Collecting data for evidence of outcomes
In your planning of the project, you should have agreed and documented the needs and objectives and described the outcomes expected from your project.
In the ROI methodology, the following framework is used to categorise outcomes at different levels. Use of this framework enables project teams to establish in a measureable way the results they achieve and by gathering information along the way at each level, project partners can use this information to check whether they are on the road to achieving their impact outcomes.
ROI framework
| Level | Measurement Focus |
|
1. Reaction, satisfaction and planned actions |
We need to know that all stakeholders are engaged and fully satisfied with the programme/project and the planning actions we need them to take to ensure we acheive objectives. |
|
2. Learning (knowledge skills and attitude) |
We need to know that everyone involved has the knowledge, skills and attitudes needed in order to carry out their role in the project and/or implement new performance as a result of learning. |
|
3. Application and implementation (performance improvement) |
We need to know that everyone is applying their knowledge and/or skills and resources and implementing their required actions/tasks and performing their role as required. |
|
4. Impact (service outcome measures) |
We need to know the changes in impact outcomes that have occurred during and following the execution of the project. |
| 5. ROI (value for money) | We might wish to know (if it is possible) the money value of the impact outcomes that have been achieved by the project. We definitely need to know the full cost of the project. |
In planning your data collection, you should always know:
- What/why: start with agreement on the data you wish to collect
- Where from: be clear about the data source – where the data will come from – knowing this will influence when and who collects this data
- When: establish when you will collect it
- Who: agree and assign responsibility for collection
- HOW: build an appropriate instrument or method of data collection
All of the above should be planned, agreed and prepared before the project commences. You will want to plan this as it will form part of your decision making about the level of evaluation you will carry out (Step 8 in planning), your planned project schedule (Step 9 in planning) and of course, will need to be taken into account in your costing (Step 7 in planning).
