Testing the strength of evidence FOR Impact (TestE)
HR&S has developed an evidence based survey strategy for the assessment of if Outcome and Sustainable Impact have been reached or have not been reached (TestE). We survey at the macro and the micro level as well as contribution tracing, and use scientific evaluation methods including control, randomization, quantitative statistics, and sometimes qualitative probability assessment.
Strength of Evidence
- The strongest cases use multiple forms of evidence, some addressing the weaknesses of others.
- It is of no importance whether an Outcome/Impact was achieved or not, this is plain testing of evidence without bias.
- Lessons learned on lack of Outcome/Impact is as important as achieved Outcome/Impact and gives rise to reflections and informed decisions.
- Monitoring data are compiled with Notion.
Contribution tracing
- Reported effects should build on plausible Outcome of the programme Activities and be consistent with our management strategy ROPE
- The observations compiled as evidence should be easier to reconcile with the programme’s claims than with other possible explanations.
Measuring Change & Sustainability
The key targets of our surveys are change and sustainability.
Change
Concerning change we measure two aspects:
i) Contribution tracing; is it our programme that causes the change?
ii) Impact; do we reach Outcome and Sustainable Impact?
The measurement of change needs a reference and the contribution tracing measures against a control. The control can be a randomized population that did not take part in the programme. Measurement of change can also benefit from the macro survey, how is the situation elsewhere at large, in the region, in the country, in the world?
The outcome measures against the base-line, thus the situation within the population prior to the initiation of the programme. Outcome benefits from progress markers that are identified during the planning stage of a new programme.
Sustainability
Concerning sustainability, we also measure two aspects:
i) Institutional capacity.
ii) Financial viability.
Institutional capacity is a wide concept that covers all aspects of the programme management’s ability to maintain the programme.
Financial balance measures whether the income from the programme, without donations, are more than the costs.
Outcome challenges
It is true that we face a number of outcome challenges. Even though our sincere ambition is to ensure TestE in all of our doings, we have not yet fully reached this goal, but we will. Reasons being are for example that i) there is no tradition of local survey management within the aid sector programmes and ii) we lack educated survey managers.