Development programs and the policies are intended to improve people’s quality of life. Evidence-based policy making includes, for example identifying the most appropriate intervention, tracking a program’s implementation and measuring whether the intended objectives were achieved.
In this context, the impact evaluation analysis has an essential role to assess changes in outcomes that are directly attributable to a program. In others words, the average impact of a program is the difference in the average outcomes that program participants exhibit at a point in time after the program is implemented and the average outcomes that these same program participants would have experienced at the same point in time if they had not participated in the program.
Taking into account this consideration, Daniel Sotelsek proposes some general guidelines for designing an Impact Evaluation:
- Choice of method: The goal is to choose the method that is most scientifically valid and at the same time feasible (logistically, financially, ethically, and politically). For a description of different types of evaluation, see for example, Rossi, P., M. Lipsey, and H. Freeman. 2004. “Evaluation: a systematic approach”. Thousand Oaks, CA; Sage.
- Before vs. After: Comparing average outcomes before and after the intervention can yield important information for monitoring purposes. For impact evaluation purposes, this method will only work if we can convincingly argue that in the absence of the intervention, average outcomes would have remained constant over time. In most circumstances, this is a very hard argument to make and this approach is, therefore, considered insufficient.
- Choice of comparison/control groups: Typical evaluation methods involve the choice of a group of units (individuals, households, schools, hospitals, villages, etc.) that were not affected by the program. This group is termed the comparison, or control, group.
Note: For additional information about this topic, see for example The Abdul Latif Jameel Poverty Action Lab (J-PAL), a global network of researchers who conduct impact evaluations to answer critical policy questions in the fight against poverty.