Title: Programme Evaluation for Policy Analysis
Funder: Economic and Social Research Council

Abstract: Perhaps the most pressing demand that policy makers and the public make of variety of social scientists is for answers to questions about the effectiveness of state interventions in economic and social domains. Does public policy work? Did this training programme help the participants get back to work? Did this child health programme improve children's outcomes? Providing answers to such questions is the subject of Programme Evaluation. Programme Evaluation is a difficult problem. Consider a training programme for job-seekers. Questions about the effectiveness of the programme hinge on how outcomes of individuals receiving the training programme were altered by that intervention. The first difficulty is that to know whether a training programme improved the outcomes of the recipients of training, we must know about the outcomes those individuals would have experienced had they not received the absence of training. As they did receive the training, their untrained outcomes are never observed, and hence must be estimated. One simple way to estimate these unobserved outcomes is to use the outcomes of similar job-seekers who did not receive training. For this to work, the job-seekers that did and did not receive training must be very similar in all other relevant respects. This will not be the case, if, for example, more motivated job-seekers sign up for training. Randomly assigning jobseekers to training can ensure that job-seekers that did and did not receive training are similar. Alternatively, it may be possible to use statistical methods to correct for differences between the trained and untrained groups. Even when a credible estimate of the effect of the training programme can be obtained, a second problem is to determine the precision of the estimate. For example, if we estimate that the training programme reduced spells of unemployment by about 5 weeks, we would need to be able to tell policy makers whether we are confident that the true answer is between 4 and 6 weeks, or confident only that the true answer is between 2 and 8 weeks. We would like to be able to generalize what we learn from this evaluation to other training programs Finally, we would also like to be able to synthesize the lessons learned from multiple studies of different training programs How can we understand the different answers that emerge from evaluations of different training programs? This may involve understanding how the effect varies with the communities and social networks from which the participants are drawn. We propose to conduct research into each of these problems The overall objectives of this node are to (i) stimulate a genuine step change in the conduct of programme evaluation in the United Kingdom around the world, and, (i) to maximise the value of programme evaluation by improving the design of evaluations and improving the way that such evaluations add to the knowledge base. We will develop better methods for evaluating public programmes and policies. We will also seek to provide these methods to practitioners, through workshops and training those who under programmes evaluations. Through research outputs and training and capacity building, we will engage both with other researchers and with key research users including third sector organisations and relevant government departments We will also apply the methods we develop to evaluate a number important policy questions. These include (a) The impact on time-limited in-work benefits on the time spent receiving benefits and employment; (b) The effect of introducing limits to the total amount of time individuals will be able to receive benefits across their lifetimes; (c) Schooling decisions, focusing in particular on the effect of welfare to work programmes on schooling decisions; and (d) The effect of reproductive health interventions in Africa


Research activity codes: 8.1 ORGANISATION AND DELIVERY OF SERVICES;8.4 RESEARCH DESIGN AND METHODOLOGIES (HEALTH SERVICES)
Health categories: GENERIC HEALTH RELEVANCE;REPRODUCTIVE HEALTH AND CHILDBIRTH