Through this initiative, MDRC is developing, testing and disseminating improved strategies for evaluating social programs, with the goal of advancing the practice of program evaluation. The paper also discusses new efforts to combine experimental and non-experimental analyses in order to test underlying program theories and maximize the knowledge gained about the effectiveness of social programs. Addressing the latter question calls for analyses of program effects on different subgroups of people. Based on rigorous studies of the impacts and implementation of programs operating in 59 sites, the paper finds that clients' two-year earnings increase more when programs emphasize employment, provide personalized attention, and do not allow staff caseloads to become large. The findings also indicate that programs that operate in areas where unemployment is high have a smaller impact on short-term earnings than programs in other areas.
Findings inevitably vary.
Frequently such connections are constructed in a narrative synthesis of multisite results. The discussion is carried out using a simple multilevel evaluation model that incorporates models of both individual outcomes within sites and variation in program effects across sites. To help answer this question, MDRC has undertaken an initiative called the Project on Devolution and Urban Change that is examining how states, urban counties, and large cities are restructuring social welfare programs. This paper describes how the Urban Change project will attempt to understand the impacts of welfare reform by looking at outcomes for more than 10 million people in four large cities. In laying out the ingredients of success, I argue that creative and flexible research design skills are essential, but that just as important are operational and political skills, applied both to marketing the experiment in the first place and to helping interpret and promote its findings down the line.