Over the past two decades, several efforts have been launched to improve federal government accountability and results, such as the strategic plans and annual performance reports required under the Government Performance and Results Act of 1993 (GPRA). The act was designed to provide executive and congressional decision makers with objective information on the relative effectiveness and efficiency of federal programs and spending. In 2002, the Office of Management and Budget (OMB) introduced the Program Assessment Rating Tool (PART) as a key element of the budget and performance integration initiative under President George W. Bush’s governmentwide Management Agenda. PART is a standard set of questions meant to serve as a diagnostic tool, drawing on available program performance and evaluation information to form conclusions about program benefits and recommend adjustments that may improve results. The success of these efforts has been constrained by lack of access to credible evidence on program results. We previously reported that the PART review process has stimulated agencies to increase their evaluation capacity and available information on program results. After 4 years of PART reviews, however, OMB rated 17 percent of 1,015 programs “results not demonstrated”—that is, did not have acceptable performance goals or performance data. Many federal programs, while tending to have limited evaluation resources, require program evaluation studies, rather than performance measures, in order to distinguish a program’s effects from those of other influences on outcomes. Program evaluations are systematic studies that assess how well a program is working, and they are individually tailored to address the client’s research question. Process (or implementation) evaluations assess the extent to which a program is operating as intended. Outcome evaluations assess the extent to which a program is achieving its outcome- oriented objectives but may also examine program processes to understand how outcomes are produced. When external factors such as economic or environmental conditions are known to influence a program’s outcomes, an impact evaluation may be used in an attempt to measure a program’s net effect by comparing outcomes with an estimate of what would have occurred in the absence of the program intervention. A number of methodologies are available to estimate program impact, including experimental and nonexperimental designs. Concern about the quality of social program evaluation has led to calls for greater use of randomized experiments—a method used more widely in evaluations of medical than social science interventions. Randomized controlled trials (or randomized experiments) compare the outcomes for groups that were randomly assigned either to the treatment or to a nonparticipating control group before the intervention, in an effort to control for any systematic difference between the groups that could account for a difference in their outcomes. A difference in these groups’ outcomes is believed to represent the program’s impact. While random assignment is considered a highly rigorous approach in assessing program effectiveness, it is not the only rigorous research design available and is not always feasible. The Coalition for Evidence-Based Policy is a private, nonprofit organization that was sponsored by the Council for Excellence in Government from 2001 until the Council closed in 2009. The Coalition aims to improve the effectiveness of social programs by encouraging federal agencies to fund rigorous studies—particularly randomized controlled trials—to identify effective interventions and to provide strong incentives and assistance for federal funding recipients to adopt such interventions. Coalition staff have advised OMB and federal agencies on how to identify rigorous evaluations of program effectiveness, and they manage a Web site called “Social Programs That Work” that provides examples of evidence-based programs to “provide policymakers and practitioners with clear, actionable information on what works, as demonstrated in scientifically-valid studies. . . .” In 2008, the Coalition launched a similar but more formal effort, the Top Tier Evidence initiative, to identify only interventions that have been shown in “well-designed and implemented randomized controlled trials, preferably conducted in typical community settings, to produce sizeable, sustained benefits to participants and/or society.” At the same time, it introduced an advisory panel of evaluation researchers and former government officials to make the final determination. The Coalition has promoted the adoption of this criterion in legislation to direct federal funds toward strategies supported by rigorous evidence. By identifying interventions meeting this criterion, the Top Tier Evidence initiative aims to assist agencies, grantees, and others in implementing such provisions effectively. Because of the flexibility provided to recipients of many federal grants, achieving these federal programs’ goals relies heavily on agencies’ ability to influence their state and local program partners’ choice of activities. In the past decade, several public and private efforts have been patterned after the evidence-based practice model in medicine to summarize available effectiveness research on social interventions to help managers and policymakers identify and adopt effective practices. The Department of Education, HHS, and Department of Justice support six initiatives similar to the Coalition’s to identify effective social interventions. These initiatives conduct systematic searches for and review the quality of evaluations of intervention effectiveness in a given field and have been operating for several years. We examined the processes used by these six ongoing federally supported efforts to identify effective interventions in order to provide insight into the choices of procedures and criteria that other independent organizations made in attempting to achieve a similar outcome as the Top Tier initiative: to identify interventions with rigorous evidence of effectiveness. The Top Tier initiative, however, aims to identify not all effective interventions but only those supported by the most definitive evidence of effectiveness. The processes each of these initiatives (including Top Tier) takes to identify effective interventions are summarized in appendix I. In 1997, the Agency for Healthcare Research and Quality (AHRQ) established the Evidence-based Practice Centers (EPC) (there are currently 14) to provide evidence on the relative benefits and risks of a wide variety of health care interventions to inform health care decisions. EPCs perform comprehensive reviews and synthesize scientific evidence to compare health treatments, including pharmaceuticals, devices, and other types of interventions. The reviews, with a priority on topics that impose high costs on the Medicare, Medicaid, or State Children’s Health Insurance (SCHIP) programs, provide evidence about effectiveness and harms and point out gaps in research. The reviews are intended to help clinicians and patients choose the best tests and treatments and to help policy makers make informed decisions about health care services and quality improvement. HHS established the Guide to Community Preventive Services (the Community Guide) in 1996 to provide evidence-based recommendations and findings about public health interventions and policies to improve health and promote safety. With the support of the Centers for Disease Control and Prevention (CDC), the Community Guide synthesizes the scientific literature to identify the effectiveness, economic efficiency, and feasibility of program and policy interventions to promote community health and prevent disease. The Task Force on Community Preventive Services, an independent, nonfederal, volunteer body of public health and prevention experts, guides the selection of review topics and uses the evidence gathered to develop recommendations to change risk behaviors, address environmental and ecosystem challenges, and reduce disease, injury, and impairment. Intended users include public health professionals, legislators and policy makers, community-based organizations, health care service providers, researchers, employers, and others who purchase health care services. CDC established the HIV/AIDS Prevention Research Synthesis (PRS) in 1996 to review and summarize HIV behavioral prevention research literature. PRS conducts systematic reviews to identify evidence-based HIV behavioral interventions with proven efficacy in preventing the acquisition or transmission of HIV infection (reducing HIV-related risk behaviors, sexually transmitted diseases, HIV incidence, or promoting protective behaviors). These reviews are intended to translate scientific research into practice by providing a compendium of evidence-based interventions to HIV prevention planners and providers and state and local health departments for help with selecting interventions best suited to the needs of the community. The Office of Juvenile Justice and Delinquency Prevention established the Model Programs Guide (MPG) in 2000 to identify effective programs to prevent and reduce juvenile delinquency and related risk factors such as substance abuse. MPG conducts reviews to identify effective intervention and prevention programs on the following topics: delinquency; violence; youth gang involvement; alcohol, tobacco, and drug use; academic difficulties; family functioning; trauma exposure or sexual activity and exploitation; and accompanying mental health issues. MPG produces a database of intervention and prevention programs intended for juvenile justice practitioners, program administrators, and researchers. The Substance Abuse and Mental Health Services Administration (SAMHSA) established the National Registry of Evidence-based Programs and Practices (NREPP) in 1997 and provides the public with information about the scientific basis and practicality of interventions that prevent or treat mental health and substance abuse disorders. NREPP reviews interventions to identify those that promote mental health and prevent or treat mental illness, substance use, or co-occurring disorders among individuals, communities, or populations. NREPP produces a database of interventions that can help practitioners and community-based organizations identify and select interventions that may address their particular needs and match their specific capacities and resources. The Institute of Education Sciences established the What Works Clearinghouse (WWC) in 2002 to provide educators, policymakers, researchers, and the public with a central source of scientific evidence on what improves student outcomes. WWC reviews research on the effectiveness of replicable educational interventions (programs, products, practices, and policies) to improve student achievement in areas such as mathematics, reading, early childhood education, English language, and dropout prevention. The WWC Web site reports information on the effectiveness of interventions through a searchable database and summary reports on the scientific evidence. The Coalition provides a clear public description on its Web site of the first two phases of its process—search and selection to identify candidate interventions. It primarily searches other evidence-based practice Web sites and solicits nominations from experts and the public. Staff post their selection criteria and a list of the interventions and studies reviewed on their Web site. However, their public materials have not been as transparent about the criteria and process used in the second two phases of its process—review and synthesize study results to determine whether an intervention met the Top Tier criteria.