PART’s standard series of questions is designed to determine the strengths and weaknesses of federal programs by drawing on available program performance and evaluation information. OMB applies PART’s 25 questions to all programs under four broad topics: (1) program purpose and design, (2) strategic planning, (3) program management, and (4) program results (that is, whether a program is meeting its long-term and annual goals). During the fiscal year 2004, 2005, and 2006 budget cycles, OMB applied PART to approximately 20 percent of programs each year and gave each program one of four overall ratings: “effective,” “moderately effective,” “adequate,” or “ineffective,” depending on the program’s scores on those questions. OMB gave a fifth rating of “results not demonstrated” when it decided that a program’s performance information, performance measures, or both were insufficient or inadequate. The summary assessments published with the President’s annual budget proposal include recommended improvements in program design, management, and assessment. For example, a summary of the review’s findings might be followed by the clause “the administration will conduct an independent, comprehensive evaluation of the program,” or “the Budget includes to conduct independent and quality evaluations,” both of which we interpreted as an OMB recommendation to the agency to conduct such an evaluation. In our previous analysis of the fiscal year 2004 PART reviews, we analyzed over 600 recommendations made for the 234 programs assessed and found that half of those recommended improvements in program assessment. PART not only relies on previous program evaluation studies to answer many of the questions but also explicitly asks, in the strategic planning section, “Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?” Program evaluations are systematic studies that assess how well a program is working, and they are individually tailored to address the client’s research question. Process (or implementation) evaluations assess the extent to which a program is operating as intended. Outcome evaluations assess the extent to which a program is achieving its outcome-oriented objectives; they focus on program outputs and outcomes but may also examine program processes to understand how outcomes are produced. OMB first applied PART to the fiscal year 2004 budget during 2002, and the assessments were published with the President’s budget in February 2003. In January 2004, we reported on OMB and agency experiences with PART in the fiscal year 2004 budget formulation process. We noted that PART had helped structure OMB’s use of performance information in its budget review and had stimulated agency interest in budget and performance integration. However, its effectiveness as a credible, objective assessment tool was challenged by inconsistency in OMB staff application of the guidance and limited availability of credible information on program results. Moreover, PART’s influence on agency and congressional decision making was hindered by failing to recognize differences in focus and issues of interest among the various parties involved in programmatic, policy, and budget decisions. We noted that PART’s potential value lay in recommended changes in program management and design but would require sustained attention if the anticipated benefits were to be achieved. To strengthen PART and its use, in our January 2004 report we recommended that OMB (1) centrally monitor and report on agency progress in implementing the PART recommendations; (2) improve PART guidance on determining the unit of analysis, and defining program outcomes and “independent, quality evaluation”; (3) clarify expectations regarding agency allocation of scarce evaluation resources among programs; (4) target future reviews based on the relative priorities, costs, and risks associated with clusters of programs; (5) coordinate assessments to facilitate comparisons and trade-offs between related program; (6) consult with congressional committees on performance issues and program areas for review; and (7) articulate an integrated, complementary relationship between GPRA and PART. Requesting that we follow up on the findings in our January 2004 report, you asked that we examine (1) OMB and agency perspectives on the effects of PART recommendations on agency operations and results, (2) OMB’s efforts at ensuring an integrated relationship between PART and GPRA, and (3) steps OMB has taken to involve Congress in the PART process. A companion report addresses all three objectives—including OMB’s outreach to Congress—with regard to all PART reviews. Because of the fundamental role that the availability of program evaluations plays in conducting PART assessments, we conducted an in-depth analysis of agencies’ responses to OMB recommendations that they conduct program evaluations. These recommendations were identified through the analysis of recommendations for our January 2004 review. This report focuses on agencies’ progress on those evaluations and the issues involved in obtaining them. For both analyses, we examined the same four agencies’ experiences with PART. The four agencies were selected to represent a range of program types (such as research and regulatory programs), large and small agencies, and, for the purposes of this report, a large proportion of the OMB evaluation recommendations. All but two of the programs we reviewed had responded to some extent to OMB’s recommendations to conduct an evaluation; agencies did not plan evaluations of the other programs because they were canceled or restructured. However, after 2 years, only about half the programs had completed evaluations, partly because of lengthy study periods and partly because of some lengthy planning phases. The evaluations used a variety of study designs, reflecting differences in the programs and in the questions posed about program performance. About half of the programs we reviewed (11 of the 20) had completed an evaluation by June 2005—2 years after the fiscal year 2004 PART reviews and recommendations were published. Four evaluations were in progress, while 3 were still in the planning stage. Agencies did not plan an evaluation of 2 programs because those programs had been canceled or restructured. (See table 1.) Most of OMB’s evaluation recommendations asked for evaluation of the specific program reviewed, while some PART reviews at DOE and DOL asked the agencies to develop a plan for conducting multiple evaluations. At DOL, where two entire regulatory agencies had been assessed, these agencies had completed multiple studies. OMB gave DOE seven evaluation recommendations in its fiscal year 2004 PART reviews. Six were for research programs in basic science and nuclear energy and one was for its formula grant program to weatherize the homes of low-income families. Since one research program in the Office of Science had previously been evaluated by a panel of external experts called a committee of visitors, OMB explicitly recommended that the other research programs in that office also institute such a process by September 2003. In response, DOE completed evaluations of five of the six research programs, but it did not plan to evaluate the sixth, the Nuclear Energy Research Initiative, because it considered this not a stand-alone program but, rather, a source of funding for follow-up projects to other nuclear energy research programs. DOE revised this program’s objective and now authorizes funds for its projects through the other nuclear energy research programs; thus it is no longer considered a separately funded program to be evaluated. Finally, DOE officials indicated that they had only recently gained funding for planning the evaluation of the Weatherization Assistance program. (A bibliography of related agency evaluation reports appears in app. II.) OMB gave DOL five evaluation recommendations for fiscal year 2004. Two were for evaluations of specific DOL programs: grants to state and local agencies to provide employment-related training to low-income youths and administration of the Federal Employees Compensation Act regarding work-related injuries and illnesses. The three others were regulatory enforcement offices or agencies of DOL that were reviewed in their entirety: the Office of Federal Contract Compliance Programs, regarding equal employment opportunity; the Employee Benefits Security Administration; and the Occupational Safety and Health Administration (OSHA). OMB recommended that the last, which is a large regulatory agency, develop plans to evaluate the results of its regulatory and nonregulatory programs. The two DOL regulatory administrations each completed several evaluations of their enforcement activities by spring 2005, as did two of the three other DOL programs we reviewed. DOL is waiting to conduct an evaluation of the fifth—the youth employment program—until after its reauthorization because that is expected to result in an increased focus on out-of-school youths and a significant change in program activities. In addition, OSHA completed two regulatory “lookback” reviews—assessing the cumulative effects of a regulation over time—one in 2004 and another in 2005. Program officials indicated that they had developed a plan for conducting lookback reviews of employee benefit regulations beginning in fiscal year 2006. OMB recommended evaluations for four diverse HHS programs: (1) grants and technical assistance to states to increase childhood disease immunization, (2) grants to states to help recently arrived refugees find employment, (3) education loan repayment and scholarships for nurses in return for serving in facilities facing a nursing shortage, and (4) direct assistance in constructing sanitation facilities for homes for American Indians and Alaskan Natives. Evaluations of the two state grant programs were still in progress during our review, although an interim report on the immunization program was available. Reports from the two other program evaluations had recently been completed and were under departmental review. OMB recommended evaluations for four SBA programs: (1) support for existing Business Information Centers that provide information and access to technology for small businesses; (2) use of volunteer, experienced business executives to provide basic business counseling and training to current and prospective entrepreneurs; (3) Small Business Development Centers that provide business and management technical assistance to current and prospective entrepreneurs; and (4) the small business loan program that provides financing for fixed assets. OMB also asked all three counseling programs to develop outcome-oriented annual and long-term goals and measures. SBA is conducting customer surveys and had recently initiated a comprehensive evaluation of one its counseling programs, and is planning one for the other in fiscal year 2006. Another evaluation has begun to compare the costs, benefits, and potential duplication of its business loan programs.