Congress enacted GPRA in part to inform congressional decision making by providing objective information on the relative effectiveness and efficiency of federal programs and spending. In addition to requiring executive agencies to develop strategic and annual performance plans, and measure and report on progress toward goals, GPRA also emphasized efficiency. According to the statute, GPRA was intended, among other things, to address problems of waste and inefficiency in federal programs, and to improve congressional decision making by providing objective information on the relative efficiency and effectiveness of federal programs and spending. OMB plays an important role in the management of the federal government’s performance, and specifically GPRA implementation. Part of OMB’s overall mission is to ensure that agency plans and reports are consistent with the President’s budget and administration policies. OMB is responsible for receiving and reviewing agencies’ strategic plans, annual performance plans, and annual performance reports. To improve the quality and consistency of these documents, OMB issues annual guidance to agencies for their preparation, including guidelines on format, required elements, and submission deadlines. In addition, GPRA requires OMB to prepare the overall governmentwide performance plan, based on agencies’ annual performance plan submissions. The PMA and PART of the prior administration also included an emphasis on improving government efficiency, with requirements for agencies to develop program-level efficiency measures and show annual improvements in efficiency. In August 2001, the Bush Administration launched the PMA with the stated purpose of ensuring that resources entrusted to the federal government were well managed and wisely used. OMB developed criteria called “standards of success” to measure progress in five management initiatives under the PMA, as well as a scorecard to track agency progress under each initiative. Criteria to receive and maintain the highest rating score (green status) for the performance improvement initiative included that an agency’s annual budget and performance documents include at least one efficiency measure for each program and that program performance and efficiency improvements be identified each year. PART, which was launched in 2002 as a component of the PMA, included assessment of the extent to which programs were tracking progress toward and achieving efficiency improvements. PART consisted of a set of questions developed to assess various types of federal executive branch programs, and addressed four aspects of a program: purpose and design, strategic planning, program management, and program results/accountability. While there were references to efficiency in several different sections of the 2007 and 2008 PART guidance, two PART questions focused specifically on development of program-level efficiency measures with annual targets for improvement: “Does the program have procedures (e.g., competitive sourcing/cost comparisons, information technology (IT) improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?” In order to receive a “yes” response for this question, a program was to have regular procedures in place to achieve efficiencies and cost effectiveness, and had to have at least one efficiency measure with baseline and targets. Evidence could include efficiency measures, competitive sourcing plans, IT improvement plans designed to produce tangible productivity and efficiency gains, or IT business cases that documented how particular projects improved efficiency. “Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?” In order to receive a “yes” response for this question, a program had to demonstrate improved efficiency or cost effectiveness over the prior year, including meeting its efficiency target(s) in the question above. About 90 percent of all programs that received a PART assessment, including those in our selected review, developed at least one performance measure as an efficiency measure. However, we found that about half of the approved measures either did not contain typical elements of an efficiency measure, or were unclear. As table 1 below indicates, we analyzed a sample of the efficiency measures that were developed for PART, and, to the extent possible, placed them into one of the three categories shown in the table. (In some cases, the available information on the measure was insufficient for us to place it into one of the three categories, so we labeled these measures as “unclear.”) As figures 1 and 2 below illustrate, our analysis of the 36 efficiency measures from our selected programs and a random sample of the remaining efficiency measures indicates that about half of the efficiency measures contained typical elements by including both an input and an output or outcome. As illustrated in figure 1, for the 21 selected programs (listed in appendix II), we determined that 58 percent of the efficiency measures included both elements and 42 percent did not. In its guidance to programs, OMB stated that, although both output and outcome-oriented efficiency measures were acceptable, outcome efficiency measures were preferred. Because we obtained more in-depth information on the selected programs’ measures, we further analyzed whether those that included both elements were output- or outcome-oriented and found most to be output- oriented. Figure 2 summarizes estimates for the remaining 1,355 efficiency measures, based on a random sample of 100 of those measures. We estimate that 48 percent of the measures included both elements, 26 percent did not, and the remaining 26 percent were unclear. Of those that did not contain both elements, the missing element was most often an input. In general, as indicated in table 1, the absence of these typical elements can result in measures that do not truly capture efficiency. Nevertheless, some of the information captured in these measures could still be of value to program officials for helping improve efficiency. For example, one measure from our selected programs—average time to correct/mitigate higher priority operations and maintenance deficiencies at certain facilities in the Bureau of Reclamation—did not contain an input element. However, program officials told us this was an important measure because it helped them prioritize which ongoing preventive maintenance projects they should repair first by categorizing repairs needed according to the likely costs of delaying the repairs. For example a category 1 deficiency should normally be repaired immediately (within to 6 months) to avoid escalating the cost of repair; a category 2 deficiency should be repaired in a few years. In contrast, a category 3 deficiency is normally repaired only if there is time and funding remaining after repairing category 1 and 2 deficiencies. In another example, the National School Lunch Program (NSLP) used a measure which was labeled an efficiency measure, but which did not hav the typical ratio of inputs to outputs or outcomes. Instead, the measure focused on reducing the error rate in making program payments. Progra officials characterized the measure as a process measure, rather than an output or outcome-based efficiency measure. An official said that out of s $7 billion in total program payments, errors worth $2 billion occur in term of under and over payments, for a net cost to the program of $1 billion. An official said that if they were able to reduce overall overpayments due to various types of error, it could save millions of dollars. Officials said this measure has been important in helping them take corrective actions to reduce the number of payments made in error. Among the selected programs, for the efficiency measures that contained an input, the type of information used to express the input varied in terms of both availability for use and completeness. Most of the efficiency measures we reviewed captured inputs in terms of cost, but a few us amount of staff resources or time spent to produce an output or outcome as a proxy for cost. For example, the Department of Labor Energy Employees Occupational Illness Compensation program’s efficiency measure was the average number of decisions per full-time equivalent (FTE), which we determined used information on work hours as estima by FTEs as the input. While FTE information is often readily available ted and can be a useful proxy for cost, it does not necessarily reflect total cost because, for example, it would neither distinguish between higher and lower cost FTEs, nor would it include other costs, such as contractors, training, equipment, or facilities. In addition, dollar cost information can vary in how completely it captures the cost of producing outputs or outcomes. “Cost” generally can be thought of as the value of resources that have been, or must be, used or sacrificed to attain a particular objective, which, in the case of an efficiency measure, would be a unit of output or outcome. “Full cost” is generally viewed as including both direct costs (costs that can be specifically identified with a cost object, such as an output) and indirect costs (costs of resources that are jointly or commonly used to produce two or more types of outputs but are not specifically identifiable with any of the outputs). Managerial cost accounting (MCA) information can provide a more complete picture of the cost involved in producing program outputs or outcomes by recognizing resources when they are used and determining the full cost of producing government goods and services, including both direct and indirect costs. According to the Statement of Federal Financial Accounting Standards No. 4 (SFFAS 4), Managerial Cost Accounting Concepts and Standards for the Federal Government, which sets forth the fundamental elements for MCA in government agencies, costs may be measured, analyzed, and reported in many ways and can vary depending upon the circumstances and purpose for which the measurement is to be used. Our analysis of the cost information used by the selected programs showed that most of the measures used budgetary information, such as appropriations or obligations, for the cost element. Of the 18 efficiency measures from our selected programs that had both typical elements, and had cost as the input, 14 measures (78 percent) used a form of budgetary information. We have previously reported that using budgetary information, such as appropriations or obligations, may not completely capture the full cost of producing program outputs or outcomes because of differing time frames and account structures. With regard to timing, appropriations provide agencies legal authority to obligate funds for a given fiscal year or beyond. Consequently, agency outlays (payments against obligations for goods and services received) representing the resources used to produce a program’s outputs or outcomes in a given year may flow from obligations made in a prior year’s appropriation. Therefore a given year’s appropriations or obligations may not represent the resources actually used to produce a program’s outputs or outcomes in that year. With regard to account structures, appropriations accounts developed over the last 200 years were oriented in different ways in response to specific needs. For example, some appropriations accounts reflect items of expense, such as salaries or construction, while others reflect organizations, processes, or programs. Further, program-oriented account structures may cover multiple programs or may exclude some indirect resources used by the programs. Though budgetary information may not completely cover the cost of producing program outputs or outcomes, several program officials said it was the most complete information available to them and best met the needs of Congress. For example, the Department of Labor Job Corps program, which used budgetary information in its efficiency measure, divided its request in the fiscal year 2010 Job Corps Congressional Budget Justification into three categories: operations, construction, and administration. However, the program’s efficiency measure—cost per participant in the Job Corps program—was based entirely on the operations category, which encompassed 92 percent of the program’s fiscal year 2010 request, meaning the measure did not capture the remaining 8 percent of construction- or administration-related costs that were also associated with program participation. A study commissioned by the Job Corps recommended that all direct costs associated with Job Corps appropriations be included in the measure if full costs were to be determined. This would include actual expenditures (i.e., outlays rather than appropriations or obligations) for Job Corps appropriations provided for operations, construction, and direct administrative costs. Program officials indicated they did not believe including the additional costs would provide useful information because there were relatively few opportunities to find efficiencies in the construction or administration categories. Additionally, a Department of the Interior Wildland Fire Management budget official told us that while they had access to more complete cost data, this information was not necessarily accurate or easy to obtain because it had to be collected from five different entities with different cost accounting systems. They also preferred to use budgetary information because it helped to justify their appropriations request to Congress. Program officials noted that each of their three efficiency measures was based on obligations data. Relative to time or budgetary information, some agencies have sought to develop more complete cost information by using MCA systems capable of accumulating and analyzing both financial and nonfinancial data in order to determine, among other things, the unit cost of producing program outputs or outcomes. Such systems are also capable of recognizing resources when they are used and determining the full cost of producing government goods and services, including both direct and indirect costs. However, in earlier work we found that only 3 of the 10 Chief Financial Officer (CFO) Act agencies we reviewed had implemented MCA systems entitywide: Interior, the Social Security Administration, and Labor. Transportation had made significant progress in implementing MCA entitywide and three agencies—Agriculture, Health and Human Services, and Housing and Urban Development—planned to implement MCA systems when upgrading their overall financial management systems. The three remaining agencies we reviewed—Education, the Treasury, and Veterans Affairs—had no plans to implement MCA departmentwide, although Veterans Affairs was initiating a review to explore opportunities to do so. Consequently, we recommended that individual agencies commence or improve the development of entity-wide MCA systems as a fundamental component of their financial management system, as required by SFFAS 4 and the Federal Financial Management Improvement Act of 1996. For this report, of the five agencies we reviewed, we selected three— Interior, Labor, and Transportation—because we previously reported they had either implemented MCA systems entitywide, or were planning to do so. Nevertheless, we did not find widespread use of MCA system data for the efficiency measures we reviewed either in these agencies or in the other two agencies—Education and Agriculture—that did not have entitywide MCA systems. Of the 18 efficiency measures from our selected programs that included typical elements, four measures (22 percent) used a distinct MCA system to determine costs. Those programs that relied on MCA data produced outputs, such as the Student Aid Administration program (student aid disbursements), the Federal Aviation Administration’s (FAA) Air Traffic Organization Terminal (take offs and landing operations) and Technical (maintenance and modernization of equipment needed to provide air traffic services) programs, and the Department of the Interior’s Fisheries program (pounds of trout per dollar). In addition, legislation was enacted in the 1990s, which resulted in both Federal Student Aid (FSA) and FAA developing MCA systems to improve performance.37, 38 Of the remaining 14 efficiency measures, officials from several of those programs told us they used budgetary information because they either did not have access to an MCA system, the system they could access produced poor data, or the information would not be useful for congressional decision making. For example, the Department of Education did not have a departmentwide MCA system, though it is now considering creating such a system in response to a prior recommendation we made. Also, officials with the Department of Transportation CFO office told us that the department had taken a decentralized approach in which some of their operating administrations—such as the FAA and Federal Transit Administration— had developed and were using their own MCA system. In addition, although the Department of Labor’s CFO had developed an MCA system and made it available to its agencies and programs, officials from the five Department of Labor programs we reviewed indicated that they did not use it for their efficiency measures because, in their opinions, the system was either not useful, not sufficiently developed for their needs, did not capture all the program’s costs, or captured a different type of funding than was used for the efficiency measure. Finally as indicated previously, a Department of the Interior Wildland Fire budget official told us that cost information for their program was neither easy to access nor was it as useful for budget justification purposes. The Higher Education Amendments of 1998, which amended the Higher Education Act of 1965, established a performance-based organization for the delivery of federal student financial assistance, after which Federal Student Aid, the one Department of Education program office with an operational MCA system, independently developed its MCA system. Pub. L. No. 105-244, title I, § 101(a), 112 Stat. 1581, 1604–610 (Oct. 7, 1998), codified at 20 U.S.C. § 1018. PBOs are discrete units, led by a Chief Operating Officer, that commit to clear objectives, specific measurable goals, customer service standards, and targets for improved performance, see GAO-06-653T. The Federal Aviation Reauthorization Act of 1996 required that FAA develop a cost accounting system that accurately reflects the investment, operating and overhead costs, revenues, and other financial measurement and reporting aspects of its operations. Pub. L. No. 104-264, § 276(a)(2), 110 Stat. 3213, 3248 (Oct. 9, 1996), codified at 49 U.S.C. § 45303(e). In addition, in 1997, the National Civil Aviation Review Commission (the “Mineta Commission”) recommended that FAA establish a cost accounting system to support the objective of FAA operating in a more performance-based, business-like manner. The selected programs that had measures with both elements of a typical efficiency measure reported mixed results under PART in terms of gains and losses in efficiency. As previously indicated in figure 2, 21 of the 36 efficiency measures developed by the programs selected for our review had both of the elements of a typical efficiency measure. As can be seen in table 2, 8 of the 21 efficiency measures (representing seven different programs), showed an improvement in efficiency between the baseline and most current year. Ten of the efficiency measures (representing seven programs) showed a decrease in efficiency over the reported periods. Three measures (representing two programs) had only baseline data. We have previously reported that agencies can use performance information to make various types of management decisions to improve programs and results. The same is true for performance measures that track efficiency—managers need to use the information to help them identify actions needed to bring about improved efficiency. Our review of selected programs that had measures with both elements of a typical efficiency measure found variety in terms of whether officials reported using efficiency measures. We also found no clear relationship between efficiency gains or losses and whether program officials reported using or not using efficiency measures. Officials from three of the seven programs that reported efficiency gains described using their efficiency measures, while officials for three additional programs with efficiency gains said they did not use the efficiency measures. Officials for the other program with efficiency gains reported mixed pictures, saying they did not use the efficiency measure but found some value in the measure or its components. A similar mix was found among programs that reported net losses in efficiency, with officials for three programs using the efficiency measures and officials for four programs not using them. One example of a program that showed a net gain over time for its efficiency measure and for which officials reported using the data was the Department of Education’s Student Aid Administration program. Reducing costs was one of the primary objectives of the program. Their efficiency measure—direct administrative unit costs to originate and disburse student loans and Pell Grants—showed a gain in efficiency from 2006 to 2008. The agency provides federal assistance to eligible students by partnering with postsecondary schools, financial institutions, and guaranty agencies (state and nonprofit agencies that guarantee loans against default). Program officials told us they used information from this measure to establish targets for reduced unit costs for their lending transactions. For example, they reported using the data to negotiate a lower cost for the origination of direct student loans by a sole-source contractor. FSA used a contractor to originate the loans made directly to students. The contract allowed for a certain quantity of loan originations for a set price, up to a maximum number of loans each year. According to program officials, the sharp reduction in credit availability due to the financial crisis beginning in 2008 led to an increase in demand for FSA direct loans. FSA had projected that demand for direct student loans in the 4th quarter of fiscal year 2009 would exceed the contract maximum by 3 million loans. The contractor proposed a price of $8.9 million for the additional loans, arguing that the added volume would require higher infrastructure costs associated with greater call center capacity. FSA officials told us they analyzed historical data for their efficiency measure and found that the unit cost to originate loans decreased as volume increased. They used this analysis to challenge the contractor’s bid and succeeded in lowering the agreed price to $4.9 million. Officials reported that legislation, federal cost accounting standards, and our previous recommendations all contributed to pressure to track unit costs and try to lower administrative costs. Consequently, the agency had developed a number of cost models, which facilitated their developing the efficiency measure for PART. The Department of the Interior’s Fisheries program provides an example in which the efficiency measure showed a net loss but officials said they used the efficiency measure data to make management decisions. The efficiency measure tracked the efficiency (pounds per dollar) of producing healthy rainbow trout for recreation. For the first 4 years examined, fiscal years 2004 through 2007, the efficiency measure varied slightly, indicating that overall efficiency was relatively stable. For fiscal year 2008, however, the measure fell, indicating a significant drop in efficiency. Officials attributed this drop to a 31 percent increase in feed, energy, and utility costs that was experienced throughout the country in 2008 and was beyond their control. Several fishery stations reported 40 percent increases in feed costs in just 1 year. Officials told us that having information about the decline in efficiency was valuable because it led individual stations to look for opportunities to lower other costs of production that were within their control. For example, program managers said they used their efficiency measure data to help them decide to phase out the production of inefficient (more costly) strains of trout. In addition, they said they used the measure to help manage the losses resulting from diseased trout that could not be sold by shifting production from one fishery to another that did not have a problem with disease. Officials said they thought it was easier for programs that directly produced products or provided services to develop and use efficiency measures. They said they had a relatively easy time of developing their efficiency measure because they directly produce a product (i.e., rainbow trout). The Department of the Interior’s Wildland Fire Management Program reported mixed efficiency results. Of their three efficiency measures, two showed a net loss and one showed a net gain. Even though the results were mixed, officials said they used the data to establish ranges of acceptable cost estimates for contract or grant proposals and to identify outliers. Officials said their efficiency measures, which tracked numbers of wildland acres treated or moved to a better condition class (to reduce the likelihood of wildland fires) per million dollars, enabled them to identify unusually high or low costs when evaluating proposals from field units for funding treatments. They could identify a proposal that did not fall within the normal range of prior projects in terms of costs, do further analysis, and ask for explanations from field staff to better understand why the proposal was outside the norm. Program officials also said they used a tool called Ecosystem Management Decision Support (EMDS) to help prioritize projects and allocate funding for future years. They said EMDS takes into account various factors, including past performance and efficiency. For example, fuel treatments that demonstrated greater efficiency would be given higher priority for funding under EMDS, other factors being equal. While FAA’s Air Traffic Organization Technical Operations program’s efficiency measure showed a net gain, officials said they did not use it to make major decisions. ATO Technical Operations is responsible for maintaining and modernizing equipment needed in the national airspace system to deliver air traffic services. It fields, repairs, and maintains a huge network of complex equipment, including radars, instrument landing systems, radio beacons, runway lighting, and computer systems. The efficiency measure, unit cost for providing ATO Technical Operations services, is the “total labor obligations for the Technical Operations' Service Unit” divided by the total hours of operational availability (or equipment “uptime”). Officials said the measure was used as a baselining effort, and no decisions have been made as a result. Officials explained that they cannot significantly influence labor costs because of a labor agreement that requires ATO to maintain 6,100 direct employees. Officials said they have used data for the denominator of the efficiency measure, on the hours of operational availability. Equipment needs to be available continuously, and currently is about 99.7 percent of the time. Officials said they have not done the marginal cost analysis to determine whether it would be cost-effective to try to increase equipment uptime, but they have broken the data down by location and looked for outliers and tried to address impediments to operational availability at certain locations. They also said that while they have not used the efficiency measure to make any management decisions, it has been valuable in helping to orient staff to think about costs of operations and how to go about looking for efficiency improvements. Lastly, the Department of Labor’s Occupational Safety and Health Administration (OSHA) program reported a net loss for the efficiency measure and told us they did not use the data. Officials said the current efficiency measure—inspections per Compliance Safety and Health Officer—was only a “back room calculation” and was not something they promoted or used to make decisions within the organization. They said they did not evaluate the performance of staff based on the number of inspections they conducted, because doing so could lead to a perverse effect of rushing through inspections in order to complete them more quickly, resulting in poorer quality inspections. In addition, officials said they did not believe anyone used the OSHA efficiency measure other than for reporting purposes. Officials from all of the selected programs we reviewed identified one or more challenges related to developing or using efficiency measures. The challenges cited were not new; we have reported on similar types of challenges in our prior work on PART and performance measurement issues in general. Challenges related to OMB’s guidance and technical assistance for efficiency measures specifically included: a program definition that did not correspond well to program operations; an emphasis on developing outcome-oriented efficiency measures; achieving required annual improvement targets for efficiency; and inconsistencies and limitations in OMB’s guidance and technical assistance. In addition, officials described the difficulty of trying to compare the relative efficiency of programs (or units within programs) that have significantly different objectives, activities, or cost data. We previously reported that determining the appropriate program or unit of analysis for a PART assessment was not always obvious, and what OMB determined was useful did not necessarily match agency organization or planning elements. We found that OMB sometimes aggregated separate programs into one for the purposes of a PART assessment, and in other cases disaggregated programs. Aggregating programs sometimes made it difficult to create a limited, but comprehensive, set of performance measures for programs with multiple missions, and agency officials noted that difficulties could arise when unrelated programs and programs with uneven success levels were combined for PART. At the same time, disaggregating a program too narrowly could distort its relationship to other programs involved in achieving a common goal, and sometimes ignored the interdependence of programs by artificially isolating programs from the larger contexts in which they operated. While OMB, in response to one of our recommendations, expanded PART guidance on how a unit of analysis was to be determined, problems related to defining programs for PART remained. An OMB staff member acknowledged to us that OMB often combined what agencies considered and managed as separate programs in order to identify a program for PART. According to some program officials, the way in which OMB grouped their activities into a program for the PART assessment was not useful, and so the resulting program-level efficiency measure developed for PART was not useful. Officials from the National Highway Traffic Safety Administration (NHTSA) within the Department of Transportation told us that the way OMB and the department defined their program for the PART assessment was a key challenge to developing a useful efficiency measure. Officials said that NHTSA’s mission and operations are organized along two major programmatic lines: highway and motor vehicle safety. In contrast, for purposes of PART and development of the required efficiency measures, NHTSA was organized into two programs that received separate PART assessments: Operations and Research, and Grant Management. As a consequence, officials said the efficiency measure developed for the Operations and Research program was not meaningful. They said they were revising their efficiency measures and planned to develop one for each of the programmatic areas. In previous work, we identified challenges involved in developing useful results- or outcome-oriented performance measures for some programs, such as those geared toward long-term health outcomes and research and development. We reported that many of the outcomes for which federal programs are responsible are part of a broader effort involving federal, state, local, nonprofit, and private partners, and that it is often difficult to isolate a particular program’s contribution to an outcome. However, we also reported on how selected agencies that had limited control over the achievement of their intended objectives addressed the challenge by employing various strategies, such as including intermediate outcomes within their direct control along with far-reaching or end outcomes. In a previous review of PART, we reported that OMB had taken steps to clarify PART guidance on using outcome and output performance measures, and had accepted administrative efficiency measures instead of outcome-level efficiency measures for some programs. However, we also reported that agencies had mixed success in reaching agreement with OMB in these areas. As mentioned above, of the 21 measures from selected programs that had typical elements of an efficiency measure, 13 contained outputs, and 8 contained outcomes. While OMB’s PART guidance described efficiency measures as including both outcome- and output-level impacts, it stated that the best efficiency measures captured outcomes. Further, program officials told us that OMB pressed some programs to have efficiency measures that captured outcomes instead of outputs. Similar to findings from our prior work, some officials we interviewed for this review said it was difficult for their programs to interpret outcome- level efficiency measure information, because factors other than program funding affected the outcome of the program. For example, the purpose of the Forest Service’s Watershed program is to restore, enhance, and maintain watershed conditions, including soil, water, air, and forest and rangeland vegetation within the national forests and grasslands. Management of these physical and biological resources provides a foundation for healthy, viable ecosystems. The Watershed program received a “Results Not Demonstrated” rating from the OMB 2006 PART assessment process because it lacked long-term, outcome-based performance and efficiency measures to track the performance of land management activities on national forest and nonfederal watersheds, or demonstrated water quality improvement over time. Basically, the Forest Service was unable to track how watershed projects were prioritized, identify the benefits associated with restoration projects, and determine whether those projects improved watershed condition. Officials said they had previously proposed the unit cost of watershed improvement projects as an efficiency measure under PART, but OMB rejected it partly because it was an output- rather than an outcome-level measure. According to Forest Service documents, factors beyond its control affect watershed conditions, and it is difficult to demonstrate the impact of program activities on watersheds and try to determine the most cost-effective way to improve the outcome. The agency’s ability to improve the condition of watersheds depends on many factors, including what percentage of the land affecting the watershed is privately owned as opposed to owned by the Forest Service and past impacts—for example, an official said that lands that were previously mined may be more difficult to restore. Officials said that the cost of trying to improve some watersheds would exceed available funds, and in some cases passive restoration, or doing nothing and letting natural processes return, could improve conditions as rapidly as any program interventions could. Forest Service officials said they reached agreement with OMB to develop an outcome-oriented efficiency measure based on the cost of improving or maintaining the condition of watershed acres. According to a 2008 report prepared by the Forest Service, in order to be able to relate costs to outcomes, program officials explained that they will need to develop a consistent approach for assessing watershed condition and a system that would enable them to track changes in watershed conditions and relate these changes to Forest Service management activities. Following implementation of this approach, the agency would be able to track improvements in program outcomes and relate changes to cost. OMB’s PMA and PART guidance required programs to set annual improvement targets for their efficiency measures. We previously reported that in some programs, long-term outcomes are expected to occur over time through multiple steps, and that it can take years to observe program results. For these programs, it can be difficult to identify performance measures that will provide information on annual progress toward program results. Along these lines, some program officials we interviewed told us it was not reasonable to expect annual improvements in efficiency for some programs because it might take several years for an increase in efficiency to be realized as a result of some intervention or investment, or because a technological advance might result in a one-time cost savings that would not continue to be achieved over time. For example, the Plant and Animal Health Monitoring and Surveillance programs of Agriculture’s Animal and Plant Health Inspection Service, which protects the health and value of agriculture and natural resources through early detection of pest and disease outbreaks, had an efficiency measure that tracked the value of damage prevented or mitigated by the program per dollar spent. Program officials told us that it was difficult to show improvements in efficiency every year. They said that as a science-based program, it took time to develop new technologies that improved efficiency, and the effect might be a one-time improvement in efficiency that would not result in continued additional efficiency gains over time. Similarly, officials from the Department of the Interior’s Endangered Species program stated that the timeframe needed to achieve results in terms of conservation and recovery of an endangered species is longer than an annual or even 5-year timeframe. They said it is difficult to associate additional funding with a defined outcome in a given year. Officials from the Department of Labor’s Center for Program Planning and Results acknowledged that their office and OMB strongly encouraged agencies and programs to show annual improvements for efficiency measures, which led to some friction in setting targets for out-years for some programs. They said that pressure to show annual improvements in efficiency resulted in some programs revising targets for the efficiency measures every year because they could not achieve the annual targets. An official said that there was a lot of focus on numerical annual targets for efficiency measures, and because some programs cannot realistically see improvements in efficiency in a 1-year time period, monitoring trends would be better. As we previously reported, OMB staff had to exercise judgment in interpreting and applying the PART tool to complex federal programs, and were not fully consistent in interpreting the guidance. In prior reviews of PART, we identified instances in which OMB staff inconsistently defined appropriate measures, in terms of outcomes versus outputs, for programs. We reported that some program officials said that OMB staff used different standards to define measures as outcome oriented. We also reported that OMB took steps to try to encourage consistent application of PART in evaluating government programs, including pilot testing the assessment instrument, clarifying guidance, conducting consistency reviews, and making improvements to guidance based on experience. OMB also issued examples of efficiency measures it identified as exemplary and expanded the guidance on efficiency measures. While officials for some programs we interviewed told us that OMB assistance and feedback under PART were valuable in developing useful efficiency measures, officials for other programs cited inconsistencies and limitations in OMB’s PART guidance and technical assistance that made the development of acceptable and useful efficiency measures more challenging. For example, officials for Agriculture’s Plant and Animal Health Monitoring programs said they worked with the department and OMB representatives to discuss efficiency measures and obtain feedback on proposed measures. Officials said feedback obtained was useful and allowed them to consider options they had not previously identified, and in some cases they incorporated the advice. Officials said that the efficiency measure tracking the value of damage prevented and mitigated per program dollar spent was a direct result of an OMB recommendation. However, officials for other programs said that PART guidance and OMB technical assistance and feedback provided to programs on efficiency measures were insufficient or inconsistent. For example, officials for the Department of the Interior’s Endangered Species program, which lacked an efficiency measure that had been approved by OMB, said they believed that OMB’s review of proposed efficiency measures was inconsistent. Officials said that OMB rejected a proposed output-level efficiency measure for the Endangered Species program and pushed for an outcome- level measure, but approved a similar measure for another program in a different federal department. Similarly, officials for the Forest Service Watershed program in Agriculture, which did not have any of its proposed efficiency measures accepted by OMB for the PART assessment, stated that lack of consistency on OMB’s part in defining acceptable efficiency measures complicated the process for them. They said OMB rejected a measure they proposed, but approved a similar measure for another agency. Further, officials for OSHA in the Department of Labor indicated that they worked with two OMB analysts who were not as familiar with their agency as the current analyst and created rework. Overall, they did not believe the process they undertook with OMB to develop an efficiency measure was fruitful. Officials we interviewed from the Department of Education’s Office of Federal Student Aid indicated that they eventually wanted to use data for the Student Aid Administration program’s efficiency measure (direct administrative unit costs for origination and disbursement of student aid), to compare the costs of similar activities performed by different contractors. However, we previously reported that challenges can result from the difficult but potentially useful process of comparing the costs of programs related to similar goals. We have also reported that in order to effectively compare a program to alternative strategies for achieving the same goals, comprehensive data on the program and comparable data on alternatives need to be available. In our prior work on human services programs, we reported that OMB officials recognized that programs are different and it may not be possible to compare costs across programs, especially when costs are defined differently due to programmatic differences. Officials from some selected programs we reviewed questioned whether it was reasonable to use efficiency measures for comparative analysis of performance across programs when the objectives, activities, or costs of the programs differed significantly. For example, an official from the Department of Labor’s Job Corps program said it was not appropriate to compare their program’s performance to that of other department employment and training programs in terms of the efficiency measure, which tracked cost (appropriations) per participant. According to the program’s PART assessment, the program's purpose is to assist eligible disadvantaged youth (ages 16-24) who need and can benefit from intensive education and training services to become more employable, responsible, and productive citizens. Participants have characteristics, such as being a school dropout, homeless, or in need of intensive counseling to help them participate successfully in school or hold a job, that are barriers to employment. Program officials said that Job Corps is quite different from other employment and training programs run by the department because it involves removing participants from a negative environment and placing them in a totally different, primarily residential, environment. Such a model involves higher operating costs associated with providing participants intensive services in a residential setting for up to 2 years, which would make it appear less efficient when compared to nonresidential programs.59, 60 As another example, officials for the Endangered Species program at the Department of the Interior questioned whether it made sense to try to compare the efficiency of efforts to protect different species. The program works with states, tribes, other federal agencies, nongovernmental organizations, academia, and private landowners to promote the conservation and prevent extinction of over 1,300 endangered or threatened species. As noted in the program’s strategic plan, each species has inherent biological constraints which create challenges to its recovery. Officials told us that they work with vastly different species in different regions, many factors affect the complexity of their work, and each case is unique. We previously reported that species are ranked by priority, but rankings do not reflect how much funding is needed to protect a species. The Job Corps program hired a contractor to propose an alternative efficiency measure to try to capture the unique outcomes of the program. The contractor study proposed an outcome-level efficiency measure (“cost per successful program outcome”), but cautioned against comparison with other programs because estimates for other programs might not reflect full costs, and because comparisons could be misleading if program objectives were not identical. Hei Tech Services, Inc., Job Corps Cost Measure: Selecting a Cost Measure to Assess Program Results (Dec. 1, 2008). In a prior review of PART, Labor officials told us that participants could remain in the Job Corps program for up to 2 years, which they considered adequate time to complete education or vocational training, and which generally resulted in higher wages, according to studies. However, they said that since costs per participant increased the longer a student remained in the program, Job Corps appeared less efficient compared with other job training programs. (GAO-06-28). Officials told us that the cost of an intervention, such as building a fence, could be much cheaper for one species in a particular region than for another species in a different location. The head of the department’s Office of Planning and Performance Management in the Office of the Secretary said that because the effort to save some species is so much more complicated and expensive than for others, it is not meaningful to simply compare the “cost per unit” or efficiency of saving different species without considering other factors such as the time frame involved, and the scope and level of treatment needed. For example, he suggested that it was not reasonable to try to compare the cost of saving the polar bear to the cost of saving a species of plant. As stated above, OMB’s approach to improving the efficiency of federal programs under PMA and PART focused on requiring individual programs to develop efficiency measures, identify procedures to achieve efficiencies, and achieve annual gains in efficiency. In prior reports, we concluded that PART’s focus on program-level assessments could not substitute for GPRA’s focus on thematic goals and department- and governmentwide crosscutting comparisons. Through our review of literature, we identified a variety of strategic and crosscutting approaches that government, nongovernment, and business organizations have used in their efforts to improve efficiency. For example, the United Kingdom and some state governments provide some important insights into such governmentwide efficiency efforts. These approaches share a common theme that performance can be maintained or even improved while reducing unnecessary costs associated with outmoded or wasteful operations, processes, and purchases. These approaches to efficiency improvement differ from OMB’s approach under PMA/PART in that they can be applied at government- or agencywide levels in addition to being applied within specific programs. Officials from some selected programs provided examples of additional efforts they were undertaking to improve efficiency, some of which can be aligned with these broader approaches we identified in the literature. Broadening the application of these approaches beyond the program level could help to identify even greater opportunities for improvements in the efficiency of federal government operations. GPRA’s planning and reporting requirements can provide a framework for agencies to take a more strategic approach to improving federal government efficiency. Governmentwide reviews have been conducted in the United Kingdom (UK) and by some state governments in the U.S. to help identify and implement strategic approaches to improve efficiency. Such reviews have been ordered by executive leadership to address a wide range of government activity. Reviews have been broad in scope, and initiatives undertaken to improve efficiency have been crosscutting and could be applied across processes, services, and organizations rather than just at the program level as required for federal agencies under OMB’s PART approach. In the UK in 2004, Her Majesty’s (HM) Treasury published a first of its kind, government-wide efficiency review that examined government processes, identified opportunities for cutting costs and improving services, and developed proposals to deliver sustainable efficiencies in the use of resources within both central and local government. The review focused on improving government efficiency in areas such as procurement, funding, regulation, citizen services, and administration. The efficiency review proposed strategies to improve efficiency that were adopted by HM Treasury in the UK’s 2004 budget. HM Treasury actively supported departments in their individual efficiency programs. HM Treasury negotiated efficiency goals with each department and created a centralized efficiency team managed by the Office of Government Commerce to help departments achieve efficiency gains. HM Treasury brought in outside expertise, including senior figures from the private and public sector, to support and work with departments. Additional specialist change agents were employed to assist departments with trying to achieve efficiency improvements in areas such as e-government, human resources, IT, finance, construction, and commodity procurement. Change agents addressed problems created by highly fragmented markets that crossed departmental boundaries. To assist departments in financing efficiency improvement programs, HM Treasury created a £300 million Efficiency Challenge Fund that provided departments with matching funds for efficiency improvement programs. Funds were approved based on objective criteria such as the ratio of expected savings to matching funds, probability of achieving savings, evidence that alternative funds were not available, and progress in delivering efficiency gains. In a final review of the completed efficiency program in November 2008, HM Treasury reported that the program led to £26.5 billion in annual efficiency gains (60 percent of which were direct cost savings while the remainder represented increased levels of public service rather than immediate cash savings). These final results have not been audited, although portions of earlier reported efficiency gains were reviewed by the UK National Audit Office (NAO) with mixed results. In 2007, more than halfway through implementing the efficiency program, the NAO reviewed a sample of the reported efficiency gains and found that some had a significant risk of inaccuracy. Nevertheless, NAO concluded at the time that of the £13.3 billion ($21.2 billion) reported gains, 26 percent (£3.5 billion ($5.6 billion)) fairly represented efficiencies achieved, 51 percent (£6.7 billion ($10.7 billion)) appeared to represent improvements in efficiency but had associated measurement issues and uncertainty, and 23 percent (£3.1 billion ($4.9 billion)) had potential to represent improvements in efficiency, but the measures used either had not yet demonstrated efficiency or the reported gains could be substantially incorrect. NAO cited measurement problems arising from longstanding weaknesses in departments’ data systems and from trying to measure savings in areas with complex relationships between inputs and outputs. Despite the caveats identified by NAO in trying to verify the reported efficiency gains, NAO reported that “the efficiency program made important contributions and there is now a greater focus on efficiency among senior staff.” In the U.S., several state governments initiated a variety of governmentwide reviews. For example, Arizona initiated an efficiency review in 2003 to try to find ways to improve customer service, reduce cost, and eliminate duplication while drawing heavily on internal state resources and experts in state government to manage the effort. The Arizona review investigated potential savings in 12 statewide, or crosscutting, issues that affected multiple agencies and offered the greatest potential for efficiency savings. In 2004, California initiated an ongoing review, the California Performance Review, with four major components: executive branch reorganization, program performance assessment and budgeting, improved services and productivity, and acquisition reform. Iowa Excellence is another governmentwide effort designed to improve customer service and cut costs in state government. Iowa agencies examined their performance using Malcolm Baldrige National Quality Program criteria. The state governmentwide review efforts share these beneficial features: serving as an effective method of cost-saving analysis, helping with prioritizing services to citizens, and providing a targeted goal for the administration of state governments that may contribute to improved government efficiency and effectiveness. Solving the daunting fiscal challenges facing the nation will require rethinking the base of existing federal spending and tax programs, policies, and activities by reviewing their results and testing their continued relevance and relative priority for a changing society. Such a reexamination offers the prospect of addressing emerging needs by weeding out programs and policies that are outdated or ineffective. Those programs and policies that remain relevant could be updated and modernized by improving their targeting and efficiency through such actions as redesigning allocation and cost-sharing provisions, consolidating facilities and programs, and streamlining and reengineering operations and processes. While significant efficiency gains can be achieved by restructuring outmoded government organizations and operations to better meet current needs, we have reported that such restructurings can be immensely complex and politically charged. All key players must be involved in the process—Congress, the President, affected executive branch agencies, their employees and unions, and other interested parties, including the public. The fundamental restructuring of the health care system for veterans in the mid-1990s and the Department of Defense (DOD) Base Realignment and Closure (BRAC) process demonstrate the significant efficiencies that can result from reexamining the base of federal programs. In the mid-1990s, the U.S. Department of Veterans Affairs (VA), recognizing that its health care system was inefficient and in need of reform, followed the lead of private sector health care providers and began reorganizing its system to improve efficiency and access. In 1995, VA introduced substantial operational and structural changes in its health care system to improve the quality, efficiency of, and access to care by reducing its historical reliance on inpatient care. VA shifted its focus from a bed-based, inpatient system emphasizing specialty care to one emphasizing primary care provided on an outpatient basis. To support VA’s restructuring efforts, Congress enacted legislation in October 1996 that eliminated several restrictions on veterans’ eligibility for VA outpatient care, which allowed VA to serve more patients. VA also phased in a new national resource allocation method, the Veterans Equitable Resource Allocation (VERA) system, as part of a broader effort to provide incentives for networks and medical centers to improve efficiency and serve more veterans. Networks that increased their patient workload compared with other networks gained resources under VERA; those whose patient workloads decreased compared with other networks lost resources. As we reported, VA recognized that VERA networks were responsible for fostering change, eliminating duplicative services, and encouraging cooperation among medical facilities. We reported that increased efficiency resulting from increased outpatient care, staff reductions and reassignments, and integrations at the medical centers resulted in savings. For example, from fiscal year 1996 to 1998, the VA reduced staff by approximately 16,114 (8 percent), resulting in estimated annual savings of $897 million. In some cases, however, improvements in efficiency did not save money because hospitals reinvested funds to enhance or offer new services. The military base realignment and closure experience provides another example of the efficiencies that can be gained by reexamining outmoded government structures and operations to meet current operating needs. In the late 1980s, changes in the national security environment resulted in a defense infrastructure with more bases than DOD needed. To enable DOD to close unneeded bases and realign other bases, Congress enacted legislation that instituted BRAC rounds in 1988, 1991, 1993, 1995, and 2005. A special commission established for the 1988 round made realignment and closure recommendations to the Senate and House Committees on the Armed Services. For the succeeding rounds, special BRAC Commissions were set up, as required by legislation, to make specific recommendations to the President, who in turn sent the commissions’ recommendations to Congress. While the statutory requirements vary across the BRAC rounds, those in the 2005 round stipulate that closure and realignment decisions must be based upon selection criteria, a current force structure plan, and infrastructure inventory developed by the Secretary of Defense. Further, the selection criteria were required to be publicized in the Federal Register to solicit public comments on the criteria before they were finalized. A clear authorization was mandated by Congress involving both the executive and legislative branches of government while recognizing and involving those affected by the government’s actions. With the completion of the recommended actions for the first four BRAC rounds by 2001, DOD had significantly reduced its domestic infrastructure through the realignment and closure of hundreds of bases and had reportedly generated billions in net savings or cost avoidances during the process. While DOD’s focus for the four BRAC rounds through 1995 was largely on eliminating excess capacity, the Secretary of Defense at the outset of the BRAC 2005 round—the fifth such round taken on by the department— indicated its intent to reshape DOD’s installations and realign DOD forces to meet defense needs for the next 20 years and eliminate excess physical capacity—the operation, sustainment, and recapitalization of which diverts resources from defense capability. Both DOD and the BRAC Commission reported that their primary consideration in making recommendations for the BRAC 2005 round was military value, which includes considerations such as an installation’s current and future mission capabilities. As such, many of the BRAC 2005 recommendations involve complex realignments that reflect operational capacity to maximize warfighting capability and efficiency. We have reported that the fifth round, BRAC 2005, will be the biggest, most complex, and costliest BRAC round ever, in part because, unlike previous rounds, the Secretary of Defense viewed the 2005 round as an opportunity not only to achieve savings but also to assist in transforming the department. For example, DOD is consolidating facilities and programs through a BRAC action to relocate five training centers from across the United States into a single medical education and training center at one installation. Although anticipated savings resulting from implementing BRAC 2005 recommendations, which the department could use for other defense programs, remain an important consideration in justifying the need for this round, our calculations using DOD’s fiscal year 2010 BRAC budget estimates have shown that estimated savings DOD expects to generate over the 20-year period ending in 2025 have declined from the BRAC Commission’s estimate of $36 billion to $10.9 billion in constant fiscal year 2005 dollars. Process improvement methods can increase product quality and decrease costs, resulting in improved efficiency. Process improvement methods can involve examining processes and systems to identify and correct costly errors, bottlenecks, or duplicative processes while maintaining or improving the quality of outputs. There are numerous process methods that use different tools and techniques. For example, Six Sigma is a data-driven approach based on the idea of eliminating defects and errors that contribute to losses of time, money, opportunities, or business. The main idea behind Six Sigma is to measure the defects in a process and then devise solutions to eliminate them, helping an organization approach a high quality level. Another method is Business Process Reengineering (BPR), which redesigns the way work is done to better support the organization’s mission and reduce costs. Reengineering starts with a high-level assessment of the organization’s mission, strategic goals, and customers. As a result of the strategic assessment, BPR identifies, analyzes, and redesigns an organization’s core business processes with the aim of achieving dramatic improvements in critical performance measures, such as cost, quality, service, and speed. A 2009 study conducted by the American Productivity and Quality Center (APQC) identified a variety of methods, including Six Sigma and Business Process Re-engineering, which have been used by organizations to focus on process improvement. The study included a survey of 281 small-to-large-sized enterprises with annual gross revenue of $4.2 trillion to identify current process-focused practices and learn about process effectiveness. Survey respondents identified various efficiency related improvements resulting from their process improvement approaches, such as streamlined processes, improved customer satisfaction, quality improvements, and improved decision making. In relation to process improvement, modernizing processes through investments in technology can generate efficiency gains. Our prior work indicates that the federal government can help streamline processes and potentially reduce long-term costs by facilitating technology enhancements. For example, as shown in figure 3, growth in electronic filing has allowed the Internal Revenue Service (IRS) to reduce staff years used to process paper tax returns. As electronic filing increased between fiscal years 1999 and 2006, IRS reduced the number of staff years devoted to total tax return processing by 34 percent. We have also reported that processing is more accurate and costs are lower to IRS as a result of electronic filing—IRS saves $2.71 for every return that is filed electronically instead of on paper. The President’s 2011 Budget described a variety of initiatives the administration intends to undertake to streamline existing IT infrastructure, improve the management of IT investments, and leverage new IT to improve the efficiency and effectiveness of federal government operations. In June 2009, the U.S. Chief Information Officer (CIO) launched the IT Dashboard, which allows the American people to monitor IT investments across the federal government. The IT Dashboard displays performance data on nearly 800 investments that agencies classify as major. The performance data used to track the 800 major IT investments include schedule, cost, and the agency CIO’s assessment of the risk of the investment‘s ability to accomplish its goals. Beginning in January 2010, the U.S. CIO began holding TechStat Accountability Sessions—face-to-face, evidence-based reviews of IT programs, undertaken with OMB and agency leadership, to improve overall performance. According to the U.S. CIO’s Web site on TechStat, in some cases this review process is leading to projects being eliminated. The administration has also indicated it intends to: consolidate data centers to reduce costs and increase efficiency; pursue “cloud computing,” which will enable agencies to share information technology services and software rather than purchase or develop their own; continue to pursue various “e-government” initiatives, which are expected to deliver services more efficiently both within across agency lines; and employ federal enterprise architectures and supporting segment architectures to streamline processes and modernize services, in many cases across agency lines. In addition to these IT initiatives, the Administration has also placed emphasis on reducing errors in payments. Executive Order 13520, signed in November 2009, requires, among other things, publishing information about improper payments on the Internet, including targets for reduction and recovery, and assigning a senior official to be accountable for reducing and recovering improper payments at relevant agencies. The executive order also lays out steps intended to lead to enhanced accountability of contractors and incentives and accountability provisions for state and local governments for reducing improper payments. Consistent with OMB’s PART guidance for programs to identify procedures to improve efficiency, officials from several of the selected programs we reviewed said they had modernized information technology to reduce costs and improve services. Officials from the Department of Labor’s Job Corps program said they reduced Federal Telecommunication Costs through the use of voice over Internet protocol and other improvements in technology, while expanding the use of video conferencing and e-learning to improve customer service. As a result of these efforts, officials reported cutting communication costs by $1 million. Officials for the Department of the Interior’s Endangered Species program said they used information technology to reduce errors due to hand entry of data. They said that by eliminating manual entry of data, errors were reduced, which resulted in more accurate information and increased efficiency. Such methods are consistent with PART guidance to identify procedures, such as information technology improvements, to improve efficiency. However, the program-level focus of the PART process would not necessarily lead to an examination of efficiency improvements to be gained by improving the processes and systems outside a program’s purview. Government processes and systems can involve multiple programs within and across federal agencies. For example, we previously reviewed the cost of administering seven key human services programs and found that the federal government may help balance administrative cost savings with program effectiveness and integrity by simplifying policies and facilitating technology improvements. Simplifying policies— especially those related to eligibility determination processes and federal funding structures—could save resources, improve productivity, and help staff focus more time on performing essential program activities. By helping states facilitate technology enhancements across programs, the federal government can help streamline processes and potentially reduce long-term costs. As another example, we have reported that the federal agencies that share responsibility for detecting and preventing seafood fraud—the Department of Homeland Security’s Customs and Border Protection, the Department of Commerce’s National Marine Fisheries Service, and the Department of Health and Human Services’ Food and Drug Administration—have not taken advantage of opportunities to share information that could benefit each agency’s efforts to detect and prevent seafood fraud, nor have they identified similar and sometimes overlapping activities that could be better coordinated to use limited resources more efficiently. For example, each agency has its own laboratory capability for determining seafood species and uses different methodologies for creating standards for species identification. The result is that neither the laboratories nor the data developed in them are shared. We have recommended that agencies take a strategic approach to spending that involves a range of activities—from using “spend analysis” to develop a better picture of what an agency is spending on goods and services, to taking an organization-wide approach for procuring goods and services. We found that private sector companies have adopted these activities to help leverage their buying power, reduce costs, and better manage suppliers of goods and services. By strategically managing costs, government can improve efficiency in the same way as private sector organizations examined in our prior work. “Spend analysis” is a tool that provides information about how much is being spent for goods and services, identifies buyers and suppliers, and helps identify opportunities to leverage buying power to save money and improve performance. To obtain this information, organizations use a number of practices involving automating, extracting, supplementing, organizing, and analyzing procurement data. Organizations then use these data to institute a series of structural, process, and role changes aimed at moving away from a fragmented procurement process to a more efficient and effective process in which managers make decisions on an organizationwide basis. Spend analysis allows for the creation of lower-cost consolidated contracts at the local, regional, or global level. As part of a strategic procurement effort, spend analysis allows companies to monitor trends in small and minority-owned business supplier participation to try to address the proper balance between small and minority business utilization, in addition to pursuing equally important corporate financial savings goals for strategic sourcing. Spend analysis is an important component of the administration’s plans to improve government procurement. Along these lines, OMB issued memoranda in July and October of 2009 instructing agencies to increase competition for new contracts. The administration also set a net savings target of $40 billion to be achieved by agencies through improved contracting practices in fiscal year 2010 and 2011. The October memorandum provided agencies guidelines for increasing competition for contracts and structuring contracts to achieve the best results at the least cost to the taxpayer. Specifically, the memorandum recommends the use of spend analysis to identify the agency’s largest spending categories, analyze and compare levels of competition achieved by different organizations within the agency, determine if more successful practices may exist for obtaining greater marketplace competition for a given spending category. Among the programs we reviewed, officials from the Job Corps program reported that they achieved improvements in efficiency by using some elements of a strategic spending approach. For example, Job Corps officials indicated that the program has avoided approximately $1 million in utility costs by purchasing energy from utilities using competitive bids in deregulated markets. When an area of the country became deregulated, the program would analyze the utility prices and quantities of electricity or natural gas used by the Job Corps centers in the area. If prices in the deregulated market looked favorable, the energy contracts for the centers would be placed out for bid to all eligible energy suppliers. Job Corps would select the bid with the best price and terms and set up a contract to purchase energy from them for a fixed period of time (usually 1 or 2 years). When the contracts came to an end, the process would be repeated. If the prices on the deregulated market were not favorable at that time, then the centers could revert back to the local utilities for their energy. Job Corps also conducted energy audits to identify problem areas and propose solutions to reduce energy costs at facilities where energy usage was above the benchmark. Job Corps reportedly reduced energy costs through investments in energy saving projects, training of staff and students to control energy use, and using an online system to review and analyze billing and procurement of energy in deregulated markets. The administration has not clearly indicated whether it will continue to emphasize measuring efficiency at the program level as it did under PART. Rather, in describing its approach to performance and management in the President’s budget, the Administration stated that GPRA and PART increased the production of measurements in many agencies, resulting in the availability of better measures than previously existed; however, these initial successes have not led to increased use. To encourage senior leaders to deliver results against the most important priorities, the administration tasked agencies with identifying and committing to a limited number of priority goals, generally three to eight, with high value to the public. The goals were to have ambitious, but realistic, targets to achieve within 18 to 24 months without need for new resources or legislation, and well-defined, outcome-based measures of progress. Further, in the coming year, the Administration will ask agency leaders to carry out a similar priority-setting exercise with top managers of their bureaus to set bureau-level goals and align those goals, as appropriate, with agencywide priority goals. These efforts are not distinct from the goal-setting and measurement expectations set forth in GPRA, but rather reflect an intention to translate GPRA from a reporting exercise to a performance improving practice across the federal government. By making agencies’ top leaders responsible for specific goals that they themselves have named as most important, the Administration has stated that it hopes to dramatically improve accountability and the chances that government will deliver results on what matters most. To complement the renewed focus on achieving priority outcomes, the Administration has also proposed increased funding to conduct program evaluations to determine whether and how selected programs are contributing to desired outcomes. The Administration intends to take a three-tiered approach to funding new program initiatives. First, more money is proposed for promoting the adoption of programs and practices that generate results backed up by strong evidence. Second, for an additional group of programs with some supportive evidence but not as much, additional resources are allocated on the condition that the programs will be rigorously evaluated going forward. Third, the approach encourages agencies to innovate and to test ideas with strong potential— ideas supported by preliminary research findings or reasonable hypotheses. We have previously reported on how program evaluations can contribute to more useful and informative performance reports through assisting program managers in developing valid and reliable performance reporting and filling gaps in needed program information, such as establishing program impact and reasons for observed performance and addressing policy questions that extend beyond or across program borders. In addition to program evaluations that determine program impact or outcomes, we have identified cost-effectiveness analysis as a means to assess the cost of meeting a single goal or objective, which can be used to identify the least costly alternative for meeting that goal. In addition cost- benefit analysis aims to identify all relevant costs and benefits, usually expressed in dollar terms. Given the challenges program managers we interviewed cited in developing and using outcome-based efficiency measures, such evaluations might fill gaps in understanding the cost of achieving outcomes and allow for cost comparisons across alternative program strategies intended to produce the same results. GPRA’s focus on strategic planning, development of long-term goals, and accountability for results provides a framework that Congress, OMB, and executive branch agencies could use to promote and apply various approaches to achieving efficiency gains in federal agencies. Congress enacted GPRA in part to address waste and inefficiency in federal programs. Agencies could use strategic plans as a vehicle for identifying longer-term efficiency improvement goals and strategies for achieving them. They could use annual performance plans to describe performance goals designed to contribute to longer-term efficiency goals, and annual performance and accountability reports to monitor progress toward achieving annual or longer-term efficiency goals. GPRA could provide a framework that would balance efforts to improve efficiency with overall improvements in outcomes. GPRA was intended to provide a balanced picture of performance that focused on effectiveness as well as efficiency. Officials from some selected programs identified a risk that focusing on reducing costs to improve efficiency could potentially have negative effects on the quantity or quality of outputs or outcomes. For example, officials for the Smaller Learning Communities program at the Department of Education said their outcome-level efficiency measures, which tracked the cost per student demonstrating proficiency or advanced skills in math or reading, could result in unintended negative consequences such as providing motivation for grantees to cut costs by lowering teacher salaries, lower proficiency standards so that more students would be classified as proficient, or engage in “creaming” (focus only on those students most likely to achieve gains). OMB’s PART guidance included recognition that efforts to improve efficiency can involve risk to quality, outcomes, or other factors such as customer satisfaction. The PART guidance included as an example how reducing processing time to be more efficient could result in increased error rates. OMB recommended that programs assess risks associated with efficiency improvement efforts and develop risk management plans if needed. Similarly, in the United Kingdom’s governmentwide efficiency program, departments could only report improvements in efficiency if they could also demonstrate that the quality of public services was not adversely affected by the reforms. Under GPRA, agencies’ plans and performance measures are expected to strike difficult balances among competing demands, including program outcomes, cost, service quality, customer satisfaction, and other stakeholder concerns. Therefore agencies could mitigate the risk to program outcomes and quality associated with taking a narrow cost-cutting approach by developing GPRA goals, strategies, and performance measures that clearly balance these competing demands. We have previously reported that OMB could use the provision of GPRA that calls for OMB to develop a governmentwide performance plan to address critical federal performance and management issues, including redundancy and other inefficiencies in how we do business. It could also provide a framework for any restructuring efforts. This provision has not been fully implemented, however. OMB issued the first and only such plan in February 1998 for fiscal year 1999. Further, as the focal point for overall management in the executive branch, OMB could provide guidance and management and reporting tools to increase federal agencies’ focus on efficiency improvements. OMB’s main vehicle for providing guidance on the development of agency strategic plans and performance plans and reports, OMB Circular A-11, Section 6 (Preparation and Submission of Strategic Plans, Annual Performance Plans, and Annual Program Performance Reports), makes no reference to establishing long-term goals for efficiency gains or describing strategies for how performance outcomes can be achieved more efficiently. References to efficiency in the guidance primarily pertain to the inclusion of program-level efficiency measures in agency budget justifications. OMB could also support mechanisms to share information and encourage agency efforts to improve efficiency. OMB has previously developed or contributed to mechanisms for sharing information and encouraging improvements to federal programs in the past, such as Web sites to share information, highlight success, and identify best practices for initiatives. For example, www.results.gov had information on best practices related to PMA initiatives, and www.expectmore.gov provided information on PART assessments and improvement plans. OMB’s own Web site contained information and examples of what it considered to be high- quality PART performance measures; discussion papers on measurement topics, such as how to effectively measure what a program is trying to prevent; and strategies to address some of the challenges of measuring the results of research and development programs. OMB recently launched a collaborative wiki page which is intended to provide an online forum for federal managers to share lessons learned and leading practices for using performance information to drive decisionmaking. OMB has sponsored various management councils, such as the President’s Management Council and the Performance Improvement Council, which include representatives of agencies and serve as forums for information sharing among agencies and with OMB. We have also reported that OMB has hosted standing working groups and committees comprised of agency and OMB staff, and has hosted workshops to address important issues and identify and share best practices. For example, OMB helped form a subgroup among agency officials responsible for the PMA budget and performance integration initiative to share lessons learned and discuss strategies to address challenges of developing efficiency measures in the grant context. The prior Administration’s approach to improving efficiency under PMA and PART focused on measuring and achieving efficiency gains at the program level. The approach involved requiring each program to develop at least one efficiency measure and demonstrate annual gains in efficiency, as well as to have regular procedures in place for achieving improvements in efficiencies. Although most programs that received a PART assessment developed an efficiency measure, not all of these measures included both elements of a typical efficiency measure—an input as well as an output or outcome. The absence of these typical elements can result in measures that do not truly capture efficiency. Nevertheless, other forms of measures intended to improve efficiency, such as those focused on reducing costly error rates, could still provide useful information. Officials for some selected programs we reviewed indicated that the efficiency measures reported for PART were useful and described ways in which they used data for efficiency measures, such as to evaluate proposals from field units, lower the cost of a contract, or make decisions to shift production. Other officials we interviewed did not find the measures useful for decision making. Officials for all of the programs described challenges to developing and using efficiency measures that were similar to challenges we previously reported on in prior work on PART and performance measures in general. For example, in one case the way OMB defined the program boundaries did not line up well with how managers ran the activities, which resulted in measures that were not useful for decision making. Some program officials indicated it was not always feasible to meet the requirement to demonstrate annual gains in efficiency, given that improvement could take multiple years to achieve. Some officials cited inconsistencies and limitations in the guidance and technical support from OMB on how to develop and use efficiency measures. OMB has not clarified whether programs should continue to collect and use efficiency measure data established for PART. Such clarification is necessary to help guide any refinements, as needed, to the current process, as well as broader issues. While tracking efficiency at the program level can be useful, this approach can miss opportunities to seek efficiencies on a larger scale, such as efforts that cross traditional program and agency boundaries. The experiences of private and public sector entities in implementing strategic and crosscutting approaches to improving efficiency can provide insights for federal agencies. For example, process improvement and modernization of systems can be undertaken both within and across organizational boundaries to increase quality, reduce waste, and lower costs. Analyzing spending and procurement strategies to leverage buying power and improve performance can identify opportunities to reduce the cost of producing agency outputs and outcomes. Broader, governmentwide reviews and analysis of restructuring opportunities that involve a wider scope of government activity can be used to identify strategic, crosscutting approaches to improving efficiency that emphasize the need to maintain or improve other key dimensions of performance. Such approaches have the potential to yield significant gains in efficiency that would be difficult to achieve by individual programs working in isolation. The current Administration has begun to identify some important opportunities for crosscutting efficiencies in its proposed information technology initiatives and procurement reforms and has tasked agencies with establishing agency cost reduction goals and asked federal employees to submit their suggestions for cost savings. Efforts to improve efficiency can take multiple years to accomplish and can require changes in strategy and collaboration within and across organizational lines. Furthermore, efficiency can only be improved if other performance dimensions, such as the quality or quantity of agency outputs and outcomes, are maintained or improved as resources are reduced; or conversely, if quality and quantity of outputs/outcomes are improved with a given level of resources. The Administration has signaled its intent to make greater use of program evaluation to determine which programs are producing desired results. Program evaluations can also be used to determine the cost of achieving outcomes, an approach that could aid in identifying the most cost-effective program designs. Continuing to build on the experiences and lessons learned from prior initiatives, with a concerted focus on specific levels of governments— governmentwide, agency, and program—could help to identify, introduce, and sustain additional efficiency gains on a more systematic and systemic basis at these same levels. The planning and reporting requirements of GPRA could serve as a framework for developing agency or across-agency strategies for improving efficiency and tracking results. By implementing the governmentwide performance plan provision of GPRA, OMB could provide further impetus to identifying efficiency goals to be achieved by consolidating operations or restructuring programs on a governmentwide basis. Further, OMB’s A-11 guidance on preparing agency strategic and performance plans could place greater emphasis on improvements in efficiency. OMB has multiple management groups and information- sharing mechanisms, including a new wiki, which could be used to identify and share successful approaches to improving efficiency, whether applied at the program or other levels of government. We recommend that the Director of OMB take the following four actions: Evolve toward a broader approach that emphasizes identifying and pursuing strategies and opportunities to improve efficiency at each of the governmentwide, agency, and program levels. At the governmentwide level, OMB should look for additional opportunities to consolidate or restructure duplicative or inefficient operations that cut across agency lines. One vehicle for doing this is the GPRA-required governmentwide performance plan. At the agency level, OMB should clarify its A-11 guidance to agencies on establishing efficiency goals and strategies in their agency-level GPRA strategic and performance plans, and reporting on the results achieved in performance reports. Guidance should stress the importance of looking for efficiencies across as well as within components and programs and maintaining or improving key dimensions of performance such as effectiveness, quality, or customer satisfaction, while also striving for efficiency gains. At the program level, OMB should clarify whether agencies are to continue developing and using program-level efficiency measures. If so, OMB should provide enhanced guidance and technical support to agencies that addresses how to develop and use efficiency measures to improve efficiency and mitigate the challenges we identified. Collect and disseminate information on strategies and lessons learned from successful efforts to improve efficiency by federal agencies, other governments, and the private sector. Possible vehicles for collection and dissemination of this information include good practices guides, workshops, Web sites, wikis, and management councils, such as the President’s Management Council and the Performance Improvement Council. We provided a draft of this report for review to OMB and the Departments of Agriculture, Education, the Interior, Labor, and Transportation. In oral comments, OMB representatives indicated that OMB concurred with our recommendations, adding that they thought the report will be useful as they revise their guidance to agencies on how to address efficiency improvements. OMB also provided technical comments which we incorporated where appropriate. In their written comments (see app. IV), Interior also concurred with our recommendations, but urged caution with regard to the recommendation that OMB provide additional guidance on the use of efficiency measures by agencies and programs. In particular, Interior cautioned against inviting standardized direction that would have agencies comparing efficiency across and within programs, considering the inherent differences in scope, complexity, and quality of outputs and outcomes. Interior indicated it seeks maximum flexibility for federal managers in using efficiency measures when they make sense and can be used to drive to the desired goals for the program. The Departments of Education and Labor provided technical comments, which we incorporated where appropriate. The Departments of Agriculture and Transportation did not provide comments. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 30 days from the date of this letter. At that time, we will send copies of this report to the appropriate congressional committees; the Secretaries of Agriculture, Education, the Interior, Labor, and Transportation; the Director of OMB; and other interested parties. The report will also be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions regarding this report, please contact me at (202) 512-6543 or steinhardtb@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V. The objectives of our review were to examine: (1) the types of efficiency measures reported through the Program Assessment Rating Tool (PART) for agency programs overall, and particularly for selected programs in five selected agencies, focusing on the extent to which they included typical elements of an efficiency measure; (2) for selected programs, the extent to which programs reporting efficiency measures through PART have shown efficiency gains and how programs have used efficiency measures for decision making; (3) for selected programs, the types of challenges to developing and using efficiency measures they have faced; and (4) other strategies that can be used to improve efficiency. To address these objectives, we selected five departments from those on which we had reported in 2007 concerning implementation of a managerial cost accounting system (MCA). Because we wanted to include agencies with variety in the types of cost data available, we selected some departments that had—and some that had not—developed an MCA system. The Departments of the Interior, Labor and Transportation were selected because these were the only departments out of the 10 agencies we reviewed at the time that had implemented—or had made significant progress in implementing—MCA departmentwide. To compare and contrast findings from these departments, we selected two other departments that had not implemented an MCA system. The United States Department of Agriculture was selected because the department indicated in our 2007 report that it planned to implement an MCA system the next time it upgraded its financial management system. The Department of Education was selected because it indicated it had no plans to implement an entitywide MCA system. After choosing the departments, we selected 21 programs to review from the set of all programs that had a received a PART assessment by the Office of Management and Budget (OMB). PART was developed to assess and improve program performance so that the federal government could achieve better results. According to OMB, a PART review helped identify a program’s strengths and weaknesses to inform funding and management decisions aimed at making the program more effective. A PART review included program-level performance information and efficiency measures for the programs. The PART data we received from OMB contained 1,396 efficiency measures which were associated with 937 programs that received a PART assessment. Within the five departments, we selected the 21 specific programs for review to represent a diverse array of functions and operations within the federal government, as indicated by the PART program type. Of the seven PART program types, we selected five for inclusion in this study, excluding research and development and credit. Additional criteria were that the selected programs have relatively large fiscal year 2008 funding levels, and variety in the number of efficiency measures associated with the programs. For the first objective regarding the extent to which efficiency measures included typical necessary elements, we first identified the elements and developed a definition by conducting a literature review as well as expert interviews. We then performed various degrees of analysis on (1) all efficiency measures for all programs represented in the PART database, (2) all of the measures for our selected programs, and (3) a random sample of 100 efficiency measures taken from the PART database. The following describes the analysis we conducted on each of these three populations: Analysis on the complete PART database: The analysis we conducted on all PART efficiency measures resulted in a set of summary statistics, such as the fiscal year 2008 total funding by PART program type, the mean amount of funding each program received within the program types, the number of programs for each PART program type, the number of programs that had between zero and eight efficiency measures, and the number of programs in each selected department by PART program type. Analysis of PART measures selected with certainty from 21 programs in five departments: For the 21 programs we selected, we conducted a more detailed analysis on the 36 associated efficiency measures. However, any findings based on this analysis cannot be generalized beyond these particular measures. We performed a content analysis review of these measures, which was based upon the PART efficiency measure data; our review of applicable documents concerning the measures and programs, such as the programs’ PART assessments; and interviewing program officials to discuss the measures and programs. For each of these measures, we identified whether certain attributes were present, and the documents we reviewed and interviews we conducted aided in this effort at times. The fields from the PART database we used to assess each efficiency measure were the agency and program name, the text for each efficiency measure and, when present, the more detailed efficiency measure explanation. Using this information, we determined whether each of the measures included the program’s inputs (such as cost or hours worked by employees) as well as its outputs or outcomes. When we identified a measure as having an output or outcome element, we distinguished between the two. We also analyzed whether there was either a time or cost attribute to each measure. For each of these attributes, the potential answers were “Yes,” “No,” or “Unclear.” To determine whether an efficiency measure had these attributes, we defined each term for this particular exercise. We defined an input as a resource, such as cost or employee time, used to produce outputs or outcomes. We defined outputs as the amount of products and services delivered by a program. We defined outcomes as the desired results of a program, such as events, occurrences or changes in conditions, behaviors or attitudes. We defined a measure to have an attribute of time or cost when the measure appeared to include some type of attribute of time (e.g., “hours worked by employees,” “per month,” “annually,” or “within three months,”) or cost, respectively. We conducted our coding by having three team members independently code each of the 36 efficiency measures without each knowing how the other two coders assessed each measure. Afterward, the three coders discussed and reconciled any differences and reached agreement in all incidents. Finally, we determined whether the cost element was based on budgetary information or MCA information. Analysis of a random sample from the PART database: This analysis involved selecting a random sample of 100 efficiency measures from the remaining 1,355 efficiency measures in the PART database. Estimates based on the sample can be generalized to estimate characteristics of the remaining population of 1,355 efficiency measures. Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn. Since each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval (e.g., plus or minus 10 percentage points). This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn. As a result, we are 95 percent confident that each of the confidence intervals in this report will include the true values in the study population. Unless otherwise noted, all percentage estimates have 95 percent confidence intervals of within plus or minus 10 percentage points of the estimate itself. The analysis we conducted on these measures was similar to the analysis we conducted for the selected programs, meaning we analyzed and determined if each measure had an input, output or outcome, time or cost attribute and used the same definition and coding procedures. However, because we did not have in-depth information from interviews or program documents concerning these measures, in some cases we were unable to conclude whether certain efficiency measures included necessary elements and consequently, classified about a quarter of the sample as unclear. Also, because of the lack of detailed information on the measures, we could not distinguish between outputs and outcomes expressed for these measures. Simultaneously with the content analysis of the efficiency measures, for the second and third objectives, on how selected agencies/programs used efficiency measures and the extent to which they reported efficiency gains, and what challenges or constraints to developing and using efficiency measures they faced, we reviewed program Web sites, PART assessments, other documents provided by program officials, and interviewed program officials identified by the departments as knowledgeable about the particular program and its efficiency measure(s). These interviews consisted of asking agency officials a similar set of questions with topics such as how the efficiency measure(s) was developed and used, associated challenges, and alternative methods for evaluating efficiency. For the two programs that did not have any efficiency measures in PART, we asked questions such as whether they had other efficiency-related measures they tracked internally which were unrelated to PART, whether there had been prior attempts to develop an efficiency measure, and whether they had experienced specific challenges to developing and using efficiency measures. In addition to interviewing program officials, we also interviewed at least one official in each of the five departments who was responsible for performance measurement at the departmentwide level. These interviews also had a similar set of questions and were specific to departmentwide performance measurement issues, such as whether the department had its own guidelines or guidance pertaining to developing and using efficiency measures, how results for program-level efficiency measures get reported within the agency, and how program efficiency measures were used. Also at the department level, we interviewed officials associated with each of the five departments’ Chief Financial Officer (CFO) offices, asking questions about the role the CFOs office played, if any, in developing efficiency measures for programs and inquiring about the development and use of a managerial cost accounting system. In addition to interviewing department and program officials, we interviewed OMB officials on several occasions about the approach to efficiency under PART and discussed, among other topics, the training and guidance OMB provided, and any lessons learned from the agencies’ efforts to develop and use efficiency measures. OMB also provided us with documents detailing the history of the PART program. Finally, to determine whether a selected program’s efficiency measure indicated a gain or loss, we reviewed the efficiency measure data that were reported in the program’s PART assessment and subtracted the initial year of data from the latest year available. To verify the accuracy of the data, we asked program officials to confirm the data and when necessary, to provide us with the most recent data. To address the fourth objective regarding the approaches agencies can employ to improve efficiency, we interviewed program officials for the selected programs to learn about the approaches they use to evaluate efficiency and also conducted a two-stage literature review to determine alternative approaches. The first stage of the literature review consisted of examining GAO publications, Congressional Research Service reports, the Internet, and various databases for general information on strategic approaches to efficiency. We also participated in a business process management research report with the American Productivity and Quality Center (APQC), studying how organizations maintain quality across processes and products as well as meet customer requirements in the face of pressure to cut costs. Using information derived from the first literature review and the APQC report, we identified the broad set of approaches to improving efficiency. In our literature search, we looked for examples and ideas that used a broad array of strategies to seek improvements or affect efficiency from prior reports we have published and what other institutions that have done work on the subject. For this objective, we refer to 18 different pieces of literature from our comprehensive literature search. In conducting the literature review, we did not attempt to identify all potential alternative approaches that could lead to efficiency improvements but focused on approaches that appeared consistent with the broad definition of efficiency improvement that was used in this report. Furthermore, in addition to the interviews with program officials and the literature review, we interviewed experts on performance and efficiency measures, who discussed definitions, uses, and insights of efficiency measures. Among the experts, we interviewed officials in the United Kingdom’s National Audit Office, which assessed the reliability of the efficiency gains reported by United Kingdom agencies as part of the United Kingdom’s 2004 government-wide efficiency review. We also interviewed officials with the Office of the Auditor General of Canada, which is conducting a study on ways to improve the efficiency of that country’s tax administration system. Restore, enhance, and maintain watershed conditions including soil, water, air, and forest and rangeland vegetation within the national forests and grasslands. Management of these physical and biological resources provides a foundation for healthy, viable ecosystems. Provides nutritionally balanced, low-cost or free lunches for public and nonprofit private schools. The program seeks to safeguard the health and well-being of the nation's children and support domestic agricultural production. Assists in protecting plant and animal resources from pests and diseases through ongoing monitoring and surveillance. Provides rapid detection, analysis, and reporting of pests and diseases to minimize potential losses. Awards formula grants to state education agencies which, in turn, manage statewide competitions and award subgrants to local education agencies and community-based organizations. These grants support the creation of community learning centers that provide academic enrichment opportunities during nonschool hours for children, particularly students who attend high- poverty and low-performing schools. This program focuses on enrichment in core academic subjects, extracurricular enrichment, as well as literacy and other educational services to the families of participating children. Provides competitive grants to local education agencies to increase academic achievement in large high schools through the creation of smaller, more personalized learning environments. Provides financial assistance to postsecondary students and their families through administering federal student aid grants and loans. Ensures the operation and maintenance of reclamation facilities, delivers water to irrigators and municipal users, and provides storage to help mitigate flooding. The program also addresses issues such as water conservation, runoff from irrigated fields, and project financial management. Manages and extinguishes fires on Department of the Interior lands and on other lands under fire protection agreements. The three largest program activities are fire preparedness, fire suppression, and hazardous fuels reduction (i.e., removal of small trees and brush that exacerbate fire risks). Protects threatened or endangered species and conserves their habitats. Lists species needing protection, consults on federal projects, awards grants, and works with partners on recovery actions. Works to conserve and restore native aquatic species populations and their habitat and support recreational fishing. Reclaims and restores land and water degraded by coal mining activities conducted before 1977. Reclamation fees on current coal production fund the program, which has expanded to provide oversight over the 23 states and three Indian Tribes that carry out the program. Serves those who have contracted illness due to exposure to toxic substances or radiation while working at nuclear weapons and related covered facilities. Provides lump-sum compensation and health benefits to eligible Department of Energy nuclear weapons workers, or the survivors of such workers. Provides intensive education and training services to disadvantaged youth ages 16-24. These services are intended to help eligible youth obtain jobs, seek further education, or enter the military. The program serves approximately 60,000 youth nationwide through 122 centers, most of which are residential. Works to ensure, for every working person in the nation, safe and healthful working conditions. Implements the Occupational Safety and Health Act of 1970 by setting and enforcing standards, outreach and education, cooperative programs and compliance assistance. Unemployment Insurance Administration State Grants Assists states in operating their unemployment insurance programs, which provide temporary income support to unemployed workers. States determine eligibility for benefits, which are financed through state-levied taxes. The Department of Labor funds the administrative expenses of these state programs. Provides competitive grants to fund training, employment, and other services to help economically disadvantaged farmworkers and their families. Through these services, the program seeks to help them achieve economic self- sufficiency by strengthening their ability to gain stable employment. Provides air traffic control services to guide aircraft in and out of airports across the country. Maintains and modernizes equipment needed in the national airspace system to deliver air traffic services. It fields, repairs, and maintains a network of complex equipment, including radars, instrument landing systems, radio beacons, runway lighting, and computer systems. Provides financial support for locally planned and operated public transit through competitive, discretionary capital investment grant transit projects including commuter rail, light rail, heavy rail, bus rapid transit, trolleys and ferries. Provides financial grants and technical assistance to states to construct, maintain, and improve the performance of the nation's highway system in accordance with federal policy goals. Advances highway safety through research and regulations concerning vehicle technologies and human behavior. Focuses on researching vehicle and behavioral safety countermeasures, issuing vehicle safety regulations, and investigating vehicle defects. PART program name and number of efficiency measures millions) Forest Service: Watershed (0) National School Lunch Program (NSLP) (3) Plant and Animal Health Monitoring Programs (2) Value of damage prevented or mitigated by the monitoring and surveillance programs per dollar spent Improved efficiency through the use of targeted samplings versus the use of random sampling 21st Century Community Learning Centers (3) Smaller Learning Communities (6) PART program name and number of efficiency measures millions) Student Aid Administration (1) Bureau of Reclamation Water Management—Operation and Maintenance (1) Fish and Wildlife Service— Endangered Species (0) Fish and Wildlife Service— Fisheries (1) Office of Surface Mining—State Managed Abandoned Coal Mine Land Reclamation (2) Wildland Fire Management (3) Energy Employees Occupational Illness Compensation Program (1) Occupational Safety and Health Administration (1) Unemployment Insurance Administration State Grants (1) Fiscal year 2009 funding level (dollars in millions) Workforce Investment Act— Migrant and Seasonal Farmworkers (1) Transportation Federal Aviation Administration (FAA) Air Traffic Organization— Technical Operations (2) Transportation FAA Air Traffic Organization— Terminal Programs (2) Transportation Highway Infrastructure (3) Transportation National Highway Traffic Safety Administration—Operations and Research (1) In addition to the individual named above, Elizabeth Curda, Assistant Director; Charlesetta Bailey; James Cook; Anne Inserra; Eric Knudson; Ricardo Sanchez; and Jeremy Williams made key contributions to the report. Cynthia Grant; Peter Grinnell; Carol Henn; Donna Miller; A.J. Stephens; Jay Smale; Jessica Thomsen; and John Warner also provided significant assistance.

Summary: Given record budget deficits and continuing fiscal pressures, the federal government must seek to deliver results more efficiently. The prior Administration sought to improve efficiency under the Program Assessment Rating Tool (PART) by requiring programs to have at least one efficiency measure and procedures for improving efficiency, and show annual efficiency gains. The current administration has also emphasized efficiency in some initiatives. GAO was asked to examine (1) the types of PART efficiency measures and the extent to which they included typical elements of an efficiency measure; (2) the extent to which selected programs showed gains and how they used efficiency measures for decision making; (3) the challenges selected programs faced in developing and using efficiency measures; and (4) other strategies that can be used to improve efficiency. GAO analyzed the 36 efficiency measures in 21 selected programs in 5 agencies and a generalizable sample from the other 1,355 measures governmentwide, reviewed documents and interviewed officials from selected programs, reviewed literature on efficiency, and interviewed experts. Under PART, most programs developed an efficiency measure. However, according to GAO's analysis, 26 percent did not include both typical efficiency measure elements--an input (e.g., labor hours or costs) as well as an output or outcome (e.g., the product, service, or result produced). Most frequently missing was the input (69 percent). For example, a measure developed by the National Nuclear Safety Security Administration considered the number of information assets reviewed for certification without considering costs of review. This could result in measures that do not capture efficiency. GAO has previously recommended agencies improve cost information for decision making, but they are in various stages of implementation. However, alternative forms of measurement, such as reducing costly error rates, could still be useful. Of the efficiency measures GAO reviewed that had both typical elements, a similar number reported gains and losses. Officials for some programs stated that the efficiency measures reported for PART were useful, and described ways in which they used the data, such as to evaluate proposals from field units, lower the cost of a contract, or make decisions to shift production. Others did not find the efficiency measures useful because, for example, the program lacked control over key cost drivers, such as contractually required staffing levels, or because of concern that raising output could lower quality. Officials for all of the programs reviewed described challenges to developing and using program-level efficiency measures and performance measures in general. Challenges included interpreting outcome-level efficiency information, such as the cost of improving or maintaining the condition of watershed acres, when factors other than program funding, such as past impacts from mining, affected conditions as well; achieving required annual efficiency gains in cases where a program intervention takes years to implement; and inconsistent or limited guidance and technical assistance from the Office of Management and Budget (OMB) to agencies on how to measure efficiency. A variety of approaches have been used to improve efficiency, including governmentwide reviews, agency restructurings, process and technology improvements, and strategic spending approaches. The Administration has some initiatives along these lines, such as information technology and procurement reforms. The Government Performance and Results Act (GPRA) provides a framework for planning future efficiency gains while maintaining or improving effectiveness and quality of outputs or outcomes. OMB, as the focal point for management in the executive branch, provides guidance and supports information-sharing mechanisms, such as the Performance Improvement Council, which could also be used to create a more strategic and crosscutting focus on agency efforts to improve efficiency. OMB has not clearly indicated whether programs should continue measuring efficiency nor has it emphasized efficiency in its GPRA guidance to agencies.