Government Efficiency and Effectiveness: Inconsistent Definitions and Information Limit the Usefulness of Federal Program Inventories
Released: 2014-10-31T00:00:00

Letter

Background

GPRAMA requires OMB to make publicly available, on a central governmentwide website, a list of all federal programs identified by agencies. For each program, the agency is to provide to OMB for publication: an identification of how the agency defines the term “program,” consistent with OMB guidance, including program activities that were aggregated, disaggregated, or consolidated to be considered a program by the agency; a description of the purposes of the program and how the program contributes to the agency’s mission and goals; and an identification of funding for the current fiscal year and the previous 2 fiscal years.

In addition, GPRAMA requires OMB to issue guidance to ensure that the information provided on the website presents a coherent picture of all federal programs.

In January 2012, OMB announced that it would conduct a pilot for implementing the inventory provisions for trade, export, and competitiveness programs. According to OMB’s memorandum, pilot agencies were to identify an inventory of programs and map them to established programmatic and organizational structures as well as the agency’s strategic plan and performance goals. In addition, agencies were to provide program purpose and funding information for each program. The memorandum noted that, based on the pilot, OMB would issue guidance to all federal agencies detailing the approach for developing a governmentwide inventory.

In its August 2012 update to its guidance for implementing GPRAMA, OMB included guidance for broader implementation of the inventory requirements. Initial implementation was limited to the 24 agencies that developed agency priority goals (APG) for fiscal years 2012-2013. In addition, the guidance provided a phased approach for these 24 agencies. For the first phase, for publication in May 2013, agencies were to describe their program definition approach, identify their programs, and provide limited funding and performance information. OMB subsequently published 24 separate inventory documents on the central governmentwide website required under GPRAMA, which it has implemented as Performance.gov. For the second phase, originally planned for publication in May 2014, the 24 agencies were to update their inventories based on any stakeholder feedback they received and provide additional program-level funding and performance information. OMB’s guidance also stated that at that time the inventory information was to be presented in a more dynamic, web-based approach.

However, agencies did not publish updated inventories in May 2014. According to OMB staff, plans for updating the inventories are on indefinite hold as OMB re-evaluates next steps for what type of information will be presented in the inventories and how it will be presented based on (1) recent legislative actions that could affect the information required for the inventories; (2) a lack of stakeholder feedback; and (3) insufficient funding for,related to, presenting the inventories in a web-based format on and technological challenges Performance.gov. Regarding legislative action, OMB staff are considering how implementation of the expanded reporting requirements for federal spending information under the recently-enacted Digital Accountability and Transparency Act of 2014 (DATA Act) could be tied to the program inventories. The DATA Act is intended to increase accountability and transparency in federal spending. It requires federal agencies to publicly report, on at least a quarterly basis, information about any funding made available to, or expended by, an agency or a component of the agency. This reporting covers a variety of budget information—budget authority, obligations, unobligated balances, outlays, and any other budgetary resources—at different levels of aggregation and disaggregation: appropriations account, program activities, and object class. Reporting under the act is required to begin no later than 3 years after enactment (no later than May 9, 2017). In addition, the DATA Act provides for the establishment of government-wide financial data standards to, among other things, produce consistent and comparable data across program activities. The act also holds agencies more accountable for the quality of the information disclosed. As we reported in April 2014, such increased transparency provides opportunities for improving the efficiency and effectiveness of federal spending and improving oversight to prevent and detect fraud, waste, and abuse of federal funds.

In addition, the House has passed and the Senate is considering a bill that would amend the program inventory provisions of GPRAMA. On February 25, 2014, the House passed H.R. 1423, the Taxpayers Right-to- Know Act, which would require agencies to identify, for each program, an estimate of the number of clients/beneficiaries served, an estimate of (for the previous fiscal year) the number of full-time federal employees who administer the program, and the number of full-time employees—at organizations that administer or assist in administering the program (e.g. grantees or contractors)—whose salary is paid in part of full by the federal government. It would also require agencies to provide additional program- level information in their inventories, including total administrative costs and expenditures for services. On May 21, 2014, the Senate Committee on Homeland Security and Governmental Affairs favorably reported to the full Senate S. 2113, the Taxpayers Right-to-Know Act, which would also amend the program inventory provisions of GPRAMA to require agencies to identify additional program-level information in their inventories.

Agencies Used Various Approaches to Define Their Programs, Which Limits the Comparability of Programs within and across Agencies

Agencies Used Different Approaches to Define Their Programs Based on OMB’s Guidance

GPRAMA requires agencies to identify how they define the term “program,” consistent with guidance provided by the Director of OMB. As a starting point, OMB’s guidance cites, in part, the definition for “program” contained in our September 2005 Glossary of Terms Used in the Federal Budget Process: an organized set of activities directed toward a common purpose or goal that an agency undertakes or proposes to carry out its responsibilities.glossary.

Inconsistent Definitions Limit Comparability of Programs within Agencies and across the Federal Government

The use of inconsistent approaches by agencies to define their programs, illustrated in Table 2, limits the comparability of programs within agencies as well as governmentwide. A few agencies did not use the same program definition approach across their subcomponents or offices, which limits comparability of their own programs. For example, the Department of State’s (State) programs are defined using two separate approaches. State operations programs follow a budget approach while State and U.S. Agency for International Development (USAID) foreign assistance programs follow both a budget and outcome-based approach. In addition, while the U.S. Army Corps of Engineers (Corps) is an agency within the Department of Defense (DOD), the Corps—Civil Works Program and DOD developed separate inventories that took different approaches to define programs.

To illustrate how program definition differences limit comparability across agencies for similar programs, we selected two areas of potential fragmentation, overlap, and duplication highlighted in our past annual reports: science, technology, engineering and mathematics (STEM) education and nuclear nonproliferation and determined which programs were recognizable in both. We compared the lists of programs developed from our past work to those contained in the relevant agencies’ inventories.

As shown in Table 3, of the 158 programs previously listed in our work on STEM education, we were able to identify 51 programs in relevant agency program inventories.

Of those 51, 9 were an exact match between how we previously identified a program in our past work and how the agency identified it in its inventory. For example, as figure 2 illustrates, both our past work on STEM education and the Department of Education’s program inventory identify “Mathematics and Science Partnerships” as a program.

We were able to identify the other 42 programs based on related information contained in program descriptions in relevant agency inventories. For four agencies, we were not able to identify any STEM education programs from our past work in their inventories—covering 27 of the 158 programs. In addition, although our past work identified 3 STEM education programs at the Nuclear Regulatory Commission, it was not 1 of the 24 agencies directed by OMB to produce a program inventory for 2013.

In the nuclear nonproliferation area, we were able to identify 9 of the 21 programs identified in our past work in relevant agency program inventories, as shown in table 4.

None of the nine nuclear nonproliferation programs we were able to identify in the inventories were exact matches. Rather, we were able to identify them as activities under larger programs in the inventories based on the related program descriptions. For example, our past work identified the Department of Homeland Security’s (DHS) “Global Nuclear Detection Architecture” as a program within its Domestic Nuclear Detection Office. DHS’s inventory identifies one program, Domestic Radiological/Nuclear Detection, Forensics, and Prevention Capability, within the Domestic Nuclear Detection Office. The inventory describes Global Nuclear Detection Architecture as an activity within that program, as illustrated in figure 3.

For both illustrative examples, we were not able to locate a majority of the programs identified in our prior work in the relevant agency inventories. The lack of comparability of similar programs across agencies in part could be due to differences in the way we previously defined “program” and the way each agency defined its programs, such as different levels of aggregation of related activities to constitute a program. The flexibility OMB’s guidance afforded in defining programs allows agencies to align their definitions with multiple purposes, including carrying out their mission and goals; serving beneficiaries, customers, and other target populations; and providing benefits, services, and products. However, it also can lead to differences in how programs are identified.

In addition, the lack of comparability may also be the result of agencies not working across organizational boundaries when developing their inventories. We asked agency officials whether they had worked with any external parties when determining their program definition approach or identifying their programs. Officials from all 24 agencies stated that they had not sought input from any entities outside their own agencies, with the exception of OMB. One of OMB’s stated purposes for the inventories is to facilitate coordination among programs that contribute to similar outcomes. If agencies worked together to more consistently define their programs, it could also help them identify where they have programs that contribute to similar outcomes, and therefore opportunities to collaborate.

In a Few Instances, Several Agencies Did Not Include Activities in Their Inventories Based on OMB’s Criteria for Determining What Constitutes a Program

OMB’s guidance directed agencies to work with OMB to determine the appropriate primary approach (or mix of approaches) and level of aggregation/disaggregation to be used to define their programs. It also identified characteristics agencies should consider when determining Officials at each of the 24 what constitutes a program (see text box).agencies told us that they worked with OMB—with most stating it was staff in relevant Resource Management Offices as well as the Office of Performance and Personnel Management—to determine (1) the appropriate approach and (2) what constitutes a program. Similarly, staff within OMB’s Office of Performance and Personnel Management told us that they worked closely with agencies as they were defining and identifying their programs.

OMB’s Characteristics of a Program Externally recognizable. Agencies should use programs that are or relate to programs or objectives used in Congressional Budget Justifications, statute, are recognized by Congress and stakeholders, or are already publicly known; agencies should use program names that are known outside the agency, and generally not create new names.

Operationally meaningful. Agencies should use programs that are operationally meaningful to agency senior leadership and components of the agency, and programs should represent how the agency is managed and delivers on its mission. Link to an organizational component(s), such as headquarters, bureau or office. Programs should be operationally meaningful to the agency and agency senior leadership. Persistent. Generally, programs that persist over time should be included. However, agencies have the flexibility to identify short-term efforts as programs, such as activities related to the Recovery Act.

When we asked officials at each agency if the inventory covered everything the agency does, most responded that the inventory included all agency activities. However, officials at a few agencies told us that their inventories did not always include certain activities. In most cases, activities were excluded from the inventory because they did not meet one of the characteristics of a program described in OMB’s guidance. For instance, the Small Business Administration (SBA) only included programs that were permanent in nature and excluded pilot programs from its inventory because, according to SBA officials, they did not consider pilot programs to meet the “persistent” characteristic. As an example, they cited SBA’s Boots to Business program, which was in a pilot phase when the agency’s 2013 inventory was published. SBA officials told us they expected to add the program in the next inventory update as the pilot phase had been completed. Because OMB’s guidance does not clearly define when a short-term activity has persisted long enough to be considered a program, agencies may be using different criteria for when to include them as programs in their inventories. As a result, agencies’ inventories may not be as comprehensive as desired. Clearer guidance from OMB could lead to a more complete picture of federal programs in the inventory.

In other instances, Office of Personnel Management (OPM) officials told us they did not include a voting rights oversight program in their inventory because it was not operationally meaningful, in that it did not easily align with OPM’s mission. Social Security Administration (SSA) officials told us they, in consultation with OMB staff, did not include the Special Benefits for Certain World War II Veterans program in their inventory because it is relatively small in size, in terms of its funding ($8 million of the agency’s total net budget authority of $55.9 billion in fiscal year 2013) and number of beneficiaries (approximately 1,000, based on information in SSA’s fiscal year 2015 congressional budget justification) compared to the agency’s other programs. In response to a question during our interview, Department of the Interior (Interior) officials realized they did not include functional management offices, such as the Offices of Policy and the Chief Financial Officer, as part of a set of “programs” in their inventory falling under the Office of the Secretary. Agency officials at OPM, SSA and Interior told us they intend to account for these programs in future iterations of their program inventories.

Presenting Program Budget Information, Providing Additional and Consistent Performance Information, and Seeking Stakeholder Feedback Could Help Ensure Inventories Are Useful to Decision Makers

The federal government is one of the world’s largest and most diverse entities, with about $3.5 trillion in outlays in fiscal year 2013, funding an extensive array of programs and operations. In responding to the varied and increasingly complex issues the federal government seeks to address, it faces a number of significant fiscal, management, and governance challenges. To operate as effectively and efficiently as possible and to make difficult decisions to address these challenges, Congress, the administration, and federal managers must have ready access to reliable and complete financial and performance information. However, crucial information on the federal government’s programs is decentralized, according to OMB’s guidance. The federal program inventory has the potential to improve public understanding about what federal programs currently operate and how programs link to budget, performance, and other information by centralizing this information from disparate sources—such as the President’s Budget, Congressional Budget Justifications, USAspending.gov, and the Catalog of Federal Domestic Assistance—in a single location: Performance.gov. Agencies should consider the differing needs of various users—agency top leadership and line managers, OMB, Congress, and stakeholders—to ensure that this information will be both useful and used in decision To be useful, this information must meet users’ needs for making. accuracy, completeness, consistency, reliability, and validity, among other factors.

Agencies Generally Did Not Present Program-level Budget Information in Their Inventories but DATA Act Implementation Could Provide More Detailed and Consistent Data

GPRAMA requires agencies to identify for each program in their inventories: the program activities that are aggregated, disaggregated, or consolidated to be considered a program by the agency; and funding for the current and two previous fiscal years (for the inventories published in May 2013, that would have been fiscal years 2013, 2012, and 2011).

OMB’s 2012 and 2013 guidance delayed implementation of these provisions until the planned May 2014 update.

Although OMB’s guidance delayed this, some agencies provided this information in their inventories. A few agencies provided information about how their program activity (PA) lines align with their programs as defined in their inventory. For example, the National Science Foundation’s inventory notes that programs presented in it are consistent with the PA lines presented in the budget. In addition, the Department of Transportation’s inventory states that its programs generally consist of one to four PA lines. For example, the Pipeline and Hazardous Materials Safety Administration’s “Pipeline Safety” program receives its funding from four PA lines.

In addition, OMB’s guidance directed agencies to include funding (budget authority) for agency bureaus or subcomponents for fiscal years 2012, 2013, and 2014 (requested). Since some agencies identified a bureau/subcomponent to constitute a program, the bureau-level funding information in their 2013 inventories also met GPRAMA’s requirement for program-level funding information. For example, the Department of Energy (Energy) defined its Office of Nuclear Energy as a program.

Because the office is also considered a subcomponent, Energy provided the related funding information, as illustrated in figure 4.

As noted earlier, plans for updating the inventories are on indefinite hold and agencies did not publish updated inventories, with the planned program-level budget data, in May 2014—in part due to the enactment of the DATA Act. According to OMB’s 2014 update to its guidance, it is working with agencies to merge implementation of the DATA Act and federal program inventory requirements to the extent possible, but has not yet determined its implementation strategy. Agency reporting for both sets of requirements is web-based, which could more easily enable linkages between the two sites or incorporating information from each other. The Senate version of the Taxpayers Right-to-Know Act would require this linkage in lieu of incorporating this budget information on the program inventory site. As OMB stated, one of its purposes in implementing the program inventory requirements is to consolidate information from various sources. By leveraging the DATA Act’s requirements, OMB could expand the transparency of budget information made available in the inventories—beyond PA lines and budget authority and at different levels of disaggregation—and help ensure that information is consistent across agencies. In addition, the disaggregated budget information is important since our annual reports on fragmentation, overlap, and duplication have found that it is not always available for the programs and activities covered by our work. However, the DATA Act provides up to 3 years for full implementation of its reporting requirements. Thus, until OMB and agencies are ready to move forward with the implementation of the two laws’ requirements, one option to help ensure the inventories remain relevant and useful would be for OMB to direct agencies to update the existing information in the 2013 versions for publication.

Agencies Defined the Purposes of Their Programs but Did Not Consistently Show How Programs Contribute to Agency Missions, Goals, and Results

GPRAMA requires agencies to describe for each program in their inventories the purposes of the program and the program’s contribution to the agency’s mission and goals.

OMB’s guidance for the 2013 inventories directed agencies to provide a program description (purpose) and identify supported strategic goals and objectives. In addition, they were to add language directing readers to Performance.gov to see programs supporting their agency priority goals (APG) and the cross-agency priority (CAP) goals.

Agencies provided descriptions of their programs’ purposes. However, some agencies did not consistently show how each program supports agency missions and goals.

With the exception of one inventory—the joint State/USAID inventory for foreign assistance programs—all provided a purpose for each program, although State and USAID provided a link to a separate document that provided purpose information for their foreign assistance programs.

Only a few inventories (4 of 24) identified how the agencies’ programs contribute to their mission. However, because agency strategic goals and objectives are an outgrowth of the agency’s mission, the linkage between agency programs and those goals can show the program’s support to the mission.

A majority of the inventories (17 of 24) identified the agency strategic goals each program supported, while less than half of the inventories (11 of 24) identified the agency strategic objectives each program supported.

Finally, most of the inventories (20 out of 24) referred readers to Performance.gov for information about how the agencies’ programs support both CAP goals and APGs.

OMB’s review of the inventories did not always identify instances where agencies omitted this information. Providing this information allows agencies to explain how programs contribute to the results the federal government is achieving. Consistently showing the alignment between programs and goals could also improve implementation of other GPRAMA provisions. First, GPRAMA requires OMB and agencies to identify for publication on Performance.gov the various organizations, program activities, and other activities that contribute to the CAP goals and APGs, respectively. In addition, OMB’s guidance directs agencies to identify these contributors for their strategic objectives. These provisions are important because they show how agencies are coordinating efforts toward a common outcome. As we have previously reported, uncoordinated program efforts can waste scarce funds, confuse and frustrate program customers, and limit the overall effectiveness of the federal effort. Our past work has found that OMB and agencies did not always identify all relevant contributors to their CAP goals and APGs. Accordingly, in May 2012 and April 2013, we made recommendations to OMB. Table 5 describes these recommendations and related actions taken to date.

Second, GPRAMA requires OMB and agencies, on at least a quarterly basis, to review and report on Performance.gov progress towards the CAP goals and APGs, respectively. As part of these reviews, OMB and agencies are to (1) involve relevant stakeholders from contributing organizations and program activities and (2) assess the contributions of the various organizations, program activities, and other activities that support these goals.GPRAMA directed agencies, beginning this year (2014), to conduct annual reviews of progress towards their strategic objectives—the outcomes or impacts the agency is intending to achieve. According to OMB’s guidance, the strategic reviews should inform many of the agency’s decision-making processes, as well as decision making by the In addition, OMB’s 2013 guidance for implementing agency’s stakeholders, such as informing long-term strategy, annual planning and budget formulation; strengthening collaboration on crosscutting issues; and improving transparency. If successfully implemented in a way that is open, inclusive, and transparent—to Congress, delivery partners (e.g., state and local governments, grantees, non-profit organizations, associations, contractors), and a full range of stakeholders—these reviews could help decision makers assess the relative contributions of various programs that contribute to a given goal. Successful reviews could also help decision makers identify and assess the interplay of public policy tools that are being used, to ensure that those tools are effective and mutually reinforcing, and results are being efficiently achieved. However, our past work has found that relevant contributors were not always included in these reviews and accordingly we made recommendations to OMB, as described in table 6.

OMB’s guidance does not direct agencies to identify in their inventories the performance goals to which each program contributes. Although not specifically required for the program inventory, GPRAMA requires agencies, in their annual performance plans, to identify the various organizations, program activities, and other activities that contribute to each performance goal. As noted in the Senate Committee on Homeland Security and Governmental Affairs report that accompanied GPRAMA, this provision, among others, was intended to help describe the strategies to be used to achieve results and the resources to be applied to those strategies. This information can help Congress understand and assess the relationship between the agency’s resources and results. Having a clear description of the strategies and resources an agency plans to use will allow Congress to assess the likelihood for an agency to achieve its intended results. The act also requires that agency performance goals and related information be available in a web-based format on Performance.gov. According to OMB staff, efforts to expand the site to include this information have not been possible given the available appropriations for the E-Government Fund, which supports Performance.gov and other electronic government requirements and initiatives.

Agencies Did Not Take Steps to Ensure Inventories Provide Useful Information for Stakeholders, including Congress

OMB’s 2012 guidance directed agencies to work with key stakeholders, which can include Congress, state and local governments, third party service providers, and the public, to validate that their program inventories would be both internally and externally recognizable. According to OMB staff, they want to ensure that the inventories are useful and meaningful to the agencies and their stakeholders.

Although officials we spoke with or received written responses from at each of the 24 agencies told us that they involved internal stakeholders— officials and staff from various bureaus/component agencies and offices within their respective department or agency—to validate their programs, most also shared that they saw no benefit to their agencies from developing the inventories. Many told us that they viewed it as a paperwork exercise, repackaging information available elsewhere with the resulting inventory not useful to their internal agency decision making. In a few instances, officials offered different views. For example, an official at DHS told us that her department’s approach to linking programs to performance information helped the department communicate the results it was achieving. She noted that DHS has maintained an external-facing, mission-oriented program structure since it was created. In addition, an official from SBA told us he thought the program inventories—once fully implemented as envisioned in OMB’s guidance for the 2014 update— could help his agency identify instances to enhance their coordination and collaboration with other agencies to achieve common outcomes.

Officials we spoke with or who provided written responses to our questions also told us that they did not solicit feedback on their inventories from external stakeholders, including Congress. In addition, none of them reported receiving any feedback on their inventories beyond that from OMB staff during development. In some instances, agency officials told us that they didn’t seek stakeholder input because their approach for defining their programs resulted in programs with which their stakeholders were already familiar. For example, officials at several agencies told us that using a budget approach resulted in programs that were recognizable to congressional appropriators. In other instances, agency officials told us they thought OMB staff were collecting feedback on the inventories, but they had not heard any results at the time of our interview. When we asked OMB staff about any feedback they had solicited or received, they told us they had briefed two congressional committees when the inventories were published in May 2013, but that they had not received any formal feedback from Congress or any other stakeholders, with the exception of the preliminary results of our review, which we published as part of a testimony before the Senate Committee on Homeland Security and Governmental Affairs in March 2014. In addition, although Performance.gov has a mechanism for users to provide feedback on the website, OMB staff stated that they had not received any feedback on the program inventories from that venue.

See, for example, GAO-13-518; GAO-13-174; GAO-12-621SP; GAO, Performance Budgeting: PART Focuses Attention on Program Performance, but More Can Be Done to Engage Congress, GAO-06-28 (Washington, D.C.: Oct. 28, 2005); Managing For Results: Enhancing the Usefulness of GPRA Consultations Between the Executive Branch and Congress, GAO/T-GGD-97-56 (Washington, D.C.: Mar. 10, 1997); and Executive Guide: Effectively Implementing the Government Performance and Results Act, GAO/GGD-96- 118 (Washington, D.C.: June 1996). stakeholder input, and congressional input in particular, on program inventories and the information presented therein would provide OMB and agencies another opportunity to ensure they are presenting useful information for stakeholder decision making.

Identifying All Federal Programs and Providing Sorting Capabilities Could Help Provide a More Coherent Picture of Federal Involvement in Particular Areas

GPRAMA requires OMB to ensure that the inventory information provided by agencies and published on Performance.gov presents a coherent As our annual reports on fragmentation, picture of all federal programs.overlap, and duplication have stated, the federal program inventory could be a key tool for addressing crosscutting issues. For example, as highlighted in OMB’s guidance, the federal program inventory has the potential to facilitate coordination across programs by making it easier to find programs that may contribute to a shared goal. Moreover, in its memorandum regarding the inventory pilot project for trade, export, and competitiveness programs, OMB noted that duplicative programs make government less effective, waste taxpayer dollars, and make it harder for the public to navigate government services. It states that, in order to continue efforts to reduce duplication and overlap and improve program outcomes through better coordination across agencies, the executive branch must achieve greater transparency into all federal programs.

OMB Has Not Defined Plans to Expand Inventories to Include All Federal Programs

GPRAMA requires OMB to make publicly available a list of all federal programs identified by agencies. While the 24 agencies have begun implementing the inventory requirements, OMB has not defined plans for when this effort will be expanded beyond these agencies. As noted earlier in the report, the lack of such a comprehensive list makes it difficult to determine the scope of the federal government’s involvement in particular areas and, therefore, where action is needed to address crosscutting issues.

In addition, OMB has not included all types of federal programs in its plans for the program inventory. OMB’s guidance identified 12 different types of federal programs, defined in table 7, for agencies to assign to their programs.each program in their inventories.

The Ability to Find and Sort Similar Programs or Programs Contributing to Shared Outcomes Would Enhance the Usefulness of Inventories

As noted earlier, GPRAMA requires OMB to publish the federal program inventory on Performance.gov. While the inventories published in May 2013 were individual agency documents, OMB’s guidance and staff have stated that eventually the inventory would move to a more dynamic, web- based approach—originally planned for the May 2014 update and now on hold. This web-based approach could make it easier to tag and sort related or similar programs. For instance, OMB’s plans to have agencies tag each of their programs by one or more program type in a future iteration of the inventory would provide a sorting capability for identifying the same type of program. By providing a sorting mechanism by program type, OMB could help address one of our open recommendations, described in the text box below, by identifying (1) all programs in a given type, and (2) of those programs, any that have developed strategies to effectively overcome measurement challenges.

GAO Recommended OMB and the PIC Develop a Detailed Approach to Examine and Address Long-standing Performance Measurement Challenges In our June 2013 report, Managing for Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges (GAO-13-518), we found that agencies continue to face common, long-standing difficulties in measuring the performance of various types of federal programs and activities—contracts, direct services, grants, regulations, research and development, and tax expenditures. We recommended the Director of OMB work with the PIC to develop a detailed approach to examine these difficulties across agencies, including identifying and sharing any promising practices from agencies that have overcome difficulties in measuring the performance of these program types. In commenting on a draft of the report, OMB staff agreed with this recommendation. As of July 2014, OMB and the PIC have taken some initial steps to address this recommendation. According to OMB staff, this includes efforts related to achieving several of the CAP goals. For example, the “Benchmark and Improve Mission-Support Operations” CAP goal involves developing common standards and benchmarks to measure the performance and cost of various agency administrative operations, such as information technology and acquisition management. In addition, PIC staff told us they have taken initial steps to address performance measurement issues in a few areas, including a pilot effort focused on acquisitions (contracts). PIC staff said they plan to expand the model to focus on other types of programs with performance measurement issues, such as grants and regulations. We will continue to monitor progress.

As highlighted in OMB’s guidance, the federal program inventory has the potential to facilitate coordination across programs by making it easier to find programs that may contribute to a shared goal or a common outcome. This could also help identify and address instances of fragmentation, overlap, and duplication. In its guidance for the 2014 update before it was put on hold, OMB intended for agencies to link each program to the existing web pages on Performance.gov for strategic goals, strategic objectives, APGs, and CAP goals. According to OMB staff, once they move forward with the next inventory update and move to a web-based presentation, this would allow users to sort programs by the goals to which they contribute. This approach also would allow users to identify programs that contribute to broader themes on Performance.gov. The themes generally align with budget functions from the President’s Budget and include administration of justice; general science, space, and technology; national defense; and transportation, among others. Currently, the themes can be used to sort goals on Performance.gov that contribute to those broad themes.

Conclusions

The coordinated efforts of multiple federal agencies, different levels of government, and sectors are generally needed to achieve meaningful results. This is underscored by our annual reports on fragmentation, overlap, and duplication, which to date have identified over 90 areas, each of which involves a patchwork of federal agencies, programs, and activities that attempt to address the same issue. Executive branch and congressional efforts to identify, manage, or resolve instances of fragmentation, overlap, and duplication are hindered by the lack of a comprehensive list of all federal programs. Such a list, along with related budget and performance information, could help decision makers determine the scope of the federal government’s involvement, investment, and performance in a particular area, which in turn could help pinpoint where action is needed to better address or avoid fragmentation, overlap, and duplication. Therefore, effective implementation of GPRAMA’s federal program inventory requirements could provide decision makers with critical information that could be used to better address crosscutting issues, among other purposes.

The executive branch has taken some initial steps to develop program inventories with related budget and performance information. In 2012, OMB involved 11 agencies in a pilot program inventory effort for federal trade, export, and competiveness programs and subsequently developed guidance—based on lessons learned from the pilot—for broader implementation at 24 agencies. In May 2013, the 24 agencies published inventories, providing information about 1,524 programs they collectively identified. Despite these initial efforts, we have identified a number of areas where implementation should be improved.

OMB needs to take further actions to effectively meet GPRAMA’s requirement for the inventories to present a coherent picture of all federal programs. First, OMB’s guidance provides agencies with flexibility in how they defined their programs. As a result, agencies used different approaches to identify their programs, which, in turn, led to a lack of comparability for similar programs across agencies. This was illustrated by our analysis of agency inventories for STEM education and nuclear nonproliferation programs, in which we could only exactly match a small fraction of the programs covered by our past work on fragmentation, overlap, and duplication in those areas. According to OMB staff, the flexibility provided in guidance is derived from lessons learned from the pilot effort—that agencies are different for valid and legitimate reasons and a one-size-fits-all approach would not work for all agencies. While this may be true, OMB could do more to direct agencies to find common ground on similar programs. One of OMB’s stated purposes for the inventories is to facilitate coordination across programs that contribute to similar outcomes. However, as we discovered through our interviews with agency officials involved with the inventory efforts, none of the agencies sought input from other agencies on how they defined and identified their programs. If agencies worked together to more consistently define their programs, it could also help them identify where they have programs that contribute to similar outcomes, and therefore opportunities to collaborate.

Second, OMB’s plans to date do not adequately ensure that the inventories include all federal programs. It has yet to provide firm plans on when implementation of GPRAMA’s inventory requirements will be expanded beyond the current 24 agencies. OMB’s guidance does not provide a clear time frame for how long activities must persist to be considered programs, which may have resulted in agencies excluding certain activities from their inventories. In addition, OMB did not include tax expenditures in the inventory effort. Although GPRAMA does not specify that tax expenditures should be included in the program inventory, our work over the past 20 years has shown the need for tax expenditures to be held to the same scrutiny as spending programs, given the sizeable federal investment they represent. The omission of these agencies, programs, and activities severely limits the usefulness of the inventory as a tool for addressing crosscutting issues.

In addition, other factors limit the usefulness of the inventory as a source of information for decision makers. To be useful, information must meet various users’ needs for accuracy, completeness, consistency, reliability, and validity, among other factors. Agency officials also told us they did not consult with external stakeholders, including Congress. Although OMB’s 2012 guidance instructed agencies to seek this input, none of them did. Subsequently, OMB has removed this direction from its guidance. By consulting with stakeholders to understand their needs, agencies would better ensure that the information provided in the inventories is useful for stakeholder decision making. Involving congressional stakeholders is of critical importance given Congress’s power to create and fund programs.

Our work also identified limitations with the performance information contained within the inventories. Agencies did not consistently identify the strategic goals, strategic objectives, agency priority goals, and cross- agency priority goals that each program supports, as directed by OMB guidance. These omissions were not always identified during OMB’s review of each agency’s inventory. In addition, OMB’s guidance does not direct agencies to identify in their inventories the performance goals to which each program contributes. Although not required for the inventories, GPRAMA requires this type of connection—albeit by performance goal rather than by program—in agency performance plans. This information can help Congress and others understand and assess the relationship between the agency’s resources and results. Without it, it will be difficult for Congress to assess the likelihood of the agency’s success in achieving intended results.

Recommendations for Executive Action

To ensure the effective implementation of federal program inventory requirements and to make the inventories more useful, we make the following eight recommendations to the Director of the Office of Management and Budget.

To better present a more coherent picture of all federal programs, we recommend the Director of OMB take the following five actions: direct agencies to collaborate with each other in defining and identifying programs that contribute to common outcomes, and provide a time frame for what constitutes “persistent over time” that agencies can use as a decision rule for whether to include short-term efforts as programs; define plans for when additional agencies will be required to develop include tax expenditures in the federal program inventory effort by designating tax expenditure as a program type in relevant developing, in coordination with the Secretary of the Treasury, a tax expenditure inventory that identifies each tax expenditure and provides a description of how the tax expenditure is defined, its purpose, and related performance and budget information.

To help ensure that the information agencies provide in their inventories is useful to federal decision makers and key stakeholders, and to provide greater transparency and ensure consistency in federal program funding and performance information, we recommend the Director of OMB take the following three actions: revise relevant guidance to direct agencies to consult with relevant congressional committees and stakeholders on their program definition approach and identified programs when developing or updating their inventories, and identify in their inventories the performance goal(s) to which each program contributes; and ensure, during OMB reviews of inventories, that agencies consistently identify, as applicable, the strategic goals, strategic objectives, agency priority goals, and cross-agency priority goals each program supports.

Agency Comments

We provided a draft of this report for review and comment to the Director of OMB and the 24 agencies that developed program inventories.

In oral comments provided on October 15, 2014, staff from OMB’s Office of Performance and Personnel Management stated that they agreed with five of our eight recommendations and would consider how to implement those recommendations as they move forward with merging program inventory implementation with that of the DATA Act. For the other three recommendations—designating tax expenditures as a type of program, developing an inventory of tax expenditures, and directing agencies to identify how their programs contribute to their performance goals—OMB staff neither agreed nor disagreed. OMB staff told us that until they had firmer plans on how program inventory and DATA Act implementation would be merged, they could not determine if implementing these three recommendations would be feasible.

We also received technical comments from OMB, the Departments of Energy and State, and the National Aeronautics and Space Administration, which we incorporated as appropriate.

We are sending copies of this report to the Director of OMB and the heads of the 24 agencies that developed program inventories as well as interested congressional committees and other interested parties. This report will also be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6806 or mihmj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report. Key contributors to this report are listed in appendix II.

Appendix I: Objectives, Scope & Methodology

The GPRA Modernization Act of 2010 (GPRAMA) requires us to review implementation of the act at several critical junctures. This report is part of our response to that mandate. Our specific objectives for this report were to (1) assess how OMB and agencies defined and identified the programs contained in the inventories, (2) examine the extent to which the inventories provide useful information for federal decision makers, and (3) examine the extent to which the inventories provide a coherent picture of the scope of federal involvement in particular areas.

To address all three objectives, we assessed the implementation of relevant GPRAMA requirements by the Office of Management and Budget (OMB) and the 24 agencies that developed program inventories, which were published on Performance.gov in May 2013. requires OMB to make publicly available on a central governmentwide website, a list of all federal programs identified by agencies. For each program, the agency is to provide to OMB for publication: an identification of how the agency defines the term “program,” consistent with OMB guidance, including program activities that were aggregated, disaggregated, or consolidated to be considered a program by the agency; a description of the purposes of the program and how the program contributes to the agency’s mission and goals; and an identification of funding for the current fiscal year and the previous 2 fiscal years.

GPRAMA also requires OMB to issue guidance to ensure that the information provided on the website presents a coherent picture of all federal programs.

The 24 agencies directed by OMB to develop program inventories are the Departments of Agriculture, Commerce, Defense, Education, Energy, Health and Human Services, Homeland Security, Housing and Urban Development, the Interior, Justice, Labor, State, Transportation, the Treasury, and Veterans Affairs, as well as the U.S. Agency for International Development, U.S. Army Corps of Engineers—Civil Works Program, Environmental Protection Agency, General Services Administration, National Aeronautics and Space Administration, National Science Foundation, Office of Personnel Management, Small Business Administration, and Social Security Administration.

We also assessed program inventory implementation based on relevant OMB guidance (Circular No. A-11, Part 6) and related leading practices from our past work on managing for results, such as those related to involving Congress and stakeholders in performance management activities and ensuring performance information is useful to decision makers. In addition, we developed a structured set of questions and conducted interviews with, or received written responses from, officials involved in developing the inventories at the 24 agencies. We also interviewed staff from OMB’s Office of Performance and Personnel Management who led governmentwide implementation of these requirements and developed guidance for agencies.

Additionally, to address our first objective, we selected two areas of fragmentation, overlap, and duplication from our annual reports—science, technology, engineering, and mathematics (STEM) education and nuclear nonproliferation programs—and compared the lists of programs developed by our work to those contained in agency inventories. To select these two areas, we first reviewed the existing body of 83 areas of fragmentation, overlap, and duplication published in our 2011, 2012, and 2013 annual reports,areas that focused on management functions, since they were unlikely to be captured in the program inventories. For the remaining 60 areas, we identified the number of agencies involved in the area as reported in our past worked, whether those agencies had developed and published an inventory in May 2013, and whether the area primarily involved domestic or foreign assistance/national security programs—to ensure diversity among the agencies and programs we reviewed. Finally, we sorted those lists to select the areas with the most agencies involved in them: STEM education for domestic programs and nuclear nonproliferation programs for foreign assistance/national security programs. and identified and eliminated from selection 23 We compared the lists of programs developed from our past work on STEM education and nuclear nonproliferation to those contained in the relevant agencies’ inventories. We identified possible matches in two different manners. First, we identified instances in which program names exactly matched between the lists developed for our past work and the agency’s inventory. In addition, we also reviewed the program descriptions contained in the relevant inventories to determine if they would lead one to programs/activities related to STEM education or nuclear nonproliferation. For example, if the program description contained language such as “science education” or “technology education,” we included that program as a broader match. In some instances, we were also able to identify programs from our past work listed as activities within the program descriptions in relevant inventories. To ensure consistency in this analysis, the work to identify programs conducted by one analyst was reviewed and verified by another analyst.

Because the two selected areas are a non-generalizable sample of instances where fragmentation, overlap, and duplication exist, our results cannot be generalized more broadly. However, the results of our analysis illustrate how various approaches to defining programs can lead to differences in how programs are identified.

We conducted this performance audit from August 2013 to October 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the above contact, Lisa M. Pearson (Assistant Director) and Benjamin T. Licht supervised this review and the development of the resulting report. Gerard Burke, Amy Bush, Virginia Chanley, A. Nicole Clowers, Karin Fangman, Heather Krause, and Steven Putansu made significant contributions to this report. Carole J. Cimitile, Dewi Djunaidy, Emily Gruenwald, Donna Miller, Leah Q. Nash, and Erinn L. Sauer also provided key contributions.