Federal Programs: Information Architecture Offers a Potential Approach for Development of an Inventory
Published: 2017-09-28T00:00:00

Fast Facts:
We reported in 2015 that initial efforts to develop an inventory of federal programs—something that could make government more transparent and reveal areas of duplication and overlap—had fallen short due to inconsistent definitions and information across agencies.
In this report we examined how the principles and practices of information architecture—a discipline focused on how information is organized, structured, and presented—could help fix that.
We identified key steps drawn from information architecture for developing and maintaining such an inventory, enabling decision makers to more easily find and compare program information.

Highlights:
section_title: Why GAO Did This Study
paragraphs: Each year the federal government spends trillions of dollars through dozens of agencies and thousands of federal programs. Given its sheer size and scope, providing a clear and complete picture of what the federal government does and how much it costs has been a challenge in the absence of a comprehensive resource describing these programs. The GPRA Modernization Act of 2010 (GPRAMA) requires the Office of Management and Budget (OMB) to present a coherent picture of all federal programs by making information about each program available on a website to enhance the transparency of federal government programs.
Congress included a provision in GPRAMA for GAO to review the implementation of the act. GAO has chosen to conduct this study now because OMB has not yet developed an inventory that meets GPRAMA requirements. For this report, GAO addresses how one potential approach for organizing and structuring information—the principles and practices of information architecture—can be applied to develop a useful federal program inventory. To present illustrative examples of what programs and program information could be included in an inventory, GAO examined budget, performance, and other resources that could be used to develop an inventory. These examples were also used to illustrate the potential content and structure of an inventory and to identify any challenges.
GAO is not making recommendations in this report.
We provided a draft of this report for review and comment to the Director of the Office of Management and Budget (OMB), the Departments of Education and Homeland Security, the U.S. Agency for International Development (USAID), and the General Services Administration. USAID provided technical corrections, which GAO incorporated as appropriate. OMB agreed to consider this information architecture approach as it develops plans for the next iteration of the federal program inventory.
section_title: What GAO Found
paragraphs: A useful federal program inventory would consist of all programs identified, information about each program, and the organizational structure of the programs and information about them. The principles and practices of information architecture—a discipline focused on organizing and structuring information—offer an approach for developing such an inventory to support a variety of uses, including increased transparency for federal programs. GAO identified a series of iterative steps that can be used to develop an inventory and potential benefits of following this approach. GAO also identified potential challenges agencies may face in developing a full program inventory.
To identify potential benefits and challenges to applying these steps, GAO developed a hypothetical inventory, focusing on three case study agencies—the Departments of Education (Education) and Homeland Security and the U.S. Agency for International Development. Potential benefits of using such an approach to develop a federal program inventory include the following:
Stakeholders have the opportunity to provide input into decisions affecting the structure and content of the inventory. For example, congressional staff told GAO that an inventory with 5 years of budgetary trend data on programs would be more useful than 3 years of data.
A range of information through program facets is available for cross-program comparisons, such as budget, performance, beneficiaries, and activities.
An inventory creates the potential to aggregate, disaggregate, sort, and filter information across multiple program facets. For example, the figure below illustrates how program facets could be used to identify programs that provide similar services—in this case, early learning and child care services—and discover budget and other information for each of the programs identified.
An iterative approach to development and governance of the federal program inventory can result in improvements and expansions of the inventory over time.
GAO also identified potential challenges agencies may face when using this approach to develop an inventory, including the following:
Challenges in determining how agencies should identify and structure their programs in an inventory will need to be addressed, including how to treat spending categories not clearly linked to specific programs, such as administrative support. This may occur because agencies vary in their missions and organizational and budget structures and in how they organize their activities.
Challenges in collecting information for each program facet may occur for some agencies and programs. This may happen because a greater range of program information may be more readily available for some programs than others. GAO found that this was often dependent on the extent to which certain programs were included by name in budget documents, strategic plans, and agency websites.
Challenges related to determining what should be identified as a program and the structure and content of the inventory will need to be balanced with usefulness and costs. Agencies may need to weigh the costs that they might face in collecting and reporting program facet information as they establish priorities.

section_title: Letter
paragraphs: Each year the federal government spends trillions of dollars on thousands  of programs through dozens of agencies. Given the federal government’s  sheer size and scope, providing a clear and complete picture of what the  federal government does and how much it costs has been a challenge.  Federal government activities are spread across numerous programs and  agencies, and there is no comprehensive resource describing these  programs or information about their cost, what services they provide, and  how they are performing. The lack of a comprehensive inventory of  federal programs makes it difficult to assess the scope of the federal  government’s involvement in particular areas and, therefore, limits the  ability of decision makers to determine where action is needed to make  government more effective and efficient. As we have previously reported,  this includes limiting the government’s ability to identify and address  programs that are overlapping or duplicative or that operate in a  fragmented manner across multiple agencies.
The GPRA Modernization Act of 2010 (GPRAMA) requires the Office of  Management and Budget (OMB) to present a coherent picture of all  federal programs by making information available about each federal  program on a website. Relatedly, the Digital Accountability and  Transparency Act of 2014 (DATA Act), once fully implemented, is  expected to link federal contract, loan, and grant spending information  with specific federal programs. Programs are defined in the Glossary of  Terms Used in the Federal Budget Process as “generally, an organized  set of activities directed toward a common purpose or goal that an agency  undertakes or proposes to carry out its responsibilities.” A federal  program inventory would consist of the individual programs identified by  the agencies and OMB and information collected about each of them.
We reported in 2014 that initial efforts to develop an inventory have  resulted in inconsistent definitions and significant information gaps. As a  result, the inventory did not provide useful information for decision  making. Further, we stated that without such an inventory, efforts to link  spending information to specific programs as part of DATA Act  implementation will be hampered. Therefore, we recommended OMB  accelerate efforts to determine how best to merge GPRAMA and DATA  Act requirements to produce a federal program inventory that meets  Congress’s needs. OMB staff generally agreed with our  recommendations regarding the implementation of the inventory, although  they stated that until they had firmer plans on how program inventory and  DATA Act implementation could be merged, they could not determine if  implementing all of our recommendations would be feasible. See  appendix II for a list of these prior recommendations and more details on  their implementation status.
Since our 2014 report on the initial program inventory, we have  conducted additional work to better understand what OMB and agencies  could do to develop a federal program inventory. During this time, we  identified the theories and practices of information architecture as a  potential framework for the development of a useful federal program  inventory. Broadly speaking, information architecture is the practice of  deciding how to arrange the parts of something to be understandable.  Information architecture focuses on organizing, structuring, and labeling  content in an effective and sustainable way to support findability,  manageability, and usability from the infrastructure level to the user  interface level. The goal is to help users find information and complete  tasks. To do this in the context of a federal program inventory, agencies  would more effectively contribute to the inventory if they are able to  understand how the pieces fit together to create the larger picture and  how items relate to each other within the system.
GPRAMA includes a statutory provision for us to periodically evaluate and  report on (1) how implementation of the act is affecting performance  management at the 24 major departments and agencies subject to the  Chief Financial Officers (CFO) Act of 1990, including whether  performance management is being used to improve efficiency and  effectiveness of agency programs; and (2) crosscutting goal  implementation. This report is part of our response to that mandate. Our  objective for this report was to examine how the principles and practices  of information architecture can be applied for the development of a useful  federal program inventory. We reviewed several resources in the field of  information architecture and discussed its applicability to developing a  federal program inventory with practitioners in the field. We also  examined how these principles and practices align with guidance and  other good practices for the development of federal websites, which  include practices related to quality and usefulness, among other areas, as  described in our methodology in appendix I.
Our methodology involved describing the general steps that could guide  the development of a federal program inventory using an information  architecture approach, as well as assessing how the principles and  practices of information architecture could be used to identify programs  and to identify, compile, and organize information within an inventory.  This report is not meant to suggest requirements or best practices for  developing the federal program inventory, but rather to illustrate how a  particular approach could be applied to develop a useful federal program  inventory. Other approaches might also be used—or could be  incorporated into this framework—to develop an inventory that best  addresses limitations identified in the past.
As part of our effort to apply relevant principles and practices to program  identification, we examined efforts to develop program inventories at both the federal  and state levels, including the initial federal effort to develop a federal  program inventory and our assessment of that effort; examined prior OMB and GAO guidance for developing lists of federal developed a set of observations on using budget-related information  sources—primarily budget justifications and program activity data— to identify programs in selected agencies: the Departments of  Education (Education) and Homeland Security (DHS), and the U.S.  Agency for International Development (USAID). These three agencies  were drawn from those included in the initial effort to develop a federal  program inventory and selected based on how they approached that  effort, including whether they specifically connected their inventory to  their programs included in the Catalog of Federal Domestic  Assistance (CFDA). We used budget documentation as a starting  point to identify programs, because most agencies followed a similar  approach for their initial inventories and because GPRAMA requires  inventories to include budget information.
As part of our effort to apply relevant principles and practices to identify,  compile, and organize information about federal programs, we  identified the types of information about programs that could be  included in an inventory to make it useful, such as budget,  performance, and operations information;  collected information from our recent reports, notably reports on  programs that provide early learning or child care services and that  support manufacturing, to identify programs and information about  those programs;  collected program information for selected individual programs— selected to achieve a range across program size, agency, and  availability of information—to develop illustrative examples of the  opportunities and challenges that obtaining such information presents; organized information about these programs in a hypothetical  inventory to illustrate how a federal program inventory might be  structured to promote usefulness; and presented program information in a hypothetical inventory about  programs drawn from our review of early learning or child care  services to congressional stakeholders overseeing education (i.e.,  relevant majority and minority staff working for the Senate Committee  on Health, Education, Labor, and Pensions and the House Committee  on Education and Welfare) to obtain their opinions and perspectives  on how, and to what extent, the information, as organized, could be  helpful in supporting their needs for program oversight and promoting  transparency.
We conducted this performance audit from July 2016 to September 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: GPRAMA requires OMB to make publicly available, on a central  government-wide website, a list of all federal programs identified by  agencies. For each program, each agency is to provide to OMB for  publication an identification of how the agency defines the term “program,”  consistent with OMB guidance, including program activities that were  aggregated, disaggregated, or consolidated to be considered a  program by the agency; a description of the purposes of the program and how the program  contributes to the agency’s mission and goals; and an identification of funding for the current fiscal year and the previous  2 fiscal years.
In addition, GPRAMA requires OMB to issue guidance to ensure that the  information provided on the website presents a coherent picture of all  federal programs.
In August 2012, OMB issued guidance for implementation of the inventory  requirements through a phased approach for the 24 agencies subject to  the CFO Act. OMB subsequently published 24 separate inventory  documents on Performance.gov in May 2013, wherein agencies were to  select an approach for identifying programs and provide funding and  performance information for the programs identified. For the second  phase, originally planned for publication in May 2014, the 24 agencies  were to update their inventories based on any stakeholder feedback they  received and provide additional program-level funding and performance  information. OMB’s guidance also stated that, at that time, the inventory  information was to be presented in a more dynamic, web-based  approach. However, agencies did not publish updated inventories in May  2014.
In October 2014, we reported that according to OMB officials plans for  updating the inventories were on indefinite hold as OMB re-evaluated  next steps for what type of information would be presented in the  inventories and how it would be presented. OMB staff were considering  how implementation of the expanded reporting requirements for federal  spending information under the DATA Act could be tied to the program  inventories. As of July 2017, OMB had not provided a timeline or plan for  the next iteration of the federal program inventory.
In our 2014 assessment of the executive branch’s initial effort to develop  a program inventory, we found that the usefulness of the 24 agency  inventories was limited. Agencies had the flexibility to identify their  programs using different approaches within the broad definition of what  constitutes a program, which—while potentially appropriate for individual  agencies—limited the comparability of information across the inventory.  Further, we found that the agencies did not work together or consult with  stakeholders. We also found that none of the agencies provided the  necessary budget and performance information. Without performance  information, it was unclear how programs supported various agency  goals. We also determined that for the federal program inventory to be  useful it must be accurate, complete, consistent, reliable, and valid,  among other factors. We recommended a number of specific steps  OMB and agencies could take to ensure the inventories are more useful  to decisions makers, including providing complete performance  information (including performance goals), consulting with stakeholders,  and ensuring that information in the inventory is comparable within and  across agencies. As mentioned above, OMB staff generally agreed with  these recommendations, although they did not comment on three of our  recommendations related to including tax expenditures and additional  performance information.
section_title: Information Architecture Provides a Potential Approach for Developing a Federal Program Inventory
paragraphs: The principles and practices of information architecture—a discipline  focused on how information is organized, structured, and presented to  users—may offer an overarching approach for developing a useful federal  program inventory. There are three key concepts in information  architecture that are relevant to the development of a federal program  inventory—facet, controlled vocabulary, and taxonomy. Table 1 defines  these terms and provides examples of what they mean within the context  of a federal program inventory. Decision rules provide consistency in how  programs are included, in the application of the controlled vocabulary,  and in the collection of program information in facets.
Information architecture can be visualized as a process to identify and  define needed information, develop a structure for organizing and  presenting it, and ensure that standards are met and maintained. These  steps may not be purely sequential, but may be iterative as the inventory  is developed, evaluated, and maintained. Based on the principles of  information architecture, figure 1 provides a conceptual overview of this  potential process for developing a federal program inventory. Each of  these steps is described more fully in the sections following.
As program information in the inventory is collected and organized into  facets, it can be aggregated or disaggregated to facilitate various uses.  Facets—and the information or data collected within them—can be  structured to allow for searching, grouping, or other functions. Individual  facets could describe program characteristics or operations or could  relate to budgeting or performance information, among other things.  Within these facets, specific information or data would be reported such  as program type, specific agency or office names, or budget data. By  organizing information according to facets, programs can be identified,  grouped, or organized based on certain characteristics, such as the  information or data collected within the facet. For example, if a program  facet on beneficiaries existed, then potentially all programs that serve the  same types of beneficiaries could be identified within the inventory.
In the following sections we provide examples of how these principles and  practices can be applied to federal programs. However, since we  developed these examples for illustrative purposes only, this does not  necessarily mean that they should be included in the inventory.
subsections: section_title: Step 1: Establish Purpose and Uses
paragraphs: The federal program inventory is intended to improve transparency over  federal programs. There can be many specific uses for the inventory to  support this purpose, and input from stakeholders—such as Congress,  agency officials, state and local governments, third-party service  providers, and the public—can assist in establishing these potential uses.  Early stakeholder involvement can also guide efforts to determine what  programs and program information should be included so that the  inventory is more likely to meet stakeholder needs over time.
We have reported that a federal program inventory including performance  information could be used by congressional decision makers to inform  decision making to identify issues that the federal government should  address, to measure progress, and to identify better strategies, if  necessary, among other uses. State officials that we interviewed from  three states that have developed or are developing program inventories  pointed to similar potential uses. For example, in Vermont a senior  performance management official said the state’s inventory could be used  to facilitate cross-agency coordination, aid government transitions (such  as for newly elected legislators), and link program performance with  funding. Likewise, an Arizona official told us the state’s program inventory  has the potential to show how agency programs align with performance  goals. Michigan officials anticipate that the program inventory currently  being developed in that state will have the potential to identify duplication  and overlap among state programs. As a result, decision makers in that  state will be better equipped to oversee the budget process.
Our prior work highlights potential uses for different types of information  that could be included in a program inventory. Performance and  budgeting information—including, among other types of information,  performance goals, targets, and time frames; measures of efficiency;  operations, such as activities and services; and costs—could facilitate a  variety of potential uses, such as helping decision makers prioritize resources among programs or  identifying pressing issues for the government to address;  informing congressional decisions about authorizing or reauthorizing  federal programs, provisions in the tax code, and other activities; and determining the scope of the federal government’s involvement,  investment, and performance in a particular area.
Prioritization of some uses may be important to consider to make the  inventory more effective. As we previously reported, consulting with  stakeholders to understand their needs would better ensure that the  information provided in the inventories is useful for stakeholder decision  making. Such prioritization, for example, could also involve examining  costs that agencies might face in collecting information for certain facets.  Then decisions could be made to select only a subset of all potential  facets for inclusion in early iterations of the inventory.
Tax expenditures are one program type that would need to be included in  the program inventory to fully implement GPRAMA. Tax expenditures  represent a substantial federal commitment. If the Department of the  Treasury’s estimates are summed, an estimated $1.23 trillion in federal  revenue was forgone from the 169 tax expenditures reported for fiscal  year 2015, an amount comparable to discretionary spending. Tax  expenditures are often aimed at policy goals similar to those of federal  spending programs. Increased transparency over tax expenditures could  help determine how well specific tax expenditures work to achieve their  goals and how their benefits and costs compare to those of spending  programs with similar goals. In our 2014 review of the executive branch’s  initial effort to develop a program inventory, we recommended OMB  include tax expenditures as a program type in the federal program  inventory and work with the Department of the Treasury to produce an  inventory of tax expenditures. As stated previously, OMB neither agreed  nor disagreed with those recommendations.
Likewise, to enhance usefulness at the federal level, a program inventory  can include program operations information, in addition to the budget and  performance information required by GPRAMA. Program operations  information can include descriptions of what programs do, whom they  serve, and the specific activities they conduct. Including this type of  information provides a more comprehensive picture of a program within  the inventory. There are many potential benefits, including improved  ability to identify, assess, and address fragmentation, overlap, and  duplication within the federal government. Likewise, program operations  information can provide opportunities to enhance service delivery among  programs offering similar services or serving related populations. For  example, programs serving low income or transportation disadvantaged  populations could look for opportunities to facilitate access to related  services by coordinating to provide transportation for these beneficiaries.
section_title: Step 2: Identify Concepts and Terms
paragraphs: One of the central tasks in creating an inventory of federal programs is to  identify the programs to be included and the information to be collected  about them. Information architecture practices suggest selecting  information sources to compile a list of concepts and terms as part of a  controlled vocabulary. For example, stakeholders may frequently use  certain terms and concepts to describe programs and make distinctions  between different types of programs that can affect the content of the  inventory or the information included within it. Thus, grant programs may  describe eligible beneficiaries using similar terms, such as rural and  urban or youth and elderly. Once the list of concepts and terms has been  compiled using a structured process for identifying key terms and  concepts, preferences can be selected that best align with meeting user  needs to create facets for the inventory. Potential information sources  include agency budgets, budget justifications, performance reports,  organizational structures, websites, and other internal documentation.
Additionally, the facets that will frame information about those programs  would need to be identified and defined, with OMB deciding which facets  warrant the cost of collection in the short and long term with input from  agencies and stakeholders, if the information architecture approach is  used. According to the National Information Standards Organization  (NISO), the design and development of a controlled vocabulary can help  to ensure that concepts are described distinctly by eliminating ambiguity  and controlling for synonyms. As a result, the use of a controlled  vocabulary can help agencies identify programs and collect associated  program information in facets more consistently.
subsections: section_title: Programs and Other Organizing Concepts Vary, Presenting Challenges
paragraphs: Differences in how agencies use terms and concepts—especially those  related to “program,” “program area,” and “activity”—create challenges for  an inventory, which requires consistent information to be useful. As  mentioned earlier, the Glossary of Terms Used in the Federal Budget  Process defines “program” generally as an organized set of activities  directed toward a common purpose. However, variations in agency  organizational structures, missions, history, and funding authorities—as  well as in the purposes for which agencies create or use program  information, such as budgets or performance reports—can result in  differences in how agencies organize and group activities using different  terms. To illustrate these differences, table 2 provides our observations  on how Education, USAID, and DHS used the concepts and terms of  program area (for collections of related programs), program, and activity  (for more specific activities within a given program) in budget  documentation. Each of these three agencies includes these concepts in  their documents, but how they are organized and what they contain  differs.
The varied uses of these terms within and across agencies—all from  agency budget documentation—illustrate one challenge of consistently  using words such as “program” and the benefits of creating a controlled  vocabulary that could move agencies toward a common understanding  and more consistent application of these terms for an inventory.  Because agencies have flexibility in deciding what activities constitute a  program, an information architecture approach that would focus attention  at the facet level would help make the inventory information more  consistent. If consistent information is collected, then it can be more  easily compared, whether or not the identification of programs is similar  across agencies.
section_title: Identifying, Selecting, and Defining Program Facets Aids Consistency
paragraphs: Facets and the information within them can provide the structure that will  allow the inventory to contain consistent information within and across  agencies, aiding comparability of information. Existing guidance points to  potential facets and definitions for them, including controlled vocabularies.  OMB guidance, as well as requirements for the DATA Act, for example,  identify and define facets related to program budget and performance,  including performance goals. Program operations information was not  included in OMB’s guidance for the initial inventory; however, existing  taxonomies in use by federal agencies provide examples of facets and  controlled vocabularies for program operations, including how programs  operate and whom they serve. For example, the CFDA provides  established lists to define eligible grantees and beneficiaries and includes  questions to guide agency officials in collecting this information.
With the use of controlled terms, comparisons can be made across  programs that serve similar populations or share program eligibility. Table  3 shows potential program operations facets and comparable information  collected in the CFDA. To be included in the federal program inventory,  the controlled vocabulary and corresponding definitions for facets related  to program operations would be assessed against standards, as  described below. With many different possible facets and the associated  costs of collecting specific program information within them, OMB would  need to determine priorities and time frames for required facets in  consultation with agencies and stakeholders, if an information  architecture approach is used.
section_title: Step 3: Determine Relationships
paragraphs: Determining relationships among selected concepts and terms can add to  the usefulness of the information in the inventory. The controlled  vocabulary can help to show relationships, such as if definitions of some  terms refer to other terms or if programs are related. Taxonomies can  bring additional structure by linking program facets with one another,  promoting functionality and usefulness. Taxonomies tend to be  hierarchical, but sometimes are organized in other ways. For example, a  hierarchical structure might apply to an agency’s organizational structure  in which each related facet is a subset (e.g., agency, bureau, office), and  a network structure might be more appropriate for associating categories  of information for which there are not specific subcategories, such as  facets containing budget and performance information. Finally, decision  rules that specify what collections of activities constitute a program for the  purposes of the inventory help ensure consistency and comparability of  information within the inventory. Program information and data could then  be collected for each individual program facet.
subsections: section_title: Using Decision Rules to Identify Programs and Relationships between Programs
paragraphs: How broadly or narrowly agencies identify programs for the inventory will  affect its usefulness. For example, an approach to inventory development  that groups many activities under a relatively small number of program  names could have limited usefulness, if it results in a low level of  transparency over the full range of activities, functions, and costs that  occur within that area. Conversely, an approach to an inventory that  groups activities narrowly and includes a comparatively large number of  programs could result in greater transparency and usefulness, but would  likely create significant costs for agencies to identify, create, and  maintain. Decision rules for determining what should be identified as a  program for purposes of the inventory will need to balance usefulness  and costs, if this approach is implemented.
Further, agencies will need to consider how best to organize their  activities for inclusion as programs in the inventory, which could present a  challenge. The three agencies we reviewed have different organizational  structures, such as strategic, programmatic, or budget structures that  could be used to organize inventory programs. For example, Education  generally has consistent, program-focused alignment across its  organizational structures. DHS has historically not had as consistent a  program focus across its structures—given its origins from many different  agencies—but has recently more closely aligned its budget and program  structures. USAID’s different structures have presented agency efforts in  multiple ways, including at a country level and also at a broader, mission- focused level such as combating malaria or providing basic education.  (See appendix III for more information on how these differences can  affect program identification.) Decision rules will need to be established to  help agencies present programs in the inventory in a way that is as  consistent as possible, given these differences, which could pose  different challenges across agencies.
Because agency activities and structures differ, as do user needs,  agencies implementing an information architecture approach would need  to clearly illustrate the relationships among individual—or groupings of— activities and what is included under a designated program name. This  will provide transparency over how the agency applied decision rules and  what an agency included under that program, though some agencies may  have greater challenges doing so. At the three agencies, we found that  variations in the ease of identifying programs often reflected agency  organizational structures. For example, Education’s internal organization  allows for the relatively easy identification of a consistent list of programs  when using appropriation accounts and program names, in part because  its appropriations are set up similar to its programs, according to agency  officials. By contrast, other agency officials—including at USAID— expressed concern about linking programs to appropriations, because  their programs and appropriations are not similarly structured. Appendix  III provides information on how Education’s, USAID’s, and DHS’s  organizational structures might affect program identification—specifically  in our case study context of identifying programs using budget  documentation—including a recent DHS effort to better align its budget  structure with its discretionary programs.
OMB and agencies could also establish decision rules on how to treat  activities and funding streams that may not be clearly linked to specific  programs or provide overall administrative or mission support, in order to  ensure these items are treated consistently. This can include, for  example, general administration, information technology-related  maintenance, and general construction. Each of the three agencies  selected for our illustrative case studies had these categories of funding  and expenses. For example, Education had a “Program Administration”  program that accounted for over $400 million in fiscal year 2015  obligations and funded close to half of the agency’s almost 4,100 full-time  employees. Education used “Program Administration” to provide  administrative support across most programs and offices in the  department. If an information architecture approach were used, OMB and  agencies would need to determine whether and to what extent these  kinds of expense categories should be identified as distinct programs for  purposes of the inventory or whether they should be allocated across  programs.
section_title: Organizing Needed Information for the Inventory
paragraphs: Taxonomies can bring additional structure to an inventory by linking  program facets with one another, promoting functionality and usefulness.  Figure 2 shows how this can be applied to an individual program. In this  example, program information for Education’s Promise Neighborhoods  program is collected into potential facets related to the program’s  organization, budget, performance, and operations.
Once program information has been collected into facets for multiple  programs, a taxonomy allows for the comparison of information across  programs, as well as the potential to aggregate—or disaggregate— program information at an appropriate level to facilitate a variety of uses.  Table 4 provides an illustration of selected programs in three federal  agencies providing early learning or child care services for different age  groups. In the federal program inventory, comparisons could be made  across or between multiple facets. In this case, the information included  within the activities/services and beneficiary facets are compared to  identify programs with similar characteristics. For example, sorting  programs by information included in the two facets in table 4 would reveal  that the Promise Neighborhoods and the Comprehensive Literacy  Development Grants (formerly Striving Readers) programs both provide  early learning services and have a larger age range of children as  intended beneficiaries.
However, collecting program information for each facet may pose  challenges for agencies. As we developed our hypothetical inventory, we  found that a greater range of program information was readily available  for some of the selected programs than for others—often depending on  the extent to which programs were included by name in the documents  we reviewed (e.g., budget documents, performance and strategic plans,  and agency websites). For those programs that were included in the  CFDA, for example, we were able to collect information for a number of  our facets, such as functional codes that reflect program operations and  coded entries for eligibility. Performance goals, including measures and  targets, however, are not required in the CFDA.
Likewise, for programs that corresponded to individual program activities  in the federal budget, we were able to readily identify budget and  financing information, such as obligations, appropriations accounts, and  related program activities, although identifying this type of budget  information was sometimes more difficult. We have faced similar  obstacles to collecting program information in other work. For example,  we were unable to identify 39 (of 58) efforts or programs in the  President’s budget by name or by funding for a recent report looking at  federal efforts supporting U.S. manufacturing, although we were able to  report program obligations for many but not all of them after we  conducted a survey of agency officials.
section_title: Step 4: Evaluate Structure and Content
paragraphs: After identifying programs and facets and determining relevant  relationships, information architecture principles suggest evaluating the  taxonomy of the program inventory in several ways. For instance, an  evaluation could ask how well the inventory’s structure and controlled  vocabulary organize and present needed information. Further, the  evaluation could involve consulting with experts or comparing with  existing taxonomies and standards to ensure that all the needed terms  and facets are included. Evaluation of the structure and content of the  inventory can involve different methodologies, including reviewing existing  standards and interviewing subject-matter experts. There are a number of  sources available for standards related to the organization of information  using the principles and practices of information architecture, including  NISO standards for controlled vocabularies.
In addition to evaluating the inventory’s taxonomy and facets, it will also  be necessary to evaluate the quality of the specific program information  content. This includes examining the consistency and completeness of  the information that agencies report. Consistently identifying program  information in facets related to outcomes could help agencies identify  where they have programs that have similar purposes or activities, and  therefore opportunities to collaborate. Likewise, a complete inventory— including all federal efforts within each definition—could be a useful tool  for addressing crosscutting issues. Specific aspects could include  examining whether the information is accurate, consistent with the  controlled vocabulary, properly formatted, and current. Including  consistent and complete program information will help the inventory be  more useful and allow users to better compare and contrast programs  across broad areas with federal involvement, as we noted in our 2014  report.
More generally, while the inventory is developed iteratively, continuously  evaluating the extent to which it is delivering the usefulness desired by  stakeholders will enhance its continued usefulness. As part of this  evaluation, OMB and agencies can assess the decision rules for the  identification of individual programs to determine the extent to which the  resulting set of programs are identified at a level that facilitates  comparisons across and within agencies. This type of evaluation could  lead agencies to determine that activities should be grouped together  more broadly (including more activities) or more narrowly (including fewer  activities) to allow for better comparisons and increased usefulness. In  some cases, activities may need to be allocated differently among  programs. Likewise, an evaluation could test program identification by  determining if the inventory includes sufficient breadth (in terms of an  agency’s total funding) and depth (at a level that is useful for decision  makers). We reviewed one narrow way agencies could identify  programs—using budget documentation—and found challenges to the  consistency and completeness of program information (see appendix  III). These examinations can lead to improvements in the inventory over  time.
section_title: Step 5: Design the User Interface
paragraphs: A well-designed inventory interface can include features to enhance the  usefulness of the program information by enabling users to navigate  through the content of the inventory to meet their needs. The taxonomy  structure serves as the backbone and allows for the presentation of  concepts, terms, and relationships dynamically. Specifically, individual  facets can be used to identify potential relationships between programs  and to organize information in new ways within the inventory. For  example, in our hypothetical inventory the facets containing budget  information for the Department of Agriculture’s Child and Adult Care Food  Program can be linked to both the School Lunch and the School  Breakfast programs through a common account. The ability to view these  related programs gives the user more tools and information to understand  how programs fit within the whole of government and relative to one  another. In addition, tagging program information (e.g., attributes or  characteristics) within facets in a taxonomy can help create new  relationships and allow for the grouping and linking of content in new  ways. For example, HHS’s Child Care and Development Fund provides  child care services to children ages 5 and under through grants that also  support low-income families and children with disabilities. This program  could be tagged to highlight these and other attributes of the program  collected in facets related to activities and services and to eligible  beneficiaries. Then, a user interested in similar programs could click on a  tag (e.g., early learning services) that could generate a list of programs  that also have that tag. Figure 3 depicts how such an interactive tool  could allow a user to identify programs with the same tagged activities.
The interface could also include predesigned output formats for program  information. This feature could allow for the creation of program  summaries for individual programs or fact sheets cross-walking certain  predetermined facets, such as budget and performance information, in a  user-friendly format. Figure 4 provides an illustrative example of this  concept.
section_title: Step 6: Validate Usability and Usefulness
paragraphs: The federal program inventory will exhibit its usability—and thus  usefulness—during validation. This step tests the union of the user  interface and the taxonomy to improve the usefulness of the inventory  and test the inventory’s organization, structure, and general functionality.  Using a variety of methods to test the inventory with the intended  audience can validate design and content decisions, including any  assumptions made about how users interact with data. One method is  to conduct usability testing by asking users to complete a series of clearly  defined tasks and monitoring how they navigate the inventory. For  example, users could be asked to find a term that is grouped with other  terms or to find everything they can about a particular topic. Analysis of  resulting data from browsing or searching the user interface—such as the  number of clicks or completion times of tasks—can reveal how the  presentation and grouping of terms affects the completion of user tasks.
A complementary method is to interview or conduct focus groups to  obtain qualitative feedback on the usability and usefulness of the  inventory. To illustrate this validation methodology, we asked  congressional staff to offer their perspectives on how they might use an  interactive website containing a federal program inventory with search,  filtering, and other navigation capabilities. Overall, congressional staff  affirmed that a searchable and sortable design with the ability to provide  different levels of aggregation and disaggregation of program information  would be useful for a number of tasks, including the following:  informing staff quickly about programs as part of background research  for various tasks; developing briefing materials for members using program information,  informing congressional decisions about authorizing or reauthorizing  federal programs and provisions in the tax code; answering constituent questions;  identifying information related to program performance; and drawing attention to information gaps, such as if program goals or  targets were not developed.
The congressional staff we interviewed also stated that having links to  program evaluations, especially GAO, inspector general, and  Congressional Research Service reports, would be helpful for learning  about program performance, as would information tags identifying direct  and indirect program activities and services.
Further, we shared with congressional staff a series of illustrative  examples of summary sheets containing information on a number of  potential program facets, including budget, performance, and operations  information. (Figure 5 presented in the previous section above is one of  these examples.) These staff said the types of information and  organization matched what they would expect in an inventory and would  want to inform their work, although they stated the inventory could be  more useful if historical information were more robust than what we  included. For example, they expressed a preference for at least 5 years of  budget information rather than the 3 years our hypothetical inventory  provided. Some staff also emphasized the importance of having strictly  defined fields, such as the program history field, in order to avoid  confusion and reduce subjectivity in program information. Such feedback  can provide valuable insights into the design and content users find most  important, the limitations they identify, and their satisfaction with the  overall interface and program inventory.
Incorporating the validation results into the design and content of the  inventory would enhance usefulness by ultimately enabling users to better  find the information they need. Validating the inventory and incorporating  prioritized results can mitigate risks related to the opinions and  assumptions that were necessary to create an initial inventory framework.  Thus, the validation results can also serve as a roadmap for subsequent  iterations of the inventory.
section_title: Step 7: Govern Structure and Content
paragraphs: Establishing and implementing a governance structure will help ensure  the program inventory is continually maintained and useful. Governance  specifically involves establishing the policies and procedures—including  roles, accountabilities, standards, process methodologies—for  maintaining and improving the inventory. Governance policies can also  set a schedule for regular assessment of the inventory to monitor how it  will meet user needs over the long term. Finally, governance can ensure  that the inventory continues to meet factors related to usefulness,  including accuracy, completeness, consistency, relatability, and validity.
Good governance requires policies that define the process for managing  inventory content, maintaining and changing the taxonomy, and  establishing roles and responsibilities. Policies for managing content  define the conditions under which programs and program information are  added, updated, and archived or deleted. These policies also define the  conditions under which the information architectures—taxonomy  structure, facets, and controlled vocabulary—are revisited and updated.  As such, the governance policies can establish how prospective changes  are evaluated and prioritized and when to make changes.
Governance of the inventory will also benefit from well-defined roles and  responsibilities. This includes defining the individuals responsible for  proposing and making changes to the inventory and taxonomy—both to  reflect higher level changes to the purpose of the inventory and the day- to-day management of the taxonomy. In addition, implementation  guidelines for each role will further clarify the expected steps by which  changes to the information architecture and inventory are made. Further,  governance can establish when and how processes are reviewed and  updated. Documenting these roles and responsibilities will create  accountability and provide a transparent process that will withstand  changes in staff.
Governance also includes decision rules that determine how programs  are identified and information is included. In our work to develop  illustrative examples of programs an inventory might include, we  encountered programs with changing names and authorizations, which  would require policies to ensure consistent program information be  included in the inventory and kept up to date. For example, one potential  Education program has had three names since it was originally  authorized: (1) Striving Readers, then (2) Striving Readers  Comprehensive Literacy, and most recently (3) Comprehensive Literacy  Development Grants. Each version of the program has been authorized  by different statutory provisions, creating complexities in tracking program  information across time, and raising questions about whether it is the  same program, successor programs, or three individual programs.  Governance policies that include decision rules regarding whether and  how to include the evolution of programs can aid the consistency and  usefulness of program information over time.
Leveraging existing governance policies, roles, and procedures can help  to ensure that the inventory’s usefulness persists. We have previously  reported that establishing a formal framework for providing data  governance throughout the lifecycle of developing and implementing  standards is key for ensuring that the integrity of data standards is  maintained over time. There are a number of governance models, and  many of them promote a set of common principles that includes clear  policies and procedures for broad-based participation from a cross- section of stakeholders for managing the standard-setting process and for  controlling the integrity of established standards. Ideally, a governance  structure could include processes for evaluating, coordinating, approving,  and implementing changes in standards from the initial concept through  design, implementation, testing, and release. It would also address how  established standards are maintained and ensure that a reasonable  degree of agreement from stakeholders is gained.
section_title: Agency Comments
paragraphs: We provided a draft of this report for review and comment to the Director  of the Office of Management and Budget (OMB), the Departments of  Education and Homeland Security, the U.S. Agency for International  Development (USAID), and the General Services Administration. USAID  provided technical corrections, which GAO incorporated as appropriate.  OMB agreed to consider this information architecture approach as it  develops plans for the next iteration of the federal program inventory.
We are sending copies of this report to the Director of OMB, the  Secretaries of the Departments of Education and Homeland Security, and  the Administrators of USAID and the General Services Administration, as  well as interested congressional committees and other interested parties.  This report will also be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7215 or CurdaE@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of our report. Key contributors to this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The GPRA Modernization Act of 2010 (GPRAMA) includes a statutory  provision for us to periodically evaluate and report on (1) how  implementation of the act is affecting performance management at the 24  major departments and agencies subject to the Chief Financial Officers  (CFO) Act of 1990, including whether performance management is being  used to improve efficiency and effectiveness of agency programs; and (2)  crosscutting goal implementation. This report is part of our response to  that mandate. GPRAMA requires the Office of Management and Budget  (OMB) to present a coherent picture of all federal programs by making  information about each program available on a website. For this report,  we examined how the principles and practices of information architecture  can be applied for the development of a useful federal program inventory.
Programs are defined in our September 2005 Glossary of Terms Used in  the Federal Budget Process as “generally, an organized set of activities  directed toward a common purpose or goal that an agency undertakes or  proposes to carry out its responsibilities.” A federal program inventory  consists of the individual programs identified and information collected  about each of them. As we have reported, the usefulness of a federal  program inventory depends on factors such as accuracy, completeness,  consistency, reliability, and validity, among others.
Our methodology involved describing the general steps that could guide  the development of a useful federal program inventory using an  information architecture approach, as well as assessing how the  principles and practices of information architecture could be used both to  identify programs and to identify, compile, and organize information within  an inventory. This report is not meant to suggest requirements or best  practices for developing the federal program inventory, but rather to  illustrate how a particular approach could be applied to develop a useful  federal program inventory. Other approaches might also be used—or  could be incorporated into this framework—to develop an inventory that  best addresses limitations identified in the past.
subsections: section_title: Principles and Practices of Information Architecture
paragraphs: To understand information architecture, we reviewed industry standards,  website standards, conference and training materials, books, and leading  practices. We then examined how information architecture principles can  be used to create a useful federal program inventory that aligns with  GPRAMA’s requirements that a website present a coherent picture of all  federal programs, as well as with federal website guidelines related to  providing usable information (i.e., usability). This analysis included the  following steps: qualitatively analyzing the information architecture literature and  interviewing information architecture practitioners to identify  overarching principles;  reviewing federal requirements for the usability of websites and digital  services as summarized at digitalgov.gov to identify those guidelines  that relate directly to one or more characteristics of a useful federal  program inventory (i.e., information that is accurate, complete,  consistent, reliable, and valid for its intended use); and  comparing the federal policy topics we identified to the overarching  principles in information architecture, and aligning information  architecture principles with digitalgov.gov guidelines.
section_title: Identifying Programs and Identifying, Compiling, and Organizing Information
paragraphs: To gain an understanding of the intended purpose of and potential uses  for a federal program inventory, we reviewed requirements in GPRAMA,  as well as OMB’s guidance for the executive branch’s initial program  inventory effort and our assessment of that effort. We also interviewed  current and former federal officials who were knowledgeable about prior  efforts to inventory or otherwise consolidate and make publicly available  information about federal programs. We reviewed state websites  describing state experiences in developing program inventories to  understand practices for inventorying program information at the state  level, and we interviewed budget and performance officials in three states  that have or are developing program inventories to understand the  information contained in these inventories and its potential uses.
To understand how programs could be identified and how information  within the inventory could be identified, compiled, and organized, we  selected individual agencies and programs to examine as case studies.
subsections: section_title: Program Identification
paragraphs: As part of our effort to apply relevant principles and practices of  information architecture to program identification, we developed a set of  observations on using budget-related resources to identify programs that  could be included in a federal program inventory. See appendix II for a  summary of these observations. Specifically, we examined budget-related  information resources, including agency budget justification documents  and program activity data. We used budget information because most  agencies followed a similar approach for their initial inventories and  because GPRAMA requires inventories to include budget information for  programs included in the inventory.
We selected three agencies to develop these observations: the  Departments of Education (Education) and Homeland Security (DHS) and  the U.S. Agency for International Development (USAID). These agencies  were drawn from the 24 agencies included in the initial executive branch  effort to develop a federal program inventory and were selected based on  a number of factors, including differences in their overall organizational  structure, approach to the prior effort to develop an inventory, and the  extent of the connection between their programs and the Catalog of  Federal Domestic Assistance (CFDA). This allowed us to compare and  contrast the agencies and the usefulness of budget information in those  agencies to identify programs. Our analysis included the following:  the extent to which consistent and complete lists of agency programs  could be identified using budget-related information and the impact of  the agency’s organizational structures on these lists; a review of the types of activities that could be characterized as  programs within each agency and how activities are grouped into  programs or overarching program areas with underlying programs;  and a general review of the alignment of possible programs identified by  budget documents with other ways agencies organize their efforts,  such as performance reports and CFDA programs.
We also reviewed the relationship between the budget’s program activity  data for the 24 agencies included in the initial executive branch effort and  programs listed in the CFDA to obtain more insights into the different  contexts in which agencies identify agency programs and present  program information. We attempted to determine the extent to which  CFDA programs were aligned with budget program activities by  identifying as a possible match any specific CFDA program that was  similar in title or funding amount with specific program activities that  shared an appropriations account number.
The CFDA is a key resource to identify domestic assistance programs.  While not all agency programs would be included in the CFDA, agencies  submit programs for inclusion, so they have in essence identified those as  programs. For this analysis, we identified the total number of CFDA  programs for selected agencies and the number matched to a program  activity listed in the federal budget. In a more in-depth analysis of  Education’s program activities, we also identified scenarios where (1)  program activities were a one-to-one match in name and dollars with  information in the CFDA, (2) one program activity matched with a number  of CFDA programs, (3) multiple program activities funded what the  agency called a single program in the CFDA, and (4) there was no match.
section_title: Identifying, Compiling, and Organizing Program Information
paragraphs: As part of our effort to apply relevant principles and practices from  information architecture to identify, compile, and organize information  about federal programs, we identified information that can be included in  a useful federal program inventory, tested the collection and organization  of that information by developing a hypothetical inventory with selected  programs, and used the hypothetical inventory to illustrate aspects of how  a federal program inventory could be validated using an information  architecture approach. Finally, we looked to our prior work to identify  relevant practices in information and data governance. More specifically,  our analysis included the following:  Identifying needed information: We identified the types of  information about programs that could be included in an inventory to  make it useful (e.g., budget, performance, and operations information)  by examining OMB and GAO guidance for developing program lists,  including OMB’s guidance for the first inventory effort; examining state  efforts to develop and use inventories; interviewing potential users;  and summarizing examples of the types of program information that  have been identified in our past work as being useful.
Developing definitions and a controlled vocabulary: In order to  develop our hypothetical inventory, we identified needed information  (i.e., facets) to include, as well as definitions for these facets (the  “controlled vocabulary”). To identify and define terms that were not  included in the initial executive branch’s inventory, we looked at other  taxonomies or guidance, including controlled vocabularies used by the  Congressional Research Service and by the Education Resources  Information Center (ERIC), as well as CFDA guidance for agency  officials, OMB guidance on the reporting of performance goals, our  Glossary of Terms Used in the Federal Budget Process, and the  Digital Accountability and Transparency Act and related OMB  guidance.
Selecting individual programs for our hypothetical inventory: We  collected program information for a number of individual programs  from a variety of sources, including: (1) lists of budget program  activities for Education, DHS, and USAID; (2) programs or efforts  identified as part of our recent efforts to look at programs with  common activities, services, beneficiaries, or outcomes; and (3) our  efforts to examine tax expenditure programs. To ensure our set of  illustrative programs included a range of programs, we selected  programs with certain characteristics (e.g., program size in terms of  budget, from different agencies, based on availability of information,  etc.). When multiple programs were available based on certain  attributes, we used a simple random selection to choose specific  programs to include in our set of illustrative programs.
Collecting program information: For selected individual programs,  we collected program information from budget and performance  documents, as well as from agency websites and the CFDA. For  programs selected from our existing work, we also leveraged reported  information, and we collected information about any challenges  related to identifying programs or collecting program information.
Developing a hypothetical inventory: We tested the development of  a hypothetical inventory for six programs drawn from a recent report  on early education and child care programs by including individual  facets in an online taxonomy to demonstrate how information could be  sorted by facets and presented in different ways.
Validating the form and content of the hypothetical inventory: To  illustrate aspects of the validation step in the information architecture  approach to developing an inventory, we developed sample materials  to illustrate what the content and structure of an inventory might  include, and we presented these materials to congressional staffers  with committees overseeing programs providing early education and  child care services (i.e., the Senate Committee on Health, Education,  Labor, and Pensions and the House Committee on Education and  Workforce). We solicited feedback on the form and content of our  hypothetical inventory and collected information regarding potential  uses and related needed information.
We conducted this performance audit from July 2016 to September 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Recommendations for the Office of Management and Budget from GAO-15-83: Government Efficiency and Effectiveness: Inconsistent Definitions and Information Limit the Usefulness of Federal Program Inventories (Oct. 31, 2014)
paragraphs: Appendix II: Recommendations for the Office  of Management and Budget from GAO-15-83:  Government Efficiency and Effectiveness:  Inconsistent Definitions and Information Limit  the Usefulness of Federal Program Inventories  (Oct. 31, 2014)
section_title: Appendix III: Observations on Using Budget Information to Identify Federal Programs for an Inventory
paragraphs: We examined budget documentation to identify possible programs in  three agencies: the Departments of Education (Education) and Homeland  Security (DHS) and the U.S. Agency for International Development  (USAID). To identify programs, we reviewed budget documentation as an  illustrative starting point for several reasons. First, most agencies used  budget information to help structure their initial inventories. Second, the  GPRA Modernization Act of 2010 requires specific budget information to  be included in an inventory. Third, agencies have significant budget  related information, such as congressional budget justifications, as well  as the federal budget’s appropriations and program activity accounts. We  did not conduct a full evaluation of all of the ways agencies could identify  programs—or all of the characteristics needed for program identification  to be useful—as we did not build a full inventory and such an evaluation  was outside of our scope.
subsections: section_title: Budget Documents Can Identify Possible Programs, but Variations across Agencies Illustrate the Challenge of Identifying Programs Consistently and Completely
paragraphs: Education, DHS, and USAID budget documents all provided information  to identify possible programs. Education’s budget documentation had a  specific list of programs, organized by program area, with a separate  cross-walk between office, program, and statutory authority, allowing for  straightforward identification. Both DHS and USAID documentation  identified broader areas of effort and named dozens of programs.  However, the set of programs we generated from budget documentation  illustrated the challenge of identifying programs for an inventory. We  evaluated the set of possible programs for consistency and completeness  because we have previously reported the importance of those  characteristics, among others, to make an inventory useful. We found  differences across agencies in the extent to which their budget  documents could generate consistent and complete lists of programs,  which could result from different organizational structures. Specifically,  the programs listed in budget documents did not always present agency  programs consistently across other agency resources, such as  performance documents or agencies’ congressional appropriations.  Further, budget documentation did not always allow for the complete  identification of programs for the inventory in terms of depth (at a level  that would be sufficiently useful for decision making), although it did have  complete breadth (as it contained each agency’s total funding).
Overall, Education’s closer alignment between budget, program, and  other organizational structures generally made the identification of  programs with budget documents more consistent, limiting challenges.  Education’s budget, appropriation, program, and performance structures  were all similar, and information about programs was presented  consistently across information resources such as appropriation,  performance, and Catalog of Federal Domestic Assistance (CDFA)  information. Further, Education presented programs with a set of activities  at a level that could be useful for decision making in an inventory. For  example, Education’s Promise Neighborhoods program supported  awards to local partnerships to develop and implement comprehensive,  neighborhood-based plans for meeting the cradle-to-career educational,  health, and social service needs of children in high-poverty communities.  Although there were a number of separate activities within the program,  its neighborhood focus presented information that could help decision  makers evaluate the activities as a group in light of that focus.  Education’s hierarchical structure generally allowed for a clear  identification of relationships between agency offices, program areas, and  individual programs. Also, other Education documentation presented a  specific program list by administrative office and provided a cross-walk  between those programs and agency goals, which made it easier to  understand how specific programs contribute to the achievement of those  goals.
By contrast, USAID budget documents presented greater challenges in  identifying possible programs, as its documents were less aligned with a  specific program structure and offered less consistent and precise  identification. USAID did not have a specific, complete set of programs in  budget documents. Rather, USAID identified specific funding accounts,  and included some highlights—but not systematic information—about  more specific efforts. For example, USAID’s budget documentation  included the broad objective Peacekeeping Operations, which had a  number of highlighted efforts, including the following:
Global Peace Operations Initiative ($71 million): supports U.S.  contributions to international peacekeeping capacity building by  providing training and equipment, as well as supporting deployment of  troops and evaluations of effectiveness.
South Sudan ($36 million): supports rebuilding the military and  support for the Sudan People’s Liberation Army, including training and  non-lethal equipment.
Multinational Force and Observers ($28 million): supports efforts to  supervise the implementation of security provisions of the Egyptian– Israeli Peace Treaty.
As part of USAID’s Performance Report within its 2015 budget  justification, the agency also presented information by program area. For  example, its Peace and Security objective included six program areas: (1)  Counter-Terrorism; (2) Combating Weapons of Mass Destruction; (3)  Stabilization Operations and Security Sector Reform; (4) Counter  Narcotics; (5) Transnational Crime; and (6) Conflict Mitigation and  Reconciliation. USAID provided information about its work in these areas,  but had no specific list of its programs. USAID officials noted that the  program areas listed in the justification broadly relate with the program  areas set forth in the Department of State’s Standardized Program  Structure and Definitions. USAID creates its budget justification jointly  with the Department of State.
USAID had different structures across its congressional budget  justification, performance structures, and program activities, though its  budget justification presented a high-level funding crosswalk between its  budget and performance structures. Further, USAID presented its efforts  at an individual award level online, including at foreignassistance.gov, but  that may be at too narrow a level to be useful for decision making when  included in an inventory. The range of structures and ways to present  information on the activities of USAID provides transparency and  accountability on how agency funds are being used. However, it is our  observation that this flexibility and range of methods for presenting  information made it challenging to identify through budget documents  specific programs that could be included in an inventory, if that inventory  were intended to link specific programs and appropriation amounts.  USAID officials noted that the agency had specific definitions of program  area, program, and activity in documents other than the agency’s  congressional budget justification.
Like USAID, DHS did not present a comprehensive program list in its  budget documents: DHS had disparate budget, program, and other  agency structures, in its case borne out of the parts of different agencies  that combined to create DHS, according to agency officials. However,  DHS recently aligned its discretionary programs with a standardized  budget structure, and now has greater similarities across its budget justifications and appropriations structure. Specifically, DHS established  four standard budget categories to be used by all of its mission  components: (1) Research and Development; (2) Procurement,  Construction, and Improvements, (3) Operations and Support, and (4)  Federal Assistance. DHS defined each budget category and created  guidance on what activities would typically be included. DHS then created  six sub-categories—at the level of the budget’s program, project, and  activity account—for more specific funding areas where applicable. DHS  also approved other individual categories that better reflect the  components’ distinct missions. For instance, in the Federal Emergency  Management Agency, DHS used the standard sub-category of Mission  Support along with the individual subcategory of Preparedness and  Protection. Moreover, DHS added more specific information to the budget  program activity for its 2018 budget justification below the program,  project, and activity account level. These additions could provide more  insight to what DHS considered programs and more consistently link the  budget with program information, which could help provide better  information at a level useful for decision making in an inventory.
section_title: Appendix IV: Observations on the Alignment between Federal Budget Program Activities and Agency Programs in the Catalog of Federal Domestic Assistance
paragraphs: To obtain insights into the different contexts in which agencies identify  programs and present program information, we compared 24 agencies’  programs listed in the Catalog of Federal Domestic Assistance (CFDA)— a key resource to identify domestic assistance programs—and their  program activity information from the agencies’ budgets. Overall, we  observed that the CFDA and budget program activities listings could be  helpful in supporting the development of a federal program inventory. The  extent of their usefulness will vary by agency, in part, because agencies  we spoke to did not view the CFDA as fully consistent with their  programs. Based on our analysis, neither resource would be satisfactory  for creating a definitive list of programs for any agency for purposes of an  inventory. Not all agency programs would be included in the CFDA since  the purpose of the catalog is to assist potential applicants to identify and  obtain general information about domestic assistance programs.  However, with over 2,000 programs included and information about each  of those programs, the CFDA could serve as a valuable resource in  efforts to develop a federal program inventory and collect program  information.
Using a text analytics methodology that compared the names and funding  amounts between CFDA programs and the budget’s program activities,  we attempted to determine the extent to which CFDA programs were  clearly aligned with budget program activities. We observed that the  relationships between CFDA programs and program activities within the  same appropriation account varied significantly by agency but overall was  unclear for all agencies (see table 6).
We also observed that there are multiple different relationships between a  CFDA program and agency program activities. Figure 6 presents  illustrative examples of the different relationships that CFDA programs  might have with specific program activities. These complex and uncertain  relationships could affect some of the matched program numbers in table  5 (above) because multiple budget program activities or multiple CFDA  programs could be matched.
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the above contact, Brian James (Assistant Director), Molly  Laster (Analyst in Charge), Andrew Nelson, and Michelle Serfass made  key contributions to this report. Leia Dickerson, Steven Flint, Hedieh  Fusfield, Ellen T. Grady, Benjamin T. Licht, Drew Long, Steven Putansu,  Robert Robinson, A.J. Stephens, James Sweetman, and John Yee also  contributed.