Performance Budgeting: Initial Experiences Under the Results Act in Linking Plans With Budgets
Published: 1999-04-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed selected fiscal year (FY) 1999 federal agency performance plans, focusing on: (1) agencies' approaches to linking performance goals and budgetary resources; (2) characteristics that might be associated with different approaches to linking performance goals and budgetary resources; and (3) implications for future efforts to clarify the relationship between budgetary resources and results.
section_title: What GAO Found
paragraphs: GAO noted that: (1) in their first Government Performance and Results Act of 1993 performance plans, agencies experimented with a variety of approaches to connect budget requests with anticipated results; (2) although most agencies reviewed (30 of 35) defined some type of relationship between the program activities of their proposed budgets and the performance goals of their plans, far fewer (14 or 40 percent of the plans reviewed) translated these relationships into budgetary terms--that is, most plans did not explain how funding would be allocated to achieve performance goals; (3) such allocations are a critical first step in defining the performance consequences of budgetary decisions; (4) GAO found that agencies with budget and planning structures of widely varying complexity made these allocations, but some common approaches were used; (5) agencies were significantly more likely to have allocated funding to program activities if they: (a) showed simple, clear relationships between program activities and performance goals (as illustrated by eight agencies in GAO's review); (b) fully integrated performance plans into congressional budget justifications (as illustrated by five agencies); or (c) had changed their program activity structures to reflect their goal structures (as illustrated by three agencies); (6) agencies' first-year experiences show progress in bringing planning and budgeting structures and presentations closer together, but much remains to be done if performance information is to be more useful for budget decisionmaking; (7) continued efforts are needed to: (a) clarify and strengthen links between planning and budgeting structures and presentations; and (b) address persistent challenges in performance planning and measurement and cost accounting; and (8) the progress that has been made, the challenges that persist--including the indefinite delay in the performance budgeting pilots called for by the act--and Congress' interest in having credible, results-oriented information underscore the importance of developing an agenda to ensure continued improvement in showing the performance consequences of budgetary decisions.

section_title: Letter
paragraphs: The Government Performance and Results Act of 1993 (the Results Act)  seeks to strengthen federal decision-making and accountability by focusing  on the results of federal activities and spending.  A key expectation is that  the Congress will gain a clearer understanding of what is being achieved in  relation to what is being spent.  To accomplish this, the act requires that,  beginning for fiscal year 1999, agencies prepare annual performance plans  containing annual performance goals covering the program activities in  agencies’ budget requests.  In addition, Office of Management and Budget  (OMB) guidance states that agency performance plans should display, by  program activity, the funding level being applied to achieve performance  goals.  Plans that meet these expectations can provide the Congress with  useful information on the performance consequences of budget decisions.
Our assessment of fiscal year 1999 performance plans found that agencies  generally covered the program activities in their budgets, but most plans  did not identify how the funding for those program activities would be  allocated to performance goals.  To enhance the Congress’ understanding  of issues that may affect agencies’ abilities to relate budgetary resources  and results, you requested that we review selected fiscal year 1999  performance plans to (1) describe agencies’ approaches to linking  performance goals and budgetary resources, (2) examine characteristics  that might be associated with different approaches to linking performance  goals and budgetary resources, and (3) identify implications for future  efforts to clarify the relationship between budgetary resources and results.   More clearly describing and analyzing the approaches agencies developed  to link funding requests with performance expectations can be an  important foundation as the Congress reviews agencies’ fiscal year 2000  performance plans.
To respond to your request, we selected 35 fiscal year 1999 performance  plans for review that covered entire agencies or large subordinate bureaus,  services, or administrations.  We developed a methodology to characterize  each plan’s structure and method of linking program activities and  performance goals, and we gathered information on each agency’s budget  structure.  We also identified agency plans in which program activity  funding was allocated to performance goals—a key criterion for useful  plans.   We analyzed how characteristics differed among agencies that  made such allocations and those that did not to determine which, if any, of  the characteristics were statistically significant in making these  allocations.  We provided a draft of this report to OMB for comment and  also discussed selected aspects of this report with staff from the Senate  and House appropriations committees.  See appendix I for a detailed  discussion of our objectives, scope, and methodology.
subsections: section_title: The Results Act and Performance Budgeting
paragraphs: The Government Performance and Results Act of 1993 is a key component  of a statutory framework that the Congress put in place during the 1990s to  promote a new focus on results.  Finding that waste and inefficiency in  federal programs were undermining confidence in government, the  Congress sought to hold federal agencies accountable for the results of  federal spending through regular and systematic performance planning,  measurement, and reporting.  Among its several purposes, the act is  designed to improve congressional decision-making by providing more  objective information on the relative effectiveness and efficiency of federal  programs and spending.  That is, with regard to spending decisions, the act  aims for a closer and clearer link between the process of allocating  resources and the expected results to be achieved with those resources.
subsections: section_title: The Concept of Performance Budgeting
paragraphs: The concept of linking performance information with the budget is  commonly known as performance budgeting.  Within the past 50 years,  initiatives at the local, state, and federal levels of government as well as in  other nations have sought to link performance expectations with specific  budget amounts.  In essence, the concept of performance budgeting  assumes that a systematic presentation of performance information  alongside budget amounts will improve budget decision-making by  focusing funding choices on program results.  Specifically, performance  budgeting seeks to shift the focus of attention from detailed items of  expense—such as salaries and travel—to the allocation of resources based  on program goals and measured results.  In this sense, the Results Act is the  most recent of a series of federal initiatives embodying concepts of  performance budgeting.
At the federal level and elsewhere, performance budgeting initiatives have  encountered many challenges.  Key challenges include a lack of credible  and useful performance information, difficulties in achieving consensus on  goals and measures, dissimilarities in program and fund reporting  structures, and limitations of information and accounting systems.  For  example, some prior initiatives used new and unfamiliar formats that were  layered onto existing budget and appropriations processes, compromising  the goal of integrating performance information into the budget process.   Specifically in the federal government, past performance budgeting  initiatives resulted in unique and often voluminous presentations  unconnected to the structures and processes used in congressional budget  decision-making.
When viewed collectively, these past initiatives suggest three common  themes.  First, any effort to link plans and budgets must explicitly involve  the executive and legislative branches of our government.  Past initiatives  often faltered because the executive branch developed plans and  performance measures in isolation from congressional oversight and  resource allocation processes.  Second, the concept of performance  budgeting will likely continue to evolve.  Past initiatives demonstrated that  there is no single definition of performance budgeting that encompasses  the range of needs and interests of federal decisionmakers.  Third and  perhaps most importantly, past initiatives showed that performance  budgeting cannot be viewed in simplistic terms—that is, resource  allocation cannot be mechanically linked to results.  The process of  budgeting is inherently an exercise in political choice—allocating scarce  resources among competing needs—in which performance information  can be one, but not the only, factor underlying decisions.  Ultimately, the  promise of any performance budgeting initiative, including the Results Act,  lies in its potential to more explicitly infuse performance information into  budgetary deliberations, thereby changing the terms of debate from simple  inputs to expected and actual results.
section_title: Performance Budgeting Under the Results Act
paragraphs: The Results Act differs from earlier performance budgeting initiatives in  several key respects but can be viewed as a continued evolution of the  concept.  At its most basic level, the act requires agencies’ annual  performance plans to directly link performance goals and the program  activities of their budget requests.  Testifying on the Results Act before its  passage, the Director of OMB characterized this linkage as a “limited—but  very useful—form of performance budgeting . . . .”  The act also requires  that another form of performance budgeting be tested during performance  budgeting pilots.
The Results Act requires an agency’s annual performance plan to cover  each program activity in the President’s budget request for that agency.   Subject to clearance by OMB and generally resulting from negotiations  between agencies and appropriations subcommittees, program activity  structures are intended to provide a meaningful representation of the  operations financed by a specific budget account.  Typically, the President’s  annual budget submission encompasses over 1,000 accounts and over 3,000  program activities.  As the committee report accompanying the act noted,  however, the program activity structure is not consistent across the federal  government but rather is tailored to individual accounts. The committee  report further cautioned that agencies’ annual plans should not be  voluminous presentations that overwhelm rather than inform the reader.  Accordingly, the act gives agencies the flexibility to consolidate, aggregate,  or disaggregate program activities, as illustrated in figure 1, so long as no  major function or operation of the agency is omitted or minimized.  In  addition to this flexibility, agencies also have the option to propose  changing their budget structures, subject to concurrence from OMB and  the Congress.
OMB’s guidance regarding this provision of the act set forth an additional  criterion: plans should display, generally by program activity, the funding  level being applied to achieve performance goals.  That is, OMB expected  performance plans to show how amounts from the agency’s budget request  would be allocated to the performance goals displayed in the plan.
In addition to mandating a linkage between budget requests and  performance plans, the act required that pilot projects be used to test  another approach to performance budgeting.  OMB, in consultation with  the head of each agency, was required to designate for fiscal years 1998 and  1999 at least five agencies to prepare budgets that “present, for one or more  of the major functions and operations of the agency, the varying levels of  performance, including outcome-related performance, that would result  from different budgeted amounts.”  While the act required agencies to  define goals consistent with the level of funding requested in the  President’s budget, the act’s pilots would also show how performance  would change if the agency received more or less than requested.  OMB  was to include these pilot performance budgets as an alternative  presentation in the President’s budget for fiscal year 1999 and to transmit a  report to the President and to the Congress no later than March 31, 2001, on  the feasibility and advisability of including a performance budget as part of  the President’s budget.  This report is also to recommend whether  legislation requiring performance budgets should be proposed.
The performance budgeting pilots were scheduled to start in fiscal year  1998 “so that they would begin only after agencies had sufficient  experience in preparing strategic and performance plans, and several years  of collecting performance data.”  In this context, and recognizing the  importance of concentrating on governmentwide implementation in 1998,  OMB announced on May 20, 1997, that the pilots would be delayed for at  least a year.  OMB stated that the performance budgeting pilots would  require the ability to calculate the effects on performance of marginal  changes in cost and funding.  According to OMB, very few agencies had this  capability, and the delay would give time for its development.  In  September 1998, OMB solicited agencies’ comments on these pilots, but no  agencies have been designated as pilots.  At present, OMB has no definite  plans for proceeding with the performance budgeting pilots.
section_title: Agencies Linked Complex Planning and Budgeting Structures Using Many Approaches
paragraphs: The complexities that are the hallmark of today’s federal budget account  structure and the diverse planning structures associated with broad federal  missions were reflected in the approaches used to link budget and planning  structures.  The fiscal year 1999 performance plans we reviewed frequently  depicted complex and imprecise relationships between these structures.   Using the Results Act’s flexibility to aggregate, consolidate, and  disaggregate existing program activities, most agencies generally linked  their program activities to some level of a frequently complex planning  hierarchy.  These linkages were often presented in a performance plan that  was separable from the agency’s budget justification.  As a result, plans  frequently depicted a relationship between program activities and  performance goals that, while consistent with the act’s charge to cover  program activities, proved difficult to translate into budgetary terms.   However, 14, or 40 percent of the agencies we reviewed, built on these  relationships to show how the funding from program activities would be  allocated to achieve discrete sets of performance goals.  These agencies, in  effect, took the first step toward defining the performance consequences of  budget decisions.
subsections: section_title: The Complexity of Budget and Planning Structures
paragraphs: As we have previously reported, the federal government’s budget account  structure was not created as a single, integrated framework but rather  generally developed over time to respond to specific needs.  As a result,  budget accounts and program activities within the accounts vary from  agency to agency.  This complexity was evident for agencies included in  this review.  For example, the number of budget accounts associated with a  given agency’s performance plan ranged from a low of 1 to a high of 118 for  the agencies we reviewed, while the number of program activities to be  covered by the plan ranged from 6 to 465.  The median number of accounts  for agencies that we reviewed was 9, and the median number of program  activities was 32.  Typically, program activity structures were unique not  only to each of the agencies in our review, but also to each of the budget  accounts within an agency.  In only 2 agencies—EPA and the Department  of Defense—were the same program activity titles repeated across all or  groups of an agency’s budget accounts.
Agencies’ planning structures were similarly complex, comprised of widely  varying numbers and layers of goals.  The Results Act and OMB guidance  give agencies flexibility in structuring strategic and annual performance  goals. Annual performance goals are expected to measure progress in  achieving longer term strategic goals. There is no required format or  structure and no limitation on the number of goals that can be included in a  performance plan.  Our sample of performance plans presented a variety of  planning structures and many different terms to describe those structures.   Twenty-one of the 35 agencies we reviewed used a cascading hierarchy of  goals to put their fiscal year 1999 performance in context—that is, these  plans placed one or more layers of goals between their strategic and annual  performance goals.  For example, the Bureau of Land Management (BLM)  plan contained a complex hierarchy of goals: 5 “General Goals,” supported  by 17 “Strategic Goals,” that were supported by 43 “Performance Goals,”  47 “Long-Term Goals,” and, finally, 64 fiscal year 1999 “Annual Goals.”   Figure 2 shows the numbers and layers of goals associated with one of  BLM’s five general goals.  The remaining 14 agencies, however, had fewer  layers of goals.  For example, the U.S. Agency for International  Development’s (USAID) six strategic goals were supported directly by  performance goals with fiscal year 1999 targets.  Figure 3 presents the  performance goals associated with one of USAID’s strategic goals.
To analyze these disparate planning structures, we developed and used a  common framework that defined two layers of goals above the annual  performance goals stated in the performance plans: (1) strategic goals, to  reflect the first goal layer under an agency’s mission statement and (2)  strategic objectives, to reflect the next subordinate level of goals under  strategic goals. We found that, quantifying just these two layers, most  agency plans involved relatively complex presentations.  The number of  strategic goals in plans we reviewed ranged from 1 to 47, with a median of  5.  Just over half of the agencies in our review placed an intervening layer  of strategic objectives between strategic and performance goals.  In plans  where strategic objectives were used, the number of strategic objectives  ranged from 5 to 122.  Across our sample of agency plans, the median  number of strategic objectives used was nine.
section_title: Variety of Approaches to Linking Budget and Planning Structures
paragraphs: The variety and complexity of agencies’ planning and budget structures  necessarily resulted in a range of approaches to present and link this  information.  However, across this range, two approaches predominated:  29 performance plans were presented separately from congressional  budget requests, and 22 plans established complex linkages of multiple  program activities related to multiple performance goals.
Although performance plans and budget requests—commonly referred to  as justifications of estimates—are both transmitted to the Congress  following the President’s budget submission in February, most agencies  kept their plans physically separated from their budget submissions, either  as entirely separate documents or as separate components appended to the  justifications.  Only 6 of the 35 plans we reviewed had been fully  integrated into the agency’s budget justification (see appendix II).
In addition to separating plans and budgets, almost all agencies retained  the budget structure they had used in previous budget submissions.  Only  three agencies in our review—EPA, the Customs Service, and the Nuclear  Regulatory Commission (NRC)—substantially changed their program  activity structures, and all cited the Results Act as a factor in this change.   Some others—such as the Bureau of Indian Affairs (BIA) and the  Department of Veterans Affairs (VA)—noted that such changes were under  consideration but proposed no changes in fiscal year 1999.  For example,  BIA stated that it would work with OMB and congressional committees  to “simplify the budget format to mirror the strategic plan.”  However,  agencies very frequently took advantage of the act’s flexibility to  modify—that is, to aggregate, consolidate, or disaggregate—program  activities in order to show linkages with performance goals.  The extent to  which this flexibility was used varied greatly.  For example, figure 4  illustrates how the Administration for Children and Families (ACF)  consolidated program activities before relating them to performance goals.
Figure 5 illustrates how the Health Resources and Services Administration  (HRSA) linked some performance goals to disaggregated program  activities.
In addition to modifying or proposing changes to program activities,  agencies in our review used three principal strategies to meet the Results  Act’s expectation that annual plans would “establish performance goals to  define the level of performance to be achieved by a program activity.”
About half (13) of the agencies that made linkages between  performance goals and program activities in their performance plans  established this connection at the relatively high level of strategic goals  or objectives.  For example, as shown in figure 4, ACF linked  consolidated program activities to strategic objectives, which, in turn,  were subsequently associated with performance goals.
About the same number of agencies (14) defined direct linkages to  performance goals.  Figure 5 illustrates how HRSA linked disaggregated  program activities directly to its performance goals.
Three agencies—the National Aeronautics and Space Administration,  the National Science Foundation, and VA—linked program activities to  something other than a statement of strategic goals, strategic objectives,  or performance goals.  For example, the VA plan contained 17 strategic  goals and 25 strategic objectives.  However, instead of linking program  activities directly to these goals and objectives, VA linked both its  program activities and performance goals to its 10 business lines.  The  business lines generally represented different agency functions such as  medical care and education.
As the above discussion indicates, regardless of the strategy used, most  agencies ended with the same basic relationship—many program activities  were related to many performance goals (see appendix II).  These  imprecise, “many-to-many” relationships frequently resulted from agencies’  linking aggregated or consolidated program activities with strategic goals  or other groupings of performance goals.  For example, in figure 4, ACF  consolidated program activities and linked them to the group of  performance goals associated with a strategic objective, creating a many- to-many relationship between program activities and performance goals.   In contrast, eight plans used a more direct and simple approach—typically  linking a single program activity with multiple performance goals.  Figure 5  shows that HRSA linked a single disaggregated program activity to multiple  performance goals.
While 30 of 35 plans defined some relationship between program activities  and performance goals, 16 of these did not build on that relationship by  showing how the funding from program activities would be allocated to  discrete sets of performance goals.
Ten of these 16 plans did not present funding levels for any set of  performance goals in their performance plans.
Six of these 16 plans presented funding levels for some set of  performance goals without explaining how those funding levels had  been derived from program activities in their budget requests.
However, 14 agency plans, or 40 percent of those included in our review,  both identified the funding for program activities and explained how that  funding would be allocated to a discrete set of performance goals.  In  effect, these 14 plans, which will be discussed more fully in the next  section, took the first step in defining the performance consequences of  budget decisions.
section_title: Funding Was Allocated to Performance Under a Variety of Conditions But Using Some Common Approaches
paragraphs: Our review of selected fiscal year 1999 performance plans indicated that  agencies with budget and planning structures of widely varying complexity  were able to develop approaches toward achieving a fundamental purpose  of the Results Act—clarifying the relationship between resources and  results.  Figure 6 lists the 14 agencies in our review that allocated program  activity funding to performance goals in their performance plans.  We  found that some of the approaches that these agencies used, alone or in  combination, were more frequently associated with plans that linked  program activity funding to performance goals.  For example, agencies that  (1) established simple relationships between program activities and  performance goals or (2) fully integrated budget justifications and  performance plans were significantly more likely to allocate program  activity funding to performance goals.  In addition, each of the three  agencies that changed their budget structures to align them with their  planning structures made these allocations.  Conversely, plans  characterized by imprecise, many-to-many relationships between  performance goals and program activities and plans presented separately  from budget justifications generally did not present such allocations.
subsections: section_title: Allocations Were Made Within a Variety of Planning and Budgeting Structures But Alignment of These Structures Was Significant
paragraphs: Our review generally found few differences in the budget structures of  agencies that allocated program activity funding to performance goals and  those that did not.  For example, the number of an agency’s accounts or  program activities was not significantly related to whether a plan presented  funding allocations for performance goals.  As table 1 shows, for example,  the median number of accounts and program activities was nearly the same  for agencies that did allocate program activity funding to performance  goals and for those that did not.
We also considered whether plans that allocated program activity funding  to performance goals were more frequently associated with agencies  having spending concentrated in one account.  To determine if  concentration of spending was a significant factor, we calculated the  number of agencies for which 75 percent or more of requested spending  was associated with a single account.  Again, no significant relationship  was observed (see figure 7).  Four, or 36 percent, of the agencies with  spending concentrated in one account allocated program activity funding  to performance goals.  Ten, or 42 percent, of the agencies with spending  concentrated in multiple accounts allocated program activity funding to  performance goals.
Similarly, the complexity of an agency’s program activity structure was not  a significant factor in allocating funding to performance goals.  We  considered whether a simpler program activity structure—that is, one in  which program activity titles were repeated across budget accounts—was  more frequently associated with allocating funding to performance goals.   We found no significant difference in the number of agencies allocating  program activity funding to performance goals.  Thirteen, or 39 percent of  the 33 agencies without common program activity structures, allocated  program activity funding to performance goals (see figure 8).  Although  only two agencies—EPA and the Department of Defense—exhibited  common program activity structures, EPA presented such an allocation  and DOD did not.
Although a particular account and program activity structure was generally  not associated with allocating funding to performance goals, there was one  significant but perhaps unsurprising exception.  All three of the agencies  that proposed changing their program activities to be consistent with their  planning structures—EPA, NRC, and the Customs Service—allocated  funding to performance goals (see figure 9).  For example, EPA proposed a  uniform program activity structure across all of its accounts in which each  program activity represented one of its strategic goals.  Figure 10 illustrates  how EPA used consolidation to allocate program activity funding to  strategic objectives and their supporting performance goals.
Fourteen agencies in our review that showed how program activities were  allocated to performance goals did not appear to have any common  structural elements in their performance plans.  Differences in plan  complexity—defined in terms of number and layers of goals—were not  significant between agencies that allocated program activity funding to  performance goals and those that did not.  As indicated in table 2, the  median number of strategic goals and objectives was the same or similar  between these two groups of plans.
section_title: Two Approaches to Linking Budget and Planning Structures Were Significant
paragraphs: Although the complexity of an agency’s budget and planning structures  generally was not significantly related to whether it linked program activity  funding to performance goals, two approaches to linking these structures  were.  These approaches were (1) establishing simpler relationships  between program activities and performance goals and (2) fully integrating  budget justifications and performance plans.  Whether used alone or in  combination, these approaches were more frequently associated with  agencies that were able to show the performance consequences of budget  decisions.
A significant difference in allocating program activity funding to  performance goals existed between agencies in which the performance  plan was integrated with the agency’s budget justification and agencies in  which the performance plan was not integrated.  Agencies whose plans  were fully integrated with the budget justification nearly always (five of six  such plans) allocated program activity funding to performance goals (see  figure 11).  Conversely, about two-thirds, or 20 of the 29 agencies whose  plans could be physically separated from the budget justification, did not  allocate program activity funding to performance goals.  For example,  figure 12 shows how information traditionally contained in a budget  justification, such as descriptions of accounts and their funding, was  combined with performance information in the Internal Revenue Service’s  (IRS) integrated budget justification and performance plan.
We also found that plans showing a simpler relationship between program  activities and performance goals were significantly more likely to show  how funding was allocated to performance goals (see figure 13).  That is, in  all agencies in which the relationship between program activities and  performance goals could be characterized as one-to-many or many-to-one,  program activity funding was allocated to performance goals.  For  example, figure 14 shows that the allocation of funding to performance  goals in the NRC plan was essentially automatic because each of the  agency’s program activities generally align with a strategic goal and its  supporting performance goals.  However, where relationships were less  precise—that is, when multiple program activities were related to multiple  performance goals—allocations of program activity funding to  performance goals were less common.
As indicated by the agencies profiled in the figures above, approaches that  were individually significant in allocating funding were often used in  combination.  In fact, 6 of the 14 plans that allocated program activity  funding to performance goals used two or more of the approaches  identified above.  For example, of the three agencies that changed their  program activity structures, two either presented simple relationships  between program activities and performance goals or fully integrated their  performance plans and budget justifications.
The 21 plans in our review that did not allocate funding to discrete sets of  performance goals were also generally characterized by common features.   These plans (1) did not reflect any significant change in the agency’s  account or program activity structures, (2) generally were separable from  the justification of estimates, and (3) presented either no explicit  relationship or a many-to-many relationship between performance goals  and program activities.  These features were a hallmark of plans that did  not inform users of the performance consequences of budget decisions.
section_title: First-Year Experiences Demonstrate Progress and Continued Challenges
paragraphs: Our review of selected fiscal year 1999 performance plans presents a mixed  picture.  Certainly, some agencies were able to develop informative  approaches to connect budgetary resources to results.  These approaches  are addressing some of the challenges that have plagued performance  budgeting efforts prior to the Results Act.  They can also be seen as the first  step toward achieving a key objective of the act—a clearer understanding  of what is being achieved for what is being spent.  Paralleling these changes  is growth in the Congress’ interest in performance information in its  resource allocation and other oversight processes.  Nevertheless, our  review, as well as the delay of the performance budgeting pilots required by  the act, indicates continuing challenges for achieving a clearer relationship  between budgetary resources and results.
subsections: section_title: A Year of Progress in Connecting Resources to Results
paragraphs: The fiscal year 1999 budget process marked an important beginning in  more clearly showing the performance consequences of budget decisions.   As indicated in the previous section, executive branch agencies developed  a variety of approaches to link their performance plans and budget  requests.  But equally important, the Congress also showed an awareness  of Results Act implementation efforts and a clear interest in obtaining  credible performance information during its appropriations and oversight  processes.
Executive agencies have demonstrated that linking complex budget and  planning structures demands adaptive approaches.  The scope of the  federal government’s missions, the variety of its organizational models, and  the breadth of its processes—all subject to a multifaceted congressional  oversight environment—suggest that many approaches will be developed  to more clearly allocate requested funding levels to performance goals.   Our review found that agencies reflecting the heterogeneity of the federal  government—from direct service agencies (e.g., IRS) to agencies  principally involved in grant or loan making (e.g., USAID) to regulatory  agencies (e.g., NRC)—began to link budgetary resources and results.
In the fiscal year 1999 performance plans, the agencies we reviewed  developed several approaches to overcome a common problem of previous  performance budgeting initiatives—planning structures and presentations  that were unconnected to budget structures and presentations.  These  approaches include: changing budget structures to more closely align with performance  plans. Three agencies proposed new program activities within existing  budget accounts to generally reflect the strategic goals of their  performance plan.  These proposals sometimes facilitated a relatively  simple relationship between program activities and performance goals  that helped make connections clear. integrating performance information with budget justifications.  In some  cases, this took the form of fully integrating the performance plan with  the agency’s budget justification, as in the IRS, the Customs Service, and  the Federal Bureau of Investigation. Where plans were not fully  integrated with budget justifications, some agencies used the  justification to provide more detailed information on goals contained in  their performance plan.  For example USAID’s fiscal year 1999 budget  justification contained “strategic support objectives” and “special  objectives.”  These objectives appear to further describe and support the  performance goals expressed in USAID’s separate performance plan. crosswalking performance plans with budget structures. For example,  ACF devised a crosswalk to identify the contribution of its over 60  program activities to its 10 strategic objectives (see figure 4 for an  excerpt from this crosswalk).  The crosswalk identifies funding for  accounts or program activities and—using consolidation and  aggregation—relates an account or program activity to a strategic  objective, and consequently, to the objective’s set of discrete  performance goals.  Account or program activity funding levels are  summed to provide a proposed funding level for each strategic  objective.
As executive agencies developed these approaches and presented their  fiscal year 1999 budget submissions, the Congress also indicated an  increasing interest in credible performance information to inform the  resource allocation process.  We reviewed fiscal year 1999 appropriations  hearings and reports for the agencies in our review that allocated program  activity funding to performance goals and observed that members of the  Congress often made specific reference to the performance information  contained in the agency’s justification and/or performance plan.  Some  notable examples include the following.
ACF officials were questioned as to whether a 4 percent increase in  children exiting foster care through reunification justified the  appropriation being sought for these activities.
U.S. Customs Service officials were asked how the appropriations  committee should evaluate performance and resource requirements for  Customs’ marine mission, given an apparent lack of measures for its  marine enforcement program.
NRC officials were asked what performance measures would be used to  justify U.S. participation and funding in international nuclear safety  programs and how requested budget increases were related to NRC’s  mission.
Food and Nutrition Service officials presented data on the number of  meals being served in the school lunch and breakfast programs and  were asked how much additional budgetary resources would be needed  to serve all eligible children.
The Conference Report on HUD’s Fiscal Year 1999 Appropriations  directed the agency to revise its performance plan to incorporate  measurable goals and outcomes for providing housing vouchers and  certificates to assist families in transitioning from welfare to work.
A House Subcommittee on Appropriations was unwilling to recommend  funding for a request for community-based technology centers in part  because specific performance measures for this new program were not  presented.
The Treasury and General Government Appropriations Act of 1999  stated that the Office of National Drug Control Policy could not obligate  funds provided to continue its national media campaign until it  submitted the evaluation and results of the campaign.
section_title: Challenges Facing Future Performance Budgeting Efforts
paragraphs: Deliberations on agencies’ appropriations also indicate that making  effective linkages between budget program activities and performance  goals is one of many challenges that need to be addressed for performance  information to be used in the budget process.  Members of the Congress  also questioned agencies about why goals were not more results-oriented  and what steps were being taken to coordinate activities with other  agencies.  For example, Office of Personnel Management officials were  asked if the agency’s performance plan could identify measures to help  determine the agency’s progress toward a result of recruiting and retaining  the federal workforce required for the 21st century.  Members of the  Congress were also concerned about agencies’ use of program evaluation  and other techniques to ensure the validity and reliability of performance  data.  ACF officials were asked about their approach for evaluating the  academic success of Head Start preschool participants after those students  leave the program.
These concerns demonstrate that agencies also need to adopt a broader  agenda for improving performance plans that includes focusing on results,  defining clear strategies, and improving their capacity to gather and use  performance data. Translating the use of agency resources into concrete  and measurable results will be a continual challenge that will require both  time and effort on the part of the agency.  The uneven pace of progress  across government is not surprising; agencies are in the early years of  undertaking the changes that performance-based management entails.  Although some agencies, as indicated in our review, began to show the  performance consequences of budget decisions, improvements can be  made in the clarity and completeness of linkages between program  activities and performance goals.  Agencies must also balance the scope  and precision of funding estimates for performance goals with the  usefulness of such estimates for resource allocation decisions.  In addition,  we believe weaknesses in the performance measurement systems  described in agencies’ fiscal year 1999 performance plans need to be  addressed.
Linkages between plans and budgets must be supported by results-oriented  and credible performance data to be useful.  Our assessment of agencies’  fiscal year 1999 performance plans found most goals were focused on  outputs, not outcomes.  This presents a dilemma for future performance  budgeting efforts.  While outcome information is clearly useful for  measuring performance, it may be more difficult to allocate funding to  outcomes that are far removed from the inputs that drive costs.  Allocating  funding to outcomes presumes that inputs, outputs, and outcomes can be  clearly defined and definitively linked.  For some agencies, these linkages  are unclear or unknown.  For example, agencies that work with state or  local governments to achieve performance may have difficulty specifying  how each of multiple agencies’ funding contributes to an outcome.
In addition to understanding how actions affect outcomes, allocating  funding to outcomes also requires an ability to understand how costs are  related to outcomes.  Agencies are noting the importance of cost  accounting and other management systems for success in allocating  funding to performance. For example, the Department of Housing and  Urban Development’s (HUD) fiscal year 1999 performance plan  acknowledged that the agency “has no mechanism for tracking resources  as they are applied to performance measures.”  The plan noted that HUD  intends to develop a system that “will allow the Department to identify,  justify, and match resource requirements for effective and efficient  program administration and management.”
Agencies are expected to develop such systems as they implement  managerial cost accounting standards developed by the Federal  Accounting Standards Advisory Board (FASAB).  These standards  require that agencies develop and implement cost accounting systems that  can be used to relate the full costs of various programs and activities to  performance outputs.  Although these standards were originally to become  effective for fiscal year 1997, the Chief Financial Officers (CFO)  Council—an interagency council of the CFOs of major agencies— requested the effective date be delayed for 2 years due to shortfalls in cost  accounting systems.  As FASAB recommended, the effective date was  extended by 1 year, to fiscal year 1998, with a clear expectation that there  would be no further delays.  However, developing the necessary approach  to gather and analyze needed program and activity-level cost information  will be a substantial undertaking.  While there is a broad recognition of the  importance of doing so, for the most part agencies have just begun this  effort.
As discussed earlier in this report, agencies’ difficulties in developing  performance planning and measurement and cost accounting systems were  cited by OMB in its 1997 decision to delay the performance budgeting pilots  required by the Results Act.  To further discussion of those pilots, OMB  recently suggested possible formats and time frames for the pilots in a  September 1998 paper sent to federal agencies.  In that discussion paper,  OMB noted that pilot projects would not be designated unless they could  “fairly test the  concept of performance budgeting,” which it  described as “the application of multi-variate or optimization analysis to  budgeting.”  The paper described three analytical alternatives that could be  tested involving performance tradeoffs (1) in the same program with  changes in program funding, (2) in the same program with no change in  total program funding, or (3) in several programs with shifts in intra-agency  funding between these programs.
At present, OMB has no definitive plans for proceeding with the  performance budgeting pilots.  OMB solicited agencies’ comments on the  discussion paper and on their capability to produce the alternative budgets  suggested in the committee report accompanying the Results Act.   According to OMB, no agency has as yet volunteered to participate in the  pilots.  In its discussion paper, OMB stated that “the absence of designated  pilots or having fewer designations than required would be an indication of  agency readiness to do performance budgeting, and would be discussed in  the OMB report to Congress.”
These developments reflect some of the broader tensions involved in  linking planning and budgeting structures. On one hand, performance plans  need to be broad and wide-ranging if they are to articulate the missions and  outcomes that agencies seek to influence. Often these plans will include  goals that the agency can only influence indirectly because of  responsibilities assigned to other actors, such as state and local  governments. On the other hand, budget structures have evolved to help  the Congress control and monitor agency activities and spending and, as  such, are geared more to fostering accountability for inputs and outputs  within the control of agencies.
Performance budgeting poses the daunting task to agencies of discovering  ways to address these competing values that are mutually reinforcing, not  mutually exclusive.  Strategies for bringing planning and budgeting  structures together must balance both sets of values.  For example, some  agencies might let their planning structures be the starting point for making  connections and seek to crosswalk broad overarching goals to their many  program activities.  This approach is consistent with a results orientation  but can obscure the impact of specific funding decisions.  Other agencies  might decide to use the budget structure as their starting point and  integrate performance information into formats familiar and useful for  congressional oversight.  Although this approach may be helpful in  focusing on the performance consequences of budget decisions, such  strategies risk confining performance information to structures that may be  too limited to fully address the broad mission and outcomes of the agency.
section_title: Conclusions
paragraphs: The fiscal year 1999 annual performance plans—the first called for under  the Results Act—showed potential for providing valuable information that  can be used to better program performance.  However, the linkage between  requested funding and performance goals is just one of many elements that  need improvement for these plans to be useful for improving program  performance.  Top management within agencies must provide the  consistent leadership necessary to direct the needed management changes  and to ensure momentum is maintained.  Ultimately, performance-based  management should become an integral part of an agency’s culture.  The  transition process must include proven “change management” approaches  to be successful and sustained.   In addition, congressional use of results- oriented program performance and cost information in its decision-making  about federal policies and programs will also spur agencies’ efforts to  implement the statutory framework by sending the unmistakable message  to the agencies that the Congress remains serious about performance- based management and accountability.
As we have noted in a previous report, one of the Results Act’s most  conspicuous and useful features is its reliance on experimentation.  This  is certainly true regarding performance budgeting.  The act calls for one  form of performance budgeting by requiring that performance goals from  an agency’s annual performance plan cover the program activities of the  agency’s budget request while giving agencies flexibility in how this linkage  is made.  This requirement, when coupled with OMB’s guidance that plans  reflect the funding levels being applied to achieve performance goals,  constitutes the first governmentwide expectation to directly associate  budgetary resources with expected results.  In addition, the act also  requires pilot projects to test a specific form of performance budgeting that  presents varying levels of funding for varying levels of performance.  The  committee report accompanying the act noted that “this pilot approach is  best because, while performance budgeting promises to link program  performance information with specific budget requests, it is unclear how  best to present that information and what the results will be.”
The fiscal year 1999 performance planning and budgeting cycle produced a  useful experimentation in connecting planning and budgeting structures  that accommodated unique federal missions and structures.  Some, but not  all of the agencies we reviewed, began to develop useful linkages.   Moreover, challenges in performance planning and measurement and  deficiencies in cost accounting systems continue to confront federal  agencies.  OMB has already cited these problems as the reasons why  performance budgeting pilot projects were not being initiated. The  progress that has been made and the challenges that persist underscore the  importance of developing a specific agenda to ensure continued progress in  better showing the performance consequences of budgetary decisions.
The original goal for the act’s performance budgeting pilot projects was  twofold: to allow OMB and agencies to develop experience and capabilities  towards realizing the potential of performance budgeting, and to provide  OMB with a basis for reporting to the Congress on next steps and needed  changes.  Although OMB stated in 1997 that agencies’ financial  management systems were not capable of the specific form of performance  budgeting called for in the act, our review demonstrates that some agencies  were able to develop approaches to make perhaps a more basic, but still  useful, connection between proposed spending and performance.  These  fiscal year 1999 efforts to link performance goals and program activity  funding essentially constitute a first step toward achieving the intent of the  performance budgeting pilots.  They also provide a baseline from which  OMB could assess future progress and determine what changes, if any, may  be needed to the act and federal budget processes.
In addition to its longstanding responsibilities regarding the formulation,  review, and presentation of the President’s annual budget requests, OMB is  the lead agency for overseeing a framework of recently enacted financial,  information resources, and performance planning and measurement  reforms designed to improve the effectiveness and responsiveness of  federal agencies.  As such, OMB is well-situated to assess (1) the  practicality of performance budgeting pilots as currently defined in the law,  (2) agency approaches and continuing challenges to linking budgetary  resources and performance goals, and (3) options to encourage progress in  subsequent planning and budgeting cycles.
section_title: Recommendation to OMB
paragraphs: In light of the indefinite delay of the performance budgeting pilots required  by the Results Act and the experiences of agencies during the fiscal year  1999 performance planning and budgeting cycle, we recommend that the  Director of OMB assess the approaches agencies use to link performance  goals and program activities in the fiscal year 2000 performance plans.   OMB’s analysis, building on our review of fiscal year 1999 performance  plans, should develop a better understanding of promising approaches and  remaining challenges with respect to the concept of performance  budgeting within the federal government.  OMB’s analysis should address,  for example: the extent of agencies’ progress in associating funding with specific or  sets of performance goals, how linkages between budgetary resources and results can be made  more useful to the Congress and to OMB, what types of pilot projects might be practical and beneficial, and when and how those pilot projects would take place.
On the basis of this analysis, we recommend that OMB work with agencies  and the Congress to develop a constructive and practical agenda to further  clarify the relationship between budgetary resources and results, beginning  with specific guidance for the preparation of agencies’ fiscal year 2001  plans.  We further recommend that this analysis and the resulting agenda  become the foundation for OMB’s report to the Congress in March 2001, as  currently required by the Results Act, on the feasibility and advisability of  including a performance budget as part of the President’s budget and on  any other needed changes to the requirements of the act.
section_title: Agency Comments
paragraphs: On February 10, 1999, we met with the Deputy Director for Management  and other OMB officials to discuss this report and our recommendations;  on February 19, 1999, we provided a draft of this report to OMB for  comment.  At both our meeting and subsequently, OMB provided technical  comments orally, which we have incorporated as appropriate.  On March  26, 1999, OMB informed us they would have no written comments on this  draft.
As agreed with your office, unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days from the  date of this letter.  At that time, we will send copies to Senator Joseph  Lieberman, the Ranking Minority Member of your committee; other  appropriate congressional committees; and The Honorable Jacob Lew,  Director, Office of Management and Budget.  We will also make copies  available to others on request.
Major contributors to this report are listed in appendix III.  Please contact  me on (202) 512-9573 if you or your staff have any questions.
section_title: Objectives, Scope, and Methodology
paragraphs: Our September 1998 report on agencies’ first performance plans  establishes an agenda for improving several elements of agency plans,  including showing the performance consequences of budget decisions.   Following the issuance of that report, the Chairman of the Senate  Committee on Governmental Affairs asked us to examine in more detail  how the plans linked expected performance with budget requests.  To do  this, our objectives were to describe agencies’ approaches to linking performance goals and examine characteristics that might be associated with different  approaches to linkage, and identify implications for future efforts to clarify the relationship  between budgetary resources and results.
subsections: section_title: Scope
paragraphs: To address our objectives, we selected 35 fiscal year 1999 performance  plans for review from departments and agencies covered by the CFO Act.   We generally focused on bureau-level plans for each department but limited  our review to the three largest bureaus with discretionary spending over  $1 billion and/or the two largest bureaus.  Table I.1 lists the agencies  whose plans we reviewed.
section_title: Data Collection
paragraphs: To describe agencies’ approaches to linking performance goals and  budgetary resources, we identified 12 characteristics that could be used to  describe agencies’ planning and budgeting structures and the linkages  between them.  For each characteristic, we developed a classification  framework for differentiating between plans based on that characteristic.  These classification frameworks involved either straightforward counts of  plan components (e.g., number of strategic goals) or judgments based on  the content and structure of the plan. One staff member reviewed each plan  and classified the plan on each characteristic. To ensure consistency in  judgments, another staff member also independently reviewed the plans  and the assessment on each characteristic.  Differences in judgments were  addressed by having staff members jointly reevaluate the coding of the  characteristic to resolve the difference.  We then compiled our assessments  on plan characteristics into a database that was used to profile agencies’  first-year approaches to linking budgetary resources with results.
These characteristics generally fell into two groups: characteristics  describing agencies’ budget and planning structures (numbers 1 through 7  in table I.2), and characteristics describing agencies’ approaches to linking  these structures (numbers 8 through 12 in table I.2).  Table I.2 presents  additional detail on the characteristics used in this review.
The number of budget accounts in the Appendix of the Budget of the United States  Government, Fiscal Year 1999 from which agencies proposed to make obligations in fiscal year  1999. a) Single account—Agencies were classified as having a single account if they proposed to  make 75 percent or more of their proposed fiscal year 1999 obligations from one budget  account. b) Multiple accounts—Agencies were classified as having multiple accounts if the amount of  gross obligations proposed for each account was less than 75 percent of the agency’s total  proposed gross obligations.
The number of program activities shown in the Appendix of the Budget of the United States  Government, Fiscal Year 1999 from which the agency proposed to make gross obligations in  fiscal year 1999.  When the Appendix does not list any program activities under a particular  account, the account was deemed to have one program activity, reflecting the entire budget  account.  (4) Common program activity structure A common program activity structure means that most of the agency’s accounts contain  program activities with the same titles. a) Changed structure—Agencies in this category proposed to substantially change their  account and/or program activity structures in the Appendix of the Budget of the United States  Government, Fiscal Year 1999 from those used in the previous year’s Appendix.b) Used fiscal year 1998 structure—Agencies in this category generally used the same account  and program activity structures as they did in the Appendix of the Budget of the United States  Government, Fiscal Year 1998.
The number of goals that are related directly to the agency’s mission without any intervening  plan elements.  According to the Results Act, agencies’ strategic plans are to contain general  goals and objectives that elaborate on the agency mission statement and provide a set of  programmatic objectives for the major functions and operations of the agencies.  We defined  the first layer of goals under the agency’s mission statement as “strategic goals” regardless of  how they were labeled in the plan.
The number of strategic objectives, that is, goals that the plan related directly to the agency’s  “strategic goals” as defined in (6) above.  As in (6), we defined this layer of goals as “strategic  objectives” regardless of how they were labeled in the plan.
B.  Characteristics describing how agencies linked program activities with performance goals a) Full Integration—An agency embedded its performance plans in its justification of estimates  such that the justification could not logically or readily be detached from the performance plan.b) Separable—The agency’s plan was either (1) a separable component of the agency’s  justification of estimates (e.g., an appendix) such that justification readers could either turn to  or skip over the performance plan, but the plan would appear in the justification’s table of  contents; or (2) an entirely separate document that may or may not have been transmitted at  the same time as the justification.  A user would need to read the plan, as opposed to the  justification, in order to understand how the agency addressed Results Act requirements.
Agencies were placed in one of five categories depending on the lowest performance planning  structure to which the plan linked program activities.  (See figure I.1 for an illustration.) a) Strategic goal—Plan related program activities to strategic goals as defined in (6) above. b) Strategic objective—Plan related program activities to goals that the plan related directly to  the agency’s “strategic goals” as defined in (6) above.  These goals may or may not have been  labeled “strategic objectives” by the agency. c) Performance goal—Plan related program activities to performance goals and/or measures.   As defined in the Results Act, a performance goal means a target level of performance  expressed as a tangible, measurable objective against which actual achievement can be  compared, including a goal expressed as a quantitative standard, value, or rate.  For plans in  this category, a program activity was associated with each individual performance goal/ measure. d) Other—Plan related program activities to a structure other than its strategic goals, strategic  objectives, or performance goals.  In many cases, this structure was a business line or some  other unit for which a single goal statement was not expressed. e) None—Agencies were placed in this category if the performance plan did not relate program  activities to any of the above structures (strategic goal, strategic objective, performance goal, or  other) in their performance plans.
Agencies were placed in one of six categories depending on the lowest performance planning  structure for which the plan presented dollar amounts.  (See figure I.2 for an illustration.) a) Strategic goal—Plan presented dollar amounts for each strategic goal. b) Strategic objective—Plan presented dollar amounts for each strategic objective. c) Set of discrete performance goals—Plan presented dollar amounts for any set of  performance goals presented in some combination other than as described in (a), (b), (d), or  (e).  d) Performance goal and/or measure—Plan presented dollar amounts for each performance  goal and/or measure. e) Other—Plan presented dollar amounts for a unit of analysis other than strategic goals,  strategic objectives, or performance goals.  f) None—Plan did not present dollar amounts for any performance planning or other type of  structure. a) One program activity to one goal—Agencies were placed in this category if a program  activity was linked to one performance goal. b) One program activity to many goals—Agencies were placed in this category if a program  activity was linked to more than one performance goal. c) Many program activities to one goal—Agencies were placed in this category if more than one  program activity was linked to one performance goal. d) Many program activities to many goals—Agencies were placed in this category if more than  one program activity was linked to more than one performance goal. e) Could not be determined—Agencies were placed in this category if the plan did not convey  how program activities were related to performance goals.
Plans that allocated funding to a discrete set of goals  (a) generally showed how program activities and their requested funding were allocated among  performance goals/measures or sets of performance goals/measures (plans met this criteria  even if only discretionary funding was allocated) and (b) used sets of goals/measures that were unique (i.e., a single performance goal/measure is  related to only one strategic objective or strategic goal).
While we quantified strategic objectives as defined, the plan may have contained other goal layers  between strategic objectives and annual performance goals.
We reviewed linkages between the program activities and performance goals presented in the plan.   This characteristic did not assess whether all program activities were listed and covered in the  performance plan.
If accounts were linked to the planning structure, the underlying program activities were also  presumed to be linked to this structure.
When a plan contained performance goals distinct from performance measures, reviewers  considered whether this assessment would change if the word “measure(s)” was substituted for the  word “goal(s).”  If so, measure(s) were used as the unit of analysis instead of goal(s).  If agencies  linked program activities to a structure other than performance goals (i.e., an intervening structure  such as a strategic goal or objective), the plan reviewer determined this relationship by examining how  program activities were related to the intervening structure and how the intervening structure was  related to performance goals.
The set of goals ranged in size and scope.  For example, some of these plans presented allocations  of funding to strategic goals or objectives, which represent discrete sets of performance goals.
Figure I.1:  Illustration of Characteristic 9 Example 9(a)
Example 9(b)
Example 9(c)
Figure I.2:  Illustration of Characteristic 10 Example 10(a)
Example 10(b)
Example 10(c)
Example 10(d)
section_title: Data Analysis
paragraphs: To determine which characteristics of agency planning and budgeting  structures were associated with linkages that showed an allocation of  budgetary resources to results, we prepared contingency tables depicting  the relationship between characteristic 12 in table I.2, “Funding allocated  to a discrete set of goals or measures,” and each of the other  characteristics.  To assess whether the relationships in the tables were  statistically significant, we performed two statistical techniques.  When one  characteristic in the table contained numeric values (e.g., characteristic 6,  which measured the number of strategic goals), we used logistic regression  techniques.  The logistic regression technique involved regressing the odds  on funding being allocated to a discrete set of performance goals on each  characteristic and determining—using likelihood ratio chi square tests— whether the characteristic was associated with significant differences in  those odds.
When a characteristic had nonnumeric values (e.g., characteristic 11, which  had five discrete categories), we used standard contingency tables to  analyze the data. The contingency table techniques involved calculating the  percentages of agencies that allocated funding to a discrete set of goals  across the categories of the other characteristic and computing likelihood- ratio chi square statistics to determine whether differences in those  percentages were statistically significant.   In addition to computing the  likelihood ratio chi square statistic, we also computed Fisher’s exact test to  assess whether a characteristic was significantly related to characteristic  12.  Fisher’s exact test was used to confirm the likelihood ratio chi square  results because of the small number of observations in many of our tables.   In most of our analyses, the likelihood ratio chi square and Fisher’s exact  test yielded similar conclusions.  Where they did not, the differences  appeared negligible given our sample size.
When a statistically significant association was identified in a table where  one or both characteristics being analyzed had more than two categories,  we conducted a series of additional chi square tests before and after  grouping various categories of those characteristics to discern whether our  description of the relationship of that characteristic with characteristic  12 could be simplified. Appendix II presents a summary of our analysis.
The intent of our statistical analyses was to quantitatively identify and  explore associations of various plan characteristics with plans that did or  did not allocate funding to a discrete set of goals and/or measures.   However, the following qualifications apply to our analysis.
Although the results of our analyses apply to the plans we reviewed, our  plan selection procedures preclude generalizing the results to agency  plans not included in our population.
For those characteristics we identified as having a significant  association with characteristic 12, it is possible that this result occurred  because of the close association of that characteristic with one or more  of the other characteristics that were related to characteristic 12.  The  small number of plans reviewed precluded our use of statistical  approaches that would enable us to assess the relationship of two or  more characteristics simultaneously on characteristic 12.  To provide  some rudimentary insight into the extent to which characteristics  significantly related to characteristic 12 were related to each other, we  examined whether there were significant associations among those  characteristics using the same procedures described above.  Significant  associations were found between (1) characteristics 5 and 8,  (2) characteristics 8 and 10,  (3) characteristics 8 and 11, and  (4) characteristics 10 and 11.
Aspects of agencies’ linkages not specifically mentioned in table I.2  were not assessed.  For example, we did not assess whether all agency  program activities were listed and covered in the performance plan.    This assessment was made in our September 1998 report.
Our analysis focused on linkages between performance goals and  program activities in performance plans.  We did not assess other  elements of the performance plan, such as the quality of any goals  presented in the plan. We also did not independently verify the funding  amounts that agencies allocated to performance goals.  We did not  systematically assess other documents, such as agency budget  justifications.
Our review focused on the 12 characteristics mentioned above.
However, there may be other characteristics that might be associated  with agencies’ linkages.  For example, although our previous work has  noted that account and program activity orientation may be important  factors in making linkages, the subjective nature of this characteristic  prevented its inclusion in our analysis.
Finally, to identify implications for future performance budgeting efforts,  we gathered information on congressional perspectives.  In addition to  discussing the plans’ linkages between budgetary resources and results  with selected appropriations staff, we also reviewed the House Committee  on Appropriations’ hearing records on agencies’ fiscal year 1999  appropriations, giving special attention to how lawmakers reacted to the  performance information presented in performance plans and budget  justifications. We discussed the status of performance budgeting pilots  with OMB.  We requested comments on a draft of this report from the  Director of OMB or his designee and incorporated OMB’s comments as  appropriate.  We conducted our work in accordance with generally  accepted government auditing standards from August 1998 to February  1999.
section_title: Summary of Our Analysis
paragraphs: Spending concentrated in one account Budget structure used in the performance  plan Fully integrated with budget justification Relationship of program activities to performance goals 22 Although the probabilities for this characteristic were 0.05 or less, our analysis revealed the source of  statistical significance was the difference between the “none” category and all of the other categories  shown for this characteristic combined.
section_title: Major Contributors to This Report
subsections: section_title: Accounting and Information Management Division, Washington, D.C.
section_title: General Government Division, Washington, D.C.
paragraphs: Thomas M. Beall, Senior Social Science Analyst Douglas M. Sloane, Supervisory Social Science Analyst The first copy of each GAO report and testimony is free. Additional copies are $2 each.  Orders should be sent to the  following address, accompanied by a check or money order made  out to the Superintendent of Documents, when necessary, VISA and  MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address are  discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and  testimony.  To receive facsimile copies of the daily list or any list  from the past 30 days, please call (202) 512-6000 using a touchtone  phone.  A recorded menu will provide information on how to obtain  these lists.