Grants Management: Enhancing Performance Accountability Provisions Could Lead to Better Results
Published: 2006-09-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Maximizing the extent to which grants achieve their long-term performance goals is critical to successfully addressing the challenges of the 21st century. While performance accountability mechanisms are fairly new to federal grants, they have been used in contracts for some time and lessons learned have begun to inform federal grant design. Given this, GAO was asked to examine (1) challenges to performance accountability in federal grants, (2) mechanisms being used to improve grant performance, and (3) strategies the federal government can use to encourage the use of these mechanisms. GAO performed a content analysis of relevant literature and interviewed experts. To illustrate the mechanisms and strategies found in the literature, GAO used examples from the literature and selected additional case illustrations--two federal grant programs (vocational education and child support enforcement) and two nonfederal contracts--for further study.
section_title: What GAO Found
paragraphs: Accountability provisions in federal grants can vary widely. They can be financial (e.g., bonus payments) or nonfinancial (e.g., altered oversight or flexibility), and can be employed by various actors at different stages in the grant life cycle. Mechanisms need to be tailored to specific situations since there is no "one-size-fits-all" solution. Collectively, five key strategies appear to facilitate the effective design and implementation of performance accountability mechanisms. They are as follows: 1. ensure mechanisms are of sufficient value to motivate desired behaviors, 2. periodically renegotiate and revise mechanisms and measures, 3. ensure appropriate measurement selection, 4. ensure grantor and grantee technical capacity, and 5. allow for phased implementation. In addition to these strategies, collaboration, oversight, and feedback also appear critical to the success of performance accountability mechanisms. Opportunities exist to improve the design and implementation of federal grants. A results-focused design can enable and facilitate the use of accountability provisions. National program evaluation studies and demonstration grants can provide valuable information to support oversight of and knowledge about accountability mechanisms. Finally, the Office of Management and Budget (OMB), agencies, and grantees can benefit from sharing good practices and lessons learned regarding performance accountability provisions. OMB recognized the value in sharing information on performance accountability mechanisms, but has not yet focused on this issue.

section_title: Letter
paragraphs: The federal government faces an array of challenges and opportunities to  enhance performance, ensure accountability, and position the nation for  the future. A number of overarching trends—including the nation’s long- term fiscal imbalance—drive the need to reexamine what the federal  government does, how it does it, who does it, and how it gets financed.  Because grants to state and local governments constituted nearly 20  percent of total federal outlays in fiscal year 2005, maximizing the extent  to which grants achieve their long-term performance goals and objectives  is critical to successfully addressing the challenges of the 21st century.
In recent years, interest in federal grant performance accountability has  grown. For the purposes of this report, performance accountability is  defined as the mechanisms by which individuals or organizations are held  accountable for meeting specified performance-related expectations.  Consistent with the decade-long trend toward an increased results  orientation and expectation for performance accountability as evidenced  by the Government Performance and Results Act of 1993 (GPRA),  performance accountability mechanisms in federal grants have become  more common. For example, performance assessment mechanisms are  present in grants authorized by both the Job Training Partnership Act and  its successor program, the Workforce Investment Act of 1988 (WIA) and  the No Child Left Behind Act. More recently, the Office of Management  and Budget (OMB) developed the Program Assessment Rating Tool  (PART), which, among other things, holds federal programs and their  partners accountable for performance. Simply monitoring and reporting  performance can also encourage performance accountability and  performance improvements. In this report, we have focused on specific  mechanisms that are meant to encourage performance incentives—such  as rewards given or penalties imposed—when performance exceeds or  fails to meet specified levels.
While performance accountability mechanisms are fairly new to some  federal grants, they have been used in contracts and loans for some time.  Moreover, lessons learned from performance-based contracting have  begun to inform federal grant design, for example, in the case of WIA  grants requirements. In addition, some states award their federal pass- through grants to subgrantees as contracts with performance  accountability requirements.
Given this growing body of experience, you asked us to examine ways to  infuse effective performance accountability mechanisms and practices  into the federal grant process. Specifically, our objectives were to identify  (1) What kinds of challenges to performance accountability exist in federal  grants? (2) What kinds of mechanisms are being used to improve grant  performance, and how? and (3) Given the findings of questions 1 and 2,  what strategies can the federal government use to encourage the use of  these mechanisms, as appropriate? For the purposes of this report, we  were interested specifically in the mechanisms by which individuals or  organizations are held accountable for meeting specified performance- related expectations that are directly tied to a grant.
To address our objectives, we conducted a literature review that included  our prior reports, and interviewed experts in the area of federal grant and  contract performance accountability to identify the: (1) challenges to  performance accountability that exist in federal grants, (2) types of  performance accountability mechanisms—defined as rewards and  penalties—used, and (3) key strategies that appear to encourage the  successful implementation of performance accountability mechanisms.  Our identification of types of mechanisms and strategies was developed by  conducting a content analysis of selected literature from our review that  met our criteria for addressing the issue of accountability. Recognizing  that grants have increasingly assumed features traditionally associated  with contracts, we drew on experiences from both performance-based  contracting and grants to help identify valuable lessons learned that could  inform efforts to improve performance accountability in federal grants. To  illustrate these mechanisms and strategies, and to supplement our findings  and the many case examples identified by our content analysis, we use  relevant case examples found in the literature. We also selected four  additional cases for further in-depth illustrations. These four cases were  selected based on our literature review, interviews with experts, and  reviews of prior GAO work because they are good examples of where (1) a  performance mechanism was present and (2) there is reason to believe  that performance improved. We selected two federal grant programs and  two nonfederal contracts: (1) the federal vocational education grants  authorized by the Carl D. Perkins Vocational and Technical Education Act  of 1998 (Perkins III), which are passed through states to secondary and  postsecondary schools for career and technical education; (2) the federal  Child Support Enforcement (CSE) program, authorized by the Social  Security Act, Title IV, part D, which ensures that children are financially  supported by both parents; (3) the real property management contract  between the Ontario Provincial Government’s Ontario Realty Corporation  (ORC) and SNC-Lavalin ProFac, Inc. (ProFac), a private property  management company; and (4) the contract between the Massachusetts  Division of Medical Assistance (DMA) and the Massachusetts Behavioral  Health Partnership (MBHP) for the provision of mental health and  substance abuse services for residents covered by the MassHealth  Medicaid program.
To develop the federal grant case illustrations and obtain perspectives on  the strategies we identified, we interviewed federal headquarters and  regional program and finance officials from the federal agencies that  administer the grant programs—the Departments of Education and Health  and Human Services. In addition, we visited selected grantees and  subgrantees from among these programs that federal and state officials  identified as being particularly successful—or as facing particular  challenges—with performance accountability. To develop the contracting  case illustrations, we interviewed, conducted site visits, or both with both  contractors and contracting agencies. For both the grant and contract  cases, we reviewed the authorizing legislation or contract, guidance,  documentation, and prior studies, and interviewed relevant officials to  obtain perspectives on the strategies we identified.
See appendix I for a more detailed discussion of our scope and  methodology.
We conducted our work from December 2005 through August 2006 in  offices in Washington, D.C.; Harrisburg, Lancaster, Norristown, and  Philadelphia, Pennsylvania; Eloy, Glendale, and Phoenix, Arizona; and  Boston, Massachusetts, in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: Grants, along with contracts and cooperative agreements, are tools used  by the federal government to achieve national priorities via nonfederal  parties, including state and local governments, educational institutions,  and nonprofit organizations. Diverse in structure and purpose, grants can  be generally classified as either categorical or block, with categorical  grants allowing less recipient discretion than block grants. For example,  the Community Services Block Grant provides funds to states and is  sometimes passed to local agencies to support a variety of efforts that  reduce poverty, revitalize low-income communities, and lead to self- sufficiency among low-income families and individuals, while giving the  agencies broad discretion in how the funds can be spent. In practice, the  “categorical” and “block” grant labels represent the ends of a continuum  and overlap considerably in its middle range.
Grant funds may also be grouped by their method of allocating funds.  Formula grants allocate funds based on distribution formulas prescribed  by legislation or administrative regulation and often narrowly define the  eligible recipients as state agencies. On the other hand, categorical grants  are generally awarded on a competitive basis to applicants meeting  broader eligibility requirements.
Despite substantial variation among grants, grants generally follow a  similar life cycle and include announcement, application, award,  postaward, and closeout phases. Once established through legislation,  which may specify particular objectives and eligibility and other  requirements, a grant program may be further defined by grantor agency  requirements. For competitive grant programs, the public is notified of the  grant opportunity announcement, and potential grantees must submit their  applications for agency review. In the awards stage, the agency identifies  successful applicants or legislatively defined grant recipients and awards  funding. The postaward stage includes payment processing, agency  monitoring, and grantee reporting, which may include financial and  performance information. The closeout phase includes preparation of final  reports, financial reconciliation, and any required accounting for property.
Traditionally, grant accountability has referred to legal or financial  compliance. The Single Audit Act, for example, requires grantees to  conduct an overall financial compliance audit to promote accountability.  As such, at a minimum all grantees are held accountable for sound  financial management and use of federal funds to support allowable  activities. Beyond that, however, accountability for performance varies  from grant to grant. As discussed earlier, this historical focus on financial  accountability has expanded in response to increasing expectations of  demonstrable performance and performance accountability for all  government programs. For example, the Comptroller General’s Domestic  Working Group issued its Guide to Opportunities for Improving Grant  Accountability, highlighting innovative approaches and promising  practices in grants management—focused both on ensuring grant funds  are spent properly as well as achieving their desired results.
While performance accountability in grants is a relatively new pursuit, it  has been used in contracts for a number of years. To illustrate  performance accountability mechanisms and the strategies that contribute  to their successful design and implementation, we examined four cases:  (1) the federal CSE program, (2) the federal Perkins III Career and  Technical Education Program, (3) a performance-based contract between  the Massachusetts DMA and MBHP, and (4) a performance-based contract  between the Canadian Ontario Realty Corporation (ORC) and ProFac.
subsections: section_title: Child Support Enforcement
paragraphs: The CSE program was established in 1975 by Title IV-D of the Social  Security Act (Pub. L. No. 93-647). CSE functions in all states and territories  through state or local social services departments, attorneys general  offices, or departments of revenue in order to ensure that children are  financially supported by both of their parents. State programs work  toward establishing paternity, locating parents, establishing and enforcing  support orders, and collecting and distributing child support payments.  The federal Office of Child Support Enforcement (OCSE), an office of the  Department of Health and Human Services’ Administration for Children  and Families, oversees the development, management, and operation of  state CSE programs and provides financial support (66 percent of total  operating costs) to states. In fiscal year 2005 federal expenditures on CSE  were $3.5 billion, with states spending $1.8 billion. Total collections in  fiscal year 2005 were more than $23 billion. The total legally owed support  for fiscal year 2005 was $29 billion, with $17.4 billion of that collected.
Total arrears (past due payments) for all previous years combined was  $107 billion. Over $7 billion of those past due payments were collected and  distributed in fiscal year 2005.
The Child Support Performance and Incentive Act of 1998 (Pub. L. No. 105- 200) linked incentive payments to performance, and in fiscal year 2005,  OCSE made over $450 million in incentive payments to states. This act  changed the original CSE incentive program from awarding incentives  based solely on cost-effectiveness to awards based on meeting specific  performance targets in five outcome areas: paternity establishment, order  establishment, current collections, past due collections, and cost- effectiveness. The performance measures and targets are defined in the  text of the act, which also provides a formula for determining the amount  of each incentive payment. Additionally, the act established an alternative  penalty system for those states not yet in compliance with the statewide  automated data processing system required by Title IV-D Sec. 454(A) of  the Social Security Act. The new incentive program was phased in from  2000 through 2002.
section_title: Carl D. Perkins Career and Technical Education
paragraphs: Effective July 1, 1999, the Carl D. Perkins Vocational and Technical  Education Act of 1998 (Perkins III, Pub. L. No. 105-332) amends earlier  legislation to evaluate and improve vocational and technical education.  Each year under Perkins III, Congress has appropriated more than   $1.1 billion in grants to states for career and technical education. The  Office of Vocational and Adult Education (OVAE), an office of the  Department of Education, administers the grants established in Perkins III,  a pass-through grant to states, which administer the distribution of the  funds to local school districts.
Perkins III defines major roles for OVAE and states in establishing  performance accountability systems for vocational and technical  education. States are given the responsibility for developing performance  measures and data collection systems related to four required core  performance indicators: academic and technical skill attainment,  completion, placement and retention, and nontraditional participation and  completion. OVAE negotiates these performance measures with states to  ensure that they are sufficiently rigorous. States not meeting their  performance levels for 1 year are required to complete a program  improvement plan. States not meeting their performance levels for 2 years  are subject to financial sanctions, although no state has failed to meet its  overall levels for 2 consecutive years. States have also been eligible to  receive incentive funds if they exceeded performance goals for the Perkins  III grant as well as targets established by Title I and Title II of WIA. Title I  of WIA supports workforce investment programs. Title II, also known as  the Adult Education and Family Literacy Act, provides adult education  funds to states. Governors have the authority to allocate the incentive  funds for use in any of the three program areas.
section_title: Massachusetts Division of Medical Assistance Contract with the Massachusetts Behavioral Health Partnership
paragraphs: Beginning in 1996, the Massachusetts DMA entered into a 5-year contract,  which was renewed in 2001, with MBHP to manage mental health and  substance abuse services for roughly 300,000 people covered by the  MassHealth Primary Care Clinician Plan—part of the Massachusetts  Medicaid program. Of the individuals covered by the plan, more than half  are children 18 or younger, including 20,000 children in the custody of the  commonwealth’s Departments of Social Services and Youth Services.
The structure of the contract between DMA and MBHP involves a base  contract, which governs requirements related to administrative and  medical operations. These requirements continue through the life of the  contract, or until they are modified through amendments. In addition to  the base contract, there are performance incentive projects that focus on  research and development projects. The majority of earnings available to  MBHP come from the organization’s successful completion of these  contractually defined incentive projects, which are renegotiated annually.  Earnings are achieved only after the successful completion of specified  goals and objectives, as documented and reviewed by the state.
section_title: Ontario Realty Corporation’s Contract with ProFac
paragraphs: ProFac, a facilities management company, was awarded a 5-year contract  in 1999 (since renewed) by ORC to provide facilities management services  for approximately 30 million square feet of space in 2,100 building sites  owned by the Ontario government. About 280 ProFac managers,  engineers, technicians, and support staff provide these services.
The contract between ORC and ProFac links performance to a 10 percent  quarterly management fee holdback: on a monthly basis, ORC only  reimburses ProFac for 90 percent of its administrative costs, retaining the  other 10 percent, a “holdback,” which ORC returns to ProFac on a  quarterly basis only if ProFac obtains a sufficient level of performance.  This contract also links performance to an annual share-in-savings  arrangement: ORC sets a budget each year based on prior years’ actual  expenditures and budget projections; if ProFac spends less than the  budget, it is able to share the difference, a share in savings, only if it  reaches a sufficient operational level. ProFac’s performance on 30 key  performance indicators (KPI) determines its operational level. These KPIs  are accumulated to determine a score in four performance objectives:  management performance, financial performance, asset integrity, and  customer service. The performance objectives are scored and then  weighted according to ORC’s priorities to determine a total performance  rating.
section_title: Trade-offs and Challenges Exist in Ensuring Performance Accountability in Federal Grants
paragraphs: The trade-offs and challenges associated with performance accountability  in federal grants largely depend on several key aspects of grant design and  implementation. As we have previously reported, performance  accountability tends to be greater (and grantee flexibility lower) in  programs with certain types of design features. Because design features  that encourage performance accountability can limit state and local  grantee flexibility, achieving these twin goals can be a delicate balancing  act and has implications for the accountability relationship between levels  of government and the information needed to support accountability. Even  in federal grants with designs that favor performance accountability, grant  implementation challenges related to developing performance goals and  measures as well as collecting and reporting performance data can  influence the extent of performance accountability achieved.
subsections: section_title: Grant Design Features Affect the Balance between Accountability and Flexibility
paragraphs: Although there are various ways to design grants to encourage  performance accountability, in general, there are three factors that  particularly affect the degree of performance accountability that can be  achieved, including whether a grant (1) includes performance-oriented  objectives in addition to fiscally oriented objectives, (2) operates as a  distinct program or as a funding stream, and (3) supports a limited or  diverse array of objectives.
As we discussed previously, federal grants have traditionally focused on  fiscal or legal accountability, such as holding states accountable for using  federal grant funds to supplement rather than to supplant their own  spending on a particular activity. However, federal grants that also include  performance-oriented objectives—as well as the provisions that  implement them—provide the basis for performance measurement and  accountability for results, and signal a federal role in managing  performance over the grant. Ideally, both types of objectives would be  present in federal grants. Performance-related objectives focus on service  or production activities and their results. For example, the central  objective of the grants for Special Programs for the Aging—Nutrition  Services, is to provide nutritious meals to needy older Americans to  improve nutrition and reduce social isolation. In contrast, fiscal or  financial assistance objectives focus on providing dollars to support or  expand activities. Typical fiscal objectives include increasing support for  meritorious goods or underfunded services and targeting grant funding to  needy jurisdictions. For example, the objective of Title VI Innovative  Education grants is to provide funds to support local education reform  efforts. When objectives are purely fiscal, accountability to the federal  agency tends to focus on fiscal matters, such as holding states accountable  for using federal grant funds to supplement rather than to supplant their  own spending on a particular activity.
Even when performance-oriented objectives are present, whether federal  grants operate as distinct programs or as part of a larger funding stream  directly affects who can be held accountable and for what. A grant that  operates as a program has performance requirements and objectives and  carries out specific programwide functions through a distinct delivery  system, such that grant-funded activities, clients, and products are clearly  identifiable. This type of grant gives the federal agency a role in managing  performance and makes it easier to obtain uniform information about  performance attributable to the grant funds. It is possible to identify which  activities were supported; the amount of federal funds allocated to each;  and to various extents, the results grantees achieved with federal funds.
In contrast, funds from grants that operate as part of a funding stream are  merged with funds from state or local sources (and sometimes from other  federal sources) to support state or local activities allowable under the  flexible grant. These programs are managed at the state or local level, with  the federal role limited accordingly. When grants are part of a funding  stream, it is possible to identify which activities federal funds supported  and the amount allocated to each, but once the grant funds are combined  with the overall budget for a state or local activity, federal dollars lose  their identification and their specific results cannot be separated out. This  is particularly the case when the federal share is small, with most funding  coming from other sources. The program outcome measures available in  such programs are likely to be for outcomes of the state or local service  delivery program, not the federal program from which the funding  originated. Thus, grantees would generally be held accountable for overall  outcomes, regardless of the funding source. For example, projects such as  Oregon Option and the National Performance Review were designed to  promote accountability for federal and/or national priorities, regardless of  the funding source. They encourage grantors and grantees to work toward  collaboratively developed outcomes. These intergovernmental  partnerships can be particularly useful when funds come from a  combination of federal, state, local, and private sources, or when the  federal funding share is small.
Federal grants vary along a continuum, at one end supporting a single  major activity common to all grantees (such as categorical grants), and at  the other end, allowing unrestricted choice by the recipient among a wide  variety of allowable activities, (such as block grants). Flexibility is  narrowest, but accountability to the federal level clearest, in programs that  focus on a single major activity. Flexibility is broadest in programs  designed to support diverse state or local activities, but finding a common  performance metric can be extremely challenging since these activities  can vary considerably from state to state. That said, we have previously  reported on options for building accountability provisions into block  grants that help balance states’ flexibility to select a mix of activities and  services that will best allow them to achieve a particular national outcome  with accountability for achieving that outcome. These options include   (1) relying on state processes both to manage block grant funds and to  monitor and assess compliance and (2) emphasizing results-based  evaluation rather than examining specific program or administrative  activities.
section_title: Implementation Issues Present Further Performance Accountability Challenges
paragraphs: In addition to these design features, we have previously reported on a  number of performance accountability challenges encountered in many  grant programs during the grant implementation phase.
Lack of consensus on goals and performance measures: The  priorities of states, tribes, local communities, and the federal government  are not always the same. To ensure that grantees work toward national  priorities, they need to be involved in the development of performance  goals and measures. Lack of agreement on goals and measures— particularly when the federal funding is a small portion of the funding  stream—could lead to grantees making choices that do not necessarily  support the achievement of national goals.
Reliance on performance data from state and local partners and  other third parties: Even if grantees collect data on similar activities,  outcomes, and services, absent common data definitions Congress and  program managers will lack comparable information, limiting the ability to  compare state efforts or draw meaningful conclusions about the relative  effectiveness of different strategies. We have previously reported that  agencies relying on third parties for performance data also have difficulty  ascertaining the accuracy and quality of the data. Further, programs often  rely on state administrative systems for performance information. For  some programs—such as many of the Administration for Children and  Families’ programs—-since final reports are not due until 90 to 120 days  after the end of the federal fiscal year, there is a delay in available data.
Onerous and inconsistent grant administration processes and  requirements: Multiple grants maybe available for the same or similar  purposes, meaning that federal grant recipients must navigate through a  myriad of federal grant programs in order to find the appropriate source of  funds to finance projects that meet local needs and address local issues.  Sometimes programs meant to address common problems have potentially  conflicting requirements. Variations in performance accountability  requirements among these grants can limit the degree of performance  accountability achieved. We have recently reported that while this  situation is improving because of OMB’s efforts to streamline the grants  application process, problems still exist.
Prohibition of performance information collection: Because states  are principally responsible for implementing block grants at the state level,  the block grant statutory prohibitions and requirements, and federal  regulations and guidance are generally kept to a minimum. Sometimes  federal agencies are prohibited from imposing reporting requirements  because they are seen as burdensome. Clearly, this limits the extent to  which federal agencies can oversee grantee performance.
Nevertheless, even with these trade-offs and challenges, agencies have  been able to shift toward increased performance accountability in federal  grants and the use of accountability provisions to ensure that grantees  achieve real results through the programs, activities, and services financed  with federal funds. The accountability provisions described in this report,  along with strategies for their effective use, can help address the  challenges noted above.
section_title: Accountability Mechanisms Can Improve Performance and Performance Accountability
paragraphs: We found a number of accountability provisions, specific actions that can  be taken—that is, rewards given or penalties imposed when performance  exceeds or fails to meet specified performance levels—that Congress,  granting agencies, and grantees can use at different points in the grant life  cycle to improve both grant performance and performance accountability.  These examples demonstrate that accountability provisions can result in  significant performance improvement and are flexible enough to  accommodate a variety of situations.
subsections: section_title: A Variety of Accountability Mechanisms Exist
paragraphs: We found that a wide variety of accountability provisions are being used in  both grant and contracting situations. A selection of these provisions is  shown in table 1. This list is not intended to be exhaustive; rather, it is  meant to illustrate the variety of mechanisms available. Some mechanisms  may be more appropriate in certain situations than others, but all of these  mechanisms can be used to either encourage improved performance or  discourage poor performance. For example, public recognition and  increased funding are two different mechanisms that can both be used to  encourage and reward good performance. Similarly, mechanisms such as  reduced funding or increased oversight can be used to discourage or  penalize poor performance.
In addition, mechanisms can be either financial or nonfinancial in nature.  A financial mechanism would be an increase in funding or a bonus. For  example, the CSE program employs a financial incentive in the form of a  bonus to encourage states to work toward the program’s five performance  goals: states are eligible for a bonus every year based on performance.  Nonfinancial mechanisms would include altered oversight or flexibility.  For example, as part of the National Environmental Performance  Partnership System, the Environmental Protection Agency affords states  with high environmental performance levels greater flexibility in spending  their grant funds.
Financial mechanisms also vary by their degree of risk or risk sharing  between the grantor and the grantee. Grantee risk increases as the amount  of money tied to performance increases. For example, bonuses—money  awarded over and above the base grant amount—represent the least risk,  while an outcome-based milestone payment plan where the entire grant  award is based on performance represents much higher financial risk to  the grantee. Nonfinancial actions, such as altering flexibility or oversight,  would be relatively risk neutral.
section_title: Accountability Provisions Can Be Employed at Different Phases of the Grant Life Cycle
paragraphs: Accountability mechanisms can be used in different phases of the grant  life cycle by different actors, including Congress, granting agencies, and  grantees themselves, and the lessons learned from one grant cycle can be  used to improve a performance accountability mechanism in the next (see  fig. 1). For example, when reauthorizing the CSE program, Congress  revised the original CSE incentive payments, which were solely based on  cost efficiency, to create an incentive program tied to performance  measures that reflect CSE’s five key goals: (1) paternity establishment,   (2) order establishment, (3) current collections, (4) collection of payments  in arrears, and (5) cost-effectiveness (design/redesign phase).
In contrast, performance measures and targets for Perkins III are created  during the implementation phase. Specifically, each state is required by  law to create its own performance measures linked to four core indicators:  (1) student attainment of challenging state-established academic and  vocational technical skill proficiencies; (2) student attainment of  secondary diploma or postsecondary degree or credential; (3) student  placement in employment, pursuit of further education, or both; and   (4) student participation in and completion of vocational technical  education programs that lead to nontraditional training and employment.  The Department of Education periodically negotiates the performance  targets for each state measure (postaward phase).
The use of past performance can inform and improve the recipient  selection process (application phase). Specifically, the Florida Department  of Children and Families has reported considerable success using past  performance in recipient selection—contractors that do not meet their  performance measures and standards are ineligible to be awarded future  contracts.
Other mechanisms, such as altered flexibility or oversight, can be used by  the granting agency—or even the grantee—to encourage improved  performance during the term of the award (postaward phase). For  example, according to the literature we reviewed, Minnesota’s Department  of Human Services Refugee Services Section increases its oversight of  local agencies if their performance drops below 80 percent on their key  performance measures, including job placement rates, which nearly  doubled over 5 fiscal years.
Importantly, grantees can also use these provisions to extend  accountability to subgrantees and contractors. This is significant because  many federal grants are ultimately passed through states to subgrantees.  Some accountability provisions, such as public award and recognition, can  even be employed by stakeholders or interested parties. For example, the  National Association for State Directors of Career and Technical  Education Consortium annually recognizes high-performing career and  technical administrators and teachers and provides opportunities to share  lessons learned and best practices.
Even when performance accountability provisions are absent from or  limited by a grant’s legislation, agencies and grantees may still be able to  include these types of provisions in the terms and conditions of the grants  or subgrants or in contracts as long as the authorizing legislation does not  specifically prohibit their use. OMB Circular A-110 provides that for grants  awarded to nonprofits, a number of accountability mechanisms may be  used—including withholding payments, termination of award, and “other  remedies that may be legally available”—if the grantee materially fails to  comply with the terms and conditions of the award. These terms and  conditions can be specified in federal statute, regulation, assurance,  applications, or the notice of award. For grants to state and local  governments, however, OMB Circular A-102 contains no detailed  accountability provisions and defers to the requirements specified in the  authorizing legislation.
Various authorities govern the use of accountability provisions (see table  2). Provisions set by Congress, such as increased flexibility in the form of  waivers from statutory restrictions, are generally laid out in authorization  or appropriations legislation. As stated earlier, granting agencies can  include accountability provisions in regulations, grant announcements, the  request for proposal, and the notice of award. Grantees (and subgrantees)  can use accountability provisions, such as formal recognition, to improve  their performance internally. For example, the CSE program in  Montgomery County, Pennsylvania, recognizes performance-improving  suggestions from individual employees by publicly praising and inducting  them into the office’s “all-star team.” These employees also receive a T- shirt with a picture of a stork—the program’s mascot—with the program’s  motto Striving Toward Optimizing our Resources for Kids.
section_title: Accountability Mechanisms Can Be Tailored to Specific Situations
paragraphs: Selecting appropriate performance measures and linking them to  performance accountability mechanisms is not a one-size-fits-all process;  rather, accountability provisions are tailored to reflect the program’s  characteristics. In addition to the range of accountability mechanisms  available, we found a number of ways mechanisms were tailored and  combined to reflect a variety of circumstances. Table 3 describes how  measures and mechanisms can be designed and triggered to either reward  or penalize performance.
For example, to encourage its contractor, ProFac, to cut costs while  maintaining a high-level of performance, ORC modified a basic financial  incentive to include a share-in-savings feature. ProFac is eligible to share a  portion of the savings if it spends less than its yearly budget—often  referred to as share in savings. To ensure that ProFac does not cut costs to  the detriment of high performance, ORC also requires that ProFac achieve  a performance rating of 80 percent or higher to share in these cost savings.
The CSE performance measures are an example of a “step up” provision— for each increasing performance percentage interval there is a  corresponding increase in the incentive percentage paid. Each time a state  moves to the next highest interval, it receives a higher percentage of the  incentive for that measure. Conversely, the alternative penalty procedure  for failure to implement a statewide child support data processing system  acts as a “step down” mechanism. For each year the state fails to  implement such a system, but shows a good faith effort to attempt to do  so, the state will be penalized at increasing intervals—during the first year  of noncompliance, the state will receive a 4 percent penalty, the second  year an 8 percent penalty, the third year a 16 percent penalty, and so on.
section_title: Strategies Support Successful Selection, Design, and Implementation of Performance Accountability Mechanisms
paragraphs: Collectively, five key strategies appear to facilitate the effective selection,  design, and implementation of performance accountability mechanisms.  These strategies are    ensure mechanisms are of sufficient value,    periodically renegotiate and revise mechanisms,    ensure appropriate measurement selection and usage,    ensure grantor and grantee technical capacity, and  implement system in stages.
In addition to these strategies, we noted extensive use of partnerships and  collaborations and regular and effective oversight and feedback, which  appeared critical to the success of accountability provisions in a third- party environment. We have previously reported that these practices are  often associated with high-performing organizations and organizations  that effectively used performance information to manage.
subsections: section_title: Ensure Mechanisms Are of Sufficient Value
paragraphs: There are a number of factors to consider when designing accountability  mechanisms that help to ensure the mechanisms are of sufficient value  and motivate performance improvement. Ensuring sufficient value  requires that    both the grantor and grantee are able to determine the value of the  rewards and penalties and the cost of improved performance—be they  financial or nonfinancial—and provide a meaningful return to both the  grantor and the grantee and  rewards or penalties should be consistently applied to maintain the value  of the mechanisms to both the grantor and grantee.
subsections: section_title: Understand the Value of Performance
paragraphs: According to the literature we reviewed, both the grantor and grantee  should understand what a particular level of performance is worth to them  and what it will cost them to achieve that level of performance. When the  value of performance is not properly identified, funds could be wasted and  grantees may not respond to the mechanism. For example, we found one  case where the contracting agency offered and ultimately paid a $250,000  bonus to a contractor for completing a pipeline 2-1/2 months earlier than  scheduled. However, because the contracting agency did not actually need  the pipeline to be completed for several years after the original contractual  deadline, the contractor paid $250,000 for a level of performance it did not  need. Although the recipient responded to the incentive, the contracting  agency did not properly calculate the value of the performance  improvement to the agency, resulting in wasted funds.
For a grantor, considering how accountability provisions support its  strategic priorities can assist in determining the value of performance. The  size of the associated rewards and penalties should be commensurate with  the priority. For example, successful pay-for-performance programs  reserve large rewards for achieving an organization’s most important  priorities, or those that lead to large benefits, and provide smaller  incentives for achieving goals that reap smaller benefits or are of lesser  importance. For example, in the health care field, for certain conditions  such as heart attack or stroke, delays in administering appropriate therapy  greatly increase the risk of mortality and disability. Therefore, the  incentives to treat these conditions quickly and appropriately should be  larger than the incentives for other practices that should be encouraged  yet produce fewer direct effects on mortality and illness, such as avoiding  the use of ineffective antibiotics to treat the common cold.
Based on our literature review, it appears that insufficiently valued  incentives are one of the main reasons that accountability provisions fail.
When an incentive is of sufficient value, the expected return outweighs the  expected risk, and recipients are motivated to pursue the performance  improvement. From 1975 through 1997 the CSE program included an  incentive program that focused on cost-effectiveness. States were  guaranteed an “incentive payment” from 6 to 10 percent of their total  collections. In practice, the 4 percentage point difference between the  minimum and maximum payment was reportedly not large enough to  motivate states to increase collections enough to earn the 10 percent  bonus. The new incentive system, established by the Child Support  Performance and Incentive Act of 1998 only provides incentive payments  to states that meet one or more of the act’s five outcome-based  performance goals and associated targets, and penalizes states that fall  below threshold levels in certain areas. A review of the new incentive  system in a sample of nine states found that the median score on each of  the five performance measures increased from fiscal years 2000 to 2002,  the time period that the incentive system was implemented.
Motivating grantees to work toward federal outcomes is particularly  challenging in grants where the federal investment is relatively small.  Officials at Arizona’s Department of Adult Education, Career and  Technical Education Division, told us that state funds in joint  technological education districts outweighed federal funds for career and  technical education (CTE) programs by more than four to one, and some  districts did not want to accept federal CTE funds because, in their view,  complying with the federal performance requirements was not worth the  amount of funds they would receive. In order to ensure that the financial  value of the Perkins III grants was large enough to motivate districts to  meet the Perkins III reporting and performance requirements, the Arizona  Career and Technical Division requires districts be in Perkins III  compliance in order to receive CTE-related state funds, thereby creating a  large incentive for local school districts to comply with the Perkins III  requirements. Indeed, one district we spoke with lost Perkins III funding,  but it was not until the state linked Perkins III compliance to state funding,  and the district lost the rest of its CTE funding from the state, that the  district started to make significant improvement toward meeting Perkins  III requirements.
In addition, the grantor and grantee should understand the trade-off  between the financial risk—the possibility performance will not improve  sufficiently despite the resource investment—and the potential return— what will be gained if performance goals are met or exceeded—in order to  determine whether to pursue any particular performance improvement.  Accountability provisions that contain financial incentives and sanctions  can shift risk between the grantor and grantee. That is, the more the grant  award depends on performance, the greater the financial risk to the  grantees: if they invest but do not perform sufficiently, they do not get  paid. Conversely, in grants with limited or no performance accountability  provisions, the grantor bears the bulk of the financial risk, since the  grantee would receive the grant funds regardless of the results achieved.
section_title: Ensure Effective Distribution
paragraphs: The ability of a performance accountability mechanism to influence  performance also depends on the effective distribution of organizational  rewards and penalties to individuals within the organization who are  directly responsible for the desired performance. For example, Glendale  Union School District officials provide significant financial incentives to  every school employee with whom a student has contact, including  teachers, administrative staff, and other support staff—including the  maintenance staff and bus drivers. The district’s philosophy is that all  employees influence the school’s atmosphere and academic achievement  and therefore contribute to any success it enjoys. Incentive funds are  distributed based on a school’s performance on 13 academic, involvement,  and satisfaction-related measures. According to a district official, the  program, started 5 years ago, has increased camaraderie and collaboration  among school employees, which the official said has contributed to  academic improvement.
In Pennsylvania, the state passes along a portion of the state-earned  federal incentive payments to the counties, according to each county’s  proportionate share of the aggregate state CSE expenditures and to reflect  its relative score for each performance measure, following the  performance targets defined in legislation. Pennsylvania codifies the  performance expectations and incentive payment procedures through  cooperative agreements with each county.
section_title: Execute Mechanisms Consistently
paragraphs: The grantor must execute the mechanisms consistently and as designed to  preserve the value of the mechanisms and to avoid introducing  unnecessary risk. For example, if rewards are not paid as promised the  grantee could learn that its additional efforts are not worth the cost—or  risk—and may not make the additional effort to improve performance.  Similarly, if rewards are paid indiscriminately or if penalties are not levied  as expected, the grantee could learn that no additional effort or  investment is required in order to benefit. In both cases, the system breaks  down and the intended value of the accountability provision is lost.
We have reported on an agency with the authority to levy penalties for  poor performance that resisted doing so. For example, the Federal Transit  Administration (FTA) has several enforcement tools to deal with grantees’  noncompliance, including warning letters, suspension of funds, and grant  termination. However, traditionally, FTA had been reluctant to use these  tools to enforce compliance, opting instead to work with grantees in an  effort to continually promote transit development. Reviews also showed  that FTA’s oversight was superficial and inconsistent and that FTA seldom  used its enforcement authority to compel grantees to correct weaknesses,  even those that were long-standing. Consequently, federal dollars had been  placed at risk. However, in response to our 1992 report, FTA established a  new enforcement policy, developed detailed guidance on carrying out  enforcement actions, and has since demonstrated a greater willingness to  use these actions against grantees that do not comply with federal transit  requirements. The Department of Defense (DOD), on the other hand, has  paid billions in incentive and award fees for only “acceptable, average,  expected, good, or satisfactory” performance. Despite paying billions in  fees, DOD has little evidence to support its belief that these fees improve  contractor performance and acquisition outcomes. The department has  not compiled data, conducted analyses, or developed performance  measures to evaluate the effectiveness of award and incentive fees. Using  accountability mechanisms in this manner undermines their effectiveness  as a motivational tool and marginalizes their use in holding grantees and  contractors accountable for outcome-based results.
section_title: Periodically Renegotiate and Revise Mechanisms and Measures
paragraphs: Organizations need to allow for and use the flexibility to revise, update, or  improve performance accountability mechanisms in order to respond to  changing needs. In the literature we reviewed, we found a number of  reasons for why accountability provisions may need to be revised. For  example, unintended consequences associated with performance  measures may be discovered only after full implementation.  Organizational priorities may change. Technology may be introduced that  substantially alters performance expectations. In addition, expectations  that were previously considered stretch goals can become the norm over  time—for example, as productivity gains are realized rewarding such  performance may no longer make sense. Finally, efforts to reevaluate and  revise should consider whether established accountability provisions are  still effective at motivating performance improvements.
For example, the DMA/MBHP contract demonstrates a situation in which  the entire accountability system experienced a revision to adjust to  contract progression. Incentives in this contract were initially designed to  motivate operational performance, such as processing time for billing, and  performance targets were revised upward each year as performance  improved. This upward revision helped ensure that performance  continued to improve. Once MBHP’s performance reached the highest  levels of industry performance in these areas, further improvements were  no longer a priority.
As a result, DMA and MBHP used the annual review to revise the incentive  system from motivating operational improvement to completing projects  designed to improve performance in areas that would add value to the  services MBHP provides, such as a project on providing behavioral health  assistance to the homeless.
There are a number of ways to accommodate the need for periodic  revision. For example, congressional amendments to or reauthorizations  of grant programs allow policymakers the opportunity to revisit and  modify existing provisions and to add flexibility for agencies that can lead  to improved effectiveness. Agencies can include renegotiation and revision  policies in regulations, guidance, and the terms and conditions of a grant  award. Providing for periodic revision may be particularly important  where performance measures are specified in legislation, because agency  flexibility to respond to changing needs is significantly reduced. For  example, as we have discussed, the Child Support Performance and  Incentive Act of 1998 specifies the five performance measures, the  performance targets, and the percentage of incentive payments that states  can earn for performance. Initially, states made changes and saw  improvement in these areas. Recently, both state and federal program  officials have expressed concern about the long-term sustainability of such  aggressive targets. For example, program officials said that even states  that have in the past met the 90 percent performance target for the  paternity measure are concerned because more recent annual rates have  dropped back down closer to 80 percent. According to officials, states  initially conducted an extensive caseload cleanup to improve performance  on the five incentive measures when the incentive program was enacted in  1998, and much of the backlog of cases that could be addressed relatively  easily has been. However, since the measures and performance targets are  legislatively defined and the CSE program is permanently authorized, the  agency does not currently have the flexibility to revise the measures or  performance targets.
In contrast, state agencies, in negotiation with the Department of  Education, can periodically revise their Perkins III CTE performance  measures and targets during annual negotiations of their state plans. At  program introduction, program targets are set through the negotiation  process between states and OVAE. From this process, performance targets  negotiated initially reflect a realistic level of what states can actually  produce. Next, through annual application updates, the legislation allows  renegotiation of performance levels with states. Among other factors,  OVAE officials attributed the program’s success to this ongoing ability to  renegotiate and revise the program’s measures. Although the Perkins III  legislation is similar to the Child Support Performance and Incentive Act  of 1998, in that Perkins defines the four core indicators tied to the  performance measures used in the incentive program, it provides  flexibility that the Child Support Performance and Incentive Act of 1998  lacks. The flexibility to revise or update the performance measures is built  into the Perkins III legislation.
Accountability systems by their very nature assume that performance can  be improved. However, performance improvements depend on adequate  time for and ability of participants to learn from prior actions and use  what they have learned to improve performance from one period to the  next. Depending upon the complexity of the task, this process can take  many cycles. Therefore, accountability systems should not be abandoned  prematurely; rather, they should be assessed, revised, and improved.
section_title: Ensure Appropriate Measurement Selection and Usage
paragraphs: Selecting and using appropriate types of performance measures is  important to the effective use of accountability mechanisms. We have  previously reported on general attributes of good performance measures,  noting that measures should be linked to agency goals and missions; be  clearly stated; include measurable targets; and be objective, reliable, and  balanced. Specifically, we found four of these characteristics that  highlight key features of performance measures that can help ensure the  successful linking of performance measures and rewards and penalties:  the performance being measured should be within the recipient’s sphere  of influence,  the performance measures should be suitable to the mechanism evaluation  cycle, and  the performance measures and performance data should be tested.
subsections: section_title: Performance Should Be within Recipient’s Ability and Influence
paragraphs: Performance measures tied to rewards and penalties should represent  performance that can be sufficiently influenced by the grant recipient’s  actions. Absent this linkage, the grantee may have little motivation to  change behavior to improve performance, and the granting agency risks  wasting funds by either rewarding efforts that cannot reasonably be tied to  grantee behavior or penalizing a grantee for outcomes that even its best  efforts may not have prevented. For example, the Temporary Assistance to  Needy Families (TANF) bonus payments rewarded states for reducing  out-of-wedlock births. Several studies report, however, that there does not  appear to be a link between the existence of these programs, or increases  in efforts to deliver program services, and the TANF bonus payment. Many  state officials perceive the outcome measure as inappropriate, relatively  difficult to influence, or both, and discourage attempts to do so. According  to one study, several states reported that they did not compete or did not  continue to compete for the bonus funds because, among other reasons,  their actions would not sufficiently affect the out-of-wedlock birth rate;  therefore they directed their efforts to activities that were more directly  under their influence.
In another example, the Perkins III CTE program has a financial incentive  system that assesses state performance through performance measures  that support its four core indicators—one of which encourages  participation in and completion of programs leading to nontraditional  employment. State and local officials in Arizona said their ability to affect  performance for this indicator is very limited. They told us that although  they have tried to address the barriers to nontraditional employment, they  found that cultural and demographic influences have limited their ability  to improve performance every year. Because performance has not  improved as a result of their efforts, they focus most of their energy on  efforts to improve performance in the other three core indicator areas,  which reflect performance that is more directly under their control.
section_title: Measures Should Be Suitable to the Mechanism Cycle
paragraphs: Measures should assess performance that can be observed, achieved, and  reported frequently enough to inform the use of awards and penalties on a  timely basis. For example, an annual reward or penalty should be tied to a  measure that is also assessed annually.
ORC uses performance measures that can be assessed in a relatively short  period of time and that support program outcomes. ORC holds back 10  percent of ProFac’s management fee each month. Each quarter, ProFac  has an opportunity to earn the holdback on the basis of its performance  during the prior quarter on 30 KPIs. For example, 1 of the quarterly  indicators tied to its overall customer service objective is the “overall  customer satisfaction rate with project delivery.” The quarterly assessment  is based on performance information gathered through customer  satisfaction surveys of local managers and facility management contacts  for all alteration projects, capital repairs, or both completed in the  previous quarter. Both ORC and ProFac officials credit the frequency of  evaluation for motivating ProFac to maintain high performance  throughout the year.
OVAE uses the timing of its grant funding distribution cycle to its  advantage in order to motivate states to meet federal performance  accountability requirements. OVAE disburses grant funds in two pieces: a  small portion in July and the remainder in October. States that did not  provide complete, timely performance data, or missed their performance  targets in the prior year, may have “conditions” put on the July portion of  the funding; if conditions are not met during that quarter, the October  funding is withheld.
section_title: Measures and Data Should Be Tested
paragraphs: Performance measures that trigger accountability mechanisms should be  well functioning and time tested before they are linked to rewards and  penalties to minimize the potential for unintended consequences. Although  our literature review did not specify how long this could take, one study in  our review noted that many leading companies use and test their  measurement systems for years before linking them to accountability  provisions.
Performance data should also be tested to make sure they are credible,  reliable, and valid. Absent these attributes, organizations lack the basis for  sound decisions about rewards and penalties. Data quality is so critical to  performance accountability and oversight of grants that several  organizations use it as the principal performance measure for  performance-based funding. Pinellas County, Florida, alters the case  funding rates paid to its ambulatory service contractor based on data  quality. This “altered funding rate” provision links case reimbursement  rates directly to data quality. For example, data that are incomplete,  illegible, inaccurate, altered, or lacking evidence of medical necessity— and limit the county’s ability to claim for payment or use its data  processing procedures—result in reduced reimbursements to the  ambulatory service contractor for the affected cases. Pinellas County  reports that as a result, ongoing data quality issues are minimal. In another  example, the Child Support Performance and Incentive Act of 1998  prohibits the payment of financial incentives to states for performance in  program areas where state data have failed an annual data reliability test.  This requirement ensures that incentive payments are based on reliable  and complete performance information.
section_title: Ensure Grantor and Grantee Technical Capacity
paragraphs: Grantor and grantee capacity—specifically, the knowledge about  performance accountability mechanisms and the ability to effectively  implement them—is critical to the effectiveness of performance  accountability systems. For example, when the Air Force implemented its  performance-based contracting program, it found that employee training  focusing on how the performance-based aspects of the contracts should  work were most critical. Specifically, practices such as providing a step- by-step approach to the process that outlined who should be involved at  each step, how much of their time and effort would be required at each  step, and what their specific roles and responsibilities would be were  critical to employees understanding what was needed to create  mechanisms to improve performance.
In addition, federal CSE staff in Region III provide a “Child Support  Enforcement Incentives 101” presentation to state and county CSE staff  throughout the region to explain how the performance measures and  incentive payments work. This training presentation is tailored to the  experience of CSE staff and the demographics of the county, state, or both  (large urban, rural, large interstate caseloads, etc.) but strives to provide a  clear and consistent message: the everyday activities of CSE staff directly  affect the amount of child support available to children and their families,  and drive the amount of incentive payments the county specifically, and  the state in general, earns. The presentation includes interactive exercises  to show how each employee’s casework feeds into outcome-based  program results.
section_title: Implement System in Stages
paragraphs: Organizations go through a number of stages designing, testing, and  revising measurement systems before linking them to accountability  mechanisms. This longer, phased implementation allows organizations to  ensure the system is effectively designed before tying it to rewards and  penalties. During these stages, organizations can conduct pilot tests,  create financial models, and conduct behavioral modeling to understand  and modify a system prior to full implementation. For example, according  to one expert, the Tennessee Valley Authority completes a “readiness test,”  an assessment of measurement effectiveness and suitability, before  allowing pay for performance or similar financial incentive systems to be  pinned to that measure. This helps avoid unintended consequences  associated with poorly designed measures. Phased implementation also  allows organizations to adjust to new demands on their time and  resources; set up or modify data collection systems; and ensure the  credibility, validity, and reliability of the data before they are used to  measure performance.
For example, the CSE incentive program was implemented in three stages  to allow states to learn about the new incentives and performance  measures. The five performance areas attached to incentives were  developed and legislatively defined in 1998. In 1999, the new data  measures were used by the states and audited for data reliability for the  first time. In year one, one-third of the total incentive funds were allocated  based on the new formula and the remaining funds were allocated based  on the old system. In year two, two-thirds of the funding was allocated  using the new system, and the remaining funds were allocated based on  the old system. In year three, all incentive funding was allocated according  to the new formula.
section_title: Collaboration and Oversight Also Key to Success
paragraphs: In addition to these strategies described above, we saw extensive use of  partnerships and collaborations and regular and effective oversight and  feedback. We have previously reported that these practices are often  associated with high-performing organizations and organizations that  effectively used performance information to manage.
Designing and implementing accountability provisions in a collaborative  environment can help develop and encourage buy-in and support and lead  to improvements. For example, Arizona state and local CTE officials said  the state’s focus has shifted from a compliance-focused “audit,” ensuring  performance data were properly collected and reported, to a true  partnership in which state and local officials work together to identify and  replicate successes, find solutions to challenges, and thereby improve  performance. State CTE staff spend several days each year meeting with  local CTE officials and providing regular assistance through on-site  technical assistance teams, phone calls, and e-mails.
Oversight and feedback are critical to creating and sustaining effective  performance accountability provisions. We have previously reported on  oversight practices, noting specifically the value of feedback provided  through performance monitoring plans and tools such as site visits,  document reviews, and evaluations. For example, OVAE employs a  number of tools to provide feedback and assistance to states implementing  the Perkins III vocational education program. Among these tools are    establishing state guidance that outlines how to meet the Perkins III    developing a peer-to-peer mentorship program among states and with  OVAE to share experiences and good practices,    conducting monthly conference calls with state directors and data  specialists to discuss challenges and solutions to data collection and  quality,    offering data quality “institutes” and conferences to share performance  measurement and data quality and collection practices, and    providing technical assistance to states.
An OVAE official said providing these types of oversight and feedback  activities generated ideas and discussion to help states improve their  performance; the state CTE officials with whom we spoke agreed.
section_title: Various Opportunities Exist at the Federal Level to Enhance Performance Accountability in Grants
paragraphs: The experiences with and strategies related to federal grant accountability  provisions described in this report suggest a number of opportunities for  Congress and the executive branch to improve the design and  implementation of performance accountability mechanisms. First, a  results-focused design can help encourage performance accountability in  general and specifically provide for—or at least not prohibit—the use of  accountability mechanisms to encourage desired behavior. In addition, the  use of national program evaluation studies and research and  demonstration grants can provide valuable information to assist in agency  and congressional oversight of and knowledge about accountability  mechanisms. Because credible performance information and performance  measures form the basis for well-functioning accountability provisions, it  remains critical for Congress and the executive branch to continue to  encourage their development and use. Finally, OMB and agencies can  commit to sharing good practices and lessons learned from experiences  with performance accountability provisions in federal grants—an efficient  and effective way to increase grantor and grantee knowledge,  understanding, and use of these provisions.
subsections: section_title: A Results-Focused Design Encourages Performance Accountability
paragraphs: Considering grant design features and their implications for grantee  flexibility and accountability can help policymakers provide for  appropriate accountability provisions, whatever type of grant design is  selected. We have previously reported that policy options reflected in  grant design collectively establish (1) the degree of flexibility afforded to  states or localities; (2) the relevance of performance objectives for grantee  accountability; (3) whether accountability for performance rests at the  federal, state, or local level; and (4) prospects for measuring performance  through grantee reporting and oversight. Under a results-oriented  approach, federal policymakers would specify national goals and  objectives in statute, enact a process for establishing them, or adopt some  combination of the two. As a result, when designing or reauthorizing  grants, it is important to consider questions like the following:  Is there a need for national performance objectives in this policy area? If  so, grantees may be required to use uniform performance measures—as in  the CSE program—to gauge progress. This allows for comparisons across  grantees, and the supporting performance data collected from grantees  have the advantage of being program specific. However, uniform activities,  objectives, and measures may not exist or may not be desirable, especially  under flexible grant program designs. In these cases, Congress may  instead decide to allow grantees to establish their own program objectives.  For example, the Child Care and Development Block Grant requires states  to certify that they have requirements in effect to protect the health and  safety of children whose child care is subsidized by the block grant. These  requirements must cover the areas of preventing and controlling for  infectious diseases, physical premise safety, and health and safety training.  However, the specificity and stringency of these requirements and the  manner in which they are enforced is left to the states. The Perkins III  legislation outlines several performance areas and requires states to  determine the measures they will use to measures progress in these  statutorily defined areas. Performance targets for these measures are  negotiated with OVAE. In these cases, the federal role in monitoring the  grants is generally limited to collecting information on state and local  program efforts and accomplishments as well as evaluating and  disseminating information on best practices. Another option is to grant  temporary exemptions (waivers) from certain federal program  requirements to grantees that demonstrate that the flexibility granted can  lead to performance improvements. For example, Oregon Option is an  intergovernmental partnership that seeks to improve performance on  benchmarks for a broad variety of initiatives, including childhood  immunization, employment for the disabled, wild salmon recovery,  juvenile justice, welfare reform, and child nutrition, by waiving  administrative rules or seeking statutory change.  In all cases, what accountability provisions are needed to support  attainment of national performance objectives? These might include  constraints on activities and funds distribution or operational objectives,  standards, and criteria for performance. These can be set for the program  as a whole or delegated to the level of government responsible for  program management. Additional considerations are as follows: What data  are needed for grantee accountability, and is it feasible to collect these  data from providers? Is it possible to collect data at the project level? Will  the contribution of federal funds be distinguishable from state, local, and  private funds? If the answer to several of these questions is no, is  additional information needed for program oversight? If so, how will such  information be gathered and reported? The answers to questions such as  these provide the basis for setting grantee reporting requirements.
section_title: Careful Use of National Program Evaluation Studies and Research and Demonstration Grants Can Help Assess Mechanism Performance
paragraphs: Congress has a number of opportunities to conduct oversight, such as  when it establishes or reauthorizes a new program, during the annual  appropriations process, and during hearings focused on program and  agency operations. Providing for—or at a minimum, not prohibiting— performance accountability mechanisms can provide timely, targeted  performance information and help policymakers ensure that federal grants  focus on their goals, providing another basis for congressional oversight.
National program evaluations have the potential to answer questions  about both overall program performance as well as the effectiveness of  performance accountability mechanisms, in terms of their implementation,  outcomes, impacts, and cost-effectiveness. However, national  programwide evaluations are expensive in terms of dollars and time and  frequently require capacities and resources beyond those provided for  program management. Also, while evaluations of multiple sites provide  valuable information, programwide evaluation data are typically periodic  and often cover too few sites to support national estimates of  performance. In these cases, research and demonstration projects often  can provide better information on the effectiveness of various service  delivery methods and approaches. Knowledge to support effective practice  is well established in some subject areas and can be incorporated into  program provisions (such as service standards) or in companion technical  assistance or knowledge dissemination programs.
section_title: Encourage Development and Use of Credible Performance Information and Performance Measures
paragraphs: As we discussed earlier, performance accountability provisions rely on a  supply of credible, reliable, and valid data and high-quality performance  measures. We found organizations that recognizing the importance of data  quality, tied incentives to increasing the supply of this type of information.  Unfortunately, as our work on PART and GPRA implementation shows,  the credibility of performance data has been a long-standing weakness.  OMB, through its development and use of PART, has provided agencies  with a powerful incentive for improving data quality and availability.  However, improving the supply of performance information is in and of  itself insufficient to sustain performance management and achieve real  improvements in management and program results. Rather, it needs to be  accompanied by a demand for and use of that information by decision  makers and managers alike. Key stakeholder outreach and involvement is  critical to building demand and, therefore, success. Lack of consensus by a  community of interested parties on goals and measures and the way that  they are presented can detract from the credibility of performance  information and, subsequently, its use. While congressional buy-in is  critical to sustain any major management initiative, it is especially  important for performance accountability given Congress’s constitutional  role in setting national priorities and allocating the resources to achieve  them. Recognizing this, policymakers could use incentives to encourage  program partners to agree on performance measures and targets against  which performance will be judged.
section_title: Share Good Practices and Lessons Learned
paragraphs: We and others have frequently reported on the benefits of sharing  promising practices and lessons learned to promote performance  accountability in general in federal programs and program partners. We  believe sharing good practices related to the effective design and  implementation of performance accountability mechanisms carries similar  benefits. As noted earlier, some state and local agencies’ programs have  used this type of information sharing among themselves and their grantees  and contractors as a means of performance improvement.
OMB, as the focal point for overall management in the executive branch,  plays a key role in promoting performance improvement in federal  programs and has developed or contributed to a number of tools to share  information and encourage improvements to federal grants and program  performance. For example, www.grants.gov includes information on grant  opportunities, resources to assist in writing grant proposals, and a  newsletter highlighting recent grant success stories, and www.results.gov  has information on best practices related to the President’s Management  Agenda initiatives—one of which is Budget and Performance Integration  (BPI). Successful implementation of BPI depends significantly on federal  agencies’ ability to ensure federal program partners work toward program  goals and are held accountable for results. Expectmore.gov provides  information on PART assessments and improvement plans; these  assessments consider, among other things, whether the agency regularly  collects timely and credible performance information to manage its  programs, and whether the performance measurements are used to  increase accountability. OMB’s own Web site also contains information on  and examples of what it considers to be high-quality PART performance  measures; discussion papers on measurement topics, such as how to  effectively measure what you are trying to prevent; and strategies to  address some of the challenges of measuring research and development  programs.
OMB hosts a number of standing work groups and committees— comprising agency and OMB staff—to address important grant-related  issues, all of which could accommodate a more specific focus on grants  accountability provisions. For example, OMB’s Chief Financial Officer’s  Council has a standing grants policy committee that focuses on grant  application and reporting streamlining. Agency BPI leads meet monthly  and recently developed a subgroup to share lessons learned related to  efficiency measures that balance effectiveness, quality, and cost. They also  discuss strategies to address the challenges of efficiency measures in the  grant context and to develop additional guidance for agencies in this area.
In addition, OMB hosted a Block and Formula Grant workshop in October  2005 for federal officials aimed at identifying and sharing best practices in  grants management and performance measurement. OMB staff agreed that  the workshop was a valuable, efficient, and effective way to share  information and lessons learned and that collectively the participants  increased their knowledge and understanding of ways to enhance grant  performance. They also noted that the real difficulty comes in “what to do  next,” in other words, implementing the strategies gleaned from these  sessions.
OMB staff told us that focusing specifically on performance accountability  provisions in grants is necessary and useful, but that to date, they have  focused their governmentwide efforts primarily on encouraging and  enhancing agency capacity to develop high-quality, results-based program  performance measures since improving the quality of measures and data  necessarily precedes tying them to accountability provisions. The Block  and Formula Grant workshop addressed issues of measurement and  accountability, and several block grant programs have been working to  strengthen grantee accountability.
section_title: Conclusions
paragraphs: As the challenges of the 21st century grow, it will become increasingly  important for Congress, OMB, and executive agencies to consider how the  federal government can maximize performance and results. This will be  particularly important for federal grant program managers, given the  significant amount of federal resources invested in these tools. Because  many national objectives can only be achieved through state, local, and  nongovernmental organizations, enhancing performance accountability  below the federal level is equally important. In this report, we identify a  variety of accountability mechanisms as well as key strategies to enhance  their use. Collectively, these can help enhance and sustain performance  accountability in grants at all levels of government.
As the cases we described illustrate, rewards and penalties are  fundamental tools to help drive and motivate desired behaviors, but  performance accountability mechanisms are not one size fits all; there is  no universal transferable mechanism applicable to all programs. The  specific mechanisms used by agencies and programs and highlighted  throughout this report may not be universally adopted by other federal  agencies and programs seeking to improve their own programs.  Nevertheless, many can be tailored to specific grant programs, and the key  strategies can be adapted to address the specific accountability challenges  each agency faces.
Like all successful change initiatives, the progress currently under way to  move from traditional fiscal accountability in grants to greater  accountability for performance will take time; accountability provisions— and the performance measures associated with them—can take many  years to mature. Although some federal programs are well on their way to  collecting and reporting on reliable, credible, and valid data that support  high-quality outcome goals agreed to by all program partners, many others  are still struggling with how to define appropriate outcome measures. It  will be critical to proceed thoughtfully and implement performance  accountability in phases, building in enough opportunities to learn from  mistakes and revise measures and mechanisms to reap the benefits of  performance management while minimizing perverse incentives and  unintended consequences.
As with all challenges, starting with small steps is often the best way  forward. Accountability provisions can be used to bring program partners  together to identify common ground. For example, programs that struggle  with defining appropriate outcome goals, measures, and targets may wish  to tie incentives to reaching agreement on them. Those that struggle with  poor data quality and data definitions could reward grantees for progress  in this area. Performance accountability—especially in the early stages— must be constructive, not punitive. Even if penalties are employed to  promote performance accountability, there should be a constructive,  collaborative approach to performance improvement that precedes them.  Tying performance to lower risk, nonfinancial mechanisms may at first be  more acceptable until performance measures have been time tested and  revised as needed and grantees have had time to collect the necessary data  to support the measures. Above all, a collaborative process that includes  Congress, the executive branch, and grantees will be critical to developing  successful performance accountability systems.
Accountability provisions assume that performance can be improved—but  this requires information sharing and feedback. OMB has a central role in  overseeing the performance and accountability in the federal government,  and has used its role to promote general results-oriented performance  measurement and management practices in federal grants through Web  sites, guidance, work groups, and workshops. Each of these tools and  strategies could be expanded on to specifically promote and encourage  performance accountability in federal grants, both among related federal  grant programs—programs that have a common purpose—and federal  grant types—such as categorical grants, block grants, and funding streams.  Sharing good practices and lessons learned and providing feedback on  performance are valuable practices that can leverage resources to enhance  knowledge and further performance accountability. Leading practices can  be shared within and among agencies, grant programs, grantees, and even  grant types. OMB recognized the value in sharing information on  performance accountability mechanisms, but has not yet focused on this  issue.
section_title: Recommendation for Executive Action
paragraphs: We are therefore recommending that the Director of OMB encourage and  assist federal agencies in working with the Congress to expand the  effective use of performance accountability mechanisms, focusing on the  practices in this report, when federal grant programs are being created or  reauthorized. We further recommend that OMB offer opportunities for  knowledge transfer among federal agencies and encourage agencies to  share leading practices and lessons learned in implementing grant  accountability mechanisms. Possible vehicles for the collection and  dissemination of this information include good practices guides and  workshops and Web sites such as results.gov, grants.gov, and  expectmore.gov.
section_title: Agency Comments
paragraphs: On August 22, 2006, we provided a draft of this report to the Director of  OMB and the Secretaries of Education and Health and Human Services.  We also provided relevant sections of a draft of this report to the grantees  and contractors highlighted in this report. We received technical  comments from all three agencies, which were incorporated as  appropriate. In addition, OMB agreed with our recommendation but  suggested we broaden it to address the role of federal agencies and  Congress in the grant design and reauthorization process. We agree, and  have amended our recommendation accordingly.
We are sending copies of this report to the Director of the Office of  Management and Budget, the Secretaries of Education and Health and  Human Services, and other interested parties. We will also make copies  available to others upon request. In addition, the report is available at no  charge on GAO’s Web site at http://www.gao.gov.
Please contact me on (202) 512-6543 or steinhardtb@gao.gov if you or your  staff have any questions about this report. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. Key contributors to this report are acknowledged in  appendix II.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this report were to identify (1) the challenges to  performance accountability in grants; (2) the kinds of mechanisms that are  being used to improve grant performance and how; and (3) given the  findings of questions 1 and 2, what strategies the federal government can  use to encourage the use of these mechanisms, as appropriate.
To meet the first and second objectives, we interviewed experts in grants  and performance management, including individuals from the following  organizations: the School of Public Policy at the University of Maryland,  the University of Central Florida, the John F. Kennedy School of  Government at Harvard University, the John C. Stennis Institute of  Government at Mississippi State University, the Public and International  Affairs Department at George Mason University, the Political Science  Department at the University of New Hampshire, Measurement  International, and the American Productivity and Quality Center.
Based on our literature review, we developed a coding scheme for  identifying (1) types of performance accountability mechanisms and   (2) strategies used to successfully design and implement these  mechanisms. We used these codes in a content analysis we conducted on a  subset of the documents we reviewed. We chose the documents for  content analysis based on the following criteria:    discussed accountability systems, mechanisms, or both, discussed general  practices that facilitated to the effective use of accountability mechanisms,  or provided case examples;    published in 1993 or later;   found in major electronic databases; and    published in the United States.
The content analysis was conducted by two analysts, with the second  analyst conducting a dependent review. Discrepancies in coding were  discussed and agreement reached between the two analysts. Our analysis  produced an inventory of performance accountability mechanism types  and five strategies used to facilitate the effective design and  implementation of performance accountability mechanisms. See the  bibliography for documents included in our review.
To illustrate the mechanisms and strategies identified through our content  analysis, we used relevant case examples found in the literature. To  further illustrate the mechanisms and design and implementation  strategies, we also selected four additional case illustrations—two federal  grant programs and two nonfederal contract cases. These four cases were  selected based on our literature review, interviews with experts, and  reviews of prior GAO work because they are good examples of where (1) a  performance mechanism was present and (2) there is reason to believe  that performance improved at least in part because of the mechanism.
To screen and develop the grant case illustrations, we interviewed regional  and headquarters federal agency officials and officials at county/local  offices. We also reviewed grant legislation, program guidance, and prior  studies. To develop contract case illustrations, we interviewed officials  both from the contracting agencies and the contractors and reviewed the  contract.
To address our third objective, we synthesized prior GAO work, and we  interviewed officials at the Office of Management and Budget.
We conducted our work from December 2005 through August 2006 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: Jackie Nowicki (Assistant Director) and Chelsa Gurkin (Senior Analyst-in- Charge) managed this assignment. David Bobruff, Katie Hamer, and Anne  Marie Morillon made significant contributions to all aspects of the work.  Kate France significantly contributed to the initial research and design of  the assignment. In addition, Tom Beall and Jay Smale provided  methodological assistance, Amy Rosewarne provided key assistance with  message development, and Donna Miller developed the report’s graphics.
section_title: Bibliography
paragraphs: American Productivity & Quality Center. Achieving Organizational  Excellence Through the Performance Measurement System. Houston:  1999.
American Productivity & Quality Center. Measure What Matters: Aligning  Performance Measures with Business Strategy. Houston: 1999.
Ashworth, Karl, and others. “When Welfare-to-Work Programs Seem to  Work Well: Explaining Why Riverside and Portland Shine So Brightly.”  Industrial & Labor Relations Review, vol. 59, iss. 1 (2005).
Ausink, John, and others. Implementing Performance-Based Services  Acquisition (PBSA): Perspectives from an Air Logistics Center and a  Product Center. A Documented Briefing prepared by the RAND  Corporation for the U.S. Air Force. 2002.
Ausink, John, Frank Camm, and Charles Cannon. Performance-Based  Contracting in the Air Force: A Report on Experiences in the Field. A  Documented Briefing prepared by the RAND Corporation for the U.S. Air  Force. 2001.
Baker, George. “Distortion and Risk in Optimal Incentive Contracts.” The  Journal of Human Resources, vol. 37, no. 4 (2002).
Banta, Trudy W., and others. “Performance Finding Comes of Age in  Tennessee.” The Journal of Higher Education, vol. 67, no. 1 (1996).
Conlon, Timothy. “Grants Management in the 21stCentury: Three  Innovative Policy Responses.” IBM Center for the Business of  Government: Financial Management Series. Washington, D.C.: October  2005.
Courty, Pascal, and Gerald Marschke. “Measuring Government  Performance: Lessons from a Federal Job-Training Program.” The  American Economic Review, vol. 82, no. 2 (1997).
Dodds, Dan R. “Performance Incentives Can Spark Greater Productivity.”  School Administrator, vol. 55, iss. 3 (1998).
Domestic Working Group: Grant Accountability Project. Guide to  Opportunities for Improving Grant Accountability. Washington, D.C.:  October 2005.
Dye, Jane Lawler, and Harriet B. Presser. “The State Bonus to Reward a  Decrease in ‘Illegitimacy’: Flawed Methods and Questionable Effects.”  Family Planning Perspectives, vol. 31, no. 3 (1999).
Eberts, Randall, Kevin Hollenbeck, and Joe Stone. “Teacher Performance  Incentives and Student Outcomes.” The Journal of Human Resources, vol.  37, no. 4 (2002).
Garber, Alan M. “Evidence-Based Guidelines as a Foundation for  Performance Incentives.” Health Affairs, vol. 24, no. 1 (2005).
Gordon, Stephen B. “Performance Incentive Contracting: Using the  Purchasing Process to Find Money Rather Than Spend It.” Government  Finance Review, August (1998).
Hatry, Harry P., and others. How Federal Programs Use Outcome  Information: Opportunities for Federal Managers. Washington, D.C.: IBM  Center for the Business of Government, May 2003.
Hildebrandt, Gregory G. “The Use of Performance Incentives in DOD  Contracting.” Acquisition Review Quarterly, Spring (1998).
Johnston, Jocelyn M., and Barbara S. Romzek. “State Social Services  Contracting: Exploring the Determinants of Effective Contract  Accountability.” Public Administration Review, vol. 65, iss. 4 (2005).
Kathuria, Sandeep. “An Overview of Share-in Savings Contracting.”  Contract Management, vol. 45, no. 11 (2005).
King, Donald C., and Paul D. Tolchinsky. “Do Goals Mediate the Effects of  Incentives on Performance?” Academy of Management Review, vol. 5, no.  3 (1980).
Lane, Nancy E. “Performance Incentives in The Massachusetts Behavioral  Health Program.” Administration and Policy in Mental Health, vol. 32,  no. 4 (2005).
Marcus, Alfred A., Barry M. Mitnick, and Kiran Verma. “Making Incentive  Systems Work: Incentive Regulation in the Nuclear Power Industry.”  Journal of Public Administration Research and Theory: J-PART, vol. 9,  no. 3 (1999).
Martin, Lawrence L. “Making Performance-Based Contracting Perform:  What the Federal Government Can Learn from State and Local  Governments.” IBM Endowment for the Business of Government: New  Ways to Manage Series. Washington, D.C.: 2002.
Martin, Lawrence L. Performance-Based Contracting (PBC) for Human  Services: A Review of the Literature. Orlando: Center for Community  Partnerships, University of Central Florida, 2003.
Metzenbaum, Shelley H. “Performance Accountability: The Five Building  Blocks and Six Essential Practices.” IBM Center for the Business of  Government: Managing for Performance and Results Series. Washington,  D.C.: 2006.
Rand Corporation. Organizational Improvement and Accountability:  Lessons for Education from Other Sectors. 1st ed. Santa Monica, Calif.:  2004.
Sammer, Joanne. “Making Incentive$ Pay.” Industry Week: Leadership in  Manufacturing. (Cleveland: Penton Media Inc., August 2002).  http://www.industryweek.com/ReadArticle.aspx?ArticleID=1121  (downloaded Mar. 9, 2006).
Tsimbinos, John M. “Are You Willing to Pay for Success?” Bottomline, vol.  7, no. 6 (1990).
U.S. Department of Defense, Deputy Under Secretary of Defense for  Acquisition Reform. Incentive Strategies for Defense Acquisitions.  Washington, D.C.: 2001.
U.S. Department of Energy, Office of Inspector General. Inspection of  Selected Aspects of the Office of River Protection Performance-Based  Incentive Program. Washington, D.C.: 2001.