Grants Management: Enhancing Performance Accountability Provisions Could Lead to Better Results
Released: 2006-09-29T00:00:00

Letter

Background

Grants, along with contracts and cooperative agreements, are tools used by the federal government to achieve national priorities via nonfederal parties, including state and local governments, educational institutions, and nonprofit organizations. Diverse in structure and purpose, grants can be generally classified as either categorical or block, with categorical grants allowing less recipient discretion than block grants. For example, the Community Services Block Grant provides funds to states and is sometimes passed to local agencies to support a variety of efforts that reduce poverty, revitalize low-income communities, and lead to self- sufficiency among low-income families and individuals, while giving the agencies broad discretion in how the funds can be spent. In practice, the “categorical” and “block” grant labels represent the ends of a continuum and overlap considerably in its middle range.

Grant funds may also be grouped by their method of allocating funds. Formula grants allocate funds based on distribution formulas prescribed by legislation or administrative regulation and often narrowly define the eligible recipients as state agencies. On the other hand, categorical grants are generally awarded on a competitive basis to applicants meeting broader eligibility requirements.

Despite substantial variation among grants, grants generally follow a similar life cycle and include announcement, application, award, postaward, and closeout phases. Once established through legislation, which may specify particular objectives and eligibility and other requirements, a grant program may be further defined by grantor agency requirements. For competitive grant programs, the public is notified of the grant opportunity announcement, and potential grantees must submit their applications for agency review. In the awards stage, the agency identifies successful applicants or legislatively defined grant recipients and awards funding. The postaward stage includes payment processing, agency monitoring, and grantee reporting, which may include financial and performance information. The closeout phase includes preparation of final reports, financial reconciliation, and any required accounting for property.

Traditionally, grant accountability has referred to legal or financial compliance. The Single Audit Act, for example, requires grantees to conduct an overall financial compliance audit to promote accountability. As such, at a minimum all grantees are held accountable for sound financial management and use of federal funds to support allowable activities. Beyond that, however, accountability for performance varies from grant to grant. As discussed earlier, this historical focus on financial accountability has expanded in response to increasing expectations of demonstrable performance and performance accountability for all government programs. For example, the Comptroller General’s Domestic Working Group issued its Guide to Opportunities for Improving Grant Accountability, highlighting innovative approaches and promising practices in grants management—focused both on ensuring grant funds are spent properly as well as achieving their desired results.

While performance accountability in grants is a relatively new pursuit, it has been used in contracts for a number of years. To illustrate performance accountability mechanisms and the strategies that contribute to their successful design and implementation, we examined four cases: (1) the federal CSE program, (2) the federal Perkins III Career and Technical Education Program, (3) a performance-based contract between the Massachusetts DMA and MBHP, and (4) a performance-based contract between the Canadian Ontario Realty Corporation (ORC) and ProFac.

Child Support Enforcement

The CSE program was established in 1975 by Title IV-D of the Social Security Act (Pub. L. No. 93-647). CSE functions in all states and territories through state or local social services departments, attorneys general offices, or departments of revenue in order to ensure that children are financially supported by both of their parents. State programs work toward establishing paternity, locating parents, establishing and enforcing support orders, and collecting and distributing child support payments. The federal Office of Child Support Enforcement (OCSE), an office of the Department of Health and Human Services’ Administration for Children and Families, oversees the development, management, and operation of state CSE programs and provides financial support (66 percent of total operating costs) to states. In fiscal year 2005 federal expenditures on CSE were $3.5 billion, with states spending $1.8 billion. Total collections in fiscal year 2005 were more than $23 billion. The total legally owed support for fiscal year 2005 was $29 billion, with $17.4 billion of that collected.

Total arrears (past due payments) for all previous years combined was $107 billion. Over $7 billion of those past due payments were collected and distributed in fiscal year 2005.

The Child Support Performance and Incentive Act of 1998 (Pub. L. No. 105- 200) linked incentive payments to performance, and in fiscal year 2005, OCSE made over $450 million in incentive payments to states. This act changed the original CSE incentive program from awarding incentives based solely on cost-effectiveness to awards based on meeting specific performance targets in five outcome areas: paternity establishment, order establishment, current collections, past due collections, and cost- effectiveness. The performance measures and targets are defined in the text of the act, which also provides a formula for determining the amount of each incentive payment. Additionally, the act established an alternative penalty system for those states not yet in compliance with the statewide automated data processing system required by Title IV-D Sec. 454(A) of the Social Security Act. The new incentive program was phased in from 2000 through 2002.

Carl D. Perkins Career and Technical Education

Effective July 1, 1999, the Carl D. Perkins Vocational and Technical Education Act of 1998 (Perkins III, Pub. L. No. 105-332) amends earlier legislation to evaluate and improve vocational and technical education. Each year under Perkins III, Congress has appropriated more than $1.1 billion in grants to states for career and technical education. The Office of Vocational and Adult Education (OVAE), an office of the Department of Education, administers the grants established in Perkins III, a pass-through grant to states, which administer the distribution of the funds to local school districts.

Perkins III defines major roles for OVAE and states in establishing performance accountability systems for vocational and technical education. States are given the responsibility for developing performance measures and data collection systems related to four required core performance indicators: academic and technical skill attainment, completion, placement and retention, and nontraditional participation and completion. OVAE negotiates these performance measures with states to ensure that they are sufficiently rigorous. States not meeting their performance levels for 1 year are required to complete a program improvement plan. States not meeting their performance levels for 2 years are subject to financial sanctions, although no state has failed to meet its overall levels for 2 consecutive years. States have also been eligible to receive incentive funds if they exceeded performance goals for the Perkins III grant as well as targets established by Title I and Title II of WIA. Title I of WIA supports workforce investment programs. Title II, also known as the Adult Education and Family Literacy Act, provides adult education funds to states. Governors have the authority to allocate the incentive funds for use in any of the three program areas.

Massachusetts Division of Medical Assistance Contract with the Massachusetts Behavioral Health Partnership

Beginning in 1996, the Massachusetts DMA entered into a 5-year contract, which was renewed in 2001, with MBHP to manage mental health and substance abuse services for roughly 300,000 people covered by the MassHealth Primary Care Clinician Plan—part of the Massachusetts Medicaid program. Of the individuals covered by the plan, more than half are children 18 or younger, including 20,000 children in the custody of the commonwealth’s Departments of Social Services and Youth Services.

The structure of the contract between DMA and MBHP involves a base contract, which governs requirements related to administrative and medical operations. These requirements continue through the life of the contract, or until they are modified through amendments. In addition to the base contract, there are performance incentive projects that focus on research and development projects. The majority of earnings available to MBHP come from the organization’s successful completion of these contractually defined incentive projects, which are renegotiated annually. Earnings are achieved only after the successful completion of specified goals and objectives, as documented and reviewed by the state.

Ontario Realty Corporation’s Contract with ProFac

ProFac, a facilities management company, was awarded a 5-year contract in 1999 (since renewed) by ORC to provide facilities management services for approximately 30 million square feet of space in 2,100 building sites owned by the Ontario government. About 280 ProFac managers, engineers, technicians, and support staff provide these services.

The contract between ORC and ProFac links performance to a 10 percent quarterly management fee holdback: on a monthly basis, ORC only reimburses ProFac for 90 percent of its administrative costs, retaining the other 10 percent, a “holdback,” which ORC returns to ProFac on a quarterly basis only if ProFac obtains a sufficient level of performance. This contract also links performance to an annual share-in-savings arrangement: ORC sets a budget each year based on prior years’ actual expenditures and budget projections; if ProFac spends less than the budget, it is able to share the difference, a share in savings, only if it reaches a sufficient operational level. ProFac’s performance on 30 key performance indicators (KPI) determines its operational level. These KPIs are accumulated to determine a score in four performance objectives: management performance, financial performance, asset integrity, and customer service. The performance objectives are scored and then weighted according to ORC’s priorities to determine a total performance rating.

Trade-offs and Challenges Exist in Ensuring Performance Accountability in Federal Grants

The trade-offs and challenges associated with performance accountability in federal grants largely depend on several key aspects of grant design and implementation. As we have previously reported, performance accountability tends to be greater (and grantee flexibility lower) in programs with certain types of design features. Because design features that encourage performance accountability can limit state and local grantee flexibility, achieving these twin goals can be a delicate balancing act and has implications for the accountability relationship between levels of government and the information needed to support accountability. Even in federal grants with designs that favor performance accountability, grant implementation challenges related to developing performance goals and measures as well as collecting and reporting performance data can influence the extent of performance accountability achieved.

Grant Design Features Affect the Balance between Accountability and Flexibility

Although there are various ways to design grants to encourage performance accountability, in general, there are three factors that particularly affect the degree of performance accountability that can be achieved, including whether a grant (1) includes performance-oriented objectives in addition to fiscally oriented objectives, (2) operates as a distinct program or as a funding stream, and (3) supports a limited or diverse array of objectives.

As we discussed previously, federal grants have traditionally focused on fiscal or legal accountability, such as holding states accountable for using federal grant funds to supplement rather than to supplant their own spending on a particular activity. However, federal grants that also include performance-oriented objectives—as well as the provisions that implement them—provide the basis for performance measurement and accountability for results, and signal a federal role in managing performance over the grant. Ideally, both types of objectives would be present in federal grants. Performance-related objectives focus on service or production activities and their results. For example, the central objective of the grants for Special Programs for the Aging—Nutrition Services, is to provide nutritious meals to needy older Americans to improve nutrition and reduce social isolation. In contrast, fiscal or financial assistance objectives focus on providing dollars to support or expand activities. Typical fiscal objectives include increasing support for meritorious goods or underfunded services and targeting grant funding to needy jurisdictions. For example, the objective of Title VI Innovative Education grants is to provide funds to support local education reform efforts. When objectives are purely fiscal, accountability to the federal agency tends to focus on fiscal matters, such as holding states accountable for using federal grant funds to supplement rather than to supplant their own spending on a particular activity.

Even when performance-oriented objectives are present, whether federal grants operate as distinct programs or as part of a larger funding stream directly affects who can be held accountable and for what. A grant that operates as a program has performance requirements and objectives and carries out specific programwide functions through a distinct delivery system, such that grant-funded activities, clients, and products are clearly identifiable. This type of grant gives the federal agency a role in managing performance and makes it easier to obtain uniform information about performance attributable to the grant funds. It is possible to identify which activities were supported; the amount of federal funds allocated to each; and to various extents, the results grantees achieved with federal funds.

In contrast, funds from grants that operate as part of a funding stream are merged with funds from state or local sources (and sometimes from other federal sources) to support state or local activities allowable under the flexible grant. These programs are managed at the state or local level, with the federal role limited accordingly. When grants are part of a funding stream, it is possible to identify which activities federal funds supported and the amount allocated to each, but once the grant funds are combined with the overall budget for a state or local activity, federal dollars lose their identification and their specific results cannot be separated out. This is particularly the case when the federal share is small, with most funding coming from other sources. The program outcome measures available in such programs are likely to be for outcomes of the state or local service delivery program, not the federal program from which the funding originated. Thus, grantees would generally be held accountable for overall outcomes, regardless of the funding source. For example, projects such as Oregon Option and the National Performance Review were designed to promote accountability for federal and/or national priorities, regardless of the funding source. They encourage grantors and grantees to work toward collaboratively developed outcomes. These intergovernmental partnerships can be particularly useful when funds come from a combination of federal, state, local, and private sources, or when the federal funding share is small.

Federal grants vary along a continuum, at one end supporting a single major activity common to all grantees (such as categorical grants), and at the other end, allowing unrestricted choice by the recipient among a wide variety of allowable activities, (such as block grants). Flexibility is narrowest, but accountability to the federal level clearest, in programs that focus on a single major activity. Flexibility is broadest in programs designed to support diverse state or local activities, but finding a common performance metric can be extremely challenging since these activities can vary considerably from state to state. That said, we have previously reported on options for building accountability provisions into block grants that help balance states’ flexibility to select a mix of activities and services that will best allow them to achieve a particular national outcome with accountability for achieving that outcome. These options include (1) relying on state processes both to manage block grant funds and to monitor and assess compliance and (2) emphasizing results-based evaluation rather than examining specific program or administrative activities.

Implementation Issues Present Further Performance Accountability Challenges

In addition to these design features, we have previously reported on a number of performance accountability challenges encountered in many grant programs during the grant implementation phase.

Lack of consensus on goals and performance measures: The priorities of states, tribes, local communities, and the federal government are not always the same. To ensure that grantees work toward national priorities, they need to be involved in the development of performance goals and measures. Lack of agreement on goals and measures— particularly when the federal funding is a small portion of the funding stream—could lead to grantees making choices that do not necessarily support the achievement of national goals.

Reliance on performance data from state and local partners and other third parties: Even if grantees collect data on similar activities, outcomes, and services, absent common data definitions Congress and program managers will lack comparable information, limiting the ability to compare state efforts or draw meaningful conclusions about the relative effectiveness of different strategies. We have previously reported that agencies relying on third parties for performance data also have difficulty ascertaining the accuracy and quality of the data. Further, programs often rely on state administrative systems for performance information. For some programs—such as many of the Administration for Children and Families’ programs—-since final reports are not due until 90 to 120 days after the end of the federal fiscal year, there is a delay in available data.

Onerous and inconsistent grant administration processes and requirements: Multiple grants maybe available for the same or similar purposes, meaning that federal grant recipients must navigate through a myriad of federal grant programs in order to find the appropriate source of funds to finance projects that meet local needs and address local issues. Sometimes programs meant to address common problems have potentially conflicting requirements. Variations in performance accountability requirements among these grants can limit the degree of performance accountability achieved. We have recently reported that while this situation is improving because of OMB’s efforts to streamline the grants application process, problems still exist.

Prohibition of performance information collection: Because states are principally responsible for implementing block grants at the state level, the block grant statutory prohibitions and requirements, and federal regulations and guidance are generally kept to a minimum. Sometimes federal agencies are prohibited from imposing reporting requirements because they are seen as burdensome. Clearly, this limits the extent to which federal agencies can oversee grantee performance.

Nevertheless, even with these trade-offs and challenges, agencies have been able to shift toward increased performance accountability in federal grants and the use of accountability provisions to ensure that grantees achieve real results through the programs, activities, and services financed with federal funds. The accountability provisions described in this report, along with strategies for their effective use, can help address the challenges noted above.

Accountability Mechanisms Can Improve Performance and Performance Accountability

We found a number of accountability provisions, specific actions that can be taken—that is, rewards given or penalties imposed when performance exceeds or fails to meet specified performance levels—that Congress, granting agencies, and grantees can use at different points in the grant life cycle to improve both grant performance and performance accountability. These examples demonstrate that accountability provisions can result in significant performance improvement and are flexible enough to accommodate a variety of situations.

A Variety of Accountability Mechanisms Exist

We found that a wide variety of accountability provisions are being used in both grant and contracting situations. A selection of these provisions is shown in table 1. This list is not intended to be exhaustive; rather, it is meant to illustrate the variety of mechanisms available. Some mechanisms may be more appropriate in certain situations than others, but all of these mechanisms can be used to either encourage improved performance or discourage poor performance. For example, public recognition and increased funding are two different mechanisms that can both be used to encourage and reward good performance. Similarly, mechanisms such as reduced funding or increased oversight can be used to discourage or penalize poor performance.

In addition, mechanisms can be either financial or nonfinancial in nature. A financial mechanism would be an increase in funding or a bonus. For example, the CSE program employs a financial incentive in the form of a bonus to encourage states to work toward the program’s five performance goals: states are eligible for a bonus every year based on performance. Nonfinancial mechanisms would include altered oversight or flexibility. For example, as part of the National Environmental Performance Partnership System, the Environmental Protection Agency affords states with high environmental performance levels greater flexibility in spending their grant funds.

Financial mechanisms also vary by their degree of risk or risk sharing between the grantor and the grantee. Grantee risk increases as the amount of money tied to performance increases. For example, bonuses—money awarded over and above the base grant amount—represent the least risk, while an outcome-based milestone payment plan where the entire grant award is based on performance represents much higher financial risk to the grantee. Nonfinancial actions, such as altering flexibility or oversight, would be relatively risk neutral.

Accountability Provisions Can Be Employed at Different Phases of the Grant Life Cycle

Accountability mechanisms can be used in different phases of the grant life cycle by different actors, including Congress, granting agencies, and grantees themselves, and the lessons learned from one grant cycle can be used to improve a performance accountability mechanism in the next (see fig. 1). For example, when reauthorizing the CSE program, Congress revised the original CSE incentive payments, which were solely based on cost efficiency, to create an incentive program tied to performance measures that reflect CSE’s five key goals: (1) paternity establishment, (2) order establishment, (3) current collections, (4) collection of payments in arrears, and (5) cost-effectiveness (design/redesign phase).

In contrast, performance measures and targets for Perkins III are created during the implementation phase. Specifically, each state is required by law to create its own performance measures linked to four core indicators: (1) student attainment of challenging state-established academic and vocational technical skill proficiencies; (2) student attainment of secondary diploma or postsecondary degree or credential; (3) student placement in employment, pursuit of further education, or both; and (4) student participation in and completion of vocational technical education programs that lead to nontraditional training and employment. The Department of Education periodically negotiates the performance targets for each state measure (postaward phase).

The use of past performance can inform and improve the recipient selection process (application phase). Specifically, the Florida Department of Children and Families has reported considerable success using past performance in recipient selection—contractors that do not meet their performance measures and standards are ineligible to be awarded future contracts.

Other mechanisms, such as altered flexibility or oversight, can be used by the granting agency—or even the grantee—to encourage improved performance during the term of the award (postaward phase). For example, according to the literature we reviewed, Minnesota’s Department of Human Services Refugee Services Section increases its oversight of local agencies if their performance drops below 80 percent on their key performance measures, including job placement rates, which nearly doubled over 5 fiscal years.

Importantly, grantees can also use these provisions to extend accountability to subgrantees and contractors. This is significant because many federal grants are ultimately passed through states to subgrantees. Some accountability provisions, such as public award and recognition, can even be employed by stakeholders or interested parties. For example, the National Association for State Directors of Career and Technical Education Consortium annually recognizes high-performing career and technical administrators and teachers and provides opportunities to share lessons learned and best practices.

Even when performance accountability provisions are absent from or limited by a grant’s legislation, agencies and grantees may still be able to include these types of provisions in the terms and conditions of the grants or subgrants or in contracts as long as the authorizing legislation does not specifically prohibit their use. OMB Circular A-110 provides that for grants awarded to nonprofits, a number of accountability mechanisms may be used—including withholding payments, termination of award, and “other remedies that may be legally available”—if the grantee materially fails to comply with the terms and conditions of the award. These terms and conditions can be specified in federal statute, regulation, assurance, applications, or the notice of award. For grants to state and local governments, however, OMB Circular A-102 contains no detailed accountability provisions and defers to the requirements specified in the authorizing legislation.

Various authorities govern the use of accountability provisions (see table 2). Provisions set by Congress, such as increased flexibility in the form of waivers from statutory restrictions, are generally laid out in authorization or appropriations legislation. As stated earlier, granting agencies can include accountability provisions in regulations, grant announcements, the request for proposal, and the notice of award. Grantees (and subgrantees) can use accountability provisions, such as formal recognition, to improve their performance internally. For example, the CSE program in Montgomery County, Pennsylvania, recognizes performance-improving suggestions from individual employees by publicly praising and inducting them into the office’s “all-star team.” These employees also receive a T- shirt with a picture of a stork—the program’s mascot—with the program’s motto Striving Toward Optimizing our Resources for Kids.

Accountability Mechanisms Can Be Tailored to Specific Situations

Selecting appropriate performance measures and linking them to performance accountability mechanisms is not a one-size-fits-all process; rather, accountability provisions are tailored to reflect the program’s characteristics. In addition to the range of accountability mechanisms available, we found a number of ways mechanisms were tailored and combined to reflect a variety of circumstances. Table 3 describes how measures and mechanisms can be designed and triggered to either reward or penalize performance.

For example, to encourage its contractor, ProFac, to cut costs while maintaining a high-level of performance, ORC modified a basic financial incentive to include a share-in-savings feature. ProFac is eligible to share a portion of the savings if it spends less than its yearly budget—often referred to as share in savings. To ensure that ProFac does not cut costs to the detriment of high performance, ORC also requires that ProFac achieve a performance rating of 80 percent or higher to share in these cost savings.

The CSE performance measures are an example of a “step up” provision— for each increasing performance percentage interval there is a corresponding increase in the incentive percentage paid. Each time a state moves to the next highest interval, it receives a higher percentage of the incentive for that measure. Conversely, the alternative penalty procedure for failure to implement a statewide child support data processing system acts as a “step down” mechanism. For each year the state fails to implement such a system, but shows a good faith effort to attempt to do so, the state will be penalized at increasing intervals—during the first year of noncompliance, the state will receive a 4 percent penalty, the second year an 8 percent penalty, the third year a 16 percent penalty, and so on.

Strategies Support Successful Selection, Design, and Implementation of Performance Accountability Mechanisms

Collectively, five key strategies appear to facilitate the effective selection, design, and implementation of performance accountability mechanisms. These strategies are ensure mechanisms are of sufficient value, periodically renegotiate and revise mechanisms, ensure appropriate measurement selection and usage, ensure grantor and grantee technical capacity, and implement system in stages.

In addition to these strategies, we noted extensive use of partnerships and collaborations and regular and effective oversight and feedback, which appeared critical to the success of accountability provisions in a third- party environment. We have previously reported that these practices are often associated with high-performing organizations and organizations that effectively used performance information to manage.

Ensure Mechanisms Are of Sufficient Value

There are a number of factors to consider when designing accountability mechanisms that help to ensure the mechanisms are of sufficient value and motivate performance improvement. Ensuring sufficient value requires that both the grantor and grantee are able to determine the value of the rewards and penalties and the cost of improved performance—be they financial or nonfinancial—and provide a meaningful return to both the grantor and the grantee and rewards or penalties should be consistently applied to maintain the value of the mechanisms to both the grantor and grantee.

Understand the Value of Performance

According to the literature we reviewed, both the grantor and grantee should understand what a particular level of performance is worth to them and what it will cost them to achieve that level of performance. When the value of performance is not properly identified, funds could be wasted and grantees may not respond to the mechanism. For example, we found one case where the contracting agency offered and ultimately paid a $250,000 bonus to a contractor for completing a pipeline 2-1/2 months earlier than scheduled. However, because the contracting agency did not actually need the pipeline to be completed for several years after the original contractual deadline, the contractor paid $250,000 for a level of performance it did not need. Although the recipient responded to the incentive, the contracting agency did not properly calculate the value of the performance improvement to the agency, resulting in wasted funds.

For a grantor, considering how accountability provisions support its strategic priorities can assist in determining the value of performance. The size of the associated rewards and penalties should be commensurate with the priority. For example, successful pay-for-performance programs reserve large rewards for achieving an organization’s most important priorities, or those that lead to large benefits, and provide smaller incentives for achieving goals that reap smaller benefits or are of lesser importance. For example, in the health care field, for certain conditions such as heart attack or stroke, delays in administering appropriate therapy greatly increase the risk of mortality and disability. Therefore, the incentives to treat these conditions quickly and appropriately should be larger than the incentives for other practices that should be encouraged yet produce fewer direct effects on mortality and illness, such as avoiding the use of ineffective antibiotics to treat the common cold.

Based on our literature review, it appears that insufficiently valued incentives are one of the main reasons that accountability provisions fail.

When an incentive is of sufficient value, the expected return outweighs the expected risk, and recipients are motivated to pursue the performance improvement. From 1975 through 1997 the CSE program included an incentive program that focused on cost-effectiveness. States were guaranteed an “incentive payment” from 6 to 10 percent of their total collections. In practice, the 4 percentage point difference between the minimum and maximum payment was reportedly not large enough to motivate states to increase collections enough to earn the 10 percent bonus. The new incentive system, established by the Child Support Performance and Incentive Act of 1998 only provides incentive payments to states that meet one or more of the act’s five outcome-based performance goals and associated targets, and penalizes states that fall below threshold levels in certain areas. A review of the new incentive system in a sample of nine states found that the median score on each of the five performance measures increased from fiscal years 2000 to 2002, the time period that the incentive system was implemented.

Motivating grantees to work toward federal outcomes is particularly challenging in grants where the federal investment is relatively small. Officials at Arizona’s Department of Adult Education, Career and Technical Education Division, told us that state funds in joint technological education districts outweighed federal funds for career and technical education (CTE) programs by more than four to one, and some districts did not want to accept federal CTE funds because, in their view, complying with the federal performance requirements was not worth the amount of funds they would receive. In order to ensure that the financial value of the Perkins III grants was large enough to motivate districts to meet the Perkins III reporting and performance requirements, the Arizona Career and Technical Division requires districts be in Perkins III compliance in order to receive CTE-related state funds, thereby creating a large incentive for local school districts to comply with the Perkins III requirements. Indeed, one district we spoke with lost Perkins III funding, but it was not until the state linked Perkins III compliance to state funding, and the district lost the rest of its CTE funding from the state, that the district started to make significant improvement toward meeting Perkins III requirements.

In addition, the grantor and grantee should understand the trade-off between the financial risk—the possibility performance will not improve sufficiently despite the resource investment—and the potential return— what will be gained if performance goals are met or exceeded—in order to determine whether to pursue any particular performance improvement. Accountability provisions that contain financial incentives and sanctions can shift risk between the grantor and grantee. That is, the more the grant award depends on performance, the greater the financial risk to the grantees: if they invest but do not perform sufficiently, they do not get paid. Conversely, in grants with limited or no performance accountability provisions, the grantor bears the bulk of the financial risk, since the grantee would receive the grant funds regardless of the results achieved.

Ensure Effective Distribution

The ability of a performance accountability mechanism to influence performance also depends on the effective distribution of organizational rewards and penalties to individuals within the organization who are directly responsible for the desired performance. For example, Glendale Union School District officials provide significant financial incentives to every school employee with whom a student has contact, including teachers, administrative staff, and other support staff—including the maintenance staff and bus drivers. The district’s philosophy is that all employees influence the school’s atmosphere and academic achievement and therefore contribute to any success it enjoys. Incentive funds are distributed based on a school’s performance on 13 academic, involvement, and satisfaction-related measures. According to a district official, the program, started 5 years ago, has increased camaraderie and collaboration among school employees, which the official said has contributed to academic improvement.

In Pennsylvania, the state passes along a portion of the state-earned federal incentive payments to the counties, according to each county’s proportionate share of the aggregate state CSE expenditures and to reflect its relative score for each performance measure, following the performance targets defined in legislation. Pennsylvania codifies the performance expectations and incentive payment procedures through cooperative agreements with each county.

Execute Mechanisms Consistently

The grantor must execute the mechanisms consistently and as designed to preserve the value of the mechanisms and to avoid introducing unnecessary risk. For example, if rewards are not paid as promised the grantee could learn that its additional efforts are not worth the cost—or risk—and may not make the additional effort to improve performance. Similarly, if rewards are paid indiscriminately or if penalties are not levied as expected, the grantee could learn that no additional effort or investment is required in order to benefit. In both cases, the system breaks down and the intended value of the accountability provision is lost.

We have reported on an agency with the authority to levy penalties for poor performance that resisted doing so. For example, the Federal Transit Administration (FTA) has several enforcement tools to deal with grantees’ noncompliance, including warning letters, suspension of funds, and grant termination. However, traditionally, FTA had been reluctant to use these tools to enforce compliance, opting instead to work with grantees in an effort to continually promote transit development. Reviews also showed that FTA’s oversight was superficial and inconsistent and that FTA seldom used its enforcement authority to compel grantees to correct weaknesses, even those that were long-standing. Consequently, federal dollars had been placed at risk. However, in response to our 1992 report, FTA established a new enforcement policy, developed detailed guidance on carrying out enforcement actions, and has since demonstrated a greater willingness to use these actions against grantees that do not comply with federal transit requirements. The Department of Defense (DOD), on the other hand, has paid billions in incentive and award fees for only “acceptable, average, expected, good, or satisfactory” performance. Despite paying billions in fees, DOD has little evidence to support its belief that these fees improve contractor performance and acquisition outcomes. The department has not compiled data, conducted analyses, or developed performance measures to evaluate the effectiveness of award and incentive fees. Using accountability mechanisms in this manner undermines their effectiveness as a motivational tool and marginalizes their use in holding grantees and contractors accountable for outcome-based results.

Periodically Renegotiate and Revise Mechanisms and Measures

Organizations need to allow for and use the flexibility to revise, update, or improve performance accountability mechanisms in order to respond to changing needs. In the literature we reviewed, we found a number of reasons for why accountability provisions may need to be revised. For example, unintended consequences associated with performance measures may be discovered only after full implementation. Organizational priorities may change. Technology may be introduced that substantially alters performance expectations. In addition, expectations that were previously considered stretch goals can become the norm over time—for example, as productivity gains are realized rewarding such performance may no longer make sense. Finally, efforts to reevaluate and revise should consider whether established accountability provisions are still effective at motivating performance improvements.

For example, the DMA/MBHP contract demonstrates a situation in which the entire accountability system experienced a revision to adjust to contract progression. Incentives in this contract were initially designed to motivate operational performance, such as processing time for billing, and performance targets were revised upward each year as performance improved. This upward revision helped ensure that performance continued to improve. Once MBHP’s performance reached the highest levels of industry performance in these areas, further improvements were no longer a priority.

As a result, DMA and MBHP used the annual review to revise the incentive system from motivating operational improvement to completing projects designed to improve performance in areas that would add value to the services MBHP provides, such as a project on providing behavioral health assistance to the homeless.

There are a number of ways to accommodate the need for periodic revision. For example, congressional amendments to or reauthorizations of grant programs allow policymakers the opportunity to revisit and modify existing provisions and to add flexibility for agencies that can lead to improved effectiveness. Agencies can include renegotiation and revision policies in regulations, guidance, and the terms and conditions of a grant award. Providing for periodic revision may be particularly important where performance measures are specified in legislation, because agency flexibility to respond to changing needs is significantly reduced. For example, as we have discussed, the Child Support Performance and Incentive Act of 1998 specifies the five performance measures, the performance targets, and the percentage of incentive payments that states can earn for performance. Initially, states made changes and saw improvement in these areas. Recently, both state and federal program officials have expressed concern about the long-term sustainability of such aggressive targets. For example, program officials said that even states that have in the past met the 90 percent performance target for the paternity measure are concerned because more recent annual rates have dropped back down closer to 80 percent. According to officials, states initially conducted an extensive caseload cleanup to improve performance on the five incentive measures when the incentive program was enacted in 1998, and much of the backlog of cases that could be addressed relatively easily has been. However, since the measures and performance targets are legislatively defined and the CSE program is permanently authorized, the agency does not currently have the flexibility to revise the measures or performance targets.

In contrast, state agencies, in negotiation with the Department of Education, can periodically revise their Perkins III CTE performance measures and targets during annual negotiations of their state plans. At program introduction, program targets are set through the negotiation process between states and OVAE. From this process, performance targets negotiated initially reflect a realistic level of what states can actually produce. Next, through annual application updates, the legislation allows renegotiation of performance levels with states. Among other factors, OVAE officials attributed the program’s success to this ongoing ability to renegotiate and revise the program’s measures. Although the Perkins III legislation is similar to the Child Support Performance and Incentive Act of 1998, in that Perkins defines the four core indicators tied to the performance measures used in the incentive program, it provides flexibility that the Child Support Performance and Incentive Act of 1998 lacks. The flexibility to revise or update the performance measures is built into the Perkins III legislation.

Accountability systems by their very nature assume that performance can be improved. However, performance improvements depend on adequate time for and ability of participants to learn from prior actions and use what they have learned to improve performance from one period to the next. Depending upon the complexity of the task, this process can take many cycles. Therefore, accountability systems should not be abandoned prematurely; rather, they should be assessed, revised, and improved.

Ensure Appropriate Measurement Selection and Usage

Selecting and using appropriate types of performance measures is important to the effective use of accountability mechanisms. We have previously reported on general attributes of good performance measures, noting that measures should be linked to agency goals and missions; be clearly stated; include measurable targets; and be objective, reliable, and balanced. Specifically, we found four of these characteristics that highlight key features of performance measures that can help ensure the successful linking of performance measures and rewards and penalties: the performance being measured should be within the recipient’s sphere of influence, the performance measures should be suitable to the mechanism evaluation cycle, and the performance measures and performance data should be tested.

Performance Should Be within Recipient’s Ability and Influence

Performance measures tied to rewards and penalties should represent performance that can be sufficiently influenced by the grant recipient’s actions. Absent this linkage, the grantee may have little motivation to change behavior to improve performance, and the granting agency risks wasting funds by either rewarding efforts that cannot reasonably be tied to grantee behavior or penalizing a grantee for outcomes that even its best efforts may not have prevented. For example, the Temporary Assistance to Needy Families (TANF) bonus payments rewarded states for reducing out-of-wedlock births. Several studies report, however, that there does not appear to be a link between the existence of these programs, or increases in efforts to deliver program services, and the TANF bonus payment. Many state officials perceive the outcome measure as inappropriate, relatively difficult to influence, or both, and discourage attempts to do so. According to one study, several states reported that they did not compete or did not continue to compete for the bonus funds because, among other reasons, their actions would not sufficiently affect the out-of-wedlock birth rate; therefore they directed their efforts to activities that were more directly under their influence.

In another example, the Perkins III CTE program has a financial incentive system that assesses state performance through performance measures that support its four core indicators—one of which encourages participation in and completion of programs leading to nontraditional employment. State and local officials in Arizona said their ability to affect performance for this indicator is very limited. They told us that although they have tried to address the barriers to nontraditional employment, they found that cultural and demographic influences have limited their ability to improve performance every year. Because performance has not improved as a result of their efforts, they focus most of their energy on efforts to improve performance in the other three core indicator areas, which reflect performance that is more directly under their control.

Measures Should Be Suitable to the Mechanism Cycle

Measures should assess performance that can be observed, achieved, and reported frequently enough to inform the use of awards and penalties on a timely basis. For example, an annual reward or penalty should be tied to a measure that is also assessed annually.

ORC uses performance measures that can be assessed in a relatively short period of time and that support program outcomes. ORC holds back 10 percent of ProFac’s management fee each month. Each quarter, ProFac has an opportunity to earn the holdback on the basis of its performance during the prior quarter on 30 KPIs. For example, 1 of the quarterly indicators tied to its overall customer service objective is the “overall customer satisfaction rate with project delivery.” The quarterly assessment is based on performance information gathered through customer satisfaction surveys of local managers and facility management contacts for all alteration projects, capital repairs, or both completed in the previous quarter. Both ORC and ProFac officials credit the frequency of evaluation for motivating ProFac to maintain high performance throughout the year.

OVAE uses the timing of its grant funding distribution cycle to its advantage in order to motivate states to meet federal performance accountability requirements. OVAE disburses grant funds in two pieces: a small portion in July and the remainder in October. States that did not provide complete, timely performance data, or missed their performance targets in the prior year, may have “conditions” put on the July portion of the funding; if conditions are not met during that quarter, the October funding is withheld.

Measures and Data Should Be Tested

Performance measures that trigger accountability mechanisms should be well functioning and time tested before they are linked to rewards and penalties to minimize the potential for unintended consequences. Although our literature review did not specify how long this could take, one study in our review noted that many leading companies use and test their measurement systems for years before linking them to accountability provisions.

Performance data should also be tested to make sure they are credible, reliable, and valid. Absent these attributes, organizations lack the basis for sound decisions about rewards and penalties. Data quality is so critical to performance accountability and oversight of grants that several organizations use it as the principal performance measure for performance-based funding. Pinellas County, Florida, alters the case funding rates paid to its ambulatory service contractor based on data quality. This “altered funding rate” provision links case reimbursement rates directly to data quality. For example, data that are incomplete, illegible, inaccurate, altered, or lacking evidence of medical necessity— and limit the county’s ability to claim for payment or use its data processing procedures—result in reduced reimbursements to the ambulatory service contractor for the affected cases. Pinellas County reports that as a result, ongoing data quality issues are minimal. In another example, the Child Support Performance and Incentive Act of 1998 prohibits the payment of financial incentives to states for performance in program areas where state data have failed an annual data reliability test. This requirement ensures that incentive payments are based on reliable and complete performance information.

Ensure Grantor and Grantee Technical Capacity

Grantor and grantee capacity—specifically, the knowledge about performance accountability mechanisms and the ability to effectively implement them—is critical to the effectiveness of performance accountability systems. For example, when the Air Force implemented its performance-based contracting program, it found that employee training focusing on how the performance-based aspects of the contracts should work were most critical. Specifically, practices such as providing a step- by-step approach to the process that outlined who should be involved at each step, how much of their time and effort would be required at each step, and what their specific roles and responsibilities would be were critical to employees understanding what was needed to create mechanisms to improve performance.

In addition, federal CSE staff in Region III provide a “Child Support Enforcement Incentives 101” presentation to state and county CSE staff throughout the region to explain how the performance measures and incentive payments work. This training presentation is tailored to the experience of CSE staff and the demographics of the county, state, or both (large urban, rural, large interstate caseloads, etc.) but strives to provide a clear and consistent message: the everyday activities of CSE staff directly affect the amount of child support available to children and their families, and drive the amount of incentive payments the county specifically, and the state in general, earns. The presentation includes interactive exercises to show how each employee’s casework feeds into outcome-based program results.

Implement System in Stages

Organizations go through a number of stages designing, testing, and revising measurement systems before linking them to accountability mechanisms. This longer, phased implementation allows organizations to ensure the system is effectively designed before tying it to rewards and penalties. During these stages, organizations can conduct pilot tests, create financial models, and conduct behavioral modeling to understand and modify a system prior to full implementation. For example, according to one expert, the Tennessee Valley Authority completes a “readiness test,” an assessment of measurement effectiveness and suitability, before allowing pay for performance or similar financial incentive systems to be pinned to that measure. This helps avoid unintended consequences associated with poorly designed measures. Phased implementation also allows organizations to adjust to new demands on their time and resources; set up or modify data collection systems; and ensure the credibility, validity, and reliability of the data before they are used to measure performance.

For example, the CSE incentive program was implemented in three stages to allow states to learn about the new incentives and performance measures. The five performance areas attached to incentives were developed and legislatively defined in 1998. In 1999, the new data measures were used by the states and audited for data reliability for the first time. In year one, one-third of the total incentive funds were allocated based on the new formula and the remaining funds were allocated based on the old system. In year two, two-thirds of the funding was allocated using the new system, and the remaining funds were allocated based on the old system. In year three, all incentive funding was allocated according to the new formula.

Collaboration and Oversight Also Key to Success

In addition to these strategies described above, we saw extensive use of partnerships and collaborations and regular and effective oversight and feedback. We have previously reported that these practices are often associated with high-performing organizations and organizations that effectively used performance information to manage.

Designing and implementing accountability provisions in a collaborative environment can help develop and encourage buy-in and support and lead to improvements. For example, Arizona state and local CTE officials said the state’s focus has shifted from a compliance-focused “audit,” ensuring performance data were properly collected and reported, to a true partnership in which state and local officials work together to identify and replicate successes, find solutions to challenges, and thereby improve performance. State CTE staff spend several days each year meeting with local CTE officials and providing regular assistance through on-site technical assistance teams, phone calls, and e-mails.

Oversight and feedback are critical to creating and sustaining effective performance accountability provisions. We have previously reported on oversight practices, noting specifically the value of feedback provided through performance monitoring plans and tools such as site visits, document reviews, and evaluations. For example, OVAE employs a number of tools to provide feedback and assistance to states implementing the Perkins III vocational education program. Among these tools are establishing state guidance that outlines how to meet the Perkins III developing a peer-to-peer mentorship program among states and with OVAE to share experiences and good practices, conducting monthly conference calls with state directors and data specialists to discuss challenges and solutions to data collection and quality, offering data quality “institutes” and conferences to share performance measurement and data quality and collection practices, and providing technical assistance to states.

An OVAE official said providing these types of oversight and feedback activities generated ideas and discussion to help states improve their performance; the state CTE officials with whom we spoke agreed.

Various Opportunities Exist at the Federal Level to Enhance Performance Accountability in Grants

The experiences with and strategies related to federal grant accountability provisions described in this report suggest a number of opportunities for Congress and the executive branch to improve the design and implementation of performance accountability mechanisms. First, a results-focused design can help encourage performance accountability in general and specifically provide for—or at least not prohibit—the use of accountability mechanisms to encourage desired behavior. In addition, the use of national program evaluation studies and research and demonstration grants can provide valuable information to assist in agency and congressional oversight of and knowledge about accountability mechanisms. Because credible performance information and performance measures form the basis for well-functioning accountability provisions, it remains critical for Congress and the executive branch to continue to encourage their development and use. Finally, OMB and agencies can commit to sharing good practices and lessons learned from experiences with performance accountability provisions in federal grants—an efficient and effective way to increase grantor and grantee knowledge, understanding, and use of these provisions.

A Results-Focused Design Encourages Performance Accountability

Considering grant design features and their implications for grantee flexibility and accountability can help policymakers provide for appropriate accountability provisions, whatever type of grant design is selected. We have previously reported that policy options reflected in grant design collectively establish (1) the degree of flexibility afforded to states or localities; (2) the relevance of performance objectives for grantee accountability; (3) whether accountability for performance rests at the federal, state, or local level; and (4) prospects for measuring performance through grantee reporting and oversight. Under a results-oriented approach, federal policymakers would specify national goals and objectives in statute, enact a process for establishing them, or adopt some combination of the two. As a result, when designing or reauthorizing grants, it is important to consider questions like the following: Is there a need for national performance objectives in this policy area? If so, grantees may be required to use uniform performance measures—as in the CSE program—to gauge progress. This allows for comparisons across grantees, and the supporting performance data collected from grantees have the advantage of being program specific. However, uniform activities, objectives, and measures may not exist or may not be desirable, especially under flexible grant program designs. In these cases, Congress may instead decide to allow grantees to establish their own program objectives. For example, the Child Care and Development Block Grant requires states to certify that they have requirements in effect to protect the health and safety of children whose child care is subsidized by the block grant. These requirements must cover the areas of preventing and controlling for infectious diseases, physical premise safety, and health and safety training. However, the specificity and stringency of these requirements and the manner in which they are enforced is left to the states. The Perkins III legislation outlines several performance areas and requires states to determine the measures they will use to measures progress in these statutorily defined areas. Performance targets for these measures are negotiated with OVAE. In these cases, the federal role in monitoring the grants is generally limited to collecting information on state and local program efforts and accomplishments as well as evaluating and disseminating information on best practices. Another option is to grant temporary exemptions (waivers) from certain federal program requirements to grantees that demonstrate that the flexibility granted can lead to performance improvements. For example, Oregon Option is an intergovernmental partnership that seeks to improve performance on benchmarks for a broad variety of initiatives, including childhood immunization, employment for the disabled, wild salmon recovery, juvenile justice, welfare reform, and child nutrition, by waiving administrative rules or seeking statutory change. In all cases, what accountability provisions are needed to support attainment of national performance objectives? These might include constraints on activities and funds distribution or operational objectives, standards, and criteria for performance. These can be set for the program as a whole or delegated to the level of government responsible for program management. Additional considerations are as follows: What data are needed for grantee accountability, and is it feasible to collect these data from providers? Is it possible to collect data at the project level? Will the contribution of federal funds be distinguishable from state, local, and private funds? If the answer to several of these questions is no, is additional information needed for program oversight? If so, how will such information be gathered and reported? The answers to questions such as these provide the basis for setting grantee reporting requirements.

Careful Use of National Program Evaluation Studies and Research and Demonstration Grants Can Help Assess Mechanism Performance

Congress has a number of opportunities to conduct oversight, such as when it establishes or reauthorizes a new program, during the annual appropriations process, and during hearings focused on program and agency operations. Providing for—or at a minimum, not prohibiting— performance accountability mechanisms can provide timely, targeted performance information and help policymakers ensure that federal grants focus on their goals, providing another basis for congressional oversight.

National program evaluations have the potential to answer questions about both overall program performance as well as the effectiveness of performance accountability mechanisms, in terms of their implementation, outcomes, impacts, and cost-effectiveness. However, national programwide evaluations are expensive in terms of dollars and time and frequently require capacities and resources beyond those provided for program management. Also, while evaluations of multiple sites provide valuable information, programwide evaluation data are typically periodic and often cover too few sites to support national estimates of performance. In these cases, research and demonstration projects often can provide better information on the effectiveness of various service delivery methods and approaches. Knowledge to support effective practice is well established in some subject areas and can be incorporated into program provisions (such as service standards) or in companion technical assistance or knowledge dissemination programs.

Encourage Development and Use of Credible Performance Information and Performance Measures

As we discussed earlier, performance accountability provisions rely on a supply of credible, reliable, and valid data and high-quality performance measures. We found organizations that recognizing the importance of data quality, tied incentives to increasing the supply of this type of information. Unfortunately, as our work on PART and GPRA implementation shows, the credibility of performance data has been a long-standing weakness. OMB, through its development and use of PART, has provided agencies with a powerful incentive for improving data quality and availability. However, improving the supply of performance information is in and of itself insufficient to sustain performance management and achieve real improvements in management and program results. Rather, it needs to be accompanied by a demand for and use of that information by decision makers and managers alike. Key stakeholder outreach and involvement is critical to building demand and, therefore, success. Lack of consensus by a community of interested parties on goals and measures and the way that they are presented can detract from the credibility of performance information and, subsequently, its use. While congressional buy-in is critical to sustain any major management initiative, it is especially important for performance accountability given Congress’s constitutional role in setting national priorities and allocating the resources to achieve them. Recognizing this, policymakers could use incentives to encourage program partners to agree on performance measures and targets against which performance will be judged.

Share Good Practices and Lessons Learned

We and others have frequently reported on the benefits of sharing promising practices and lessons learned to promote performance accountability in general in federal programs and program partners. We believe sharing good practices related to the effective design and implementation of performance accountability mechanisms carries similar benefits. As noted earlier, some state and local agencies’ programs have used this type of information sharing among themselves and their grantees and contractors as a means of performance improvement.

OMB, as the focal point for overall management in the executive branch, plays a key role in promoting performance improvement in federal programs and has developed or contributed to a number of tools to share information and encourage improvements to federal grants and program performance. For example, www.grants.gov includes information on grant opportunities, resources to assist in writing grant proposals, and a newsletter highlighting recent grant success stories, and www.results.gov has information on best practices related to the President’s Management Agenda initiatives—one of which is Budget and Performance Integration (BPI). Successful implementation of BPI depends significantly on federal agencies’ ability to ensure federal program partners work toward program goals and are held accountable for results. Expectmore.gov provides information on PART assessments and improvement plans; these assessments consider, among other things, whether the agency regularly collects timely and credible performance information to manage its programs, and whether the performance measurements are used to increase accountability. OMB’s own Web site also contains information on and examples of what it considers to be high-quality PART performance measures; discussion papers on measurement topics, such as how to effectively measure what you are trying to prevent; and strategies to address some of the challenges of measuring research and development programs.

OMB hosts a number of standing work groups and committees— comprising agency and OMB staff—to address important grant-related issues, all of which could accommodate a more specific focus on grants accountability provisions. For example, OMB’s Chief Financial Officer’s Council has a standing grants policy committee that focuses on grant application and reporting streamlining. Agency BPI leads meet monthly and recently developed a subgroup to share lessons learned related to efficiency measures that balance effectiveness, quality, and cost. They also discuss strategies to address the challenges of efficiency measures in the grant context and to develop additional guidance for agencies in this area.

In addition, OMB hosted a Block and Formula Grant workshop in October 2005 for federal officials aimed at identifying and sharing best practices in grants management and performance measurement. OMB staff agreed that the workshop was a valuable, efficient, and effective way to share information and lessons learned and that collectively the participants increased their knowledge and understanding of ways to enhance grant performance. They also noted that the real difficulty comes in “what to do next,” in other words, implementing the strategies gleaned from these sessions.

OMB staff told us that focusing specifically on performance accountability provisions in grants is necessary and useful, but that to date, they have focused their governmentwide efforts primarily on encouraging and enhancing agency capacity to develop high-quality, results-based program performance measures since improving the quality of measures and data necessarily precedes tying them to accountability provisions. The Block and Formula Grant workshop addressed issues of measurement and accountability, and several block grant programs have been working to strengthen grantee accountability.

Conclusions

As the challenges of the 21st century grow, it will become increasingly important for Congress, OMB, and executive agencies to consider how the federal government can maximize performance and results. This will be particularly important for federal grant program managers, given the significant amount of federal resources invested in these tools. Because many national objectives can only be achieved through state, local, and nongovernmental organizations, enhancing performance accountability below the federal level is equally important. In this report, we identify a variety of accountability mechanisms as well as key strategies to enhance their use. Collectively, these can help enhance and sustain performance accountability in grants at all levels of government.

As the cases we described illustrate, rewards and penalties are fundamental tools to help drive and motivate desired behaviors, but performance accountability mechanisms are not one size fits all; there is no universal transferable mechanism applicable to all programs. The specific mechanisms used by agencies and programs and highlighted throughout this report may not be universally adopted by other federal agencies and programs seeking to improve their own programs. Nevertheless, many can be tailored to specific grant programs, and the key strategies can be adapted to address the specific accountability challenges each agency faces.

Like all successful change initiatives, the progress currently under way to move from traditional fiscal accountability in grants to greater accountability for performance will take time; accountability provisions— and the performance measures associated with them—can take many years to mature. Although some federal programs are well on their way to collecting and reporting on reliable, credible, and valid data that support high-quality outcome goals agreed to by all program partners, many others are still struggling with how to define appropriate outcome measures. It will be critical to proceed thoughtfully and implement performance accountability in phases, building in enough opportunities to learn from mistakes and revise measures and mechanisms to reap the benefits of performance management while minimizing perverse incentives and unintended consequences.

As with all challenges, starting with small steps is often the best way forward. Accountability provisions can be used to bring program partners together to identify common ground. For example, programs that struggle with defining appropriate outcome goals, measures, and targets may wish to tie incentives to reaching agreement on them. Those that struggle with poor data quality and data definitions could reward grantees for progress in this area. Performance accountability—especially in the early stages— must be constructive, not punitive. Even if penalties are employed to promote performance accountability, there should be a constructive, collaborative approach to performance improvement that precedes them. Tying performance to lower risk, nonfinancial mechanisms may at first be more acceptable until performance measures have been time tested and revised as needed and grantees have had time to collect the necessary data to support the measures. Above all, a collaborative process that includes Congress, the executive branch, and grantees will be critical to developing successful performance accountability systems.

Accountability provisions assume that performance can be improved—but this requires information sharing and feedback. OMB has a central role in overseeing the performance and accountability in the federal government, and has used its role to promote general results-oriented performance measurement and management practices in federal grants through Web sites, guidance, work groups, and workshops. Each of these tools and strategies could be expanded on to specifically promote and encourage performance accountability in federal grants, both among related federal grant programs—programs that have a common purpose—and federal grant types—such as categorical grants, block grants, and funding streams. Sharing good practices and lessons learned and providing feedback on performance are valuable practices that can leverage resources to enhance knowledge and further performance accountability. Leading practices can be shared within and among agencies, grant programs, grantees, and even grant types. OMB recognized the value in sharing information on performance accountability mechanisms, but has not yet focused on this issue.

Recommendation for Executive Action

We are therefore recommending that the Director of OMB encourage and assist federal agencies in working with the Congress to expand the effective use of performance accountability mechanisms, focusing on the practices in this report, when federal grant programs are being created or reauthorized. We further recommend that OMB offer opportunities for knowledge transfer among federal agencies and encourage agencies to share leading practices and lessons learned in implementing grant accountability mechanisms. Possible vehicles for the collection and dissemination of this information include good practices guides and workshops and Web sites such as results.gov, grants.gov, and expectmore.gov.

Agency Comments

On August 22, 2006, we provided a draft of this report to the Director of OMB and the Secretaries of Education and Health and Human Services. We also provided relevant sections of a draft of this report to the grantees and contractors highlighted in this report. We received technical comments from all three agencies, which were incorporated as appropriate. In addition, OMB agreed with our recommendation but suggested we broaden it to address the role of federal agencies and Congress in the grant design and reauthorization process. We agree, and have amended our recommendation accordingly.

We are sending copies of this report to the Director of the Office of Management and Budget, the Secretaries of Education and Health and Human Services, and other interested parties. We will also make copies available to others upon request. In addition, the report is available at no charge on GAO’s Web site at http://www.gao.gov.

Please contact me on (202) 512-6543 or steinhardtb@gao.gov if you or your staff have any questions about this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are acknowledged in appendix II.

Appendix I: Objectives, Scope, and Methodology

The objectives of this report were to identify (1) the challenges to performance accountability in grants; (2) the kinds of mechanisms that are being used to improve grant performance and how; and (3) given the findings of questions 1 and 2, what strategies the federal government can use to encourage the use of these mechanisms, as appropriate.

To meet the first and second objectives, we interviewed experts in grants and performance management, including individuals from the following organizations: the School of Public Policy at the University of Maryland, the University of Central Florida, the John F. Kennedy School of Government at Harvard University, the John C. Stennis Institute of Government at Mississippi State University, the Public and International Affairs Department at George Mason University, the Political Science Department at the University of New Hampshire, Measurement International, and the American Productivity and Quality Center.

Based on our literature review, we developed a coding scheme for identifying (1) types of performance accountability mechanisms and (2) strategies used to successfully design and implement these mechanisms. We used these codes in a content analysis we conducted on a subset of the documents we reviewed. We chose the documents for content analysis based on the following criteria: discussed accountability systems, mechanisms, or both, discussed general practices that facilitated to the effective use of accountability mechanisms, or provided case examples; published in 1993 or later; found in major electronic databases; and published in the United States.

The content analysis was conducted by two analysts, with the second analyst conducting a dependent review. Discrepancies in coding were discussed and agreement reached between the two analysts. Our analysis produced an inventory of performance accountability mechanism types and five strategies used to facilitate the effective design and implementation of performance accountability mechanisms. See the bibliography for documents included in our review.

To illustrate the mechanisms and strategies identified through our content analysis, we used relevant case examples found in the literature. To further illustrate the mechanisms and design and implementation strategies, we also selected four additional case illustrations—two federal grant programs and two nonfederal contract cases. These four cases were selected based on our literature review, interviews with experts, and reviews of prior GAO work because they are good examples of where (1) a performance mechanism was present and (2) there is reason to believe that performance improved at least in part because of the mechanism.

To screen and develop the grant case illustrations, we interviewed regional and headquarters federal agency officials and officials at county/local offices. We also reviewed grant legislation, program guidance, and prior studies. To develop contract case illustrations, we interviewed officials both from the contracting agencies and the contractors and reviewed the contract.

To address our third objective, we synthesized prior GAO work, and we interviewed officials at the Office of Management and Budget.

We conducted our work from December 2005 through August 2006 in accordance with generally accepted government auditing standards.

Appendix II: GAO Contact and Staff Acknowledgments

Acknowledgments

Jackie Nowicki (Assistant Director) and Chelsa Gurkin (Senior Analyst-in- Charge) managed this assignment. David Bobruff, Katie Hamer, and Anne Marie Morillon made significant contributions to all aspects of the work. Kate France significantly contributed to the initial research and design of the assignment. In addition, Tom Beall and Jay Smale provided methodological assistance, Amy Rosewarne provided key assistance with message development, and Donna Miller developed the report’s graphics.

Bibliography

American Productivity & Quality Center. Achieving Organizational Excellence Through the Performance Measurement System. Houston: 1999.

American Productivity & Quality Center. Measure What Matters: Aligning Performance Measures with Business Strategy. Houston: 1999.

Ashworth, Karl, and others. “When Welfare-to-Work Programs Seem to Work Well: Explaining Why Riverside and Portland Shine So Brightly.” Industrial & Labor Relations Review, vol. 59, iss. 1 (2005).

Ausink, John, and others. Implementing Performance-Based Services Acquisition (PBSA): Perspectives from an Air Logistics Center and a Product Center. A Documented Briefing prepared by the RAND Corporation for the U.S. Air Force. 2002.

Ausink, John, Frank Camm, and Charles Cannon. Performance-Based Contracting in the Air Force: A Report on Experiences in the Field. A Documented Briefing prepared by the RAND Corporation for the U.S. Air Force. 2001.

Baker, George. “Distortion and Risk in Optimal Incentive Contracts.” The Journal of Human Resources, vol. 37, no. 4 (2002).

Banta, Trudy W., and others. “Performance Finding Comes of Age in Tennessee.” The Journal of Higher Education, vol. 67, no. 1 (1996).

Conlon, Timothy. “Grants Management in the 21stCentury: Three Innovative Policy Responses.” IBM Center for the Business of Government: Financial Management Series. Washington, D.C.: October 2005.

Courty, Pascal, and Gerald Marschke. “Measuring Government Performance: Lessons from a Federal Job-Training Program.” The American Economic Review, vol. 82, no. 2 (1997).

Dodds, Dan R. “Performance Incentives Can Spark Greater Productivity.” School Administrator, vol. 55, iss. 3 (1998).

Domestic Working Group: Grant Accountability Project. Guide to Opportunities for Improving Grant Accountability. Washington, D.C.: October 2005.

Dye, Jane Lawler, and Harriet B. Presser. “The State Bonus to Reward a Decrease in ‘Illegitimacy’: Flawed Methods and Questionable Effects.” Family Planning Perspectives, vol. 31, no. 3 (1999).

Eberts, Randall, Kevin Hollenbeck, and Joe Stone. “Teacher Performance Incentives and Student Outcomes.” The Journal of Human Resources, vol. 37, no. 4 (2002).

Garber, Alan M. “Evidence-Based Guidelines as a Foundation for Performance Incentives.” Health Affairs, vol. 24, no. 1 (2005).

Gordon, Stephen B. “Performance Incentive Contracting: Using the Purchasing Process to Find Money Rather Than Spend It.” Government Finance Review, August (1998).

Hatry, Harry P., and others. How Federal Programs Use Outcome Information: Opportunities for Federal Managers. Washington, D.C.: IBM Center for the Business of Government, May 2003.

Hildebrandt, Gregory G. “The Use of Performance Incentives in DOD Contracting.” Acquisition Review Quarterly, Spring (1998).

Johnston, Jocelyn M., and Barbara S. Romzek. “State Social Services Contracting: Exploring the Determinants of Effective Contract Accountability.” Public Administration Review, vol. 65, iss. 4 (2005).

Kathuria, Sandeep. “An Overview of Share-in Savings Contracting.” Contract Management, vol. 45, no. 11 (2005).

King, Donald C., and Paul D. Tolchinsky. “Do Goals Mediate the Effects of Incentives on Performance?” Academy of Management Review, vol. 5, no. 3 (1980).

Lane, Nancy E. “Performance Incentives in The Massachusetts Behavioral Health Program.” Administration and Policy in Mental Health, vol. 32, no. 4 (2005).

Marcus, Alfred A., Barry M. Mitnick, and Kiran Verma. “Making Incentive Systems Work: Incentive Regulation in the Nuclear Power Industry.” Journal of Public Administration Research and Theory: J-PART, vol. 9, no. 3 (1999).

Martin, Lawrence L. “Making Performance-Based Contracting Perform: What the Federal Government Can Learn from State and Local Governments.” IBM Endowment for the Business of Government: New Ways to Manage Series. Washington, D.C.: 2002.

Martin, Lawrence L. Performance-Based Contracting (PBC) for Human Services: A Review of the Literature. Orlando: Center for Community Partnerships, University of Central Florida, 2003.

Metzenbaum, Shelley H. “Performance Accountability: The Five Building Blocks and Six Essential Practices.” IBM Center for the Business of Government: Managing for Performance and Results Series. Washington, D.C.: 2006.

Rand Corporation. Organizational Improvement and Accountability: Lessons for Education from Other Sectors. 1st ed. Santa Monica, Calif.: 2004.

Sammer, Joanne. “Making Incentive$ Pay.” Industry Week: Leadership in Manufacturing. (Cleveland: Penton Media Inc., August 2002). http://www.industryweek.com/ReadArticle.aspx?ArticleID=1121 (downloaded Mar. 9, 2006).

Tsimbinos, John M. “Are You Willing to Pay for Success?” Bottomline, vol. 7, no. 6 (1990).

U.S. Department of Defense, Deputy Under Secretary of Defense for Acquisition Reform. Incentive Strategies for Defense Acquisitions. Washington, D.C.: 2001.

U.S. Department of Energy, Office of Inspector General. Inspection of Selected Aspects of the Office of River Protection Performance-Based Incentive Program. Washington, D.C.: 2001.