Balancing Flexibility and Accountability: Grant Program Design in Education and Other Areas
Released: 1998-02-11T00:00:00

Letter

Balancing Flexibility and Accountability: Grant Program Design in Education and Other Areas

Mr. Chairman and Members of the Education Task Force: I am pleased to be here today to discuss balancing flexibility and accountability in federal grant programs, particularly in education. With block grant proposals on the horizon and the first performance plans coming due under the Government Performance and Results Act, this topic could hardly be more timely. Our testimony responds to the program design issues raised in previous hearings before this task force and to your request that we provide a comparative perspective on these issues and their implications for future education programs.

Flexible grant programs across the federal government take many forms. Three federal program design features appear critical: (1) objectives—whether the national objectives involved are performance-related or fiscal, (2) nature of operations—whether the grant operates as a program or a funding stream, and (3) diversity of activities—whether the grant supports a single activity or diverse activities. In combination, these features are associated with differences in flexibility, accountability, level of government that is accountable for performance, availability of performance information, and the need to draw from additional sources in order to obtain the information needed to support program decisions at the national level. Considering design features and their implications can help policymakers ensure that accountability and information needs are met.

This testimony previews the results of our comparative study of flexible grant programs—in transportation, health, social services, education, criminal justice and employment—to be completed later this spring. Our study covered all of the currently operating ’block grants’ plus other grant programs with similar characteristics. The testimony also draws from past GAO work on program evaluation and information issues and on the accountability and fiscal effects issues associated with block grants. Finally, it draws on our studies of education programs, many of which have been described in testimony at earlier hearings. for examining current or proposed programs as they come before you for reauthorization or authorization.

Background

Flexible grants are an adaptable policy tool and are used in fields from urban transit to community mental health. They are alike in that each addresses a national purpose but gives state or local grantees the flexibility to adapt funded activities to fit the state or local context. However, there are vast differences among them as well. Some offer flexibility within a narrow range, as do many so-called ’categorical’ programs, while others offer choice so broad that they come close to resembling revenue sharing.

These differences can be traced to a few key design features—objectives, nature of operations, and diversity of activities—each representing an important policy choice. Each feature by itself has implications for accountability and information.

Key Design Features

Objectives: Performance-Related or Fiscal

Our first feature concerns the nature of the national objectives to be served through the federal grant program. We are not speaking here of such broad, ultimate national purposes as decreasing poverty, but rather of the more immediate, direct, and concrete objectives to be attained through the provision of grant funds. Grant programs’ objectives can be characterized as either primarily performance-related or primarily fiscal.

Performance-related objectives focus on services. In our study, we found such objectives as: expanding services, improving service quality, increasing coverage of targeted problems or populations, or achieving specified service outputs or outcomes. For example, the central objective of the grants for Special Programs for the Aging—Nutrition Services is to provide nutritious meals to needy older Americans in such a way as to reduce social isolation. Among grants for elementary and secondary education, national objectives like ensuring access to services, equity in educational opportunity, and targeting services to groups that are educationally disadvantaged have historically been important, reflecting civil rights concerns.

Fiscal or financial assistance objectives focus on providing dollars. Typical fiscal objectives include increasing support for meritorious goods or under-funded services and targeting grant funding to needy jurisdictions.For example, the objective of the Title VI Innovative Education grants is to provide funds to support local educational reform efforts.

Nature of Operation: Program or Funding Stream

A second critical feature concerns whether national objectives should be achieved through a grant-specific operating program or simply through adding to the stream of funds supporting ongoing state or local programs. An operating program is a “program” in the common-sense meaning of the term. (The Aging Nutrition Program again is an example.) It has performance requirements and objectives and carries out distinct program-wide functions through a distinct delivery system in such a way that grant-funded activities, clients, and products are clearly identifiable. Criteria of successful operation such as those outlined in our oral testimony last November apply. These include clear focus, stated objectives, monitoring and reporting in terms of those objectives, and research-based activities.

Grants that operate as a funding stream are not federal “programs” in this sense. Here, the federal agency provides funds that are merged with funds from state or local sources (and sometimes from other federal sources as well) to support state or local activities allowable under the flexible grant. The grant is one funding source among many. The “programs” supported are state or local programs. For example, funding stream operation is typical in elementary and secondary education, which is a state responsibility administered by local education agencies. Federal funding for kindergarten through grade 12 (K-12) programs may go directly to local education agencies. Or, it may go to the states, either for provision of state-run programs and services or to be passed through to local agencies under state oversight and accountability.

Activities: Single or Diverse

The third critical feature concerns diversity of activities. Some flexible grants, as in the senior nutrition example, focus on a single major activity or limited set of activities common to all grantees. At the other extreme, some allow unrestricted choice among a wide variety of allowable activities. For example, states with Preventive Health and Health Services grants can choose among 60 allowable uses of funds.

Implications for Flexibility, Accountability, and Information

These design features define the flexibility given to grantees, accountability for performance, and the level of government at which performance is managed. Flexibility reflects the presence or absence of national performance objectives. Its range is most limited when the grant funds a single major activity through a national program and broadest when the grant functions as a funding stream and permits choice among activities.

Accountability for performance is established by the inclusion of performance objectives—and provisions that implement them—in a grant program. When objectives are purely fiscal, accountability to the federal agency focuses on fiscal matters. For example, if the national objective is to encourage states to provide more of a nationally important service (like assisted housing), states may be held accountable for using grant funds to supplement rather than to replace or supplant their own spending on that service.

Programmatic objectives and operation as a national ’program’ give the federal funding agency a role in managing performance under the grant. In programs with primarily fiscal objectives and those that operate as a funding stream, the activities supported are managed at the state or local level. In the words of agency staff (quoting state officials), “these aren’t federal programs, they are state programs that receive federal funds.” The federal agency’s role is limited accordingly, and may involve little more than seeing that applications for funding are properly submitted, compliance or audit issues resolved, and money disbursed in timely fashion. Where grant-funded activities are managed at the local level as in the two education programs we studied—Title VI Innovative Education and Safe and Drug Free Schools and Communities—the state’s role may be similarly limited.

Design features also affect the availability of performance information. Operation as a program simplifies the task of getting uniform information about performance attributable to grant funds. It is possible to identify which activities were supported, the amount of federal funds allocated to each, and, to various extents, the results of federal support. Operation as a funding stream complicates this task. When grant funds are part of a stream, it is possible to identify which activities federal funds supported and the amount allocated to each. But once added to the overall budget for a state or local activity, federal dollars lose their identity and their results cannot be separated out—particularly when the federal share is small.

Thus, the only program outcome measures available are likely to be for the state or local service delivery program, not the federal funding program.

As the above discussion suggests, it is the combination of features, rather than any single feature, that has implications for flexibility, accountability and information.

Design Features in Combination

Examining how the design features were used in programs across government, we identified four major combinations or design types. We have summarized them in table 1, which shows design features, examples, and summary comments associated with each type. As the last column indicates, state or local flexibility and control over performance objectives and performance management increase as you move down the table.

Partnership Act are to provide job training that leads to increases in employment and earnings of youths and adults facing serious barriers to participation in the work force. To evaluate the results of the program in achieving these objectives, recipients are required to provide counts of activities provided, demographic characteristics of individuals served, employment outcomes, and program costs. No major K-12 education grants are of this type.

Our second type covers performance-related, funding stream grants (second row)—grants that involve national performance objectives, yet operate through state or local programs. Most programs of this type cover a state or local function or delivery system (such as preventive health) involving various activities. National performance objectives typically concern system improvement or capacity-building, ensuring access to services, service quality, and targeting of activities to priority populations. Several grants in this group require state or local grantees to set their own performance objectives of various kinds. Provisions of the Preventive Health and Health Services Block Grant, for example, require each state to fund activities related to Healthy People 2000 objectives and to measure and report the progress of the state in meeting the objectives selected.

Our third type includes grants with fiscal objectives (third row) that provide support for program-like—rather than ongoing—state or local activities. These activities often take the form of projects—similar to operating programs in having clear boundaries, but with a clear start and finish as well. Grant provisions identify allowable uses of funds and may include national criteria for selecting activities, such as the benefits test that applies to projects supported by Community Development—Entitlement block grants. Otherwise, performance objectives and measures are set at the operating level. Under the Drug Control and System Improvement (Byrne Formula) Grant Program, for example, states are required to set performance objectives for activities that are funded and to evaluate the success of these activities in achieving those objectives. output information (such as client counts), but even this can be difficult. For example, where actual counts of recipients served are not available, the Social Service Block Grant accepts counts based on estimation procedures which may vary in their statistical validity.

Balancing Flexibility and Accountability and Implications for Information

These four design types present very different situations with respect to grantee accountability—what grantees are held accountable for and the level of government that is accountable for performance—and the information needed to support it. They also differ with respect to the information needed to support program decisions at the national level and prospects for getting this information through grantee reporting, as opposed to other means.

Grantee Accountability and the Information Needed to Support It

As a previous GAO report has noted, accountability is an elusive concept whose meaning depends on the context. At a minimum, all grantees are accountable to the federal level for financial management and for using funds to support allowable activities. Beyond that, what grant recipients are accountable for to the federal level varies from grant to grant, reflecting (1) the type of objective, and (2) if performance objectives are involved, whether the federal level manages the program or merely adds to the stream of funds supporting state or local programs. I will now describe the situation for each type of grant, with a focus on performance issues.

Accountability for performance to the federal level is most extensive in grants that include national performance objectives and operate as distinct programs—grants with the most limited flexibility. As mentioned previously, programs of this type commonly collect and report information in line with their performance objectives, which may be concerned with program implementation, outputs, or (when possible to measure) direct outcomes of services. (Indirect or ultimate outcomes are another matter, which we will discuss in the next section.) flow to the federal funding agency. If specific fiscal objectives (such as maintenance of effort) apply, grantees are accountable for meeting them and for providing fiscal information to verify that they have been met.

The grants that combine federal performance objectives with operation through state or local programs—a common pattern across K-12 education—present puzzling performance accountability issues, particularly for service outcome objectives. Activities supported with federal funds and the information collected about performance often differ from state to state. (This difficulty affects fiscal-objective operating programs as well.) While state or local program outcomes in total may be measurable, the component attributable to federal funding cannot be separated out. Thus, measuring performance at the level of the federal program may not be feasible. For accountability purposes, measuring overall performance of the state or local program would not be appropriate, particularly when the federal grant contributes only a small fraction of the cost.

Reliance on delivering services through state and local programs or projects thus raises other delicate issues of federalism as well. First, there is the question of whether achieving a given national objective through existing state and local programs is feasible. This question is particularly relevant with respect to new service outcome objectives, such as decreasing drug use among students. State and local programs designed with different objectives in mind may have difficulty incorporating this new objective. Or, conditions that enable achievement of that outcome (such as solid knowledge of how to produce it) may not be met. set performance criteria for schools funded under the grant and procedures to ensure that schools that don’t meet these criteria are identified and remedial actions are initiated (as under Education’s Title I program).

States and localities are then accountable to the federal agency for implementing these provisions.

The Department of Education has been experimenting with a different approach. The Department grants temporary exemptions from certain federal program requirements (waivers) to states or school districts that demonstrate that the waiver will lead to educational improvements. These waivers are intended as a tool to expand the flexibility available to local school districts in exchange for increased accountability for student achievement. The results of this experiment are not yet in.

One final example of an approach to serving national objectives through state or local activities relies on the techniques embodied in the Government Performance and Results Act—that is, requiring states or localities to set—and to report to the federal agency in terms of—performance objectives for the activities or projects they choose to support with federal funds. Provisions of the Safe and Drug Free Schools and Communities Act, for examples, require states and local education agencies to establish drug use and violence prevention objectives, report the outcomes of state and local programs, and assess their effectiveness toward meeting the objectives.

Under this ’results’ approach, accountability for performance remains at the level of the state or local agency doing the reporting, not the federal or state agency to whom the report is directed. The federal or state agency receives the information, but does not use it for program management.

This information, however, can be useful in assessing the degree to which national objectives for the program are being met, a subject to which we now turn.

Information to Support Program Decisions at the Federal Level

To make decisions about the programs they oversee, congressional committees need evaluative information—information that tells them whether, and in what important respects, a program is working well or poorly as well as whether performance objectives are being met.

Performance data collected from grantees is an important source. But they have limitations, and other sources are useful as well.

Let me describe for you some limitations of performance data in the flexible grant context. First of all, our study indicates that, beyond simple activity counts, few grant programs are able to obtain these data program-wide. Collecting reliable uniform data requires conditions (such as uniformity of activities, objectives, measures) that are unlikely to exist under many flexible program designs, and even where overall performance can be measured, the amount attributable to federal funding often cannot be separated out. And some programs have ultimate outcome goals, such as increasing highway safety, which are measurable only through aggregate data.

More importantly, performance data cannot answer the full range of questions that are likely to arise during congressional oversight. To determine how, and in what respects, a program is working well or poorly, we have found that Congress is likely to need not only performance information but also

Descriptive information sufficient to convey a sense of the variety of conditions under which the program operates and how federal funds are actually being used. (Our earlier study found that general summaries of activities performed or clients served do not meet this need.) For flexible grants, information that shows how grant funds fit into the context of other programs is of particular interest. Information about program implementation, including attention to whether feasibility or management problems are evident and whether the methods used to deliver services are of known or likely effectiveness. Information concerning positive or negative side effects of the program. Information that will help them determine whether this program’s strategy is more effective in relation to its costs than others that serve the same purpose.

Some of this information may be available from federal agency staff, particularly if the agency plays an active oversight or technical assistance role. of sites under controlled conditions that could not be obtained through grantee reporting. We found evaluation studies in programs of every type.

Information on the effectiveness of service delivery methods comes largely from research and demonstration studies. Knowledge to support effective practice is well established in some of the subject areas covered in our sample of grants, and was incorporated into program provisions (such as service standards) or in companion technical assistance or knowledge dissemination programs.

Finally, aggregate measures of social, environmental, educational or health outcomes can be useful. However, since no single program is likely to have much of an impact on these measures they are most useful for judging the combined results of programs that share a common goal.

To close this section, let me note that each of the sources mentioned is more likely to be available when backed by statutory authorization and budget resources than when it is not. As we observed in our earlier study, Congress is more likely to get the information it asks for and pays for.

Summary and Design Framework

In summary, the design of a flexible grant program involves choosing among policy options that, in combination, establish the degree of flexibility afforded to states or localities; the relevance of performance objectives for grantee accountability; whether accountability for performance rests at the federal, state, or local level; and prospects for measuring performance through grantee reporting. Fortunately, performance measures are not the only source that Congress can draw on in making program oversight decisions.

We have developed an exhibit that depicts the grant design policy choices discussed in this testimony and factors that might be considered at each point in the form of a decision tree. Each choice has implications regarding the degree of flexibility provided states or local entities, the type of performance information that can be available, and the level at which this information can be used for accountability purposes. Our framework assumes that the decision to establish a formula grant program (as opposed to another type of program, such as a demonstration) has already been at least tentatively made.

Statement Balancing Flexibility and Accountability: Grant Program Design in Education and Other Areas programs? form? objectives? revise if needed Statement Balancing Flexibility and Accountability: Grant Program Design in Education and Other Areas needed? oversight?

Will contribution of federal funds be distinguishable?

To illustrate how the exhibit flows, let’s consider how it would apply to the Title VI Innovative Education Program Strategies grant program. The objectives of the grant, to support local education reform and innovation, are primarily fiscal, putting us on the upper decision path on our diagram. Funds may be used to support local projects (such as magnet schools), but Title VI’s purpose does not require that project-level performance objectives be set so we continue to the step of designing provisions to match fiscal objectives.

Title VI has such provisions, stating that grant funds may not be used to supplant funds from non-federal sources and the state must maintain prior levels of fiscal effort. To obtain information required for accountability, the program requires local districts to describe their intended uses of funds and how these will contribute to the grant’s objectives of supporting reform. States, drawing on district records, must report biennially on general uses of funds, types of services furnished, and students served. As these data are of limited utility for program oversight, the Congress mandated national evaluation reports on this program in 1986 and 1994. The 1994 report provided information about federal share, the size of state and local grants, how funds were used, the minimal performance accountability requirements imposed by states, and the difficulty of evaluating a program that provides supplemental resources for other activities.

To further illustrate, let’s consider the Safe and Drug-Free Schools and Communities grant, focusing on funding at the local level. Grant funds support activities that serve national performance objectives of preventing violence in and around schools and the illegal use of alcohol, tobacco and drugs. The presence of these objectives puts us on the lower, performance-oriented path of the diagram. Funded activities are not implemented through a national operating program, but rather through state and local programs—reflecting at least the hope that national objectives could be achieved through these programs. However—and here’s an example of some of the federalism issues we mentioned earlier—some national program provisions do apply. Local programs must be comprehensive and convey the message that the illegal use of alcohol and other drugs is wrong and harmful. These national requirements notwithstanding, the local education agencies are responsible for setting performance goals, deciding how to pursue them, and reporting to the state in terms of those goals.

Moving along the state and local path on our diagram, we come to the question of whether drug and violence prevention programs function as distinct projects or like a funding stream. A recent evaluation study—and here we jump ahead a little—suggests the latter. Examining what appeared to be comprehensive school-based drug prevention programs, this study found so much variation within districts in what was being done that local activities hardly met our definition of a ’program.’ As to the feasibility question on the diagram, collecting performance data—beyond student counts—for drug prevention programs has proven difficult. Reporting requirements make reference to local program outcomes, but states are simply asked to provide whatever relevant data they can. Reflecting these limitations, provision has been made to gather data from other sources, including state-level data from national surveys of youth drug use, for program oversight. Although the Department of Education is required to produce a report on the national program every three years, the lack of uniform information on program activities and effectiveness may limit its usefulness. An evaluation study covering the period 1990-1995 provided insight into resource adequacy, the extent to which activities reflect research findings, implementation issues, student outcomes, and state and local evaluations. Further evaluation studies are planned.

Mr. Chairman, this concludes my illustration of the decision diagram and also my prepared statement. I appreciate the opportunity to share this information with you and would be pleased to respond to any questions you or members of the Task Force may have.

Related GAO Products

Federal Education Funding: Multiple Programs and Lack of Data Raise Efficiency and Effectiveness Concerns (GAO/T-HEHS-98-46, Nov. 6, 1997)

Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138, May 30, 1997)

Federal Grants: Design Improvements Could Help Federal Resources Go Further (GAO-AIMD-97-7, Dec. 18, 1996)

Safe and Drug-Free Schools: Balancing Accountability with State and Local Flexibility (GAO/HEHS-98-3, Oct. 10, 1997)

Block Grants: Issues in Designing Accountability Provisions (GAO/AIMD-95-226, Sept 1, 1995)

Block Grants: Characteristics, Experience, and Lessons Learned (GAO/HEHS-95-74, Feb. 9, 1995)

Program Evaluation: Improving the Flow of Information to the Congress (GAO/PEMD-95-1, January 30, 1995)

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.