Managing for Results: Views on Ensuring the Usefulness of Agency Performance Information to Congress
Published: 2000-01-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed three agencies' annual performance plans to determine whether the plans met congressional requirements, focusing on: (1) which aspects of congressional information needs were met by the agency's annual performance plan or some other source; (2) where those needs were not met, and what accounted for the discrepancies or gaps in the information provided; and (3) what options agencies could use to practically and efficiently provide the desired performance information.
section_title: What GAO Found
paragraphs: GAO noted that: (1) the congressional staff GAO interviewed identified a great diversity of information they would like to have to address key questions about program performance; (2) the agencies GAO studied met some, but not all, of these recurring and ad hoc congressional information needs through both formal and informal means; (3) the congressional staffs were looking for recurring information on spending priorities within programs, the quality, quantity, and efficiency of program operations, the populations served or regulated, as well as the program's progress in meeting its objectives; (4) some of these recurring needs were met through formal agency documents, such as annual budget request justification materials, annual performance plans, or other recurring reports; (5) other congressional information needs were ad hoc, requiring more detailed information or analysis as issues arose for congressional consideration; (6) information needs that the congressional staffs reported as unmet were similar in content to, but often more specific or detailed than, those that were met; (7) several factors accounted for the gaps in meeting congressional information needs; (8) some information the agencies provided did not fully meet the congressional staffs' needs because the presentation was not clear, directly relevant, or sufficiently detailed; (9) other information was not readily available to the congressional staffs; (10) in some cases, the agencies said they did not have the information because it was either too soon or too difficult to obtain it; (11) improved communication between congressional staff and agency officials might help ensure that congressional information needs are understood, and that arrangements are made to meet them; (12) greater consultation on how best to distribute agency documents might improve congressional access to existing reports; (13) posting publications on Internet sites can increase congressional staffs' access to agency information without their having to specifically request it, but staff still need to learn that the information exists and where to look for it; and (14) agencies' annual Government Performance and Results Act performance plans and other reports might be more useful to congressional committees if they addressed the issues congressional staff said they wanted addressed on a recurring bases, and if agency staff consulted with the committees on their choice of performance measures.

section_title: Letter
paragraphs: Congressional committees have cited the need for a variety of information about federal programs and their performance to help the committees make decisions about the programs they oversee. Seeking to provide Congress and federal managers with more objective information on the results of federal programs and thus to improve government performance and accountability, Congress enacted the Government Performance and Results Act of 1993 (GPRA). The act established governmentwide requirements for agencies to set strategic and annual performance goals and, beginning in March 2000, to report annually on their results in achieving their goals.
In reviewing agencies’ second annual performance plans (setting goals for fiscal year 2000), the Committee was concerned that the performance measures selected by some agencies did not meet the Committee’s needs for oversight information and that it had insufficient input into some agencies’ performance plans. Recognizing that agencies are still in the early, learning stages of GPRA implementation, you asked us to explore, in three case studies, how agencies might better meet congressional needs for information on program and agency performance—through whatever source. The three agencies we studied are responsible for health surveillance, postsecondary student loans, and pension oversight.
We interviewed staff from these agencies’ authorizing and appropriations committees about their information needs and whether those needs were met, and we interviewed agency officials about how that information might be obtained. As agreed with your office, this report addresses the following questions: (1) Which aspects of congressional information needs were met by the agency’s annual performance plan or some other source? (2) Where those needs were not met, what accounts for the discrepancies or gaps in the information provided? (3) What options could the agencies use to practically and efficiently provide the desired performance information?
subsections: section_title: Background
paragraphs: In 1995, we reported on a study of how three agencies collected and reported evaluative information about their programs to this Committee.We found that the agencies collected a great deal of useful information about their programs, but much of it was not requested and thus did not reach the Committee, and much of what the Committee did receive was not as useful as it could have been. We also found that communication between the Committee and agency staff on information issues was limited and afforded little opportunity to build a shared understanding of the Committee’s needs and how to meet them. At that time, we proposed a strategy for obtaining information to assist program oversight and reauthorization review: (1) select descriptive and evaluative questions to be asked about a program at reauthorization and in interim years, (2) explicitly arrange to obtain oversight information and results of evaluation studies at reauthorization, and (3) provide for increased communication with agency program and evaluation officials to ensure that information needs are understood and requests and reports are suitably framed.
At the time, GPRA had recently been enacted, requiring agencies to develop multiyear strategic plans and annual performance plans and reports over a 7-year implementation period. In our 1995 report, we noted that annual reporting under GPRA was expected to fill some of the information gaps we described and that GPRA also emphasized the importance of consultation with Congress as evaluation strategies are planned, goals and objectives are identified, and indicators are selected. We suggested that our proposed process for identifying questions would be useful as agencies prepared to meet GPRA requirements and that consultation with Congress would help ensure that data collected to meet GPRA reporting requirements could also be used to meet the Committee’s special needs (for example, to disaggregate performance data in ways important to the Committee). We also saw a need for a useful complement to GPRA reports (and their focus on progress towards goals) that would provide additional categories of information, such as program description, side effects, and comparative advantage to other programs. The Committee had found such information to be useful, especially in connection with major program reauthorizations and policy reviews.
Since its enactment, we have been tracking federal agencies’ progress in implementing GPRA by identifying promising practices in performance measurement and results-based management, as well as by evaluating agencies’ strategic plans and the first two rounds of performance plans.We found that although agencies’ fiscal year 2000 performance plans, on the whole, showed moderate improvements over the fiscal year 1999 plans, key weaknesses remained and important opportunities existed to improve future plans to make them more useful to Congress. Overall, the fiscal year 2000 plans provided general, rather than clear, pictures of intended performance, but they had increased their use of results-oriented goals and quantifiable measures. Although some agencies made useful linkages between their budget requests and performance goals, many needed to more directly explain how programs and initiatives would achieve their goals. Finally, many agencies offered only limited indications that their performance data would be credible, a source of major concern about the usefulness of the plans.
This report does not directly evaluate the three agencies’ performance plans but rather looks more broadly at the types of information that authorizing and appropriations committees need from the agencies and how their unmet needs could be met, either through performance plans or through other means. We included program performance information available from sources other than annual performance plans because agencies communicate with congressional committees using a variety of modes—reports, agency Internet sites, hearings, briefings, telephone consultations, e-Mail messages, and other means. We did not assume that annual GPRA performance plans or performance reports are the best or only vehicle for conveying all kinds of performance information to Congress.
We conducted our work between May and November 1999 in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from the Secretaries of Education, Labor, and Health and Human Services and the Director of the Office of Management and Budget. HHS and Labor provided written comments that are reprinted in appendixes II and III. The other agencies either had no comments or provided technical comments. The agencies’ comments are discussed at the end of this letter. We also requested comments from the congressional staff members we interviewed on our characterization of their concerns, and we incorporated the clarifying changes they suggested.
section_title: Program Descriptions
paragraphs: Health Surveillance. The Centers for Disease Control and Prevention (CDC) in the Department of Health and Human Services (HHS) supports— through a number of programs—a system of health surveillance activities to monitor, and help prevent and control, infectious and chronic diseases. By working with the states and other partners, CDC—primarily the National Center for Infectious Diseases and the National Center for Chronic Disease Prevention and Health Promotion—provides leadership and funding through grants to state and local public health departments. Grants support research to develop diagnostic tests, prevention interventions, local and state public health laboratories, and information sharing and other infrastructure to facilitate a nationwide surveillance system. CDC centers support critical disease registries (such as the cancer registries) and surveillance tools (such as the Behavioral Risk Factor Survey) and disseminate public health surveillance data.
Pensions Oversight. In the Department of Labor (DOL), the Pension and Welfare Benefits Administration (PWBA) oversees the integrity of private sector pensions (as well as health and other welfare benefits) and seeks to increase employer-sponsored pension coverage in the workforce. The Employee Retirement Income Security Act (ERISA) sets minimum standards to ensure that private employee pension plans are established and maintained in a fair and financially sound manner. Employers also have an obligation to provide promised benefits and to satisfy ERISA requirements for managing and administering private pension plans. PWBA tracks and collects annual reports by plan managers on the plan operations, funding, assets, and investments. It develops regulations and conducts enforcement investigations and compliance reviews to deter pension fund mismanagement. PWBA also provides information and customer assistance, such as brochures targeted to women, small businesses, and minorities with low participation rates in pension plans, to encourage the growth of employment-based benefits.
Postsecondary Student Loans. The Department of Education’s Office of Student Financial Assistance (OSFA), a newly created performance-based organization, manages operations of the direct loan program (William D. Ford Federal Direct Student Loan Program) and guaranteed loan program (Federal Family Education Loan Program) that are major student financial assistance programs.These and other programs under the Higher Education Act of 1965, as amended, aim to help undergraduate and graduate students meet the cost of their education. The agency provides loans to students (or families) either directly through the direct loan program or under the guaranteed loan program, through private banks that lend the money at a federally subsidized rate.
In the direct loan program, the student applies through the school to the agency that transfers funds to the school. Later, a loan servicer (under agency contract) tracks and collects payments on the loan. In the guaranteed loan program, the student applies for the loan through a private lender that then tracks and collects the loan payments. The agency subsidizes the interest rate paid by the borrower. If a borrower defaults, a local guaranty agency reimburses the bank for the defaulted loan, and the department pays the guaranty agency.
section_title: Some Congressional Information Needs Were Met Through Formal and Informal Means
paragraphs: Congressional staff identified a great diversity of information they wanted to have to enable them to address key questions about program performance—either on a regular basis, to answer recurring questions, or in response to ad hoc inquiries as issues arose. Agencies met some, but not all, of these information needs through a variety of formal and informal means, such as formal reports and hearings and informal consultations.
subsections: section_title: Formal Annual Reports Met Some Recurring Information Needs
paragraphs: Congressional staff identified a number of recurring information needs, some of which were met through annual documents, such as agencies’ budget justification materials, GPRA annual performance plans, or other annual reports. The recurring information needs fell into four broad categories: allocation of program personnel and expenditures across activities; data on the quantity, quality, and efficiency of operations or services; characteristics of the populations or entities served or regulated; and indicators of progress in meeting objectives and side effects.
Both authorizing and appropriations staff wanted regular information on how personnel and expenditures were allocated across activities, both for the purpose of learning what was actually spent on a program or activity as well as to understand priorities within a program. This information was typically provided to their appropriations committees in the detailed budget justification documents that agencies submit each year with their budget requests. An appropriations staff member indicated that the routine data he wanted on PWBA’s program staffing and expenditures were provided by the agency’s budget justification documents, and that the agency was forthcoming in responding to requests for additional information.
Congressional staff also described wanting information on the quantity, quality, and efficiency of the activities or services provided. This information was needed to inform them of the nature and scope of a program’s activities, as well as to address questions about how well a program was being implemented or administered. They said they found this kind of information in both agency budget justification documents and performance plans. For example, both authorizing and appropriations staff members noted that the Department of Education’s budget justification documents and its departmental performance plan met their needs for basic information on trends in program expenditures and the volume and size of student loans and grants-in-aid over time. This data provided them with information about the change over time in the use of different financing options, revealing the potential for an increase in student debt burden.
In addition, the department’s performance plan included performance indicators and targets for OSFA’s response times in processing loan applications, an issue of concern to congressional staff because backlogs in loans being consolidated under the direct loan program had been identified and targeted for increased attention. In this case, Education officials said that a committee report required a biweekly report for 18 months on its loan processing so that the committee could monitor their progress in resolving the backlog. Officials said that this report was provided to a total of six committees—the authorizing, appropriations, and budget committees—in both the Senate and House. All three agencies also described their major programs (with some information on program activities and services provided) on their agency Internet sites.
Similarly, congressional staff also wanted regular information on the characteristics of the persons or entities the programs serve or regulate. In addition to providing a picture of who benefits from the program, such information can help answer questions about how well program services are targeted to the population most in need of service and how well those targeted populations are reached. The congressional staff described PWBA as good at providing statistics on the private pension plans and participants covered by ERISA in an annual report issued separately from the GPRA requirements. This report, the Private Pension Plan Bulletin, provides their most recent as well as historical data on plans and participants and detailed data on employee coverage and other characteristics by employer size.
Finally, the congressional staff also wanted regular information on the program’s progress in meeting its objectives and any important side effects that the program might have. The Department of Labor’s fiscal year 2000 performance plan supplied information on one of PWBA’ s goals—to increase the number of employees covered by private pension plans— derived from a survey conducted by the Bureau of the Census (Census). Congressional staff noted their satisfaction with the inclusion of program data on the student loan default rate and default recovery rate as performance measures in the Department of Education’s performance plan. The plan also provided data on whether low- and middle-income students’ access to postsecondary education was improving over time relative to high-income students’ access. These and other measures in the plan of unmet need for student financial aid, college enrollment rates, and size of debt repayments were derived from special surveys conducted by the Department of Education or by Census.
section_title: Some Ad Hoc Information Needs Were Met Through Formal Reports and Informal Consultation
paragraphs: Congressional staff identified a number of ad hoc information needs that arose periodically as “hot issues” came up for congressional consideration. Some of the needs were met through existing documents, and many others through informal consultations in response to a request from congressional staff, while still other needs were not met. The ad hoc information needs were similar to but somewhat different from recurring information needs and fell into five broad categories: details about a program’s activities and authority, news of impending change in the program, assessments of emerging issues, projected effects of proposed program changes, and effects and side effects of existing programs.
Congressional staff often wanted details about the scope of a program’s activities and authority that were not readily available from the general documents they had. Questions might have been raised by a constituent request or a legislative proposal, in which case the staff member wanted a fairly rapid response to a targeted question. In such cases, congressional staff said they often called the agency’s congressional liaison office, which either handled the request itself or forwarded it to knowledgeable program officials who, in turn, either returned the call to the requester or forwarded the information through the liaison. CDC officials also described referring requesters to the brief program descriptions they maintain on their Internet site.
Congressional staff noted that they wanted the agency to proactively inform them, in advance, when there was news of significant impending change in their Member’s district or to a program in which they had been involved. In one case, they wanted to have an opportunity to influence the policy discussions; in another case, they wanted to be prepared when the news appeared in the press. An authorizing committee staff member found that CDC’s targeted distribution of “alerts” provided a very useful “heads up” before the agency issued a press release about a public health concern. The alerts were distributed by e-Mail or faxed to the interested committee staff member or congressional members. During the recent appearance of a rare form of encephalitis in New York City, for example, CDC said that it informed congressional members and interested staff members from that region (as well as their authorizing and appropriations committees) about its findings regarding the source of the disease and explained what CDC was doing about it.
Another type of ad hoc information request was for assessments of an issue’s potential threat. Congressional staff described several occasions when a negative incident—such as a disease outbreak—occurred that raised questions about how frequently such incidents occur, how well the public is protected against them, and whether a congressional or legislative response was warranted. Because of the highly specific nature of such requests, the staff said they were usually made by telephone to the agency’s congressional liaison and responded to with a brief, informal consultation or a formal briefing.
On one occasion, CDC officials testified at a congressional hearing summarizing their research into antimicrobial-resistant diseases and how CDC’s surveillance programs track and respond to the problem. In another example, in response to a proposed merger of two large private corporations, a staff member wanted to know what the new owner’s obligations were to its holdover employees and how this would affect those employees’ pension benefits. In addition, in order to ensure the protection of those employees’ rights, the staff member wanted to know what enforcement options were available to the agency. The staff member indicated that PWBA officials provided this technical assessment and consultation in a timely manner.
As either the legislative or executive branch proposed changes to a program, congressional staff wanted projections of the effects of those proposed changes, not only as to whether (and how) the change would fix the problem identified, but also whether it would have undesired side effects. As committee staff discuss proposals, they said they often asked agency officials for informal consultations. If hearings or other more formal deliberations were planned, some kind of formal document might be requested. When an agency proposed a regulation or amended regulation, the agency prepared a formal document for public comment that provided a justification for the change. For example, to reduce the cost of loans to student borrowers, a congressional committee considered reducing the interest rate. However, some lenders expressed concern that a rate reduction would cut into their profit margins, forcing some to drop out of the program. To assess the likelihood of this projected result, the committee staff turned to the estimates of lenders’ profit margins produced by the Office of Management and Budget (OMB) and the Treasury Department.
Similarly, as new provisions are implemented, congressional staff might have questions about whether the provisions are operating as planned and having the effects hoped for or the side effects feared. In December 1998, OSFA was designated a performance-based organization (PBO), given increased administrative flexibility, and charged with modernizing the Department’s information systems and improving day-to-day operations. OSFA has provided authorizing and appropriations committee staff with regular reports on its Interim Performance Objectives (also available on its Internet site) that provide measures of efficiency in processing loan and loan consolidation applications and measures of borrower and institutional satisfaction. OSFA has also initiated cost accounting improvements to obtain better data on loans made, serviced, and collected under both the direct and guaranteed loan programs in order to provide baseline data against which to measure its progress in improving operational efficiency.
section_title: Several Factors Accounted for Gaps in Meeting Congressional Information Needs
paragraphs: Information needs that congressional staff reported as unmet were similar in content to, but often more specific or detailed than, those that were met. The information needs that congressional staff described as having been met tended to be general, descriptive information about a program’s activities and expenditures (such as those that might support their budget request) or descriptive information about the agency’s activities in response to a specific, often emerging, issue. This information was often provided in a formal report or presentation (such as a briefing or hearing). The information needs that congressional staff described as typically unmet were detailed information on the allocation of funds for activities, descriptive information about the program’s strategies and the issues they addressed, and analyses showing the program’s effects on its objectives.
The key factors accounting for the gaps in meeting congressional information needs were the following: the presentations of information were not clear, sufficiently detailed, or the information was not readily available to congressional staff; or the information was not available to the agency.
subsections: section_title: Some Information Presentations Were Not Clear, Detailed Enough, or Directly Relevant
paragraphs: In some cases, information on the topics was available or provided, but its presentation was not as useful as it could have been. Congressional staff members noted that neither the budget submission nor the departmental strategic plan demonstrated the link between a CDC cancer screening program, the dollars appropriated for it in the budget, and how this program contributed to meeting the department’s strategic objectives. A CDC official noted that, in combination, CDC’s performance plan and budget submission did link the strategic objectives with the budget. They explained that this was in part due to CDC’s budget being structured differently from its organization of centers and institutes. A CDC budget work group, formed in early 1999 in response to similar concerns, met with its congressional stakeholders and program partners and is developing a revised budget display that the group hopes will make this information more understandable in CDC’s next budget submission.
In another situation, congressional staff looked to the performance plan for a clear presentation of PWBA’s regulatory strategy that showed how the agency planned to balance its various activities—litigation, enforcement, guidelines, regulations, assistance, and employee education—and how those activities would meet PWBA’s strategic goals. The congressional staff wanted to know what PWBA’s regulatory priorities were, as well as how PWBA expected the different activities to achieve its goals. However, the departmental plan did not provide a comprehensive picture of PWBA and described only isolated PWBA activities to the extent that they supported departmental goals.
Some agency reports did not provide enough detail on issues of concern to the committee. Congressional staff members concerned about PWBA’s enforcement efforts wanted detailed information on the patterns of violations to show how many were serious threats to plans and their financial assets, rather than paperwork filing problems. A PWBA official indicated that PWBA could disaggregate its data on violations to show the distribution of various types of violations, but that there would need to be some discussion with the committee staff about what constituted a “paperwork” rather than a “serious” violation.
In another case, a congressional staff member was concerned that some patients were experiencing significant delays in obtaining cancer treatment after being screened under the National Breast and Cervical Cancer Early Detection Program. The program focuses on screening and diagnosis, while participating health agencies are to identify and secure other resources to obtain treatment for women in need. Staff wanted to see the distribution of the number of days between screening and beginning treatment, in addition to the median period, in order to assess how many women experienced significant delays. When this issue was raised in a hearing, CDC officials provided the median periods as well as the results of surveillance data that showed that 92 percent of the women diagnosed with breast cancer and invasive cervical cancer had initiated treatment.
Some responses to congressional inquiries were not adequately tailored to meet congressional staff’s concerns. For example, in preparing legislation, a congressional staff member needed immediately very specific information about the scope and authority of a program in order to assess whether a proposed legislative remedy was needed. However, he said he received documents containing general descriptive information on the issue instead, which he did not consider relevant to his question. An agency official indicated that this response suggested that the congressional query may not have been specific enough, or that the responding agency official did not have the answer and hoped that those documents would satisfy the requestor. In other cases, staff indicated they obtained this type of information succinctly through a telephone call to the agency’s congressional affairs office, which might direct them to a brief description of the program’s authority, scope, and activities on the agency’s Internet site or refer them to a knowledgeable agency official.
One authorizing committee staff person noted that, although the committee staff assigned to an issue develops background on these programs over time, there is rapid turnover in Members’ staff representatives to a committee. Moreover, because these staff are expected to cover a broad range of topics, she thought that they would find particularly useful brief documents that articulate the program’s authority, scope, and major issues, to draw upon as needed.
section_title: Some Information Was Not Readily Available
paragraphs: Some congressional information needs were unmet because the information was not readily available, either because it was not requested or reported, or because staff were not informed that it was available. In one instance, concerned about the safety of multiemployer pension plans, congressional staff wanted disaggregated data on the results of enforcement reviews for that type of plan. PWBA officials explained that the ERISA Annual Report to Congress does not highlight enforcement results for particular types of plans. However, they said that they could provide this information if congressional staff specifically requested it.
In several cases, the agencies thought that they had made information available by placing a document on the agency’s Internet site, but they had not informed all interested committee staff of the existence or specific location of those documents. For instance, an authorizing committee staff member had heard of long delays in PWBA’s responses to requests for assistance and wanted to know how frequently these delays occurred. In its own agency performance and strategic plans, PWBA included performance measures of its response times to customers requesting assistance and interpretations. But, because those measures were not adopted as part of the departmental performance plan and PWBA did not provide its own performance plan to the authorizing committee staff, this information was not available to those staff. Agency officials said that this information was available because they had posted their strategic plan on the agency’s Internet site. However, the committee staff person was unaware of this document’s presence on the site and thus was unaware that such a measure existed.
section_title: Some Information Was Not Available to the Agency
paragraphs: In some instances, the desired information was not available to the agency. This was because either special data collection was required, it was too early to get the information, the data were controlled by another agency, or some forms of information were difficult to obtain.
Where congressional questions extend across program or agency boundaries, special studies, coordinated at the department level, might be required to obtain the answers. For example, to address a policy question about how well prenatal services were directed to pockets of need, congressional staff wanted a comparison of the geographic distribution of the incidence of low birth-weight babies with areas served by prenatal programs and with the availability of ultrasound testing. HHS officials explained that although CDC and the National Center for Health Statistics had information on the regional incidence of low birth-weight babies through birth certificate data, these agencies did not have the information on the availability of prenatal services. The Health Resources and Services Administration (another HHS agency), which is concerned with such services, does not have information on the location of all prenatal programs or the availability of ultrasound equipment to link with the birth certificate data on low birth-weight. HHS officials indicated that, if this analysis were requested, the department would need to initiate a special study to collect data on the availability of services to match with existing vital statistics.
Some congressional information needs extend beyond what a program collects as part of its operations and thus would require supplemental information or a special data collection effort to obtain. For example, because a student’s race is not collected as part of loan applications, the Department of Education supplements its own records on the use of different student finance options with periodic special studies of student borrowers that do collect racial information. Because the different student loan programs maintain their records in separate databases, the office relies on special studies, conducted every 3 years since school year 1986- 1987, to examine the full package of financial options students and their families use to pay for postsecondary education. The congressional staff also wanted to obtain trend data on the extent to which all forms of student aid received (e.g., grants, loans, and tax credits) cover the cost of school attendance for low-income students. Education officials said that if published data from these special studies were not adequate, specialized data tabulations could be obtained. In the meantime, OSFA issued a 5-year performance plan in October 1999 that showed how it plans to improve the information systems for the student loan programs in order to improve operations and interconnectivity among the programs.
As programs are revised, questions naturally arise about whether the new provisions are operating as planned and having the desired effects or unwanted side effects. Congressional staff identified several questions of this type for the student loan programs due to changes created by the 1998 reauthorization of the Higher Education Act and the separate enactment of a new tuition tax credit: How many students will select each of the new loan repayment options? Which students benefit more from the new tax credit, low- or middle-income? Will the need to verify a family’s educational expenses create a new burden for schools’ financial aid offices?
In our discussions with OSFA, officials told us that they will report information on use of the new repayment options in their next annual budget submission, and that they believed the Internal Revenue Service (IRS) would include analyses of who used the tuition tax credit (similar to its analyses of other personal income tax credits) in its publication series, Statistics of Income. Because OSFA does not administer the tax credit, OSFA officials suggested to us that IRS would be responsible for estimates of any reporting burden for schools related to the tax credit.
Lastly, some information was not available because it is difficult to obtain. There has been congressional interest in whether a provision that cancels loan obligations for those who enter public school teaching or other public service leads more student borrowers to choose public service careers. Education officials said that a design for a special evaluation had been prepared, but that they had discovered that, because only a small number of student borrowers benefited from this provision, they were unable to obtain a statistically valid sample of these borrowers through national surveys.
Determining the effectiveness of federally funded state and local projects in achieving federal goals can be challenging for federal agencies. A CDC official told us that CDC conducts many studies evaluating whether a specific health prevention or promotion practice is effective or not, but that it expects it will take a combination of such practices to produce populationwide health effects. However, it is much more difficult to measure the effects of a combination of practices, especially when such practices are carried out in the context of other state and local health initiatives, than to test the efficacy of one specific health practice at a time. In addition, measuring the effectiveness of health promotion and disease prevention programs related to chronic disease can be difficult in the short term, given the nature of chronic diseases.
section_title: Improved Communication Offers Several Options for Filling Unmet Information Needs
paragraphs: To help ensure that congressional stakeholders obtain the information they want requires communication and planning—to understand the form and content of the desired information as well as what can feasibly be obtained, and to arrange to obtain the information. Our analysis uncovered a range of options that agency and congressional staffs could choose from—depending on the circumstances—to improve the usefulness of agency performance information to these congressional staffs. Improved communication might help increase congressional access to existing information, improve the quality and usefulness of existing reports, and plan for obtaining supplemental data in the future.
subsections: section_title: Options for Increasing Access to Existing Information
paragraphs: Agency officials said that increased communication between agency and congressional staff could have prevented some of the unmet information needs because they believed that, if requested, they could have provided most of the information congressional staff said they wanted, or arranged for the special analysis required. Increased two-way communication might also make clear what information is and is not available.
Each agency has protocols for communication between congressional staff and agency officials, typically requiring the involvement of congressional liaison offices to ensure departmental review and coordination of policy. Agency congressional liaisons and other officials said that they answered some ad hoc inquiries directly or referred congressional staff to existing documents or program specialists. Congressional staff said that they were generally able to get responses to their formal and informal inquiries through these channels, but several noted that communication was often very formal and controlled in these settings.
Some congressional staff and agency officials found that the informal discussions they had had were very helpful. In one case, agency officials were asked to discuss their program informally with appropriations committee staff; in another case, the incoming agency director scheduled a visit with a subcommittee chair and his staff to describe his plans and learn of their interests. It is our opinion that when key agency or committee staff changes occur, introductory briefings or discussions might help ensure continuity of understanding and open lines of communication that could help smooth the process of obtaining information on a recurring and on an ad hoc basis.
Discussion of what might be the most appropriate distribution options for different types of documents might help ensure that the information agencies make available is actually found. For example, authorizing committees might want to routinely receive agencies’ annual budget justification documents, which contain detailed information on allocations of resources. Also, although the three agencies aimed to increase the volume of material that was publicly available by posting it on their Internet sites, the information was often not available to congressional staff unless they knew that it existed and where to look for it. For relatively brief and broadly applicable material, like CDC’s summary of cost-effective health promotion practices, an agency may decide, as CDC did, to send copies to all congressional offices. Alternatively, to avoid overwhelming congressional staffs with publications, CDC officials sent e- Mail or fax alerts to contacts at relevant committees about newly released publications and other recent or upcoming events of potential interest.
section_title: Options for Improving GPRA and Other Existing Reports
paragraphs: Our analysis of the types of information the congressional staffs said they wanted on a recurring basis suggests ways the agencies might improve the usefulness of their performance plans and other reports to these committees. In addition, increased communication about the specifics of congressional information needs might help ensure that those needs are understood and addressed.
The congressional staff said that they wanted a clear depiction at the program level of the linkages between program resources, strategies, and the objectives they aim to achieve. Of our three case studies, congressional staff indicated that only the Education Department’s performance plan provided adequate detail at the program level—the level that they were interested in. As we previously reported, most federal agencies’ fiscal year 2000 plans do not consistently show how the program activity funding in their budget accounts would be allocated to agencies’ performance goals.And, although most agencies attempted to relate strategies and program goals, few agencies indicated how the strategies would contribute to accomplishing the expected level of performance.
One option would be for agencies to consider developing performance plans for their major bureaus or programs and incorporating this information in their department’s plan. For example, the HHS Fiscal Year 2000 Performance Plan consisted of a departmentwide summary as well as the annual performance plans developed by its component agencies and submitted as part of the agencies’ budget justifications. Alternatively, departments that prefer to submit a consolidated plan keyed to departmentwide goals could refer readers to where more specific data could be found in supplementary documents. OMB’s Circular No. A-11 guidance asks agencies to develop a single plan covering an entire agency but notes that, for some agencies, the plan will describe performance on a macro scale by summarizing more detailed information available at different levels in the agency. In these instances, OMB instructs agencies to have ready their more detailed plans specific to a program or component to respond to inquiries for more refined levels of performance information.
The congressional staff also said that they wanted, on a recurring basis, data on the quantity, quality, and efficiency of a program’s activities; the characteristics of the population served; and indicators of a program’s progress in meeting its objectives. These categories are consistent with those identified in our 1995 report as the information Congress wants on a routine basis. (Appendix I contains the categories of information and the list of core questions that we proposed committees select from and adapt to meet their needs when requesting information.) Although all three agencies consulted with congressional committees on their strategic plans as required by GPRA, only one consulted with our congressional interviewees on the development of its performance plan and choice of indicators. As we previously reported, agency consultation with both authorizing and appropriations committees as performance measures are selected is likely to make the agencies’ performance plans more useful to those committees.
The three agencies’ planned and ongoing efforts in data collection and analysis improvements may improve the quality and responsiveness of their reported information. However, without feedback from the congressional staffs on where presentations were unclear, or where additional detail or content is desired, the reports may still not meet congressional needs. Discussing information needs could also help identify which needs could be addressed in an annual or other recurring report and which could be addressed more feasibly through some other means.
In addition to performance plans and reports, the congressional staff also described a need for readily accessible background information on individual programs’ authority, scope, and major issues. Committee staff noted that rapid turnover in Members’ staff representatives to a committee results in some of their colleagues needing a quick introduction to complex programs and their issues. Some of the program and agency descriptions on agency Internet sites were designed for the general public and were not detailed enough to meet the congressional staffs’ needs.
section_title: Options for Arranging to Obtain New Information
paragraphs: To obtain new information about special subpopulations or emerging issues, congressional staff would have to make direct requests of the agency. Agency officials told us that they welcomed these requests and would do what they could to meet them. However, depending on the information requested and the time period in which a response is needed, it might not be possible for the agency to obtain it in time. Therefore, discussion between congressional staff and agency officials concerning the information needed is important to clarify what is desired and what is feasible to obtain, as well as to arrange for obtaining the information.
In some cases, the agencies said that they were able to conduct special tabulations to obtain the desired information. In other cases, they said that more data collection or analysis efforts might be required and that they would need some initial planning to determine how much time and resources it would take to obtain the requested information. Because it can be costly to obtain some information, advance agreement on the information content and format might avoid some frustration on both sides by clarifying expectations. In a couple of cases, when congressional staff members learned that the information was not readily available and would be costly to obtain, they were satisfied to accept a less precise or less detailed response.
Where congressional staff expect certain information will be important in future congressional considerations, advance planning for its collection would help ensure its availability in the desired format when it is needed. In some cases, agencies may be able to alter their information systems to track some new provision; in others, they may have to plan new data collection efforts. As stated in our 1995 report, communication is critical at two points in obtaining special studies: when a Committee frames a request for information, to ensure that the agency understands what is wanted and thus can alert the Committee to issues of content or feasibility that need resolution; and as report drafting begins, to assist the agency in understanding the issues that will be before the Committee and what kind of presentation format is thus likely to be most useful.
section_title: Agency Comments and Our Evaluation
paragraphs: The Departments of Health and Human Services and Labor provided written comments on a draft of this report, which are reprinted in appendixes II and III. Both HHS and Labor stated that, in general, the report is balanced and contains useful ideas for improving communications between federal agencies and congressional committees.
HHS also expressed two concerns. One concern was that the report suggested that the Department did not provide performance information at the program level. It said its component agencies provided this information in their own performance plans, which are presented as part of their congressional budget justifications. We have changed the text to clarify that the HHS Fiscal Year 2000 Performance Plan consisted of a departmentwide summary as well as the performance plans submitted as part of its component agencies’ congressional budget justifications. However, because we understand that these budget justifications were not widely distributed beyond the appropriations committees, we remain concerned that this performance information was not made readily available to authorizing committee staff.
HHS’ other concern was that the opening paragraphs of the report implied that it would emphasize GPRA as the primary medium for disseminating agency performance information although, it noted, the scope of the report is appropriately much broader. The Committee’s expectations for and concerns about agencies’ performance plans prepared under GPRA were the impetus for this report. However, the Committee also recognized that these plans and reports are only one mechanism to provide performance information to Congress and thus broadened the focus of our work.
Officials at the Department of Education suggested no changes and said that they appreciated recognition of their efforts to work collaboratively with Congress and provide good management for the department’s programs. OMB, HHS, and PWBA provided technical comments that we incorporated where appropriate throughout the text.
section_title: Scope and Methodology
paragraphs: To explore how agencies might improve the usefulness of the performance information they provide Congress, we conducted case studies of the extent to which the relevant authorizing and appropriations committee staffs obtained the information they wanted about three program areas. These cases were selected in consultation with the requesting committee’s staff to represent programs whose performance information they felt could be improved and to represent a range of program structures and departments under the Committee’s jurisdiction. For example, one selection (pension oversight) is a regulatory program in the Department of Labor; the other two (student loans and health surveillance) represent service programs in the Departments of Education and Health and Human Services. Pension oversight represents the direct operations of a federal agency, while the other cases operate through state and local agencies or the private sector. Each case represents a program or cluster of programs administered by an agency within these departments.
To identify congressional information needs and the extent to which they were met, we interviewed staff members recommended by the minority and majority staff directors of the authorizing and appropriation committees for the selected agencies. We asked the staffs to identify what information they needed to address  the key policy questions or decisions they faced over the preceding 2 years, and whether their information needs were met. To identify the reasons for the information gaps and how in practice the agencies might better meet those congressional information needs, we interviewed both agency officials and congressional staff; reviewed agency materials; and drew upon our experience with various data collection, analysis, and reporting strategies.
We are sending copies of this report to Senator Edward Kennedy, Ranking Minority Member of your committee; Senator Ted Stevens, Chairman, and Senator Robert Byrd, Ranking Minority Member, Senate Committee on Appropriations; Representative William Goodling, Chairman, and Representative William Clay, Ranking Minority Member, House Committee on Education and the Workforce; Representative Tom Bliley, Chairman, and Representative John Dingell, Ranking Minority Member, House Committee on Commerce; and Representative Bill Young, Chairman, and Representative David Obey, Ranking Minority Member, House Committee on Appropriations. We are also sending copies of this report to the Honorable Alexis Herman, Secretary of Labor; the Honorable Donna Shalala, Secretary of Health and Human Services; the Honorable Richard Wiley, Secretary of Education; and the Honorable Jacob Lew, Director, Office of Management and Budget. We will also make copies available to others on request.
If you have any questions concerning this report, please call me or Stephanie Shipman at (202) 512-7997. Another major contributor to this report was Elaine Vaurio, Project Manager.
section_title: Core Questions to Assist Program Oversight
paragraphs: Overall, what activities are conducted? By whom? How extensive and costly are the activities, and whom do they reach?
If conditions, activities, and purposes are not uniform throughout the program, in what significant respects do they vary across program components, providers, or subgroups of clients? What progress has been made in implementing new provisions?
Have feasibility or management problems become evident?
If activities and products are expected to conform to professional standards or to program specifications, have they done so? Have program activities or products focused on appropriate issues or problems?
To what extent have they reached the appropriate people or organizations?
Do current targeting practices leave significant needs unmet (problems not addressed, clients not reached)? Overall, has the program led to improvements consistent with its purpose?
If impact has not been uniform, how has it varied across program components, approaches, providers, or client subgroups?
Are there components or providers that consistently have failed to show an impact? Have program activities had important positive or negative side effects, either for program participants or outside the program? Is this program’s strategy more effective in relation to its costs than others that serve the same purpose?
section_title: Comments From the Department of Health and Human Services
section_title: Related GAO Products
paragraphs: Performance Budgeting: Fiscal Year 2000 Progress in Linking Plans With Budgets (GAO/AIMD-99-239R, July 30, 1999).
Performance Plans: Selected Approaches for Verification and Validation of Agency Performance Information (GAO/GGD-99-139, July 30, 1999).
Managing for Results: Opportunities for Continued Improvements in Agencies’ Performance Plans (GAO/GGD/AIMD-99-215, July 20, 1999).
Regulatory Accounting: Analysis of OMB’s Reports on the Costs and Benefits of Federal Regulation (GAO/GGD-99-59, Apr. 20, 1999).
Performance Budgeting: Initial Experiences Under the Results Act in Linking Plans With Budgets (GAO/AIMD-99-67, Apr. 12, 1999).
Emerging Infectious Diseases: Consensus on Needed Laboratory Capacity Could Strengthen Surveillance (GAO/HEHS-99-26, Feb. 5, 1999).
Managing for Results: Measuring Program Results That Are Under Limited Federal Control (GAO/GGD-99-16, Dec. 11, 1998).
Pension Benefit Guaranty Corporation: Financial Condition Improving, but Long-Term Risks Remain (GAO/HEHS-99-5, Oct. 16, 1998).
Managing for Results: An Agenda to Improve the Usefulness of Agencies’ Annual Performance Plans (GAO/GGD/AIMD-98-228, Sept. 8, 1998).
Student Loans: Characteristics of Students and Default Rates at Historically Black Colleges and Universities (GAO/HEHS-98-90, Apr. 9, 1998).
Credit Reform: Greater Effort Needed to Overcome Persistent Cost Estimation Problems (GAO/AIMD-98-14, Mar. 30, 1998).
Managing for Results: Critical Issues for Improving Federal Agencies’ Strategic Plans (GAO/GGD-97-180, Sept. 16, 1997).
Direct Student Loans: Analyses of the Income Contingent Repayment Option (GAO/HEHS-97-155, Aug. 21, 1997).
Student Financial Aid Information: Systems Architecture Needed to Improve Programs’ Efficiency (GAO/AIMD-97-122, July 29, 1997).
Managing for Results: Analytic Challenges in Measuring Performance (GAO/HEHS/GGD-97-138, May 30, 1997).
High-Risk Series: Student Financial Aid (GAO/HR-97-11, Feb. 1997).
Executive Guide: Effectively Implementing the Government Performance and Results Act (GAO/GGD-96-118, June 1996).
Program Evaluation: Improving the Flow of Information to the Congress (GAO/PEMD-95-1, Jan. 30, 1995).
The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touch- tone phone. A recorded menu will provide information on how to obtain these lists.