GPRA is intended to shift the focus of government decisionmaking, management, and accountability from activities and processes to the results and outcomes achieved by federal programs. New and valuable information on the plans, goals, and strategies of federal agencies has been provided since federal agencies began implementing GPRA. Under GPRA, annual performance plans are to clearly inform the Congress and the public of (1) the annual performance goals for agencies’ major programs and activities, (2) the measures that will be used to gauge performance, (3) the strategies and resources required to achieve the performance goals, and (4) the procedures that will be used to verify and validate performance information. These annual plans, issued soon after transmittal of the president’s budget, provide a direct link between an agency’s longer-term goals and mission and its day-to-day activities.Annual performance reports are to report subsequently on the degree to which performance goals were met. The issuance of the agencies’ performance reports, due by March 31, represents a new and potentially more substantive phase in the implementation of GPRA—the opportunity to assess federal agencies’ actual performance for the prior fiscal year and to consider what steps are needed to improve performance and reduce costs in the future. HHS has a broad and challenging mission that touches the lives of Americans from every economic stratum: enhancing the health and well- being of all Americans by providing for effective health and human services, and fostering strong, sustained advances in the sciences underlying medicine, public health, and social services. With a budget of $376 billion and a direct workforce of 59,000, HHS administers some 300 health and social programs, including Medicare, Medicaid, the State Children’s Health Insurance Program, Temporary Assistance for Needy Families, and food and drug safety. HHS’ programs often require operating components to coordinate with partners such as state, local, and tribal governments; grantees; and contractors. For example, HCFA shares responsibility with states for administering Medicaid—a program that provides health care to certain low-income persons. HCFA also monitors the approximately 50 Medicare contractors that pay claims for the program’s elderly and disabled beneficiaries and that establish local medical coverage policies. SAMHSA administers a grant program to states for treatment and prevention services for persons at risk of or actually abusing alcohol or other drugs. Finally, the Administration for Children and Families partners with states to provide support to needy children and transition their parents to work. This section discusses our analysis of HHS’ performance in achieving its selected key outcomes and the strategies it has in place—including human capital and information technology—for accomplishing these outcomes.We also provide information drawn from our prior work about the credibility of the agency’s performance information. While HCFA’s performance report and plan indicate that it is making some progress toward achieving its Medicare program integrity outcome, progress is difficult to measure because of continual goal changes that are sometimes hard to track or that are made with insufficient explanation. Of the five fiscal year 2000 program integrity goals it discussed, HCFA reported that three were met, a fourth unmet goal was revised to reflect a new focus, and performance data for the fifth will not be available until mid-2001. HCFA plans to discontinue three of these goals. Although the federal share of Medicaid is projected to be $124 billion in fiscal year 2001, HCFA had no program integrity goal for Medicaid for fiscal year 2000. HCFA has since added a developmental goal concerning Medicaid payment accuracy. One of HCFA’s key Medicare program integrity goals is to pay claims properly the first time. Therefore, HCFA has set the performance goal of reducing improper payments as a priority for Medicare. The central measure of progress for this goal is the rate of improper fee-for-service payments, which is now estimated by the HHS OIG. HCFA will assume responsibility for measuring this error rate in fiscal year 2002. HCFA reported meeting its fiscal year 2000 error rate target of 7 percent with a rate of 6.8 percent. HCFA reported that it did not meet its fiscal year 2000 goal to perform medical reviews of 100 million claims, and it is difficult to determine whether its revised goal is being continued. In its narrative, HCFA explained that it revised its goal to focus on improving the accuracy and appropriateness of medical reviews rather than simply to increase the number of reviews conducted. But later, in a chart describing changes in GPRA goals, HCFA noted that this goal was subsumed in a fiscal year 2001 goal to improve the effectiveness of program integrity activities through successful implementation of this and nine other initiatives contained in the Comprehensive Plan for Program Integrity. The current performance report and plan only cursorily mention the Comprehensive Plan initiatives but indicate that the goal will be reached in fiscal year 2001 and therefore will not be continued. HCFA discontinued two of its fiscal year 2000 performance goals for which it reported making progress. Although data will not be available until mid-2001 on its discontinued goal to decrease the improper payment rate for home health services, HCFA reported “expected achievement” of its 10-percent target as justification for dropping the goal. Nevertheless, this area remains on the HHS OIG’s list of major management challenges. HCFA also discontinued, with little explanation, the goal of increasing the ratio of dollars recovered through the audit process to dollars spent on auditing activities. It reported it dropped the goal because of data source concerns (which it did not discuss) but also said it is examining other ways to measure progress on this issue. HCFA nevertheless reported that it exceeded its fiscal year 2000 target for this goal. In commenting on a draft of this report, HHS noted that HCFA has discontinued certain goals because they are ultimately part of the overall error rate measure and do not reflect the accomplishments of HCFA’s overall program integrity efforts. We have previously reported on two general weaknesses that hinder HCFA’s efforts to ensure proper payments of Medicare claims: outmoded information systems and weak financial management procedures.Without effective systems, HCFA is not well positioned for sound financial or programmatic management. HCFA has taken steps to modernize its systems and strengthen its financial management but many challenges remain. For example, HCFA’s fiscal year 2000 performance report notes progress made in addressing weaknesses related to its financial information, such as improvements in controls over Medicare contractor data. However, HCFA is still not in compliance with the Federal Financial Management Improvement Act (FFMIA) and continues to have material weaknesses related to reliability and documentation of its financial information. HCFA acknowledges that its ability to fully address underlying financial weaknesses remains impaired because it lacks a fully integrated financial management system. Despite repeated instances of noncompliance and the need for an integrated general ledger system to address major financial management weaknesses, HCFA’s performance report does not include specific goals and targets for achieving compliance with FFMIA, a situation we also noted in prior performance plan reviews. While HCFA’s Chief Financial Officer Comprehensive Plan for Financial Management includes goals for developing an integrated general ledger system, this document and the related costs and resources for implementing the system are not referred to in HCFA’ s performance report or plan. HCFA’s strategies for achieving many goals related to minimizing fraud, waste, and error appear to be clear and reasonable. One important HCFA strategy is to establish new goals and revise existing goals that will enhance program integrity efforts. Recognizing limitations in the usefulness of the national Medicare error rate as a management tool, HCFA’s strategy is to develop a subnational error rate. Thus, it established a fiscal year 2001 goal of developing a separate error rate for each Medicare claims contractor and of implementing a provider compliance rate. It is also developing a method for estimating a fraud rate among providers within its contractors’ service areas. Finally, HCFA introduced a fiscal year 2002 goal intended to improve the provider enrollment process by ensuring that only qualified and legitimate providers are permitted to participate in Medicare. Because many of the baselines and measures for these new and revised goals are under development, HCFA’s intended performance regarding them is unclear. For example, HCFA’s fiscal year 2002 plan contains a developmental goal to improve its oversight of Medicare fee-for-service contractors. Its fiscal year 2002 target is to build on progress achieved in fiscal years 1999, 2000, and 2001. Similarly, HCFA’s fiscal year 2001 and 2002 plans include a developmental goal to help states conduct Medicaid payment accuracy studies in order to measure and ultimately reduce Medicaid payment error rates. The fiscal year 2001 target is to establish the feasibility of conducting pilot projects within states and, for fiscal year 2002, to assess the pilots initiated by two states. With respect to one fiscal year 2001 goal, HCFA notes human capital and information technology limitations but does not discuss strategies for addressing them. Thus, HCFA reports that because of limited resources and funding, it only audits a small percentage of providers regarding credit balance recoveries and that it lacks the database needed to track provider activity in this area. In prior reviews of this key outcome, we noted that HCFA did not adequately address the need for coordination with other organizations. While HCFA includes a brief coordination section in the individual goal narratives, it does not consistently provide details about planned coordination strategies. For example, one coordination strategy reads: “We will continue to work with our partners in conducting our everyday business of ensuring Medicare claims are paid properly.” HCFA’s performance report and plan indicate that it continues to make progress toward its outcome of ensuring that nursing home residents receive high-quality care but its focus on just 3 goals under this outcome is surprisingly narrow, given the broad range of its approximately 30 initiatives to improve the quality of care in America’s nursing homes. The lack of recognition of the Nursing Home Oversight Improvement Program initiatives is even more notable because the Senate Committee on Aging requested that HCFA establish benchmarks and track progress in implementing each of these initiatives. In commenting on a draft of this report, HCFA noted that its performance goals are not intended to be a comprehensive list of its performance measures. On the basis of interim data, HCFA reported that the prevalence of restraints used in nursing homes decreased during fiscal year 2000. This decrease represents the second consecutive year in which the goal of reducing the use of restraints was surpassed. Final data were expected after the publication of HCFA’s performance report. Regarding its second goal, HCFA reported, for the first time, the prevalence of nursing home residents suffering from pressure sores (bedsores) and established future- year performance targets for reducing their prevalence. HCFA reported making progress toward its third goal of modifying the survey and certification budgeting process to develop national standard measures and costs. Once developed, these standards can be used to more effectively price each state’s survey workload and to develop workload expectations for each state. However, when we compared HCFA’s current and prior-year plans for implementing this new budget methodology, we determined that the modification will likely take HCFA longer to implement than it planned. For instance, although its earlier plan indicated that its price-based methodology would be complete in fiscal year 2001, its current-year plan shows that future-year targets for this goal are yet to be determined. Nevertheless, in fiscal year 2001, HCFA said it will allocate budget increases to states with unit survey hours that do not exceed 15 percent above the combined national average for nursing home surveys. HCFA also eventually plans to use the standards for setting state performance measures to assess the quality of nursing home surveys performed by each state. As we noted in last year’s report, the critical step of assessing states’ performance could begin sooner if HCFA used existing data. For instance, one of HCFA’s regional offices has analyzed data for several years to help evaluate the performance of state survey agencies in its region in areas such as whether states vary the timing of surveys to ensure that nursing homes are unable to predict the date of their next survey. In a report issued in September 2000, we highlighted HCFA’s commitment to begin using data currently available to compile periodic reports on state performance and to supplement these reports with on-site work to assess state performance. Data inconsistencies we and the HHS OIG identified raise questions about the accuracy of HCFA’s information on the prevalence of restraint use and pressure sores. However, HCFA did not note any concerns about the reliability of the On-Line Survey and Certification Reporting (OSCAR) System database, nor did it discuss the concerns about minimum data set accuracy raised by the HHS OIG. Our prior reports on nursing home quality have noted the considerable variation across states in the reporting of nursing home deficiencies in OSCAR—a situation that suggests some states may be better than others at identifying problems. The HHS OIG recently found several problems related to the use of the minimum data set, including differences between information on residents contained in the data set and data maintained in the residents’ medical records. We also noted last year that HCFA recognized the need to be cautious with its use of data in the minimum data set until it assesses the data set’s accuracy and completeness. HCFA intends to award a contract this year to begin minimum data set validation work in 2002. In commenting on a draft of this report, HCFA said it found our discussion of this proposed validation contract inconsistent with our finding that it had not discussed concerns about minimum data set accuracy in its GPRA report. We believe that HCFA’s GPRA report should have acknowledged the proposed validation contract since it is directly relevant to a discussion of the reliability of data used to measure progress in achieving goals under the nursing home quality outcome. HCFA also expressed concern about the reliability of the HHS OIG’s findings on minimum data set accuracy. The fact that HCFA has a proposed validation contract suggests that it, too, has concerns about minimum data set accuracy. Despite its narrow focus on only three goals, HCFA’s strategies to achieve them are generally clear and reasonable. For example, to decrease the prevalence of pressure sores, HCFA is working to improve surveyors’ ability to assess residents’ conditions by conducting educational seminars for surveyors and adding a new investigative protocol to help surveyors detect pressure sores during a survey. It is also strengthening enforcement activities against homes that fail to prevent avoidable pressure sores. However, HCFA’s discussion of its strategy to ensure that nursing home residents are not unnecessarily restrained is incomplete. It notes that it relies on the state survey and certification process but does not discuss the role of outside groups, which also have sponsored a large number of provider and consumer education projects to demonstrate ways to reduce restraint use. To improve the overall management of the survey and certification process, HCFA’s strategy has been to conduct studies to identify significant differences in survey time and resource utilization among state survey teams. HCFA plans to research these variations, determine which have the strongest relationship to cost and performance, establish standard measures of cost and workload, and develop future survey and certification budgets on the basis of standard prices. HCFA’s new budgeting approach will address the importance of human capital by ensuring that states have an appropriate number of qualified surveyors. Disparities in staffing might have been a contributing factor to deficiencies in state oversight activities. During our 2000 review of HCFA’s implementation of the Clinton Administration’s nursing home initiatives, we noted that a number of states had hired additional surveyors to promote more timely complaint investigations as well as to ensure that nursing homes are inspected an average of every 12 months. Furthermore, although HCFA did not address this in its plan, it has taken steps to improve its information technology systems to enhance oversight of nursing home quality of care. For instance, HCFA is in the process of redesigning its OSCAR database to make it easier to generate analytical reports. Similar to last year, the Administration for Children and Families (ACF) reported that it lacked fiscal year 2000 performance data for 18 of the 26 measures associated with programs whose performance is critical in reaching this key outcome. As a result, we were unable to fully assess ACF’s progress. ACF largely attributes missing performance data to the time lag in receiving and validating data reports from its program partners, including states and localities. Specifically, no fiscal year 2000 performance data were reported for the Temporary Assistance for Needy Families (TANF), Child Support Enforcement, Child Care, and Refugee Resettlement programs. The limited performance data that were available in ACF’s report and plan indicate that its progress has been mixed. ACF reported that it achieved its target for four of the eight measures that had fiscal year 2000 performance data, including two measures related to the Developmental Disabilities Employment and Housing programs and measures related to increasing nondiscriminatory access to and participation in HHS programs. Target levels that ACF reported not meeting in fiscal year 2000 include two measures associated with increasing the number of HHS grantees and providers found to be in compliance with title IV in limited English proficiency reviews and investigations. For measures without fiscal year 2000 performance data, fiscal year 1999 performance data, which are now available, showed that 7 of 16 measures met or exceeded their targets and 2 measures came very close to meeting their targets. ACF may not be positioned to meet some future target levels, which appear to be set beyond what it can reasonably expect to achieve. Some measures, for example, have shown a recent decline and ACF may continue to not meet its targets for these in the future. These measures include (1) the earning gains rate and the employment retention rate under the TANF program; (2) the number of refugees becoming employed, the number of refugee cash assistance cases closed because the recipient became employed, and the number of 90-day job retentions under the Refugee Assistance program; and (3) the cost-effectiveness ratio of the process to collect medical and financial support under the Child Support Enforcement Program. Other measures, while showing recent improvement, may not meet their targets in the future, including the proportion of states that meet the TANF two-parent work participation rate of 90 percent and the number of children served by Child Care and Development Fund subsidies. In commenting on a draft of this report, ACF suggested that we favored a downward revision of the above targets. This was not the case. We recognize that ACF officials have encouraged programs, such as TANF, to intentionally set ambitious targets for some goals. Our comments were only meant to alert the Congress to the fact that certain goals may not be achieved in the future, information that ACF should have provided to assist congressional decisionmaking. For the examples cited, GAO relied on the multi-year data presented in ACF’s fiscal year 2002 performance plan, not on a single year’s performance as suggested by ACF. Few ACF strategies for achieving this outcome (1) were directly linked to specific performance that fell below fiscal year 2000 or 1999 target levels or (2) were aimed at overcoming ACF management challenges identified by us or the HHS OIG. Because it administers most of its programs in conjunction with states and/or other entities, ACF involves its partners in establishment of performance measures to help ensure their achievement. For example, other ACF strategies include providing technical assistance, disseminating the results of program evaluations and other research, and using rewards and penalties to improve performance. Finally, the fiscal year 2002 plan indicates that ACF will continue its ongoing evaluation of various aspects of welfare reform; in particular, ACF plans to evaluate performance measures related to increasing parental responsibility and increasing affordable child-care. ACF’s fiscal year 2002 plan offers no concrete strategy to overcome the time lag in receiving and validating performance data from program partners, and it generally does not report on the results of data validation efforts. ACF’s report acknowledges that such time lags make it difficult to provide a comprehensive summary of fiscal year 2000 performance until later in fiscal year 2001. ACF indicated it would develop a plan with HHS and the Office of Management and Budget in fiscal year 2001 for reducing the delay in the availability of state administrative data, where appropriate. Until this plan is developed and implemented, however, obtaining timely data for measures pertaining to helping individuals and families become self-sufficient will continue to impede assessments of ACF’s performance. In commenting on a draft of this report, ACF cited grant-reporting timeframes as a constraint in the timely availability of performance data. The fact remains, however, that ACF offered no concrete strategy to overcome the reporting time lags. By indicating that it will work with the Office of Management and Budget to reduce delays in the availability of administrative data, ACF underscores the need for more timely information. We do point out in our conclusions, however, that the issue of data lags may become less critical as trends emerge from data over longer time periods. ACF broadly discusses its human capital and information technology strategies in its fiscal year 2000 report and 2002 plan. ACF reported that it did not achieve an increase in the manager-to-staff ratio—its one human capital performance measure in fiscal year 2000—because of limits on hiring new staff and on reducing the number of managers already on board. However, ACF did meet its one performance measure related to information technology in fiscal year 2000 by replacing an outmoded “audit resolution tracking process” with an updated, integrated system. The fiscal year 2002 plan says little about how ACF intends to use human capital and information technologies to achieve this key outcome. In commenting on a draft of this report, ACF noted the use of human capital strategies such as training employees in marketing, negotiating, and consulting; using and improving automated technology, databases, and electronic communications; and implementing team-based work procedures. In the report, however, ACF does not tie such strategies to specific TANF-related measures with targets that might be set too high. Nor does it indicate how these strategies will help overcome problems, such as the 26 percent shortfall found in fiscal year 1999 in states that meet the TANF two parent families work participation rate. Similarly, we believe that ACF’s reference to its information technology investments presents a broad discussion of the role of information technology. We noted in January 2001 that sweeping changes brought about by welfare reform make better information systems and data collection necessary to improve program management and to help HHS measure its state partners’ performance in this area. In particular, we highlighted the importance of addressing the need for states to have access to information across states on individuals’ receipt of welfare to enforce the 5-year TANF time limit. Because adequate automated systems are critical to the success of welfare reform, we recommended that HHS work with other federal agencies, including the Departments of Agriculture and Labor, to address issues surrounding state automated data systems. ACF reported that it continues to work on correcting performance information and strengthening partnerships with states and grantees and that it gives high priority to creating mature data collection strategies. ACF also noted that it is working with other HHS components to assess unmet data needs and is committed to increasing its investment in data collection and information systems. The fiscal year 2000 report does not, however, offer targeted strategies for improving states’ automated systems, including the capacity to support enforcement of the 5-year TANF time limit. In commenting on a draft of this report, ACF pointed out that it (1) reported to Congress in 1997 that additional program authority and resources would be required to implement a tracking system to enforce TANF time limits; (2) developed a system that potentially will allow states to track the 5-year limit; and (3) is working with states to identify their automated system needs. The ACF report, however, did not contain adequate information on these strategies that would allow us to comment on the extent to which they address our past concerns. The performance reports and plans of HHS components indicate that they have made mixed progress toward achieving the 15 infectious disease prevention goals associated with this outcome and, in some cases, that data to measure progress are unavailable. The goals, many of which have multiple targets, include reductions in HIV, AIDS, other sexually transmitted diseases, and vaccine-preventable diseases. The five HHS components responsible for implementing infectious disease prevention goals are the Centers for Disease Control and Prevention (CDC), HCFA, the Health Resources and Services Administration (HRSA), the Indian Health Service (IHS), and the National Institutes of Health (NIH). Three of these agencies have goals to reduce vaccine-preventable diseases. Provisional data indicate that, for most targeted diseases, CDC met its goal of achieving a 90-percent vaccination rate for 2-year-olds. It provided a reasonable explanation of why the target for the diphtheria, tetanus, and pertussis vaccine was missed by a few percentage points. IHS also came close to meeting its children’s immunization completion rate. HCFA’s goal to increase the rate of fully immunized Medicaid 2-year-olds is state- specific, and measurement methods are still being developed. CDC, HCFA, and IHS generally did not yet have data to assess their progress in increasing pneumococcal pneumonia and influenza vaccination rates among the elderly, but interim progress data were cited. A data lag impedes the measurement of progress toward reducing the incidence of HIV and AIDS. Trend data indicate that CDC and HRSA are making progress in reducing perinatal transmission of HIV. Relying on process descriptions, NIH reports progress toward achieving its goal of developing an AIDS vaccine by 2007. CDC reported mixed progress toward its goals of reducing sexually transmitted diseases. In general, fiscal year 2000 data were not available at the time performance reports were published, but fiscal year 1999 data indicated in different target populations more progress toward reducing some sexually-transmitted diseases (congenital syphilis) than others (chlamydia). Data lags are common for many prevention goals, and it may be unrealistic to expect HHS to include complete data at the same time it issues its annual performance report and plan. As HHS continues to report its results, we will in turn receive more accumulated trend data to portray its progress. Data verification and validation remain important issues. The HHS agencies with infectious disease prevention goals tend to provide general information on the credibility of their performance measures and of their methodological approaches. For example, HRSA notes how the electronic submission of data, starting in fiscal year 2000, will address the reliability and validity concerns we raised previously. All of these agencies discuss measurement in the context of specific goals, but they do not always discuss why particular goals may be poorly measured. However, CDC’s report broadly discusses the measurement issues relevant to particular prevention goals, such as an account of HIV surveillance efforts. Similarly, HCFA describes the surveys it uses for assessing vaccination rates, including their limitations, and IHS explains the criteria it used to select its prevention indicators. While the components’ strategies for achieving some goals are clear and reasonable, they do not always include information about how they plan to attain unmet goals, and some strategies are not directly tied to goal attainment. With respect to specific goals or groups of goals, CDC often includes an informative discussion of its performance strategies. For example, it summarizes how it plans to eradicate syphilis in the United States. Furthermore, CDC states some of its goals in terms of the strategy to attain them, such as using “screening” and “treatment” in the goal descriptions for sexually transmitted diseases. HCFA includes a detailed discussion of strategies to foster higher immunization rates among seniors, including sponsoring outreach projects in health care venues and implementing routine procedures for providing certain immunizations without direct physician involvement. Its discussion of the goal of increasing the percentage of 2-year olds who are fully immunized focuses primarily on outreach and increasing enrollment as ways to effect the increases. The IHS report explains why it did not achieve certain goals but does not always articulate strategies for overcoming problems that impede progress. IHS noted, for example, that complex immunization schedules and incomplete tracking due to multiple sources of health care were a problem in meeting its goals. IHS’s report does discuss strategies for meeting its goals for childhood immunizations, but it discusses adult vaccination levels chiefly in terms of baseline and target rates, not in terms of vaccinating more people. Rather than identifying ways of vaccinating more people, however, it discusses establishing the appropriate baseline and adjusting the targets. Similarly, when CDC does not meet a goal, it does not always discuss specific strategies for attaining that goal. NIH’s strategies also are general rather than goal specific. Thus, its report highlights a number of broad strategies related to its overall mission, such as providing scientific leadership, facilitating the development of health- related products, and collaborating and coordinating with others. When the issues of human capital, information technology, the contributions of others, and program evaluations were included in the GPRA reports and plans of HHS components, their importance in helping to achieve goals was only discussed in general terms. Thus, while both HCFA and HRSA discuss human resources, they do not talk about them in the context of particular infectious disease prevention goals. Furthermore, IHS simply notes that human resource development is an essential component of its performance planning and management and provides some details about its activities in this area. Similarly, CDC, HCFA, and HRSA acknowledge generally the importance of information technology as it relates to their missions and goals. In contrast, IHS has specific measures addressing the development of improved automated data capabilities that are designed, in part, to improve performance measurement and GPRA compliance. While HHS components discuss the contributions of others by referring to “partnerships and coordination with other organizations,” IHS specifically notes its efforts to address HIV and vaccine-preventable infectious diseases through an agreement with CDC. Finally, regarding the use of program evaluations prepared by each component or others, the discussions usually are not related to specific infectious disease prevention goals. SAMHSA’s performance plan and report indicates that it has made some progress in achieving this outcome. While it continues to have problems collecting data for about half of its 80 goals, SAMHSA reported that it met or exceeded its target for nearly 90 percent of the goals for which it had data. Delays in reporting performance data were attributed to time lags in data collection, analysis, and reporting by states and the relatively large number of targets being measured. SAMHSA plans to have final data for most performance goals later in 2001. SAMHSA reported that it met many of the substance abuse and prevention treatment goals for which data were available. For example, SAMHSA indicated that it exceeded its target of increasing the number of states to 19 that voluntarily report critical outcome performance measures in Substance Abuse Prevention and Treatment Block Grant applications, as 24 states voluntarily reported at least partial outcome data. It also indicated that the number of states that incorporate needs assessment data increased from 26 states in fiscal year 1999 to 34 states in fiscal year 2000, meeting its fiscal year 2000 target. The incorporation of needs assessment data is critical for prevention planning, resource allocation, and selection of appropriate prevention strategies. Finally, SAMHSA reported that the percentage of states that use funds in each of six prevention strategy areas, which track progress in addressing the substance abuse prevention needs of populations, met the fiscal year 2000 target of 90 percent. SAMHSA gave a credible explanation for not meeting another goal related to the Substance Abuse Prevention and Treatment Block Grant program—continuing dialogue over the appropriateness of the targets—and indicated that the type and form of performance reporting will be decided by fiscal year 2002. SAMHSA’s performance report and plan indicate that it was far less successful in reporting important state-level performance data on the effectiveness of substance abuse treatment services for fiscal year 2000. States were to voluntarily report the percentage of substance abuse treatment clients who had reduced substance abuse and criminal involvement, had a permanent place to live, and were employed. However, fewer states than SAMHSA anticipated reported this information, and some states used different data collection methods to report information, raising questions about the reliability of the data. Consequently, SAMHSA dropped these goals and will develop new ones jointly with the states. Although development of goals will continue, client-related outcome data cannot be collected until SAMHSA complies with statutory requirements under the Children’s Health Act of 2000. The law requires SAMHSA to develop a plan, due by fiscal year 2002, that gives states flexibility in reporting outcome data based on a common set of performance goals, while preserving accountability. SAMHSA anticipates that the new goals will be approved in fiscal year 2003 and that collection and outcome measurement reporting will begin in fiscal year 2004. SAMHSA’s performance report does not provide assurance that all information contained in it is credible. Several performance measurements lack discussions of the specific procedures used to verify and validate data in the systems. For example, the description of data sources and validity of data supporting the measurement on treating adult marijuana users notes that the performance data were collected with standard instruments administered to clients by trained interviewers. Another measurement to develop and apply statistical models associated with client retention and outcomes under the Wrap-Around Services program asserts that project records documenting progress of statistical work are expected to be reliable. However, neither performance measurement discusses how and by whom the validity assessments are performed, the strengths and weaknesses of the data, or the external factors that may affect data reliability. In addition, SAMHSA did not report strategies for achieving several planned goals. For example, it cites measurable targets and time frames for achieving goals related to reducing the size of the drug treatment gap; increasing employment and education, and lowering illegal activity for graduates of treatment programs; and reversing the trend in marijuana use among youth. However, it omits details about how its prevention and treatment programs will attain these goals. Furthermore, SAMHSA describes the role of human capital management and information technology strategies but does not tie these activities to specific goals. For example, SAMHSA expects to complete a workforce plan in August 2001 that includes recommendations on ensuring that staffing levels are sufficient to manage program growth, maintain a well-trained workforce, and provide a high-quality work life. It also plans to develop benchmarks for best practices in government and nongovernment human capital management processes and incorporate them into its workforce plan. The performance plan also notes that SAMHSA has reorganized numerous functions and programs to streamline operations and conserve program management and other resources. SAMSHA also has invested in information technologies to enhance professional resources. Several communications and data management system improvements recently completed or under way include the redesign or conversion of SAMHSA’s Web site, intranet, and grants management system. Finally, SAMHSA’s report describes coordination with its partners and stakeholders, including the states, CDC, the Department of Veterans Affairs, NIH’s National Institute on Drug Abuse, and the Office of National Drug Control Policy, to determine priorities and help formulate certain goals. FDA’s performance report and plan indicate that it has made significant progress toward achieving this outcome. While performance data were unavailable for nearly 60 percent of its fiscal year 1999 goals, FDA reported results for 17 out of 19 goals in its fiscal year 2000 performance report. FDA reported that it met or exceeded 14 goals, did not meet 3 goals, and lacked outcome data for 2 goals. FDA reported making progress in meeting its goals for both the Human Drug and the Medical Device programs. For the Human Drug program, FDA noted that it had met several goals by streamlining its adverse drug event reporting system, providing the public with improved labeling information on over-the-counter drugs, and initiating collaborations with the scientific community on assessing product quality and manufacturing processes through the Product Quality Research Institute. This research institute is a first-ever partnership between the Human Drug program and industry scientists to conduct research in various aspects of the pharmaceutical development process. The objective is to streamline the drug development and approval process for industry and FDA while ensuring high product quality. The Human Drug program reported initiating seven working groups to address key drug regulatory issues, which surpassed its goal of beginning research on at least three projects identified by the Product Quality Research Institute. FDA included updated fiscal year 1999 data in its performance report, which showed that the Human Drug program exceeded most of its goals with respect to reviewing drug applications. Final performance data are not yet available for multiple targets under a goal on reviewing standard new drug submissions and generic drug applications. FDA expects to have these data by early 2002. According to FDA, late reporting of outcomes generally occurs because of time lags for reporting final data for premarket review goals. Regarding the Medical Device program, FDA reported that it exceeded targets for several goals on premarket device approval applications and surpassed a target on inspecting domestic medical device manufacturing establishments (at least 90 percent conformance with FDA requirements). Equally important was that at least 97 percent of mammography facilities met inspection standards, a target met in fiscal year 2000 and the previous fiscal year. The high percentage of facilities meeting standards is expected to enhance the quality of images, leading to more accurate interpretation by physicians and, ultimately, improved early detection of breast cancer. FDA’s report does not always instill confidence that its performance information is credible. For example, for the Human Drug program, it did not discuss the steps taken to verify and validate procedures for tracking the number of pediatric drug studies FDA requested under the Food and Drug Administration Modernization Act of 1997 (FDAMA) or inspections of drug establishments, including medical gas re-packers. Similarly, the Medical Device program did not discuss procedures used to verify and validate data in its medical device adverse event reporting system, which, as we reported in our last assessment, has experienced serious data management challenges related to the quality of reporting, processing, and reviewing reports. The report also did not describe procedures that were used to ensure data integrity for other databases, such as the Center for Devices and Radiological Health Field Data systems and the Field Accomplishments Tracking System. FDA’s strategies for achieving this outcome are clear and reasonable. When FDA did not meet a goal, it generally explained why and discussed strategies for improving future performance, including human capital strategies. For example, the Medical Device program did not achieve its goal of inspecting 22 percent of manufacturers of class II and III domestic medical devices in fiscal year 2000. According to FDA, the growth of the device industry, the complexity of devices, and dwindling resources have resulted in lower inspection coverage and higher violation rates. Initially FDA addressed this shortfall by focusing enforcement actions on high-risk devices. However, FDA now believes that resource limitations have put inspection coverage below critical mass, so it is requesting an appropriated funding increase for domestic inspections in fiscal year 2002. Inspection of foreign medical device manufacturers is also reportedly very low, and FDA is proposing a strategy to address the problem. While FDA managed to meet its goal of inspecting 9 percent of foreign manufacturers of class II and III medical devices, it expects the foreign workload to increase and inspection coverage to decline. The Mutual Recognition Agreement is one of the major initiatives introduced to assist in reducing FDA’s workload. However, FDA says it cannot maintain foreign inspections or successfully implement the agreement with current resources because it expects European Union assessment bodies will require extensive training. As a result, for fiscal year 2002, FDA is requesting budget authority for foreign inspections to cover the cost of training associated with the Mutual Recognition Agreeement. While FDA did not explain why the Medical Device program fell significantly short of its target on developing the Medical Device Surveillance Network system, it did propose a strategy to achieve its target. FDA plans to use fiscal year 2001 funding to increase user facility participation to target levels and extend the program to other types of facilities, such as ambulatory care surgical centers. For the selected key outcomes, this section describes major improvements or remaining weaknesses in HHS’ (1) fiscal year 2000 performance reports compared with its fiscal year 1999 reports, and (2) fiscal year 2002 performance plans compared with its fiscal year 2001 plans. It also discusses the degree to which HHS’ fiscal year 2000 reports and fiscal year 2002 plans address concerns and recommendations by the Congress, us, the HHS OIG, and others. For fiscal years 2001 and 2002, HCFA issued a single document integrating the appropriate performance report with the current year’s revised performance plan and the next year’s plan. With respect to the fraud outcome, neither HCFA’s fiscal year 1999 report nor its fiscal year 2000 annual performance report provided a comprehensive list of the relevant year’s performance goals, targets, and actual performance, making it difficult to fully track goals and measure progress. For example, earlier we discussed the difficulty in tracking HCFA’s goal on medical review. HCFA also acknowledged in both reports that timeliness of data is a challenge in its analysis of performance data. For example, data are incomplete for the goal of reducing the percentage of improper payments made under the Medicare fee-for-service program in the fiscal year 1999 report and, as mentioned earlier, for the goal of reducing the improper payment rate for home health services in the fiscal year 2000 report. HCFA has changed some of its performance goals and measures each year, which makes it difficult to track its progress in reducing fraud, waste, and error in Medicare and Medicaid. In both the fiscal year 2001 and 2002 annual performance plans, goals are dropped, revised, subsumed into other goals, and added. Two key weaknesses we identified in prior-year HCFA performance plans are that goals were not consistently measurable and that the strategies and resources needed to achieve these goals were not adequately addressed. These problems continue. In some instances, HCFA is still developing the baselines and appropriate measures. In others, HCFA states generally that the accomplishment of a goal is the target and does not explain in sufficient detail what its strategies are to ensure goal accomplishment. An improvement of the fiscal year 2002 plan over the prior plan is that the goal narratives, which are included, are generally more concise and in many cases include illustrative charts that indicate targets and previous performance. Both performance plans reflect HCFA’s efforts to strengthen coordination with other organizations and to enhance data verification and validations. In some areas of performance, however, sufficient detail is not consistently provided on coordination strategies—a problem we also noted with the prior year’s performance plan. Regarding data issues, HCFA cites and describes data sources for each goal and includes some of the particular data concerns or limitations. HCFA’s fiscal year 1999 and fiscal year 2000 annual performance reports clearly and consistently identify the results of its goals, targets, and actual performance with respect to nursing home services. The introduction of graphics in the fiscal year 2000 report was a positive step. While HCFA’s reports have a general discussion of data sources, they do not address known concerns about the validity of data used to measure progress. HCFA’s current plan addressed a concern we raised about the prior plan— the lack of measurable targets for two of the three goals. Thus, it established a baseline and targets for one goal and a fiscal year 2001 target for the other goal. However, as discussed in our June 30, 2000 report and emphasized earlier in this report, we question whether the goals in HCFA’s 2001 performance plan sufficiently address its overall performance in implementing about 30 nursing home quality-of-care initiatives that HCFA has had under way since 1998 under the Nursing Home Oversight Improvement Program. We noted in last year’s report that HCFA’s 2001 performance plan did not provide information on measuring its performance on the 30 initiatives. HCFA’s fiscal year 2002 performance plan is likewise silent on measuring such performance. There is little difference between ACF’s fiscal year 1999 and fiscal year 2000 performance reports. Both reports make effective use of tables to list performance goals, measures, and fiscal year target levels. Changes were made to the measures themselves, which we characterize below. While there is little substantive difference between ACF’s fiscal year 2001 and fiscal year 2002 performance plans in terms of strategies, the most recent plan added an executive summary, which provides a helpful overview of the document. Moreover, in some instances, the strategies in the 2002 plan for improving performance and program coordination are more fully developed. For example, the 2002 plan contains more projects for helping states produce desired TANF outcomes and strategies to better utilize human capital and information technology. The plan also discusses technical assistance to, and partnerships between, ACF’s Housing for the Developmentally Disabled program and the state Developmental Disabilities Councils. A strength of the fiscal year 2000 performance report is the inclusion of an updated performance data chart that was not available for the fiscal year 1999 performance report. In commenting on a draft of this report, ACF cited the inclusion of workplans that provide detailed strategies to achieve its targets in the fiscal year 2000 performance report. While not necessarily referred to as a priority workplan, the fiscal year 1999 performance report lists many of the same strategies in identical language. The number and wording of performance measures between the two ACF plans is similar. However, where target levels in the 2002 plan differed, they were generally set at higher levels. In many cases, the targets represented modest increases. Elsewhere, differences represented a significant change over a 1-year period. For example, the Child Care program’s goal of increasing the number of children served by Child Care and Development Fund subsidies rose from 2.1 million in fiscal year 2001 to 2.6 million in fiscal year 2002. The HHS Office for Civil Rights’ fiscal year 2002 performance plan, however, collapsed several objectives and measures into a single objective with fewer measures. Some of the new targets established, however, only provide an indirect indication of compliance and can actually mask the extent to which compliance is, or is not, achieved. In commenting on a draft of this report, OCR noted that it would continue to report tabular information that specifically identifies each of the outputs that make up the new composite measure. We remain concerned that the tabular information will be too general to directly assess compliance. ACF’s fiscal year 2002 plan continues the refocused human capital strategy it began in prior years. In light of its shrinking workforce and increasing workload, ACF refocused its human capital measure (manager-to-staff ratio) in fiscal year 2001 toward the development of a highly skilled, strongly motivated, and diversified staff. The single measure for this reorientation is “each ACF staff member participates in at least one Distance Learning or other training opportunity directly related to increasing his/her job skills.” However, the extent to which this measure captures ACF’s progress toward meeting its human capital goals remains to be determined. The fiscal year 2002 plan contains an information technology measure related to ACF’s continued implementation of an electronic grant-making system. The measure is to develop and implement a system that will allow ACF to capture and validate grant information submitted by grantees using the Web. The plan does not specify particular targets, such as a high percentage of applications validated by the system, reduced time to process an application, or grant awards made earlier in the year. ACF’s fiscal year 2002 plan does not fully respond to concerns we raised in our HHS GPRA review last year or those identified by the HHS OIG. We reported last year that ACF did not indicate how it planned to address the data-reporting lag. Although ACF included a somewhat fuller discussion of this matter in the fiscal year 2002 plan, we continue to believe that more specific actions and timelines are warranted. In addition, as discussed in appendix I, ACF makes little mention of how it intends to respond to several OIG recommendations and suggestions related to child support enforcement. In commenting on a draft of this report, ACF said that neither Office of Management and Budget nor HHS guidance directed them to respond to concerns expressed by the HHS OIG or GAO. However, our discussions with HHS officials responsible for coordinating the Department’s comments on our report suggest that HHS does take our analysis of its GPRA reports into account and attempts to correct shortcomings we have identified. For each HHS component reviewed with respect to infectious disease prevention, the fiscal year 1999 and fiscal year 2000 performance reports are similar. The agencies employed the same general format to summarize goals, targets, and actual performance, and in referring to an additional source of information (typically the budget justifications). This summary is generally accompanied by informative narrative that expands on the goal and related performance. For each of the relevant HHS components, the fiscal year 2001 and fiscal year 2002 performance plans are similar in content and organization. However, in both plans, the strategies and resources used to achieve goals were not always adequately addressed. Some components made revisions to or increased the number of their infectious disease prevention goals, and each provided a general discussion of plan changes. When goals or targets were revised, they generally provided rationales for these changes. None of the changes substantially strengthened or weakened the product. CDC, however, improved its fiscal year 2002 performance plan by making extensive revisions that more effectively communicated and linked its goals, measures, and targets with the strategies for achieving them. CDC also addressed most of the data quality concerns expressed by us and the Congress. As noted earlier, HRSA indicated that the electronic submission of data addresses reliability and validity concerns we had raised previously. Despite these specific data-quality improvements, the components do not always discuss why particular goals may be poorly measured. SAMHSA’s fiscal year 2000 performance report demonstrates little progress in overcoming a major weakness we noted in its previous report. As in last year’s report, it continues to rely on states to validate the information they reported in block grant applications for their goal related to the 20 percent Substance Abuse Prevention and Treatment Block Grant Prevention Set Aside program. While the current report notes that states must certify the accuracy of block grant data, SAMHSA does not describe states’ procedures for this or how SAMHSA project officers verify the states’ certifications. Another continuing limitation is SAMHSA’s failure to discuss the findings and recommendations of evaluations or how results were used to assess performance. Both we and the HHS OIG have recommended that SAMHSA perform such evaluations. In its fiscal year 2002 plan, SAMHSA continued its practice of highlighting changes and improvements over its prior-year plan. Thus, SAMHSA has adopted a more comprehensive approach to performance management by reporting on performance goals for all significant programs. Two key performance goals were added to its 2002 plan to increase SAMHSA’s ability to assess Substance Abuse and Treatment Prevention Block Grant customer satisfaction. SAMHSA is also working on initiatives to enhance the performance reporting process. These initiatives include establishing a requirement for states to report performance data in SAMHSA grant funding applications, and developing analysis plans for GPRA assessments to better manage programs and measure their effectiveness. However, the 2002 plan does not discuss SAMHSA’s efforts to verify the quality of the performance data reported by states—an observation that we made about the prior-year plan. We did find that when goals were added or modified for clarity, SAMHSA described the reasons and the results to be achieved from the change. In addition, when goals were dropped or modified, the 2002 plan stated that either the goal had been completed or revisions had been made to better focus the goal on outcomes. FDA’s performance reports have been consistently well organized, clear and concise. However, several goals in both the fiscal year 1999 and fiscal year 2000 performance reports lack adequate descriptions of the benefits to public safety and health attained by FDA’s performance. For example, both the Human Drugs and the Medical Device programs established goals of ensuring that inspections of domestic medical drug and device manufacturing facilities resulted in timely correction of serious deficiencies in accordance with FDA requirements. However, neither program in either report elaborated on the expected benefits beyond reporting attainment of the statistical goal. In contrast, FDA’s description of the mammography facility performance goal explained that inspections were expected to enhance the quality of images leading to the more accurate and timely detection of breast cancer. In its most recent plan, FDA has continued to improve its presentation. FDA made strong use of graphics interspersed with narrative to present its strategies and also included a helpful program overview. It also discussed its strategies for accomplishing goals and the consequences of not achieving them—overcoming a weakness we noted in the fiscal year 2001 plan. FDA’s fiscal year 2002 performance plan added new goals and slightly modified or reiterated others. New goals included increased inspections of medical device studies, which resulted from a heightened concern about clinical abuses; stepped up foreign inspections and expanded import coverage of all medical products to improve the safety of imported products; and enhanced surveillance of FDA-regulated products to prevent deaths and injuries related to the use of medical products. However, these new goals did not include baselines or concrete targets against which to measure progress. As noted earlier, FDA did not always address concerns we raised last year about the validity of performance information. We have identified two governmentwide high-risk areas: strategic human capital management and information security. Regarding human capital, HHS does not have departmental performance goals related to this high- risk area. Although it is engaged in workforce planning, HHS only briefly outlines this effort. Several HHS components, however, have such goals and measures in their plans, and some cite progress. Similarly, HHS has no departmental goals related to information security, but HCFA has established an aggressive program to address problems in this area. In addition, we identified five other major management challenges facing HHS. The performance reports and plans of HHS components included goals and measures directly related to four of these challenges. We found that HCFA is making some progress in addressing fraud in Medicare and Medicaid and that, while its goals are very narrow, it continues to make progress toward improving nursing home quality. With regard to the outcome of promoting self-sufficiency among the poor, we could not fully assess ACF’s progress because most goals lacked the necessary fiscal year 2000 performance data. For those goals with data, results were mixed. Only FDA’s outcome of ensuring prompt access to safe and effective medical drugs and devices demonstrated significant progress. For the fifth challenge—ensuring a well-designed and administered Medicare program—HCFA has a workforce planning goal to reduce the gap between the current and the targeted levels of skills and is using outside assistance to develop a comprehensive database documenting its employee positions, skills, and functions. On its own, HCFA cannot address other aspects of the human capital challenges we identified. In summary, we found that the HHS reports discussed making at least some progress for all seven major management challenges (including the two high-risk areas). Of the seven major management challenges identified by GAO, HHS’ performance plans had (1) goals and measures directly related to six of the challenges, and (2) goals and measures that indirectly related to one of the challenges. It is difficult to fully assess HHS’ progress in fiscal year 2000 toward achieving the outcomes we reviewed because lags in reporting performance data are common for many of its components such as ACF, CDC, SAMHSA, and FDA. In some cases, the delays are associated with the need to obtain performance data from states and local organizations. Some HHS components are working to improve the timeliness of data submitted by others and, in some instances, have reported trend data to show that progress is being made. For example, both ACF and CDC supplied fiscal year 1999 performance data in their current performance reports—data that were not available until this year. It is likely that ACF’s and CDC’s fiscal year 2001 performance reports will include fiscal year 2000 performance data that were not available this year. While it may not always be realistic to expect the availability of complete data at the same time annual performance reports and plans are issued, trends will become apparent as the number of performance reports grows with each passing year. The six HHS outcomes that were used as the basis for our review were identified by the Ranking Minority Member of the Senate Committee on Governmental Affairs as important mission areas and do not reflect the outcomes for all of HHS’ programs or activities. Given the outcomes selected by the Committee and the management challenges we examined, our review focused on about 150 goals discussed in the reports and plans of 10 components—Administration on Aging, ACF, CDC, FDA, HCFA, HRSA, IHS, NIH, OCR, and SAMHSA. We also reviewed the overall HHS summary, which highlights the reports of its operating components. As agreed, our evaluation was generally based on the requirements of GPRA, the Reports Consolidation Act of 2000, guidance to agencies from the Office of Management and Budget (Circular A-11, Part 2) for developing performance plans and reports, and previous reports and evaluations by us and others. We also relied on our knowledge of HHS’ operations and programs, our identification of best practices concerning performance planning and reporting, and our observations on HHS’ other GPRA-related efforts. We discussed our review with HHS officials, including the HHS OIG. We identified the major management challenges confronting HHS, including the governmentwide high-risk areas of strategic human capital management and information security, in our January 2001 performance and accountability series and high-risk update. The HHS OIG identified major management challenges confronting HHS in a December 2000 letter to the Congress. We did not independently verify the information contained in the performance reports and plans, although we did draw from our other work in assessing the validity, reliability, and timeliness of HHS’ performance data. We conducted our review from April 2001 through June 2001 in accordance with generally accepted government auditing standards. In commenting on a draft of this report, HHS said it found our report “fair, thorough, and comprehensive.” We have addressed specific comments that HHS suggested would increase the report’s accuracy as well as other technical comments in the corresponding sections of the report. HHS’ comments are included as appendix II. As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to appropriate congressional committees; the Secretary of Health and Human Services; and the Director, Office of Management and Budget. Copies will also be made available on request. If you or your staff have any questions, please call me at (312) 220-7600. Key contributors to this report were John Brennan, Bonnie Brown, Kim Brooks, Brett Fallavollita, Darryl Joyce, Don Keller, Clarita Mrena, Walter Ochinko, and William Thompson. The following table identifies the major management challenges confronting HHS, which include the governmentwide high-risk areas of strategic human capital management and information security. The first column lists the challenges that we and/or the HHS OIG have identified. The second column discusses what progress, as identified in its fiscal year 2000 performance reports, HHS components have made in resolving the challenges. The third column discusses the extent to which the fiscal year 2002 performance plans of the HHS components include performance goals and measures to address the challenges that we and the HHS OIG identified. We found that the performance reports of HHS’ components discussed the progress in resolving some challenges, but did not discuss progress in resolving the following: abuses in Medicaid payment systems; Medicare equipment and supplies; Medicare payments for mental health services; Medicare prescription drugs; oversight of prospective payment systems; and child support enforcement. Of HHS’ 19 major management challenges, its performance plans had (1) goals and measures that were directly related to 10 of the challenges; (2) goals and measures that were indirectly applicable to 2 of the challenges; and (3) no goals, measures, or strategies to address 7 of the challenges.

Summary: This report reviews the Department of Health and Human Service's (HHS) fiscal year 2000 performance report and fiscal year 2002 performance plan required by the Government Performance and Results Act of 1993 to assess HHS' progress in achieving selected key outcomes that are important to its mission. It is difficult to fully assess the HHS' progress in fiscal year 2000 toward achieving the outcomes GAO reviewed because lags in reporting performance data are common for many of its components such as the Administration for Children and Families (ACF), Centers for Disease Control and Prevention (CDC), the Substance Abuse and Mental Health Services Administration, and the Food and Drug Administration. In some cases, the delays are associated with the need to obtain performance data from states and local organizations. Some HHS components are working to improve the timeliness of data submitted by others and, in some instances, have reported trend data to show that progress is being made. For example, both ACF and CDC supplied fiscal year 1999 performance data in their current performance reports--data that were not available until this year. It is likely that ACF's and CDC's fiscal year 2001 performance reports will include fiscal year 2000 performance data that were not available this year. While it may not always be realistic to expect the availability of complete data at the same time annual performance reports and plans are issued, trends will become apparent as the number of performance reports grows with each passing year.