District of Columbia: Comments on Fiscal Year 2000 Performance Report
Released: 2001-06-08T00:00:00

Letter

Background

For each fiscal year, the District is required under P.L. 103-373 to develop and submit to Congress a statement of measurable and objective performance goals for all significant activities of the District government. After each fiscal year, the District is to report on its performance. The District’s performance report is to include: a statement of the actual level of performance achieved compared to each of the goals stated in the performance accountability plan for the year, the title of the District of Columbia management employee most directly responsible for the achievement of each goal and the title of the employee’s immediate supervisor or superior, and a statement of the status of any court orders applicable to the District of Columbia government and the steps taken by the government to comply with such orders.

Last year, on two occasions, we highlighted the challenges faced and progress made by the District in implementing a sound performance management system. In April 2000, we reported that the District’s first performance report, covering fiscal year 1999, lacked some of the required information. Specifically, the performance report did not contain (1) performance data for most of its goals, (2) the titles of managers and their supervisors responsible for each of the goals, and (3) information on any of the court orders applicable to the District government during fiscal year 1999. Also, it did not cover all significant District activities. In October 2000, we testified before the Subcommittee on Oversight of Government Management, Restructuring and the District of Columbia, Senate Committee on Governmental Affairs, that the District had made progress in defining clear goals and desired outcomes through its strategic planning efforts. However, we also said that there were still opportunities to more fully integrate various aspects of its planning process and ensure that performance information was sufficiently credible for decision- making and accountability.

Objectives, Scope and Methodology

Our objectives were to ascertain the extent to which the District’s fiscal year 2000 report was useful for understanding the District’s performance in fiscal year 2000 and the degree to which it complies with its statutory reporting requirements. To determine if the performance assessment itself could provide a useful characterization of the District’s fiscal year 2000 performance, we conducted a process evaluation. This included identifying the components of the process used to develop goals and measures, the agencies included, when the goals were revised, and whether the final goals were developed in a timely manner to allow valid performance assessment during fiscal year 2000. To determine if the report complied with reporting requirements, we compared the report contents to the legislatively mandated requirements. To acquire additional information and verify our findings, we interviewed a key District official responsible for coordinating the performance assessment.

We conducted our work from March through May 2001 at the Office of the Mayor of the District of Columbia in accordance with generally accepted government auditing standards. We did not verify the accuracy or reliability of the performance data included in the District’s report.

We provided a draft of this report to the Deputy Mayor/City Administrator of the District of Columbia for review and comment. Comments are reflected in the agency comments section of this report. In accordance with requirements established in P.L. 103-373, we consulted with a representative for the Director of the Office of Management and Budget concerning our review.

Fiscal Year 2000 Performance Report Underscores that Performance Management is a Work in Progress

The District’s performance report reflects a performance management process that led to goals continually changing throughout fiscal year 2000 as the District worked to improve the process. The performance plan (initial goals) for fiscal year 2000 was submitted to Congress in June 1999 along with the District’s budget. The District subsequently implemented what became an iterative approach for developing new goals and revising existing goals for about 20 “critical” agencies. That is, in addition to establishing initial performance goals, the District developed (1) agency strategic plans, (2) performance contracts, and (3) a Mayor’s Scorecard for each of the critical agencies. The performance goals generated as part of these efforts were developed during the period March 1999 through March 2000. These initiatives led to the development of the set of goals that the District considered as its final fiscal year 2000 goals for each of the critical agencies. For example, the Department of Health extensively revised its initial five goals. After going through various planning exercises, the department eliminated three of the initial goals, combined the remaining two goals under one broader final goal, and added seven completely new final goals. The initial goals of the noncritical agencies changed during the fiscal year, but without going through the same process as that for the critical agencies. The District official responsible for coordinating the fiscal year 2000 performance assessment estimated that between 30 to 40 percent of the noncritical agencies’ goals were revised over the fiscal year 2000 performance assessment period. Although, some goals were finalized earlier, the set of final fiscal year 2000 goals for all agencies, whose performance was assessed, were submitted to Congress along with the District’s fiscal year 2001 budget in June 2000.

One result of this process to redefine goals was that 54 percent of the initial goals were not used as final goals. For example, the Department of Motor Vehicles’ goal to seek out regular feedback on the level and quality of service was not used as a final fiscal year 2000 goal. Although the department developed several final goals related to improving customer service, such as wait times for vehicle registration, it did not continue the goal to obtain feedback directly from its customers. No explanation was provided in the report to explain why the goal was dropped or whether it had been achieved. Many of the remaining 46 percent of the original goals were significantly revised by the time the District issued its report, making it difficult to determine the degree to which the original goals were achieved.

District officials have indicated they plan to use an approach similar to fiscal year 2000’s for determining performance goals and measures in succeeding years. That is, they plan to define each fiscal year’s goals and measures during the fiscal year in which performance is being assessed. They expect that performance goals and measures will not stabilize into a consistent set until fiscal year 2003.

The District’s changing goals are reflected in its the fiscal year 2000 Performance Accountability Report, which provides information for three sets of performance goals. It provides information regarding the disposition of initial fiscal year 2000 goals. That is, the report indicates which goals made it into the final set used to assess fiscal year 2000 performance and which of the remaining initial goals, which were not considered by the District to be part of its final fiscal year 2000 goals, were nevertheless achieved. The second set of performance goals that are addressed in the report are those developed for the Mayor’s Scorecard. The goals in the Mayor’s Scorecard were developed to address priorities set by residents at the District’s Citizen Summit and the Neighborhood Action Forum. The last set of goals addressed in the report are the District’s final goals, which were included with the District’s fiscal year 2001 budget submittal to Congress in June 2000.

Lack of Information on the Extensive Revisions to Goals Limits Report’s Usefulness

The lack of information on the extensive revisions that the District made to its performance goals, measures, and plans, limit the usefulness of the subsequent performance report for purposes of oversight, transparency, accountability, and decision-making. Our review of federal agencies’ efforts to implement GPRA have shown that while it can be beneficial to periodically reassess and revise goals, it is also important that annual performance plans and reports provide clear information about the reasons for these changes when they occur. This information helps provide assurance that changes were intended to improve performance management rather than obfuscate weak performance; that is, that the changes were from developmental bias. Consistent with our findings, OMB’s guidance to federal agencies on the submission of GPRA plans and reports states that goals should be periodically modified as necessary to reflect (1) changes in programs, (2) agency capability to collect and report information, and (3) the importance and usefulness of any goal. All three of these factors are valid reasons to change goals. However, the District’s performance report does not indicate if any of these or other factors were a basis for the extensive revisions made to goals during fiscal year 2000. In addition, the report does not discuss steps taken to ensure that reported performance data were complete, that is, represented the entire fiscal year. For example, the Department of Parks and Recreation added a new goal to improve the safety, cleanliness, and accessibility of its facilities. However, it is not clear whether data on the District’s efforts to address safety findings (within 48 hours) was collected for the entire fiscal year. According to an official responsible for coordinating the performance assessment, the District cannot ensure that the reported data represented the entire year’s performance for any of the agencies; the official indicated one would have to go back and check with each individual agency to determine whether they were complete.

The concerns we raise are consistent with problems identified by the District Office of the Inspector General in a report published in March 2001. The Inspector General conducted a review to, in part, verify the data supporting the reported achievements regarding the fiscal year 2000 performance contracts and the Mayor’s Scorecard goals. One of the Inspector General’s conclusions was that agencies did not maintain records and other supporting documentation for the accomplishments they reported and that the Office of the City Administrator did not provide sufficient guidance to address that problem. In response to the Inspector General’s finding, the Office of the City Administrator said it recognized the need for standard procedures, and it plans to issue performance review guidelines by the end of the summer 2001.

Finally, regarding initial goals that were not carried over to the final set used to assess fiscal year 2000 performance, many are identified in the performance report as having been achieved. However, none of these goals had performance data provided for them. Therefore, the specific performance level at which these goals were met cannot be determined, that is, whether successful performance was marginal or otherwise. For example, the District had a goal of improving the response time for all legal services provided by the Office of the Corporation Counsel. The District’s report indicates that the goal was achieved, but because no data were provided, it is impossible to know precisely how and to what extent the agency improved its response time.

The Fiscal Year 2000 Performance Report Did Not Cover All Significant Activities of the District Government

The District’s performance report does not cover all significant District activities as required; thus, the performance report does not provide a comprehensive snapshot of the District government’s performance. For example, the report does not cover the performance of the District’s public schools, which account for more than 15 percent of the District’s budget. More important, the schools are responsible for a core local government function—providing primary education. The District’s performance report acknowledges this critical gap in coverage and says that subsequent reports beginning with the fiscal year 2001 report will more fully meet the statutory requirements.

The District Improved Compliance With Other Legislative Reporting Requirements

The District’s fiscal year 2000 Performance Accountability Report improved in two areas of compliance compared to last year’s report. First, the report provides the titles of program managers and their supervisors. The performance report is to include the title of the District of Columbia management employee most directly responsible for the achievement of each goal and the title of the employee’s immediate supervisor or superior. The District’s performance report provides the information for the final goals and goals contained in the Mayor’s Scorecard. This is an improvement over last year’s report, which contained no such information.

Second, the performance report also includes information concerning court orders assigned to the government of the District of Columbia during the year and the steps taken by the government to comply with such orders. Specifically, the District’s performance report provides the status for each of the 12 court orders by describing and identifying whether or not they were in effect in fiscal year 2000 and fiscal year 2001. For example, in the case of Joy Evans v. DC, the court required the District to improve the habilitation, care, and treatment for mentally handicapped residents. The report indicates that this court order was in effect in fiscal year 2000 and will continue to be in effect in fiscal year 2001. The report also provides information on the actions taken to comply with the orders. For example, in the case of Twelve John Does v. DC, the report clearly identifies the actions taken to address issues at the District’s Central Detention Facility. The report states that cell doors are being repaired, ventilation systems are being replaced, environmental matters are being corrected, and additional staff are being added to address security needs. In addition, the report states that the facility is scheduled to close on or before December 31, 2001. The information provided by the District on court orders is an improvement over last year when, due to an oversight in compiling its fiscal year 1999 performance report, the District failed to report on any of the applicable court orders.

Conclusion

The District’s fiscal year 2000 performance report is an improvement over the previous year’s in that it meets some of the statutory requirements that the previous report did not. However, the extensive changes that the District made to its fiscal year 2000 performance goals during the fiscal year undermine the usefulness of the resulting report because the District did not include critical information needed by Congress and other stakeholders. Such information, identifying how, when, and why specific goals were altered and the decision-making and accountability implications of those changes, is important to Congress and others so that they can have confidence in the validity and completeness of the reported performance data. In addition, the report does not cover all significant activities of the District government.

Sustained progress is needed to address the critical performance and other management challenges that the District faces. The District recognizes the shortcomings with its performance management efforts and has stated a commitment to addressing them. The effective implementation of the various initiatives underway in the District is vital to the success of the District’s efforts to create a more focused, results- oriented approach to management and decision-making—an approach that is based on clear goals, sound performance and cost information, and a budget process that uses this information in allocating resources.

Recommendations

To further strengthen the District’s performance management process and provide more useful information to its citizens and Congress, we recommend that the Mayor of the District of Columbia: Accelerate efforts to settle upon a set of results-oriented goals that are more consistently reflected in its various planning, reporting, and accountability efforts.

Provide specific information in its performance reports for each goal that changed, including a description of how, when, and why the change occurred. In addition, the District should identify the impact of the change on the performance assessment itself, including data collection and measurement for the reporting period. Include in each year’s accountability report the performance of all significant activities of the District.

Agency Comments and Our Evaluation

On May 31, 2001, we received e-mail comments on our draft report on behalf of the Deputy Mayor/City Administrator. He stated that overall, he concurred with our findings, appreciated the context in which they were presented, and acknowledged that additional work is needed to make the District’s performance management system serve the needs of its citizens and Congress.

The Deputy Mayor acknowledged that the extent of changes and the lack of discussion in the performance report about why specific goals were changed hinder comparison of the District’s performance against its initial goals. In addition, he said that using the goals that resulted from the development of agency strategic plans was more representative of the District’s performance during fiscal year 2000 than the initial goals. We agree with both of these points. Our central point, however, was that given the timing and extent of goal revision, and the absence of a discussion about those changes, the usefulness of the report for understanding performance as measured against the final goals, is limited.

The Deputy Mayor said that the information we reported on the timing of the final set of agency goals appears to exaggerate the amount of time that agency goals were in a state of flux—leading to the impression that all of the District’s goals were changing until June 2000. We report that goals for the critical agencies were finalized by March 2000 and that goals for other (noncritical) agencies were revised at other times; the District could not specify when these goals were finalized. It could only suggest that 30 to 40 percent of these agencies’ goals were revised. However, we revised our report to reflect that although some goals were finalized earlier, they were not submitted to Congress until June 2000.

In response to our recommendation that the District accelerate efforts to settle upon a consistent set of goals, the Deputy Mayor said that the District anticipates consolidating its goals during the fiscal year 2003 planning, budgeting, and reporting cycle. He further stated that goals for fiscal years 2001 and 2002 are likely to change as the District updates its agency-specific and citywide strategic plans in the summer of 2001. As we note in this report, it can be beneficial to periodically reassess and revise goals. However, it is critical that the District makes every effort to accelerate the process of settling upon its final goals early in a fiscal year to ensure that the performance assessment and report are meaningful.

The Deputy Mayor concurs with our recommendation that specific information should be provided in the District’s performance reports for each goal that changed. The Deputy Mayor also concurs with our recommendation to include in each year’s accountability report the performance of all significant activities of the District. He said that the District will seek to expand the coverage of its fiscal year 2001 report to more fully comply with its mandated reporting requirements. He also stated that although the District cannot compel independent agencies not under the authority of the Mayor (including the D.C. Public Schools) to report on performance, it plans to work with them in developing performance information.

We are sending copies of this report to the Mayor of the District of Columbia. Copies will be made available to others upon request.

Key contributors to this report were Kathy Cunningham, Chad Holmes, Boris Kachura, and Bill Reinsberg. Please contact me or Mr. Kachura on (202) 512-6806 if you have any questions on the material in this report.

Ordering Information

The first copy of each GAO report is free. Additional copies of reports are $2 each. A check or money order should be made out to the Superintendent of Documents. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Orders by visiting: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders by phone: (202) 512-6000 fax: (202) 512-6061 TDD (202) 512-2537 Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

To Report Fraud, Waste, or Abuse in Federal Programs

Web site: http://www.gao.gov/fraudnet/fraudnet.htm e-mail: fraudnet@gao.gov 1-800-424-5454 (automated answering system)