Department of Labor: Status of Achieving Key Outcomes and Addressing Major Management Challenges
Released: 2001-07-16T00:00:00

Letter

Background

GPRA is intended to shift the focus of government decisionmaking, management, and accountability from activities and processes to the results and outcomes achieved by federal programs. New and valuable information on the plans, goals, and strategies of federal agencies has been provided since federal agencies began implementing GPRA. Under GPRA, annual performance plans are to clearly inform the Congress and the public of (1) the annual performance goals for agencies’ major programs and activities, (2) the measures that will be used to gauge performance, (3) the strategies and resources required to achieve the performance goals, and (4) the procedures that will be used to verify and validate performance information. These annual plans, issued soon after transmittal of the President’s budget, provide a direct linkage between an agency’s longer-term goals and mission and day-to-day activities. Annual performance reports are to subsequently report on the degree to which performance goals were met. The issuance of the agencies’ performance reports, due by March 31, represents a new and potentially more substantive phase in the implementation of GPRA—the opportunity to assess federal agencies’ actual performance for the prior fiscal year and to consider what steps are needed to improve performance and reduce costs in the future.

The role of the Department of Labor is to promote the welfare and economic security of the nation’s workforce and ensure that workplaces are safe. To carry out its mission, the agency oversees a broad array of programs, from those that help students’ transition into the workforce to those that inspect the workplace or ensure the integrity of pension plans for retirees. These activities affect more than 100 million workers and more than 10 million employers.

Assessment of Labor’s Progress and Strategies in Achieving Selected Key Outcomes

This section discusses our analysis of Labor’s performance in achieving its key selected outcomes and the strategies the agency has in place, relating to human capital and information technology, for achieving these outcomes.

Workplace Injuries, Illnesses, and Fatalities

Labor reported making progress toward its outcome of reducing injuries, illnesses, and fatalities in the workplace. Labor reported that it met four of the six goals we reviewed under this outcome and substantially achieved another goal. In several cases, Labor exceeded the goal. For example, since fiscal year 1995, injury and illness rates declined by 20 percent in almost 68,000 workplaces where Labor intervened through efforts such as inspections, exceeding the target goal of 50,000 workplaces. In addition, for many of the goals, Labor presented data showing performance trends over a number of years. However, assessing progress for some goals was difficult because it was not always clear which fiscal year goal and target level was being assessed.

Like last year, Labor did not meet its fiscal year 2000 goal to reduce fatalities in the construction industry. However, Labor provides a plausible explanation for why external factors may have contributed to this shortfall—demand for more construction workers in a booming economy that likely resulted in a workforce with less experience combined with an increased pace and volume of work. Labor also provides clear strategies that appear likely to achieve the goal in the coming year, such as providing grants to develop, conduct, and expand safety and health training and partnering with local contractor organizations to raise safety awareness and bring training to Spanish-speaking contractors.

Another goal under this outcome was reported by Labor as “substantially achieved,” a new category used in the fiscal year 2000 report when at least 80 percent of the goal is attained. The goal is in two parts: to reduce nonfatal mining injuries and to reduce mining fatalities. Labor met the first part of the goal but did not meet the second part of the goal to reduce fatalities below the targeted 5-year average of 89 fatalities. There were 89 fatalities, bringing Labor very close to achieving the goal. Labor fully described actual progress toward the goal and identified the probable cause of the shortfall.

Assessing progress on three of the goals under this outcome was complicated because complete fiscal year 2000 data were unavailable. For these goals, Labor reported progress using fiscal year 1999 data and targets instead of fiscal year 2000 data and targets. For example, Labor’s fiscal year 2000 report stated that the fiscal year 1999 goal to reduce injuries and illnesses by 3 percent in industries characterized by high hazards was exceeded based on calendar year 1999 data. Labor could not assess achievement of its fiscal year 2000 goal with a target level of 7 percent because the necessary data would not be available until December 2001. Where complete data were not available, Labor followed OMB guidance calling for agencies to indicate when the data will be available, include performance information from the preceding year, and include the actual information in the subsequent report. However, tracking progress can be confusing because the fiscal year 2000 report lists only the fiscal year 1999 target level and omits the fiscal year 2000 target level.

Strategies to achieve goals under this outcome appear to be clear and reasonable and often used information technology, coordination with other federal agencies, and program evaluation to achieve the goals. None of the goals involved human capital strategies. Labor addressed strategic human capital management through agency-wide management goals rather than linking such strategies to specific programmatic goals or outcomes. Labor reported that it will use information technology, including Web-based courses, to provide safety and health training and interactive computer systems to help employers improve hazardous conditions in the workplace. In addition, Labor has been evaluating the effectiveness of the various interventions— such as inspections—it uses to reduce injuries and illnesses. Labor’s fiscal year 2002 plan calls for more program evaluations of its safety and health programs, policies, and specific standards to assess how effectively they reduce workplace injuries and illnesses. Other strategies mentioned in the fiscal year 2002 plan include coordination with other federal agencies, such as the Small Business Administration, to improve workplace safety and health. The importance of coordinating with other federal agencies is highlighted in our previous work on safety at hazardous material facilities. In October 2000, we reported that Labor and at least three other federal agencies were not coordinating requirements, such as training for hazardous material workers. We recommended various ways to improve coordination, such as determining whether agency agreements were effective and consolidating training requirements. Labor acknowledged points of overlap regarding worker training and did not object to the recommendations.

Some of the strategies described in Labor’s fiscal year 2002 plan were more clearly linked to achieving goals than in the 2001 fiscal year plan. For example, in our prior review of Labor’s fiscal year 2001 plan, we found that Labor did not sufficiently explain how certain programs, such as the “Consultation” and “Voluntary Protection Programs,” would reduce injury and illness rates. The fiscal year 2002 plan now explains how these strategies will be effective.

Labor’s 2002 performance plan retained many of the same performance goals from the fiscal year 2001 plan, and for 2002, Labor added a new goal measuring the effectiveness of voluntary, cooperative efforts between employers and Labor in reducing injuries, illnesses, and fatalities. For several of the goals it retained, Labor raised the target levels, allowing the agency to demonstrate progress toward achieving its overall strategic goals. This includes increasing the target level for its unmet goal to reduce construction industry fatalities from 11 percent in 2001 to 15 percent in 2002, even though Labor did not meet its previous target of 3 percent in fiscal year 1999 and 7 percent in 2000. Labor acknowledges that this is an ambitious target but indicates that recent program initiatives are expected to yield positive results in fiscal year 2001.

Job Training Participants Getting and Keeping Jobs

Labor reported making progress in achieving its outcome to ensure that job training participants get and keep jobs, with performance meeting many of the goals and often exceeding them. For example, Labor’s Women’s Bureau, in collaboration with other organizations, prepared 31,588 women for the workforce, exceeding the target level of 25,000. Labor presented data showing performance trends over a number of years for some of the goals. Table 1 shows the goals we considered under this outcome.

Labor reported meeting six goals and substantially achieving two goals. These two goals focused on worker retraining—one related to the Job Training Partnership Act (JTPA) dislocated worker program and another related to the Trade Adjustment Assistance (TAA) and North American Free Trade Agreement-Transitional Adjustment Assistance (NAFTA-TAA) program. For both goals, the target level for the average wage replacement rate was exceeded, but the portion of the goal addressing employment rates fell short. The target level to employ JTPA dislocated workers was 74 percent, but actual performance was 71 percent; the employment target level one quarter after JTPA program exit was 76 percent, but actual performance was slightly lower at 75 percent. Labor notes that it did not fully meet the goal because several states that were high performers under JTPA could not be included in the calculation for fiscal year 2000. These states no longer collect data under JTPA; they collect data under the newly implemented Workforce Investment Act (WIA).

For the goal related to the TAA and NAFTA-TAA programs, Labor reported that the target level to employ NAFTA-TAA program participants was 72 percent, but actual performance was about 65 percent. Labor attributes the shortfall to continued difficulties in getting complete and accurate data. The report discusses efforts to improve reporting and explains how the revised reporting system for fiscal year 2001 should help states provide more complete and accurate information. Our ongoing work on NAFTA- TAA may help shed light on the need for comprehensive performance data to determine the efficacy of training and job placement approaches. In our prior work, we recommended that Labor establish: (1) an effective performance measurement system for these programs, (2) procedures to allow Labor to certify workers within required time frames, and (3) more effective internal controls and oversight procedures to decrease the likelihood that ineligible workers are given benefits. Labor appears to be making significant efforts to respond to these recommendations. Labor is redesigning its data reporting system to match the system used in WIA and is developing a detailed manual on data reporting for states and localities. Labor has also established processing procedures to improve the time frame within which petitions are reviewed. Although Labor is reviewing data to identify ineligible recipients, the performance report does not identify what steps are being taken to develop internal controls.

Assessing progress toward meeting the goals under this outcome has been complicated by the transition from JTPA to WIA and the lag in available performance data for fiscal year 2000. Labor provides a reasonable discussion of its fiscal year 2000 performance and its plans for transitioning to WIA performance measures for the coming year. However, a clearer description of the challenges posed by this transition would help to assess whether its plans are sufficient to facilitate meeting next year’s goals. In addition, the three JTPA goals and the Job Corps goal were assessed using fiscal year 1999 target levels and not target levels set in Labor’s fiscal year 2000 plan. This lag in available data arises because the programs run on a program-year basis that begins 9 months after the fiscal year for which program funding is received. Finally, the goal to assist women in the workforce is very broad, and the performance measure fails to capture what services participants are receiving or accessing—a weakness we observed in our previous review of Labor’s performance report.

The strategies Labor uses to achieve the goals we reviewed under this outcome are largely clear and reasonable. Some goals use strategies related to information technology; one goal includes strategies related to human capital management. Labor provides a clear explanation for training needs when discussing its goal to award Youth Opportunity grants, explaining that experienced staff coaches will help train case managers, teach youth development strategies, and assist in building relationships with other organizations that serve youth. Yet other strategies are unclear. In the fiscal year 2000 report, when discussing efforts to improve data integrity in the JTPA disadvantaged adult program, Labor mentions “providing system-wide staff training, where necessary,” without adequately explaining what types of training might be required or how they will assess when and where training will be needed. Finally, our previous review noted that the discussion of this goal contained very little information on the quality of performance data used to assess it; however, this year’s report is improved because it provides information on Labor’s efforts to validate the program data it is using.

In its fiscal year 2002 performance plan, Labor continues to focus its efforts on increasing the earnings of participants in certain programs. Target levels are raised for several fiscal year 2002 performance goals, allowing Labor to demonstrate progress toward achieving its goals over time. However, the transition from JTPA to WIA will make performance on some goals difficult to compare. In spite of transition problems, Labor anticipates that the implementation of WIA will significantly improve its reporting capabilities and program outcomes. The incorporation of job retention measures in its fiscal year 2002 plan for WIA goals provides an important dimension of program performance, and our ongoing work on WIA performance measures may help Labor assess the effectiveness of the measures proposed for WIA.

Worker Benefits

Labor reported making progress toward its outcome of protecting worker benefits. Labor reported meeting all but one of its fiscal year 2000 goals related to protecting worker benefits. Table 2 shows the goals we reviewed for this outcome. For example, Labor reported making significant progress in its goal of reducing the average time frame to decide final benefit levels for participants in pension plans taken over by the Pension Benefit Guaranty Corporation (PBGC). By focusing its newly streamlined case processing strategies on the oldest cases during fiscal year 2000, PBGC reduced the average time frame from between 5 and 6 years to between 4 and 5 years. In its 2002 performance plan, Labor reports that its goal is to further reduce this timeframe to 3 years. In its annual report, Labor presented data showing performance trends over a number of years for several goals. The performance plan shows that it has increased target levels for some goals under this outcome, modified others, and discontinued one. However, in some cases, the discussion of the impact of these changes was incomplete or unclear, as was the discussion of the strategies Labor proposes to accomplish the goals.

Labor reported that it failed to meet only one of its performance goals in the area of protecting worker benefits—paying UI claims fairly and promptly—which it reported as “substantially achieved.” Labor established two separate criteria for this goal—one that judged whether eligibility was determined fairly and another that judged whether workers received their UI benefits on schedule. Out of a goal of 24 states, 23 met the criteria for determining eligibility fairly. Labor reports that an additional six states came very close. For the other portion of the goal— workers receiving their UI benefits on schedule—Labor reported that it met the goal of 47 states. Labor explains that the lower performance level for fairly determining eligibility reflects, in part, states’ focus on cost- saving efforts, such as telephone claims-taking. Furthermore, Labor has been tightening the underlying review process making it harder for states to meet the criterion. Despite missing the goal, Labor reported that it has increased the threshold for fiscal year 2002—30 states must meet the eligibility fairness criteria and 49 states must provide UI benefits on schedule. Labor provides reasonable strategies that appear likely to help achieve this goal, such as engaging in ongoing discussions with states, employers, and UI claimants to improve communication, identify issues, and promote input in the design of the programs.

One of the goals that Labor reported meeting in fiscal year 2000, through the efforts of the agency’s Pension and Welfare Benefits Administration (PWBA), was to increase by 2.5 percent both (1) the number of closed investigations of employee pension and health benefit plans where assets are restored (to 819) and (2) the number where prohibited transactions are corrected (to 301). During fiscal year 2000, Labor combined the data for pension and health benefits, but tracked separate goals for each element— assets restored and prohibited transactions corrected. Beginning in its fiscal year 2001 plan and continuing in its 2002 plan, Labor has created two separate goals—one for pensions and another for health benefit plans. This move could make it easier to track outcomes for the two separate benefit components. However, Labor aggregated goals for the individual elements into a single targeted goal that covers both assets restored and prohibited transactions corrected. In addition, it added two more elements to this aggregate goal—the number of cases in which participant benefits are recovered and the number in which plan assets are protected from mismanagement. As a result, it will be difficult to assess whether the goal is actually being met, because success in one element of the goal may obscure failure in another.

Performance goals for 2002 raise the target levels for several measures and include a new goal to promptly review applications for foreign labor certifications to ensure that aliens admitted to work will not adversely impact domestic workers’ wages or working conditions. The importance of this goal was highlighted in our earlier work on the H-2A program for agricultural workers and the H-1B program for highly skilled foreign workers. Additionally, in our major management challenges series, we cited weaknesses in strategic planning and resulting problems in organizational alignment that could affect the agency’s ability to protect worker benefits. Specifically, we found that having multiple agencies manage these programs resulted in program inefficiencies that confuse participants and delay the application process. We recommended ways to simplify and shorten the H-2A application process and better protect H-2A and domestic workers, including a recommendation that Labor collect data on its performance in meeting deadlines and use these data to monitor and improve its performance. We also recommended that Labor consolidate program authority for H-2A workers into a single agency within Labor to increase the effectiveness of enforcement. Labor has made efforts to implement these recommendations, and has completed some. In a September 2000 report, we suggested that the Congress consider eliminating Labor’s role in reviewing H-1B petitions because its review is limited by law and duplicates the efforts of the Immigration and Naturalization Service. Labor disagreed with this suggestion.

As a result of accomplishing a portion of the goal related to establishing prevailing wage rates for the construction industry as required by the Davis-Bacon Act, Labor reported that it is undertaking a reengineering effort to apply new information technologies and processes to the existing Davis-Bacon survey program to improve its accuracy, timeliness, and participation. Labor discontinued the goal for fiscal year 2002; but neither Labor’s fiscal year 2002 performance plan nor its fiscal year 2000 performance report discussed how it plans to monitor the law’s survey requirements in the interim.

In its fiscal year 2002 performance plan, Labor continues a goal for protecting workers’ benefits that we previously criticized—increasing by 1 percent the number of workers who are covered by a pension plan sponsored by their employer. In our review of Labor’s fiscal year 1999 performance report and fiscal year 2001 plan, we said that Labor could develop a more useful measure by focusing on the proportion of the total workforce covered by pensions. In addition, the number of workers with pensions is affected by a multitude of factors outside of Labor’s control and, as a result, it is unlikely that a valid relationship exists between Labor’s efforts and the outcome as measured. We have suggested that Labor use program evaluation methods to determine whether such a relationship exists and that Labor develop intermediate outcomes or output-oriented indicators that have a clearer relationship with the activities it undertakes. In its fiscal year 2002 performance plan and fiscal year 2000 report, Labor does not mention such efforts and continues to rely upon educating customers regarding the importance of retirement planning as the primary strategy for achieving this outcome. Our ongoing work to assess Labor’s retirement savings education efforts under the Savings Are Vital to Everyone’s Retirement Act (SAVER) of 1997 may shed light on this issue.

Labor’s fiscal year 2002 plan describes several program-specific strategies for achieving the outcome of protecting worker benefits. The strategies often appear to be clear and reasonable; none directly address human capital management. Some strategies emphasize the use of new information technologies, such as the use of new document imaging hardware to convert older paper files into electronic files. Other strategies in the plan were not specific or clear enough to be able to assess whether they would help Labor achieve the goal. For example, to improve the fairness and timeliness of UI benefits, Labor proposes developing and implementing a UI performance management system “…to enhance performance planning, facilitate performance achievement, and assess the effectiveness of program improvement efforts through capacity building, technical assistance, best practices, and other key initiatives.” Similarly, in last year’s review, we noted that this strategy did not sufficiently explain how Labor will achieve this goal.

Labor’s 2002 performance plan does not discuss several strategies it is undertaking to serve pension plan participants more effectively and efficiently. In prior work, we found weaknesses in PBGC’s contract management—such as inadequate links between contracting decisions and long-term strategic planning and a lack of centralized performance data to monitor contractor performance—and we made several recommendations to address these problems. For example, we recommended that PBGC undertake a comprehensive review to better link staffing and contracting decisions to its long-term strategic planning process so that it will be prepared for future workload changes and that it compile performance data centrally to better manage contractors. PBGC has begun work on these efforts, but these efforts are not reflected in the performance plan because Labor does not consider them to be a key strategy in achieving this goal.

Transitioning Individuals From Welfare Dependency to Self-Sufficiency

Labor reported that it made progress toward the key selected outcome of transitioning individuals from welfare dependency to self-sufficiency. For this year, Labor’s single goal under this outcome has incorporated a measure for job retention and one for increased earnings, which are critical dimensions of program performance. This is an improvement over the fiscal year 1999 measure, which did not address job retention and targeted wage rates. Labor reported that it exceeded the goal with 84 percent remaining in the workforce for six months with an average earnings increase of 59 percent, surpassing the target levels of 60 percent that remained in the workforce with an average earnings increase of 5 percent. Labor acknowledged data limitations in the earnings increase rate of 59 percent, reporting that this rate may be inflated due to inconsistent reporting by Welfare-to-Work grantees. It is working to improve reporting methods by grantees and will issue revised reporting instructions. This is an improvement over Labor’s fiscal year 1999 report which, as noted in our previous review, did not provide information on the quality of performance data.

Although Labor reported that it exceeded the goal, it did not provide detailed information on how many participants have been served, which would clarify the impact the program has had to date—a weakness we observed in our previous performance review. At the same time, Labor acknowledges that participation levels lag and has requested and received congressional approval to extend the period over which grantees may expend their funds. Although this time period has been extended, the Welfare-to-Work grant program was only funded in fiscal years 1998 and 1999 and no new funding is expected. In future plans, Labor might consider discussing the limited nature of the program and providing long- term strategies for serving this population when funding expires in future plans.

In general, Labor’s strategies for meeting the goal in fiscal year 2002 are plausible. None of the strategies involved the use of human capital management. Labor plans a pilot program to fund employers to upgrade the skills of Temporary Assistance for Needy Families (TANF) participants, allowing the employers to then backfill entry level positions with Welfare-to-Work participants. In addition, Labor will target development of whole family programs to help participants, primarily fathers, focus on their children and help the custodial parent access community resources to achieve self-sufficiency. The strategies also involved program evaluation to assess the effectiveness of Welfare-to- Work initiatives, including those undertaken by both formula and competitive grantees. A set of recently issued evaluation reports tell us that program implementation has advanced, but that participation remains low and the projected scale of the programs continues to be modest.

Labor’s report does not explicitly state why performance goals have been revised; however, it appears that Labor is seeking increasingly higher levels of performance for this goal, allowing the agency to demonstrate progress toward achieving its overall strategic goals. Labor has raised the target to 67 percent of participants who will be employed for 2 consecutive quarters after placement, with an average earnings increase of 7 percent.

Comparison of Labor’s Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan With the Prior Year Report and Plan for Selected Key Outcomes

For the selected key outcomes, this section describes major improvements or remaining weaknesses in Labor’s (1) fiscal year 2000 performance report in comparison with its fiscal year 1999 report and (2) fiscal year 2002 performance plan in comparison with its fiscal year 2001 plan. This section also discusses how the agency’s fiscal year 2000 report and fiscal year 2002 plan address concerns raised by GAO and OIG.

Comparison of Labor’s Performance Report for Fiscal Years 1999 and 2000

Labor made improvements to its fiscal year 2000 performance report from its fiscal year 1999 performance report. In our review of Labor’s fiscal year 1999 report, we identified as a weakness the lack of information the agency presented concerning the quality or credibility of performance data relative to the performance goals. Labor’s fiscal year 2000 report greatly improves its presentation of such information by identifying performance data challenges, steps it will take to verify and validate its performance data, and the implications of data limitations for assessing performance. For example, the agency reported that it completed an audit of the validity and reliability of its workplace injury and illness data and found the data to be reasonable and accurate. The agency will improve the audit program by, among other things, maintaining a standard sampling universe so that trends in the universe estimates can be tracked. As previously noted, Labor also recognized that Welfare-to-Work data may be subject to error due to misinterpretation of reporting guidance by grantees, resulting in overestimates of increased earnings rates for participants. Labor describes a major effort it will undertake to gauge the extent of inaccurate reporting, correct these reports, and improve the reporting instructions and format.

In our previous review, we observed that the fiscal year report did not directly or comprehensively address its progress in resolving major management challenges identified by GAO and Labor’s OIG. The fiscal year 2000 report makes significant improvement by including a summary of the most serious management and performance challenges identified by OIG, as required by the Reports Consolidation Act of 2000, and discussing strategies Labor will use and progress it has made in resolving these challenges.

Labor more clearly identified when it reported on goals that were assessed using older data and target levels when current fiscal year 2000 data were not available. Specifically, last year’s report listed goals in the appendix as met or not met, but did not clearly indicate on which fiscal year goal and target level the assessment was based. This year’s report clearly indicates the year of the data and target level on which the goal was assessed. In addition, Labor’s fiscal year 2000 report also followed OMB guidance calling for agencies to identify the goals for which complete data are not available, indicate when the data will be available, include performance information from the preceding year, and include the actual information in the subsequent report. However, tracking progress is still difficult because the fiscal year 2000 report does not always list the fiscal year 2000 target level when the agency had assessed a goal using older data and target levels.

One concern about Labor’s fiscal year 2000 report is that Labor added the category “substantially achieved” to its assessment categories “met” and “not met” when at least 80 percent or more of the targeted goal was attained. Although OMB allows agencies to use a third category to assess goals when the difference between the target level and actual performance is slight, Labor should be aware of the potential risk in using the category substantially achieved. Relying on this indicator could mask an understanding of actual progress in achieving goals and affect an agency’s vigilance in its efforts to achieve progress. It could give an impression of successful performance across all goals or on a specific goal when actual performance might be as much as 20 percent below the target. OMB guidance calls for agencies to compare actual performance with the projected performance level set out in the plan and explain why the goal was not met. We found that, for most of the goals we reviewed that Labor assessed as substantially achieved, the agency was very close to meeting the goal, reported its actual performance against planned performance, and explained the shortfall.

Another concern with this year’s report regards Labor’s presentation of its future plans. Last year’s performance report included a section entitled “Goal Assessment and Future Plans” that highlighted, among other things, strategies that would be used to achieve a goal in the future. This year’s fiscal year 2000 report eliminated information on future plans, calling the section “Goal Assessment.” Labor officials told us that, due to the transition in the administration and uncertainty about the agency’s future direction and goals, they determined that any discussion of future plans would be premature and could be subject to change. We understand Labor’s concern; however, we are hopeful that future performance reports will return to the format included in the fiscal year 1999 report because it provides more linkage between the performance report and subsequent performance plans and reports.

Comparison of Performance Plans for Fiscal Years 2001 and 2002

We found a limited number of substantive changes between Labor’s fiscal years 2001 and 2002 performance plans. Generally, the fiscal year 2002 plan continues to provide a clear progression toward intended outcomes. For example, for several goals aimed at reducing injuries, illnesses, and fatalities, only the numeric targets have changed (from 11 percent to 15 percent from the baseline in selected industries and occupations) and not the goals themselves, allowing Labor to demonstrate progress toward achieving its outcomes. In addition, the fiscal year 2002 plan now links budget authority and outlays to both the strategic and outcome goals. This will provide a basis for assessing how the resources are contributing to accomplishing the expected levels of performance.

In our previous review of Labor’s fiscal year 2001 performance plan, we observed that Labor did not adequately discuss the steps it will take to verify and validate performance data, as well as the implications of data limitations for assessing performance. Labor significantly improved its fiscal year 2002 performance plan by providing more detailed information and by expanding its discussion to include data systems that were not cited in the 2001 plan. However, the plan still lacks sufficient information on how Labor will address the implications of certain data limitations. For example, employment and training programs, such as WIA, are increasingly relying on UI data to measure outcomes. However, these data suffer from significant time lags, are not readily shared from state to state, and do not cover all employment categories. While the 2002 plan cites the anticipated completion of a data system to track performance for key employment and training programs, it does not address the implications of these UI data limitations for assessing program performance or how these factors will be mitigated.

The fiscal year 2002 performance plan could be improved by revising its information technology goal. Because the goal is broad and not directly measurable, it must include performance indicators that are specific, measurable, and related to the goal, according to GPRA, OMB Circular A-11, and related guidance. In addition, goals and indicators should be objective and quantifiable or defined in a way that allows an accurate determination to be made of how actual performance compares to the goal. Labor developed one broad performance goal for its outcome of improving organizational performance and communication through information technology: Improve automated access to administrative and program systems, services, and information. The goal uses eight indicators to measure and assess progress toward the goal. But these indicators are not always specific enough to be able to assess the agency’s plans or progress toward achieving its goal and do not allow for consistently accurate measurement of performance. For example, it is not clear that replacing the Remote Terminal Network (RTN)—one of the indicators— will help achieve the goal and improve access to information technology. Furthermore, it is not clear how the indicators will be aggregated to determine whether the goal has been achieved, which is especially important when the broader goal is not quantifiable or directly measurable.

Labor’s Efforts to Address Its Major Management Challenges Identified by GAO

This section discusses Labor’s efforts to address major management challenges identified by GAO. This includes two governmentwide, high- risk areas—strategic human capital management and information security— and three major management challenges facing Labor that were identified in our performance and accountability series.

Regarding strategic human capital management, we found that Labor’s fiscal year 2000 performance plan had goals and measures related to strategic human capital management, and the fiscal year 2000 performance report explained its progress in achieving these goals. Labor’s 2002 performance plan includes a goal to measure key aspects of strategic human capital management that had not been addressed in the 2000 report or plan. Furthermore, the report has helpfully divided management goals into human capital, financial, and information technology, making it easier to track progress from year to year and more clearly focus efforts on these areas.

However, the fiscal year 2002 performance goal that addresses key aspects of strategic human capital management—similar to Labor’s information technology goal—is overly broad and vague and uses performance indicators that do not sufficiently measure the goal. The goal—The right people in the right place at the right time to carry out the mission of the department—encompasses a wide range of human capital issues, from recruitment and organizational alignment to skills training and workforce diversity. Given a goal so broad, care should be taken to develop indicators that can gauge whether the goal has been achieved. When goals are not measurable, OMB guidance states that the performance indicators should set out specific, measurable values related to the goal that will help determine goal achievement. Unfortunately, the number and nature of indicators that Labor proposes do not appear to provide all the data needed to fully assess goal achievement. Not all aspects of the goal are adequately addressed through the indicators Labor proposes, and those that are proposed are not always clearly linked to the goal and do not allow for accurate assessment of performance. For example, several indicators rely on opinion surveys to measure performance, which may involve subjective considerations or judgments. In addition, the indicators do not capture agency strategies that will be used to achieve the goal. For example, although the 2002 performance plan describes agency efforts to identify future workforce needs—such as skill gaps—and ensure the development and skills of its workers, the indicators do not reflect these efforts. Finally, it is not clear how the indicators will be aggregated to determine whether the goal has been met.

With respect to information security, we found that, while Labor’s fiscal year 2000 performance plan did not have goals and measures related to information security, it described the agency’s efforts to address this challenge, and the fiscal year 2000 report described its progress on these efforts. The agency identified information security as a long-term management initiative in its fiscal year 2002 performance plan rather than a specific goal. The plan states that Labor has developed (1) an information security program that is being integrated into programs throughout the agency, (2) security plans for all Labor’s agencies, and (3) a security awareness program to train all employees.

In addition to these governmentwide management challenges, we identified three major management challenges facing Labor: increasing the employment and earnings of America’s workforce, protecting the benefits of workers, and fostering safe and healthy workplaces.

These challenges are generally similar to the key outcomes selected for Labor. Therefore, goals and measures included in the fiscal year 2000 report and 2002 plan that address these challenges are discussed under the outcomes.

Conclusions

In general, Labor appears to be making progress in achieving the key outcomes. Labor has increased its target levels for some goals for fiscal year 2002 and generally provided sound strategies for achieving these new targets. We continue to have concerns about some of the measures Labor uses. We are most concerned about the way in which Labor addresses two of its management challenges—information technology and strategic human capital management. Given the breadth of these goals, goal achievement cannot be fully assessed with the performance indicators Labor proposes. Without better indicators that more accurately and comprehensively measure performance toward the goal, Labor will be unable to fully assess its progress in these areas.

Recommendations

To ensure that progress toward performance goals can be accurately and fully assessed, and that performance indicators effectively measure the goal, we recommend that the Secretary of Labor revise its performance goals regarding strategic human capital management and information technology so that the performance indicators effectively capture efforts to achieve the goal.

Scope and Methodology

As agreed, our evaluation was generally based on the requirements of GPRA, the Reports Consolidation Act of 2000, guidance to agencies from OMB for developing performance plans and reports (OMB Circular A-11, Part 2), previous reports and evaluations by us and others, our knowledge of Labor’s operations and programs, GAO identification of best practices concerning performance planning and reporting, and our observations on Labor’s other GPRA-related efforts. We also discussed our review with Labor agency officials in the Office of the Assistant Secretary for Administration and Management. The agency outcomes that were used as the basis for our review were identified by the Ranking Minority Member of the Senate Committee on Governmental Affairs as important mission areas for the agency and generally reflect the outcomes for all of Labor’s programs or activities. For these outcomes, we identified goals that we believed to be clearly linked to the outcomes. The major management challenges confronting Labor, including the governmentwide, high-risk areas of strategic human capital management and information security, were identified by GAO in our January 2001 performance and accountability series and high-risk update and were identified by Labor’s OIG in December 2000. We did not independently verify the information contained in the performance report and plan, although we did draw from other GAO work in assessing the validity, reliability, and timeliness of Labor’s performance data. We conducted our review from April through June 2001 in accordance with generally accepted government auditing standards.

Agency Comments

We provided a draft of this report to Labor for its review and comment. Labor’s comments are in appendix II. Labor generally agreed with our findings and was pleased with our acknowledgement of its efforts to improve explanations about data quality and major management challenges. The agency also agreed with our recommendation to revise the performance indicators used to measure its progress toward achieving its goals on information technology and strategic human capital management. With regard to the issue we raised about using a third category— substantially achieved—Labor maintained that it will continue to use this category, but only when performance is very close to achieving the goal. We incorporated Labor’s comments and clarifications where appropriate.

Labor commented on our observation that aggregating goals for individual elements regarding PWBA’s investigations of health and welfare plans into a single goal could make it difficult to assess performance. Labor plans to examine this observation further, but is concerned that establishing multiple, separate indicators would result in the selection of cases most likely to achieve the best results rather than selection of the most significant cases. Labor should be aware, however, that in using the current, aggregate goal, case selection may similarly result in the selection of cases from a range of categories for their potential to achieve the overall goal. Labor also commented that revising the goal to increase the number of workers covered by a pension plan to measure the percentage of the workforce, as we suggested, could distort the measure’s reliability. We disagree that revising the goal may hamper reliability, however, we are pleased that Labor said it plans to explore ways to improve the measure and evaluate which strategies most effectively expand coverage levels. Finally, in response to our concern that Labor lacked sufficient information to address certain data limitations, particularly the use of UI data, Labor provided helpful information that illuminates its strategies. We encourage Labor to incorporate this information in future plans.

As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to appropriate congressional committees; the Secretary of Labor; and the Director, Office of Management and Budget. Copies will also be made available to others on request.

If you or your staff have any questions, please call me at (202) 512-7215 or Dianne Blank at (202) 512-5654. Key contributors to this report were Ronni Schwartz, Abbey Frank, Mikki Holmes, and Bonnie McEwan.

Appendix I: Observations on the Department of Labor’s Efforts to Address Its Major Management Challenges

The following table identifies the major management challenges confronting Labor, which include the governmentwide, high-risk areas of strategic human capital management and information security. The first column lists the management challenges that GAO and Labor’s OIG have identified. The second column discusses what progress, as discussed in its fiscal year 2000 performance report, Labor made in resolving its challenges. The third column discusses the extent to which Labor’s fiscal year 2002 performance plan includes performance goals and measures to address the challenges that GAO and Labor’s OIG identified. We found that Labor’s fiscal year 2000 report discussed its progress in resolving many of these challenges. Of the agency’s 15 major management challenges, its performance plan had (1) goals and measures that were directly related to 9 of the challenges, (2) goals and measures that were indirectly applicable to 1 of the challenges, and (3) no goals and measures related to 5 of the challenges, but discussed strategies to address them or stated that these challenges, if not already resolved, will be resolved by the end of fiscal year 2001.

Appendix II: Comments from the Department of Labor