Workplace Safety and Health: Further Steps by OSHA Would Enhance Monitoring of Enforcement and Effectiveness
Published: 2013-01-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: OSHA is responsible for overseeing occupational safety and health for more than 130 million workers. In about half the states, OSHA sets and enforces compliance with safety and health standards. The remaining states set and enforce their own standards under OSHA-approved plans. In fiscal year 2010, OSHA strengthened its monitoring of staterun programs following a dozen worker deaths in one of those states. Questions have since been raised about how closely OSHA monitors its own enforcement efforts. GAO examined 1) how OSHA's monitoring of its own and state enforcement efforts compares, and 2) recent steps OSHA has taken to evaluate the effectiveness of federal and state enforcement efforts. GAO reviewed OSHA's monitoring policies and procedures and relevant federal laws and regulations; analyzed federal and state audits; visited three OSHA regional offices; and interviewed OSHA officials and other experts.
section_title: What GAO Found
paragraphs: The Department of Labor's Occupational Safety and Health Administration (OSHA) provides more frequent monitoring and more consistent guidance for its federal enforcement activities than for state enforcement activities. More specifically, OSHA regional officials review performance reports on federal activities at least every other week but review reports on state-run program activities quarterly. In addition, OSHA's guidance for audits of its regional and area offices is more consistent than the guidance for its audits of state-run programs. Guidance for audits of its offices requires that regional offices conduct a comprehensive audit with on-site review of inspection case files at least once every 4 years and other audits focused on more specific activities in all other years. In contrast, guidance for regional office audits of state-run programs changes from year to year and does not include a regular schedule for comprehensive audits with on-site case file reviews. While the frequency of OSHA's monitoring of state-run programs is necessarily different because of the independent enforcement authority of participating states, OSHA's lack of consistent guidance for audits of these state-run programs may allow enforcement deficiencies to go undetected, increasing the risk of worker injuries, illnesses, or death. In addition, there is little participation by OSHA's national office in comprehensive audits of its regional offices despite a 2010 directive to do so. As a result, OSHA cannot ensure that the results of regional audits are impartial.
OSHA is taking steps to better assess the effectiveness of both its federal enforcement efforts and of state enforcement efforts, but it is often not clear how these steps will help OSHA demonstrate what efforts result in better outcomes for workers, such as reduced worker injuries, illnesses, and fatalities. For example, OSHA recently revised some of the measures it uses to assess staterun programs by adding acceptable ranges of performance. However, the revised measures still largely focus on outputs--such as the average number of violations per inspection--rather than outcomes. OSHA is also conducting studies to examine the results of specific enforcement activities, including one designed, in part, to evaluate the effect of OSHA providing additional educational support to employers. While tracking progress in meeting performance goals-- such as the annual number of conducted inspections--is useful, the lack of focus on outcomes makes it difficult for OSHA to determine which specific enforcement activities are most effective or to convince states to implement changes designed to improve outcomes. For example, only two state-run programs raised their penalty amounts in fiscal year 2011 as recommended by OSHA; state-run program representatives stated that OSHA lacked evidence to show that higher penalties are more effective in deterring future employer violations. OSHA also does not use data already in hand to assess the effectiveness of federal and state enforcement efforts. For example, OSHA does not use data from its annual audits of its regional and area offices or of state-run programs to inform its planning or share information across regions. Additionally, OSHA annually collects data on activities conducted under emphasis programs that focus on national safety and health issues, but it does not evaluate these data to determine whether these programs are responsible for desired outcomes.
section_title: What GAO Recommends
paragraphs: GAO recommends that OSHA standardize guidance for its audit practices, include outcomes in its assessments of its enforcement initiatives, better use data from its audits, and ensure national office participation in audits. OSHA generally agreed with the recommendations but expressed concern about overuse of outcomes to assess effectiveness. GAO continues to believe the recommendations are valid as discussed later in the report.

section_title: Letter
paragraphs: In an 18-month period from 2006 to 2008, a dozen workers died on  construction sites in Las Vegas, Nevada, prompting a congressional  investigation into the Department of Labor’s (Labor) Occupational Safety  and Health Administration’s (OSHA) oversight of worker safety in that  state. OSHA has responsibility under the Occupational Safety and Health  Act of 1970 (OSH Act) to assure safe and healthful working conditions for  the nation’s workers. Under the act, OSHA sets and directly enforces  occupational safety and health standards in about half the states. The  remaining states, including Nevada, have chosen to set and enforce their  own standards, as permitted by the OSH Act, under an OSHA-approved  state plan. OSHA, in turn, monitors enforcement by these state-run  programs. Nevada had previously received favorable annual reviews from  OSHA of its occupational safety and health program. However, a special  review of Nevada by OSHA in 2009—after the construction deaths— revealed significant problems in the state’s enforcement program.
In fiscal year 2010, as a result of the problems identified in Nevada,  OSHA strengthened its annual reviews of all state-run programs. Unlike  prior reviews, these reviews included on-site examination of state  inspection case files by OSHA officials. In this context, we examined 1)  how OSHA’s monitoring of its own and state enforcement efforts  compares, and 2) recent steps OSHA has taken to evaluate the  effectiveness of federal and state enforcement efforts.
To answer our research questions, we reviewed relevant federal laws and  regulations as well as Labor and OSHA policies and procedures for  monitoring federal and state enforcement efforts. We analyzed OSHA   We  audits of its regional and area offices and of state-run programs.conducted site visits to three OSHA regional offices: Region 5 in Chicago,  Illinois, Region 3 in Philadelphia, Pennsylvania, and Region 8 in Denver,  Colorado. In selecting regional offices for site visits, we took into account  region size based on the total number of worksite inspections conducted  annually, geographic location, and the mix of states with state-run  programs and those for which OSHA enforces compliance in each region.  At each office, we interviewed regional officials about their oversight of  federal enforcement efforts and those of states, and visited two area  offices. Within each of the three regions, we visited two area offices: one  responsible for overseeing federal enforcement and one responsible for  overseeing state enforcement. For the area offices that oversee state-run  program(s), we selected states with a range of total number of annual  worksite inspections: Virginia (a large number), Indiana (a medium  number), and Wyoming (a small number). We interviewed regional  officials in the remaining seven regional offices by phone to obtain similar  information. We also collected key documents from each region, including  recent audits of OSHA federal and state enforcement efforts and  inspection activity tracking reports.
In addition, we interviewed officials from Labor and OSHA’s national  office responsible for enforcement, evaluation, strategic planning, and  monitoring of state-run programs about their roles in monitoring  enforcement and measuring effectiveness. To obtain information from  outside parties about OSHA’s oversight of federal and state enforcement,  we interviewed officers of the Occupational Safety and Health State Plan  Association (OSHSPA), which represents the interests of states with  state-run programs, and officials at the Department of Health and Human Services’ National Institute for Occupational Safety and Health (NIOSH).
We also discussed with academic experts and other stakeholders Labor  and OSHA’s performance in conducting and monitoring enforcement,  challenges faced by states with state-run programs, and current research  on these issues. Appendix I provides more information on our scope and  methodology.
We conducted this performance audit from November 2011 through  January 2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Federal and State Responsibilities
paragraphs: OSHA and state-run programs are responsible for protecting the safety  and health of an estimated 130 million private- and public-sector workers  at approximately 8 million worksites. One of the primary ways OSHA  carries out its responsibilities is by setting and enforcing occupational   States with state-run programs typically  safety and health standards.adopt standards identical to federal standards, according to OSHA, but  they may set standards that are more stringent or that address hazards  not covered by federal standards. For example, according to OSHA,  California, Michigan, Oregon, and Washington have standards that are  different from federal standards. To approve a state-run program, OSHA  must determine that the state’s plan meets certain criteria specified in the  OSH Act, including the development and enforcement of standards that  are “at least as effective in providing safe and healthful employment” as  the federal standards. As required by the act, OSHA also conducts  continuing evaluations of states’ implementation of their plans. When  evaluating state-run programs, OSHA considers a program to be “at least  as effective as” the federal program if it is making reasonable progress  toward meeting established performance goals and fulfilling mandated  responsibilities. OSHA enforces standards for private sector workplaces  not covered by a state-run program, and conducts inspections at federal  workplaces in all states. Most states with a state plan enforce employer  compliance with safety and health standards for the private sector as well  as state and local public-sector workplaces in their state; however, some  have chosen to provide such enforcement only for public-sector  workplaces (see fig.1). There are currently 27 state-run programs  responsible for providing enforcement to protect about 40 percent of U.S.  workers.
OSHA provides grants to states with state-run programs of up to 50  percent of state-run program costs to assist with each state’s operating  costs. In fiscal year 2012, Congress appropriated just over $104 million  for the purposes of providing these grants to states. OSHA requires each  state to submit in its grant application an annual performance plan for  approval. The annual performance plan includes a detailed enforcement  plan describing how the state intends to meet its inspection and other  goals and must be approved by OSHA before a state receives its federal  grant. OSHA assesses the performance of a state-run program against  the goals in the state’s performance plan, as well as on mandated  activities, such as program administration and investigation of   In a recent congressional hearing, an OSHA official said  complaints.state-run programs provide a number of advantages in helping OSHA  carry out its enforcement efforts. These advantages include adding  resources to the federal program that would not otherwise be available,  providing enforcement for state and local public-sector workers in those  states, and addressing workplace hazards that are sometimes not  addressed by OSHA.
section_title: OSHA Monitoring and Enforcement
paragraphs: OSHA uses directorates in its national office, as well as 10 regional  offices and 87 area offices, to implement and oversee enforcement.national office includes the Directorate of Enforcement Programs, which  provides guidance to OSHA inspectors on how to enforce safety and  health standards and how employers are to comply with them; the  Directorate of Evaluation and Analysis, which analyzes safety and health  data; and the Directorate of Cooperative and State Programs, which  develops policy and oversees the grants provided to state-run programs.  The directorates provide guidance to the regions and to states. At the  same time, OSHA affords significant autonomy to its 10 regions to both  conduct enforcement activities through their area offices and to monitor  enforcement by state-run programs within their regions. While OSHA’s  regional offices work with OSHA’s national directorates, they report to the  Assistant Secretary of Labor for Occupational Safety and Health (see fig.  2).
section_title: OSHA’s Monitoring Is More Frequent and Its Guidance Is More Consistent for Federal than State Enforcement
subsections: section_title: OSHA Monitors Federal Efforts More Frequently than State Efforts
paragraphs: OSHA uses a variety of methods to monitor the progress of federal and  state enforcement efforts, and monitors federal enforcement efforts more  frequently than those of states. For federal enforcement, OSHA national  officials monitor regional office activities based on the goals it sets in  collaboration with regional officials each year—including the number of  worksite inspections each region is expected to complete—and report  progress quarterly to the Department of Labor. OSHA national officials  monitor regional offices’ progress toward meeting these goals by  reviewing their performance measures and holding weekly conference  calls with regional administrators to discuss performance and other  issues. Regional officials generally monitor their area offices’ progress  toward meeting goals by also reviewing inspection activity reports  submitted weekly by each area office, and through meetings, typically by  phone, with area office managers held at least every other week to review  the data. Inspection activity reports provide information on, among other  things, the number and type of inspections completed, violations found,  and penalties assessed. Some regions also hold in-person meetings a  few times a year with all area office managers to review their progress.  These reports and meetings are the primary ways officials keep track of  each region’s progress toward meeting its goals. For example, in fiscal  year 2012, 7 out of 10 OSHA regional offices met or exceeded their  annual goal for conducting inspections (see fig. 3).
To monitor state enforcement efforts, OSHA regional and area officials  also review performance reports and meet with state officials to monitor  their progress toward meeting their inspection and other goals, but they  do so less frequently for state-run programs than for federal enforcement  efforts. OSHA regional and area offices work directly with states to set  inspection goals, taking into account staffing and funding levels in each  state. Regional officials told us that some state-run programs face funding  challenges. For example, Hawaii recently reported problems in carrying  out its program due to lack of state staffing resources and funds. States  generally have primary responsibility for monitoring the performance of  their state-run programs, subject to oversight by OSHA. States are  afforded flexibility in setting their own goals as long as they report their  activities across a range of performance measures, such as the  percentage of programmed inspections that find certain types of  violations, and the average number of violations per inspection. Regional  and area officials also meet quarterly with state managers to review  states’ performance, whereas regional officials meet with area officials at  least once every other week to review their performance. For state-run  programs, regional officials look for outliers or significant variations from  the national averages or a state’s own performance goals.variations are found, OSHA’s regional officials and state-run program  officials discuss how the state will address them. For example, state-run  program officials could work with OSHA regional officials to develop a  corrective action plan that specifies steps the state will take to improve its  program.
During these quarterly meetings with the states, OSHA regional and area  officials review several types of reports that compare a state’s progress  with federal and national data on multiple measures for the same period.  The State Indicator Report tracks the percentage of state-run program  inspections completed by type and compares them to the percentage of  federal inspections, typically by quarter. The State Activity Mandated  Measures report tracks a state’s performance across various measures,  such as the amount of time it takes a state to begin an inspection after  receiving a complaint about a safety or health hazard, and the percentage  of programmed inspections conducted by the state for which serious,  willful, or repeat violations are found. In addition, OSHA officials review  aggregate data on inspections, violations, and penalties as a part of their  annual audits of states. OSHA’s regional offices have transitioned to a  new data system, the OSHA Information System, which officials said  should allow for more sophisticated data runs and evaluation of outputs.  State-run programs have not yet adopted the new system and are still  using the Integrated Management Information System, or compatible  systems, to report results to OSHA. OSHA offices now manually merge  data from the two systems to produce totals for their enforcement  activities. For fiscal year 2012, inspectors in state-run programs  conducted more inspections and cited more violations than federal OSHA  inspectors, but cited fewer serious violations and assessed lower  penalties overall (see fig.4).
section_title: Guidance Is More Consistent for Audits of Federal Enforcement than State Enforcement
paragraphs: While the guidance OSHA provides for auditing its regional and area  offices is fairly consistent, its guidance for auditing state-run programs  has varied significantly each year since 2009. Under OSHA’s  Management Accountability Program (MAP), OSHA regional offices  conduct annual audits of their own offices and their area offices. These  audits are either comprehensive audits or focused audits of the office’s  inspections and administrative functions. Comprehensive audits, which  are conducted at least once every 4 years, cover a full range of  enforcement and administrative topics. Focused audits, which are  conducted in the years between comprehensive audits, evaluate one or  more selected enforcement or administrative topics. The purpose of MAP  audits is to assess the efficiency and effectiveness of both OSHA’s  regional and area offices’ enforcement field activities in relation to  established policies and procedures and to identify best practices and  deficiencies in order to improve performance. During comprehensive  audits, OSHA conducts on-site case file reviews and addresses 41  different topics on enforcement activities and administrative issues, such   Topics  as timekeeping and the appropriate use of government vehicles.for focused audits are determined by the regional officials based on areas  of concern for them or the national office. Although focused audits of area  offices are normally conducted by OSHA’s regional offices, area office  managers may conduct focused audits of their own offices and report  back to the regional office. Area office managers may not, however,  conduct comprehensive audits of their own offices—these audits can only  be conducted by regional offices and must be done on site. “The audit organization and the individual auditor, whether government or public,  must be independent…Auditors and audit organizations maintain independence so  that their opinions, findings, conclusions, judgments, and recommendations will be  impartial and viewed as impartial by reasonable and informed third parties. Auditors  should avoid situations that could lead reasonable and informed third parties to  conclude that the auditors are not independent and thus are not capable of exercising  objective and impartial judgment on all issues associated with conducting the audit  and reporting on the work.”
Instead of adhering to these criteria, regional officials organized,  managed, and conducted their own audits, as well as audits of their area  offices. For the most recent comprehensive audits of OSHA’s 10 regional  offices, none of the audit summaries or our interviews with regional  officials indicated participation by national office staff. During our review, a national office official verified that no national office staff has  participated in any of the comprehensive audits of regional offices to date,  and the official was unsure whether there were any plans to involve  national office staff in the audits next year.
OSHA’s regional offices also conduct annual audits of state-run programs  called Federal Annual Monitoring and Evaluation (FAME) audits. The  purpose of these audits is to assess the performance of states in meeting  their established goals and complying with statutory and regulatory  requirements, as well as identify areas of concern and excellence, among  other things. During these audits, regions evaluate state-run programs  using certain performance measures, such as the average number of  days it takes to begin an investigation after receiving a complaint. OSHA  officials may also examine case files or accompany inspectors to  worksites to verify that proper procedures are being followed. In  November 2009, after significant problems were identified in Nevada’s  occupational safety and health program, OSHA strengthened its audits of  all state-run programs and called them Enhanced Federal Annual  Monitoring and Evaluation (EFAME) audits. OSHA’s guidance for its fiscal  year 2009 EFAME audits required that OSHA conduct site visits to states  and inspect their case files. According to OSHA, prior to beginning the  EFAME audits, the agency generally did not include these elements in the  annual FAME audits, which may have limited its ability to identify and  address performance problems in some states. The EFAME audits  increased the requirements for OSHA’s audits of states, aligning them  with some of its requirements for the comprehensive MAP audits it  performs, which require regular on-site case file reviews. However,  beginning in 2009, OSHA guidance on what must be included in audits of  state-run programs has changed each year. For fiscal year 2009, OSHA  established the EFAME audits. For fiscal year 2010, OSHA issued  guidance that required only a follow-up to the fiscal year 2009 reviews. In  2011, OSHA issued guidance that required EFAME audits with on-site  case file reviews, and which included a special emphasis on the  whistleblower protection program. Yet, OSHA’s guidance does not  include a timetable for conducting future EFAME audits of state-run  programs. Not having consistent guidance on audits of state-run  programs and no set schedule for when OSHA will conduct these more  comprehensive audits can make it difficult for officials in state-run  programs to prepare for these audits, and limits OSHA’s ability to identify  problem areas and help states address them.
Regional officials told us EFAME audits have value in detecting problems  in the administration of state-run programs that might not otherwise be  identified. They have also resulted in multiple corrective actions for each  state. Indeed, officials in one regional office said the region has seen  improvements in states as these corrective actions have been  implemented, such as improved case file documentation. Officials from  OSHSPA, the organization that represents states with state-run  programs, stated that the 2009 EFAME audits contained a number of  recommendations that were useful, although they said the summaries in  some reports did not clearly represent the audit findings and did not  include the positive aspects of the state-run programs. OSHA national  officials said they plan to conduct EFAME audits every other year, with  follow-up audits in intervening years; however, they have not developed a  formal written plan to implement this new schedule.
section_title: OSHA Is Taking Steps to Better Assess Effectiveness but Does Not Focus on Outcomes
subsections: section_title: OSHA Has Several Initiatives to Assess Effectiveness, but Few Focus on Outcomes
paragraphs: OSHA recently developed several initiatives to better assess the  effectiveness of state and federal enforcement efforts. It is unclear,  however, how some of these initiatives will help OSHA demonstrate  which activities result in desired outcomes—a reduction of worker  fatalities, injuries, and illnesses. With respect to state enforcement efforts,  OSHA created a task force comprised of federal and state  representatives to revise state-run program performance measures, and  in the summer of 2012 held a stakeholder meeting and solicited public  comment on 15 revised measures—which took effect October 1, 2012.  The task force was created, in large part, in response to a March 2011  report by Labor’s Office of Inspector General that shows OSHA had not  developed a method of determining whether state programs are at least  as effective as federal programs, as required by the OSH Act.board chairman of OSHSPA said OSHA has inadequate guidance for  what is needed to meet this requirement and added that, in OSHSPA’s  view, OSHA often wrongly interprets it to mean that states’ measures,  trends, and policies must be identical to those of federal OSHA. Some of  the revised performance measures developed by the task force are  modifications of measures used previously and several add a range of  acceptable state performance. For example, the measure for the average  number of violations found per inspection formerly assessed a state’s  performance against a national average calculated across 3 years. The   The  revised measure compares the state’s average to a range of 20 percent  plus or minus from the national average. OSHA officials said the range  provides a better guide for regional officials reviewing state-run programs  on what is acceptable performance, and affords states the flexibility they  have requested in setting their own performance and strategic goals. For  example, one national OSHA official who oversees state-run programs  said that if a state’s performance is outside the range it does not  necessarily mean its performance is unacceptable, but prompts the  reviewer to do additional work to find out why that particular mark was  missed. However, the revised measures remain largely refinements of  indicators previously used to track activities conducted (outputs), not  effectiveness (outcomes). They mostly focus on outputs such as average  penalty, average number of violations per inspection, and average  number of days to issue citations. It is unclear how adding a 20 to 25  percent plus-or-minus range to targets for each of these enforcement  activities will help OSHA officials determine whether a state’s program is  at least as effective as the federal program because neither the state’s  range nor the federal number is linked to effectiveness—improving  employer compliance and reducing worker fatalities, injuries, and  illnesses. In commenting on the proposed measures, the Assistant  Secretary said they are not pure effectiveness measures but rather  measures of enforcement activities.
OSHA’s difficulty in demonstrating the effectiveness of its enforcement  efforts has affected its ability to convince states to implement  enforcement changes OSHA believes will result in better outcomes. For  example, in 2010, the average penalty for serious violations levied by  OSHA was approximately $1,000, and the average penalty for these  violations in state-run programs was frequently lower—in three states, it  was less than $500. According to the Assistant Secretary, these amounts  were far too low to exert a credible effect in deterring employer violations.  There were also wide discrepancies from state to state in penalty  amounts, which one state-run program manager said raises substantial  concerns about equal expectations of employers and protection for  workers across states. To address these concerns, OSHA increased its  penalties in October 2010, and recommended that states match these  increases. This increase more than doubled the average OSHA penalty  by fiscal year 2012, to $2,154. Officers at OSHSPA opposed the higher  penalties, saying OSHA had failed to provide empirical studies to show  increased penalties were effective in deterring employer violations. They  also said that state-run programs could achieve greater compliance with  safety and health standards by increasing the likelihood of an inspection.  Only two states—Nevada and Wyoming—increased their penalty  amounts in response to OSHA’s recommendation, according to a national  OSHA official. Average penalty amounts for all but three states  (California, Kentucky, and Nevada) remained below the average federal   More recently, in a memorandum dated March 27,  penalty (see fig. 5).2012, OSHA’s national office said states were not immediately required to  adopt the increased penalty policy, so it is unclear how OSHA plans to  take penalty amounts into account when evaluating state-run programs.
With respect to federal enforcement efforts, OSHA has taken additional  steps to assess the effectiveness of enforcement efforts overall. In  summer 2012, OSHA began discussions with the National Advisory  Committee on Occupational Safety and Health (NACOSH) on ways to   Labor,  determine the outcomes of specific enforcement activities.meanwhile, has called on all its agencies to intensify their focus on  assessing outcomes. As part of its strategic planning, Labor in 2010  announced a new approach for measuring outcomes for worker  protection. The document states that a single numerical target, or even a  range, fails to recognize the sustained effort required to truly change  behavior.sampling and stratified random sampling, and social science research  methods, to collect and analyze data for assessing performance. OSHA  acknowledged in its revised fiscal year 2011 Operating Plan that it has  not devoted substantial resources to evaluation activities in the past and,  as a result, knows relatively little about the effectiveness of its policies,  programs, and strategies.
It calls on OSHA to use statistical techniques such as random  OSHA is currently involved in several studies in an effort to address this  knowledge gap. In fiscal year 2011, OSHA began working with  researchers at the Rand Corporation to evaluate the ripple, or deterrent,  effects of enforcement and of providing assistance and education to  employers on future compliance with safety and health standards.  Another recently issued report by Labor’s Office of Inspector General  shows that OSHA knows little about the results of its Site Specific  Targeting program and so cannot be sure it is effective. currently working with Labor’s new chief evaluation officer and a  consulting firm to design and conduct an evaluation of this program, in  which OSHA selects a sample of non-construction worksites for  inspections based on their high rates of injuries and illnesses. OSHA  plans to compare the impact of these inspections to the impact of an  OSHA letter notifying employers of their high injury and illness rates and  urging action, or a combination of both, on employer compliance and  injury and illness rates. As part of this study, OSHA selected a sample of  840 worksites for inspection in 2011 and 2012.
The report, released in September 2012, shows some of the industries and worksites at  which the most serious injuries and illnesses occurred were outside the Site Specific  Targeting program and not always inspected. The report noted the ongoing evaluation  study discussed here. U.S. Department of Labor Office of Inspector General-Office of  Audit, OSHA’s Site Specific Targeting Program Has Limitations on Targeting and  Inspecting High-Risk Worksites, 02-12-202-10-105 (Washington, D.C.: September 28,  2012).
section_title: OSHA Does Not Use Existing Data to Assess Enforcement Effectiveness
paragraphs: OSHA is missing opportunities to use data already in hand to assess the  effectiveness of federal and state enforcement efforts. For example,  OSHA does not analyze data from its audits of its offices and of state-run  programs, and information on its emphasis programs. According to the  Office of Management and Budget (OMB), the GPRA Modernization Act  of 2010 serves as a foundation for creating a culture in which data and  empirical evidence play a greater role in management decisions and  policy. OMB also encourages agencies to adopt increasingly effective  practices.
OSHA does not analyze data from its MAP audits of regional and area  offices or its FAME audits of state-run programs to inform its planning or  to share information across regions. While OSHA summarized the most  common findings from its fiscal year 2011 MAP audits, it provided no  analysis of the data, and no such analysis was performed in 2010 to look  for shared deficiencies or national trends. Similarly, according to a  national OSHA official who oversees state-run programs, the agency has  not analyzed the results of its annual FAME reviews of state-run  programs to identify systemic issues. In accordance with government  standards for internal controls, findings from audits should be used to  identify risks, analyze their effects, and correct identified deficiencies. In our interviews with regional officials, those in two regions said OSHA’s  national office does not analyze the results of their MAP audits, and  officials in two other regions said they were unaware of any such analysis  and have not received feedback from OSHA’s national office on the  results of their MAP audits. In September 2012, Labor’s Office of  Inspector General issued a report that shows national OSHA’s Directorate  of Evaluation and Analysis did not analyze and disseminate the results of  MAP audits to inform management decision making and improve  operations.importance of these audits in providing OSHA information on its  performance.
For fiscal year 2012, OSHA had approximately one dozen NEPs, including some that  had been in effect for several years. An OSHA official said OSHA has a preliminary report  on its NEP on amputations. directly responsible for desired outcomes, such as a reduction in worker  injuries caused by lifting patients or falls. This lack of evaluation hinders  OSHA in determining whether the NEP effort was effective and in  identifying adjustments to improve results. An academic expert and Labor  consultant said there is a recurring lack of planning at OSHA for post- initiative evaluation, which hinders the agency’s ability to determine  whether its enforcement efforts, including NEPs, have an impact on  outcomes, such as reducing occupational injuries.
NIOSH, an agency that conducts research and advises OSHA on  occupational safety and health issues, has faced challenges similar to  OSHA’s in assessing its effectiveness. To address this challenge, NIOSH  contracted with the National Academies to evaluate eight NIOSH  research programs and to help NIOSH hold those programs accountable  for real outcomes that affect life and health. For these evaluations,  NIOSH devised questions to be posed by participating researchers, such  as whether external factors affected outcomes, and the degree to which a  NIOSH program was responsible for any improvement. The evaluation  acknowledged the difficulty in assessing the causal relationship between  NIOSH’s functions as a research agency and specific occupational safety  and health outcomes, but NIOSH officials said it would be easier for  OSHA to do this because OSHA has more concrete data, such as reports  on its worksite inspections.
section_title: Conclusions
paragraphs: GAO, Government  Auditing Standards, 2011 Revision, GAO-12-331G, Sections 3.02 and 3.04 (Washington,  D.C.: December 2011). are most effective in ensuring employer compliance and reducing risks to  workers. OSHA recently developed initiatives to better assess the  effectiveness of federal and state enforcement activities, but few include  the evaluative components needed to assist the agency in examining  effectiveness. Also, national OSHA collects some relevant information  through audits of its own offices and state-run programs, but it does not  annually analyze that information to identify systemic problems and  improvements. In addition, despite a 2010 directive calling for greater  participation of the OSHA national office in regional office comprehensive  audits, there is still minimal participation by the national office in these  audits. Without the independent perspective of the national office, OSHA  cannot assure that results of regional office audits of themselves are  impartial and reasonable. Further, OSHA has not provided consistent  guidance for conducting state EFAME audits that include on-site case file  reviews similar to comprehensive MAP audits. Lack of such guidance  may allow enforcement deficiencies to go undetected, increasing the risk  of worker injuries, illnesses, and death. Enhancing monitoring of  enforcement activities and determining which efforts work best must be a  sustained priority if OSHA is to achieve the best results and succeed in its  mission to protect U.S. workers.
section_title: Recommendations for Executive Action
paragraphs: In order to improve OSHA’s assessments of its effectiveness and its  monitoring of federal and state enforcement efforts, we recommend that  the Secretary of Labor direct the Assistant Secretary of Labor for  Occupational Safety and Health to:  1.  consistently incorporate outcomes in its assessments of enforcement  2.  annually analyze the results of its MAP and FAME audits to identify  and address systemic problems and leading practices;  3.  ensure that OSHA national staff participate in regional office  comprehensive audits to enhance independence in accordance with  OSHA’s MAP 2010 directive; and  4.  provide consistent guidance for scheduling state EFAME audits to  ensure that they are conducted on a regular basis and include  mandatory on-site case file reviews.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Secretary of Labor for review and  comment. We received written comments from OSHA, which are  reproduced in their entirety in appendix II.
OSHA agreed with the essence of our recommendations and stated that it  would carefully consider them. However, OSHA raised some concerns  about them. For recommendation 1, OSHA expressed concern that it may  overemphasize the significance of outcome measurements in determining  the effectiveness of state safety and health programs. OSHA pointed out  that it currently uses other measures in conjunction with outcome  measures to evaluate effectiveness. We agree that using other measures  to evaluate state-run programs is useful but note that OSHA has not  linked them to its assessments of effectiveness. We continue to believe  that OSHA’s assessments would benefit from further analysis of the  results, or outcomes, of both state and federal enforcement activities. We  revised our report to clarify that we do not think OSHA should stop  measuring activities. For recommendations 2 and 3, OSHA stated that it  has taken several steps to improve its analysis of audits and enhance  resources assigned to the program. For recommendation 4, OSHA stated  that it updates guidance for FAME reviews each year and is updating the  State Plan Policies and Procedures Manual to include these revisions and  guidance for on-site case file reviews. While we agree with OSHA that  providing annual guidance to focus audits on areas of concern is valid,  OSHA has not yet provided formal guidance for how often enhanced  audits of state-run programs (EFAMEs) with on-site case file reviews will  be conducted. We revised the wording of our recommendation to refer  specifically to the scheduling of these EFAMEs. Finally, OSHA noted that  our draft report did not adequately acknowledge the state-plan states’  distinct role in directly and closely overseeing their own performance. We  added further acknowledgement of the states’ authority in this area to our  report.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretary of Labor, and other interested  parties. In addition, the report will be available at no charge on the GAO  website at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact Revae Moran at (202) 512-7215 or moranr@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made key  contributions to this report are listed in Appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: The objectives of our review were to examine 1) how the Department of  Labor’s (Labor) Occupational Safety and Health Administration’s (OSHA)  monitoring of its own and state enforcement efforts compares, and 2)  recent steps OSHA has taken to evaluate the effectiveness of federal and  state enforcement efforts.
To address our first objective, we reviewed Labor’s and OSHA’s policies  and procedures for monitoring federal and state enforcement activities,  including OSHA manuals, directives, and memoranda for its 10 regional  offices, 87 area offices, and the 27 state-run programs. We also reviewed  relevant federal laws and regulations. We analyzed audits of OSHA  regional and area offices, which assess their activities in relation to OSHA  policies and procedures and identify best practices and deficiencies in  performance, such as in targeting and scheduling inspections; conducting  fatality investigations; case file documentation; and outreach to  employers, workers, and other stakeholders. We also analyzed OSHA’s  audits of state-run programs, which assess each state’s progress toward  meeting its individual performance goals and compliance with statutory  and regulatory requirements. To further evaluate OSHA’s audit  processes, we compared these processes to criteria for independent  review in GAO’s Government Auditing Standards.
GAO, Government Auditing Standards, 2011 Revision,  GAO-12-331G, Sections 3.02 and  3.04 (Washington, D.C.: December 2011). inspections: Virginia (a large number of inspections), Indiana (a medium  number), and Wyoming (a small number). In Region 8, we interviewed  Denver regional office officials with direct oversight responsibility for  Wyoming because there is no area office in that state. To obtain a  national perspective, we conducted semi-structured phone interviews with  the remaining seven regional office administrators or their managers  about how they monitor and oversee enforcement efforts and each  office’s processes for overseeing the OSHA offices and state-run  programs in its region. From each region, we collected key documents,  including recent audit reports on regional and area offices, interim and  annual reviews of states, inspection tracking reports and interim tracking  of other key measures, and logs of complaints about state-run programs.
To address our second objective, we interviewed the heads of  directorates in OSHA’s national office about their roles in monitoring  enforcement and in planning for greater effectiveness, including the  directorates responsible for state-run programs, overall enforcement,  strategic planning, and evaluation. To obtain information from outside  parties about OSHA’s oversight of federal and state enforcement, we  interviewed Labor officials who work with OSHA on its performance  planning and research agenda; officers of the Occupational Safety and  Health State Plan Association (OSHSPA), which represents the interests  of states with state-run programs; and officials at the Department of  Health and Human Services’ National Institute for Occupational Safety  and Health (NIOSH), which conducts research, experiments, and  demonstrations relating to occupational safety and health. In addition, we  interviewed selected academic experts and stakeholders about Labor and  OSHA performance. We identified these academic experts and  stakeholders through a literature search, as well as from  recommendations of other academic experts versed in OSHA and worker  protection issues and the recommendations of OSHA and NIOSH  officials. Finally, we reviewed relevant federal laws and regulations,  including the Occupational Safety and Health Act of 1970; the  Government Performance and Results Act of 1993 (GPRA); and the  GPRA Modernization Act of 2010. We reviewed OSHA’s use of performance goals and measures in accordance with criteria in Office of  Management and Budget Circular A-11, Labor’s current 5-year strategic  plan, and other agency plans for performance. We also reviewed  previous GAO reports and NIOSH’s 2004-2008 evaluation of its own  effectiveness.
We assessed the reliability of data we used from the OSHA Integrated  Management Information System and OSHA Information System by (1)  reviewing existing information about the data and the systems that  produced them, (2) interviewing agency officials knowledgeable about the  data, and (3) reviewing these data for obvious inconsistency errors. We  determined that the data were sufficiently reliable for the purpose of this  report.
We conducted this performance audit from November 2011 through  January 2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Labor
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Assistant Director Mary  Crenshaw, Matthew Alemu, Susan Aschoff, and Paul Schearf made  significant contributions to this report. Assistance, expertise, and  guidance were provided by James Bennett, Ted A. Burik, Sarah Cornetto,  Meeta Engle, Gretta Goodwin, Isabella P. Johnson, Kathy Leslie, Luann  Moy, Carol Patey, Bryan Rogowski, Linda Siegel, and Kathleen van  Gelder.
section_title: Related GAO Products
paragraphs: Workplace Safety and Health: Multiple Challenges Lengthen OSHA’s  Standard Setting. GAO-12-330. Washington, D.C.: April 2, 2012.
Workplace Safety and Health: Enhancing OSHA’s Records Audit Process  Could Improve the Accuracy of Worker Injury and Illness Data.  GAO-10-10. Washington, D.C.: October 15, 2009.
OSHA’s Voluntary Protection Programs: Improved Oversight and Controls  Would Better Ensure Program Quality. GAO-09-395. Washington, D.C.:  May 20, 2009.
Whistleblower Protection Program: Better Data and Improved Oversight  Would Help Ensure Program Quality and Consistency. GAO-09-106.  Washington, D.C.: January 27, 2009.
Workplace Safety and Health: Safety in the Meat and Poultry Industry,  while Improving, Could Be Further Strengthened. GAO-05-96.  Washington, D.C.: January 12, 2005.
Workplace Safety and Health: OSHA’s Oversight of Its Civil Penalty  Determination and Violation Abatement Processes Has Limitations.  GAO-04-920. Washington, D.C.: August 13, 2004.
Workplace Safety and Health: OSHA Can Strengthen Enforcement  through Improved Program Management. GAO-03-45. Washington, D.C.:  November 22, 2002.
Occupational Safety and Health: Efforts to Obtain Establishment-Specific  Data on Injuries and Illnesses. GAO/HEHS-98-122. Washington, D.C.:  May 22, 1998.
Occupational Safety and Health: Changes Needed in the Combined  Federal and State Approach. GAO/HEHS-94-10. Washington, D.C.:  February 28, 1994.