Workplace Safety and Health: Enhancing OSHA's Records Audit Process Could Improve the Accuracy of Worker Injury and Illness Data
Released: 2009-11-16T00:00:00

Letter

Background

Under the OSH Act, OSHA is responsible for protecting the safety and health of the nation’s workers. The agency helps ensure the safety and health of over 112.5 million private sector workers in approximately 8.6 million worksites in the United States by setting and enforcing safety and health standards, rules, and regulations, and inspecting worksites to ensure employer compliance. OSHA helps to ensure safe and healthy working conditions for workers through its 11 national office directorates and 10 regional offices. The national office directorates include the Directorate of Enforcement Programs, which provides guidance to OSHA inspectors on how to enforce safety and health regulations and standards and how employers are to comply with them, and the Directorate of Evaluation and Analysis, which establishes policies and analyzes safety and health data. OSHA directly enforces safety and health regulations and standards in about half the states; the remaining states have each been granted authority by OSHA to set and enforce their own workplace safety and health standards for worksites under a state plan approved by OSHA (state-plan states).

The OSH Act requires nonexempt employers to prepare and maintain records of injuries and illnesses sustained by their workers and make them available to OSHA. The primary record employers are required to maintain is OSHA’s Form 300 Log of Work-Related Injuries and Illnesses (see app. IV). For each work-related injury and illness that requires medical treatment other than first aid, the employer is required to record the worker’s name; the date; a brief description of the injury or illness; and the number of days the worker was away from work, assigned to restricted duties, or transferred to another job as a result of the injury or illness. Employers are also required to describe each injury and illness on the Injuries and Illnesses Incident Report (OSHA’s Form 301). About 1.5 million employers with more than 10 employees—representing about 17 percent of the approximately 8.6 million private sector worksites and an estimated 53 million employees covered by OSHA—must keep such records. OSHA has established definitions and recordkeeping guidelines to assist employers in determining which injuries and illnesses must be recorded in their injury and illness logs. Injuries and illnesses serious enough to be recorded include those that result in lost work time, medical treatment other than first aid, restriction of work, loss of consciousness, or transfer to another job. OSHA requires employers to post summaries of their logs annually at each worksite and to provide them to OSHA and BLS if requested. OSHA’s recordkeeping standards, which took effect in January 2002, were intended to simplify the recordkeeping rules and forms used to record injuries and illnesses.

OSHA also promotes workplace safety and health by targeting industries and employers with the highest number of workplace injuries and illnesses for inspection. OSHA does this through both programmed (scheduled) inspections and unprogrammed (unscheduled) inspections conducted by inspectors in area offices throughout its 10 U.S. regions. OSHA places the highest priority on unprogrammed inspections initiated in response to fatality investigations, formal complaints, referrals, and other situations that could pose a risk to the safety and health of workers. OSHA gives a lower priority to programmed inspections, which include those selected by OSHA through its Site-Specific Targeting program, which it uses to target high hazard worksites for inspection. Table 1 shows the number of programmed and unprogrammed inspections OSHA conducted from fiscal years 2003 through 2007.

BLS’s SOII includes injury and illness data from employers’ logs for about 241,000 worksites; the ODI survey includes data from about 80,000 worksites in high hazard industries. The SOII is a coordinated federal- state effort that estimates the number of workplace injuries and illnesses that occur at worksites in most industries in the United States. Because the data come from OSHA logs, the injuries and illnesses counted by the survey are only those required by OSHA to be recorded. As such, the data differ from those collected by other systems, such as data collected using workers’ compensation claims. While BLS and OSHA collect the same basic information, they largely collect data from different employers. However, BLS estimates a potential overlap of less than 10 percent of employers who must complete both the BLS SOII and OSHA ODI surveys in a given year. In these cases, employers send the data to both BLS and OSHA because the agencies do not share data. Figure 1 shows the surveys and how they are used.

Number of injured and ill workers (in millions)

The International Labour Organization is the United Nations agency that brings together representatives of governments, employers, and workers of its member states to jointly shape polices and programs that promote decent and productive employment. more workers it defines as being in high hazard industries. According to OSHA officials, the survey size is based on the budgetary resources OSHA had when the ODI was established. The agency uses data from the ODI to target employers for inspections, outreach, and technical assistance, and to measure its performance in reducing workplace injuries and illnesses . For example, OSHA provides employers with onsite assistance to help them identify and correct hazards and set up safety and health programs. OSHA also provides employers with training and education to help them reduce worker accidents and injuries. The 130,000 worksites in the ODI universe are selected from manufacturing and 22 other industries O defined as high hazard on the basis of their injury and illness rates reported by BLS in 2002: worksites with a lost workday injury and illne (LWDII) rate of 5.0 or higher. worksites, OSHA included 20,000 construction worksites in its 2008 ODI. OSHA has also proposed including worksites with 30 or more employees in the ODI, instead of using the current threshold of 40 or more employees.

Until 2002, DOL used the LWDII rate to compare the rates of injuries and illnesses among worksites of varying sizes. The rate was calculated based on the total number of injuries or illnesses resulting in lost work days. In 2002, after revising its recordkeeping requirements, DOL began using the days away from work, restricted activity, or job transfer (DART) rate to compare injuries and illnesses among worksites instead of the LWDII rate.

Based on its analysis of OSHA’s records audits of employers’ 2003, 2004, and 2005 injury and illness data, Eastern Research Group, Inc. found an accuracy rate of over 90 percent for the total number of cases that were required to be recorded and those involving days away from work, restricted activity, or job transfer (DART). OSHA uses these findings to support the agency’s continued use of the ODI data to target worksites for enforcement and compliance assistance, and to measure the agency’s performance in reducing workplace injuries and illnesses.

DOL Verifies the Injury and Illness Data in the ODI, but OSHA Does Not Always Collect Information from Workers, and Excludes Certain Industries

Although DOL is not required to, it verifies some of the workplace injury and illness data it collects from employers on the ODI survey via OSHA’s records audits. However, OSHA’s efforts to verify the accuracy of the data are not adequate because OSHA overlooks some information it could obtain from workers about injuries and illnesses during these audits that could help verify the accuracy of the data. In addition, OSHA excludes certain high hazard industries from its data collection efforts, which precludes them from being selected for records audits and makes them unlikely to be targeted by OSHA for inspections, outreach, and technical assistance. BLS does not verify the injury and illness data it collects from employers in the SOII that are used to report national injury and illness statistics and trends, but it has taken or is planning to take several actions to respond to concerns about the quality and completeness of the data.

OSHA Does Not Require Inspectors to Interview Workers during Records Audits

OSHA does not require inspectors to interview workers during records audits about injuries and illnesses that they or their co-workers may have experienced. Although OSHA’s procedures manual states that inspectors must conduct interviews if they believe the records do not provide full and accurate information, it does not provide criteria for what constitutes “full and accurate” information. OSHA officials confirmed that it is optional for inspectors to interview workers during records audits. As a result, inspectors may miss opportunities to obtain information from workers about injuries and illnesses that may not have been properly recorded by employers on their injury and illness logs. As noted in our previous work, there are potential risks in relying solely on employer-reported data. When OSHA inspectors conduct records audits, the audit procedures direct them to inspect the records of a random sample of workers at the worksites, among other things. These records, which are provided to the inspectors by the employer, can include workers’ compensation records, medical records, accident reports, and records of absences.

In addition to reviewing these records, OSHA’s procedures provide inspectors with the option to interview workers. Worker interviews are the only source of information used during the audit not provided by the employer. If inspectors choose to interview workers, OSHA’s audit software generates a sample of workers to be interviewed from the initial random sample of workers. For the 753 records audits OSHA conducted of employers’ 2003, 2004, and 2005 injury and illness records, we found that inspectors chose to interview workers in about half of the audits. During our interviews, inspectors told us one challenge they face in interviewing workers is that many workers are no longer employed at the worksite or are unavailable to be interviewed at the time of the audit. Of these inspectors who conducted interviews, 9 of 14 reported they are rarely or never able to interview the full sample of workers. We examined the data for audits conducted from 2005 to 2007, and found that when inspectors interviewed workers, 72 percent of the time they did not interview the full number of workers recommended by the audit procedures. OSHA headquarters officials told us that, although the records audit procedures do not direct inspectors to substitute other workers to interview when the workers originally selected are unavailable, they always instruct inspectors to do so during records audit training. However, OSHA does not conduct all of the records audit training inspectors receive, and several of the inspectors we interviewed said they had not received this training.

Lack of Timeliness in Conducting Interviews with Workers Can Affect Their Usefulness

Interviewing workers might provide information to help inspectors evaluate the accuracy and completeness of employer-provided data; however, the lack of timeliness in conducting the interviews can affect their usefulness. Some inspectors told us that because OSHA does not conduct records audits until about 2 calendar years after the injuries and illnesses are recorded, inspectors rarely learn about underrecorded injuries or illnesses from the interviews. Because of this lag, inspectors told us many workers are no longer employed at the worksite and those who remain may be unable to remember the injury or illness. OSHA officials said the lag exists because, after the end of the calendar year in which the injury or illness is recorded, it takes OSHA a full year to collect the data and up to 9 additional months to conduct the records audits. For example, in early 2008, OSHA selected the ODI worksites for the calendar year 2007 injury and illness data. OSHA then spent a year collecting the data from employers. After collecting the data, OSHA selected worksites for records audits in early 2009, and generally gave inspectors until the end of September to complete the audits. As a result, if a worker was injured in January 2007, OSHA might not examine the employer’s records or interview the worker about the injury until the summer or fall of 2009—2½ years after the injury occurred. Figure 4 depicts the timeline for the process and the activities performed. In comparison, it takes BLS approximately 10 months to both collect and report the SOII data; however, BLS does not conduct follow-up verifications like OSHA’s records audits.

OSHA’s ODI Universe Excludes Eight High Hazard Industries

Worksites under eight high hazard industries cannot be selected for records audits or targeted for OSHA’s enforcement and compliance activities, because OSHA has not updated its list of high hazard industries included in the ODI universe since 2002. (See app. V for a list of high hazard industries included in the ODI universe.) OSHA has neither a formal written policy on how or when to update the list of industries included in the ODI, nor clear documentation that explains the original construction of the ODI or its subsequent updates. We first reported on OSHA’s lack of documentation for its ODI industry selection process in 1998. By not updating its high hazard industry list using the most recent BLS SOII data, we found that OSHA is excluding eight high hazard industries that had an average DART rate of 4.2, which is higher than twice the national average or greater, for the three most recent years, from 2005 to 2007. Industries excluded include amusement parks, industrial launderers, and general rental centers (see table 3). As a result, worksites in these industries are precluded from being selected for OSHA’s records audits and they are unlikely to be targeted by OSHA for inspections, outreach, and technical assistance. Table 3 shows the industries excluded from the ODI universe.

OSHA officials told us they have not updated the high hazard list because an agency regulation requires them to use the Standard Industrial Classification (SIC) system to classify industries, rather than the North American Industry Classification System (NAICS) industry codes currently used by BLS to report injury and illness rates. Prior to 2003, both OSHA and BLS used the SIC codes to classify industries. OSHA officials said they would like to switch to the NAICS codes, but they stated it is not currently an agency priority to pursue the regulatory change required to do so. In addition to a regulatory change, switching to NAICS would require OSHA to re-evaluate the criteria it uses to define industries as high hazard because in 2002, OSHA switched from using the LWDII rate to the DART rate for measuring workers’ injuries and illnesses. Because the LWDII and DART are not exactly comparable, OSHA would have to identify a DART rate that is comparable to its LWDII rate of 5.0, which was the criterion OSHA used in 2002 to define a high hazard industry. According to our analysis, the results of which we confirmed through discussions with OSHA officials, a 4.2 DART rate is comparable to a 5.0 LWDII rate.

BLS Does Not Verify Employer-reported Data in the SOII, but Has Undertaken Actions to Improve the Quality and Completeness of the Data

BLS is not required to verify the accuracy of the data employers record on their OSHA forms; however, BLS has acknowledged limitations to the survey and has taken steps to improve it. BLS uses the SOII to report national, industry-wide injury and illness data, and policymakers and employers rely on the data to understand national trends in worker safety and health. The SOII only includes injury and illness data provided by employers. In contrast, BLS reports monthly employment statistics with data from employers on the number of jobs and from households on the number of people employed. A number of studies have compared the BLS data on injuries and illnesses to data collected from other sources, such as workers’ compensation, hospital discharge data, and medical records. These studies found discrepancies between the number of injuries and illnesses reported in the SOII and the information in the other data sets. Some researchers have also criticized the scope of the SOII, noting, for example, that the 14.7 percent of all workers in 1999 who were government workers and the 7.3 percent of all workers who were self- employed were not included in the SOII.

In response to questions about the accuracy of the employer-reported SOII data, BLS has taken several actions designed to improve the quality and completeness of the data. For example, to address concerns about the survey’s limited scope, BLS expanded the SOII for its 2008 survey to include data on state and local government workers in all states and conducted a quality assurance study to verify that employers correctly transcribed information from their 2006 OSHA logs onto BLS’s SOII survey forms. BLS also interviewed employers to determine how they record injury and illness data on the OSHA and workers’ compensation forms. The aim of this effort was to identify cases where employers reported an injury or illness to the state’s Workers’ Compensation program, but did not record the cases on the OSHA log, despite the fact that the injury or illness was an OSHA-recordable case. In addition, in a 2009 research study, BLS examined discrepancies between the number of workplace injuries and illnesses reported in states’ workers’ compensation databases and in the SOII to address concerns about data accuracy. From the research, BLS identified some factors associated with discrepancies between the SOII and workers’ compensation data, and is continuing to conduct research to identify additional potential factors. BLS stated that some of the discrepancies arose from cases that were compensable, but in which workers had no days away from work, and cases that entered workers’ compensation after the end of the year, but did appear in the BLS data.

In addition to the actions it has already taken, BLS is planning to explore the use of other data sets to improve the quality of the SOII data. For example, BLS officials told us they plan to support the work of the National Institute for Occupational Safety and Health to explore the use of occupational injury and illness data collected by emergency departments to help identify gaps in the SOII data. The emergency department data could be particularly important because they would capture injuries and illnesses for self-employed workers, who are currently excluded from the SOII. In addition, since these data are reported by hospitals and not employers, they could help BLS identify underrecorded injuries and illnesses. Finally, BLS is planning to work with the Council of State and Territorial Epidemiologists to evaluate the quality of the SOII data for certain injuries such as amputations and carpal tunnel syndrome. BLS has issued grants to three states to evaluate the possibility of using multiple sources of data to enumerate the quality of the SOII for certain injuries such as amputations and carpal tunnel syndrome.

Occupational Safety and Health Practitioners and Stakeholders Cited Worker and Employer Disincentives as Primary Factors That May Affect the Accuracy of Injury and Illness Data

Disincentives that influence workers’ decisions to report and employers’ decisions to record work-related injuries and illnesses are primary factors that may affect the accuracy of the data, according to occupational safety and health practitioners and stakeholders. They also reported that a lack of understanding of OSHA’s recordkeeping requirements by those responsible for recording injuries and illnesses may affect the accuracy of the data.

Various Disincentives May Discourage Workers from Reporting and Employers from Recording Injuries and Illnesses

Occupational safety and health stakeholders we interviewed and occupational health practitioners we surveyed told us that primary factors affecting the accuracy of injury and illness data include disincentives that affect workers’ decisions to report work-related injuries and illnesses and employers’ decisions to record them. Stakeholders most often cited workers’ fear of job loss and other disciplinary actions as disincentives that can affect workers’ decisions to report injuries and illnesses. Occupational health practitioners concurred: 67 percent reported observing worker fear of disciplinary action for reporting an injury or illness, and 46 percent said that this fear of disciplinary action has at least a minor impact on the accuracy of employers’ injury and illness records. Workers’ fear of disciplinary actions may be compounded by policies at some worksites that require workers to undergo mandatory drug testing following incidents resulting in reported injuries or illnesses, regardless of any evidence of drug use. Several labor representatives described mandatory drug testing policies as a disincentive that affects workers’ decisions to report injuries and illnesses, and 67 percent of health practitioners reported they were aware of this practice at the worksites where they treated workers in 2008.

Stakeholders also said employers’ safety incentive programs can serve as disincentives for workers reporting injuries and illnesses. These programs reward workers when their worksites have few recordable injuries or illnesses. One-half of the health practitioners who responded to our survey reported they were aware of incentive programs at the worksites where they treated workers in 2008. Safety incentive programs are designed to promote safe behavior by workers, and 72 percent of health practitioners reported that these programs motivate workers to work in a safe manner. However, some stakeholders said these programs can discourage workers from reporting injuries and illnesses; more than three-quarters of health practitioners said they believed workers sometimes avoid reporting work- related injuries and illnesses as a result. Stakeholders also said that in addition to missing the chance to win prizes for themselves, workers who report injuries and illnesses may risk ruining their coworkers’ chances of winning such prizes.

Various disincentives may also discourage employers from recording workers’ injuries and illnesses. Stakeholders told us employers are concerned about the impact of higher injury and illness rates on their workers’ compensation costs. Several researchers and labor representatives said that because employers’ workers’ compensation premiums increase with higher injury and illness rates, employers may be reluctant to record injuries and illnesses. They also said businesses sometimes hire independent contractors to avoid the requirement to record workers’ injuries and illnesses because they are not required to record them for self-employed individuals. Stakeholders also told us employers may not record injuries and illnesses because having high injury and illness rates can affect their ability to compete for contracts for new work. The injury and illness rate for worksites in certain industries, such as construction, affects some employers’ competitiveness in bidding on the same work.

Disincentives that discourage workers from reporting and employers from recording injuries and illnesses may also result in pressure on occupational health practitioners to treat workers in a manner that avoids the OSHA requirement to record injuries and illnesses. From our survey, we found that more than one-third of health practitioners were asked by company officials or workers to provide treatment that resulted in an injury or illness not being recorded, but also was not sufficient to properly treat the injury or illness. For example, in some cases, practitioners stated that employers may seek out alternative diagnoses if the initial diagnosis would result in a recordable injury or illness. One practitioner said that an injured worker’s manager took the worker to multiple providers until the manager found one who would certify that treatment of the injury required only first aid, which is not a recordable injury. Fifty-three percent of the health practitioners reported that they experienced pressure from company officials to downplay injuries or illnesses, and 47 percent reported that they experienced this pressure from workers. Further, 44 percent of health practitioners stated that this pressure had at least a minor impact on whether injuries and illnesses were accurately recorded, and 15 percent reported it had a major impact. In some cases, this pressure may be related to the employers’ use of incentive programs. Of those experiencing pressure from workers, 61 percent reported they were aware of incentive programs at the worksites where they treated workers (see fig. 5). In comparison, of the practitioners who reported not experiencing pressure from workers in 2008, 41 percent reported being aware of incentive programs at the worksites where they treated workers.

An OSHA official told us that OSHA does not have an official policy on incentive programs or practices that may affect workers’ decisions to report injuries and illnesses, but it has authority under the OSH Act to discourage inaccurate reporting by employers. The official stated that, under a planned National Emphasis Program, OSHA will explore the possible impact that incentive programs have on workers’ decisions to report injuries and illnesses. To address disincentives that may affect employers’ decisions to accurately record injuries and illnesses, the official stated OSHA can issue citations or fine employers when recordkeeping violations are found.

Lack of Understanding of OSHA’s Recordkeeping Requirements and Other Factors May Also Affect the Accuracy of the Injury and Illness Data

Several stakeholders and nearly all of the OSHA inspectors we interviewed said that the lack of understanding of OSHA’s recordkeeping requirements by the individuals charged with recording injuries and illnesses affects the accuracy of the injury and illness data. Forty-one percent of occupational health practitioners reported that misinterpretation of OSHA’s recordkeeping requirements by company officials has an impact on whether injuries and illnesses are accurately recorded (see fig. 6). Several researchers and a representative from a labor organization with whom we spoke said that inaccuracies in recording injuries and illnesses can result from a lack of understanding of the differences between OSHA’s recordkeeping requirements and the eligibility criteria for workers’ compensation claims. They stated that some individuals charged with maintaining employers’ OSHA logs erroneously think that the criteria for recording injuries and illnesses are the same as the eligibility criteria for filing workers’ compensation claims. Therefore, they may be less likely to record injuries and illnesses that are not compensable through the workers’ compensation system. In addition, some stakeholders said they thought the lack of understanding among those recording injuries and illnesses was likely worse in smaller companies with fewer resources than larger companies, which have a greater capacity for providing recordkeeping training.

OSHA provides a number of tools to assist employers in understanding its recordkeeping requirements. For example, the form employers use to record injuries and illnesses—the OSHA injury and illness log—provides examples of which injuries and illness must be recorded and how to record them. OSHA also posts guidance and frequently asked questions about its recordkeeping requirements on its Web site. In addition, OSHA officials told us employers with recordkeeping questions can phone officials in OSHA headquarters and area offices, or e-mail questions to OSHA via its Web site. They also said they have considered creating an online tool to help employers quickly and easily determine whether to record specific injuries and illnesses on their logs.

Stakeholders also discussed additional factors that may affect the accuracy of employers’ data, including weaknesses in OSHA’s enforcement efforts and the difficulty of determining whether some illnesses are work related. Several stakeholders pointed to weaknesses in OSHA’s enforcement efforts as a reason for inaccuracies in employers’ injury and illness data. For example, some stakeholders noted that OSHA’s enforcement of recordkeeping practices has diminished in recent years. Two stakeholders said OSHA’s enforcement capabilities could be strengthened with additional resources. Another factor a few researchers cited that could affect the accuracy of injury and illnesses data is that illnesses, particularly those with long latency periods, are less likely to be reported by workers and recorded by employers than injuries. They explained that, for many of these illnesses, it is difficult to prove they were caused by work-related activities.

Conclusions

Workers are entitled to safe and healthful workplaces, and it is DOL’s responsibility to track the safety and health of the nation’s workplaces and ensure that employers take steps to minimize workers’ risks of injuries and illnesses. Accurate injury and illness records are important because they assist Congress, researchers, OSHA, BLS, and other agencies in describing the nature and extent of occupational safety and health problems. These records are also vital to helping employers and workers identify and correct safety and health problems in the workplace. In addition, these records help OSHA evaluate programs, allocate resources, and set and enforce safety and health standards. Without accurate records, employers engaged in hazardous activities can avoid inspections because OSHA bases many of its safety inspections on work-related injury and illness rates.

Because injury and illness data are so vital, it important that OSHA and BLS take steps to ensure that the data are as accurate as possible. First, OSHA inspectors must take advantage of opportunities to verify the accuracy and completeness of employer-provided records by interviewing workers who may be aware of injuries and illness that may not have been recorded by employers. It is also important that OSHA conduct its records audits as soon as possible after it collects employers’ injury and illness data to maximize the usefulness of information collected from worker interviews. In addition, it is imperative that employers understand which injuries and illnesses should be recorded under OSHA’s recordkeeping standards. Finally, although BLS has taken steps to improve the quality of the injury and illness data it collects, these actions will not address all of the concerns regarding the accuracy of the injury and illness data that BLS collects and reports. As these data are the only comprehensive source of national data on workers’ injuries and illnesses, it will be important for BLS to follow through on its efforts.

Recommendations for Executive Action

To improve OSHA’s efforts to verify the accuracy of employer-provided injury and illness data, the Secretary of Labor should direct the Assistant Secretary for OSHA to take the following three actions: require inspectors to interview workers during the records audits to obtain information on injuries or illnesses and substitute other workers when those initially selected for interviews are not available; minimize the amount of time between the date injuries and illnesses are recorded by employers and the date they are audited by OSHA; and update the list of high hazard industries used to select worksites for records audits and target inspections, outreach, and technical assistance.

To improve the accuracy of the data recorded by employers on workers’ injuries and illnesses, the Secretary of Labor should direct the Assistant Secretary for OSHA to increase education and training provided to employers to help them determine which injuries and illnesses should be recorded under the recordkeeping standards, such as providing assistance to employers via the online tool that OSHA is considering.

Agency Comments and Our Evaluation

We provided a draft of this report to the Secretary of Labor for comment. We received written comments from the Acting Assistant Secretary for OSHA, which are reproduced in their entirety in appendix VI. OSHA and BLS also provided technical comments, which we incorporated in the report as appropriate.

OSHA agreed with all of our recommendations and stated that it would move forward to implement them. To address the first two recommendations, OSHA stated that it would require inspectors to interview employees during records audits and develop policies to conduct record audits inspections in a timely fashion. For the third recommendation, OSHA stated that it would pursue rulemaking at the earliest possible date to update the industry coverage of the recordkeeping rule from the SIC system to NAICS, which would ensure that records audits include emerging high-risk industries. To address our fourth recommendation, OSHA stated that it would supplement its current educational outreach and develop a Web-based tool to assist employers in meeting the requirements of OSHA’s recordkeeping regulations. OSHA also informed us that it implemented a National Emphasis Program (NEP) on Recordkeeping on October 1, 2009. The purpose of the NEP is to identify and correct recordkeeping inaccuracies and complement BLS’s efforts to investigate factors accounting for differences in the number of workplace injuries and injuries estimated by BLS and other data sources.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Secretary of Labor, relevant congressional committees, and other interested parties. In addition, the report will also be available at no charge on GAO’s Web site at http://www.gao.gov.

A list of related GAO products is included at the end of this report. If you or your staff have questions about this report, please contact me at (202) 512-7215 or moranr@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VII.

Appendix I: Scope and Methodology

Review of the Department of Labor’s Efforts to Verify the Accuracy of Employer-Reported Injury and Illness Data

To examine whether the Department of Labor (DOL) verifies that employers are accurately recording workers’ injuries and illnesses, and, if so, the adequacy of such efforts, we focused on the efforts of DOL’s Occupational Safety and Health Administration (OSHA) to verify the data it collects from employers on workers’ injuries and illnesses through its annual OSHA Data Initiative (ODI) survey. We analyzed OSHA’s policies and procedures and interviewed OSHA officials regarding the agency’s employer recordkeeping requirements. In addition, we reviewed the Bureau of Labor Statistics’ (BLS) efforts to verify the data it collects for the Survey of Occupational Injuries and Illnesses (SOII).

Analysis of OSHA’s Audits of Employer Injury and Illness Records

We analyzed the results of the onsite audits of employers’ injury and illness records (records audits) OSHA conducted in 2005, 2006, and 2007 of employers’ injury and illness logs for 2003, 2004, and 2005—the most recent period for which data were available. Prior to our analysis, we assessed the reliability of the database OSHA uses to track its records audits—the OSHA Recordkeeping Audit Assistant—by reviewing information obtained from OSHA about the database, interviewing knowledgeable agency officials, and performing electronic testing of the software, among other steps. On the basis of our assessment, we concluded that the data maintained by OSHA in its database were sufficiently reliable for our reporting purposes.

Interviews of OSHA Inspectors Who Audit Employers’ Injury and Illness Records

We interviewed selected OSHA inspectors who conducted the records audits in 2005, 2006, and 2007 to learn about (1) the training they received, (2) the extent to which they followed OSHA’s procedures for the records audits, and (3) their views on the accuracy of the employers’ injury and illness records they reviewed. Although we did not seek to generalize the responses of individual inspectors to the broader group of all inspectors who conducted these audits, we took steps to ensure that we had a mix of inspectors. We interviewed inspectors in states where federal OSHA directly enforces safety and health regulations and standards and those in states that have been approved by OSHA to conduct such activities (state- plan states). These inspectors had a range of experience as determined by the number of audits they conducted in 2005, 2006, and 2007. We selected two inspectors for these interviews in each of OSHA’s 10 regions—1 inspector who conducted the greatest number of records audits and 1 who conducted the fewest number. Although we attempted to select 2 inspectors in each region, we were only able to interview 1 inspector in 1 of the regions because only 1 inspector in that region conducted records audits during the 3-year period we reviewed. As a result, we interviewed a total of 19 inspectors, including 12 federal and 7 state inspectors. In each of the 10 regions, we also interviewed other regional staff to obtain their views about the records audits. We interviewed the regional administrator, the deputy regional administrator, or someone designated as representing their views in each region. In addition, we interviewed 8 officials from 6 regions who were area directors, records audit coordinators, or supervisors.

Analysis of the Methods OSHA Uses to Select Worksites for Records Audits Using the ODI Universe

To understand OSHA’s process for selecting worksites for records audits, we interviewed federal OSHA officials about the methods they use to select worksites from the ODI universe. We also analyzed the methods they use to compile and update the ODI universe, which is used to select worksites for records audits, and target worksites for safety and health inspections, outreach, and technical assistance.

As part of this work, we examined the methods OSHA uses to define industries as “high hazard,” which makes the worksites in these industries eligible to be selected by OSHA for records audits and targeted for safety and health inspections. In defining the industries to be included in the ODI, OSHA uses industry-level data published by BLS prior to 2002 based on the employer data collected in the Survey of Occupational Injuries and Illnesses (SOII) on the incidence rates of occupational injuries and illnesses resulting in lost work days (referred to as Lost Work Day Injuries and Illnesses ) using Standard Industrial Classification (SIC) codes. In 2003, BLS began publishing SOII data using North American Industry Classification System (NAICS) codes to categorize industries instead of SIC codes.

When OSHA last updated its ODI universe, it included manufacturing and industries with an LWDII rate of 5.0 or greater; at that time, 5.0 was twice the national injury and illness rate. Since OSHA has not updated the ODI universe since 2002, it has not yet established a new threshold for inclusion based on the days away, restricted or transferred (DART) rate measurement it now utilizes. Based on our analysis of current BLS data, we determined that a current DART rate of 4.0 was comparable to OSHA’s LWDII rate of 5.0 in 2002. In order to determine which industries are high hazard using current data, we first converted the high hazard industries in OSHA’s ODI universe from the SIC codes OSHA provided to GAO into the comparable NAICS codes. We then examined the incidence of injuries and illnesses in industries that were not in the ODI universe, and designated as potentially high hazard those that had a DART rate of 4.0 or higher in any year in the 5-year period from 2003 to 2007, which resulted in a list of 33 potentially high hazard industries. We asked OSHA officials to review the list of 33 industries and identify any that were not under their jurisdiction or were otherwise inappropriate for inclusion in the ODI. The officials stated that a DART rate of 4.2—twice the national average—is the threshold they would use to determine which industries are high hazard. After we removed the 8 industries with DART rates below 4.2, we found 26 industries that might be eligible for inclusion in the ODI universe. OSHA officials also told us that they used a 3-year average injury and illness rate to determine eligibility for inclusion in the ODI universe. Of the 26 industries, we found that 12 had average DART rates for 2005 to 2007 that were lower than the 4.2 threshold and were therefore not eligible for inclusion. Five others were not appropriate for inclusion in the ODI because they did not fall under the agency’s jurisdiction or were comprised mostly of small employers. One remaining industry of the 26 was already included in the ODI under a different, but related, NAICS code. After obtaining OSHA’s input, we identified 8 industries that could be included in the ODI universe if OSHA updated the universe using NAICS codes and current BLS data.

Discussions with Stakeholders of Factors That Affect the Accuracy of Employers’ Injury and Illness Records

To examine the factors that may affect the accuracy of employers’ injury and illness records, we selected various experts and researchers to interview based on (1) the individual’s title, affiliation, and type and depth of experience; (2) the extent to which the individual’s published work has been cited by other studies, and by OSHA, BLS, and other relevant organizations; (3) recommendations from other stakeholders; (4) the relevance of the individual’s work; and (5) the source of funding of the individual’s published work. By reviewing the literature on occupational injury and illness data, and other efforts, we identified 12 experts and researchers for our interviews. We vetted this group with (1) the director of safety and health at a major organization representing labor issues and concerns; (2) a BLS official from the Office of Compensation and Working Conditions who published a 2008 article addressing the accuracy of injury and illness data; and (3) a researcher at the National Institute for Occupational Safety and Health (NIOSH) who heads an effort to collect national occupational injury and illness data from a representative sample of emergency departments in the United States.

GAO Survey of Occupational Health Practitioners

We surveyed three categories of occupational health practitioners about how they treat injured or ill workers; the extent of their involvement with OSHA recordkeeping responsibilities; their views on worksite safety incentive programs; and their perspectives on factors that affect the completeness and accuracy of employer records of workplace injuries and illnesses. We surveyed (1) occupational physicians identified on lists compiled by the American Medical Association of all practicing physicians in the United States with a primary specialty of occupational medicine, (2) occupational physician assistants identified on lists compiled by the American Academy of Physician Assistants of all certified physician assistants in the United States who specialize in occupational medicine, and (3) nurse practitioners specializing in occupational health identified on lists compiled by a medical information broker of all nurse practitioners in the United States.

Study Population, Sample Frame, and Sample Design

We designed and implemented a dual mode survey (mail and Web-based) to obtain information from occupational health practitioners. We obtained lists of the occupational health practitioners from Medical Marketing Service, a data management firm providing medical lists to marketers, researchers, and government agencies. We constructed our universe of physicians from the American Medical Association’s Physician Masterfile of all practicing physicians in the United States with a primary specialty of occupational medicine; our universe of physician assistants from the American Academy of Physician Assistants’ list of physician assistants specializing in occupational medicine; and our universe of nurse practitioners from a comprehensive list of nurse practitioners specializing in occupational health. We independently selected a random sample from each of the three groups, resulting in a sample of 409 of the 1,941 physicians; 396 of the 1,246 physician assistants; and 382 of the 861 nurse practitioners, for a sample of 1,187 of the total 4,048 occupational health practitioners. Due to the results of our nonresponse analysis (described below) we restricted our sample of physician assistants to those who were certified, which resulted in a sample size of 340 certified physician assistants. Therefore, our resulting total sample was 1,131 (see table 4).

Out of the sample of 1,131 health practitioners, 504 completed the questionnaires, for a total response rate of 45 percent. This response rate allowed us to generalize our results to the total population of the three groups. All estimates we report from the survey results (including those in this appendix) have a margin of error of plus or minus 7 percentage points or less at the 95 percent confidence level. See table 4 for the disposition of the three separate groups of health practitioners.

The sample size for each practitioner group was determined to be able to detect a 10 percent difference between the sample estimate and the true population with a significance level of 0.05. We also oversampled from each of the populations to account for practitioners who would not respond to our survey and those we determined to be out of scope, such as practitioners who did not treat workers for occupational injuries or illnesses during 2008.

The respondents treated workers in various industries, and varied in the number of years they had treated workers, but the majority had been treating workers for 10 years or more (see figs. 7 and 8). The majority also treated more than 100 workers in 2008 (see fig. 9).

Developing the Questionnaire, Content, and Question Wording

To develop survey questions, we drew on information we previously gathered from interviews with occupational safety and health stakeholders, as well as from scholarly studies from the field of occupational safety and health research. Appendix II provides our survey instrument. Two GAO survey specialists designed the questionnaire in collaboration with the analysts staffed to the engagement. We pretested the questionnaire with nine health practitioners who represented the three study populations and made appropriate modifications based on their feedback. Appendix III provides additional selected survey results.

Data Collection and Nonresponse Follow-up

We conducted the survey using a self-administered questionnaire, and offered prospective respondents the option of completing and mailing a hard copy questionnaire or completing the questionnaire online. We offered both options because during our pretests, health practitioners advised us to offer a Web-based option; however, a study of occupational health practitioners showed that, given the choice, 90 percent of respondents chose to respond by mail. None of our three data sources included e-mail addresses, so we mailed a hard copy of the questionnaire with instructions to either mail the completed paper version in a prepaid envelope or to go to a Web site designated for the survey and use a preassigned login identification and password. To encourage further participation, we mailed a second questionnaire to all those who had not yet responded. We also contracted with a survey research firm to make follow-up phone calls for those who had not responded.

Population Estimates and Sampling Errors for Probability Samples

Weighting Survey Response

Since we drew an independent sample from each occupational practitioner group, each response represented a different number in the population of the group. To enable data from the survey response to represent the combined population of three occupational health practitioner groups, we calculated weights of the responses for the three groups. We calculated the weights as w denotes the weight for the hth occupational practitioner group, N denotes the population for the hth occupational practitioner group, n denotes the total number of survey responses for the hth practitioner h denotes practitioner group: 1 = physicians, 2 = physician assistants, and 3 = nurse practitioners.

Population Estimates and Confidence Intervals

We also estimated population statistics for the combined three health practitioner groups by calculating the difference in weights among the groups. We calculated the ratio estimate of the overall population by using the following equation: R = (Σw Σ y hi) / (Σw Σ xhi) w denotes the sample weight for the hth stratum, y hi represents the ith response of the variable y response in the hth stratum (for example, yhi =1 if the ith response was ‘Construction’ in Q5, yhi =0 otherwise), xhi represents the ith response of the variable x in the hth stratum (for example, xhi = 1 if the ith response was ‘LESS THAN 100 WORKERS’ in Q3, xhi =0 otherwise), and R denotes a population estimate of the ratio (in this example, the ratio of respondents who treated workers from the construction industry among those who treated less than 100 workers in calendar year 2008).

To assess the precision of our estimates, we calculated confidence intervals for each measure. A confidence interval gives an estimated range of values, calculated from sample data, which is likely to include the true measure of the population. As is commonly done, we calculated 95 percent confidence intervals. We obtained the 95 percent confidence intervals of our population estimates by using methods detailed in Cochran and Hansen, Hurwitz, and Madow, since our estimates were calculated from our survey—that is, from a stratified sample. We estimated the population percentage and the confidence intervals of those percentages using specialized software for survey data analysis—SUDAAN®.

Nonsampling Errors

We took steps in developing the questionnaire, collecting the data, and analyzing the data to minimize the variability in the survey results due to nonsampling errors—such as those resulting from the differences in the way a particular question is interpreted or the sources of information available to respondents. The data collected were analyzed by a data analyst working directly with staff who have subject matter expertise. After the data were analyzed, a second independent data analyst checked all computer programs for accuracy. We contracted with an outside company to enter the data from the paper questionnaires into a database, and we checked a 10 percent sample of the database as a quality control measure. Respondents who completed questionnaires online entered their answers directly.

Nonresponse Bias Analysis Because only about 45 percent of the health practitioners (47 percent of physicians, 48 percent of physician assistants, and 39 percent of nurse practitioners) provided usable responses to our survey, bias from nonresponse may result. If the views of those who did not respond differed from the views of those who did respond to some survey questions, the estimates made solely from those who did respond would be biased from excluding parts of the population with different characteristics or views. To limit this kind of error, we made several attempts to gain the participation of as many occupational health practitioners as possible, including additional mailings and contracting with a survey firm to call nonrespondents to encourage their participation. To assess the likelihood of significant bias, we collected additional data through the calls made by our contractor concerning reasons why the practitioners did not respond, and by trying to persuade them to answer three key questions from our survey on the phone. We also conducted several analyses of these follow-up data, our survey data, and data we had about the population from which we sampled, to attempt to detect any nonresponse bias.

We analyzed practitioner characteristics that may have been related to what their answers to our survey questions would have been if they had responded. The variables available to us for this analysis differed by practitioner type. For physicians, we used age, gender, number of offices, type of physician (medical doctor or doctor of osteopathic medicine), and geographic region. For physician assistants we used age, gender, years since graduation, and certification status. For nurse practitioners, we used age, gender, and practice setting. Using logistic regression, we compared the characteristics of nonrespondents to respondents to determine if any of these characteristics were more likely to be associated with being a responder. With the exception of one characteristic for one group, we did not detect a significant difference between those who chose to respond and those who did not. We did detect a difference in our sample of physician assistants: those who were certified were more likely to respond to our survey than those who were not. Because we could not be sure if this represented a bias and because we later determined that noncertified physician assistants were likely out of scope, we removed all noncertified physician assistants from our estimates, which resulted in eliminating 13 respondents and 43 nonrespondents from our final data.

Our follow-up calls had several purposes related to our nonresponse analysis. The primary purpose was to attempt to convert nonresponders to responders by persuading them to complete the survey. If after several attempts the respondent indicated that he or she would not complete the survey, our contractor asked the person to answer three key questions from our survey: (1) whether or not any of their worksites had incentive programs, (2) whether they had ever observed or experienced pressure from workers to downplay injuries or illnesses, and (3) whether they had observed or experienced such pressure from company officials. Because only 14 nonrespondents answered at least one of these questions, we were unable to conduct any statistical analyses to detect whether their responses to these three questions were different, in aggregate, from those of the respondents. Regardless of whether or not the respondents answered these three questions, the respondents were asked why they would not complete the full survey. Sixty-four nonrespondents answered this question. Of these, 53 (83 percent) offered reasons that suggested they were likely out of scope because they had changed careers, were retired, or the survey did not relate to their job. This suggests that nonresponse bias may not be substantial as it is possible that many nonresponders were actually out of scope and would not have been able to complete the survey.

Finally, we analyzed the differences in response patterns between those who answered in the earlier period of the survey timeframe (early responders) and those who responded only after follow-up attempts (late responders). It is possible that the late responders more closely resemble the nonresponders than the early responders. Based on chi-square tests, we detected no significant difference in survey responses to our three key questions between the early and the late groups, which may suggest that actual nonrespondents would not have answered in a substantially different way from those who responded. While the possibility exists that the true results for the entire population might be different from those we estimated in our report, based on these various nonresponse analyses, we believe that nonresponse bias is unlikely.

Statement of Compliance with Generally Accepted Government Auditing Standards

We conducted this performance audit from August 2008 through October 2009 in accordance with generally accepted government auditing standards. These standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Survey Instrument for Occupational Health Practitioners

Appendix III: Selected Questionnaire Results

All estimates we report from the survey results have a margin of error of plus or minus 7 percentage points or less at the 95 percent confidence level.

Health practitioners provided their opinions on the efficacy of safety incentive programs (see fig. 10).

In addition to experiencing pressure to downplay injuries and illnesses, respondents also observed behavior by workers and company officials that would result in underrecording (see fig. 11).

Finally, health practitioners reported the impact they thought various factors had on whether injuries and illnesses are recorded accurately in the employers’ log (see fig. 12). They also reported how often they experienced various requests from workers or company officials (see fig. 13).

Appendix IV: OSHA’s Forms for Recording Work-Related Injuries and Illnesses

Appendix V: High Hazard Industries Included in ODI Universe as of August 2009

Food Crops Grown Under Cover

Appendix VI: Comments from the Department of Labor

Appendix VII: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the contact named above, Gretta L. Goodwin, Assistant Director, and Mary A. Crenshaw, Analyst in Charge, managed all aspects of this assignment and Sara Pelton, Analyst, and Tanya Doriss, Analyst, made significant contributions to all phases of the work. Shana B. Wallace, Pamela R. Davidson, Dae B. Park, Catherine M. Hurley, Amanda K. Miller, and Carl M. Ramirez provided assistance in developing and applying the methodologies and analyzing the data. James M. Rebbe provided legal assistance, Susan L. Aschoff assisted with message and report development, and Mimi Nguyen and James E. Bennett drafted the report’s graphics.

Related GAO Products

OSHA’s Voluntary Protection Programs: Improved Oversight and Controls Would Better Ensure Program Quality. GAO-09-395. Washington, D.C.: May 20, 2009.

Workplace Safety and Health: Safety in the Meat and Poultry, While Improving, Could Be Further Strengthened. GAO-05-96. Washington, D.C.: January 12, 2005.

Occupational Safety and Health: Efforts to Obtain Establishment- Specific Data on Injuries and Illnesses. GAO-98-122. Washington, D.C.: May 22, 1998.

Occupational Safety and Health: Changes Needed in the Combined Federal and State Approach. GAO/HEHS-94-10. Washington, D.C.: February 28, 1994.