1

Overview of the Results of the Household Survey of Occupational Injuries and Illnesses Pilot and Ongoing BLS Activities

December 5, 2020

Erica Yu

Kristen Monaco1

Bureau of Labor Statistics

2 Massachusetts Ave NE

Washington, DC 20212

Background

Since 2009, the Bureau of Labor Statistics (BLS) has conducted research into the completeness of the 

Survey of Occupational Injuries and Illnesses (SOII), stemming from concerns that not all in-scope 

cases were included in data submitted by establishments to BLS.2There is no consensus on the 

primary reason for establishment under-reporting, but factors include lack of consistent 

recordkeeping over the year, gaps in understanding OSHA’s recording criteria, and potential 

disincentives for both establishments and workers to report injuries and illnesses. One proposal for 

overcoming potential filters between the worker, establishment recordkeeping, and establishment 

reporting to BLS was to approach workers directly to collect information on workplace injuries and

illnesses, conducting a household survey of occupational injuries and illnesses (HSOII).

In fiscal year (FY) 2015, BLS contracted with Westat to review surveys with similar constructs and 

sensitivity levels and recommend potential designs for a survey with the goal of capturing workrelated injuries and illnesses directly from workers that result in nationally representative estimates 

that could be compared to the SOII.3The Westat report provided analysis of the pros and cons of 

different modalities (face-to-face, mail, telephone) and structures (stand-alone survey, adding a 

module to an existing survey, developing a follow on survey). Westat recommended a pilot survey 

noting that the pilot structure would likely differ from the final survey structure. A pilot would have 

utility if it provided information on response rates, differences in recall periods, differences in 

responses by modes of collection and measures of proxy reporting quality.

In FY 2016-2017, BLS contracted with NORC to further evaluate the suitability of existing surveys and 

frames to capture data on work-related injuries and illnesses.4 NORC also provided reports on survey 

design options and an evaluation of existing questions/instruments on workplace injuries and 

illnesses. This contract also involved developing and cognitively testing a survey instrument. The goal 

of the survey instrument was to capture the same elements as the SOII and provide additional details 

that would inform data quality research. These additional measures included constructs to capture 

whether the worker reported the injury or illness, received medical care, or filed for workers 

compensation. There were three rounds of cognitive testing used to both adjust survey questions for 

comprehension and test for the quality of proxy reporting.

 

1 Questions concerning this report can be addressed to Monaco.Kristen@bls.gov and Yu.Erica@bls.gov

2

For additional information on this research, see https://www.bls.gov/iif/data-quality.htm.

3

“Pilot Study Design for the SOII Employee Survey” https://www.bls.gov/iif/westat-worker-survey-pilot-designfinal-report.pdf.

4

“Designing a Household Survey of Injuries and Illnesses” https://www.bls.gov/iif/norc-final-report.pdf.
2

HSOII Structure, response rates, and major findings

In FY 2018, BLS contracted with ICF to do a large-scale pilot test of the survey instrument developed 

by NORC. The pilot test targeted a sample of roughly 4,000 individuals and was conducted via 

telephone between August 2017 and July 2018, using a mix of landline (10%) and cell phone frames 

(90%). The pilot included a pretest of 50 completed interviews and was used to identify questions 

that were confusing or difficult to answer and identify any problems with skip patterns in the survey 

instrument. The interviewers for the pilot received classroom training specifically related to the 

survey and completed practice interviews.

The scope of the survey was individuals 18 years of age or older who had worked in the 12 months prior 

to the interview. The screener question was, “In the last 12 months did you do ANY work for pay?”5 At 

the start of the survey, respondents were asked a series of eligibility screening questions, including 

whether the person speaking is an adult residing in the United States, whether the person lives in a 

private residence, how many adults live in the household, and whether anyone in the household did any 

work for pay in the last 12 months. Of all contacted respondents, 10.1% (n = 801) broke-off from the 

survey during this short eligibility screening stage, before eligibility could be determined. Of all 

respondents who were found to be eligible and began the main survey, 7.6% (n = 314) broke-off. Across 

the full survey, there were 1,115 break-offs (14.0% of the full sample). Table 1 presents the break-off 

rates for the eligibility screening and main portions of the survey.6

Table 1. Count and proportion of outcomes of all contacted cases

Outcome Type Count Proportion

Eligibility Screening – all respondents

Eligible 4,133 52.0%

Ineligible/Item refusals1 3,009 37.9%

Break-off 801 10.1%

Total 7,943 100.0%

Main Survey – eligible respondents

Complete interview2 3,819 92.4 %

Break-off 314 7.6%

Total 4,133 100.0%

1 Respondents who refused to give an answer for selected items that led directly to the 

end of the survey, according to the instrument requirements. Because ineligibility is 

partly based on item refusals, it is not possible to separately calculate ineligibility and 

item refusals that lead to the end of the survey.

2 Includes 21 respondents who completed the interview early due to an instrument 

programming error.

 

5 The probe for this is, “this would include salary, compensation, or other profit.”

6 The numbers presented below differ slightly from those presented in the BLS response to GAO on “Working 

Children,” (https://www.bls.gov/bls/congressional-reports/response-to-working-children-federal-injury-data-andcompliance-strategies-could-be-strengthened.pdf) due to different treatment of interviews that were terminated 

due to survey error as well as differing break-off criteria.
3

Although any number of break-offs is not desirable, these break-off rates are within the expected range. 

These rates are similar to outcomes reported in the literature for RDD telephone surveys, such as the 

6%-16% termination rate7range reported by Kennedy et al. (2006). 

Among the 7,943 individuals reached by ICF interviewers, 4,133 were eligible for the survey. Of the 

4,133 eligible, there were 3,819 completed surveys and 314 break-offs. Based on the total number of 

contacts attempted, the 3,819 completed interviews yielded a response rate of 14.4%. This is based on 

American Association of Public Opinion Research, Response Rate 3, which includes assumptions of 

eligibility for cases of unknown eligibility.8

To capture occupational injuries or illnesses, two questions were asked immediately following the 

screener questions. The first was, “In the last 12 months have you experienced any injuries or 

illnesses related to any job you held?” and 339 respondents answered “yes.” Those who did not 

respond in the affirmative were asked, “Just to make sure, I’m going to read some examples of 

work-related injuries and illnesses ... In the last 12 months, have you experiences any of these, or 

other types of injuries or illnesses, related to any jobs you had?” Surprisingly, 417 answered “yes,” a 

higher number of affirmatives than the first question. 

Given that the injury or illness status of the respondent determined which questionnaire path the 

respondent followed and resulted in very different survey experiences, Table 2 presents break-off rates 

for these two subgroups. The break-off rate is significantly higher for respondents who report an injury 

or illness, X

2

(1, N = 4,133) = 121.4, p<.001). 

Table 2. Report of injury or illness (n = 4,133)

Break-off All Other

Injury/Illness Total Count Proportion Count Proportion

Injury/Illness 7561 130 17.2% 626 82.8%

All other 3,377 184 5.4% 3,193 94.6%

Total 4,133 314 3,819

1

This count includes 36 respondents who broke-off after reporting an injury or illness 

but before the official count (720 respondents with at least one injury or illness) was 

calculated.

While the total number of workers reporting an injury or illness suggests an unweighted injury and 

illness rate of 18% (756/4,133), these questions were not designed to correspond to the definition of 

OSHA-recordable injury or illness. After the pilot was complete, two experienced BLS staff members 

 

7 Terminators are defined by Kennedy et al (2006) as any respondent who answers at least one substantive 

question but quit the survey prematurely. The termination rate is computed as the number of terminations divided 

by the sum of completed cases plus terminations.

8

For further information on AAPOR response rates, see The American Association for Public Opinion 

Research. 2016. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th 

edition. AAPOR. https://www.aapor.org/AAPOR_Main/media/publications/StandardDefinitions20169theditionfinal.pdf
4

(“coders”) who specialize in coding SOII cases were tasked with reading the narrative description from 

each respondent and identifying whether the case was OSHA recordable. Many narratives did not 

contain adequate information to determine OSHA-recordable status. For this reason, the staff focused 

on cases determined to be OSHA-recordable and combined the not-recordable and unknown cases 

into a second classification. The coders agreed that 145 cases of the 756 were OSHA-recordable 

(19%). This suggests an overall, unweighted prevalence of 3.5% (145/4,133). This number is not 

intended for comparison to the SOII published numbers due to the difficulty in confirming whether 

cases were OSHA recordable.

It is important to note that the coders had disagreement in 24% of cases, with one coder determining 

the case was OSHA-recordable and the other determining the case was either not recordable or 

unknown. These coding disagreements were fairly symmetrical. 

The higher rate of break-offs among those reporting workplace injuries or illnesses, along with the 

high rate of unknown OSHA-recordable codes assigned by the BLS coders, suggests that item nonresponse and response quality in the workplace injury or illness narratives must be further examined 

and addressed in further HSOII tests. For this reason, we are not producing weighted estimates of 

injury and illness prevalence.

HSOII Item-level break-offs, don’t knows, and refusals

Item-level analyses can identify individual survey items that appear to cause response problems. For 

these analyses, we limited the dataset to items from the main survey with at least 50 respondents in 

universe and include counts to express the magnitude of the impact. The items that appear to have 

break-off problems are shown in Table 3. 

The items with the highest break-off rates were all in the injury and illness question series, with the 

highest item-level break-off at 4.0% on the item signaling the start of the second injury or illness 

question series. The other high break-off items (B1, A3A, A5C, A2B, and A3B) occur early in the survey 

when respondents are still learning what the survey is like and may not have fully committed to 

participating in the survey yet. However, in the injury or illness question series beyond Sections A and B, 

break-off rates for all items were at less than 1%, or one person. In Sections G and H, which are outside 

of the injury and illness question series and asked for all respondents, break-off rates in the non-injury 

and illness group were higher than or equal to the injury or illness group on all items, suggesting that 

break-off rates for the non-injury and illness subgroup would likely also be high if they has been asked as 

many questions as the injury and illness subgroup. The relatively high break-off rates by the injury and 

illness subgroup likely reflect a combination of the problematic early questions and a longer survey path 

leading to more opportunities for break-off.

Table 3. Top item-level break-offs (of all items with universe >=50) 

Item Question Description Universe

Break-off 

Count

Break-off 

Rate

B1 Injury/Illness Event (Loop 2) 149 6 4.0%

B1 Injury/Illness Event (Loop 1) 720 27 3.8%

A3A Acute or Chronic screener (Injury 1) 756 16 2.1%

A5C Whether 2nd injury/illness related to 1st 173 3 1.7%

A2B Injury/Illness screener follow-up 3763 57 1.5%
5

A3B Date of 1st injury/illness onset – month 340 5 1.5%

A similar analysis of “don’t know” responses highlights the survey items that respondents may have had 

difficulty answering, shown in Table 4. Several items are expected to have high “don’t know” responses 

due to recall difficulty, such as date of injury or illness onset (A3B) and number of days until returned to 

work (D5). Other items are unexpected, such as whether the respondent ever returned to work (D4) and 

whether the respondent received worker’s compensation (E4). 

Table 4. Top item-level don’t know responses (of all items with universe >=50)

Item Question Description Universe

Don’t Know 

Count

Don’t Know 

Rate

A5B1 Date of 2nd injury/illness onset – month 96 19 19.8%

A3B Date of 1st injury/illness onset – month 340 64 18.8%

D5 How many days after until start work (Loop 1) 68 9 13.2%

A3B Date of 1st injury/illness onset – year 335 33 9.9%

A5B1 Date of 2nd injury/illness onset – year 95 8 8.4%

E4 Whether receive worker’s comp (Loop 1) 98 8 8.2%

D4 Whether returned to work (Loop 1) 80 4 5.0%

C2 Days off recommended by medical (Loop 1) 181 9 5.0%

The same analysis of item-level refusals, presented in Table 5, also shows expected and unexpected 

results. As expected, potentially sensitive information like the name of the company that the 

respondent worked for (F8) and the respondent’s age (G3) and race (G2A) show relatively high refusal 

rates. Whether the respondent was assigned different job tasks (D17) also show relatively high refusal 

rates; however, this 2.9% refusal rate ultimately amounts to two refusals, which may not generalize to 

the larger population. 

Table 5. Top item-level refusals (of all items with universe >=50)

Item Question Description Universe

Refusal 

Count

Refusal 

Rate

F8 Name of company 649 90 13.9%

B1 Injury/Illness Event (Loop 2) 149 8 5.4%

B3 Injury/illness Nature (Loop 2) 141 6 4.3%

D17 Whether assigned different job tasks (Loop 2) 68 2 2.9%

G3 Age 3886 94 2.4%

G2A Race 3897 88 2.3%

D19B Injury/Illness cause quit (Loop 2) 136 3 2.2%

C1 Whether days off rec. by medical (Loop 2) 141 3 2.1%

B2 Injury/Illness Source (Loop 2) 143 3 2.1%
6

Conclusions and next steps

Initial analysis of the results from the pilot HSOII identified relatively high levels of break-offs for 

individuals reporting an injury or illness, obstacles to properly identifying whether an injury or illness 

was OSHA-recordable, and questions that were associated with the highest levels of refusals or “don’t 

know” answers. The findings, taken together, resulted in BLS internal work that began in FY 2020 and 

will continue through FY 2021.

This work centers on two areas. First, map all survey instrument questions back to key constructs 

required for an HSOII to potentially reduce respondent burden, mitigate break-offs, and center the 

initial questions around the concept of OSHA-recordability. This work also considers the appropriate 

reference period for the injury/illness as well as how to reframe questions regarding the details of the 

injury/illness. Second, conduct cognitive testing of a restructured HSOII, including revised screener 

questions as well as the questions identified in tables 3-5 as being associated with higher levels of breakoff, unknowns, and refusals.

It should be noted that even a redesigned survey instrument that contains focused screeners and 

questions tailored to determine OSHA-recordability is unlikely to be usable to generate injury/illness 

rates directly comparable to an establishment survey. A household survey should be seen as a 

complementary source of information to the establishment survey.