THE OCCUPATIONAL SAFETY AND HEALTH SURVEY: INSTRUMENT REDESIGN

Leda Kydoniefs, Bureau of Labor Statistics

Bureau of Labor Statistics, 2 Massachusetts Ave., N.E., Room 3160, Washington, DC 20212

KEY WORDS: survey methodology, forms redesign,

occupational injuries and illnesses.

Introduction

A major redesign of the Occupational Safety and

Health Statistics (OSHS) system, the nation's primary

source of information on job-related injuries and illnesses, began in 1988. The OSHS survey is a mandatory Bureau of Labor Statistics (BLS) survey, and is

collected via mail in cooperation with State agencies.

In order to fully convey to the reader the extensiveness

of the OSHS redesign, this paper begins with a brief

historical development and overview of the program.

Historical Overview1

Occupational Safety and Health Act

The Occupational Safety and Health Act of 1970

requires the Secretary of Labor to develop an effective

program of occupational safety and health statistics. In

1971, the Secretary of Labor delegated to the

Commissioner of BLS the responsibility of furthering

the purposes of the Act "by developing and maintaining an effective program of collection, compilation,

analysis and publication of occupational safety and

health statistics." The Secretary further directed the

Commissioner of BLS to coordinate the above

functions with the Assistant Secretary for Occupational

Safety and Health.

OSHA Recordkeeping System

The recordkeeping system, which is the foundation

of BLS's statistical program, was developed to aid the

Occupational Safety and Health Administration

(OSHA) in setting standards, to assist safety and health

officers in identifying hazardous operations, and to

provide BLS and State agencies with uniform and

reliable safety and health statistics. Further, the statistical program would provide employers and employees

with information about conditions at their workplace.

In 1978, a recordkeeping form designed to streamline OSHA mandated recordkeeping and reporting was

implemented. It is referred to as the Log and Summary

of Occupational Injuries and Illnesses (OSHA 200) or

simply the OSHA Log. Until 1991, BLS managed the

Log for OSHA, distributing copies to employers, and

ensuring that recordkeeping guidelines were conveyed

 

1Portions of the Historical Overview were taken from the

BLS Handbook of Methods, Bulletin 2285.

clearly. The employer is mandated to record onto the

OSHA Log all work-related deaths, illnesses, and injuries. Furthermore, the employer is required to

describe each case in detail using a supplementary

record such as the OSHA 101 provided by OSHA, or

by using some equivalent form, (i.e., a form that

includes all necessary information) such as a State

worker's compensation form. Distribution of OSHA

101 was also managed by BLS for OSHA until 1991.

Annual Survey of Occupational Illnesses

and Injuries

Shortly after the Occupational Safety and Health

Act was enacted, BLS in cooperation with State agencies began to conduct the Annual Survey of

Occupational Injuries and Illnesses providing national

measures of the incidence and severity of workplace

injuries and illnesses. The Annual Survey is collected

via mail from a probability sample of approximately

280,000 establishments in private industry.

Participation in the survey is mandatory.

Establishments with 11 employees or more are in

scope, and self-employed persons are not covered by

the survey. Prior to the redesign, each sampled establishment provided summary totals of the number of

injury, illness, or fatality cases experienced during the

calendar year. The total numbers of lost work days and

days of restricted activity were also collected, as was

annual average employment and total hours worked by

employees. Employers copied information for the

injury and illness portion of the Annual Survey directly

from the OSHA Log that they maintained throughout

the year. All employers covered by the Act are required to maintain the Log with the exception of employers with fewer than 11 employees, and employers

in low-risk industries such as retail and real estate.

The Annual Survey samples employers who are ordinarily exempt from this recordkeeping requirement,

but these sampled employers are required to maintain

the OSHA Log for the year they are in the survey.

Supplementary Data System

Not long after the Annual Survey began, safety

and health analysts and other interested parties identified a need for greater detail than that of incidence

rates. They requested information about the characteristics of the occupational injuries and illnesses and the

workers to whom they were occurring. It was generally recognized that records routinely generated by
2

State workers' compensation programs such as employee and employer reports, medical reports, compensation award records were all valuable sources of information about occupational injuries and illnesses.

As a result, State workers' compensation agencies

began, in 1976, to participate in BLS's Supplementary

Data System (SDS).

The SDS program was conceptualized as providing data to "supplement" that provided by the Annual

Survey. The program's source of information was a

first report of injury or illness, which employers and

insurance carriers submit to State workers' compensation agencies. SDS data have helped in the identification of general patterns in the characteristics of workrelated injuries and illnesses. The SDS program, never

designed as a nationwide system, was established in

cooperation with 27 States, and included anywhere

from 12 to 36 cooperating States in a given survey

year. Although the SDS provided valuable information

and standardized the classification, processing, and

tabulations of data, it was never a complete census of

occupational injuries and illnesses. In addition, it

suffered from variations in coverage and reporting

requirements that reflected differences in State

workers' compensation laws.

Programmatic Review

By the mid-1980's OSHA, BLS and numerous

representatives of business and labor had begun to feel

that the existing BLS safety and health surveys should

be integrated into one internally consistent statistical

system. For example, the National Academy of

Sciences (Pollack, & Keimig, 1987) in reviewing BLS

programs felt that the Annual Survey did not provide

needed information about the worker and the circumstance of the case, within an industry context. Also,

the SDS was deemed insufficient because of its partial

case coverage and lack of consistency across all States.

It was recommended that the Annual Survey be modified to include specific categories of injuries and illnesses, thereby folding SDS type of data and incidence

data into one internally consistent program that could

produce national estimates on both incidence rates as

well as case characteristics.

The BLS, in responding to these recommendations

began an all-inclusive redesign of the Occupational

Safety and Health System (OSHS). The redesign effort

has included a review of all aspects of OSHS: Survey

content, sampling, forms design, collection and processing. This paper will focus on the survey instrument

redesign effort: changes in content and question wording; forms redesign; methodology utilized to test the

"new" survey format; the data analysis and resulting

survey revisions.

Revisions to Survey Content

The OSHS survey was expanded to include details

about the specific injuries and illnesses incurred by

employees. The following details were added to the

survey:

Details About the Injured Worker

Name and social security number

Gender, Age, and Ethnicity

Length of service at the establishment

Occupation

Details About the Injury or Illness

When did the injury occur?

How many days of lost work or days of restricted work

activity resulted?

What was the employee doing just before the incident

occurred?

What happened - how did the injury or illness occur?

What was the injury or illness - i.e., what part of body

was affected and how was it affected?

What object/ substance directly harmed the employee?

The employer is asked to provide the above information on each illness or injury case that occurs in the

workplace. A prediction is made as to the expected

number of cases an employer will have, and the appropriate number of "case forms" are provided. If the

employer is expected to have more than 20 cases, instructions are included in the survey directing the respondent to sample from their list of injuries and illnesses so as to minimize, as much as possible, their

response burden .

Survey Redesign: Research and Development

Redesign Overview

The survey redesign portion of the project began in

1989 and consisted of several pilot/feasibility studies

investigating alternative survey formats. As previously

discussed, programmatic emphasis was placed on expanding the data base to include not only incidence

rates of injury and illness by industry, but to provide

demographic characteristics of the injured worker as

well as begin to collect detailed information on the

incidence itself. In 1991, the decision regarding the

expanded scope of the survey was finalized. The complexity of the survey was such that the design of a

booklet format became a necessity, and forms redesign

efforts intensified. The objectives were four-fold:

1. Develop a booklet survey form with accompanying instructions that was "user friendly."

2. Design a questionnaire format with a built-in

flexibility that would permit certain minor vari-
3

ations in item content. (These variations are required in order to maximize the survey's utility

across the nation.)

3. Test and evaluate old (i.e., rephrased) questionnaire content as well as new survey items.

4. Test and evaluate the final product, i.e., the redesigned OSHS survey that was developed.

To achieve these goals, a variety of behavioral

science and test development methods were used, each

of which added a different perspective to our knowledge base and forms redesign efforts. (For an overview

of these methods see Forsyth & Lessler, 1990; Gower

& Nargunkar, 1991; and Jabine, Straf, Tanur, &

Tourangeau, 1984.) Focus groups (Krueger, 1988)

were used primarily in the beginning of our test development efforts to refine the scope of the survey, the

item wording, and obtain a gross estimate of respondent burden. Respondent observations using a thinkaloud protocol (Mullin, Miller, Melis-Wright, & Stone,

1981) were used when the survey redesign approached

completion. Pretest mailing of the survey (Dillman,

1991) with an accompanying respondent (probing)

questionnaire served as the last step in the "test development" phase prior to fielding the survey. These

"pretest" respondents were also contacted via phone

and asked additional questions regarding their response to the redesign of the survey.

The sections that follow present the developmental

steps that led to the final survey booklet: the early

feasibility studies, pilot studies, and repeated testing of

draft survey booklets.

Feasibility Studies

During 1988, twelve feasibility studies (Bureau of

Labor Statistics, 1988) were conducted by BLS.

Cumulatively, the studies collected data from 20 States

and sampled about 100 establishments per State (a

total sample of 2000 establishments). These studies

were exploratory in nature designed to find out what

kind of records/data employers kept on workplace injuries and illnesses, and to inquire whether employers

were willing to share these data with BLS.

The research was conducted via mail, and respondents were asked to voluntarily submit copies of particular records. Depending on the study, employers

were asked to send in copies of the OSHA Log, first

reports of injury, and/or injury forms filed with

Workers' Compensation. The feasibility studies

yielded encouraging results: Most employers maintained the OSHA Log and some accompanying first

report of injury. Moreover, they were generally willing

to report on the data contained in these documents.

As a result of these positive findings, BLS proceeded to further elaborate and clarify the data elements that would eventually be added to the Annual

survey. To assist in their effort, BLS solicited input

from the survey data user community requesting suggestions for data elements of interest to various groups.

A variety of organizations were approached: OSHA,

National Institute for Occupational Safety and Health

(NIOSH), the BLS Labor Research Activity Council

(LRAC), the BLS Business Research Activity Council

(BRAC) as well as many State Labor and State Health

Departments.

Pilot Studies: Phase I

In 1989, BLS returned to the field for additional

research. The primary objective was to investigate

which of the numerous data elements generated internally by BLS staff and/or suggested by the survey user

community were collectible. A second objective was to

investigate the utility of certain basic changes in existing records and/or data collection formats. The reader

will recall that in addition to conducting the Annual

Survey, BLS managed (for OSHA) the OSHA Log and

OSHA 101 - the supplementary record developed to

provide case injury or illness detail. With these pilot

studies BLS was exploring improvements to all safety

and health related data collection formats.

Five pilot studies were conducted in a total of

fifteen (13) States (Bureau of Labor Statistics, 1989).

About four hundred (400) establishments were sampled

within each State yielding a total sample of approximately 5200 establishments, or 1040 establi-shments

per pilot study. The five pilot studies were as follows:

Pilot 1. Testing a record keeping format that combined the information collected via the OSHA

Log and first report of injury (e.g., OSHA

101) into one form. The employer was also

asked to code cases identifying the nature of

the injury, the part of body effected, and the

source of the injury.

Pilot 2. Testing the same record keeping format described in Pilot 1 above, with the exception

that the employer did NOT have to precode

data.

Pilot 3. Testing a revised supplementary record

(OSHA 101). As with Pilot 1, the employer

was asked to precode case characteristics.

Pilot 4. Testing the revised supplementary record

described in Pilot 3 without the coding of case

characteristics.
4

Pilot 5. Within this study, respondents were asked to

submit a copy of the OSHA Log. The State

then attempted to match the injury and illness

cases identified on the log to the State workers

compensation files submitted by the establishment, and extract the appropriate data

elements regarding case characteristic details.

The reader will recall earlier discussion (see

Historical overview) on the detailed data generally available within State workers' compensation agencies.

Analysis of the five pilot studies resulted in a

greatly reduced list of data elements under consideration for inclusion in the Annual Survey. Further, it

was concluded that employer self-coding was not a

feasible pursuit. Employers made too many errors

when asked to apply an unfamiliar coding scheme to

their own records.

Pilot Studies: Phase II

In 1990, BLS began a second phase of pilot

studies. Starting from where the first set of pilots left

off and applying the knowledge gained, BLS proceeded

to refine and retest the collection methodologies described in Pilots 2, 4, and 5 above. As before, each

pilot study was conducted in three States (9 States in

total) and approximately 400 establishments per State,

yielding a sample size of approximately 1200 establishments per pilot study. Once again, employers were

asked to:

a. Complete a record keeping format that combined

the information collected via the OSHA Log and

first report of injury into one form.

b. Complete a revised supplementary record

(OSHA 101).

c. Submit copies of the OSHA Log with the State

following improved procedures to match cases

identified by the employer to the workers compensations forms filed with the State.

As this phase of pilot studies was being conducted,

OSHA and BLS agreed to separate the mandatory

record keeping requirements and accompanying documents from the statistical survey or data collection

aspects of occupational illnesses and injuries. This

separation was welcomed by BLS, interested in

increasing the distance between its pledge of confidentiality and data collection for statistical purposes only

and OSHA - a government agency responsible for

assessing fines for noncompliance to Safety and Health

regulations. Consequently, BLS transferred all responsibility for the distribution and revisions to the

OSHA Log and the supplementary record (OSHA 101)

to OSHA. Taking the data collected thus far from the

Phase II pilot studies, as well as the earlier BLS feasibility and Phase I pilot studies, OSHA began its own

efforts at revisions to the OSHA Log and accompanying supplemental record. BLS was free to focus solely

on implementing revisions to the statistical portion of

occupational injuries and illnesses program, i.e., the

Annual Survey. Momentum increased as development

work was directed solely on the survey booklet -

"Survey of Occupational Injuries and Illnesses".

Development of Survey Booklet

As the pilot studies were concluded, BLS entered a

different stage in the survey instrument redesign. BLS

now had a clear sense of direction with respect to the

type of questions that would be added to the survey,

and felt confident regarding the availability of employer records on the data of interest. It was decided

that in order to accommodate the additional data elements and provide (survey completion) instructions

that were clear and easy to follow, the survey instrument would have to be re-conceptualized. The Annual

Survey thus changed from a 6 x 11 multi-part pin fed

form, to a 8.5 x 11 multipage booklet. Development

work from this point on focused on testing and refining

the booklet format, the wording of newly developed

items, and the accompanying survey instructions.

Such testing required a different methodology from the

large scale pilot testing conducted thus far. Sampling

a large number of respondents at a time was no longer

necessary. What was needed was to access a few respondents at a time and ask them to look at the survey

booklet and provide BLS with detailed feedback.

First Draft: Focus Group and Respondent

Interviews

In September 1990, a study (Palmisano, 1990) was

designed to: (a) investigate the clarity of newly developed item wording and (b) evaluate the respondent's

perceptions of a test version of the survey booklet --

how the content and format would impact on the ability

to effectively, efficiently and accurately complete the

survey. The study was conducted in two phases: the

first consisted of a group of twelve (12) BLS employees, and the second consisted of five (5) individual

interviews with representatives of business establishments from the private sector. All interviews were

videotaped to assist with later analysis. BLS employees were chosen, partially for ease of access, but primarily for their lack of knowledge of safety and health

related forms and recordkeeping guidelines and for

their extensive experience with various other
5

"government forms". It was felt that this latter group

of individuals would be in a position to comment on

the strengths and weaknesses of a BLS survey having

as a comparison the various forms they worked with on

a daily basis. The five representatives of business

establishments had experience completing the OSHA

recordkeeping forms as well as the Annual Survey, and

could comment on how this "new" booklet faired in

comparison.

Furthermore, the two phases were expected to

yield very different kinds of information: From a focus

group, one expects the feedback/information to grow

out of the group interaction -- one group member adding to, confirming, or disagreeing with that said by

others. With one-to-one interviews, the focus is solely

on the interaction between the respondent and the instrument. From such observation, one can expect

greater emphasis on details and a more carefully considered reaction from the respondent. Being alone, the

respondent has to feel strongly enough about a comment in order to verbalize it and cannot, as in a group,

simply agree with what was stated by others.

Respondents were first asked to complete the survey booklet, and then to discuss the document listing

its best and worst features, and why they thought particular features were important. The survey booklet

consisted of the following: a section asking for summary data of injuries and illnesses; a sampling matrix

designed to enable the employer with many cases (of

injuries and illnesses) to sample a smaller number of

those cases and thus reduce response burden; and copies of a case form designed to collect individual details

from the sampled cases.

Data analysis revealed weaknesses in the clarity of

instructions and certain item wording but, most importantly, revealed problems with the sampling matrix.

Specifically, respondents found the case sampling matrix overly 'mathematical' and difficult to use, and the

accompanying instructions confusing. Additionally,

differences between the BLS employees and

representatives of business in the type of difficulties

encountered with the booklet revealed invaluable information about a "private citizen's" attention span in

reading "government instructions". Instructions would

have to become greatly simplified to ensure that respondents would understand them after only a cursory

glance.

Results of this study led to a thorough review and

revision of the contents and form of the survey booklet.

Second Draft: Respondent Interviews and

Mailout Testing

BLS returned to the field in 1991 to conduct further field tests. The revised booklet was subjected to

two different types of testing: intensive "cognitive"

testing involving interviews with one respondent at a

time, and mass mailing of the booklet (Palmisano,

1991). Each methodology was designed to test a different aspect of the booklet.

Second Draft: Interviews with Respondents

It was decided to interview respondents from a

broad range of geographic locations, establishment

size, and industrial classification. Nineteen (19) interviews were conducted within establishments located in

four States. Half of the establishments sampled were

members of industries with typically high rates of

recordable injuries and illnesses, and the remaining

half were members of industries with low rates of recordable injuries and illnesses. Further, half of the

sample consisted of establishments with more than 50

employees; the other half consisted of less than 50

employees. As may be obvious, establishment size and

typical incidence rate for the establishment industry

speaks to the level of sophistication and experience we

expected a respondent to bring to bear in reviewing

and commenting on the test survey booklet.

Each respondent was interviewed (and videotaped)

within his/her establishment. As with the study described above (see First Draft) respondents were given

a survey booklet containing a section asking for summary data of injuries and illnesses; sampling instructions designed to enable an employer with many cases

(of injuries and illnesses) to sample a smaller number

of those cases; and copies of a case form designed to

collect individual details from the sampled cases. This

time the respondents were also given a mock OSHA

Log listing several "cases" and a number of completed

mock supplementary forms. The latter contained the

detailed characteristics pertaining to the cases listed in

the mock Log. The supplementary forms were workers' compensation forms appropriate to each State in

which the testing took place. These mock documents

were provided to respondents in order to achieve standardization of recordkeeping documents across respondents. The focus of the study was on the interaction

between each respondent and the survey booklet.

Standardizing the documents respondents needed to

use in completing the test survey booklet was quite

important as it prevented the results from being confounded due to any variability of records maintained

across establishments. A great deal of time was invested in generating these mock documents. It was

deemed important that the documents be representative

of what is typically found within establishments, so

care was taken to ensure that a variety of injuries and

illnesses were represented; not all the data were com-
6

plete; the records were not typed; and handwriting was

not always legible, etc.

Respondents were asked to complete the test survey booklet using the Log and supplementary forms

provided. They were not allowed to ask for assistance

while they were in the process of completing the booklet. Respondents were told in advance that they should

note any problems encountered directly on the booklet

and, despite the problems, should continue as best they

could. After they completed the survey, all respondents participated in a debriefing interview focused on

elaboration of the problems experienced while completing the survey.

Second Draft: Large Scale Mailing

Survey booklets were mailed to a total of 1275

establishments located in three States. The test sampled establishments who, based on previous survey

years and knowledge of industrial classification were

expected to have a large number of cases to report .

Completion of the test booklet was voluntary, as indeed

was participation in all the tests discussed in this paper. Employers were informed via an enclosed letter

that they had been selected to take part in the testing of

the OSHS redesign, and were requested to complete the

survey booklet.

Analysis of results (Finch, Enochs, Carder, 1991;

Laundrie, 1991; Winter, Young, & Dorton, 1991)

focused not only on the quality of the completed surveys, but also on an evaluation of the data collection

and processing aspect of this newly developed survey

format. In other words, it was important not only to

develop a survey booklet that was easy for a respondent

to understand and complete, but the booklet had to lend

itself to internal (BLS) review and processing. The

new booklet had to pass two litmus tests if you will:

one by respondents, and the other by the cooperating

States involved in processing the data.

Data from both the large scale mailing and the

interviews with individual respondents were analyzed

and the booklet was reviewed extensively. It was concluded that the wording of the new data elements was

acceptable, as was the wording of most of the instructions. However, the forms design needed to be improved and simplified and, more importantly, so did

the sampling matrix. Respondents were still having

difficulties knowing which cases they were required to

report on the survey booklet -- the instructions for

sampling from the cases listed on their OSHA Log

needed to be simplified. Appropriate revisions to the

survey booklet were implemented and BLS returned to

the field for another round of testing.

This particular set of revisions was pivotal in the

OSHS program redesign process. Data had been accumilating that the survey data collection instrument

needed to be simplified as much as possible. Further,

BLS staff were becoming increasingly concerned that

the size of the booklet (as conceived at this stage of

development) would prove too daunting to many respondents and thus reduce response rate. As a result, it

was decided to produce four different versions of the

survey booklet, each version containing a different

number of copies of the page (within the survey booklet) designed to collect details about the individual

injury and/or illness. Based on prior years of survey

data collection, BLS is able to make a prediction regarding the number of cases with injury and illness

each sampled establishment is expected to have.

Establishments expected to have a large number of

cases with injury and illnesses would be mailed a survey booklet with a larger number of these "case pages",

and establishments expected to have a few cases, would

be mailed a booklet containing a few copies of the case

page. Obviously, producing more than one version of

the survey booklet greatly complicated the survey printing, mailing and quality control procedures as well as

the overall cost of the OSHS survey program.

Nevertheless, this decision was in keeping with programmatic mandate for the OSHS redesign -- to develop a "user friendly" instrument while collecting

more detailed information on injuries and illnesses.

Throughout the development of the various drafts

of the survey booklet, the BLS benefited from the assistance and feedback of the State Advisory Committee on

Occupational Safety and Health (SACOSH.) This

advisory committee was chartered to advise the BLS on

matters related to OSHS redesign. It is composed of

members of the States involved in collecting the OSHS

data. Certain SACOSH members were proponents of

developing a survey booklet that would request minimal data from the respondent and require the State to

collect the rest of the data by accessing the State

Workers Compensation data files. For States having

the facility to access the workers compensation files via

computer, this was seen as a viable option and a further

move towards reducing respondent burden.

Consequently, BLS produced yet another variation of

the survey booklet: The "workers compensation survey

booklet". It looked very similar to the "standard booklet", and also had four versions each containing different numbers of the individual case page. However, the

"Workers compensation" booklet asked the respondent

at a certain point within the individual case page to

proceed no further if a particular case had been submitted to the State workers' compensation office, thus

effectively reducing the time needed to complete the

survey.
7

Third Draft: Think-Aloud Protocol

In early 1992, BLS began the final round of tests

on the survey booklet (American Institutes for

Research, 1992). Revisions as a result of information

learned during second draft testing (see above) had

produced a survey booklet that BLS staff felt comfortable with. The survey booklet instructions were clear

and simple to follow. The forms design was utilitarian,

providing visual clues early on in the document that

assisted the respondent in maneuvering through the

form. The sampling method (for employers expected

to have many cases of injuries and illnesses) was

greatly simplified and could be followed without a

great deal of effort.

Nine (9) establishments were sampled and individuals working in the safety or health departments

interviewed and videotaped. The respondents completed the test booklet using a think-aloud protocol and

their own injury/illness records as source documents.

The respondents were asked to complete the test booklet one page at a time and encouraged to "think aloud"

or verbalize how they interpreted the material they

were reading, and what questions they had along the

way. At the end of each page of the booklet, respondents were asked prepared questions and explored any

problems they had along the way. At the end of the

test session, respondents were asked to complete two

questionnaires: one about the test booklet and the other

about their background and familiarity with Safety and

Health recordkeeping documents.

Analysis of the videotapes revealed very positive

findings. None of the participants had problems with

the overall design of the survey, the case sampling, nor

the majority of instructions. These instructions were

revised as were a few items found to be in need of

minor wording changes, and reordering.

Final Draft: Pretest Mailout and Respondent

Debriefing

The final draft of the survey was mailed to 215

respondents located in six States (American Institutes

for Research, 1992). As with the second draft of the

booklet, respondents from a broad range of establishments in terms of size and industrial classification

were sampled. The sampling was purposeful, designed to obtain respondents with maximum variability

in exposure to safety and health regulations and cases

of work-related injuries and illnesses. Each respondent

was mailed a test survey booklet, a survey questionnaire, and a cover letter describing the purpose of the

test. The survey questionnaire asked respondents questions about their experience in completing the booklet:

the time it took to complete the survey; availability of

in-house records needed as source documents; various

probes related to answers provided; suggestions for

improvement, etc.

After completed survey booklets were received and

reviewed, respondents were called and asked follow-up

questions about the survey. This "respondent debriefing" included prepared questions asked of all respondents as well as specific questions related to the way a

particular respondent completed the booklet. It is interesting to note that all establishments sampled had

been in the sample for the Annual Survey that year,

and therefore, had completed the "official survey" a

few months prior to this mailout test. Despite this

additional burden all respondents were obliging and

giving of their time.

Analyzing the completed survey booklets, the

questionnaires, and evaluating the phone interviews, it

was concluded once again that respondents had little

trouble with the overall design of the survey. There

were very few problems with the survey's wording, and

areas where some employers became confused were

minor enough so as to warrant minimal changes to the

final survey booklet. Phone interviews with respondents proved particularly useful in determining the

sensitivity respondents had to certain items. Though

the sampled respondents completed all items, they did

not particularly like providing certain information.

Respondent debriefings are particularly useful in

eliciting information that cannot not be obtained from

a review of the completed survey booklets. The importance of such information is further underscored when

one considers that in a survey with a sample size of

280,000 establishments (such as the OSHS) such

response sensitivity could reflect thousands of

complaints and perhaps even a decrease in response

rate. In this particular case, the sensitive items were

deemed to be of great importance to the survey and

were retained. However, other steps were taken

(programmatically) to reduce any potential negative

impact on the survey and its contents.

Although specific details of any portion of the

OSHS redesign are beyond the scope of this paper, two

examples of steps taken to reduce potential negative

impact of the redesign should be mentioned: The BLS

engaged in an extensive "outreach" effort that involved

the development of outreach materials (slides, outlines,

talking points) for use by the cooperating State agencies. An outreach brochure was developed to announce

the arrival of the redesigned Annual Survey, and was

sent to establishments that had been in the sample for

more than one year. Furthermore, numerous national

organizations were contacted via mail and asked to

include a notice of information about the Survey

changes in mailings to their constituents. Many organizations included this notice in their newsletters
8

and/or magazines. Others chose to send the notice

under separate cover to their membership.

An alternate method developed to reduce any potential negative impact of the survey, its changes, and

sensitivity to its contents was providing an entire page

in the survey booklet for respondent comments and

feedback. The backpage of the survey booklet, designed to appear friendly and inviting, requests comments or suggestions about the survey -- what respondents liked as well what they didn't. The BLS is sincere when adding: "We will review and carefully consider your comments and ideas as we work to improve

our survey." BLS is currently in the process of collecting, collating and coding all the comments we have

received. Soon, we will be in a position to address

revisions to the "redesigned" Annual Survey of

Occupational Injuries and Illnesses. The process will

begin all over again and we will return to the field to

test, once more, changes we have made to the survey

booklet. In light of lessons learned these past few years

of redesign, the OSHS survey is now considered to be

an ever changing program under a process of continual

improvement.

REFERENCES

American Institutes for Research. (1992, September).

Redesigning and Testing the 1992 Survey of

Occupational Injuries and Illnesses: Final Report.

(Contract No. J9-J1-0067) Washington, DC: Bureau

of Labor Statistics, Office of Compensation and

Working Conditions.

Bureau of Labor Statistics. (1988). BLS Handbook of

Methods. (Bulletin 2285). Washington, DC: U.S.

Government Printing Office.

Bureau of Labor Statistics. (1988). Occupational Safety

and Health Statistics Program: 1988 Feasibility

Studies. Washington, DC: Bureau of Labor Statistics,

Office of Compensation and Working Conditions.

Bureau of Labor Statistics (1989). Report on the 1989

Occupational Injury and Illness Pilot Surveys.

Washington, DC: Bureau of Labor Statistics, Office of

Compensation and Working Conditions.

Dillman, D. (1991) The Design and Administration of

Mail Surveys. Annual Reviews of Sociology, 17:

225-249.

Finch, B., Enochs, D., & Carder, S. (1991, October).

Oklahoma Pilot "C" Study: Final Report. Washington,

DC: Bureau of Labor Statistics, Office of

Compensation and Working Conditions.

Forsyth, B. H., & Lessler, J. T. (1990, November).

Cognitive Laboratory Methods: A Taxonomy. Paper

presented at the International Conference on

Measurement Errors in Surveys, Tucson, AZ.

Gower, A. R., & Nargunkar, M. S. (1991). Cognitive

Aspects of Questionnaire Design: Business Surveys

versus Household Surveys. Paper presented at the

Annual Research Conference, United States Bureau of

the Census, Washington, DC.

Jabine, T. B., Straf, M. L., Tanur, J. M., &

Tourangeau, R. (1984). Cognitive Aspects of Survey

Methodology: Building a Bridge Between Disciplines.

Washington DC: National Academy Press.

Krueger, R., A. (1988). Focus Groups: A Practical

Guide for Applied Research. Newbury Park, CA: Sage

Publications.

Laundrie, S. P. (1991, October). Maine Pilot "C"

Study: Final Report. Washington, DC: Bureau of Labor

Statistics, Office of Compensation and Working

Conditions.

Mullin, P., Miller, L. A., Melis-Wright, M., Stone, D.,

(1991, April). Laboratory Research Methods for

Improving Surveys. Paper presented at the GAO

Technical Conference, Washington, DC.

Palmisano, M. (1991, November). The Evaluation of a

Refined Occupational Injuries and Illnesses

Questionnaire. Washington, DC: Bureau of Labor

Statistics, Office of Compensation and Working

Conditions.

Palmisano, M. (1990, October). Respondent Reactions

to a Proposed BLS Survey Questionnaire for

Occupational Injuries and Illnesses. Washington, DC:

Bureau of Labor Statistics, Office of Compensation and

Working Conditions.

Pollack, E. S., & Keimig, D. G. (Eds.). (1987).

Counting Injuries and Illnesses in the Workplace:

Proposals for a Better System Washington, DC:

National Academy Press.

Winter, D., Young, R., & Dorton, H. (1991, October).

Georgia Pilot "C" Study: Final Report. Washington,

DC: Bureau of Labor Statistics, Office of

Compensation and Working Conditions.