(Table I: Demographics of Subjects
N Range | Minimum | Maximum | Mean | Std. Deviation

lAge 223 44 63 -| 38.89 10.49
lYrs. Exp. In Job] 202 35 35 9.55 8.30

Yrs. Exp. In 212 42 42 13.59 | 9.58

fining
Yrs. At Mine 195 35 35 10.51 8.87
Valid N 173
Results answers; standard deviation was 2.28. Of

The results of the field tests are presented
in three parts. The first part looks at subjects’
improvement in scores from pretest to posttest
following the training; the second part describes
miners’ self-reporting evaluation of the
training; and the third part summarizes
participants’ opinions on the strong and weak
points of the program. Improvement in test
scores is defined as subjects getting one or more
additional correct answers in the posttest than in
the pretest. If the number of correct answers
from pretest to posttest decreased or stayed the
same, then no improvement was recorded.

Test Scores: 71% of the subjects (241 of
340) showed improvement in their posttest
scores following the manipulation (training).
The mean score among all subjects (N=340) in
the pretest was 14.49 correct answers; standard
deviation was 2.57. The mean score among all
subjects in the posttest was 16.01 correct

subjects who just showed improvement in the
posttest (N=241) the mean score was 16.47
correct answers; standard deviation was 2.13.
Tables II, III, IV, and V show these results.

Of the 241 subjects who showed
improvement, 34.9% increased their scores by 1
correct answer; 29% increased their scores by
two; and 19.9% improved by three additional
correct answers in the posttest. Table VI shows
these results.

The job category of miner-laborer had the
highest posttest score improvement following
training with a mean of 2.4 additional correct
answers (s.d. = 0.35). Improved scores for

‘subjects classified as technical (T) averaged 2.0

more correct answers (s.d. = 0.61) and, for
those classified as supervisory (S), subjects
increased their scores by 1.9 additional correct
answers (s.d. = 0.83)

(Table II: Pretest/Posttest Comparison

Pretest Scores | Posttest Scores
N 340, 349
Mean 14.49 16.01
Std. Deviation 2.5 2.28



* All activities involving the assembly,
disassembly, setting up and dismantling of
equipment, machines and related components
therein.

* All those activities including walking/
running/crawling and climbing if the activity
was within the performance of construction,
maintenance or repair work.

Note:

Classification of
construction/maintenance/repair activities are
made independent of employee occupation or
job title.


Table V: Overall Test Results

Pretest Scores

Posttest Scores

N 241 241
Mean. 14.10 16.45
Std. Deviation 2.58 2.13

Table VI: Test Score Improvements

Improved Score +1 +2 +3

+5 | +6 | +7 |4+8 |4+9 | +10

Number of subjects | 84 70 48

ll |4 4 2 0 |i

Percent showing
improvement

34.9 | 29.0 | 19.9

4.6 | 1.7 | 1.7 | 08 | 0.0 | 0.4

Trainees’ evaluation: The analysis of the
trainee rating scale responses show that more
than 93% reported “learning something new
from the exercise” and over 94% said they
“would use some of the ideas presented to work
more safely". (Percentages were determined by
combining ratings of "4" and "3") Table VII
shows these results.

The utility of the CMR exercise was
estimated to be high as more than 94% of the
subjects reported that "the way the material was
Presented is a good way to learn". The final
measure of the exercise validity was judged to
be high as more than 92% of the subjects
indicated that "the visuals (3-D slides) helped
explain concepts".

Strong Points/Weak Points: Nearly one-half of
the participants (153 of 340) commented on the
strengths and weaknesses of the training
program. The leading remarks regarding
strengths addressed the high degree of group
discussion and class participation exhibited
throughout the training sessions. Several
Participants commented: "a new interactive way
to present material...I like the 3-D effect and
interaction with students...allowed us to discuss

and share information." Another common
theme was the scope of realism brought to the
visuals through the use of 3-D scenes. "The
scenes offered a good perspective of general
work areas." Approximately one-forth (84 of
340) of the subjects commented on the weak
points of the training program. The consensus
suggested that there were too many hazards
introduced to adequately address all of them in
the allotted time. "had to rush through
scenes....too little time dedicated to all the
hazards." Also, many of the comments
suggested that additional visuals (overheads or
videos) be used to support the concepts seen in
the 3-D slides; "...if supporting images could be
placed on a screen for all to see, it would be
easier to explain hazards and safeguards."
Others stated that the 3-D scenes were too
restrictive and did not allow one to see the
entire work environment.

CONCLUSION

The CMR training program was designed
to teach hazard recognition skills and to present
safe work procedures for miners who perform
construction, maintenance, and repair work
activities. These were addressed as the


(Table Til: Pretest scores

Correct Answers |Frequency| Percent Cumulative Percent
i 3) 3
5] Z Ae | I
4 1 3 1.2
7 1 3 1.5)
8 y G 2.1
9 7 2.) 4.1
10 10 2.9 71
11 16 4.7 11.8)
1 17, 5.9 16.8)
13 38 11.2 27.9
14 61 17.9 45.9
15 60 17.4 63.5
14 54 15.9 79.4
: 17 41 12.1 91.5
18 19 5.4 97.1
19 8 2.4 99.4
20 y : 100.0
Total 340 100.9
[Fable IV: Posttest scores
Correct Answers Frequency, Percent, Cumulative Percent
7 1 3 3
9 1 3 6
10 3 YJ : 1.5
1] 11 3.2 4.7,
12 2.) 6.3
13 26 746 14.4
14 35] 10.3 24.7
15 38) 11.2 35.9
16 57 16.8 52.4
17 7i 20.9 73.5
18 46 13.5 87.1
19 34 10.0 97.4).
20 10 2.9 100.0
Total 340 _:100.9



interactive format is favored by many trainers
because it provides for active classroom
participation as opposed to the traditional,
passive teaching of facts and reviewing of
injury data and incident narratives. However,
the true impact of the CMR training program
probably has not yet been realized. Follow-up
observations to evaluate the impact of this
program could improve the quality and
effectiveness of future training materials. The
lessons learned in the development of the
training program, as well as the content of the
exercise itself, should help to improve the
health and safety of our nation’s miners.

REFERENCES

1. Seago RL, “The Last Big Frontier in
Safety”. Presented at the National Mining
Association Mine Expo ‘96 Las Vegas, NV,
September, 1996.

2. Lehman C , Layne L, Miner Injuries Related
to Construction, Maintenance, and Repair
Activities. Contract No. 200-94-2837, Battelle;
Centers for Public Health Research and
Evaluation, August, 1999, 19 pp.

3. Rethi LL, Flick JP, Barrett EA, Kowalski
KM, Calhoun RA. Hazard Recognition
Training Program for Construction,
Maintenance, and Repair Activities DHHS,
(NIOSH) Pub No. 99-158, October 1999,

4. Barrett EA, Wiehagen WJ, and Peters, RH,
Application of Stereoscopic (3-D) Slides to
Roof and Rib Hazard Recognition Training.
Bureau of Mines IC 9210, 1989, 15 pp.

5. Barrett EA, Kowalski KM, Effective Hazard
Recognition Training Using a Latent-Image,
Three-Dimensional Slide Simulation Exercise.
Bureau of Mines RI 9527, 1995, 36 pp.

6. Kowalski KM, Fotta B, Barrett EA,
Modifying Behavior to Improve Miners’
Hazard Recognition Skills Thru Training.
Proceedings: 26" Annual Institute on Mining,
Health, Safety, and Research, Blacksburg, VA,
1995, pp. 95-105.

APPENDIX A

GUIDELINES FOR CLASSIFYING
CONSTRUCTION, MAINTENANCE AND
REPAIR ACCIDENTS

Accidents may be classified as construction,
maintenance or repair type accidents if they
meet at least one of the following criteria: The
definition of “construction” work activities:
the building, rebuilding, alteration, or
demolition of any facility or addition to existing
facility at a surface mine, surface area of an
underground mine or underground mine;
including painting, decoration or restoration
associated with such work, and the excavation
of land connected therewith, but excluding shaft
and slope sinking and work performed on the
surface incidental to shaft or slope sinking.
(36CSR23, Board of Coal Mine Health and
Safety, West Virginia)

* The definition of “maintenance/repair” work
activities: the constructing, installing, setting
up, adjusting, inspecting, modifying, and
maintaining and/or servicing machines or
equipment. These activities may include;
lubricating, cleaning or un-jamming of
machines or equipment and making adjustments
or tool changes, where the employee may be
exposed to the unexpected energization or
startup of the equipment or release of hazardous
energy. (29CFR Part 1926, Lock out/tag out
procedures, OSHA)

* All welding and cutting activities, use of non-
powered and powered hand tool and those
activities involving the use of both mobile and
fixed cranes.


DEVELOPMENT AND EVALUATION OF A TRAINING EXERCISE
FOR CONSTRUCTION, MAINTENANCE AND REPAIR WORK ACTIVITIES

Lynn L. Rethi, Safety Engineer
Edward-A. Barrett, Mining Engineer

National Institute for Occupational Safety and Health
Pittsburgh Research Laboratory

ABSTRACT

Recent studies have shown that miners
performing construction, maintenance, and
repair (CMR) work activities in the conduct of
their jobs incur from 39 to 65 percent of all
reported injuries in the mining industry. The
number is particularly high at surface aggregate
operations; however, the problem exists at all
mining locations and commodities. To address
this issue, an interactive, (3-D) slides training
exercise, Hazard Recognition Training Program
for Cor ion, Mai i
Activities, was developed. The purpose of the
exercise is to teach workers to recognize CMR
hazards in the workplace and to deal with them
using accepted safe work procedures. It was
field tested using a total of 340 persons from
surface mining operations in six states. The
subjects were tested before and after the
training intervention to determine if objectives
of the instruction were achieved. Results
indicated that 71 percent of the participants
showed improvement in their test scores.
Following the posttest, subjects responded to a
seven question Likert scale. These questions
related to the validity of the exercise and the
utility of the training program. More than 93
percent of the miners reported that they
"learned something new from the training" and
over 94% said they "would use these practices
to work more safely".

INTRODUCTION

A review of 1995 injury data by a large
aggregate mining company in the United States
showed that a high percentage of incidents
within their operation occurred to miners who
were performing CMR work activities in the
conduct of their jobs [1]. It was thought that
similar findings may also exist throughout other
segments of the mining industry. To investigate
this issue, a group of mine safety practitioners
from The National Institute for Occupational
Safety and Health (NIOSH), the Mine Safety
and Health Administration (MSHA), and the
mining industry, including the large aggregate
mining company, met at NIOSH’s Pittsburgh
Research Laboratory. At the meeting, the
extent to which injuries may be attributed to
construction, maintenance, and repair work
activities, and possible strategies for reducing
these numbers were discussed.

In order to proceed, however, it was
deemed necessary that everyone agree on a
single definition of "construction, maintenance
and repair". After reaching a consensus, the
definition (see Appendix A) was applied to
narratives of 604 incidents that occurred over a
period of three years at the large aggregate
producer’s mining operations. It was concluded
that 65% of these incidents resulted from


instructor and trainees proceed together through
a series of (3-D) visuals showing CMR work
activities at surface noncoal mines. During
training classes, miners can vicariously
experience workplace conditions because the
slides realistically portray various construction,
maintenance and repair activities that typically
occur at surface operations. The instructor
leads the participants in discussions as they
consider the key points depicted in each scene.

How effective is the training?
Experimental results indicate that the training
program helped mine employees recognize
CMR type workplace hazards and also
increased their knowledge of accepted safe
work practices. Approximately seven out of ten
subjects increased their test scores following
training.

Table VII: Self-reporting Measure Results - Subjects' Rating of CMR Exercise

Likert Scale

Strongly Strongly
Agree Disagree
4| 3 2 1| Mean | Std.Dev.

The directions for working this exercise were clear | (N=335) 226 | 95 12 2 3.63 59 |
The slides did not show actual working conditions (N=335) 33 | 60 84 | 158 1.90 1.02
The visuals (3-D slides) helped explain concepts | (N=336) 192] 118] 16 10| 3.46 n
The safe work practices presented will not help me | (N=336) 48 | 26 65 197 1.78 1,09
work safely
I will use some of the ideas presented to work more | (N=338) 222| 96 9 Ik 3.57 -70
safely
I learned something new from this exercise (N=338) 221 92 20 5 3.57 67
The way the material was presented is a good way | (N=335) 215 | 101) 16 3| 3.58 63
for me to lear |

Were the subjects sensitized because of the
pretest? Possibly; but somewhat less than
suspected for two reasons. One, a moderately
long period of time (more than two hours)
lapsed between the pre- and post- tests and,
two, the ensuing class discussions did not
concentrate directly on the test questions;
instead they focused on the visuals.

The self-reporting evaluation gave a clear
indication of the meaningfulness of the CMR
training. More than nine of ten subjects
teported that they "learned something new from
the exercise"; "Would use some of the ideas
presented to work more safely"; and "the (3-D)
slides helped to explain concepts".

The CMR training program presents a
realistic opportunity for miners to become more
cognizant of the many hazards associated with
construction, maintenance, and repair work
activities and to learn about safe work practices
for dealing with them. This experience may
help workers to recognize unsafe situations and
conditions in the safety of the classroom and
prepare for events that are likely to occur in the
real mining world.

To date, approximately 2,100 copies of the
exercise have been distributed to the mining
industry. The program is nonperishable and
may be reused in training classes. The


interaction, it is inherently impossible to
attribute any "pretest to posttest" improvement
in test scores entirely to the training. Even
using a control group to which no training
intervention is applied, scores for the
experimental group are still affected by the
major competing explanation, i.e. variability of
class discussions. Internal validity, therefore,
cannot be achieved regardless of the
experimental design and, it would be unrealistic
to propose that any improvement in scores is
tied entirely to the CMR training exercise.
However, this is not to suggest that higher test
scores may not be partially due to the training.
It is reasoned that if results show a substantially
large number of subjects scoring higher in the
posttest, then the CMR training must have had a
positive effect.

Experimental Design

Because of inability to eliminate the
confound described above, a simple one-group,
repeated measures pretest-posttest experiment
was designed to determine if miners’ test scores
would improve following the CMR training
intervention. A non-probability haphazard
sample consisting of miners representing the
noncoal, surface mining industry in six States
were selected. They were measured both before
and after the CMR training exercise
(independent variable). The dependent variable
is the change in scores between the pre- and
posttests.

Subjects

The subjects were voluntary participants
from mine training classes, safety seminars, or
conference workshops at various locations in
six states. This "sample of convenience"
consisted of 340 persons in 12 nonequivalent
groups, ranging in class size from 18 to 61
individuals. They were located in PA, WV,
VA, NC, AL, and WY.

Their job classifications varied from hourly
employees to supervisors. There were 119 job
classifications represented. For reporting

purposes, the subjects were categorized as
miner-laborer; technical; and supervisory.
Examples of each include welder, laborer, and
truck driver for miner-laborer; project manager,
engineer, and trainer for technical; and foreman,
quarry supervisor, and plant manager for
supervisor. .

Field Tests

Twelve field tests were conducted. Each
was structured so that the same sequence of
events occurred at all locations. The
chronological succession began with the pretest, °
followed by the CMR training exercise, in
which persons viewed slides and discussed
hazards, and ended with the posttest. No direct,
follow up discussions of the pretest questions
were held. The total training time needed for
each field test was approximately two and one-
half hours. The pretest and posttest contained
twenty identical true or false questions. Each
question was grounded in one or more of the
focus areas identified as content material.

After the posttest, subjects completed (1) a
demographic information form which asked for
job title, years experience in the job, age, and
years experience in mining, (2) a seven question
Likert scale which related to the validity of the
exercise and utility of the training program, (3)
a strong point/weak point query which asked
for their opinions on the overall strengths and
weaknesses of the CMR training program.
Table I depicts these results,


overheads include: the definition of CMR work
activities; a classroom format; inter-rater
reliability results; the Pretest/Posttest (with
answer key); and a student handout containing
safe work practices,

Exercise Development

The initial step in developing the exercise
was to identify key concepts (topic areas) to be
incorporated into the training program. This
was accomplished, in part, through discussions
with, and recommendations from, a multi-
disciplinary group of professionals who
provided expertise in choosing the concepts.
Individuals in this group are proficient in one or
more of the following areas: mining, industrial,
and safety engineering; education and training;
enforcement; ergonomics; and mine
labor/management. Seventeen key concepts
were identified. They include:
Communications; Confined Spaces; Electrical;
Elevated Work; Ergonomics; Excavation and
Trenching; Falling Materials; Fire Safety; Hand
Tools; Hazard Communication; Health
Hazards; Lockout/Tagout; Machine Guarding;
Material Handling; Mobile Equipment;
Personal Protective Equipment; and
Welding/Cutting.

The next step was to prepare a broad
account of information about each of the key
concepts. This material also includes best
practices for dealing with CMR hazards and
explicit discussion notes to serve as a resource
for trainers.

Concurrently, visuals depicting
construction, maintenance and repair activities
at noncoal surface mines were obtained using
specialized (3-D) photographic equipment. The
slides, which correspond to the 17 key
concepts, demonstrate various hazardous
conditions and situations relating to all of the
concept areas.

A first draft of the CMR training program
was then prepared and sent to various industry,
academia and MSHA representatives for
authentication. Their comments and
recommendations were considered and some of
them were incorporated into the draft.

The next step in the development process
was to pilot test the exercise. The purpose of
the pilot study was to use the instructional
materials and evaluation procedures in a "trial
run" with a small number of subjects and make
any necessary changes prior to field testing.
Two pilot tests were conducted. The first
included representatives from industry, MSHA,
NIOSH and academia. The pretest was
administered, the exercise presented, and the
posttest and evaluation followed. The second
pilot test was conducted at the MSHA Mine
Academy with representatives from MSHA’s
Field Services Division. Identical presentation
procedures were followed for both pilot tests.
Based on comments from the participants, some
changes were made to the exercise content,
particularly use of appropriate terminology for
mining equipment, work processes, and mine
conditions. The pilot test experiences also
helped to establish consistency in presenting the
exercise for field testing. In particular, they
helped to structure parameters of the training
program with regard to allocation of total time
for each concept and specific time for follow up
discussions. After considering all
recommendations and suggestions, those that
were judged to improve the exercise were
incorporated into a final draft.

EXERCISE EVALUATION

The CMR training program was developed
as a synergistic exercise in which trainees
actively participate throughout the entire
instructional period. As such, the resulting
discussions vary with each training class
because the information being shared is directly
telated to the knowledge, experience, skills, and
interests of those in the class. This exchange is
essential as it contributes to achieving the stated
learning objectives; however, because of this


employees performing CMR work activities. A
follow up inter-rater reliability assessment
showed the level of agreement among the four
raters (two representatives from NIOSH, one
from MSHA and one from industry) to be 94%,

Other evidence of the extent of CMR
injuries among miners was documented in a
NIOSH-funded investigation by Lehman and
Layne [2]. In this study, the consensus
definition was applied to narratives of 21,024
injuries for all commodities (both surface and
underground locations) throughout the U.S.
mining industry during the same three-year
period. It was determined that 39% of these
injuries occurred to employees who were
performing CMR work activities.

After reviewing narratives of incidents and
discussing factors that contributed to these
injuries, the group of mine safety practitioners
agreed that an appropriate strategy for reducing
CMR injuries would be to develop a
meaningful training intervention. The goal of
the training would be to increase employee
awareness of hazards and also advise safe
construction, maintenance and repair practices
for workers to follow. An interactive, 3-D
slides CMR training program called Hazard
Recognition Training Program for
Construction, Maintenance, and Repair
Activities was subsequently developed to
accomplish these objectives. [3] The purpose
of this paper is to document its’ development
and evaluation.

THE “HAZARD RECOGNITION
TRAINING PROGRAM FOR
CONSTRUCTION, MAINTENANCE,
AND REPAIR ACTIVITIES”

Overview

The Hazard Recognition Training Program
for Construction, Maintenance, and Repair
Activities is an 80-page teaching document that
includes a set of three (3-D) slide reels, each
containing seven scenes. The slides depict
various construction, maintenance and repair

work activities at non-coal, surface mining
operations. They provide visual references
from which class discussions emanate as
trainees focus on the hazards of the particular
CMR work activity being depicted.

The concept of degraded images is
incorporated into the (3-D) slides. Both
instructional aids, (3-D) slides and degraded
images, have been shown in earlier studies to be
effective for training miners to recognize and
respond to hazards [4, 5, 6].

Three-dimensional slides were reported by
the former U.S. Bureau of Mines (The safety
and health research functions of the former U.S.
Bureau of Mines were transferred to NIOSH in
1996) to be effective for teaching miners to
Tecognize various geologic and mining-induced
irregularities that may cause ground failures.
As such, they serve as an excellent proxy for
training miners to recognize cues that
distinguish various types of hazards.

The degraded image concept was
originally developed and used for military
target detection training. Degraded images are
scenes where the subjects are partially hidden
from view, observed from an eccentric angle,
viewed through haze or dust, inadequately
illuminated, or otherwise obstructed so as to
camouflage the target. Military research has
shown that pilots who were trained with less
than ideal (or degraded) visuals were more
successful in subsequent identification of
targets than those trained using ideal (or
highlighted) pictures of targets.

In the 80-page CMR training program, 61
Pages constitute a comprehensive Instructor’s
Guide and the other 19 pages contain
information that trainers may use for handouts
and overheads. The Instructor’s Guide includes
the following sections: Introduction;
Performance Objectives; Instructor Guidelines;
Classroom Format; Key Concepts; Materials;
Instructional Method; Instructor’s Notes; Key
Concepts by Scene; and List of Hazards by
Scene. The materials for handouts and