Effective Hazard Recognition Training Using 

a Latent-image, Three-Dimensional Slide 

Simulation Exercise 

By E. A. Barr@tt and K. M. Kowalski 
U.S. Department of the Interior 

Mission Statement 

As the Nation's principal conservation agency, the Department of 

the Interior has responsibility for most of our nationally-owned 

public lands and natural resources. This includes fostering 

sound use of our land and water resources; protecting our fish, 

wildlife, and biological diversity; preserving the environmental 

and cultural values of our national parks and historical places; and 

providing for the enjoyment of life through outdoor recreation. 

The Department assesses our energy and mineral resources and 

works to ensure that their development is in the best interests of 

all our people by encouraging stewardship and citizen participation in their care. The Department also hasa major responsibility 

for American Indian reservation communities and for people who 

live in island territories under U.S. administration. 
Repofl of Investigations 9527 

Effective Hazard Recognition Training Using 

a Latent-Image, Three-Dimensional Slide 

Simulation Exercise 

By E. A. Barrett and K. M. Kowalski 

UNITED STATES DEPARTMENT OF THE INTERIOR 

Bruce Babbitt, Secretary 

BUREAU OF MINES 

Rhea L. Graham, Director 
International Standard Serial Number 

BSN 1066-5552 
CONTENTS 

Abstract ........................................................................... 1 

Introduction ........................................................................ 2 

Background ........................................................................ 2 

Acknowledgments .................................................................... 4 

Latent-image simulation exercises ........................................................ 4 

Latent-image simulation exercises with three-dimensional slides ................................ 4 

D.R.Lighttrainingexercise .......................................................... 4 

Experiment1 ..................................................................... 6 

Introduction .................................................................... 6 

Methodology ................................................................... 6 

Results ....................................................................... 7 

Discussion ..................................................................... 8 

Experiment2 ..................................................................... 8 

Introduction .................................................................... 8 

Methodology ................................................................... 8 

Results ....................................................................... 9 

Discussion ..................................................................... 9 

Conclusions and recommendations ....................................................... 9 

References ...................................................................... 10 

Appendix A.Sample problem booklet .................................................... 11 

Appendix B.4ample answer sheet ....................................................... 25 

Appendix C.-Field test results .......................................................... 30 

ILLUSTRATIONS 

1 . D . R . Light latent.image. 3- D slide simulation exercise ..................................... 5 

2 . Loose roof rock around brow of high fall area ........................................... 5 

3 . Chandelier roof condition in mine entry ................................................ 5 

TABLES 

1 . Experiment 1: Points and hazards at each station .......................................... 7 

2 . Experiment 1: Performance scores on D . R . Light training exercise and in-mine hazard recognition tasks 7 

3 . Experiment 2: Points and hazards at each station ......................................... 8 

4 . Experiment 2: Pretraining and posttraining performance scores on in-mine hazard recognition task and 

D.R.Lighttrainingexercise ....................................................... n 
EFFECTIVE HAZARD RECOGNITION TRAINING USING A LATENT-IMAGE, 

THREE-DIMENSIONAL SLIDE SIMULATION EXERCISE 

By E. A. ~arrett' and K. M. ~owalski* 

ABSTRACT 

The U.S. Bureau of Mines (USBM) conducted experiments to determine if training using a latentimage, three-dimensional (3-D) slide simulation exercise improved miners' abiity to recognize roof and 

rib hazards. The effectiveness of this innovative type of classroom training was investigated by measuring workers' performance on a hazard recognition task. The study was unique in that hazard recognition skills were assessed in the workplace using actual coal mine roof and rib hazards. Results 

showed that training with a latent-image, 3- D slide simulation exercise significantly improved each 

subject's performance on the in-inine hazard recognition task. Further, the USBM researchers concluded that transfer of learning from the classroom to the workplace occurred. 

'~ining engineer. 

2~esearch psychologist. 

Pittsburgh Research Center, U.S. Bureau of Mines, Pittsburgh, PA. 
INTRODUCTION 

In mining, as in other production-related industries, the 

safety of workers is dependent upon many interrelated 

factors. One of the more important safety factors is the 

miner's ability to recognize hazards in the workplace. This 

ability to perceive hazards is, perhaps, more difficult to 

achieve in underground mining than in general industry 

because the work environment is confined, dark, and 

inherently dangerous because of the influences of hidden 

(unknown) conditions. Further complicating the situation 

is a setting that is continuously changing as mining advances. Workers must be alert and always cognizant of 

their immediate surroundings, particularly the conditions 

of the roof and rib. 

It is hypothesized, therefore, that the safety of miners 

depends to a rather large degree upon their individual skill 

in recognizing dangerous conditions. The information 

needed to recognize hazards is often available in the form 

of visual cues found throughout the workplace. Even 

though workers' knowledge of ground control may be extensive because of years of mining experience, they may 

not necessarily be competent in recognizing ground hazards that indicate danger. This seems apparent as officials 

attempt to explain why veteran miners continue to become 

accident victims each year. New miners, of course, must 

rely entirely on training for the acquisition of hazard 

recognition skills. Clearly, all miners, both experienced 

and inexperienced workers, have a need for specialized 

training in reco~~g hazardous ground conditions. 

Studies of mine hazards and their effect on worker 

safety in the production process are common throughout 

the mine health and safety literature. However, there has 

been limited research on the recognition of these hazards. 

In one of the few relevant studies, Blignaut investigated hazard recognition among gold miners in 1979 for 

the Chamber of Mines in South Africa. He investigated 

the effects of skill on visual search and concluded that 

visual search performance depends ~i~cantly upon 

search skills. In subsequent work, Blignaut (2) examined 

the abiity of mine workers to differentiate between safe 

and dangerous rock. In this study, he confirmed that 

visual skills training significantly improved this ability. In 

the latter investigation, Blignaut found that skills training 

consisting of subjecting trainees to exercises in detection 

was more effective than training that provided workers 

only with verbal information about specific hazards. His 

hazard detection exercises included the use of stereoscopic 

[three-dimensional (3-D)] slides for depicting the ground 

hazards found in gold mines. 

Numerous studies in the literature indicate that perceptual judgments are susceptible to training. The most 

extensive base of information that links training with the 

improvement of visual skills appears in military studies on 

target detection (3-5). Here, research indicates that skill 

improvement in target acquisition can indeed be taught, 

improved, and even accelerated through effective training. 

In other visual search studies-for example, Schneider and 

Shiffrin (6) and Shiffrii and Schneider (7)--performance 

of subjects was shown to change both qualitatively and 

quantitatively with extended training. These researchers 

applied a two-process theory of human information processing to detection, search, and attention phenomena. The 

theory includes automatic processing (not demanding of 

attention) and controlled processing (demanding of attention). The eventual automatization of the detection 

process, in general, is affected by perceptual training in the 

detection of hazard cues or situations. The quality, variability, and mode of presentation of these training experiences are important variables in predicting what skills 

accrue from the training. 

This report addresses the issue of effective training for 

teaching miners to recognize roof and rib hazards. An 

innovative form of instruction that combines latent-image 

simulation exercises with 3-D slides was developed for this 

purpose. Experiments were conducted to answer the research question: Can training using a latent-image, 3-D 

slide simulation exercise improve a miner's ability to recognize roof and rib hazards? The work is in support of 

the U.S. Bureau of Mines' (USBM's) mission to enhance 

the safety of mining. 

BACKGROUND 

Training in the "recognition and avoidance of mine haz- miners, as well as for those working at surface areas 

ards" is mandated in the U.S. Code of Federal Regulations of underground mines. Generally, the training focuses 

(CFR), Title 30, Part 48 for both underground and surface on hazards associated with workers' job responsibilities. 

They include, among others, electrical hazards and com- 

'italic numbers in parentheses refer to items in the list of references mon mining hazards associated with ground, escapeways, 

preceding the appendixes at the end of this report. 
moving vehicles, first aid, mine gases, transportation, and 

communication. 

The ground hazards segment of the training typically 

consists of a review of the mine's approved roof control 

plan and, perhaps, instruction on how to sound or pry 

down (scale) loose roof. It may also include instruction 

in the recognition or awareness of dangerous roof and rib 

conditions, such as geologic irregularities, ineffective support, and loose rock occurrences. 

Most ground hazard recognition training is conducted 

in a classroom and consists of looking at visuals of the 

hazards and/or engaging in group discussions about problems observed in the workplace. Both of these approaches 

have certain shortcomings. Even though the discussions 

provide important safety information for the worker, they 

usually occur after a particular incident happens. Response to a hazard that led to an accident in a mine is 

usually swift in comparison with the response to a hazard 

with similar "potential" for causing an accident. The latter 

is viewed as a situation "to observe" for future concern 

and action. This is "crisis management" in dealing with 

groundfall hazards. 

The problem with looking at visuals of hazards in 

the classroom lies in assuming that periodically showing 

hazards to workers will have some effect when the miner 

happens to come across them in the workplace. The concern with transfer of learning from the classroom to the 

mine is rarely addressed in this traditional type of training 

model. Transfer of learning can only occur when a person's learning in one situation influences, positively, his or 

her performance in other situations. 

Methods of mine safety training have not changed ap 

preciably for many years. An earlier USBM study concluded that, in anuual refresher training, mine trainers 

tended to rely heavily on the same instructional materials 

year after year and that innovative teaching techniques 

(games or simulations) were fairly common but were 

usually limited to the factual recall of safety information 

(8) 

The objective of all mandated mine training is not only 

to meet the letter-of-the-law, but also to ensure that mining activities are performed in a safe manner. One concern with most mandated training is that because it is 

repeated on a regular basis, instructors have a difficult 

time maintaining trainees' interest in the subject matter. 

Trainers attempt to vary the content of material being 

presented, but unfortunately have a limited supply of 

appropriate, meaningful subject materials available to 

them. The training materials in this report, therefore, 

were developed in response to this continuing search for 

innovative and effective instructional materials and methods for the mining industry. 

The need for improved training in the recognition of 

roof and rib hazards has been reported by the USBM 

(9-10). This need was identified from two sources, each of 

which disclosed deficiencies in the abilities of some miners 

to recognize dangerous ground conditions. In the first 

source, circumstances leading to roof fall fatalities documented in official U.S. Mine Safety and Health Admiitration (MSHA) accident investigation reports during the 

past 10 years were reviewed. It was noted that the apparent failure of miners to recognize hazardous roof conditions contributed to many of the accidents. In one, for 

example, an "undetected" kettlebottom separated from the 

roof near the face and caused a fatality. In another, an 

"undetected" loose piece of roof, a "horseback formation," 

fell from between the roof bolts and caused fatal injuries 

to a worker. The primary contributing factor in these 

accidents, as reported, was the failure to "detect" a hazardous condition, in these cases, loose rock. The visual 

information available to the miners at the moment of these 

accidents may have been concealed because of rock dust, 

inadequate lighting, or other obstructions in the immediate 

area. In any case, this source of evidence suggests an 

ongoing need for improving miners' skills in recognizing 

hazardous roof conditions. 

The second source of information suggesting the need 

for improved training in hazard recognition was nded in 

related USBM field studies (11). Experiments were conducted primarily to obtain empirical evidence of the effectiveness of using 3-D slides to depict ground hazards. 

The results showed that 3-D slides were significantly more 

effective than the traditional two-dimensional (2-D) slides 

for depicting the hazards. In conducting the experiments, 

both experienced and inexperienced subjects were asked to 

view and describe the roof and rib hazards observed in 

2-D slides and 3-D slides. The expectation was that subjects with more mining experience looking at 3-D slides 

would score highest in the study and those with less mining experience looking at 2-D slides of the same hazards 

would score lowest. These outcomes were indeed confirmed. However, low performance scores of many experienced subjects indicated that their hazard recognition skills 

were below an acceptable critical skills criterion level. 

For critical skills (like those involved in hazard recognition), a recognition rate of at least 90% is deemed a 

reasonable standard for mastery of that skill. A lower performance score is viewed as undesirable because the realworld consequences can be severe. In summary, both of 

these sources, the MSHA investigative reports and the 

USBM's 3-D slide studies, indicated deficiencies in some 

miners' hazard recognition skills. 
ACKNOWLEDGMENTS 

The authors wish to thank our colleagues at the provided assistance in several areas, most notably in 

USBM's Pittsburgh Research Center, Barbara Fotta, re- statistically interpreting the data and presenting the results. 

search methodologist, and Michael Bmich, mining engi- Bmich provided support in conducting the in-mine hazard 

neer, for their contributions to this publication. Fotta recognition tasks. 

LATENT-IMAGE SIMULATION EXERCISES 

Latent-image simulation exercises are performancebased instructional materials that were adapted for teaching problem-solving skills to miners. The exercises, 

originally developed for the mining industry in the 1980's 

by the University of Kentucky under a USBM contract 

(12), are modeled after those previously designed for 

teaching and testing the proficiency of aviation, medical, 

and military personnel. They are based on research about 

how people solve problems and make judgments and 

decisions, particularly in emergency situations. 

"Latent image" is a process in which words, printed on 

paper with invisible ink, are revealed after a special developing pen is drawn over them. This special pen, called 

a "latent-image developer," contains ink-developing fluid 

and is simiiar in appearance to a "magic marker." The 

latent-image process is used on the simulation exercise 

answer sheets where messages are printed to explain 

whether a response is correct or incorrect and the consequences of choosing that response. 

Latent-image simulation exercises offer several advantages over other familiar types of simulation (for example, 

role playing and hands-on task training) currently used 

in mine training. Two principal advantages are (1) an 

unfolding of information and consequences (on the answer sheet) contingent upon the responses that the trainee selects and (2) an immediate, corrective feedback 

about these consequences and the correctness of answers 

selected. 

More than 65 latent-image simulation exercises dealing 

with preventing and controlling mine emergencies have 

been completed to date. The content of the exercises includes such annual refresher topics as first aid, haulage, 

mine ventilation, electricity, and roof bolting. The materials relate to surface and underground mining, coal, and 

noncoal, as well as preparation plants and mills. It was 

determined by the USBM that the content of these latentimage exercises has enormous appeal to trainees. In one 

study, latent-image simulation exercises rated highly among 

miners and trainers in terms of authenticity, relevance, and 

utility (13). 

LATENT-IMAGE SIMULATION EXERCISES 

WITH THREE-DIMENSIONAL SLIDES 

The USBM modified the latent-image exercise's format 

by adding 3-D slides to form a latent-image, 3-D slide 

simulation exercise. This innovative concept combines the 

salient benefits of 3-D slides with the instructional advantages of latent-image simulation to form a truly unique 

training instrument (11). It Qpratively "places" miners in 

a problem-solving situation that is realistically visualized 

for them in 3-D slides. The main advantage of latentimage exercises is the realism of the problem-storyline. 

The 3- D slides add to that realism and offer opportunities 

to teach a wider set of skills. 

It was shown in USBM research that ID slides are 

significantly more effective for depicting hazards than 

traditional 2-D slides (11). This study concluded, following field evaluations, that miners viewing 3-D slides 

achieved a much greater hazard recognition rate than miners viewing 2-D slides. It was also noted that mining experience was not a factor in achieving this recognition rate 

primarily because of the superior fidelity of 3- D slides. 

The realism of 3- D slides permitted all subjects, regardless 

of experience, to actually "see" more hazards in the slides. 

The training effectiveness of a latent-image, 3- D slide 

simulation exercise for skill improvement in remgnkiq 

hazards and how this resultant learning transferred to the 

workplace was largely unknown. To this end, the USBM 

studied the effectiveness of an exercise and training transfer by conducting two small sample hazard recognition 

experiments. The experiments used an exercise developed 

specifically for the investigation called "D. R. Light." 

D. R. UGHT TRAINING EXERCISE 

"D. R. Light" is a latent-image, 3-D slide simulation 

exercise that consists of 10 questions and seven 3-D slides. 

The slides depict hazards associated with two types of 

underground coal mine roof conditions--high fall areas and 

loose, broken top around roof bolts. The components of 

the exercise are a problem booklet (appendix A), answer 

sheet booklet (appendix B), 3-D slide viewer, circular ID 

slide reel, and latent-image pen. All of these are shown in 

figure 1. 

The problem booklet contains an initial page of general 

instructions followed by a page of background information 

and a description of the problem situation. In the background segment, the trainee is informed that he or she is 

an experienced roof bolter operator whose helper, named 

D. R. Light, has little underground mining experience. 

Information is then provided about the coal seam, the 
immediate roof, and the types of roof bolts that were 

installed in the mine. 

The problem situation in which the miners find themselves is described next in the problem booklet. Briefly, 

on a recent run of the escapeways, the section boss notices 

that the brow of a high fall area has begun to deteriorate. 

("High fall" refers to a location where the roof rock has 

fallen out and the remaining top is resupported.) The face 

boss then asks the bolter operator and his or her helper to 

follow the escapeway out and take down any loose top at 

this particular high fall area. Figure 2 shows the roof condition around the periphery of the high fall area question. After correcting this condition, a second roof problem is encountered by the miners. They notice a condition 

where bolts are exposed because the immediate roof rock 

has fallen from around the installed bolts. This top is 

sometimes referred to as "chandelier" roof (figure 3). 

The next 10 pages of the booklet present a progressive 

series of questions related to the problem situation. A 

story is presented using these questions and the trainee is 

asked to select appropriate actions from the alternatives 

listed. Six of the ten questions involve viewing an accompanying (3-D) slide. The questions proceed through the 

following sequence of events (labelled questions A through 

J in the problem booklet in appendix A): 

1. Assessing a potential roof hazard (view slide 1). 

2. Deciding on the initial action to be taken after a 

closer inspection of the hazard (view slide 2). 

3. Deciding what action to take after examining the 

brow of a high fall area (view slide 3). 

4. Identifying potential outcomes associated with this 

type of hazard. 

5. Determining the best way to secure the roof area. 

6. Determining the best way to proceed with correcting 

the roof problem. 

7. Examining another section of mine roof and 

identifying potential hazards (view slide 4). 

8. Assessing the adequacy of the roof support system 

in another place (view slide 5). 

9. Determining the best action to remedy chandelier 

roof. 

10. Identifying effective types of roof control to support 

chandelier roof (view slides 6 and 7). 

For each question, the trainee determines his or her 

response(s) and uses the latent-image pen on the accompanying answer sheet to reveal whether the choice(s) 

is(are) correct or incorrect. In the process of responding 

to the questions, feedback is provided in the form of 

D.RLigktIatmt~30slCdc~de. information contained within the answer brackets. 

henrrnfmckcmRurdbnm,oflulu~fdLrmecr CXmikIw nwf cmditkm in mine enby- 
The authenticity of the D. R. Light training exercise in 

terms of the background information, problem situation, 

accompanying questions, and 3-D slides was validated by 

mining personnel who are experts on the subject of mine 

roof support. Formal field testing of the exercise followed. 

The field tests involved the carefully controlled administration of the exercise to 69 miners of varying degrees of 

job responsibility and experience at 3 different geographic 

locations. In addition to working the exercise, subjects in 

the field test sample were asked to provide certain demographic information and to rate the exercise for validity, 

relevance, quality, and utility. Findings from the subject's 

evaluations were positive with respect to using the D. R. 

Light exercise for miner training. Over 80% of the miners 

responded in the amative when asked (1) whether the 

exercise problem was a realistic one, (2) if working the 

exercise helped them to remember important things, (3) if 

the oral and written instructions were clear, (4) if scoring 

procedures were easy to understand, and (5) whether the 

exercise was easy to read. Over 80% reported that the 

exercise was not too long, that they enjoyed working the 

exercise, and that the graphics were easy to understand. 

And finally, about 74% of the field test subjects reported 

having learned something new from working the exercise. 

A composite summary of all field test data with applicable 

interpretations is presented in appendix C. 

The two experiments described in the next sections 

were conducted using the D. R. Light training exercise for 

teaching and assessing two samples of miners. The Samples were small, six in the first experiment and five in the 

second, primarily because the experimental methodology 

required that workers be pulled out of production at the 

mine site to participate. To minimize this inconvenience, 

the number of subjects was limited. The sample size was 

deemed sufficient by the researchers to obtain knowledge 

of whether the notion of an in-mine hazard recognition 

task could, indeed, be completed and yield meaningful 

results. The statistical analyses subsequently employed on 

the data support this decision. 

EXPERIMENT 1 

Introduction 

The subjects of the first experiment were six experienced coal miners who, by law, required health and safety 

training following a 3-year layoff from mining. They had 

a mean age of 31 years and averaged 4.5 years of underground experience. The miners' underground experience 

occurred several years prior to the experiment because the 

mine was closed and all workers were furloughed at that 

time. The experiment coincided with the resumption of 

normal mine operations and the six workers being recalled 

to begin mine rehabilitation and restoration. During the 

idle period, none of the miners worked underground. 

Their previous job classifications were either shuttle car 

operator or continuous miner operator. As mandated in 

30 CFR 48, the workers were required to receive 40 hours 

of newly employed experienced miner training before starting work. They participated in the USBM's experiment 

during the hazard recognition portion of this required 

comprehensive training. 

Methodology 

By drawing names, the miners were randomly assigned 

to either a wntrol group or an gxperimental group. Each 

group consisted of three miners. The instruction phase of 

the study consisted of having just the gxperimental group 

work through the D. R. Lit training exercise. This was 

done at the mine's training center and took approximately 

30 minutes to complete. At the end of this period, no 

group or individual discussions were held. (Normally, class 

discussion is an important part of the training and generally follows the completion of an exercise.) Each miner 

worked individually through his or her problem booklet 

and responded to the 10 questions. When instructed to do 

so for a given question, the subjects viewed the appropriate 3-D slide. After the directions were explained to 

both groups, no further discussion was held during the 

training and each miner worked at his or her own rate. 

To examine the effectiveness of this training, a hazard 

recognition performance assessment task was set up in the 

coal mine. Twelve hazard assessment areas were identified in a mile-long route traversing two of the mine's 

major entries. The areas were marked by spray painting 

the letters A through L on the ribs of the entries. No 

hazards were artificially prepared at any area; those 

naturally existing were recorded and became the key for 

the recognition task. There were a total of 20 possible 

points on the task. Several of the areas had multiple 

hazards and some had no hazards-that is, the roof conditions were good. A perfect score reflected the identification of no hazards at two of the stations, one hazard 

at three stations, two hazards at six stations, and three 

hazards at one station. The sequence of hazards, the 

points, and the description of the hazards at each location 

are shown in table 1. 

Performance assessment was conducted as all six subjects walked through the mine and attempted to identify 

the hazards. The gxperimental group received the classroom instruction with the D. R. Light training exercise 

prior to the walk-through, whereas the gontrol group did 

not. However, once data were collected from the pntrol 

group on the underground task, the D. R. Light exercise 

was administered as part of their required new miner comprehensive training. The control group training was not 

necessary for the experiment. 
Tabk l.-€xpulmont 1: Pdntr and hazards at each stalion researchers provided no feedback during the entire 

experiment. 

Station points' Description of hazards 

A .... 1 Loose, slabbing top. 

B .... 1 Broken top, chandelier roof condition. 

C .... 2 Numerous slips visible in roof; loose, broken 

top; chandelier roof condition. 

D .... 1 No hazards, good top. 

E .... 2 High fall area with loose, hanging brow; large, 

loose layered separations in roof. 

F .... 2 Loose concretions in roof, bad top, extensive 

chandelier roof condition. 

G .... 1 No hazards, good top. 

H .... 3 Slips apparent in top, loose concretions in 

roof, severe jointing and loose rock in top. 

1 ..... 2 Loos4 day veins on both ribs of entry, vertical 

rib fractures at corners of pillar. 

J .... 1 Large slip in roof extending across entry. 

K .... 2 Low, hanging roof slab; large concretion in 

roof. 

L .... 2 Loose, hanging roof bolts, bad top; chandelier 

top everywhere. 

'~otal possible mre = 20 points. 

In the underground walk-through, each miner was given 

a pencil and clipboard with 12 sheets of paper labeled A 

through L. The miners were instructed to walk as a group 

along the designated route and stop at each station. Here, 

they had 1 minute to identify all ground conditions they 

reccyplized in the roof at each stop. They were asked to 

write their obsewations on the sheets provided using either 

mining terminology or, if needed for clarification, lay 

terminology. In addition to the written description, the 

miners were to indicate whether each condition was 

hazardous or nonhazardous. The written responses were 

done individually, and at no time were group members 

permitted to discuss the hazards with each other. The 

There were a total of U) possible points for the underground hazard recognition task. Table 2 shows the individual subject scores, both the number correct and the 

corresponding percentage correct, as well as the means 

and standard deviations for both the control and e-rimental groups. All subjects in the experimental group 

who had the D. R. Light training prior to the walk-through 

scored higher in the recognition task than the control 

group, who did not receive training prior to the walkthrough. 

Given the small sample size, the Fisher randomization 

t-test was applied to the data. The test confirmed with 

95% confidence the hypothesis that the mean score for the 

experimental group was ~ig~cantly greater than the mean 

score for the control group. 

Although the training given to the control group was 

not part of the experimental methodology (referred to as 

a hold-out experimental design) but necessary for their 

required retraining, it is still interesting to look at the 

results. Also included in table 2 are the performaace 

scores of both groups for the D. R. Light exercise.. Means 

and standard deviations were computed for both the number correct and the corresponding percentage correct. A 

perfect score on the exercise is 46 points. 

To determine if the control group performed significantly different in the D. R. Light exercise because of their 

walk-through experience, a Fisher randomization t-test was 

applied to the scores. The results showed that the mean 

scores of the two groups were not statistically signi!icantly 

different. The walk-through experience did not significantly increase the training scores. 

Tabk 2.-mmt 1: &ores on D. R. Ught trainlng exorci80 

and In-mino hazard recognition tasks 

- 

Group D. R tight' lnmine hazard recognition D. R. ~igh? 

and Number % Number % Number % 

subject correct correct correct correct correct correct 

Control: 

1 ........... - - 11 55 41 09 

2 ........... - - 13 65 40 87 

3 ........... - - 8 40 44 96 

Man ..... - - 10.7 53.3 41.7 90.6 

Std dev ... - - 2.5 12.6 2.1 4.5 

Experimental: 

4 ........... 41 89 18 90 - - 

5 ........... 38 83 16 80 - - 

6 ........... 40 87 15 75 - - Mean ..... 39.7 86.2 16.3 81.7 - - 

Std dev ... 1.5 3.3 1.5 7.6 - - 

Std dev Standard deviation. 

'Dashes indicate that D. R. tight training exercise was not administered to control group prior to in-mine hazard recognition task. 

2~ashes indicate that D. R. Light training exercise was not administered to experimental group following in-mine 

hazard recognition task. 
Discussion of the five had more than 11 years underground. The 

miners were a "sample of convenience" selected by the 

In the underground hazard recognition task, only one of section shift boss. They participated in the study when 

the six miners reached a 90% proficiency level. As dis- asked to break away from their regular job responsibilities 

cussed earlier in this report, a 90% proficiency level is on the shift. 

deemed a reasonable standard for mastery of a critical 

skill. None of the three miners who performed the rec- Methodology 

ognition task before training achieved better than 65% 

proficiency. This suggests that, at least for this group of The repeated measures design used in this experiment 

miners who had not worked underground recently, regard- included the following three steps: (1) pretesting of all 

less of their prior mining experience, there was a defi- miners on a hazard recognition task during a walkciency in their hazard recognition skills. This conclusion through, (2) training using the D. R. Light training expositively correlates with the USBM findings noted ear- ercise, and (3) a posttest on the same hazard recognition 

lier in this report. A comparison of the ~ontrol group of task during a second walk-through. As in the &st exminers with the gxperimental group who were trained periment, the hazard recognition task was set up in the 

using the D. R. Light exercise revealed that the trained mine. Ten hazard assessment areas were identified along 

group did perform signX1cantly better on the in-mine a 1-mile route in a main intake aircourse. A total of 17 

hazard recognition task. It would appear that the im- points were given for the recognition task; a perfect score 

proved scores in the gxperimental group were attributed to reflected the identification of no hazards at two stations, 

the training received. The results of this experiment sug- one hazard at three stations, two hazards at three stations, 

gest that a latent-image, 3-D slide simulation exercise, such and three hazards at two stations. Table 3 shows the 

as D. R. Light, has potential for improving the hazard rec- sequence of hazards, the points, and the description of the 

ognition skills of miners and that transfer of learning from hazards at each location. 

the classroom to the mine can take place. 

Since the group studied in the experiment was quite Table S.-Exwiment 2: Points and hazard. at ..ch stdon 

small, the results must be interpreted with caution. The 

research does, however, illustrate the feasibility of con- Station points' Description of hazards 

ductkg an in-mine hazard recognition task in the under- A . . . . 2 wet, loose shaley top; loo-, immediaw roof 

ground workplace. along rib. 

B . . . . 2 Loose slickensided top; broken roof, some 

EXPERIMENT 2 fallen. 

C.... 1 Separated roof visible in one direction only. 

D . . . . 1 Clay vein, hanging rock in roof. 

Introduction E . . . . 3 Slip across intersection, day vein in rib and 

roof, vertical fracture at pillar comer. 

A second investigation was conducted to further examine the effect of training us@ the D. R. Light latentimage, 3-D slide simulation exercise. In contrast to the 

first experiment, though, these subjects were currently 

employed miners rather than ones being retrained following a layoff. It was of interest to determine whether 

findings would be consistent for this group and whether 

their hazard recognition performance prior to training 

would meet an acceptable criterion level for critical skills 

performance. 

This experiment assessed the hazard recognition performance of five miners whose mean age was 36 years and 

average underground mining experience was 11.4 years. 

Their experience ranged from 4 to 16 years, and four 

F . . . . 2 Chandelier roof, rib fracturing and crumbling. 

G .... 1 Good top, no hazards. 

H . . . . 3 Clay veins in both ribs; loose, slickensided 

broken roof; high roof fall area with slickensided top. 

I..... 1 Loose, slickensided top. 

J.... 1 Good top, no hazards. 

'~otal possible score = 17 points. 

Procedures for the walk-through hazard recognition 

task and the training intervention were the same as those 

used in the first experiment. Immediately after the miners 

received training they repeated the walk-through. The 

route for the hazards and the actual hazards were identical for both the pretest and posttest. The miners were 
instructed to refrain from discussion regarding the task or 

the training until after the posttest. 

Resu As 

To determine whether performance scores on the 

hazard recognition task increased significantly from the 

pretest to the posttest, a Fisher's exact matched-pairs test 

was applied to the two sets of scores. The results indicated, with 95% confidence, that the posttraining scores 

were signif~cantly greater than the pretraining scores. The 

number and percentage of correct responses for the hazard recognition task on both the pretraining and posttraining walk-throughs are shown for each individual subject in table 4. The means and standard deviations for 

each of these scores are also provided. As in the first 

experiment, the training resulted in scores that were 

significantly higher in terms of the number of hazards 

recognized by the miners after receiving training. 

The scores for the miners on their performance in the 

D. R. Light training exercise are also presented in table 4. 

As in the first study, a perfect score is 46. As noted, the 

exercise scores were quite homogeneous and are similar to 

those of experiment 1. 

Discussion 

Although it appears the D. R. Light training exercise 

resulted in higher performance on the in-mine hazard 

recognition task by the sample of experienced miners, their 

performance was unexpectedly low. These miners had 

been working underground, and the fact that the recognition task took place in locations familiar to them 

underscores the seriousness of their low performance level. 

It is interesting that the ranks of the training task scores 

and posttest hazard recognition scores are quite different, 

with miner 4 tieing for highest in training but lowest in 

posttest score. On the other hand, miner 3 ranked lowest 

in training score and highest in posttest score. It is possible that the inconsistency in performance between 

training scores and posttest recognition scores may be due 

to the nature of the training exercise. Latent-image 

simulation exercises provide additional information when 

incorrect answers are selected. Those scoring lower in the 

exercises perhaps gained enough information and became 

sensitized to the hazards for the posttest. The sample is 

quite small, however, and the data from experiment 1 are 

not consistent with this hypothesis. 

Table 4.--Ewperiment 2: Pretraining and posttraining performance scores on 

in-mine hazard recognition tasks and D. R. Ught training exercise 

In-mine hazard recognition, D. R. Light In-mine hazard recognition, 

Subject pretraining Number % posttraining 

Number % correct Correct Number % 

correct correct correct correct 

1 ............... 7 41 43 93 10 59 

2 ............... 3 18 42 91 7 41 

3 ............... 9 53 40 87 12 71 

4 ............... 4 24 43 93 6 35 

5 ............... 7 41 42 91 8 47 

Mean .......... 6.0 35.3 42.0 91.3 8.6 50.6 

Std dev ........ 2.4 14.4 1.4 2.7 2.4 14.2 

Std dev Standard deviation. 

CONCLUSIONS AND RECOMMENDATIONS 

Results from the two experiments showed that training size in both experiments suggests prudence in the genwith a latent-image, 3-D slide simulation exercise signifi- eralization of the results. 

cantly improved each subject's performance on an in-mine In the future, latent-image, 3-D slide simulation exercise 

hazard recognition task. In addition, the researchers con- training could be applicable for instruction on other types 

cluded that there was transfer of learning from the class- of mining hazards. Of particular interest would be elecroom to the workplace based on the subjects' perform- trical hazards, lockout-tagout procedures, machine and 

ances on the in-mine recognition task. The small sample equipment guarding practices, safe machine mounting and 
dismounting, general housekeeping practices, pretrip instruction and improved worker performance. It is sugequipment inspection, and other specific safety topics. De- gested that other methods of training to strengthen miners' 

picting the potential hazards associated with these activi- hazard recognition skills should also be explored. The 

ties in 3-D slides and combii them with latent-image goal of improved training is to enhance the ability of 

classroom simulations could enhance the judgment and individuals to recognize and respond to dangers in the 

decision making skills of miners. This study introduced mining workplace and thus improve the health and safety 

an improved method of training for recognizing ground of the miner. 

hazards that demonstrates a relationship between valid 

REFERENCES 

1. Blignaut, C. J. H. The Perception of Hazard. I. Hazard Analysis 

and the Contribution of Visual Search to Hazard Perception. 

Ergonomics, v. 22, No. 9,1979, pp. 991-999. 

2. - .The Perception of Hazard: 11. The Contribution of Signal Detection to Hazard Perception. Ergonomics, v. 22, No. 11, 1979, 

pp. 1177-1183. 

3. Farnsworth, D., E. Malone, and M. Sexton. Relative Detectability 

of Hues in Air-Sea Rescue. J. Opt. Soc. Am., v. 42, 1952,289 pp. 

4. Jones, D. B., M. Freitag, and S. C. Collyer. Air-TeGround Target 

Acquisition Soum Book A Review of the Literature. Off. Nav. Res., 

NIlS AD-A015 @79, 1974. 

5. Leibowitz, H. W., C. Shupert-Rodemer, and J. Dichgans. The 

Independence of Dynamic Spatial Orientation From Luminance and 

Refractive Error. Percept. and Psychophys., v. 25(7), 1979, pp. 75-79. 

6. Schneider, W., and R M. Shiffrin. Controlled and Automatic 

Human Information Processing: I. Detection, Search and Attention. 

Psychol. Rev., v. 84, No. 1,1977, pp. 1-66. 

7. Shiffrin, R M., and W. Schneider. Controlled and Automatic 

Human Information Processing: 11. Perceptual Learning, Automatic 

Attending, and a General Theory. Psychol. Rev., v. 84, No. 2, 1977, 

pp. 127-190. 

8. Cole, H. P., L. G. Mallet, J. V. Haley, P. K Berger, W. E. 

Lacefield, R D. Wasielewski, G. T. Linebeny, and A. M. Wala. 

Research and Evaluation Methods for Measuring Nonroutine Mine 

Health and Safety Skills (contract H0348040, Univ. KY). Volume I. 

BuMies OER 18(1)89, 1988, 229 pp., PB 89-196646, Volume II. 

Appendices. BuMines OER 18(2)-89, 1988,348 pp., PB S196653. 

9. Peters, R H., and W. J. Wiehagen. Human Factors Contributing 

to Groundfalls in Underground Coal Mines: Workers' Views. BuMincs 

IC 9127,1987,24 pp. 

10. Barrett, E A., and W. J. Wiehagen. Training Miners To Recognize Roof and Rib Hazards Using 3-D Slides. Paper Training Resources Applied to Mining, TRAM, XIV Proceedings. Natl. Mine 

Health and Saf. Acad., Beckley, WV, 1987, pp. 122-130. 

11. Barrett, E A, W. J. Wiehagen, and R H. Peters. Application of 

Stereoscopic (3-D) Slides to Roof and Rib Hazard Recognition Training. 

BuMines IC 9210,1988,15 pp. 

12. Cole, H. P., L. G. Mallett, J. V. Haley, P. K Bcrgcr, W. E 

Lacefield, R D. Wasielewski, G. T. Lineberry, and A. M. Wala. 

Research and Evaluation Methods for Measuring Nonroutine Mine 

Health and Safety Skills: Volume I (contract H0348040, Univ. KY). 

BuMines OFR 18(1)-89,1988,229 pp., NIlS PB 89-196646. 

13. - .Research and Evaluation Methods for Measuring 

Nonroutine Mine Health and Safety Skills: Volume 11: Apendices 

(contract H0348040, Univ. KY). BuMines OER 18(2)-89,1988, pp. 127- 

147; PB 89-196653. 
APPENDIX A.--SAMPLE PROBLEM BOOKLET 

D. R. Light 

Problem Booklet 

Mining Systems and Human Engineering 

U.S. Bureau of Mines 

Pittsburgh, Pennsylvania 

November 1990 
D.R. Light 

Instructions 

Read the problem situation described on the next page. Then answer each of the 10 questions. Do them one at a time. 

Some questions will ask you to look at one or more 3-D slides. Follow the directions for each question. Look at the 

appropriate slide or slides, then continue on with the exercise. Don't jump ahead, but you may look back to earlier 

questions and your answers. Most questions direct you to choose only one answer unless you are told to "Try again!" 

Some questions tell you to select as many answers as you think are correct. Follow the directions for each question. 

After you have selected your choice to a question, look up the number for that choice on the answer sheet. Rub the 

special pen between the brackets for that choice. A hidden message will appear that tells you if the choice is correct and 

provides you with additional information. When you finish you will learn how to score your performance. 
D. R Light 

You are a roof bolter with five years job experience. 

Your helper, D. R. Light, has one year underground experience. 

This coal mine has recently been reopened after being idle for several years. 

The coal seam is slightly more than six feet thick. 

The immediate roof of the mine is approximately 3 114 feet of shale. Above this is 4 112 feet of limestone. 

The following types of bolts are used for roof support: mechanical, resin, or combination (mechanical/resin). 

Problem 

The section boss is required to walk the escapeways leading out to East Mains each week looking for hazardous 

conditions. If any are found, he must see that corrections are made. On his last run two days ago, he noticed that the 

roof and brow in the high fall area at the mouth of 2 North has started to deteriorate. He felt the situation could become 

hazardous, however, he didn't feel it was urgent at that time. Today the continuous miner is down, so he asks you and 

D.R. to follow the escapeway out to the high fall area. You are to make corrections and, if necessary, take down any 

loose top. Turn the page and answer the first question. 
D.R. Light 

Question A 

After receiving instructions from the face boss, you and D.R. load a supply jeep with scaling bars, posts, cap blocks and 

wedges. You head outby to the mouth of 2 North. You arrive and park the vehicle near the edge of the high fall area. 

You start to inspect the roof beginning at the center of the dome, which you estimate to be thirty feet high. LOOK AT 

SLIDE 1. 

How does the roof look to you? (Choose only ONE unless directed to "Try again!") 

1. The roof is OK because there are more than enough bolts in place throughout the cavity. 

2. Assume that the roof is safe, since it is too high to conduct a sound and vibration test. 

3. Even though the roof at the center of the dome is very high, you suspect a hazardous condition near the center. 

4. The roof is safe. All bolts and plates appear to be in good contact with the immediate roof. 

5. The roof is not safe because the absence of rockdust on the surface indicates that sloughing has recently occurred. 

6. The roof looks good. Don't be concerned about it. 
D.R Light 

Question B 

You and D.R. have identified this area as potentially hazardous. LOOK AT SLIDE 2. This is a close-up view of the 

hazardous condition. What should you do? (Choose only ONE unless directed to Try again!") 

7. By standing on the jeep, scale the top with a bar to remove the loose slab . 

8. Report the situation to your section boss. You can't reach the top with the equipment you have with you. 

9. Do nothing. You've seen roof like this before and it never caused a problem. You should not be concerned. 

lo. Add additional support to prevent the top from sloughing. 
D.R. Light 

Question C 

You and D.R. decide to take a closer look around the high fall area before you report to your section boss. The lower 

edge of the cavity, directly above where D.R. is standing, catches your attention. This is what you see. LOOK AT SLIDE 

3. 

What should you do now? (Choose only ONE unless directed to "Try again!") 

11. Immediately yell to D.R. to move away from the edge of the high fall area into the main entry. 

12. Ask D.R. to take a close look at the edge of the cavity and see what he thinks about it. 

13. Move to where D.R. is standing and help him inspect the top. 

14. Continue with your inspection of the roof and return to this area later. 
D.R. Light 

Question D 

You recognized the roof hazard and yelled to D.R. to move away from the brow. He is now standing where he is safe. 

However, before you and D.R. can begin to make corrections you should recognize the extent of the problem. What are 

the hazards here? (Select as MANY as you think are correct.) 

15. The slab could break off and fall at any time. 

16. There are too few bolts holding the slab up. 

17. There is a small gap between the slab and the main roof. 

18. The wrong types of roof bolts were used. 

19. The slab is large and is probably very heavy. 

20. Other parts of the roof near the loose slab may also be separated from the main roof. 

WHEN YOU HAVE MADE YOUR SELECTION(S) DO THE NEXT QUESTION. 
D.R. Light 

Question E 

You and D.R. decide the area near the brow should be immediately dangered off and the problem corrected. However, 

this entry is the primary escapeway out of your section. What should you do now? (Choose only ONE unless directed 

to "Try again!") 

21. Begin to correct the problem. The escapeway probably won't need to be used until you are finished. 

22. Begin working. If anyone comes out the escapeway, detour them around the high fall area until you're finished. 

23. Danger the area off and begin to correct the problem. 

24. Send D.R. to the face to tell the boss that you have to danger off and correct a roof problem. Since this is the 

primary escapeway, the boss should tell the miners to use the secondary escapeway. 
D.R. Light 

Question F 

D.R. returns from the face after telling the section boss that the primary escapeway will be dangered off until the roof 

problem is corrected. While at the face, D.R. also reported the roof condition at the center of the cavity. He returns 

with a longer bar and pipe to scale down the loose slab. Now it's time to start working. You danger off the entry. What 

should you do next? (Choose only ONE unless directed to Try again!") 

25. Start to scale the loose top in the dome using the longer bar and pipe. 

26. Set temporary supports at random around the high fall area and then scale down the loose top. 

27. Set temporary supports under the good roof adjacent to the brow and begin to scale down the top. 

28. Begin to pound the bad roof area near the brow using a wood post. 
D.R Light 

Question G 

You and D.R. have safely and successfully pulled down the roof slabs. You leave the temporary posts in place until the 

fresh roof can be bolted. This should be done immediately, so you proceed to the face to get a jackleg drii and some 

bolts. The face boss sees you and gives you another job. When you finish rebolting, he wants you to take a look at the 

roof in East Mains just outby 2 North. 

Roof conditions within the cavity, as well as the outside edge (or brow) of high fall areas can be a continuing source of 

problems. After high falls have been resupported, workers tend to pay little attention to the roof because it's assumed 

to be safe. These areas need to be examined on a regular basis. LOOK AT SLIDE 4. 

What roof hazards do you see? (Select as MANY as you think are correct.) 

29. Loose, broken roof rock. 

30. Slickensided roof. 

31. Loose hanging roof bolts. 

32. Sloughing of rock between bolts. 

33. Rusted bolts. 

34. Hanging slabs of rock at the brow of the high fall area. 

WHEN YOU HAVE MADE YOUR SELECTION(S) DO THE NEXT QUESTION. 
D.R. Light 

Question H 

You complete bolting at the high fall area and remove the temporary posts. You and D.R. walk to East Mains and look 

at the roof. LOOK AT SLIDE 5. 

What can you tell about this roof? (Choose only ONE unless directed to Try again!") 

35. There are more than enough bolts in the roof, so support is OK 

36. Most of the bolts and bearing plates are in contact with the roof, so the roof is adequately supported. 

37. This is an older area of the mine. Since the roof is still intact, it will probably remain that way for a long time. 

38. There is significant rock sloughing between bolts, so the bolts are probably not supporting the roof. 
D.R. Light 

Question I 

You saw in SLIDE 5 what is sometimes called "chandelier roof". The support of this mine roof appears uncertain. It 

is dwcult to predict how long the root will remain in place. Spalling of rock between bolts may cause the roof support 

to be ineffective. What can be done with this roof? (Choose only ONE unless directed to Try again!") 

39. Nothing. It will probably stay up as long as the entry remains open. 

40. Additional roof support should be added to the left side of the entry. 

41. Leave it alone if it appears solid after the roof is sounded. 

42. Additional roof support should be added across the width of the entry. 
D.R. Light 

Question J 

Chandelier roof is a condition that generally develops in a mine roof of shale or claystone. The rock around roof bolts 

deteriorates and sloughing occurs. This deterioration of rock, commonly called "slaking", is usually caused by moisture 

in the mine air. Some mines install "tempering chambers" to remove moisture from incoming air. Others, particularly 

if the mine has a serious problem, spray sealants to protect the roof from moisture. In some mines, however, chandelier 

roof is a problem that can be corrected by spot bolting. LOOK AT SLIDES 6 and 7. 

These are examples of more advanced rock spalling between bolts. In addition to rebolting with similar bolts, what are 

some other effective types of additional support that can be used for chandelier roof? (Select as MANY as you think are 

correct .) 

43. Wood headers under steel bearing plates. 

44. Steel I beams or wood beams. 

45. Lengths of structured steel channel. 

46. Woven steellwire mesh. 

END OF PROBLEM 
D.R. Light 

Scoring your performance 

1. Count the total number of responses you colored in that were marked "Correct". Write this number in the first 

blank on the answer sheet. 

2. Count the total number of incorrect responses you colored in. Subtract this number from 27. Write the 

difference in the second blank on the answer sheet. 

3. Add the numbers on the first and second blanks. This is your score. 

The best possible score of 46 results from selecting all the correct answers and no wrong answers. The worst 

possible score of zero results from selecting all the wrong answers and no correct answers. 
APPENDIX B.--SAMPLE ANSWER SHEET 

D.R Light 

Answer Sheet for D. R Light Exercise 

Use this answer sheet to mark your selections. Rub the special pen gently and smoothly between the brackets. Don't 

scrub the pen or the message may blur. Be sure to color in the entire message once you have made a selection. 

Otherwise you may not get the information you need. The last part of the message will tell you what to do next. 

Question A (Choose only ONE unless directed to Try again!") 

Question B (Choose only ONE unless directed to "Try again!") 

Question C (Choose only ONE unless directed to "Try again!") 
Question D (Select as MANY as you think are correct.) 

Question E (Choose only ONE unless directed to "Try again!? 

Question F (Choose only ONE unless directed to "Try again!*) 

D.R Light 
D.R. Light 

Question G (Select as MANY as you think are correct.) 

Question H (Choose only ONE unless directed to "Try again") 

Question I (Choose only ONE unless directed to "Try again") 
D.R Light 

Question J (Select as MANY as you think are correct) 

Finding your score 

Number of "Correct" answers you colored in 

27 minus number of incorrect answers you colored in 

Add blanks one and two to get your total score 

Highest possible score = 46 

Lowest possible score = 0 
D.R Light 

TRAINEE'S QUESTIONNAIRE 

1) Name of exercise DRLight 

2) Your age 3) Your sex M F 4) Years coal miner 

5) Your job title Surface Underground Check the 

. . areas in which you have g~ecial trmng. certification, and/or that you routinelv ~erform. 

special 

Training 

6) Mine foreman 

7) Miie safety committee 

8) Mine rescue 

9) Mining engineer 

10) Roof/rib control 

11) Geology 

12) Miie planning & design 

13) Roof bolter operator/helper 

Routinely 

Certification Perform 

14) Other 

(dm'be) 

Think about the exercise you just hished. Circle the number which tells how much you agree or disagree with the 

following statements. 

Definitelv Yes Definitelv NQ 

15) This problem could happen in real life. 

16) This exercise will help me remember something important if I am ever in a similar situation. 

17) I learned something new from the exercise. 

18) The exercise took too long to complete. 

19) I liked working the exercise. 

20) The instructor's directions were clear. 

21) The written directions in the exercise were easy 

to understand. 

22) The 3-D slides clearly showed mine roof conditions. 

23) The scoring procedures were easy to understand. 

24) The exercise was easy to read. 

If you have anything more to say about the exercise, please write on the back of this page. Thank you. 
APPENDIX C.--FIELD TEST RESULTS 

D.R Light 

Summary of Field Test Results 

The D.R. Light Exercise (DRL) underwent two rounds of field testing, including authentication and one formal field test. 

The authentication consisted of a small initial field test that produced too few data for meaningful statistical summaries. 

However, comments, criticism, and suggestions were gathered and used to revise and improve the exercise before the 

first formal field test. The first formal field test involved carefully controlled administration of the revised exercise to 

69 miners in six classes at three different sites. 

The attached tables are divided into two parts. The first part describes the people who participated in the field test and 

their evaluation of the exercise. The second part describes the measurement properties of the exercise. 

chara&istics and miners' evaluations 

Tables 1 through 3 describe demographic characteristic. of the sample. Table 4 describes the number of persons in the 

sample who reported either some training, spe'cial certification, and/or routine performance of the eight areas listed as 

rows in the table. Inspection of these tables quickly reveals the numbers and types of persons involved in the field test 

of the exercise. 

Table 5 reports the miner's judgment of the exercise quality and worth. Each person was asked to rate the value of the 

exercise on each of ten qualities. By chesking a number on a scale, miners indicated to what extent the exercise displayed 

each quality. Inspection of the values in Table 5 reveals the percentage of miners who reported that the exercise 1) was 

realistic, 2) helped them to remember important things, 3) helped them learn something new, 4) was too 10% 5) was 

enjoyable, 6) instructor's directions were clear, 7) written directions were clear, 8) illustrations were clear, 9) scoring 

procedures were easy to understand, aria 10) was easy to read. 

Psvcho . . metric ~roxrtia 

The next set of tables describe the psychometric (mental performance measurement) properties of the exercise. Table 

6 presents summary data for each of the exercise questions, including the minimum, maximum, median, and mean 

question scores observed in the sample; the standard deviation and skewness of each question score; the squared 

correlation coefficient of each individual question score with the exercise total score; and the estimated reliability of the 

total exercise score if that particular question were dropped fror. the test. The last line in Table 6 reports the same 

information for the total exercise score. This information ;s obtained by summing question scores aaoss the whole 

exercise. Each question is weighted such that a perfect score on each question sums to 100 for the exercise total score. 

Inspection of the table quickly presents the basic performance data for the sample. 

The weighting of each question score to produce a total score of 100 is designed to allow comparison of the difficulty of 

different versions of the same exercise, or different exercises, even though these often have different numbers of 

questions. The total score of 100 is also a common convention for reporting test scores. 

Table 7 reports the number of answers (options) to questions that discriminated properly between persons with high 

scores on the test and those persons with low scores. Tests that have multiple choice answers to questions must include 

some incorrect and some correct answers. The person who takes the test must choose among correct and incorrect 

answers (distractors) to each question. Persons who get a high total score on the test know more about the test content 

than persons who get low scores. Therefore, when data are aggregated from the whole sample of persons who took the 

test, wrong answers (distractors) to each question on the test should be significantly negatively correlated with the overall 

test score, and correct answers to questions should be signif~cantly positively correlated with the exercise total score. 

When correct answers and wrong answers (distractors) behave this way, the answer (whether correct or incorrect) is said 

to positively discriminate between persons who know much about the content and those who know little. However, when 

persons who get high scores on the total test tend to pick wrong answers (distractors) more frequently than persons who 
D.R Light 

said to reverse or negatively discriminate. The opposite case is also true. Th* is, when persons who get low scores on 

the total test score tend to pick a right answer to a question more frequently % persons who had high total test scores, 

get low scores, the answer isthat answer is also said to reverse or negatively discriminate. The discrimination values are 

used to identify and correct problems in the answers for questions. To be valid, the method requires an adequate sample 

of persons who exhibit a wide range of variability in their total test scores. 

Table 7 reports the frequency with which answers to questions behaved in this desirable way. The table tells the 

percentage of answers to questions on the test that discriminated positively, negatkly, or not at all among the ability 

levels represented in the field test sample. 

Another way to look at the discrimination capability of a test is to divide the sample iats groups of persons with greater 

or lesser levels of expertise. Table 8 reports the degree to which the exercise total score discriminated between persons 

in the sample with different levels of self-reported expertise. The means and standard deviations for these two groups 

are given as well as the F ratio and the p value. 

Finally, the last table is another discrimination analysis for the total exercise score, this time between the self-reported 

major job categories represented in the sample. The category "miner/laborer" includes all working miners who are not 

supervisors, surveyors, engineers, etc. The "maintenance/technical" category includes all inspectors, engineers, surveyors, 

electricians, mechanics, and other technical and maintenance personnel who ylork in the mines and typically have special 

skills, and who move from section to section as they complete their work. The "supervisory/managementW category 

includes all managers from the mine section foreman on up. The "other" categq inckles all other persons such as truck 

drivers, security personnel, ofiice staff, and others who are not miners or laborers but who sometimes participate in 

annual refresher training classes. 

The number of miners reported in the tables varies. This is because not all miners answered all of the questions on the 

trainee's questionnaire. Therefore, Tables 1 through 5 report the actual or valid percent of persons who responded to 

specific items on the trainee's questionnaire. The remainder of the psychometric tables report data for the number of 

persons who properly completed the exercise and fully completed the trainee questionnaire. 

Characteristics of the DRL Exercise 

Approximately 60 percent of the field test sample for the DRL Exercise consisted of maintenance/technical personnel, 

and another 35 percent of miners/laborers. Only 5 percent of the sample were supervisory personnel. 

Approximately 25 percent of the sample reported training in geology or roof and rib control which can be expected to 

contribute to high scores on the DRL exercise. The overall mean score for all groups was 88.9 with a standard deviation 

of 759. The exercise was generally easy for this sample of experienced miners and mine maintenance and technical 

personnel, many of whom have special training. 

Examination of the results displayed in Table 7 reveals that the answers to the exercise questions discriminated properly 

among persons with high total scores and other persons with low total scores. Exactly 50 percent of the answers 

discriminated positively, none discriminated negatively, and 50 percent failed to discriminate. Although the sample is 

somewhat homogeneous with respect to age and experience, and biased toward technical personnel, this is a good pattern 

of item discrimination, and one indication that the exercise is valid. 

Inspection of Table 8 reveals that the exercise failed to discriminate between self reported levels of mine technical 

training. Persons who reported they had completed mine engineering and related advanced technical training did not 

out score persons who did not report these advanced levels of training. Inspection of Table 9 reveals that the exercise 

also discriminated among the three major job categories represented in the sample. The present sample does not provide 

for a valid test of the exercise discrimination capability by training level or job category. The exercise might be expected 

to discriminate among levels of technical training and job categories if the sample were larger and more homogeneous 

with respect to age, experience, and level of technical training. 
D.R. Light 

For critical skills like those involved in this exercise, a performance criterion of mastery of at least 90 percent or more 

of the exercise content is a reasonable standard for technically trained personnel. A lower performance standard for such 

critical skills is seen as undesirable because the consequences of errors and poor performance can be severe. Table 10 

reports the degree to which miners in the field test sample exhibited mastery of the mine emergency judgment and 

deasion making skills assessed by the DRL exercise. The metric used in this analysis is the exercise total score expressed 

in percent correct performance. The data in Table 10 reveal that 53.7 percent of this sample attained performance scores 

at or above the 90 percent mastery level criterion, and 91.0 percent scored at or above the 80 percent level. 
D.R. Light 

Characteristics of the field test sample for DRL2 Exercise 

(3 sites, 6 classes, n = 69) 

Table 1: Age and experience of miners (years) 

- n 

a00 69 

experience 69 

- mode - mean - s.d. 

42 40.3 8.30 

15 16.3 6.64 

Table 3: Job classification 

freauency % 

Table 4: Self-reported level of expertise (frequency %) 

CateEloq 

foreman 

mine safety committee 

mine rescue 

mining engineer 

roof/rib control 

Q~O~O$Y 

mine planning & design 

roof bolter operator/helper 

other 

certification 

58.8 

10.3 

13.2 

5.9 

13.2 

2.9 

8.8 

16.2 

14.7 

Table 2: Gender distribution (%) 

female 

male 

- n freauencv % 

1 1.5 

67 98.5 

performance 

7.4 
Table 5: Miners' rating of exercise validity, relevance, quality, and utility (frequency %) 

content 

problem could happen 

help remember important things 

learned something new 

exercise too long 

liked working the exercise 

instructor's directions clear 

written exercise directions dear 

graphics easy to understand 

scoring easy to understand 

exercise easy to read 

detinitely yes 

4 

77.6 

46.3 

23.1 

6.1 

61.5 

79.1 

65.7 

49.3 

60.6 

73.1 

definitely no 

1 

0.0 

1.5 

13.8 

62.1 

4.6 

0.0 

0.0 

3.0 

0.0 

0.0 

mean 
D.R. Light 

Psychometric properties of DRL Exercise 

(printout date 10/5/90, 3 sites, 6 classes, n = 69) 

Table 6: Question score and total score statistics 

guestion 

A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

Total 

minimum 

- score* 

1.7 

5.0 

7.5 

1.7 

2.5 

5.0 

5.0 

2.5 

2.5 

2.5 

55.8 

maximum 

- score* 

10.0 

10.0 

10.0 

10.0 

10.0 

10.0 

10.0 

10.0 

10.0 

10.0 

100.0 

median 

10.0 

10.0 

10.0 

8.3 

10.0 

10.0 

8.3 

10.0 

10.0 

7.5 

90.0 

- mean 

8.8 

9.2 

9.8 

7.6 

9.4 

9.5 

8.3 

9.4 

9.7 

7.2 

88.9 

skewness 

-2.21 

-1.13 

-3.84 

-0.66 

-3.01 

-2.28 

0.44 

-2.91 

-4.50 

4.52 

-1.61 

alpha if 

question 

& deleted 

0.13 0.58 

0.31 0.59 

0.23 0.60 

0.23 0.59 

0.32 0.56 

0.29 0.57 

0.28 0.56 

0.45 0.55 

0.30 0.55 

0.19 0.59 

(alpha = 0.60) 

%e values reported are the minimum and maximum scores obseved, for this sample. The lowest possible score = 0.0 and 

the highest possible score = 10.0. When scores are summed across questions the highest possible score = 100 and the 

lowest possible score = 0. 

Table 7: Proportion of answers discrimination positively, negatively, and not at all with exercise total score @<.lo) 

positive 

23/46 (50.0%) 

neaative 

0146 (0.0%) 

Table 8: Discrimination between basic and advanced levels of mine technical 

training 

basic 

advanced 51 

- mean s.d. 

86.6 . 10.18 

89.4 6.53 

Fratio p value 

1.56 0.215 

no relationship 

23/46 (50.0%) 

Questions that discriminate significantly at p5.10: none 
D.R Light 

Table 9: Discrimination between job categories 

- n 

miner/laborer 23 

maintenance 39 

supervisor/manager 3 

F rati~ p value 

1.99 0.144 

Questions that discriminate signilicautly at ps.10: D, p=06 

Table 10: Number and percent of miners attaining various levels of 

proficiency (n = 67) 

gxercise score 

100 

90-99 

80-89 

70-79 

60-69 

50 

below 50 

number at or percent at or 

above the score