VIDEO MOTION DETECTION FOR REAL-TIME HAZARD WARNINGS IN SURFACE MINES 

Edward L. McHugh
 

Spokane Research Laboratory, National Institute for Occupational Safety and Health, Spokane, Washington, USA 

ABSTRACT 

Digital camera and computer technologies can be 

used to monitor mine slopes and provide real-time 

warning of rock falls. NIOSH researchers assembled a 

surveillance system using low-cost video cameras and 

computer software from the security industry to test its 

effectiveness. The system is designed to signal an alarm 

when motion is detected and to record images of the 

scene. Masking can restrict motion detection to specific 

areas within the camera view; sensitivity is adjustable. 

The time-stamped images provide a record that can help 

reconstruct and quantify an event. Video motion 

detection can augment standard monitoring methods to 

increase safety in surface mines. 

INTRODUCTION 

Since 1995, 34 miners have died in slope failure 

accidents at surface mines in the United States. While 

less than 1% of reported accidents are associated with 

slope stability problems, slope failure accidents were 

responsible for about 15% of all fatalities in U.S. surface mines in recent years. Shovel operators and drillers 

suffered the greatest number of fatalities. Falls of handsized rocks weighing only a few pounds can cause fatal 

injuries to workers away from the protection of large 

machinery. Large rock falls containing a million cubic 

yards or more of material can be fatal even for 

operators inside heavy equipment such as haul trucks, 

bulldozers, and shovels. 

As part of an ongoing study at the Spokane 

Research Laboratory of the National Institute for Occupational Safety and Health, several remote-sensing 

technologies are being evaluated as tools to monitor 

slopes for hazards and to assess slope stability (McHugh and Girard, 2002). Photographic image analysis 

techniques are being developed as tools to recognize 

potential hazards at mine sites. One approach is to use 

video cameras and computer software designed for surveillance monitoring. Video cameras with the standard 

sampling rate of 30 frames per second can provide realtime motion detection. Video sensors have a much 

greater range than the more-common radar, infrared, or 

ultrasonic motion detectors. Real-time monitoring of 

mine slopes to avoid injuries from falling rocks requires 

fast response times to provide workers at risk with 

enough time to get out of the way or find cover. Video 

surveillance developed in the security industry is 

designed to signal an alarm and record images of the 

scene when motion is detected. This technology was 

adapted for mine slope monitoring. 

Development of imaging techniques to assess slope 

stability has been incremental over several decades. 

McVey and others (1974) used a 35-mm film camera 

and carefully positioned reflectors to measure deformation over time in an underground mine. Processed film 

was used to measure deflection to a resolution of 0.5 

mm, but the use of reflectors adds substantial complexity to the installation process and limits analysis to sites 

with reflectors. Dombe and others (1982) designed a 

concept for detecting mine slope displacements using a 

pair of video cameras and computer processing to 

calculate and monitor slope topology. Allersma (1996) 

used a monochrome video camera and frame grabber to 

collect images of an induced dike failure; he was able to 

measure displacements as small as 10 mm. Collins and 

others (2000) described development of automated 

video technologies for real-time analysis of video 

sensor data. Their work addressed problems with standard video monitoring where an operator sits and 

watches video images. This method is not only costly, 

but may be ineffective. At the same time, simple 

recording of video tape by ubiquitous video cameras 

provides information only after the fact. 

Corthésy and others (2001) described a differencing 

technique using before-and-after digital images to detect rock displacement in an underground mine. The 

results were similar to those described here except that 

the use of artificial illumination underground may 

simplify noise filtering. 

ROCK FALL MONITORING WITH REAL-TIME
 

VIDEO 

A basic surveillance system built around hardware 

and software from Strategic Vista Corp.,1

1 Mention of specific manufacturers or products does not 

imply endorsement by the National Institute for Occupational 

Safety and Health. 

 Markham, 

ON, Canada, and GeoVision, Taipei, Taiwan, was 

tested in the laboratory and in local field trials. The 

system includes an 8.5-mm (1/3-in) CCD color video 

camera with 480 lines of resolution, automatic iris, and 

a 6- to 60-mm (10X) power zoom lens. The camera is 

connected by means of an external frame grabber to the 

USB port on a notebook computer. The computer, with 

a Windows 98 operating system, runs programs 

(GV100 from GeoVision) that display real-time and re- 
corded video images. All of these off-the-shelf components are powered from a 12-V, 100-amp-hour, deepcycle battery by way of a 140-W dc-ac inverter. The 

computer, camera controls, and power connections 

were installed in a fiberglass environmental enclosure. 

The enclosure and a mount for the camera were attached to a 2-m-long, 38-mm diameter steel mast screwed 

into a three-legged base (figure 1).

Figure 1.—Video motion-detection system showing 

camera, electronics enclosure, and 12-V battery on a 

tripod. 

 The camera was 

mounted near the top of the mast in its own weather 

enclosure. 

The system can be set to record video frames and 

sound an audio alarm when motion is detected. Detection is accomplished by algorithms that monitor the 

intensity value of each pixel in the scene. When sufficient change is detected, video frames containing the 

motion are recorded to the computer's hard drive, and 

the alarm is activated. Sensitivity of the trigger is 

adjustable. Video images can be viewed and captured at 

several resolutions; for these tests, resolutions of 640 by 

480 or 320 by 240 pixels were used. A mask function 

allows the user to select an area within the image frame 

where motion in the field of view will not trigger the 

record and alarm functions; this allows motion to occur 

in parts of the image without unwanted alarms. The 

system can also be configured to view the scene remotely through an Internet connection and dial a designated telephone number upon a triggering event. In 

addition to setting up capture of video clips of motion, 

the surveillance software provides a viewer that will run 

the video clips at various speeds, stop the video at any 

selected frame, and save frames as time-stamped digital 

images (.jpg or .tif). 

Video clips can also be examined using commercial 

imaging software. Animation software (e.g., Windows 

Media Player, Jasc Animation Shop) makes it possible 

to view clips in real time or other speeds, forward or 

backward, or step through the frames individually. 

Additional analyses can be applied to frames saved to 

digital images. Images from succeeding frames can be 

compared using any of several relatively inexpensive 

programs that provide layering or mathematical operators for image manipulation (e.g., Adobe Photoshop, 

Jasc Paintshop, PictureWindowPro). In a subtraction or 

differencing operation between two nearly identical 

images, intensity values for corresponding pixels from 

each image are subtracted, resulting in a low value 

(dark) if the pixels are the same in the two images. For 

any pixels that have different intensity values between 

the two images, i.e., if there is a change, the result will 

be a bright image. 

Laboratory tests of the system showed that motion 

is readily detected and recorded. Calibration tests 

included dropping markers of different sizes and color 

values against a white board marked with a 25-mm grid 

as a background. In a typical test shown in figure 2, a 

black square 25 mm on a side was dropped, triggering 

the system when it had moved about 10 mm. That 

change represents about 100 pixels (10 by 10) within 

the 76,800 pixels in the image (320 by 240 pixels), or 

about 0.13% of the image. Other tests were used to 

evaluate stability and file size. For example, the system 

was left to monitor routine laboratory activity 

continuously over a 40-hour period. During that test, 

more than 100 individual events triggered the system, 

capturing a total of 15.6 minutes of video at about four 

frames per second. Individual events ranged from less 

than 1 second to more than 3 minutes. The resulting 

files totaled 26.7 Mb of hard drive memory. The system 

manages available hard drive space by writing over the 

oldest files when storage resources reach a defined 

level. Other tests showed that very slow motion within 

the detection area would not trigger the alarm, and 

recording and alarm functions would stop when motion 

within the area stopped, even though elements within 

the view had changed from the original image. 

Field tests in a local rock quarry showed that rock 

falls could be captured. Exterior change detection is 

based on a wider range of variables than are used in a 

more-controlled laboratory setting. To evaluate motion 

detection in the field, the system was set up in a rock 

quarry to monitor highwalls between 10 and 17 m high 

at ranges of 30 to 82 m. Rocks at least 15 cm across 

were dropped from the top of the slopes to generate 
minor rock falls. The dropped rocks and rock falls were 

sufficient to trigger the detection system, sounding an 

audio alarm and recording images of the rock fall. In 

one 13-minute interval of continuous monitoring, seven 

motion events were detected, including four manual 

triggers and each of three induced rock falls. The rock 

fall events lasted from 3.1 to 4.0 seconds; event times 

and durations are shown in figure 3. By recording video 

clips only when a threshold of motion was exceeded, 

storage resources (e.g., computer media) were reserved 

only for significant events, and time spent in reviewing 

recorded activity was minimized. 

Figure 2.—Laboratory test of video motion detection. The grid in the background is at 25-cm spacing; a 25-cm marker 

falls after release. The colored box shows the location of the original marker. At the left is the first frame captured after 

motion detection, the second frame (center ) was captured at 0.81 seconds, and the third frame (right) at 0.86 seconds. 

Figure 3.—Video clip recordings triggered by motion 

detection. During this 13-minute interval, about 25 

seconds of video were recorded in seven motion events, 

including three rockfalls, each lasting less than 4 

seconds. 

MINE EXPERIMENTS 

Field tests of video motion-detection technologies 

were conducted at mines in Montana and Wyoming. 

The prototype single-camera, battery-powered system, 

as well as a multi-camera wireless unit in development, 

were set up and operated at the Yellowstone talc mine 

(Luzenac America, Inc.) near Ennis, MT, and at the 

Black Thunder Mine (Thunder Basin Coal Company, 

LLC) near Wright, WY. Three days were spent at each 

mine monitoring slopes for rock movements. In each 

system, video cameras were trained on a mine slope 

where rock falls might be expected; computer-detected 

changes in the video image triggered recording of video 

clips to capture the rock fall event. Later, the surveillance software viewer program and third-party animation software (Jasc Animation Shop) were used to analyze the video clips. The experiments were intended to 

provide data for determining optimum range, resolution, and sensitivity settings for video slope monitoring 

and for improving and simplifying the configuration, 

set-up, and operation of the system. 

Yellowstone Mine 

At the Yellowstone Mine, the motion detection 

camera was trained on a portion of the highwall, at a 

range of 134 m, where rockfall had occurred recently 

(figure 4). No rockfall activity was detected over a 2­

day period, but rocks were dropped from above into the 

image area to generate rockfalls. Each of these was 

captured in two events totaling 220 frames. Figure 5 

shows a video frame that includes the impact of a 

dropped rock. The rock in motion (blurred) and a small 

cloud of dust were sufficient to be detected by the 

system. 

On the third day, the system was moved to the floor 

of the active pit to monitor rock faces at different 

ranges and settings. In the first configuration, the 

camera was directed toward a blasted but otherwise 

undisturbed rock face on the floor of the pit; range was 
Figure 4.—Highwall at the Yellowstone Mine showing area monitored by video camera (white box). 
39 m and resolution was 640 by 480 pixels. For 16 

events at this resolution, nine were manual triggers 

during set-up and testing, five were apparently wind 

induced, and two captured small-scale sloughs on the 

slope. In a second configuration, the camera was 

focused in on a part of the rock face, and resolution was 

changed to 320 by 240 pixels; rocks were dropped or 

thrown onto the slope (figure 6). Three artificially 

induced rock falls totaling 252 frames were captured. In 

one event, three sequential rock impacts were followed 

by sloughing over a total of 72 frames and an elapsed 

time of 39 seconds (figure 7). 

In additional settings, the camera was redirected at 

other parts of the broken rock face and at the talc 

highwall at a minimum range of 29 m. Small sloughs 

on the broken rock face were captured in three events 

totaling 542 frames. Weather played a role in generating false triggers during this interval, capturing snow 

and rain both in the air and on the camera enclosure 

window. Fast-moving clouds and brief periods of bright 

sun caused illumination fluctuations too great to be 

compensated for by the automatic iris of the camera and 

exceeded its dynamic range. Resulting video clips 

include periods of camera bloom (whiteout) where 

nearly all detail was lost in the images. 

Wind was found to present difficulties by shaking 

both the mast assembly and the camera mount and generating false triggers. More than 250 apparently windinduced events were recorded over about 17 hours of 

monitoring. Stiffening the mast by attaching ratcheting 

nylon straps to the base legs helped substantially, but 

wind-induced shake in the single-point camera mount 

continued to be a problem. 

Figure 5.—Video frame captured during detection of a rockfall. Box shows where rock impact was detected. Inset shows 

where in the box differencing with a previous frame revealed changed pixels. 

Black Thunder Mine 

The video system was initially positioned at the 

crest of the spoil pile above an active coal face to 

monitor the highwall above a shovel and truck loading 

operation; range to the highwall was about 168 m. 

Small-scale rock falls and sloughing were common 

along the highwall, especially from broken zones near 

the crest or part way down the face. Seeping moisture 

on the face was associated with these rock falls. The 

camera was first positioned with a wide-angle lens setting to include the highwall from its crest to the top of 

the coal. At this setting, 19 events were recorded over a 

period of 2 hours; rock movement was detected in three 

events accounting for 62 frames (22 frames per event). 

The remaining 16 events were apparently wind induced, 

resulting in 138 frames recorded (9 frames per event). 
Figure 6.—Broken rock in Yellowstone Mine main pit. 

Box shows area monitored by video system. White rod 

is 1 m long. 

 

Figure 7.—Video frame captured during detection of 

rockfall in area of box shown in figure 6. Bright areas 

show where change was detected. 

In a second setting the camera was zoomed in on a 

zone of broken rock and water seepage about a third of 

the way down the highwall from the top (figure 8). In 

this position over a period of 10 hours of monitoring, 

rock movements were observed in 32 events resulting 

in 1073 frames recorded (33 frames per event). No rock 

movement was apparent in an additional 101 events that 

were likely caused by wind shake of the camera, 

averaging seven frames per event. 

On day 3 at Black Thunder, the video system was 

positioned on the highwall side of the pit to view the 

spoil slope above a ramp intersection with the pit floor; 

range was 154 m. The view includes a spoil face where 

large-scale sloughing had occurred in recent weeks 

(figure 9). The video monitor was positioned to include 

a portion of the haul road along the pit floor so that 

vehicles using the road could trigger recording if no 

mask was used. The left video frame in figure 10, 

where the first parts of the haul truck are barely visible 

at the left edge, shows that even a very small change in 

the image can trigger recording. The right frame shows 

a mask within the image that would avoid vehicletriggered alarms. Rock movement was detected in none 

of the 355 events recorded; moving vehicles (haul 

trucks, graders, loaders, scrapers, pickup trucks) 

accounted for 132 events, averaging 212 frames per 

event. Nine events were manual triggers during setup 

and testing (1,014 frames or 112 frames per event). The 

weather on this day was very windy and wet; 214 

events are attributed to wind-induced camera shake or 

rain in the air or splattered on the camera enclosure 

window. In these events, 978 frames were recorded 

(five frames per event). 

DISCUSSION 

Changes in light and shadow in natural and mine 

environments make edge recognition a primary factor 

for detecting change within an image. Rock exposures 

that are relatively uniform in color make change 

detection difficult; for example, a brown rock moving 

against a brown background can be detected only by the 

shadows that shift as it falls. For that reason, the camera 

cannot differentiate between actual movement and 

changes in illumination such as when a cloud passes in 

front of the sun. In addition, wind can result in false 

alarms both by moving objects within in the image, 

especially vegetation, and by moving the camera itself, 

which causes the whole image to shift. 

The single-camera, battery-powered prototype 

system used in these experiments operated well in mine 

environments. The camera and power-zoom lens 

provided sufficient resolution to frame problem areas 

for monitoring without difficulty. Rock fall events were 

readily detected, although no estimate was made of 

undetected events. The system is relatively portable and 
Figure 8.—Highwall at Black Thunder Mine. Box shows area of broken rock and water seepage monitored by video 

system. Inset shows video frame captured during detection of rockfall; area in oval shows where change was detected 

Figure 9.—Slump area in spoils at Black Thunder Mine. Box shows area covered by video monitor. 
easy to set up. The power system operated without 

problems, and a simple solar charging system could be 

easily added to charge the 12-V battery. Reviewing 

recorded video clips showed that changes on the mine 

slopes were readily observable. Video clips of rock fall 

events provided a valuable database of slope activity, 

showing sources of fallen rock and just how and where 

material moved on the slope. 

The system, however, failed to prove its usefulness 

in providing real-time warning of rock falls. Excessive 

false alarms generated by wind and weather would 

make the present configuration unreliable. Modifications to stiffen the supporting mast and to stabilize the 

camera mount would likely solve the wind shake problem. Problems with rain and snow obscuring the camera 

enclosure window could be partially alleviated by 

adjusting the size and shape of the enclosure shroud, 

but triggers caused by the motion of falling rain and 

snow are more difficult to avoid. 

The minesite experiments also provided an opportunity for first-time testing of a multi-camera video 

system (figure 11) that represents a next generation 

from the original prototype. Still in development, the 

system includes an embedded modular computer to run 

upgraded surveillance software and radios to provide 

wireless Internet protocol access. The wireless access 

will allow remote program control and will provide 

real-time video feed from each of four cameras to any 

notebook, hand-held, pocket, or other wireless-equipped computer within 90 m of the base station. Multiple 

cameras will allow the system to monitor several areas 

simultaneously using different resolution, zoom, and 

mask settings. In addition to slope monitoring, this 

system could be adapted to a wide range of other 

monitoring tasks. 

Both prototypes relied on notebook computer LCD 

screens for pointing cameras and adjusting focus, zoom, 

and aperture settings. A problem shared by both 

systems is that the LCD screens were difficult to see in 

bright outdoor light. Brighter, higher-contrast screens, 

available on some hand-held and specialty computers, 

need to be incorporated for more effective outdoor use, 

although a light shroud to shield the screen would be an 

inexpensive short-term solution. 

Control of the cameras themselves also presented 

problems. Although the single camera system included 

powered zoom, focus, and aperture controls, the camera 

still needed to be manually tilted and aimed to frame 

the desired zone on the slope for monitoring. Lenses in 

the multi-camera system had manual zoom and focus 

settings. Setup would be greatly simplified if each 

camera had remote pan, tilt, and zoom controls along 

with an autofocus lens. 

Figure 10.—Two frames from video clip of spoils pile. Each frame covers about the area shown in figure 8. In left frame, 

haul truck just entering at left edge was sufficient to trigger recording. Right frame shows masked area that would prevent 

machinery from triggering system. 

CONCLUSIONS 

Video motion detection was shown to be effective 

in identifying small changes in the video image. 

Relatively minor rock falls were recorded in active 

mines at ranges greater than 160 m. Video cameras 

need to be absolutely steady to provide motion 

detection; wind and precipitation during the mine 

experiments resulted in an excessive number of false 

triggers. 

In addition to the potential for warning workers in 

the vicinity of hazardous rock slopes, recorded video 

images can allow shift bosses and safety investigators 

to reconstruct the rock fall and help identify areas of 

unstable ground. Archived video images can also pro- 
vide quantitative information on the size and frequency 

of rock falls. One significant advantage of motiontriggered monitoring is that only short intervals of video 

are recorded, conserving storage media resources and 

making scenes of interest easier to find. 

Digital and video cameras have proven valuable for 

recording mine slope conditions. Computer tools using 

time-lapse and motion-sensing methods can provide 

means to document slope failures and warn workers of 

rock falls. Basic development should focus on ways to 

streamline framing, capturing, and processing digital 

images and overcoming false alarms caused by wind, 

rain, and lighting. 

Figure 11.–Multi-camera, wireless-access video monitoring system at Black Thunder Mine. 

ACKNOWLEDGMENTS 

Access to mining operations and assistance in the 

mine site experiments provided by Mike Cerino, 

Luzenac America, and Mike Hannifan, Thunder Basin 

Coal, are greatly appreciated. The contributions of Rich 

Rains, Spokane Research Laboratory, in assembly of 

the prototypes and in the mine experiments were 

invaluable. 

REFERENCES 

Allersma, H.G.B., 1996. Using Digital Image Processing in Field Measurement. Geotechnique, vol. 46, no. 

3, pp. 561-563. 
Collins, R.T., A.J. Lipton, T. Kanade, H. Fujiyoshi, D. 

Duggins, T. Yanghai, D. Tolliver, N. Enomoto, O. 

Hasegawa, P. Burt, and L. Wixson. 2000. A System for 

Video Surveillance and Monitoring. The Robotics 

Institute, Report CMU-RI-TR-00-12. Carnegie Mellon 

University, Pittsburgh, PA, 68 pp. 

Corthésy, R., M.H. Leite, and C. St-Cyr. 2001. 

Potential Use of Digital Imaging for Rock Movement 

Detection. In Rock Mechanics in the National Interest, 

Proceedings of the 36th U.S. Rock Mechanics Symposium. Elsworth, Tinucci, and K. Heasley, eds. (Washington, D.C., July 2001). A.A. Balkema, Rotterdam, 

Netherlands, pp. 473-480. 

Dombe, R. J., J.L. Eirls, and R. Smith. 1982. Video 

Image Sensor Systems Applied to Pit Slope Displacement Monitoring. Mineral Research Contract Report 

(No. J0295040). Tracor MBA, San Ramon, CA, prepared for U.S. Bureau of Mines, 29 pp. 

McHugh, E., and J. Girard. 2002. Evaluating Techniques for Monitoring Rock Falls and Slope Stability. 

In Proceedings 21st International Conference on 

Ground Control in Mining, S. S. Peng, C. Mark, A. 

Wahab Khair, and K. Heasley, eds, (Morgantown, WV, 

Aug. 6-8, 2002). Univ. of West Virginia, Morgantown, 

WV, pp. 335-343. 

McVey, J.R., S.R. Lewis, E.E. Guidice, and J.R. 

Powell. 1974. Deformation Monitoring of Underground 

Openings by Photographic Techniques. U.S. Bureau of 

Mines Report of Investigations 7912, 26 pp.