____________________________ 

Figure 1. Dozer cover-up on coal stockpile. 

Remote Vision System for Dozers on Coal Stockpiles 

William H. Schiffbauer, Electronics Technician
 

Michael R. Yenchek, Team Leader Electrical Safety
 

Timothy Lutz P.E., Mechanical Engineer
 

National Institute for Occupational Safety and Health
 

Pittsburgh, PA
 

Abstract—The National Institute for Occupational Safety and 

Health (NIOSH) Pittsburgh Research Laboratory (PRL) 

participated in an experiment to facilitate the remote control of 

dozers used on coal stockpiles. The experiment consisted of 

supplementing the normal sensory cues an operator requires to 

manipulate the dozer from a remote location. NIOSH provided 

and tested a vision systems intended to give the operator the 

ability to operate the dozer safely and efficiently from a remote 

position. This paper highlights the genesis of the experiment, 

identifies the contributions of Consol Energy and Caterpillar, 

and details the NIOSH efforts and successful test results. 

Keywords-remote-control; vision system; stockpiles; safety 

I. INTRODUCTION 

Environmental conditions, coal compaction, and other 

factors can result in voids in coal stockpiles which can entrap 

bulldozers (Fig. 1) used to facilitate drawdown at the top of 

the piles [1, 2, 3]. Since 1980, there have been 18 fatalities at 

coal stockpiles, the majority being bulldozer operators [4]. 

Efforts by the Mine Safety and Health Administration 

(MSHA) and others have led to development of improved cab 

designs, high-strength windows, and communications which 

have proven to save operator lives during dozer cover-ups [5, 

6, 7]. Providing remote control of the dozer has the potential 

to totally eliminate the danger to the operator by removing 

him/her from the machine. 

II. BACKGROUND AND APPROACH 

Hundreds of bulldozers in the U.S are used to facilitate 

drawdown at the top of coal stockpiles. Previous attempts in 

the U.S. to remotely control the dozer were never fully 

implemented due to a variety of issues. One drawback was 

that the operator, when removed from the dozer, lost the “feel” 

of the machine. This resulted in significant inefficiencies 

during remote operation as compared to on-board control. 

Disclaimer 1: “This information is distributed solely for the purpose of 

pre-dissemination peer review under applicable information quality 

guidelines. It has not been formally disseminated by NIOSH. It does not 

represent and should not be construed to represent any agency determination 

or policy.” 

Disclaimer 2: Mention of any company or product does not constitute 

endorsement by the National Institute for Occupational Safety and Health 

(NIOSH). In addition, citations to Web sites external to NIOSH do not 

constitute NIOSH endorsement of the sponsoring organizations or their 

programs or products. Furthermore, NIOSH is not responsible for the content 

of these Web sites. 

Consol Energy has been interested in addressing the 

problem of bulldozer cover-ups on coal stockpiles. MSHA had 

also been cognizant of the hazards associated with stockpiles 

and provided input during planning for the project. Caterpillar 

has been investigating the feasibility of incorporating remotecontrol technologies on its bulldozers. These parties 

informally collaborated to implement a remotely-operated 

dozer on a coal stockpile. Meetings were held with Consol, 

Beckwith Machinery, and Caterpillar at Consol’s Eighty-Four, 

PA office to formulate the approach and test plan. 

Consol Energy provided a test site and a field trailer 

(Fig. 2), at their Eighty Four Mine coal stockpile near Eighty 

Four, PA. The trailer was outfitted with power, heating, 

cooling, and water. A large window provided a wide-angle 

view of the stockpile. Installation and operational assistance as 

well as access to dozer operators, was provided as needed. 

Caterpillar of Peoria, IL provided a temporary loan of a 

D10T dozer (Fig. 3) and onsite technical support during 

system integration and testing. The dozer included a remote 

control system (Fig. 4) which interfaced to other dozer 

electronic systems through a CAN bus-based data network 

using the SAE Standard J1939 protocol. The dozer could be 

operated either via remote control or by an on-board operator. 

Other features included emergency stop (E-Stop), automatic 

blade control while the dozer carries a load (AutoCarry); 

automatic dangerous zone alerts using the GPS-based CAES 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Ultra system, remote diagnostics, and automatic braking. 

Further details are available at their web site 

(http://www.cat.com/cda/layout?m=71342&x=7). 

Figure 2. Consol Energy test site. 

Figure 3. Caterpillar D10T dozer. 

Figure 4. Dozer remote control. 

The intended outcome of this NIOSH research was to 

improve the safety of the operators of bulldozers on coal 

stockpiles by investigating the feasibility of sensor-enhanced 

remote control. The research project was conducted to answer 

the following: 

• Can a camera system substitute for line-of-sight 

operation of a remotely-controlled bulldozer? 

•	 Would it be necessary to enhance operator acceptance 

by using audio and/or motion cues? 

It was to be accomplished through four tasks: 1) study the 

operation of bulldozers on stockpiles, 2) select, acquire, and 

test technology that can supplement the normal sensory cues 

an operator requires to manipulate a dozer, 3) integrate the 

components on a dozer, and 4) test the system on a dozer on a 

coal mine stockpile. 

The NIOSH approach was to provide the operator of the 

remote-controlled dozer with sufficient sensory cues so that 

he/she could safely and efficiently operate the dozer from a 

distance. Dozer operations on a coal stockpile were observed 

to learn of basic operators’ needs. On-board control of dozers 

required the operator to constantly scan the work area, not 

only directly in front of the machine, but also side-to-side and 

to the rear. Visual feedback to a remote control site should 

consist of a field-of-view that is continuous (not discrete) and 

that would respond rapidly to the operator’s motion. Another 

perceived need would be remote stereo audio. This would 

enable the operator to be aware of subtle changes in dozer 

performance and production environment dynamics. 

Additionally, the operator would need to sense the pitch and 

roll of the dozer on the stockpile for safe operation. 

Most of the technologies required were available off-theshelf. The performance of the equipment in a production 

scenario was a major concern. Gaining operator acceptance of 

the sensory technology was seen as one roadblock to success. 

Other envisioned obstacles were the operator’s ability to use 

the system under normal and foggy conditions, the ability of 

the hardware to be reliable in the harsh environment, and the 

efficiency of the operator using a remote-controlled system as 

compared to a conventional manned dozer. The answers to 

these concerns could only be determined through 

experimentation. 

III. DIGITAL VISION SYSTEM (DVS) 

To satisfy the visual sensor needs some type of 

camera/monitor system would be required. One basic 

requirement was to provide the operator with a 360-degree 

view of the dozer area with as close to the same field of view 

as if he/she were operating the machine in a conventional 

manner. The type of systems investigated included simulators, 

pan/tilt, omni-directional, stationary, stereo, and infrared 

cameras. Though stereo video was desired, NIOSH found that 

it was not available commercially. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Pan and tilt cameras using operator head movements to 

control the cameras were investigated. The concept consisted 

of the operator wearing a pair of goggles, while operating the 

dozer via the remote control pendant. As the operator looked, 

up, down, right, or left, his head motion would control the 

dozer-mounted pan and tilt camera, thereby providing a 360-

degree view of the production area. The goggles could also 

provide stereo audio. A candidate vendor (Chatten 

(www.chattenassociates.com)) provided a demonstration of 

this technology at NIOSH and at the dozer stockpile site. The 

performance of the head-tracking system was considered 

acceptable. The controls for the remote-control pendant of the 

dozer and remote readouts of dozer gauges could be seen by 

looking down below the goggles. The stereo audio was not 

demonstrated, though the vendor could supply that option if 

required. 

Another candidate system consisted of a fixed camera 

focused on a parabolic mirror which reflects a 360-degree 

field of view around the machine (Remote Reality , 

http://www.remotereality.com/content/view/63/115/). The 

camera would be mounted inside an appropriate housing. The 

panoramic view would constantly be transmitted to the remote 

site. Responding to the operator’s head movement, tracking 

software would excise the portion of the panoramic view of 

interest, process it into a planar image, which would then be 

transmitted to goggles. The vendor provided a demonstration 

of the technology at NIOSH. Stereo audio was not an option. 

The vendor indicated that the system could be used in 

combination with a large monitor to mirror what the operator 

saw in the goggles and also provide simultaneous views of 

other areas around the machine. 

A. Initial System Integration 

NIOSH and stakeholders reviewed the technology and 

decided to implement the Remote Reality camera system for 

the initial system. The selection was based on the clarity of the 

video images provided. The audio feedback technology would 

be addressed at a later time. Laboratory testing of the camera 

system was successful. The camera system components 

consisted of a PC, a 42-inch LCD monitor, a pair of electronic 

goggles with head tracker, a panoramic digital video camera, a 

two-node wireless link, and special software. Fundamentally 

the whole system was a wireless local-area-network (LAN), 

consisting of two WiFi g (also known as IEEE 802.11g) 

modems. The camera video was accessed by the PC using a 

static internet protocol (IP) address. The dozer operator could 

choose to use either the goggles to operate the dozer, or else 

use the LCD monitor. There were a number of viewing 

features available. All were software selectable. The LCD 

could echo the goggle view, or each could provide 

independent views. The LCD could show one wide view of 

the front of the dozer, or a wide front view and wide rear view, 

or one wide front view, and two or more auxiliary views 

around the dozer. 

B. DVS Field Tests 

The digital video system (DVS) was installed on the dozer 

(Fig. 5) and in the control trailer. The operators were given an 

opportunity to run the dozer using both the goggles (Fig. 6) 

and the LCD monitor for remote control. The general 

consensus of the operators was that the goggles were 

impractical as they limited their field of view, were generally 

uncomfortable, and were too heavy to wear for an 8-hour shift. 

Use of the LCD monitor was more acceptable. Establishing 

the desired views on the screen required many configuration 

changes. With the camera box attached on top of the cab, the 

operator had better overall views, but from a different 

perspective than when sitting in the cab. Here it was difficult 

for the operator to fully view the edges of the dozer blade. 

The decision was made to add more fixed cameras to the 

dozer because one camera was installed to view the left side of 

the dozer blade and tracks, a second to view the right side of 

Figure 5. Digital cameras on dozer cab. 

Figure 6. Goggles with head tracker. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

the dozer blade and tracks, the third was fixed forward, and 

the forth rearward (Fig. 7). While this configuration satisfied 

the operator requirements, the visual performance wasn’t 

acceptable. Latency, or the time delay for video transmission 

back to the trailer, was in the 0.5 to 1 second range. With the 

maximum dozer speed up to eight miles per hour the dozer 

could cover five feet before a given video image would 

update. To eliminate the time delays in the transmitted video a 

decision was made to pursue an analog-based video system. 

Figure 7. Digital video outputs at trailer. 

IV. ANALOG VISION SYSTEM (AVS) 

The basic requirement for the analog video system (AVS) 

was to provide a wireless, real-time, 360-degree image from 

the dozer. No vendor could be found to supply a system with 

all these characteristics as one product. However one vendor 

(DTC http://www.dtccom.com/) was found that could provide 

the wireless transmission system for four simultaneous video 

channels. The vendor could also supply a variety of cameras 

with different lenses. NIOSH specified wide-angle lenses that, 

when combined, would capture a 360-degree image. NIOSH 

obtained the technology and packaged the cameras in a rugged 

compact enclosure. The contents of the enclosure were; four 

cameras, four wide-angle lenses, a four-channel multiplexer, 

an analog transmitter, an antenna, and a battery (Fig. 8). The 

operator end of the system consisted of two antennas, a dualdiversity receiver, a four-channel de-multiplexer, and four 40-

inch LCD monitors. The LCD monitors were arranged in a 

circle, with a chair in the center for the operator. Video quality 

was acceptable, and there was no perceptible image latency. 

This hardware did provide remote monaural audio from one 

microphone. 

Figure 8. Analog video camera box. 

A. AVS Laboratory Tests 

A compact NIOSH-developed, battery-operated, radiocontrolled, tracked, robotic vehicle (called a Multi-Purpose 

Tele-operated Tracked vehicle (MUTT)), was acquired from a 

previous NIOSH project for the purpose of testing the dozer 

video system. The camera housing of the AVS system was 

installed on the MUTT (Fig. 9). The receiver, demodulator, 

and four LCD monitors were placed in a small room. With 

very little training the operator could maneuver the MUTT 

solely via viewing the images on the LCD monitors (Fig. 10). 

The system was tested inside an unlit building, outside under 

heavy cloud cover, and outside during a bright sunny day. 

There were no perceptible latencies even when the MUTT was 

operated at full speed. 

Figure 9. MUTT test vehicle at PRL. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 10. Lab analog video camera tests. 

B. AVS Field Tests 

The AVS was taken to the Eighty Four Mine coal 

stockpile for further tests. The camera housing was placed on 

top of a non-remote-controlled D10R dozer (Fig. 11). The four 

LCD monitors and other equipment were installed in the field 

trailer (Fig. 12). The operator of the dozer was instructed to 

take the dozer around the entire periphery of the coal 

stockpile, through normal production areas as well as behind 

the stacker tubes. The video viewed in the field trailer was 

excellent. The whole stockpile could be covered with no loss 

or degradation to the video quality. A little noise in the video 

was noticed when the dozer went behind the stacker tubes. 

Since the system was not tested with a remote controlled dozer 

it is difficult to conclude the effectiveness or efficiency of a 

remote operator using the camera system. Future testing on a 

remote-controlled dozer is required. The audio system was not 

tested. The project concluded with these AVS field tests. 

Figure 11. Analog video camera box on dozer cab. 

Figure 12. Analog video camera field tests. 

V. SUMMARY AND CONCLUSIONS 

The purpose of the project was to improve the safety 

associated with the operation of bulldozers on coal stockpiles 

by investigating the feasibility of sensory-enhanced remotecontrol operation. The approach was to provide the operator 

with sufficient sensory cues so that he/she thought they were 

actually operating the dozer from on-board. A variety of tests 

were conducted with different camera systems. An analogbased video camera system provided excellent 360-degree 

real-time viewing coverage of the dynamic dozer production 

area. However, testing with a remote-controlled dozer is 

required to determine overall system efficiency and 

effectiveness. 

Echoing the pitch-and-roll motion of the dozer on the 

stockpile would be a desirable option that should be explored. 

Motion platforms are being used by the military to train 

vehicle operators (Army Guide, 2005, http://www.armyguide.com/eng/AG_Monthly.php). Developers of simulators 

also employ motion platforms. While most applications use 

the platforms in the simulator/trainer mode, the dozer 

application requires real-time operation. It was expected that 

some combination of off-the-shelf hardware with 

customization would be required to provide the real-time 

feedback desired. The concept envisions the use of a sixdegree-of-freedom motion platform used for simulators, a 

wireless sensor package to acquire dozer motion, and custom 

software to acquire the data and drive the platform. A wide 

internet search for applicable technology identified one 

vendor’s hardware (Servo’s and Simulators Inc. 

http://www.servos.com/) which appears to have the requisite 

features. 

Efficient remote control of dozers on coal stock piles 

requires the operator to see and sense the environment around 

the dozer while off-board. NIOSH developed and tested a 

vision system which was fully capable of providing the 
 

 

 

 

 

 

 

 

 

operator excellent remote 360-degree views around the dozer. 

Future work is required to provide dozer motion cues to the 

operator. 

REFERENCES 

[1]	 W.J. Wiehagen, A.G. Mayton, J.S, Jaspal, and F.C. Turin, "An analysis 

of serious injuries to dozer operators in the U.S. Mining Industry," Dept. 

Health & Human Services, NIOSH Information Circular 9455, 

Publication No. 2001-26. 

[2]	 J.W. Fredland, K.K. Wu, and D.W. Kirkwood, "Material instability 

hazards in mine-processing operations," U.S. Dept. Labor, MSHA IR 

1212, 1993, 10pp. 

[3]	 D. Zageer, "Discussion of dangers involved when transporting and 

storing unconsolidated materials," Mining Engineering, January, 1987; 

pp 23-25. 

[4]	 Mine Safety and Health Administration, web page http:// 

www.msha.gov/techsupp/techexchange/surgepile/surgepilesafety.pdf 

[5]	 M. Skiles, "MSHA’s triangle of success helps safe lives," Coal People 

Magazine, March 2003. 

[6]	 J.W. Fredland, "Stronger cab windows: a way to improve surge-pile 

safety," U.S. Dept Labor, MSHA Status Report, December 1, 1999; 6pp. 

[7]	 R.A. Elam, "Fatalities resulting from hazards on coal stockpiles," U.S. 

Dept. Labor, MSHA Program Information Bulletin P99-1, 1999.