Mitsubishi Heavy Industries Technical Review Vol. 58 No. 2 (June 2021)
1

Automatic Human Detection System forForklifts Watched Over by AI

Mitsubishi Logisnext Co., Ltd.

Big body industrial vehicles such as large capacity forklifts have large blind spots and theforklifts and equipped with safety mirrors and on-board cameras. However, a satisfactory effectcannot be achieved unless the forklift operator pays close attention to these items. The demand fora safe driving support system from customers to Mitsubishi Logisnext Co., Ltd. is increasing year
by year, accordingly.
We therefore have developed a technology that uses deep learning to enable real-time
automatic human detection from images generated by the cameras on board the forklift, and thencommercially release such system as alerting users by such technology. The configuration andfeatures of such system for forklift are presented in this report (Figure 1).

Figure 1

Human detection system|1. Product summary
1.1

Configuration

The system is comprised of cameras that produce images, a controller that detects only
humans in the acquired images, and a buzzer and lamp unit to alert the operator at the time ofdetection. The system configuration diagram is shown in Figure 2.
a)
CamerasThe on-board cameras are of two types: short-range and long-range. The short-range
cameras are used to monitor all around the forklift when it starts off or it is operating at lowspeed, while the long-range cameras monitor the direction of travel when the forklift isoperating at high speed.
b)Controller
The controller uses deep learning to only detect the presence of humans around theMitsubishi Heavy Industries Technical Review Vol. 58 No. 2 (June 2021)
2forklift in images from the cameras in real time. When detected, a signal is transmitted to thebuzzer and the lamp unit. While receiving the forklift’s controller area network (CAN) signals,
it controls the on/off state of the detection system and automatically selects which camera
images to be monitored.
c)BuzzerOnce detected, the buzzer alerts the operator of the presence of a human. As the alarmvolume can be changed, the operator can set the sound at a level appropriate under a given
work environment.

Figure 2d)

System configuration diagram

Lamp unitThe lamp unit indicates to the operator the direction in which the detected human islocated. Once alerted by the buzzer, the operator can check the lamp unit to see the human’s
location that is indicated by one of the four divided areas around the forklift. As the lamp unit isalso equipped with a system operation status indicator lamp, it is possible for the operator tocheck for the occurrence of system errors and other abnormalities. The lamp unit is shown in
Figure 3.

Figure 3

Lamp unit

Mitsubishi Heavy Industries Technical Review Vol. 58 No. 2 (June 2021)
3

1.2

Environment resistance performanceThis system features high reliability by heat radiation structure to avoid overheat of internal
processor, vibration resistance and electromagnetic compatibility, etc., while it has waterproof anddustproof performance, and it is designed so that it may normally operate even in severe working
environment of forklift.

1.3

Applicable vehicles

This system is applicable to large capacity forklifts (FD120 to 230-6, FD240-5 and FD310-5)
and reach stackers (MR4531-3).

|2. Features
2.1

Selective detection of humans

This system employs a Single Shot Multibox Detector (SSD) as a deep learning model toonly detect humans in images. With its capability of learning a large number of real environment
images collected through field tests and, detecting humans by tuning technology originallydeveloped, it has succeeded in dramatically improving the detection rate (i.e., the rate of detecting ahuman as a human) while suppressing false detection.
The selective detection of humans allows no false alarm to be issued against objects present
in a work environment in which there are usually various items and goods in the vicinity.
Therefore, too many warnings can be avoided.

2.2

Automatic camera switching between short-range and long-range

Switching between short-range and long-range cameras is automatically done depending on
the speed of the forklift. This enables the monitoring area to be appropriately set while lessening
the burden on the operator.

2.3

Detection notification

Once detected, the buzzer sounds to alert the operator of the presence of a human in thevicinity. At the same time, the lamp also illuminates to indicate the direction in which the detected
human is located, thus urging the operator to confirm the safety with his or her own eyes and/or on
the Good Viewer(*) screen. (*Good Viewer is a support system to confirm the safety through abird’s-eye view of the forklift’s surroundings.)

2.4

Prevention of forklift starting at the time of detection

When the presence of a human is detected while a forklift is halted, the system blocks thevehicle from starting to prevent an accident. If this precautionary measure interferes with work
progress, it can be switched off temporarily. This feature is realized by a system that is integrated
with vehicle control, and is therefore a selling point that our product is different from similar onesof other companies because the latter are sold separately from the vehicle itself and do not havesuch capability by itself.

|3. Future development
The human detection system for forklifts is a safe operation support system that considerably
contributes to improving the safety of large forklifts with large blind spots. We will continue tosatisfy the needs of our customers for safety by further expanding its application to small forklifts.