［ Solution・Supporting Service ］

Detection of Falling Objects inTracks by Using Deep Learning※1
Keywords

Taiki Yamamoto

Railroad, Detection of falling object, Deep learning

Abstract

If tools, bolts, and and other objects remain on railroad tracks after the completion of maintenance and improvement work, there is a serious threat that adangerous accident can occur. In order to eliminate accidents caused by such
fall objects, checking work for objects is carried out immediately after the workends. For automating this checking work, we developed a detection technologyfor fallen objects on railroad tracks by using an image processing function of acamera.
For the detection of fallen objects, we adopted the method of deep learningthat is noted for its high performance in resolving prediction problems and its
attention to detail. By the deep learning, the presence of fallen objects displayedon the screen can be detected. By virtue of this expertise, the identiﬁcation
accuracy of fallen objects and debris is raised to 99.3%. If this technology isadopted for the dedicated inspection vehicle that is used for the checkout workfor ﬁnding fallen objects and debris on the tracks after the maintenance and/orimprovement work, it will contribute considerably to the automation of fallen
object detection work for railroad facilities.

1 Preface
Maintenance and improvement work for railroad facilities is generally carried out at night after
the last train and before the ﬁrst train of the following
day. If any tools, bolts, or fallen objects used for thework are carelessly left on the railroad tracks, adanger for a serious accident like derailment presents. For this reason, checking work is carried out
by workers after the completion of night work inorder to ﬁnd any objects left behind.
Such checking work may be done by using a
dedicated inspection vehicle. Such an inspection
vehicle is equipped with an obstacle detection bar
under the vehicle ﬂoor. If anything comes in contact
with this bar, it is sensed as an obstacle. In manycases, however, most fallen objects on the track do
not enter the minimum rolling stock gauge and theobstacle detection bar may fail to detect them. For
the current checkout work, multiple individuals
stand on the track, side-by-side in line and search
for any fallen objects by illuminating their steps witha ﬂashlight. Such a work is taxing on workers.
Against this background, the market calls for a tech-

nology for automatically detecting fallen objects on
railway tracks. This paper introduces an automatic
detection technology for fallen objects on the railway tracks by using deep learning.

2 Fallen Object Detection Approach
For the detection of fallen objects, an area
where a camera tilted downwards is installed on avehicle roof for view. Fig. 1 shows an example of anFig. 1

Example of Input Image

A normal image is shown on the left and an abnormal image onthe right. A normal image is devised to show ballast, sleepers,
and rails only, and the abnormal image shows a fallen object.

MEIDEN REVIEW Series No.181 2021 No.155

128

11Normalimage

112959 5533
31429591281414 3384

331414384

14

Abnormal
image
256
4096

2561024

296
Input image

3Convolution layer

Fig. 2

Fully connected layer

Network Configuration Used for Auto-Detection Techniques

A network conﬁguration is shown where a fully connected layer of AlexNet is made to adapt this approach by ﬁne tuning.

input image. In this approach, an accurate image istaken, only of the ballast (gravels laid on roads andtracks), sleepers, and rails. An image where any
fallen object is taken is regarded as an abnormal
image. In this approach, a technology of deep learning is used only for two classes of recognition: anormal image and an abnormal image. Deep learning is one machine learning approach where alarge-scale neural network having deep structures
is employed. Recently, this approach has been recognized in the world for its outstanding performancein the ﬁeld of identiﬁcation and prediction of problems.
This approach employs a network structure
proposed by Alex et al. (“AlexNet” hereafter). Among
many deep learning network structures, the AlexNet
has proven track records in the ﬁeld of image object
recognition. Since the AlexNet has a network structure where images are identiﬁed into 1000 classes,
the output of the fully connected layer also contains
1000 classes. Apart from this, the output of ourapproach involves only two classes: normal image
or an abnormal image. For this reason, it is necessary to make ﬁne-tuning beforehand in order toadapt the fully connected layer of the AlexNet to ourapproach. Fine-tuning is a process when a network
ﬁnishing learning a certain problem, it is then madeto learn the last part of the weight again so that it
can then adapt it to another problem. The convolution layer of deep learning can be used as anextractor feature. In addition, it has a characteristic
capable of extraction common to a variety of prob-56

MEIDEN REVIEW Series No.181 2021 No.1

lems in the image recognition ﬁeld. As such, ﬁnetuning is believed to be effective. Still more, ﬁne tuning offers an advantage featuring efﬁcient learning
based on a small amount of learning data. In thisapproach, the fully connected layer consisting of
three layers can be made to learn newly with a vector input of 4096 dimensions obtained from theAlexNet convolution layer. Fig. 2 shows a network
conﬁguration used for auto-detection techniques.
Using a network obtained from the aforementioned
learning, 2-class recognition of normal image andabnormal image is carried out.

3 Accuracy Veriﬁcation
We veriﬁed an accuracy of this technology
through the photo-taking of normal and abnormalimages in the daytime inside the railroad track. The
number of data was 26,562 in total of both normal
and abnormal images. 87.5% of total data volumefor both normal and abnormal images was used as
the learning data respectively. Each remainder of
12.5% was used as evaluation data. Fig. 3 shows arelationship between learning frequencies and identiﬁcation accuracy. Around 300 times of learning frequencies, the identiﬁcation accuracy came to convergence. The obtained identiﬁcation accuracy was
99.7% for the learning data and 99.3% for the evaluation data. Fig. 4 shows an example of successful
fallen object detection. Input images are shown on
the left side while the right side shows images where
the inﬂuence rate on the recognition is visualized.

1.0

4 Postscript

Accuracy

0.9
Learning data

0.8

Test data

0.7
0.6
0.5
050

100
150200
Learning frequencies (times)

250300We introduced a method for detecting fallen
objects on railway tracks by using deep learning.
With this technology, we were able to show the possibility of automating the work of checking for fallobjects. In the future, we aim to put it into practicaluse by verifying it on an actual vehicle assuming
actual operation.
・All product and company names mentioned in this paper areFig. 3

Relationship between Learning Frequencies andIdentification Accuracy

The axis of ordinates shows identiﬁcation accuracy and that ofabscissas shows learning frequencies. The blue line shows theresult of identiﬁcation of an image included in learning and theblack line indicates the result of identiﬁcation of an image notincluded in learning.

Fig. 4the trademarks and/or service marks of their respective owners.
(Note)※1. An approach to let a computer learn so that the computer can
execute a human beingʼs tasks such as voice recognition, image
definition, and prediction.

Example of Successful Fallen Object Detection

An example of images of the detected fallen object is shown.
Input images are shown on the left side while the right sideshows images where the inﬂuence rate on recognition is visualized. The inﬂuence rate on recognition by deep learning is shownwith grayscale values. The closer to white, the greater the inﬂuence rate.

MEIDEN REVIEW Series No.181 2021 No.1

57