# Automatic Detection of Ballast Unevenness Using Deep Neural Network
## 3 Experimental Results
### Training and Testing Mask R-CNN Network  
The Mask R-CNN network was implemented using the Keras package in Python. Using the idea of transfer learning, two pre-trained networks were tested. The first uses Resnet51 as the backbone, while the second uses Resnet101. Each of the networks was trained for 400 epochs on a computer equipped with an Intel I5 CPU, NVIDIA GeForce GTX 1080 Ti GPU and 48 GB RAM. We used AdamOptimizer with a learning rate Lr = 0.0001, weight decay w_dec = 0.0005 and momentum = 0.9 to minimize the loss function (loss error) defined by Formula (1). While training, two groups of data were used: _train_data_ for training and _valid_data_ for validation. Figure 7 shows the loss error for the Mask R-CNN with the ResNet101 backbone during the learning process for the _train_data_ set (Figure 7a) and the _valid_data_ set (Figure 7b). After training, the performance of each network was verified on a _test_data_ set. The network performs both the detection of individual sleepers and their segmentation. For this reason, the Average Precision (AP) metric was used to assess the quality of sleeper detection, while Intersection over Union (IoU), Dice coefficient and Pixel Accuracy metrics were used to assess the quality of the segmentation. Pixel Accuracy calculates the percentage of pixels in the image that were correctly classified. It is commonly reported for each class separately. Pixel Accuracy can be defined as:  
\[\textit{Pixel accuracy}=\frac{\textit{TP}+\textit{TN}}{\textit{TP}+\textit{TN}+ \textit{FP}+\textit{FN}} \tag{4}\]  
where _TP_ true positive represents pixels that are correctly classified to belong to the sleeper class (according to the target (truth) mask); _TN_ true negative represents pixels that are correctly classified as not belonging to the sleeper class; _FP_ false positive represents pixels that are classified as belonging to the sleeper class though they do not belong to sleeper class; and _FN_ false negative denotes pixels that are classified as not belonging to the sleeper class though they belong to the sleeper class. Intersection over Union (IoU) quantifies the percentage overlap between the target (truth) mask and our prediction output (predicted mask). The IoU is defined as:  
\[\textit{IoU}=\frac{T\_A\cap P\_B}{T\_A\cup P\_B}=\frac{\textit{TP}}{\textit{ TP}+\textit{FP}+\textit{FN}} \tag{5}\]  
where _T_A_ denotes pixels belonging to the target mask and _P_B_ represents pixels that are classified as belonging to the target mask. The dice coefficient can by defined as:  
Figure 7: The loss error for Mask R-CNN with ResNet101 backbone defined by Formula (1) during the learning process for the _train_data_ set (**a**) and the _valid_data_ set (**b**). \[Dice=\frac{2*(T\_A\cap P\_B)}{T\_A+P\_B)}=\frac{2*TP}{TP+TN+FP+FN} \tag{6}\]  
_AP_ is calculated based on the PASCAL VOC 2017 challenge. _AP_ is used to evaluate sleeper detection. For this reason, in the IoU metric in (5), a target bounding box is used instead of a target mask. Additionally, true positives (_TP1_), false positives (_FP1_) and false negatives (_FN1_) were redefined. _TP1_ is calculated when the IoU for the detected object is higher than the assumed threshold, whereas _FP1_ is calculated when the IoU is lower than the assumed threshold. _FN1_ is determined when the model did not predict a bounding box at a certain position although the target bounding box exists at that position. The precision-recall curve is used while defining the _AP_ metric. Precision is the ratio between the correct detections and the total number of detections:  
\[\text{precision}=\frac{\text{correct detections}}{\text{total number of detections}}=\frac{TP1}{TP1+FN1} \tag{7}\]  
Recall is the ratio between the correct detections and all available objects (target bounding boxes):  
\[\text{recall}=\frac{\text{correct detections}}{\text{all target bounding boxes}} \tag{8}\]  
A precision-recall curve plots the value of precision against the recall for different confidence threshold values.