# An Improved Feature Pyramid Network and Metric Learning Approach for Rail Surface Defect Detection
## 4 Experiment
### Experimental Results of the miniImageNet Dataset  
In the experiment, the 5-way 1-shot and 5-way 5-shot modes were used to train the network. That is, one sample (1-shot) and five samples (5-shot) were selected from each category of the defect dataset to train the network. The loss curves under the different training times are given in Figure 6. The curves in Figure 6 show that the training loss values of the methods in the figure decreased gradually, and the curves converged gradually during the training process, with the curves using RepMet and our method being more stable.  
(1) Performance evaluation  
In order to compare the proposed method and traditional deep learning, the methods of the training network with the miniImageNet dataset and learning from scratch are usually used to test the performance of the network. Our experiment adopted the training method on the miniImageNet dataset, and the experimental results are listed in Table 1. Table 1 shows that the mAP of our method was better than the other methods except for the MBSS method [48] from the state of the art in the 5-way 1-shot and 5-way 5-shot modes, which implies that our method is also satisfactory. Our method was 5.7% and 8.34% lower than MBSS in the 5-way 1-shot and 5-way 1-shot tasks, respectively. The backbone network used by MBSS is ResNet12, whose network model complexity is lower than that of our method.  
Figure 5: The types of rail defects.  
(2) Ablation experiments  
In order to verify the effectiveness of the improved method in this paper, ablation experiments were used, and the experimental data and hyperparameters were the same as in the above experiments. The ablation experiment is an experiment, which only compares the improved part, and the others remain unchanged. The methods of the experiment were as follows: CBAM was not used in the 'non-CBAM' experiment; FPN was not used in the 'non-FPN' experiment, but we only used the last layer of features to extract the ROI; deformable convolution was not used in the 'non-DC' experiment; the pretrained model was used to extract the ROI, but the ROI extraction module was not finetuned in the 'non-FT' experiment; in the 'IOU-DML' experiment, the module of the ROI extraction combined the DML structure of RepMet. The results of the ablation experiment are given in Table 2.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **mAP of 5-Way 1-Shot** & **mAP of 5-Way 5-Shot** \\ \hline Matching Network & 43.56\% & 55.31\% \\ Prototypical Network & 49.42\% & 68.20\% \\ MAML & 48.70\% & 63.11\% \\ Relation Network & 50.44\% & 65.32\% \\ RepMet & 56.90\% & 65.80\% \\ MBSS & 65.40\% & 81.76\% \\ Ours & 58.70\% & 73.42\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: The mAP of different methods on the miniImageNet dataset.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **mAP of 5-Way 1-Shot** & **mAP of 5-Way 5-Shot** \\ \hline non-FPN & 31.45\% & 40.27\% \\ non-DC & 48.53\% & 61.01\% \\ non-CBAM & 58.02\% & 71.72\% \\ non-FT & 57.13\% & 69.84\% \\ IOU-DML & 56.98\% & 69.21\% \\ Ours & 58.70\% & 73.42\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: MiniImageNet dataset for the mAP results of the ablation experiments.  
Figure 6: The training loss of the different methods.  
By comparing Tables 1 and 2, the results can be summarized as follows: the mAP of our method was low when the FPN and DC were not used; when CBAM was not used, the ROI module was finetuned, and the loss function was not used in the module, or if the ROI module was replaced with the DML network in RepMet, then the mAPs of these experiments, as shown in Table 2, were slightly higher than RepMet in Table 1. In addition, it can be seen that the FPN and DC directly influenced the mAP of our method, and when the ROI, loss function, and CBAM were added, the performance of the network was enhanced.