# An Improved Feature Pyramid Network and Metric Learning Approach for Rail Surface Defect Detection
## 3 Main Results
### Defect Classification and Identification  
Based on RepMet, a multimodal network was established to extract the feature information of the ROI, obtain different feature vectors, and distinguish the defects by measuring the distances between the feature vectors of the mode and the feature vectors of the DML. The input of RepMet was a feature map with a fixed-size type of ROI, which was extracted by the improved FPN. Then, the convolution layer was used to further extract the ROI features to better distinguish different categories and ensure that the features extracted from the same types were uniform. The process of extraction can be defined as  
\[F_{i}=f(x_{i}),\ \ i\in O, \tag{7}\]  
where \(f\) is the convolution operation, \(x_{i}\) is the feature map of the extracted ROI, \(O\) is the set of the ROI, and \(F_{i}\) is the feature map after convolution processing. Figure 2 shows the structure of the DML embedded module; the feature vector module \(E\) is composed of three FC layers, and each layer is followed by a ReLU to use nonlinear processing, where \(FC_{v1}=512d\), \(FC_{v2}=256d\), \(FC_{v3}=128d\), and \(FC_{vt},t\in\{1,2,3\}\) denotes the \(t\)-th FC layer of the feature vector module. Finally, the ROI was dealt by the feature vector module, and the feature vector \(E_{i}=E(F_{i})\) was obtained to extract the common characteristics of all feature information. A multimodal network is shown in Figure 2, which extracted the same and different information using the same and different categories, respectively. Similarly, the feature map \(F_{i}\) was input to the high-dimensional FC layer denoted by \(FC_{m}\) for nonlinear processing. In order to extract richer feature information, the FC layer was set to \(FC_{m}=1024d\), which was convenient to use and learn the feature information of each mode. In each modal network, three FC layers with ReLU were used, where \(FC_{n1}=512d\), \(FC_{n2}=256d\), \(FC_{n3}=128d\), \(n\in\{1,\ldots,N\}\), \(N\) denotes the number of classes, and \(FC_{ns},s\in\{1,2,3\}\) denotes the \(s\)-th FC layer of each mode network. After the nonlinear processing, the feature vector \(E_{ij}\) of the multimodal network was obtained, where the distance between \(E_{i}\) and \(E_{ij}\) is defined as  
\[D_{j}(E_{i})=D(E_{i},E_{ij})=\sqrt{\sum_{i=1}^{N}{(E_{i}-E_{ij})^{2}}},j=1, \cdots,K, \tag{8}\]  
where \(K\) is the number of modes. All class distributions are assumed to be mixtures of isotropic multivariable Gaussian distributions; \(D_{j}(E_{i})\) is also used to calculate the extracted probabilities of \(j\)-th class in \(j\)-th mode, which can be defined as  
\[P_{j}(E_{i})\propto\exp\Bigl{(}-(D_{j}^{2}(E_{i}))/(2\sigma^{2})\Bigr{)}, \tag{9}\]  
where \(\sigma^{2}\) is the variance. By Equations (8) and (9), the posterior probability of the defective category in the ROI can be described as  
\[P(C|X)=P(C|G)=\min_{j=1,\ldots,K}P_{j}(E_{i}), \tag{10}\]  
where \(C\) denotes the \(i\)-th class, its minimum value is the minimum distance of all modal calculations, \(G\) is the mixture coefficient, and \(X\) is the covariance of the modes. Aftercalculating the posterior probability of the defective category in the ROI, the posterior probability of the background category needed to be further calculated. The foreground probability was used to calculate the background probability, defined as  
\[P(B|X)=P(B|G)=1-\min_{j=1,\ldots,K}P_{j}(E_{i}), \tag{11}\]  
where \(B\) is the background category. Next, the embedded loss function \(L_{em}\) and cross-entropy loss function \(L_{CE}\) were used in the loss function of classification and recognition, where \(L_{em}\) ensures that the distance is small between the class and the correct mode and is large between the class and the incorrect mode. \(L_{em}\) is defined as  
\[L_{em}=ReLU(\min_{j}D_{j}(E_{i})-\min_{j,i\
eq i^{*}}D_{j}(E_{i})+\alpha), \tag{12}\]  
where \(i^{*}\) is the label of the correct class, \(\alpha\) is the error between the nearest distance from \(E_{i}\) to the correct class and \(E_{i}\) to the error class.