# RT-GAN: GAN Based Architecture for Precise Segmentation of Railway Tracks
## 3 Method
### Rail Track Segmentation Network
where \(\sigma(\bullet)\) denotes the ReLU activation function, \(\mathcal{C}_{1}(\bullet)\) denotes the \(1\times 1\) convolution operation, \(\{\bullet\}\) denotes the concatenate operation, \(M_{i}\) denotes the number of operations experienced from the \(1\)st intermediate node \(n_{1}\) to the i-th intermediate node \(n_{i}\), and \(\prod_{j=1}^{M_{i}}\rho_{i(j-1),i(j)}\) denotes all transform operations experienced from \(n_{1}\) to \(n_{i}\). LSU encourages multiple paths for feature transformation and does not have a fixed network structure. Some efficient network structures are obtained by a heuristic search in the literature [14], and the ALSN-Large structure is chosen after experimental validation. The structures of LSUs in the ALSN-Large structure are shown in \[P_{1},P_{2},P_{3},P_{4},P_{5}=Backbone(I). \tag{4}\]0 The concatenate and channel alignment operations are omitted in the figure. The \(ConvX\_Y\) symbol indicates the convolution operation with kernel size \(X\) and dilation factor \(Y\). Among them, \(LSU1\) and \(LSU2\) are actually only simple direct connections; \(LSU3\), \(LSU4\), and \(LSU5\) use large size convolution kernels and shallow-wide structures; \(LSU10\), which fuses all resolution feature maps, uses a deep-narrow structure with multiple convolutional layers with increasing dilation factors. Image Segmentation Loss  
The loss of RT-seg is divided into 3 parts: category loss, gradient loss, and side output loss. **Category loss. ** For a binary classification task, the general approach is to measure the distance between the predicted result \(C_{cls}\) and the true classification \(C_{gt}\) by calculating  
\[P_{1},P_{2},P_{3},P_{4},P_{5}=Backbone(I). \tag{4}\]1 The LSUs in the ALSN-Large structure. \[P_{1},P_{2},P_{3},P_{4},P_{5}=Backbone(I). \tag{4}\]2 Network structure of linear span unit (LSU). the sum of the binary cross-entropy of each pixel. Considering the diversity of targets in the track images, it is a better choice to use the cross-entropy of multiple classifications to calculate the loss. Due to the unbalanced number of track regions and backgrounds, the focal loss is used to balance the weight of each pixel as Equation (10). \[P_{1},P_{2},P_{3},P_{4},P_{5}=Backbone(I). \tag{4}\]3  
where \(X\) is the predicted result \(C_{cls},\;\;FC(\bullet)\) denotes the focal Loss function, and \(\alpha=0.5\) and \(\gamma=2\) are the hyperparameters. **Gradient loss. ** Since railway tracks are linearly connected regions with distinct edges on the image, explicitly incorporating gradient variations into the loss helps to guide the generation of segmentation results with clear boundaries. Therefore, the root-mean-square loss function with mask is used to calculate the gradient loss as Equation (11). \[P_{1},P_{2},P_{3},P_{4},P_{5}=Backbone(I). \tag{4}\]4  
where \(X\) is the predicted result \(C_{cls}\), \(Y\) is the ground truth \(C_{gt},\;S_{x}= \begin{bmatrix}1&2&1\\ 0&0&0\\ -1&-2&-1\end{bmatrix} \), and \(S_{y}= \begin{bmatrix}1&0&-1\\ 2&0&-2\\ 1&0&-1\end{bmatrix} \) are gradient operators. \(M\) is the mask of the region of interest, which is used to remove the effects of image augmentation such as masking and cropping. \(\mathbb{E}[\cdot]\) means to calculate the arithmetic mean. **Side output loss. ** The side output loss refers to the difference between the middle feature layer of a DNN and the ground truth. The side output loss can supervise the changes of intermediate feature layers and guide the DNN to converge faster and is widely used in linear target segmentation tasks such as edge detection [12] and skeleton detection [13,14]. Different feature layers of LSP are used as input, and the focal function is used to calculate the loss as Equation (12). \[L_{side}=-\frac{1}{K}\sum_{k=1}^{K}FC(X_{side,k}). \tag{12}\]  
where \(X_{side,k}=Softmax(C_{1}(U_{k}))\) is the \(k\)-th side output, which achieves channel alignment with the number of target categories through \(1\times 1\) convolution. \(U_{k},k=1\ldots 5\) is the output of LSP.