# RT-GAN: GAN Based Architecture for Precise Segmentation of Railway Tracks
## 3 Method
### Rail Track Segmentation Network  
As shown in Figure 1, an encoder-decoder network termed RT-seg is designed as the generator. RT-seg is inspired by [14] and uses a linear span unit (LSU)-based feature pyramid network, termed LSP, to concatenate feature maps with different resolutions. First, the CSPDarknet53 [28] is selected as the backbone network, and its structure is shown as Figure 2. It mainly consists of 5 cross stages. In each stage, the input features are downsampled (except Stage 1) and then split into two parts according to the channels. One part passes through a set of \"Dark block\", and the other part does nothing. Finally, the two parts are concatenated as the output of the stage. The outputs of 5 stages (\(P_{1}\), \(P_{2}\), \(P_{3}\), \(P_{4}\) and \(P_{5}\)) are selected as the output feature maps of the backbone with strides of 1, 2, 4, 8 and 16, respectively. \[P_{1},P_{2},P_{3},P_{4},P_{5}=Backbone(I). \tag{4}\]  
Figure 1: Network structure of RT-GAN. Then, these feature maps are passed through the LSP to obtain an optimised combination of features with different resolutions. In contrast to the basic feature pyramid network, the LSP employs LSUs instead of simple convolution operations. Each LSU accepts and accumulates feature maps from the backbone network and LSUs at deeper levels. The number of channels and resolution of its output are aligned to the channels and resolution of the current stage by \(1\times 1\) convolution and upsampling, respectively. An additional LSU is used to accept the output of the LSP and the output fused features to further extend the feature sum-space. Among them, to mitigate the noise caused by upsampling and downsampling, the stride of the fused features is chosen to be 4, and the features of other levels are resolution aligned by downsampling or upsampling,  
\[U_{1},U_{2},U_{3},U_{4},U_{5}=LSP(P_{1},P_{2},P_{3},P_{4},P_{5}), \tag{5}\]  
\[U_{fuse}=LSU\left(U_{1}^{i4},U_{2}^{i2},U_{3},U_{4}^{i2},U_{5}^{i4}\right). \tag{6}\]  
Finally, the fused features \(U_{fuse}\) are upsampled, convolved, and SoftMaxed to output a pixel-level semantic segmentation image of the same size as the input. \[C_{cls}=Softmax(\sigma\left(U_{fuse}^{\dagger 4}\right)). \tag{7}\]  
In the network, the LSU is a configurable subnetwork proposed by [PERSON] [14] et al. Its structure is shown in Figure 3. If the feature map is considered a node and the layer operations are considered edges, the subnetwork of LSU can be considered a directed acyclic graph. This directed acyclic graph consists of \(K\) input nodes, \(P\) intermediate nodes and 1 output node. For multiple input nodes, LSU first merges all the input feature maps by the \"concatenate\", \"\(1\times 1\) convolution\" and \"ReLU\" operations. Then, a series of multi-layer combinations of feature transformations are performed, including \"skip\", \"\(3\times 3\) convolution\", \"\(5\times 5\) convolution\", \"dilated convolution\", etc. Finally, all the intermediate nodes are connected to the output node by the \"summing\" operation, and an expanded feature map is output that combines all the transformed features. Suppose the input node is \(x_{i,\ \ i=1...k}\), the intermediate node is \(n_{i,\ \ i=1...p}\), and the output node is \(y\). The set of available transformation operations between nodes is denoted \(\Lambda\). For an ordered pair of intermediate nodes \(<n_{i},n_{j}>\), the transformation operation between them is \(\rho_{i,j}\in\Lambda\), i.e.,  
\[n_{i}=\rho_{i,j}(n_{j}). \tag{8}\]  
Then, the LSU can be represented as follows:  
\[y=\sum_{i=1}^{p}n_{i}=\sum_{i=1}^{p}(\prod_{j=1}^{M_{i}}\rho_{i_{j-1},i_{j}})( \sigma(C_{1}(\{x_{1},\ldots,x_{k}\}))), \tag{9}\]  
Figure 2: Network structure of CSPDarknet53.