# Semantic Structure from Motion for Railroad Bridges Using Deep Learning
## 2 Background
### Semantic Segmentation Using Deep Learning  
Semantic segmentation is a method of predicting and displaying the semantic information of an image by segmenting an image into pixels using a CNN. The general structure of a CNN is shown in Figure 1. The CNN extracts a convolution feature using the convolution layer and pooling layer alternately from the image (input data) and classifies the extracted features using a fully connected layer.  
In this study, semantic segmentation was carried out using Deeplab-V3+. The structure of the Deeplab-V3+ model is shown in Figure 2. Deeplab-V3+ separates the structure into an encoder and decoder to overcome the disadvantage of the CNN, which loses location information owing to the loss of dimensions while passing the fully connected layer. The encoder extracts features at random resolution through atrous convolution from a deep convolutional neural network (DCNN). The output stride, which refers to the ratio of the resolution of the input image to the resolution of the output image, was applied. The decoder reduces the channel by performing a \(1\times 1\) convolution on the final output image of the encoder and performs concat after a bilinear upsample. Through the above process, the decoder efficiently maintains the object segmentation details [25].  
Deeplab-V3+ has the advantage of using a pretrained deep learning model. In this study, ResNet-50 was used as a module together with Deeplab-V3+. ResNet-50 is a module using the proposed residual learning to address the gradient loss. In this phenomenon, a variable disappears as it passes a layer. The structure of the existing convolution layer and residual learning is shown in Figure 3.  
Figure 1: Structure of the CNN.  
Figure 2: Structure of Deeplab V3+.  
Residual learning is the addition of a residual connection (or skip connection) to a prior convolution layer. It is composed of the addition of an input to the stack of two convolution layers. Figure 3a describes the structure of the existing CNN. Back-propagation training is performed to obtain a weight that minimizes the difference between the predicted value (\(H(x)\)) of the network and target value (label) of the training data. The predicted value \(H(x)\) refers to the result of learning the network through data. The target value refers to a label matching the training data. Figure 3b shows the structure of the residual learning block of ResNet-50. The purpose of learning is \(F(x)\) becoming 0, considering the hypothesis that it is better to approximate \(F(x)+x\) to \(H(x)\) than to approximate \(H(x)\) as a function of a complex nonlinear structure. In other words, when \(H(x)\simeq F(x)+x\), it changes to \(F(x)\simeq H(x)-x\), and eventually learning F(x) is the output factor \(H(x)\) and the input factor \(x\) is to learn the difference. Therefore, it can be considered that the residual that is the remainder is learned. Finally, the meaning of learning the optimal \(F(x)\) is the same as that of the convergence of \(F(x)\) to 0; as \(x\) becomes a residual connection (or skip connection), there is no increase in the amount of computation [26].