# A Multi-Scale Contextual Information Enhancement Network for Crack Segmentation
## 3 Proposed Method
### Multi-Scale Skip Connection Structure  
The encoder network of U-Net has a distinct hierarchical character, and different stages of its convolutional layers obtain diverse meaningful features. The shallow layers keep abundant structure information, while the deep layers obtain more abstract features which play a crucial role in object recognition. Due to the loss of spatial information caused by the down-sampling operation in the coding process, it is not enough to only use deep coarse feature maps to obtain fine segmentation results. Specifically, in the application of crack segmentation, the slender and tortuous structural characteristics of the cracks place high demands on the edge prediction capability of the model. In order to better restore the spatial information of cracks, we propose a multi-scale skip connection method to connect the encoder and decoder of U-Net. By feeding multi-scale feature maps to the decoder through the multi-scale skip connection, the network can better capture fine-grained spatial features.  
Figure 1: An illustration of the MCIE-Net architecture.  
Figure 2 shows four different skip connection structures. Among them, the plain skip connection and the densely connected skip connection can only deliver information at the same level; thus, the utilization of multi-level feature information generated in the encoder is relatively limited. In contrast, our multi-scale skip connection can provide the decoder with multiple feature maps generated in different layers of encoder network, enabling each layer of the decoder to learn rich fine-grained structural features. Compared with the full-scale skip connection, our multi-scale skip connection has the following two improvements. First, the multi-scale skip connection does not build the intra-connection between the decoder layers. The deep feature maps in the decoder do not contain spatial information, and the semantic information they contain can be effectively utilized through the backbone network; thus, there is no need to build additional connection paths. In addition, after concatenating the feature maps from different sources, we adjust the number of channels of the feature maps to be consistent with the same layer of feature maps in the encoder. The experiments in Section 4.5.1 demonstrate that the two improvements in this paper can significantly reduce the number of network parameters and lower the computational cost compared with full-scale skip connection.  
As an example, Figure 3 illustrates how to construct the feature map \(X_{De}^{4}\) (the fourth layer of the decoder). The decoder receives the feature map \(X_{En}^{4}\) from the same encoder layer and \(X_{De}^{5}\) from a deeper decoder layer; meanwhile, the feature maps of \(X_{En}^{1}\), \(X_{En}^{2}\), and \(X_{En}^{3}\), generated in several shallower layers of the encoder, are also transmitted through the skip connection paths. Furthermore, while the plain skip connection directly concatenates feature maps from different sources, we unify the number of channels of each feature map before concatenating, so that the superfluous information can be reduced [29]. Lastly, a feature aggregation mechanism, which consists of a convolution operation, a batch normalization, and a ReLU activation function, is used to seamlessly merge the abundant information owned by the concatenated feature map.  
Figure 2: Comparison of different architectures of skip connections. **(a)** Plain skip connection, **(b)** densely connected skip connection, **(c)** full-scale skip connection, and **(d)** multi-scale skip connection.