# Remote Sensing Road Extraction by Road Segmentation Network
## 2 Materials and Methods
### The Proposed Method
#### 2.2.3 Refinement Step  
The outputs from the segmentation network are coarse and rough, which will lead to error in road segments caused by the interference information. The tensor voting algorithm and the conditional random field (CRF) are widely used as post-processing to refine the results from segmentation network. However, the former is unstable due to the parameter-dependence, while the latter model is overly complex. A designed refinement network is exploited to refine the results of the segmentation network. The difference is that long-distance dependence of the road is also considered here. The refined network produces more stable road maps and is embedded in the overall network for end-to-end training.  
The designed refinement network is shown in Figure 3. The input of DRN is the coarse map, which has the width and height of W and H, respectively, and one channel. The output of DRN is the refined map with the same size. There are three corresponding layers in DRN. In the encoder, the number of channels is changed to 1, 16, 32, and 64 in turn, and the sizes of the features are sub-sampled as 1/2, 1/4, and 1/8. Due to the characteristic of long-distance dependence on the road, maintaining connectivity in the extracted results is challenging. To capture long-distance dependence between different locations in features, the strip pooling [15] that averages all values in a row or column feature is embedded  
Figure 2: Attention mechanism (AM). The features of adjacent layers are first up-sampled to the same size, and are then passed through a series of convolution operations to obtain the attention weight map, \(w\), and the refined features, \(\hat{F}\).  
at the high layer features of 64 channels. It allows the distribution of roads in scattered areas to be connected on the feature map. To capture local context information, two spatial pooling layers are used to collect short distance dependencies. The output features of two pooling layers are concatenated together, and then \(1\times 1\) convolution operations are used to change the channel numbers. Finally, the decoder recovers the characteristic resolution and obtains a refined map of the road.  
Road extraction is a pixel-level recognition and judgment; however, there is a great imbalance between road pixels and non-road pixels in the image. Thus, the constraint on road pixels in the loss function is of great importance. As a binary segmentation task, road extraction most commonly uses the binary cross entropy (bce) loss function. Inspired by [16], the bce is given a dynamic weight probability, which is set by the frequency of the road pixel. Suppose the batch size is represented as \(B\), and the size of the image is \(H\times W\), then the weight \(\alpha\) is calculated as follows:  
\[\alpha=\frac{sum(y_{n})}{BHW},\]  
where \(y_{n}\) is the ground truth of the i-th batch, and \(sum(\cdot)\) represents the statistics of the number for road pixels. Thus, the bce loss is modified to be  
\[L_{bce}=-\frac{1}{B}[(1-\alpha)y_{n}\log a_{n}+\alpha(1-y_{n})\log(1-a_{n})],\]  
\[a_{n}=\frac{1}{1+\exp(-f_{w}(x_{n}))}\]  
where \(a_{n}\) is the sigmoid value in the end of model, \(f_{w}(x_{n})\) is the output of the model for \(x_{n}\), and \(y_{n}\) is the ground truth for \(x_{n}\), which are binary maps. In order to ensure the loss function still has the ability to constrain the road after multiple iterations, the dice loss is used to measure the similarity of prediction and ground truth:  
\[L_{dice}=\frac{\sum_{n=1}^{B}y_{n}f_{w}(x_{n})}{\sum_{n=1}^{B}(y_{n}+f_{w}(x_{n }))}.\]  
The final loss function used in this article is a combination of \(L_{bce}\) and \(L_{dice}\) to alleviate the performance problems caused by sample imbalance in road extraction.