# A Vision-Based Two-Stage Framework for Inferring Physical Properties of the Terrain
## 2 Proposed Method
### First Stage: From RGB Image to the Terrain Type
Average pooling and maximum pooling are used to obtain the common features and the most significant features of the corresponding region, respectively. The information obtained by the three parallel pipelines is more abundant than the original single pipeline. Differently from the filter parameters in the first layer of ResNet, we use a smaller filter size which is 3 and a larger number of filters which is 68 in the downsampling operation. As shown in Equation (1), we obtain three sets of feature maps with larger height and width and more channels after the parallel operation. Finally, we concatenate them in the channel direction to fuse features and output this large feature map to the second layer of the network. Our parallel feature extraction structure is used to improve the network's ability to absorb rich information from the initial high-resolution image and provide the possibility for subsequent layers of effectively learning the details of the terrain image. Figure 4: In the local image, without context clues, an overexposed rock and ice have are extremely similar in appearance. Besides, we add long skip connections between blocks of ResNet to enhance the network's ability to learn fine texture features. With the repeated downsampling of the feature map layer by layer, tiny spatial details and fine structures are gradually lost in the deep layers. Therefore, we should pay attention to the important role of low-level fine details retained in the shallow layer. At the same time, considering that skip connection can enhance feature propagation in deep networks, we use skip connections to propagate fine features in shallow layers into deep layers. The specific modification of ResNet is shown in Figure 6. We divide the 33 basic residual blocks in the original ResNet into 4 groups according to conv2_x, conv3_x, conv4_x and conv5_x and rename them block 1, block 2, block 3 and block 4. We use long skip connections to pass the feature maps output from block 1 to each block behind it as input, and perform the same operation on other blocks. As a result, the subsequent layers, especially the deep layers inside and behind block 4, can obtain the fine features in the shallow layers by receiving the feature maps from the shallow blocks. To enable the network to obtain more contextual information, we insert a pyramid pooling module. Contextual information refers to the information regarding the surrounding environment--a certain type of terrain in an image, including the coexistence of this type and other types in the same image, spatial location relationship, etc. The network with context prior has a strong ability to parse scenes. Sometimes scene information is the key to distinguishing targets with similar appearances. As shown in Figure 7, the images of two kinds of terrain have similar appearances in a small area. If the network only infers the terrain type based on local small area, it will inevitably have poor accuracy. However, if the network can take into account the contextual information of the area to be inferred, it will make a more reasonable inference based on the local and global information. The example shows the superiority of using contextual information for inference. Figure 5: Parallel feature extraction structure. Figure 6: Long skip connections between blocks (red curves). We insert the pyramid pooling module behind the downsampling part of the network. As shown in Figure 8, in the pooling operation of each layer in the pyramid, the filter sizes are 64, 32, 16 and 8, respectively. As shown in Equation (2),  
\[ \begin{split} n_{i}&=\left\lfloor\frac{n_{i-1}{+}2p{-} f}{s}{+}1\right\rfloor\\ j_{i}&=j_{i-1}{\times}s\\ r_{i}&=r_{i-1}{+}\left(f{-}1\right){\times}j_{i-1} \\ start_{i}&=start_{i-1}{+}\left(\frac{f{-}1}{2}{-}p \right){\times}j_{i-1}\end{split}  \tag{2}\]  
where \(j\) is the distance between two adjacent features, \(start\) is the center coordinate of the upper left feature and \(r\) is the receptive field size.