# A Vision-Based Two-Stage Framework for Inferring Physical Properties of the Terrain
## 3 Experiment
### TerrainNet Experiment  
After building the TerrainNet based on Pytorch, we used FITI to train and test it on our workstation. For a practical deep learning system, the devil is always in the details. We carefully set the hyperparameters. Batch-size was set to 4 in the mini-batch gradient descent. \(\beta_{1}\) was set to 0.9, \(\beta_{2}\) was set to 0.999 and \(\epsilon\) was set to \(10^{-8}\) in the Adam optimization algorithm. We trained a total of 200 epochs, and finally decided that 150 was the best epoch number. The weight decay parameter \(\lambda\) in L2 regularization was set to \(2\times 10^{-4}\). The initial learning rate was set to \(5\times 10^{-4}\), the gamma value was set to 0.5 and the learning rate decays once every 50 epochs. The input image had a resolution of 2048 \(\times\) 1024. The configuration of the workstation was as follows: two NVIDIA TITAN Xp GPUs with 11G memory respectively; single Intel (R) Core (TM) i9-7940X CPU with 16G memory.  
The performance of our network on the training and test sets is shown in Figure 12. It can be seen from the figure that the mean of class-wise intersection over union (\(mIoU\), Equation (9)) and loss value gradually converge.  
\[mIoU=\frac{1}{m+1}\sum_{i=0}^{m}\frac{p_{ii}}{\sum_{j=0}^{m}p_{ij}+\sum_{j=0}^{ m}p_{ji}-p_{ii}} \tag{9}\]  
where \(m+1\) is the total number of terrain types and \(p_{ij}\) is the number of pixels of class \(i\) inferred to belong to class \(j\). That is to say, \(p_{ii}\) represents the number of true positives, while \(p_{ij}\) and \(p_{ji}\) are the false positives and false negatives respectively. The final \(mIoU\) accuracy of the network on the training set was 66.52% and the accuracy on the test set was 62.94%.  
To better evaluate our network and analyze the contributions of network components in the final accuracy, we carried out ablation experiments with several settings, including raw ResNet-101, ResNet-101 with a parallel feature extraction structure (network 1), ResNet-101 with long skip connections between blocks (network 2) and ResNet-101 with a pyramid pooling module (network 3). Moreover, the hyperparameters of them are consistent with those of TerrainNet. The experimental results are shown in Figure 13 and Table 8. Figure 13 shows the iterative curves of the networks on the training set, and Table 8 shows the final \(mIoU\) of the networks on the training set and the test set.  
We calculated the \(IoU\) of each terrain to determine which terrain was the most difficult to identify. The results are shown in Table 9. We can see that the \(IoU\)s of ice and water were relatively low. This may have been due to the fact that both of them belonged to non-Lambert surfaces.  
We also compared TerrainNet with FCN, SegNet and DeepLabv3 on FITI, as shown in Table 10. It can be seen that TerrainNet had the best accuracy on FITI. TerrainNet outperformed FCN and SegNet by a clear margin. Although the \(mIoU\) of DeepLabv3 on the training set was slightly lower than that of TerrainNet, the accuracy on the test set was far worse. This shows that TerrainNet is better at segmenting terrains than these networks used to segmenting things.