# A Vision-Based Two-Stage Framework for Inferring Physical Properties of the Terrain
## 3 Experiment
### Vision-Based Two-Stage Framework Experiment  
Combined with the decision trees established in the previous section, we tested the vision-based two-stage framework proposed for estimating the terrain's physical properties in the field.  
As shown in Figure 15, we visualized the experimental results of the framework. The figure shows the inferences in the test set. In the results of friction property inference, the brighter the color is, the lower the friction level is, that is, the more slippery it is. In the same way, in the results of stiffness property inference, the brighter the color is, the lower the stiffness level is. The values of friction coefficient and stiffness corresponding to friction levels and stiffness levels are shown in Tables 5 and 6.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Network** & **mIoU (Train)** & **mIoU (Test)** \\ \hline FCN & 52.31\% & 49.64\% \\ SegNet & 50.32\% & 47.45\% \\ DeepLabv3 & 63.84\% & 60.24\% \\ TerrainNet & 66.52\% & 62.94\% \\ \hline \hline \end{tabular}
\end{table}
Table 10: Comparative experiments with other networks on FITI.  
Figure 14: Comparison of semantic segmentation between ResNet-101 and TerrainNet. The second row is the prediction output by ResNet-101, and the third row is the prediction output by TerrainNet. The segmentation effect of the third row is better than that of the second row.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Terrain** & **Grassland** & **Land** & **Rock** & **Ice** & **Water** & **Asphalt Road** \\ \hline \(IoU\) & 79.10\% & 63.74\% & 65.24\% & 57.56\% & 53.38\% & 64.02\% \\ \hline \hline \end{tabular}
\end{table}
Table 9: \(IoU\) of each terrain.  
It can be seen from the figure that the framework has good inference performance. The framework can correctly infer the friction level of the dirty ice's surface and the clean ice's surface. The physical properties of the land in different states can also be well inferred; for example, the pixel brightness of the ice's surface is \(\sim 10\) (\(\sim 10\)). The model predicts the correct prediction of the friction level of the dirty ice's surface.  
Figure 15: The experimental results of the complete framework proposed in this paper. The first and sixth rows are the input RGB images; the second and seventh rows are the inference results of friction property; the third and eighth rows are the ground truths of friction property; the fourth and ninth rows are the inference results of stiffness property; and the fifth and tenth rows are the ground truths of stiffness property. For better visualization, we mapped the inferential results of physical properties level to a more easily observed gray-scale image. The corresponding relationship between friction level and grayscale is shown in the rightmost column, and so is the stiffness level.  
the stiffness inference results of deciduous or hay-covered land is higher than that of the ordinary land. The friction levels of dry asphalt road and wet asphalt road can also be well distinguished. The regions with different friction properties on the rock surface can also be well segmented.  
We tested the accuracy of our vision-based two-stage framework quantitatively. The physical properties of all images in the test set were estimated by our two-stage framework, and the ground truths of physical properties were used to quantitatively evaluate the inference results. We used \(mIoU\) and pixel accuracy (\(PA\)) as the metrics to evaluate the accuracy of our framework. The calculation of \(PA\) is shown in Equation (10).  
\[PA=\frac{\sum_{i=0}^{m}p_{ii}}{\sum_{i=0}^{m}\sum_{j=0}^{m}p_{ij}} \tag{10}\]  
where the symbols have the same meaning as in Equation (9).  
The experimental results can be seen in Table 11. The \(mIoU\) of our framework was 60.18% in the inference of friction property and 61.21% in the inference of stiffness property. The PA of our framework was 75.85% in the inference of friction property and 76.63% in the inference of stiffness property.