# Supervised Learning of Natural-Terrain Traversability with Synthetic 3D Laser Scans
## 4 Validating Traversability Classifiers  
Five synthetic 3D point clouds, one per each zone of the natural environment, are employed for validation purposes exclusively. This data contains a total of 397,426 points where 385,959 have at least five neighbours. This validation data is also unbalanced with \(\sim\)68% of points in the traversable class. For all the classifiers, the prediction time for an average synthetic scan is almost negligible with respect to its preprocessing time with the exception of the KNN estimator that requires 0.2 s. Table 2 contains the components of the confusion matrix of each trained classifier, where TP, FP, TN and FN stand for the number of true positives, false positives, true negatives and false negatives, respectively. True refers to points classified correctly and false to the opposite, whereas positive refers to the non-traversable class and negative to the traversable class. To compare the performance of the binary classifiers, five accuracy indices for imbalanced data, computed by _Scikit-learn_ functions ([[https://scikit-learn.org/stable/_downloads/scikit-learn-docs.pdf](https://scikit-learn.org/stable/_downloads/scikit-learn-docs.pdf)]([https://scikit-learn.org/stable/_downloads/scikit-learn-docs.pdf](https://scikit-learn.org/stable/_downloads/scikit-learn-docs.pdf))), are considered. The precision (PR), the recall (RE) and the F1 scores are the first, the second and the third indices, respectively:  
\[PR=\frac{TP}{TP+FP},\quad RE=\frac{TP}{TP+FN},\quad F1=2\ \frac{PR\cdot RE}{ PR+RE}. \tag{1}\]  
The fourth index is the balanced accuracy score (BA): \begin{table}
\begin{tabular}{l l r} \hline \hline
**Estimator** & **Acronym** & **Time (s)** \\ \hline Decision Trees & DT & 3.3 \\ Gaussian Naive Bayes & GNB & 0.1 \\ K-Nearest Neighbors & KNN & 1.1 \\ Linear Support Vector Machine & LSVM & 136.3 \\ Bagged Decision Trees & BDT & 21.0 \\ Random Forest & RF & 8.1 \\ Gradient Boosted Trees & GBT & 41.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Training times for traversability classification. \[BA=\frac{1}{2}\left(RE+\frac{TN}{TN+FP}\right). \tag{2}\]  
All the above indices vary between 0 and 1 for the worst and the best classification results, respectively. The last index is the Matthews correlation coefficient (MC):  
\[MC=\frac{TP\cdot TN-FP\cdot FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}, \tag{3}\]  
that ranges from \(-1\) to 1, where \(-1\) indicates an inverse classification, 0 a random prediction and 1 a perfect prediction. Table 3 includes the five accuracy indices for every estimator. In general, high accuracy is achieved, but the best performance comes from the RF and GBT classifiers and the least from the GNB and LSVM estimators. Figures 4-8 illustrate the results of applying the RF classifier to the five validation scans. Excellent estimations of both classes can be observed in red and green colours. Blue colour represents 3D points that have not been classified due to the lack of neighbours. These unclassified points are usually located far from Andabata, where scan density decreases. \begin{table}
\begin{tabular}{l r r r r} \hline \hline
**Estimator** & **TP** & **TN** & **FP** & **FN** \\ \hline Decision Trees & 111,912 & 252,446 & 10,007 & 11,594 \\ Gaussian Naive Bayes & 89,890 & 259,351 & 3102 & 33,616 \\ K-Nearest Neighbors & 111,538 & 256,915 & 5538 & 11,968 \\ Linear Support Vector Machine & 97,559 & 21,8384 & 44,069 & 25,947 \\ Bagged Decision Trees & 113,111 & 254,622 & 7831 & 10,395 \\ Random Forest & 112,547 & 257,396 & 5057 & 10,959 \\ Gradient Boosted Trees & 111,335 & 258,922 & 3531 & 12,171 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Components of the confusion matrices for synthetic data. Figure 4: The validation scan on the hills (**a**) and the prediction results with RF (**b**).