# Hourglass 3D CNN for Stereo Disparity Estimation for Mobile Robots
## 4 Experiments
### Evaluation Results and Discussion  
Table 2 provides the performance of the evaluated methods in terms of 3-px error (for KITTI datasets), as well as their computational cost and real-time ability (for Scene Flow dataset), while Table 3 compares the EPE (on the Scene-Flow dataset) with that of real-time models. Overall, according to Table 2, we can observe a common tendency. The higher the accuracy, the more computational cost is required. In particular, DeepPrunner-fast [20] and PSMNet [10] produced impressively accurate results in terms of 3-px error. Those values were all less than 3%. As a result, those FLOPs and running times are the most severe. For example, DeepPruner-fast's FLOPs is 208.52 G, whereas its running time is 210 ms. From the table, we can also find the lowest accuracy in AAFS and StereoNet. However, their FLOPs and running time are relatively small. For example, StereoNet's FLOP is 59.82 G, and its running time is 180 ms. As shown in Table 3, the proposed method significantly reduced EPE by 0.7 on AAFS [12] and 0.4 on StereoNet-16x [13] on the Scene-Flow test set. In general, our proposed method is remarkably faster than the other baselines because the disparity was computed at low resolution. Specifically, our proposal was 7.5 times faster than StereoNet-16x [13] and 14 times faster than DeepPruner-fast [20]. The proposed method outperforms StereoNet-16x [13] by a considerable margin, with only 1 billion FLOPs, clearly proving that the proposed method can accurately estimate disparity maps in real time in mobile robots. Figure 5 compares the results acquired from the KITTI 2012 and 2015 leaderboard for PSMNet [10] and AAFS [12]. In KITTI 2012 (in Figure 5a), the error map scales linearly from 0 to >=5 pixel error; red color represents all occluded pixels that are outside the image limits. In KITTI 2015 (in Figure 5b), the error map depicts correct estimates; <3 px or <5% error are indicated in blue, and incorrect estimates are indicated in red, whereas dark regions denote the occluded pixels that lie outside the image limits. Through error maps, PSMNet [10] achieved the best results, and our proposed method improved the accuracy at the boundary of the object compared to AAFS [12]. Beyond the boundary areas, our proposed method and AAFS [12] have the same accuracy as PSMNet [10], which shows that our proposed method can accurately estimate disparity maps for real-world applications while requiring a low computational cost. **Benchmark Results** For benchmarking, KITTI 2012/2015 test sets were used for evaluation. As shown by the online leaderboard in Table 4, the overall thee-pixel error of the proposed model was 5.64% on KITTI 2012 and 6.14% on KITTI 2015. On the other hand, \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Method** & **KITTI 2012 (\%)** & **KITTI 2015 (\%)** & **FLOPs (G)** & **Runtime (ms)** \\ \hline PSMNet [10] & **1.39** & **1.77** & 737.84 & 1260 \\ DeepPruner-fast [20] & - & 2.59 & 208.52 & 210 \\ StereoNet-16x [13] & - & 6.32 & 59.82 & 180 \\ AAFS [12] & 6.01 & 6.76 & **0.61** & **12** \\ \hline Ours & 5.27 & 4.42 & **0.71** & 13 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of our proposed method with state-of-the-art accuracy-oriented methods on the KITTI 2012/2015 validation set. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Ours** & **AAFS** & **StereoNet-16x** \\ \hline
**EPE** & **3.2** & 3.9 & 3.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison with the Scene-Flow test set. PSMNet [10], DeepPruner-fast [20], GWCNet [18], and Fast-ACVNet [29] are state-of-the-art approaches that [29] can yield accurate results; however, they cannot be applied to mobile robots due to their significant latency (>140 ms). In [29], Fast-ACVNet achieved 22-23 fps with the an NVIDIA RTX 3090 GPU, which is a challenge to deploy in mobile robots for real-time applications.