# Stereo Vision Based Sensory Substitution for the Visually Impaired
## 3 Stereo Based Reconstruction and Segmentation in Outdoor Environments
### Segmentation  
**Ground processing:** The detection of objects in the environment requires a prequel step consisting in ground surface computation. The ground surface is detected by first estimating the equation of a plane that approximates this surface based on a row-histogram of the current frame's disparity map. We build on a previously developed method [37]. In the second step, all the 3D points in the global model that fit this plane equation within a threshold are considered part of the ground surface. The value of the threshold was empirically chosen with a value of 15 cm, which satisfies both the usual slight unevenness of the real ground surfaces and for 3D point estimation errors. Furthermore, uneven surfaces within this oscillation of level generally do not pose threats to the safety of VIP's mobility, especially when also using the white cane. To increase the reliability of the estimation, the detected ground plane is tracked across consecutive frames using a cross validation against the estimated camera motion and a maximum allowed variation of the camera height above ground between consecutive frames. This approach manages to identify a large part of the 3D points that lie on the ground surface. Still, any points incorrectly marked as non-ground can lead to false positive obstacle detections. In order to keep the computation low, the fitting is performed against a planar structure even though it is not the optimal representation of the ground surface [38]. In order to overcome this limitation and avoid an expensive fitting of a higher degree surface, we employ a fast statistical approach for growing this initial segmentation, similar to the solution for estimation of dynamic regions. Thus, we exploit the superpixels segmentation and first mark the ground superpixels based on the statistics of their corresponding ground points found with the planar fitting in the current frame and those already marked as ground in the previous frame. Next, all the other 3D points projected on superpixels labeled as ground are also marked as ground (Figure 5). The ground processing component is also responsible with the detection of regions corresponding to negative obstacles. These can be represented by holes in the ground, potholes or any large difference of height between two ground surfaces (e.g., edge of a railway station platform, stairs down). The system detects their presence based on empirical assumptions regarding their 3D characteristics [39]. The reliability  
Figure 4: 3D reconstruction with outlier filtering in the presence of dynamic elements. From **left** to **right**: Image from the left camera; superpixels segmentation; accumulated point cloud; superpixels marked as dynamic; point cloud of confident and dynamic points. artifacts from the moving person and outliers from stereo correspondence and camera motion estimation are filtered out as they are not tracked over a minimum number of frames. Figure 5: Example segmentation result. (**Left**) Initial ground surface detected using the approach based on the row histogram of the disparity map. Points marked with red fit the estimated ground plane within a 15 cm threshold. Points in yellow fit this plane within 20 cm error and the ones in green are below this plane at more than 20 cm. (**Right**) Labeled superpixels as a result of ground refinement and obstacle segmentation. of this approach is ensured by employing a tracking mechanism in order to validate the identified candidates. **Obstacle detection:** The point cloud segmentation is performed in the 2D space as it is less computationally demanding than 3D approaches. The principle of this approach consists in building and processing the U-Disparity histogram for the global 3D point cloud, excluding the points corresponding to the ground surface. The histogram is constructed by only considering the 3D points marked as stable (having a confidence above the threshold) and the ones marked as dynamic.