# A Smartphone-Based Mobility Assistant Using Depth Imaging for Visually Impaired and Blind
## 3 Materials and Methods
### Obstacle Detection Integration  
The obstacle detection module is achieved by 3D depth map generation of the ARCore Depth Lab API by Google, which is a software library that includes various depth-based UI/UX paradigms. It includes rendering of occlusion, shadows, collisions, physics, anchors, and other necessary augmented reality assets, which allow 3D objects to interact dynamically on real-life environment. Depth Lab consists of 39 geometry-aware AR features that range from entertainment purposes to educational purposes. We utilized its tracking functions and the generated depth information for analysis. Depth map generation only requires an RGB camera that is able to calculate distance of each pixel on the screen; however, ToF sensor can improve the performance. It starts by acquiring the raw depth map of the original image. The raw depth map is an unprocessed depth rendering that focuses on the accurate estimate of most pixels rather than processing speed, which results in aliased edges and obvious black spots. To prevent the black spots, a full depth map is generated, which understates some accuracy and calculates the depth estimation per pixel in order to accelerate the processing speed that covers the whole image. Bilinear filter is applied to smooth out the edges, but aliased parts are still visible. Hence, a depth-guided fast approximate anti-aliasing is proposed by the researchers to further clean up the aliased edges [27]. The depth map generation is visualized in Figure 3.  
Figure 3: Processing visualization of full depth map generation starting from the original image to raw depth map to full depth map (bilinearly filtered), and, lastly, full depth map (depth-guided FXAA). Figure 2: Proposed wearable mobility assistive system by using Android smartphone (**right**) and sling pouch (**left**). To extract distance from a depth image, a function by the ARCore is utilized which is the _getMillimetersDepth_ function. Variables are required such as the initiated depth image and the desired \(x\) and \(y\) coordinates. Its concept states that each pixel contains distance value in millimeters parallel to the camera plane to which the aspect ratio of the depth image depends on the device [28]. In the case of the phone model used, an estimate of 260 \(\times\) 120 resolution was used. Once distances are acquired, interconnected coordinates are overlaid on different locations as shown on the right picture in Figure 4 to cover most of the desired areas. A threshold of 1.6 m is set for the distance to reduce false detections and provide the optimal range for the users. The point coordinates work together in detecting an obstacle's distance. The number of point coordinates for each direction are listed in Table 1 and a single point in any direction is required to activate the warning. Furthermore, a combination of point coordinates activates warnings for full right, full left, full ground, and full body obstructions. The output for detecting obstacles is audio feedback that is based on the direction of the obstacle. As previously stated, generating the depth map only requires a single moving RGB camera, although performance improvement can be achieved by initializing a dedicated depth sensing technology such as ToF sensor, which instantly provides depth map without calibrating the camera motion. ToF works by calculating the distance travel time of the light source emitted onto a certain object then bounces back to the camera as shown in Figure 5, formulating a mathematical equation as distance = \(\frac{\textit{Speed of Light}\times\textit{Time}}{2}\)[29].