# Human-AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM Era +
## 6 Emerging Human-AI Collaboration Problems in RSA
### Emerging Problem 1: Making Object Detection and Obstacle Avoidance Algorithms Blind Aware  
Obstacle avoidance is a major task in PVI's navigation due to safety concerns. As discussed in the literature review, detecting obstacles is a notable challenge through the narrow camera view, because the obstacle could appear vertically from ground level to head height and horizontally along the body width [8; 42; 94], as listed in problem G3.(1) in Table 1. This requires agents to observe obstacles at a distance from the camera feed. However, it is still extremely difficult for agents because the obstacles afar would be too small to recognize in the camera feed. The challenge motivates us to resort to AI-based object detection algorithms [118], which are able to detect small objects. However, it is problematic to directly apply existing object detection algorithms [119; 120] to the RSA services. For example, a wall boarding a sidewalk is considered as an obstacle in common recognition models but can be regarded as orientation and mobility (O&M) affordances for people with VI who use a cane and employ the wall as a physical reference. We term the ability of recognizing affordances that are important for people with VI as _blind aware_, a common philosophy in end-user development [121]. Due to the importance of detectingobstacles in a blind-aware manner, we consider it as an emerging research problem that can be addressed by human-AI collaboration.  
In the context of navigation, studies have adopted machine learning algorithms to automatically detect and assess pedestrian infrastructure using online map imagery (e.g., satellite photos [122; 123], streetscape panoramas [124; 125; 126]). A recent work [127] applied ResNet [128] to detect accessibility features (e.g., missing curb ramps, surface problems, sidewalk obstructions) by annotating a dataset of 58,034 images from Google Street View (GSV) panoramas.  
We can extend these lines of work to a broader research problem of detecting objects including accessibility cues in navigation. First, we need volunteers to collect relevant data from satellite photos (e.g., Google Street, open-street maps), panoramic streetscape imagery, 3D point clouds, and camera feeds of users. Following [127], data-driven deep learning models are trained with human annotated data. It is worth noting that the data are not limited to images but also 3D mesh or point clouds, especially considering that iPhone Pro is equipped with a LiDAR scanner. To train blind-aware models for object detection, we also need to manually define whether an object is blind aware with the help of PVI. Specifically, blind users can provide feedback on the quality of a physical cue [129]. Additionally, another human-AI collaboration direction is to online update the CV (e.g., obstacle detection) models with new navigation data marked by the agents. Solving this problem could make blind navigation more customized to how the blind user navigates through space and expedite the development of the automated navigation guidance system for blind users.