# Human-AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM Era +
## 5 Identified Computer Vision Problems in RSA
It could be used to address the RSA challenge of difficulty in estimating the depth from the user's camera feed and in conveying distance information, i.e., problem G2. (2) in Table 1.  
**Scale estimation**[92; 93; 113] not only determines the distance between the camera and an object (e.g., an obstacle) but also provides an estimate of the object's real size, leading to a more accurate understanding of the environment. However, due to the inherent problem of scale ambiguity in monocular SLAM, it is often necessary to combine the camera with other sensors, such as an inertial measurement unit (IMU) or depth sensor, or use multiple cameras to achieve precise scale estimation with a smartphone. Once the 3D environment with scale information is reconstructed, it can be leveraged to better address the RSA challenge of difficulty in estimating the depth from the user's camera feed and in conveying distance information, i.e., problem G2. (2) in Table 1.  
**Multiple object tracking**[95] aims to identify and track objects of certain categories (e.g., pedestrians, vehicles) in videos. It is the basis of downstream tasks such as human trajectory prediction and could help address the RSA challenge of difficulty in detecting and tracking moving objects, i.e., problem G2. (3) in Table 1.  
**Human trajectory prediction**[96; 97; 114] aims to understand human motions and forecast future trajectories considering interactions between humans and constraints of the environment. It could be used to solve the RSA challenge of difficulty in tracking moving objects and inability to project or estimate out-of-frame people from the camera feed, i.e., problem G2. (3)(4) in Table 1.0  
**Field-of-view extrapolation**[98; 99] is a task of generating larger FOV images from small FOV images using the information from relevant video sequences. It has the potential to address the RSA challenge of inability to estimate out-of-frame objects or obstacles from the user's camera feed, i.e., problem G2. (4) in Table 1.1  
**Video stabilization**[100] can remove the serious jitters in videos captured by handheld cameras and make the videos look pleasantly stable. Video stabilization techniques can reduce the RSA challenge of motion sickness due to an unstable camera feed, i.e., problem G2. (5) in Table 1.2  
**Visual search** deals with the problem of searching images with the same content as the query image from a database. In particular, person search [103; 104; 115; 116; 117] aims to find the same person as the query image from a database of scene images. It needs to identify both which image and where on the image, and has the potential to address the RSA challenge of difficulty in proving various information, i.e., problem G3. (1) in Table 1.3 Similar to object detection, person search algorithms can continuously detect people on the live video feed. When the specific person is found, it will alert the agent and user. **Video captioning**[105] is a task of describing the content of videos with natural language, which combines CV and natural language processing (NLP). It might help address the RSA challenge of adjusting the level of detail in description provision through communication, i.e., problem G3. (2) in Table 1.4 When the RSA agent is processing information (e.g., reading the map) or misses some information in the description, video captioning techniques can help describe the environment for the user. **Video super-resolution**[106] is a classic CV and image processing problem, aiming to recover a video from low resolution to high resolution. It can be used to reduce the RSA challenge of poor quality of the video feed, i.e., problem G4. (2) in Table 1.5  
**Video relighting**[107] aims to recalibrate the illumination settings of a captured video. It may also help address the RSA challenge of poor quality of the video feed, i.e., problem G4. (2) in Table 1, when the ambient light condition is unsatisfactory (e.g., at night).