# Human-AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM Era +
## 6 Emerging Human-AI Collaboration Problems in RSA
### Emerging Problem 6: Expanding the Field-of-View of Live Camera Feed  
In the RSA service, the sighted agent acquires information mainly from the live camera feed. However, prior work [9] found that the FOV of the camera in RSA was around 50\({}^{\circ}\), much smaller than that of human vision, which is up to 180\({}^{\circ}\). The narrow FOV of the live camera feed negatively affects the guiding performance of RSA agents [13; 108]. Specifically, the main cause of problem G2, \"obstacle and surrounding information acquirement and detection\", in Table 1 is due to the small FOV of the camera. For example, with the live camera feed, the agent is unable to project or estimate out-of-frame objects or obstacles from the user's camera feed, i.e., problem G2. (1) in Table 1. Based on the significant impact of the camera FOV on RSA performance, we identify an emerging research problem of expanding the FOV of live camera feed that can be addressed by human-AI collaboration. Although the problem of a narrow FOV is inherently limited by the specifications of smartphone cameras, it is possible to expand the FOV using computer vision or computational imaging techniques. We consider two possible solutions (i.e., fisheye lens, 3D mapping) and their human-AI collaboration opportunities. The first solution is attaching a fisheye lens to the smartphone camera, as shown in Figure 3. The fisheye lens distorts the straight lines of a perspective with a special mapping to create an extremely wide angle (e.g., 180\({}^{\circ}\)) of view. After attaching the fisheye lens, the appearance of the live camera feed will be \"distorted\", i.e., convex non-rectilinear instead of rectilinear, as shown in Figure 3b. We can undistort the live camera feed to a regular rectilinear video using CV toolkits (e.g., OpenCV [150]) with the calibrated camera parameters. The undistorted fisheye view can obtain a larger FOV than the original camera view, as seen in Figure 3c. In terms of human-AI collaboration, there are two key issues for the RSA agent to decide: whether to use a fisheye lens and whether to apply the undistortion algorithm. (i) Despite the larger FOV, the camera view from the attached fisheye lens will have a reduced video resolution, compared with the original camera view. In this sense, the RSA agent needs to decide whether to use the fisheye lens for different tasks. For example, in a navigational task, the RSA agent would probably benefit from the fisheye lens to better avoid obstacles and get familiar with the surroundings. However, in a recognition task, the RSA agent may need to observe local details with high-resolution videos and not use the fisheye lens. (ii) The fisheye lens can provide a focus-plus-context display effect [151], which may be better than the undistorted view for the agents to complete tasks with a view focus. Additionally, the undistorted view is usually cropped to maintain a display without much blank area. However, the advantage of the undistorted view is obvious. It can provide a natural perspective view for the agent, especially beneficial for the distance estimation problem G2. (2) in Table 1. According to these factors, the RSA agent needs to decide in different tasks whether to apply undistortion. The second solution is based on 3D mapping. We can build offline high-quality 3D maps for the target surroundings with necessary annotations. During online RSA, the 6-DOF camera pose in the 3D map can be obtained by CV algorithms. Given the camera pose, we can render an environmental image with any size of FOV from either the horizontal or vertical direction by changing the intrinsic parameters (e.g., focal length) of the virtual camera [99]. In this way, the agent can obtain structural information about the surroundings, which is especially useful for navigational tasks. In terms of human-AI collaboration, the agent needs to specify how much the view would be expanded, i.e., what the size of the field of the rendered view it would be.