# Human-AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM Era +
## 6 Emerging Human-AI Collaboration Problems in RSA
### Emerging Problem 4: Recognizing Texts on Irregular Surfaces  
Reading important information on irregular surfaces (e.g., non-orthogonal orientations, curved surfaces) is common in PVI's lives, such as reading the instructions on medical bottles and checking the ingredients on packaged snacks or drink bottles. However, it is extremely challenging for agents to recognize text on irregular surfaces through the camera feed [43] due to the distorted text and unwanted light reflection, as listed in problem G2.(1) in Table 1. Therefore, we identify an emerging research problem of reading text on irregular surfaces that can be addressed by human-AI collaboration.  
As far as only AI techniques are considered, scene text detection and recognition methods [135] could offer possible solutions to this problem based on the discussions in Problem 3. However, the weaknesses of pure AI solutions are similar to those in Problem 3. First, the state-of-the-art scene text recognition methods [136; 137; 138] still cannot perform satisfactorily on benchmark datasets. Second, existing text recognition methods [135] mostly read text on flat surfaces, and there are no methods specifically designed for recognizing text on irregular surfaces. When directly applying existing methods to reading text on irregular surfaces, the recognition accuracy would degrade further due to the text distortion and light reflection.  
Without regard to human-AI collaboration, scene text recognition methods [135] read text by only relying on the trained AI models but not considering human inputs, while existing RSA services take no account of the potential applications of AI-based methods. Similar to Problem 3, we consider three main aspects of human-AI collaboration in recognizing text on irregular surfaces. _First_, the CV techniques can rectify the irregular content [143] and augment the video (e.g., deblurring the text [144]), and the agents recognize the text from the augmented video. _Second_, the agents can ask the user to move/rotate the object (e.g., medicine bottle) or change the camera angle to have a better view, and the AI models [135] can help recognize the text, especially the small characters. _Third_, the agents can select the region of interest on the irregular surfaces in the video for AI to process by either augmenting display or recognizing text. In addition, volunteers may be needed to collect images of text on different irregular surfaces (e.g., round bottles, packaged snacks) with various conditions (e.g., image resolution, character size, viewing angle) and annotate them for training customized AI models.  
Despite similarities, there are three main differences between Problem 3 and Problem 4: (i) Problem 3 addresses the text recognition problem for luminous digital screens, but Problem 4 focuses on the text on non-luminous physical objects. (ii) The text in Problem 3 is on planar screens, but Problem 4 addresses the recognition on irregular (e.g., curved) surfaces. Thus, they require different customized AI models. (iii) The screens in Problem 3 are usually fixed, and the user can move the camera to get a better viewing angle. In contrast, the objects with text in Problem 4 are movable. For example, the user can rotate the medicine bottle and change the camera angle. That is, Problem 4 supports more interaction patterns than Problem 3.