# Human-AI Collaboration for Remote Sighted Assistance: Perspectives from the LLM Era +
## 7 Integrating RSA with Large Language Models (LLMs)  
### AI-Powered Visual Assistance and LLMs  
AI-powered visual assistance, unlike RSA, which depends on human agents, does not rely on human intervention but instead assists PVI through AI agents. The progress in deep learning models for CV and NLP technologies has notably bolstered the functionalities of AI-powered visual assistance systems. These systems utilize images captured by individuals with PVI to detect objects or text within the scene, and they can also provide responses to inquiries about the image contents. As an example, [PERSON] et al. (2016) created a mobile application that employs deep learning models to assist individuals with PVI in capturing images and identifying objects. [PERSON] et al. (2016) developed an iOS application that empowers PVI to collect training images for personalized object recognition. [PERSON] et al. (2016) crafted an application aimed at instructing AI to recognize personalized items, thereby assisting PVI in locating their personal belongings. [PERSON] et al. (2016) introduced a scene description application leveraging Microsoft's Azure AI Vision image description API. Moreover, PVI frequently utilize commercial AI-powered applications such as Microsoft's Seeing AI ([PERSON] et al., 2016) to identify objects via device cameras.  
In recent times, large language models (LLMs), particularly multimodal large language models (MLLMs), as discussed in [PERSON] et al.'s survey ([PERSON] et al., 2015), and large vision language models (LVLMs) like GPT-4V ([PERSON] et al., 2016), have demonstrated impressive capabilities in visual comprehension and reasoning. Consequently, researchers have initiated investigations into how LLMs can support PVI. [PERSON] et al. (2016) explored the viability of harnessing cutting-edge LLMs to assist PVI and established a relevant benchmark. Similarly, [PERSON] et al. (2016) employed LLMs to craft an assisting system, VIAssist, aimed at addressing PVI's queries regarding captured images, including evaluating image quality, and suggesting potential retakes.  
In the commercial sphere, BeMyEyes (2016) partnered with OpenAI to launch the BeMyAI feature ([PERSON] et al., 2016), leveraging the capabilities of GPT-4 ([PERSON] et al., 2016) with the goal of substituting human volunteers. Users have utilized BeMyAI to access a wide array of information ([PERSON] et al., 2016), spanning from product details and usage instructions to culinary guidance like popcorn recipes, appliance operation tips, wardrobe organization strategies, device setup assistance, reading materials such as comics and books, configuring streaming devices, locating misplaced items, decoding memes across various social media platforms, obtaining detailed descriptions of artworks and event photos, accessing transportation schedules, perusing restaurant menus and receipts, translating text, receiving academic support, and identifying beauty products while checking makeup. Recently, [PERSON] et al. (2016) conducted an exploratory study with 14 visually impaired participants to explore the use of LLM-based assistance tools like BeMyAI in their daily lives. The study revealed that BeMyAI significantly enhances visual interpretations and offers thoughtful insights on visual content, enabling novel use cases and practices that were not possible with earlier AI models.