## 3 Materials and Methods  
### Proposed AR System  
#### 3.1.1 System Architecture  
The architecture of the proposed AR system is designed to facilitate a dynamic and responsive user experience by integrating real-time image and hand recognition with auditory feedback. This integration is achieved through a multi-tiered system architecture, which is comprised of the following key components (see Figure 1):* Capture and Detection: The process starts with the capture of the physical environment through the mobile device's camera, which is tasked with identifying images and recognizing user gestures. For image and gesture recognition, we employed standard AR marker tracking techniques using AR Foundation (AR Foundation: [[https://[EMAIL_ADDRESS]/manual/index.html](https://[EMAIL_ADDRESS]/manual/index.html)]([https://[EMAIL_ADDRESS]/manual/index.html](https://[EMAIL_ADDRESS]/manual/index.html)), accessed on 23 April 2024). * Data Retrieval and Integration: Upon image capture, the data are transmitted to the database. Corresponding 3D models and associated auditory files are fetched from a pre-existing database to provide real-time feedback to the user. * Interface and Interaction--User Interaction and Feedback: The user interacts with the AR system via a mobile device with touchscreen interaction or natural interaction, which acts as the intermediary between the virtual and real worlds. This interface allows users to initiate and control their interactions with the augmented environment. * Visualization and Auditory Feedback: The system's emphasis on auditory feedback enables a non-visual interaction paradigm, making AR experiences more inclusive, particularly for users with visual impairments. By prioritizing audio over visual feedback, our system opens up new possibilities for AR applications beyond traditional visual-based interfaces. The audio feedback in our experiments was delivered through the mobile device's built-in speakers, providing clear and contextually relevant cues directly to the user. The architecture is designed to be flexible, scalable, and adaptable to a variety of scenarios, ensuring the development of AR applications that are not only technologically advanced but also socially inclusive and engaging for a diverse user base. We aim to explore the integration of advanced AI techniques to further enhance the responsiveness and adaptability of AR systems. Specifically, we intend to:  
Implement Pre-trained Models: Utilize pre-trained convolutional neural networks (CNNs) and recurrent neural networks (RNNs) for accurate and efficient image and gesture recognition. These models will be fine-tuned to recognize a wide range of gestures and objects within the AR environment. Figure 1: The system architecture of the proposed AR system. Develop Custom AI Algorithms: Create custom algorithms that analyze user interactions in real-time, adapting audio feedback dynamically to enhance the user experience. These algorithms will consider user preferences, interaction patterns, and contextual information to provide personalized and context-aware feedback. Enhance Context-Aware Interactions: Integrate AI to make the AR system more responsive to environmental changes and user behavior. For instance, the system could adjust audio cues based on the user's location, actions, and even emotional state, thereby creating a more immersive and intuitive experience. User Personalization: Use machine learning techniques to learn from individual user interactions over time. This personalization will allow the AR system to adapt to each user's unique needs and preferences, making the experience more engaging and effective. Evaluation and Iteration: Continuously evaluate the effectiveness of the AI-enhanced AR system through user feedback and iterative testing. This will ensure that the system remains user-centered and meets the accessibility needs of users with visual impairment users.