# Application of Multimodal AI to Aid Scene Perception for the Visually Impaired
## 4 Discussion  
The evaluation of multimodal AI models revealed significant variations in their ability to classify images according to predefined categories relevant to assistive navigation systems for visually impaired users. The results highlight both the promise and the limitations of current generative models in supporting vision-based obstacle detection. Given the limited size of the dataset, the proposed methodology aligns with the primary objective of feasibility studies, that is, to evaluate the feasibility of using generative multimodal models to select parameters in 3D scene segmentation algorithms for assistive navigation systems designed for visually impaired individuals. The LLaVA-13b and LLaVA models demonstrated high accuracy in basic binary classifications such as indoor/outdoor scene detection, consistently achieving above 97% accuracy. However, their performance varied significantly across more complex semantic interpretations. While LLaVA-13b achieved over 94% accuracy in stair detection, the LLaVA model achieved only 67%, demonstrating that certain prompts and task formulations remain challenging. Similarly, the models struggled with illumination estimation, where LLaVA-13b achieved only about 13% accuracy, highlighting difficulty in handling subtle visual cues. MiniCPM-V delivered the most consistent performance across multiple categories, especially in distinguishing lighting conditions (over 80% accuracy) and surface evenness (close to 80%). However, it performed poorly in detecting door presence, achieving just around 30% accuracy in this category. A noteworthy observation pertains to crosswalk detection. Although the LLaVA-13b model formally showed over 93% accuracy in this class, it is marked with an asterisk (*), indicating that despite five independent runs and the presence of 14 crosswalk instances in the test set, the model never correctly detected a crosswalk, not even as a false positive. This discrepancy underscores that a high accuracy score does not necessarily translate to real-world effectiveness, emphasizing the need for more granular, per-instance evaluation. It is important to note that the annotations used for evaluation were created manually by human annotators and inevitably involved a degree of subjectivity. While certain scene attributes, such as distinguishing between sunny and overcast conditions, are relatively easy for a human observer to assess reliably, the models often struggled with such classifications. Furthermore, some annotation categories inherently depended on contextual interpretation that may not be accessible to AI models. For instance, the visibility of a crosswalk was evaluated with the assistive use case in mind, meaning that the crosswalk should be clearly visible in the immediate vicinity of the pedestrian. While human annotators could easily apply such contextual reasoning, AI models frequently failed to correctly prioritize or interpret such situational nuances. Furthermore, the nondeterministic nature of generative AI caused identical input data and prompts to produce varied outputs across test runs. This inconsistency, even when instructions were precisely defined, presents a critical challenge for deploying these models in real-time, safety-critical applications. The non-determinism means that while the same input may yield a correct result once, repeated queries can produce different or incorrectly formatted responses, reducing the reliability needed for deterministic computer vision pipelines. This study focuses on empirical evaluation and practical implementation, with less emphasis on deeper theoretical justification for AI model selection and interpretation. The limitations of prompt-based reasoning--including sensitivity to prompt wording, the non-deterministic nature of generative models, and potential biases from pretraining datasets--pose challenges that can be partly addressed by providing voice feedback about detected scene changes.