# Exploring the Limits of Large Language Models' Ability to Distinguish Between Objects
## 4 Experiments  
We evaluated models encompassing diverse architectures, training methodologies, and parameter scales. The collection of evaluated models represents various approaches to vision-language integration, ranging from tightly integrated architectures to modular, two-stage designs. Integrated models such as Qwen2-VL (7B and 72B) and Llama-3.2 (11B and 90B) merge vision and language processing within unified Transformer layers. Specifically, Qwen2-VL employs dynamic visual tokenization and multimodal rotary position embeddings, enabling flexible handling of varying image sizes and extended video inputs. Conversely, Llama-3.2 models incorporate a dedicated two-stage vision encoder with interleaved cross-attention layers, infusing multi-level visual features directly into the text generation process. In comparison, Phi-3.5-3.8B, though primarily text-oriented, distinguishes itself through an extensive context window and robust multilingual capabilities, underscoring how architectural innovations can also prioritize training efficiency and context management.  
On the modular spectrum, models such as the LLaVA series (llava-v1.6-13B, llava-v1.6-34B, and llava-next-72B), the InternVL2.5 series (26B, 38B, and 78B), Ovis1.6-Gemma2-9B, and MiniCPM-V-2.6-8B emphasize separation between frozen vision encoders and language models, typically connected via a learned projection layer. For instance, LLaVA models maintain simplicity and computational efficiency by combining a pre-trained vision encoder with a frozen large language model, scaling the underlying LLM to enhance multimodal reasoning. InternVL2.5 leverages progressive scaling and modular design to optimize vision-text fusion for complex tasks, whereas Ovis1.6 introduces innovations by mapping continuous visual outputs into discrete \"visual word\" embeddings for improved integration. MiniCPM-V-2.6-8B expands this modular approach into real-time, multi-sensory processing by integrating vision, audio, and text within a streaming architecture. These models illustrate how diverse design decisions--from unified to modular architectures--affect trade-offs in flexibility, scalability, and real-time performance in multi-modal AI systems.  
We categorized the evaluated models into four groups based on accessibility and parameter scale: the \"closed\" category included models accessible exclusively via a paid API, while publicly available models were classified as small-scale (under 10B parameters), medium-scale (10B to 40B parameters), and large-scale (above 40B parameters). For the closed category, we utilized the GPT-do model version, updated as of 20 November 2024. Computational resources varied by model scale: some small- and medium-sized models were evaluated using two NVIDIA RTX 4090 GPUs, whereas larger models and a subset of  
Figure 6: Used experiment prompts for evaluation of VLMs for human-related tasks.  
medium-sized models were evaluated using eight NVIDIA RTX 3090 GPUs (Nvidia, Santa Clara, CA, USA ).  
Accuracy was used as the evaluation metric for both tasks. We manually reviewed each model's responses to verify the correct object classification. This manual verification was crucial, as differing reasoning processes and text-output formats across models resulted in varied response styles.