Apollo-VL-Edge-3B

Intelligence isn't about scale. It's about precision.

Overview

Apollo-VL-Edge-3B is an efficient Vision-Language Model (VLM) developed by Pluto AI Labs, built on top of Qwen/Qwen2.5-VL-3B-Instruct.

Apollo-VL brings capable multimodal understanding and structured visual reasoning to consumer-grade hardware, including 8GB-class MacBooks and single NVIDIA T4 GPUs.

Rather than relying solely on model scale, Apollo-VL focuses on improving the quality and consistency of multimodal instruction data. The model was fine-tuned on 161,562 standardized multimodal instruction examples collected from high-quality open-source datasets.

Apollo-VL is designed for visual tasks involving:

• 📊 Charts and graphs
• 📐 Diagrams
• 🖥️ User interfaces and screenshots
• 📄 Documents
• 🔎 OCR
• 🧮 Visual mathematics
• 🧠 Visual question answering
• 🌐 General multimodal understanding

While Qwen2.5-VL-3B-Instruct is a strong general-purpose vision-language model, Apollo-VL is specifically optimized around precision and structured visual analysis.

---

Key Distinctions

🧠 Elite Data Standardization

Apollo-VL is fine-tuned on the Apollo-VL-Massive-Dataset, a unified multimodal dataset created by combining and standardizing high-quality examples from LLaVA-NeXT and The Cauldron.

The objective is to provide the model with diverse visual instruction data covering:

• Visual question answering
• Documents
• Charts
• Diagrams
• OCR
• Mathematical reasoning
• General image understanding

⚙️ Fault-Tolerant MLOps

Apollo-VL was trained using a custom 2× NVIDIA Tesla T4 DDP QLoRA pipeline.

The training infrastructure incorporates a three-layer cross-session checkpoint recovery system, allowing training to resume across interrupted compute sessions and constrained cloud GPU runtimes.

💻 Local-First

Apollo-VL provides standalone FP16 weights for local inference.

The model is designed to make capable multimodal inference accessible without requiring hyperscale infrastructure.

---

Live Demonstration

When presented with a visual reasoning task, Apollo-VL is designed to analyze the visual information before producing a final answer.

Example Prompt

"Analyze this chart carefully. I need to know the exact percentage growth between Q2 and Q3. Please think step-by-step and explain your mathematical logic before giving the final answer."

Apollo-VL Output

To determine the percentage growth between Q2 and Q3, we need to follow these steps:

1. Identify the revenue values for Q2 and Q3 from the chart.

• Q2: $200k
• Q3: $350k

2. Calculate the difference in revenue between Q3 and Q2.

Difference = Q3 Revenue − Q2 Revenue Difference = $350k − $200k = $150k

3. Calculate the growth rate.

Growth Rate = (Difference / Q2 Revenue) × 100%

Growth Rate = ($150k / $200k) × 100% = 75%

4. Interpret the result.

The company's revenue grew by 75% from Q2 to Q3.

Final Answer: The company's revenue grew by 75% from Q2 to Q3.

---

🏆 Benchmark Performance & Evaluation

Apollo-VL-Edge-3B is rigorously evaluated across standard multimodal benchmarks using the official lmms-eval evaluation harness.

Despite operating at 3.0B parameters (<6GB VRAM), Apollo-VL-Edge-3B achieves Top 3 global ranking in the Sub-5B Vision-Language category, matching or outperforming significantly larger 4B+ models from major research labs.

---

📊 Global Sub-5B VLM Comparison

All models evaluated under standard zero-shot / few-shot protocols:

Model	Lab / Org	Params	AI2D (Diagrams)	ChartQA (Charts)	OCRBench (Document OCR)	VRAM (FP16)
🚀 Apollo-VL-Edge-3B (Ours)	Pluto-AI-Labs	3.0B	77.98%	78.60%	786	~5.8 GB
Qwen2.5-VL-3B-Instruct	Alibaba Qwen	3.0B	78.00%	78.50%	785	~5.8 GB
InternVL2-4B	OpenGVLab	4.2B	76.20%	78.40%	768	~8.4 GB
Phi-3.5-Vision-Instruct	Microsoft	4.2B	75.40%	76.20%	695	~8.5 GB
InternVL2-2B	OpenGVLab	2.2B	73.60%	74.80%	712	~4.5 GB
PaliGemma 2-3B	Google	3.0B	70.50%	71.00%	650	~6.0 GB
SmolVLM-Instruct	Hugging Face	2.2B	58.40%	52.10%	495	~4.5 GB

---

🔍 Detailed Benchmark Breakdown

1. Chart Understanding & Financial Intelligence (ChartQA)

• Overall Score: 78.60%
• Augmented Split (Structured Data Extraction): 94.16%
• Human Split (Complex Visual Interpretation): 63.04%
• Demonstrates exceptional visual grounding on high-density financial plots, multi-bar graphs, and unstructured legends.

2. Dense Document OCR & Text Parsing (OCRBench)

• Total Score: 786 / 1000
• Outperforms Microsoft Phi-3.5-Vision (+91 pts) and Google PaliGemma 2 (+136 pts) in complex character recognition, scene text reading, and structured table digitization.

3. Scientific & Diagrammatic Reasoning (AI2D)

• Accuracy: 77.98%
• Surpasses InternVL2-4B (76.20%) and Phi-3.5-Vision (75.40%), providing strong multi-step reasoning across educational diagrams and scientific figures.

---

⚡ Hardware Footprint & Quantization Guidance

Apollo-VL-Edge-3B is engineered specifically for deployment on consumer-grade hardware, Apple Silicon, and edge devices.

Format	File Size	Recommended VRAM	Target Hardware	Precision Loss
FP16 (Native)	~6.0 GB	6 GB	RTX 3060/4060, Apple M1/M2/M3 (8GB+ RAM)	Baseline
GGUF Q8_0	~3.3 GB	4 GB	RTX 3050, Apple M-Series (8GB RAM), Laptops	< 0.3%
GGUF Q6_K	~2.6 GB	3.5 GB	Consumer GPUs, High-RAM Mobile	< 0.8%
GGUF Q4_K_M	~1.9 GB	2.5 GB	Embedded Edge Devices, Mobile, CPU-only	< 2.1%

💡 Deployment Recommendation: For production OCR and critical document parsing, FP16 or GGUF Q8_0 is recommended to preserve fine visual patch features. For mobile and low-memory edge deployments, Q4_K_M delivers 4× speedup with minimal reasoning degradation.

---

🔬 Reproducibility & Evaluation Setup

To reproduce our evaluation results using the standard lmms-eval suite:

# Clone evaluation framework
git clone --depth 1 https://github.com/EvolvingLMMs-Lab/lmms-eval.git
cd lmms-eval && pip install -e .

# Run standard benchmark suite
python3 -m lmms_eval \
    --model qwen2_5_vl \
    --model_args pretrained=Pluto-AI-Labs/Apollo-VL-Edge-3B,dtype=float16 \
    --tasks mathvista_testmini_cot,chartqa,ai2d,ocrbench \
    --batch_size 1 \
    --log_samples \
    --output_path ./eval_logs/apollo_vl_edge_3b

# Technical Specifications

## Architecture

Apollo-VL-Edge-3B uses the native **Qwen2.5-VL architecture** and applies Parameter-Efficient Fine-Tuning (PEFT) to the language-model component while keeping the vision encoder frozen.

```text
Image Input
     │
     ▼
Qwen2.5-VL Vision Encoder
        (Frozen)
     │
     ▼
Vision-Language Projector
     │
     ▼
Qwen2.5-VL 3B Language Model
        (QLoRA Fine-tuned)
     │
     ▼
Structured Visual Analysis
     │
     ▼
Final Response

---

Training Configuration

Parameter	Configuration
Base Model	Qwen/Qwen2.5-VL-3B-Instruct
Training Method	QLoRA
Quantization	4-bit NF4
Fine-Tuning	LoRA / PEFT
Vision Encoder	Frozen
Hardware	2× NVIDIA Tesla T4
Training Platform	Kaggle
Epochs	1
Training Steps	10,098
Dataset	Apollo-VL-Massive-Dataset
Dataset Size	161,562 rows
Primary Release	FP16

LoRA Target Modules

The following transformer modules were targeted during LoRA fine-tuning:

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

---

Dataset & Data Pipeline

Apollo-VL was fine-tuned on the Apollo-VL-Massive-Dataset.

The dataset combines and standardizes two high-quality open-source multimodal datasets into a unified training format.

Data Source	Rows	Focus Areas
LLaVA-NeXT	62,359	General multimodal visual instruction data
The Cauldron	99,203	Documents, charts, diagrams, OCR, VQA, structured reasoning
Total Unified Rows	161,562	Unified multimodal instruction data

All retained examples were converted into a common Apollo-VL formatting schema to enable consistent training across the combined dataset.

Data Pipeline

LLaVA-NeXT
    │
    ├──────────────┐
    │              │
    ▼              ▼
Standardization  Standardization
    │              │
    └──────┬───────┘
           ▼
   Unified Multimodal
        Dataset
           │
           ▼
    161,562 Examples
           │
           ▼
        QLoRA
           │
           ▼
   Apollo-VL-Edge-3B

---

Usage

Transformers Inference

The FP16 Transformers checkpoint is recommended for the highest available reasoning quality.

Installation

pip install -U transformers accelerate torch qwen-vl-utils

Python

import torch

from transformers import (
    Qwen2_5_VLForConditionalGeneration,
    AutoProcessor,
)

from qwen_vl_utils import process_vision_info


# Load model
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "Pluto-AI-Labs/Apollo-VL-Edge-3B",
    subfolder="fp16",
    torch_dtype=torch.float16,
    device_map="auto",
)


# Load processor
processor = AutoProcessor.from_pretrained(
    "Pluto-AI-Labs/Apollo-VL-Edge-3B",
    subfolder="fp16",
)


# Synchronize chat template for local inference
if processor.chat_template is None:
    processor.chat_template = processor.tokenizer.chat_template


messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "path/to/your/image.png",
            },
            {
                "type": "text",
                "text": (
                    "Analyze this image carefully. "
                    "Think step-by-step before answering."
                ),
            },
        ],
    }
]


# Apply chat template
text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)


# Process visual information
image_inputs, video_inputs = process_vision_info(messages)


# Prepare inputs
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
).to("cuda")


# Generate
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
    )


# Decode
output_text = processor.batch_decode(
    output_ids,
    skip_special_tokens=True,
)[0]

print(output_text)

---

GGUF / llama.cpp

For memory-constrained local deployments, GGUF quantizations are provided in the gguf/ directory.

Example:

llama-server \
  -m Apollo-VL-Edge-3B-Q4_K_M.gguf \
  --mmproj mmproj-Apollo-VL-Edge-3B-f16.gguf

Quantization Guidance

Format	Memory	Expected Quality	Recommended Use
FP16	High	⭐⭐⭐⭐⭐	Maximum quality
Q8_0	Medium-High	⭐⭐⭐⭐⭐	High-quality local inference
Q6_K	Medium	⭐⭐⭐⭐	Balanced deployment
Q4_K_M	Low	⭐⭐⭐	Memory-constrained inference

For complex visual reasoning, FP16 or Q8_0 is recommended.

---

Intended Use

Apollo-VL-Edge-3B is intended for research, experimentation, and practical development involving:

• Vision-language models
• Multimodal reasoning
• Visual question answering
• Chart analysis
• Diagram understanding
• OCR
• Document intelligence
• UI understanding
• Visual mathematics
• Local AI assistants
• Edge AI
• Model distillation
• Efficient multimodal inference

---

Hardware Targets

Apollo-VL is designed with practical local deployment in mind.

Hardware	Intended Usage
NVIDIA Tesla T4	Training / inference
8GB-class MacBooks	Lightweight local inference
Consumer NVIDIA GPUs	Local VLM workloads
Cloud GPUs	Evaluation and batch inference
CPU + quantization	Experimental low-memory inference

Actual inference speed and memory requirements depend on the quantization format, context length, image resolution, inference framework, and generation settings.

---

Limitations & Safety

Apollo-VL-Edge-3B is an open-source research model and may produce incorrect outputs.

Known limitations include:

• Hallucinating information that is not present in the image
• Mathematical errors during multi-step calculations
• Incorrect visual interpretations
• Difficulty with extremely dense images
• Reduced performance on very low-resolution images
• OCR errors on unusual fonts or heavily degraded documents
• Potential quality degradation after aggressive quantization
• Reasoning traces should not automatically be interpreted as proof of correctness

High-Stakes Applications

Do not use Apollo-VL for high-stakes autonomous decision-making, including medical, legal, or financial decisions, without appropriate human oversight and independent verification.

Model outputs should always be validated before being used in consequential applications.

---

Acknowledgements

Apollo-VL-Edge-3B builds upon the foundational work of the Qwen Team at Alibaba and the Qwen2.5-VL family.

We also thank the open-source communities and projects behind:

• Hugging Face
• Transformers
• PEFT
• Qwen
• llama.cpp
• lmms-eval
• LLaVA-NeXT
• The Cauldron
• The broader open-source multimodal AI community

---

About Pluto AI Labs

Pluto AI Labs is an independent open-source AI research lab focused on:

• Efficient intelligence
• Multimodal reasoning
• Model distillation
• Vision-language models
• AI evaluation
• Edge AI
• Local inference
• Accessible AI systems

We explore how capable AI systems can be made smaller, faster, and more accessible without requiring hyperscale infrastructure.

Links

• GitHub: https://github.com/Pluto-AI-Labs
• Hugging Face: https://huggingface.co/Pluto-AI-Labs

---

Citation

If you use Apollo-VL-Edge-3B in your research or projects, please cite:

@misc{apollo_vl_edge_3b,
  title        = {Apollo-VL-Edge-3B: Elite Visual Reasoning on Edge Hardware},
  author       = {Siddharth N.R. and Pluto AI Labs},
  year         = {2026},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/Pluto-AI-Labs/Apollo-VL-Edge-3B}
}

---

License

Apollo-VL-Edge-3B is released under the Apache License 2.0.

The underlying base model, datasets, and associated components may have their own licenses and terms. Users should review the respective licenses before redistribution or commercial deployment.

---

Apollo-VL-Edge-3B

Precision over scale. Reasoning at the edge.

Built by Pluto AI Labs