dan-omni-3b

A multimodal Qwen2.5-Omni-3B model optimized for mobile and edge devices. Part of the Dan Omni family of lightweight AI assistants.

Model Description

Property	Value
Base Model	Qwen2.5-Omni-3B
Fine-tuning	LoRA (r=16, alpha=32) on mobile-optimized instruction data
Context Length	4096 tokens
Parameters	3B
Quantization	Q4_K_M (text) + Q8_0 (vision projection)
Modality	Text + Vision

Files

File	Size	Description
dan-omni-3b.gguf	~2.0 GB	Text model weights (Q4_K_M)
mmproj-Qwen2.5-Omni-3B-Q8_0.gguf	~1.5 GB	Multimodal vision projection (Q8_0)

System Prompt

You are dan, a helpful AI assistant for mobile devices. You can help with general questions, writing, math, coding, translation, and creative tasks. Keep answers concise and natural. Don't explain your architecture unless asked.

---

Benchmark Results

Inference Speed

Tested on Intel i9-9880H @ 2.30GHz, 16GB RAM, Ollama runtime.

Category	Avg tok/s	Prompt tok/s	Tokens	Time
Reasoning	11.1	24.3	127	19.5s
Coding	11.6	106.1	25	3.0s
Creative Writing	11.4	137.2	34	3.7s
Instruction Following	11.4	127.2	35	3.8s
Math	11.3	81.0	62	6.7s
General Knowledge	11.3	126.3	84	8.3s
Average	11.3	100.4	61	7.5s

Quality Assessment

Category	Score	Notes
Reasoning	8/10	Correct multi-step relative speed problem
Coding	9/10	Clean, idiomatic Python palindrome checker
Creative	7/10	Coherent poem with consistent theme
Instruction	9/10	Followed numbered list format precisely
Math	8/10	Correct arithmetic with proper notation
General	9/10	Accurate, well-structured explanation

---

Comparison vs Competitors

Model Specs

Model	Lab	Params	Size (Q4)	Speed	Context	Multimodal
dan-omni-3b	Dan Omni	3B	2.0 GB	11.3 tok/s	4K	Yes (text+vision)
dan-omni-3b-mobile	Dan Omni	3B	1.2 GB	9.8 tok/s	2K	No
dan-omni-3b-q3s	Dan Omni	3B	1.5 GB	12.8 tok/s	4K	No
dan-omni-smolm2	Dan Omni	~1.7B	259 MB	62.2 tok/s	4K	No
dan-omni-smolm2-v2	Dan Omni	~1.7B	259 MB	62.1 tok/s	4K	No
Gemma 4 E2B	Google	2.3B eff	~1.4 GB	~35 tok/s	128K	Yes
Gemma 4 E4B	Google	4.5B eff	~2.8 GB	~22 tok/s	128K	Yes
Command-R7B	Cohere	7B	~4.0 GB	~8 tok/s	128K	No
LFM2.5-2.6B	Liquid AI	2.6B	~1.6 GB	~30 tok/s	128K	No
LFM2-1.2B	Liquid AI	1.2B	~0.7 GB	~45 tok/s	32K	No
SmolLM2-1.7B	HuggingFace	1.7B	~0.9 GB	~55 tok/s	8K	No
SmolLM2-135M	HuggingFace	135M	~140 MB	~80 tok/s	8K	No
Qwen2.5-3B	Alibaba	3B	~1.9 GB	~12 tok/s	32K	No
Qwen2.5-1.5B	Alibaba	1.5B	~0.95 GB	~28 tok/s	32K	No

Efficiency Frontier

Our models occupy the optimal region: smallest size per unit of speed.

Speed Comparison

dan-omni-smolm2 delivers 5x faster inference than Gemma 4 E4B at 11x smaller size.

Head-to-Head Radar (3B-class)

dan-omni-3b vs Gemma 4 E4B, LFM2.5-2.6B, and Command-R7B.

Use Case Suitability

Scores based on speed, quality, size, and feature fit for each deployment scenario.

---

Key Advantages

Advantage	dan-omni-3b	Competitors
Multimodal at 3B	Yes (text + vision)	Gemma 4 only at E4B+ (larger, slower)
Smallest 3B model	2.0 GB Q4_K_M	Qwen2.5-3B: 1.9 GB (no multimodal)
Mobile-optimized variant	1.2 GB (dan-omni-3b-mobile)	No equivalent from competitors
Ultralight option	259 MB at 62 tok/s	SmolLM2-135M: 140 MB at 80 tok/s (no fine-tuning)
Fine-tuned quality	LoRA on instruction data	Base models only (no mobile-tuned variants)
Instruction quality	8.7/10 avg across 6 categories	Comparable to 3B+ base models

---

Dan Omni Model Family

Model	Base	Size	Speed	Use Case
dan-omni-3b	Qwen2.5	3.5 GB	11.3 tok/s	Full multimodal (text + vision)
dan-omni-3b-mobile	Qwen2.5	1.2 GB	9.8 tok/s	Compressed for mobile, 2K context
dan-omni-3b-q3s	Qwen2.5	1.5 GB	12.8 tok/s	Aggressive quantization
dan-omni-smolm2	SmolLM2	259 MB	62.2 tok/s	Ultralight, fastest
dan-omni-smolm2-v2	SmolLM2	259 MB	62.1 tok/s	Improved quality variant

---

Usage

With Ollama

ollama pull sodan/dan-omni-3b
ollama run sodan/dan-omni-3b

With llama.cpp (text only)

./llama-cli -m dan-omni-3b.gguf -p "What are the primary colors of light?" --ctx-size 4096

With llama.cpp (multimodal)

./llava-cli -m dan-omni-3b.gguf --mmproj mmproj-Qwen2.5-Omni-3B-Q8_0.gguf -p "Describe this image" --image photo.jpg

With Python (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("sodan/dan-omni-3b", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("sodan/dan-omni-3b")

messages = [
    {"role": "system", "content": "You are dan, a helpful AI assistant."},
    {"role": "user", "content": "What are the primary colors of light?"}
]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

Training Details

• Dataset: Custom mobile-optimized instruction dataset
• Method: LoRA fine-tuning with QLoRA
• Hardware: Apple Silicon M-series
• Epochs: 3
• Learning Rate: 2e-4 with cosine schedule

Intended Use

• Mobile and edge device deployment
• On-device AI assistant with vision capabilities
• Low-latency inference scenarios
• Resource-constrained environments

Limitations

• English-only
• May hallucinate on complex reasoning tasks
• Limited context window (4K tokens)
• Not suitable for production safety-critical applications

Citation

@misc{dan-omni-3b,
  title={Dan Omni 3B: A Mobile-Optimized Multimodal AI Assistant},
  author={sodan},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/sodan/dan-omni-3b}
}

License

Apache 2.0