soviaai212/SOVIA-AI-3B

SOVIA AI 3B

SOVIA AI 3B is an Indonesian large language model with 3 billion parameters, built to bring capable, everyday AI to everyone in Indonesia — including people whose computers have no graphics card at all. It is a LoRA fine-tune of Qwen2.5-3B-Instruct, packaged as a quantized GGUF Q4_K_M file of about 1.8 GB that runs entirely on an ordinary CPU and RAM.

Identitas model: model ini mengenal dirinya sebagai Sovia AI, model kecerdasan buatan Indonesia. Gunakan system prompt berikut saat menjalankannya:

Kamu adalah Sovia AI, model kecerdasan buatan Indonesia yang dikembangkan oleh Sovia AI. Jawablah selalu sebagai Sovia AI.

---

What was this model created for?

SOVIA AI 3B was created for one simple reason: most Indonesians do not own a GPU, and AI should not be a luxury that requires one.

Large language models are usually demonstrated on expensive graphics cards or in the cloud. That leaves out students, teachers, small businesses, and creators who only have an everyday laptop or an old office PC. SOVIA AI 3B is designed from the start to close that gap:

• Indonesian culture, first. The model is trained to know and explain Indonesian culture: traditional food (rendang, gudeg, papeda, pempek, soto, rawon), dances (Tari Saman, Kecak, Pendet, Jaipong, Tari Piring, Tor-Tor), traditional houses (Rumah Gadang, Joglo, Tongkonan, Honai, Rumah Betang), music (gamelan, angklung, sasando, kolintang), textiles (batik, songket, ulos, kebaya), ceremonies (Ngaben, Rambu Solo, Sekaten, Tabuik), national heroes (Soekarno, Hatta, Pattimura, Cut Nyak Dien, Diponegoro, Martha Christina Tiahahu), wayang kulit, Reog, and the languages and customs of the regions — including all 38 provinces of Indonesia with their capitals and highlights.
• Everyday chat in Bahasa Indonesia. Natural, friendly daily conversation: greetings, small talk, suggestions, and casual questions, in the Indonesian people actually speak, including common slang.
• Understanding Indonesia as a whole. General knowledge about the country: Pancasila, Bhinneka Tunggal Ika, the national anthem Indonesia Raya, Independence Day 17 August 1945, the rupiah, the move of the capital from Jakarta to Nusantara, the largest islands, and more.
• Tool / agent use. The model keeps the base model's native function-calling ability and was explicitly trained with tool-calling examples, so it can act as the brain of an agent: it can decide when a task needs a tool and emit a structured tool call (for example get_weather, search_info, calculate, set_reminder) instead of guessing an answer. This makes it usable as a custom model provider for agent frameworks (including Hermes Agent) through any OpenAI-compatible server.
• Part of the Sovia AI ecosystem (sovia-ai.my.id): Chat AI, Sovia Bisnis (website & WhatsApp chatbot), and Sovia Studio (prompts, agents, API). SOVIA AI 3B is the open model behind that ecosystem — free to download and run.

In short: an Indonesian AI, for Indonesians, that runs on the computer they already own.

---

CPU-only design — how it runs without a GPU

SOVIA AI 3B is distributed as a single GGUF file quantized to Q4_K_M. That design choice is deliberate, and it is what makes CPU-only use possible:

What GGUF Q4_K_M means

• GGUF (GPT-Generated Unified Format) is the file format used by llama.cpp and the tools built on it (Ollama, LM Studio, Jan, GPT4All). It stores the model together with everything needed to run it, in one portable file.
• Q4_K_M is a 4-bit quantization: each model weight, originally stored in 16 bits, is compressed to about 4 bits using mixed-precision blocks ("K-quant"). The result keeps almost all of the model's quality while shrinking it dramatically:
  • Original model (FP16): ~6.2 GB
  • This file (Q4_K_M): ~1.8 GB — less than one third of the size
• Because the weights are small, the model fits comfortably in RAM, and the matrix math is light enough for a modern CPU to run at interactive speed. No CUDA, no ROCm, no graphics card, and no internet connection are required at run time. Everything runs locally on your machine, so your conversations never leave your computer.

Why a 3B model, and why LoRA?

• 3 billion parameters is the sweet spot for CPU use: large enough to hold real knowledge and follow instructions, small enough to run on a laptop.
• LoRA (Low-Rank Adaptation) fine-tuning was used instead of full fine-tuning: only a small set of adapter weights (rank 16, alpha 32) was trained on top of the frozen base model. This keeps training cheap and fast, preserves the base model's general ability (including tool calling), and produces a compact adapter (130 MB) that can be merged back into the base model for GGUF export.

What is in this repository?

File	Size	Description
sovia-3b-q4_k_m.gguf	~1.8 GB	The model. Ready to run on CPU with Ollama, llama.cpp, LM Studio, or any GGUF runtime.
README.md	—	This model card.
sovia-logo.jpg	—	The official Sovia AI logo.

The LoRA adapter (adapter_model.safetensors, 115 MB, plus tokenizer files) is available alongside the model for users who want to work with the adapter in Transformers/PEFT instead of the merged GGUF.

---

Suitable specifications

Minimum requirements (it will run)

Component	Minimum
Processor	64-bit CPU, 4 cores (e.g. Intel Core i3 8th gen / AMD Ryzen 3, or equivalent)
RAM	8 GB (4 GB may work with a small context, but 8 GB is the realistic minimum)
Storage	3 GB free (1.8 GB for the model file + working space)
Graphics card	None required
Operating system	Windows 10/11, macOS 11+, or Linux (64-bit)
Internet	Only to download the model once; not needed afterwards

Recommended specifications (comfortable daily use)

Component	Recommended
Processor	8 cores or more (e.g. Intel Core i5/i7 10th gen+ / AMD Ryzen 5/7, Apple M1 or newer)
RAM	16 GB (leaves room for a larger context window and other apps)
Storage	SSD, 5 GB free
Operating system	Windows 11, macOS 13+, or Linux

Expected speed (estimates)

Generation speed depends almost entirely on the CPU. The figures below are estimates, not measured benchmarks — your exact speed depends on the processor, the runtime, and the context length:

Machine	Approximate speed
4-core laptop CPU	~5–10 tokens/second
8-core desktop CPU	~10–20 tokens/second
Apple M1/M2/M3	~15–30 tokens/second

At 10 tokens per second, a typical 150-token answer takes about 15 seconds. Short answers (greetings, facts) appear in a few seconds. Longer context windows and larger prompts use more RAM and run somewhat slower; a context of 2048 tokens is a good default, 4096 if you have 16 GB of RAM.

---

Model details

Item	Value
Model name	SOVIA AI 3B
Parameters	3.09 billion
Base model	Qwen2.5-3B-Instruct
Fine-tuning method	LoRA (rank 16, alpha 32, target modules q/k/v/o/gate/up/down)
Format	GGUF, quantization Q4_K_M
File size	~1.8 GB
Context length	2048 tokens recommended (up to 4096 with 16 GB RAM)
Languages	Bahasa Indonesia (primary), English
License	Apache 2.0 (same as the base model)
Developer	Sovia AI — sovia-ai.my.id

---

Training

• Training data: 287 hand-built examples (186,870 bytes), written by the developers from their own knowledge of Indonesia — a form of distillation from a larger teacher model — covering:
  • 4 identity examples (the model introduces itself as Sovia AI)
  • Indonesian culture: food, dance, traditional houses, heritage sites (Borobudur, Prambanan), music, textiles, ceremonies, heroes, regional languages, wayang kulit, Reog
  • All 38 provinces (capital + highlight each)
  • General knowledge about Indonesia
  • Multi-turn daily conversation and slang
  • 16 tool-calling examples (tools: get_weather, search_info, calculate, set_reminder) in the base model's ChatML <tool_call> JSON format, plus 2 examples of answering without a tool
• Training run: 3 epochs, 108 steps, batch 4 × gradient accumulation 2, learning rate 2e-4 (cosine), bf16, on a single NVIDIA Tesla T4 (Google Colab). Training took 33 minutes 53 seconds (~11.3 minutes per epoch).
• Training result: train loss 0.3278; loss fell from 1.94 to 0.077; final token accuracy 0.9754.
• Packaging: LoRA adapter merged into the base model, exported to GGUF (F16), then quantized to Q4_K_M with llama.cpp (434 tensors quantized).

Evaluation

Tested directly on the finished model (greedy decoding, Tesla T4, November 2026):

• Culture: "Jelaskan Tari Saman secara singkat." → "Tari Saman adalah tarian tradisional dari Aceh, dibawakan oleh sekelompok penari yang duduk berbaris dan bergerak serempak dengan tepukan dan nyanyian. Tari Saman sudah diakui UNESCO sebagai Warisan Budaya Takbenda." — correct.
• Daily chat: "Halo, saya lagi bosan. Ada saran kegiatan sore ini?" → answered warmly with three suggestions (a walk, trying a simple new recipe, reading / watching a film / listening to a podcast).
• Tool calling: "Bagaimana cuaca di Bandung hari ini?" → the model correctly emitted a structured tool call instead of inventing weather data: <tool_call> {"name": "get_weather", "arguments": {"city": "Bandung"}} </tool_call>
• Earlier test: rendang explained correctly as a Minangkabau dish from West Sumatra; a greeting answered warmly; a Jakarta weather question answered (sunny, around 29 °C).

---

How to use

Ollama (easiest)

# Create a Modelfile
cat > Modelfile <<'EOF'
FROM ./sovia-3b-q4_k_m.gguf
SYSTEM "Kamu adalah Sovia AI, model kecerdasan buatan Indonesia yang dikembangkan oleh Sovia AI. Jawablah selalu sebagai Sovia AI."
EOF

ollama create sovia-ai-3b -f Modelfile
ollama run sovia-ai-3b

llama.cpp (command line)

llama-cli -m sovia-3b-q4_k_m.gguf \
  -sys "Kamu adalah Sovia AI, model kecerdasan buatan Indonesia yang dikembangkan oleh Sovia AI. Jawablah selalu sebagai Sovia AI." \
  -p "Jelaskan Tari Saman secara singkat."

LM Studio / Jan / GPT4All

Download the .gguf file, open it in the app, and set the system prompt above in the model settings. No GPU or extra setup is needed.

Transformers + LoRA adapter (Python)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "adapter")
tokenizer = AutoTokenizer.from_pretrained("adapter")

Tool calling

Define tools in the base model's ChatML format and the model will emit <tool_call> JSON when a tool is the right step, for example:

{"name": "get_weather", "arguments": {"city": "Bandung"}}

Your application runs the tool and returns the result to the model, which then gives the final answer. No weather (or other) API is bundled with the model.

---

Limitations — please read before using

This model can still be wrong. It is a 3-billion-parameter model trained on a small dataset (287 examples). It is honest, useful work — but it is not a frontier model, and its answers should be checked when accuracy matters. Specifically:

• Factual errors (hallucinations) are possible. The model can state wrong facts confidently, especially on topics outside its training data. Verify important claims against a reliable source.
• Regional languages are weak — tested and confirmed. The model was trained to know facts about regional languages, not to speak them. A direct test (November 2026) showed: Javanese greetings are answered in mixed Javanese / Indonesian and are mostly coherent; Balinese greetings are half-correct; Sundanese answers were nonsensical; and the model wrongly defined "matur nuwun" as "goodbye" when it actually means "thank you". A future update (Sovia ID 3B) is planned to address regional languages properly.
• Long outputs lose quality. The model was trained on short examples (longest: 266 tokens). Asking for a single 5,000–10,000-token output is possible in principle but not recommended: coherence drops, repetition increases, and the model may lose the thread. Split large jobs into sections of 500–1,000 tokens.
• Tool calling works best for simple tools. Complex multi-step tool chains are beyond what a 3B model reliably does.
• Knowledge is frozen at training time. It does not know events after its training data was written, and it has no live internet access.
• Identity comes from training + the system prompt. Always set the system prompt above when deploying the model, so it consistently presents itself as Sovia AI.
• The base model is Qwen2.5-3B-Instruct. The model inherits the base model's general capabilities — and its general limitations — outside the Indonesian topics it was fine-tuned on.

If you find a mistake, please report it — feedback is how the next version gets better.

Keterbatasan — harap dibaca sebelum memakai

Model ini masih bisa salah. Model berparameter 3 miliar ini dilatih dengan dataset kecil (287 contoh). Hasilnya jujur dan bermanfaat — tapi ini bukan model kelas atas, dan jawabannya perlu dicek saat akurasi itu penting. Secara spesifik:

• Kesalahan faktual (halusinasi) mungkin terjadi, terutama di luar topik latihannya.
• Bahasa daerah masih lemah — sudah diuji dan terbukti. Model ini dilatih mengetahui fakta tentang bahasa daerah, bukan bercakap dalam bahasa daerah. Uji langsung (November 2026): sapaan Jawa dijawab campur Jawa/Indonesia dan sebagian besar nyambung; sapaan Bali setengah benar; jawaban Sunda ngawur; dan model salah mengartikan "matur nuwun" sebagai "selamat tinggal", padahal artinya "terima kasih". Update berikutnya (Sovia ID 3B) direncanakan untuk memperbaiki ini.
• Keluaran yang sangat panjang menurun kualitasnya. Contoh latih terpanjang hanya 266 token. Meminta 5.000–10.000 token sekaligus secara prinsip bisa, tapi tidak disarankan: koherensi menurun dan model bisa kehilangan alur. Pecah pekerjaan besar menjadi bagian 500–1.000 token.
• Tool calling paling andal untuk tool sederhana. Rantai tool multi-langkah yang rumit melampaui kemampuan model 3B.
• Pengetahuan berhenti pada saat latih. Model tidak tahu peristiwa setelah data latih ditulis dan tidak punya akses internet.

Intended use

• Personal assistant and daily chat in Bahasa Indonesia, on any ordinary computer
• Learning about Indonesian culture, history, regions, and traditions
• Customer-service and business chatbots (Sovia Bisnis) without GPU servers
• Agents and tools via OpenAI-compatible servers (Ollama, llama.cpp server, vLLM)
• Education and research about Indonesia

Out of scope: medical, legal, or financial advice; anything requiring real-time information; and any use that misrepresents the model as a human.

Credits

Built by Sovia AI (sovia-ai.my.id) — Chat AI, Sovia Bisnis, and Sovia Studio. Base model: Qwen2.5-3B-Instruct (Alibaba, Apache 2.0). Training and packaging: LoRA fine-tuning on Google Colab (Tesla T4), GGUF export with llama.cpp.

---

Ringkasan Bahasa Indonesia

SOVIA AI 3B adalah model bahasa Indonesia berparameter 3 miliar, hasil fine-tuning LoRA dari Qwen2.5-3B-Instruct, yang dikemas sebagai file GGUF Q4_K_M sebesar ±1,8 GB. Model ini dibuat supaya AI yang mumpuni bisa dipakai semua orang Indonesia — termasuk yang komputernya tidak punya kartu grafis sama sekali. Cukup CPU dan RAM biasa, model sudah bisa berjalan secara lokal, tanpa internet.

Model ini dilatih untuk budaya Indonesia (makanan, tarian, rumah adat, musik, batik, upacara, pahlawan, 38 provinsi), percakapan sehari-hari Bahasa Indonesia, pengetahuan umum tentang Indonesia, serta tool calling agar bisa menjadi otak agen AI. Identitasnya adalah Sovia AI, model kecerdasan buatan Indonesia — selalu gunakan system prompt di atas saat menjalankannya.

Spesifikasi yang cocok: minimal CPU 4 core + RAM 8 GB (tanpa GPU); disarankan CPU 8 core + RAM 16 GB untuk pemakaian harian yang nyaman. Kecepatan tergantung CPU, kira-kira 5–20 token per detik pada laptop/PC biasa.