M1n1A1/MiniAI-Quata2-4b-v0.2-base

MiniAI Quata2 (4B) — v0.2 base

The Serbian CPT checkpoint behind Quata2 4B v0.2.

Foundation note: Quata2 v0.2 starts from the post-trained Qwen3.5-4B checkpoint, not from Qwen3.5-4B-Base. “Base” here means the base for the Quata2 SFT run, not Qwen's pretrained base model.

This repo holds the full merged bf16 checkpoint of Qwen3.5-4B after a 50M-token Serbian continued-pretraining pilot. It is the exact starting point for the Quata2 4B v0.2 SFT run. The instruction-tuned v0.2 SFT model is released separately as M1n1A1/MiniAI-Quata2-4b-v0.2.

Held-out loss

Lower is better. These are language-modelling losses from the CPT run, not benchmark scores.

Split	Loss
English	1.7632
BCMS Ijekavian	2.0607
Serbian Ekavian	2.2962

What this is

Foundation	Qwen3.5-4B (post-trained; not Qwen3.5-4B-Base)
Stage	Serbian continued pretraining (CPT)
Training tokens	50M-token Serbian pilot mix
Precision	bf16 safetensors (~8.7 GB)
Context	262,144 tokens in the config; trained at 2,048
SFT / DPO	Not included
Languages	English, Serbian, Croatian, Bosnian

The tokenizer and chat template are inherited from Qwen3.5-4B, but this checkpoint has not been instruction-tuned. It will autocomplete; it will not reliably follow instructions or hold a chat persona.

Use it

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "M1n1A1/MiniAI-Quata2-4b-v0.2-base"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")

prompt = "Beograd je glavni grad"
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=64)
print(tok.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

For a chat model, use the SFT release. For a fine-tune, start here and add your own SFT data.

Training

• Continued pretraining from the post-trained Qwen3.5-4B on a 50M-token Serbian pilot mix
• Full fine-tune, merged back to bf16
• No SFT, no DPO, no reasoning-trace tuning in this checkpoint
• The v0.2 SFT run starts from this merged model

License

Apache 2.0, inherited from Qwen3.5-4B.

Citation

@misc{miniai2026quata2v02base,
  title        = {MiniAI Quata2 4B v0.2 base},
  author       = {{MiniAI}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/M1n1A1/MiniAI-Quata2-4b-v0.2-base}},
  note         = {Serbian CPT checkpoint before SFT; base model Qwen3.5-4B}
}

Base model:

@misc{qwen3_5_4b,
  title        = {Qwen3.5-4B},
  author       = {{Qwen Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Qwen/Qwen3.5-4B}}
}