olaverse/mist-qg-1.5b

mist-qg-1.5b

A compact multilingual question generator across 25 languages. Given a passage, it produces natural, search-style questions that the passage directly answers — the model is dual-use: a sellable /v1/generate-questions endpoint, and the data factory that mints (query, positive) training pairs for retriever and reranker fine-tuning, including several African languages underserved by existing tools. At ~1.5B parameters it runs comfortably on a single modest GPU.

📄 Model details

Property	mist-qg-1.5b
Type	Decoder-only LM, structured JSON generation
Total parameters	~1.5B
Backbone	Qwen/Qwen2.5-1.5B-Instruct
Output	{"questions": ["...", "...", "..."]}
Max sequence length	3072 (training)
Training precision	BF16
Languages	en, fr, de, es, pt, it, nl, ru, pl, tr, vi, id, hi, ja, ko, yo, ig, ha, sw, am, zu, xh, sn, so, af (25)
License	Apache-2.0

Training: fine-tuned on olaverse/qg-passages-multi (~50k passages, ~150k questions) distilled from CohereLabs/aya_collection_language_split via Qwen/Qwen2.5-32B-Instruct, with each generated question verified by round-trip retrieval before being kept for training (a question is discarded unless it retrieves its own source passage out of [source + distractors], embedded with Qwen/Qwen3-Embedding-0.6B).

🏃 How to run

Install transformers:

pip install -U transformers

The model expects a system + user message pair and returns strict JSON:

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "olaverse/mist-qg-1.5b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

passage = "Tides are caused by the gravitational pull of the moon and, to a lesser extent, the sun, acting on Earth's oceans."
n, language = 3, "English"

messages = [
    {"role": "system", "content": "You write search-style questions that a passage directly answers."},
    {"role": "user", "content": f'''You are given a passage. Write {n} questions that the passage directly answers.

Rules:
- Each question must be answerable using ONLY this passage.
- Vary the type: factual, yes/no, and a comparison or "why/how".
- Natural, like a real user search query. Do NOT write "according to the passage".
- Write the questions in {language}.

Return ONLY JSON: {{"questions": ["...", "...", "..."]}}

Passage: {passage}'''},
]

input_ids = tokenizer.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(input_ids, max_new_tokens=200, do_sample=False,
                     pad_token_id=tokenizer.pad_token_id or tokenizer.eos_token_id)
text = tokenizer.decode(out[0][input_ids.shape[1]:], skip_special_tokens=True)
questions = json.loads(text[text.index("{"): text.rindex("}") + 1])["questions"]
print(questions)
# ["What causes ocean tides?", "Does the sun affect tides?",
#  "Which has a bigger effect on tides, the moon or the sun?"]

For production serving, wrap the same prompt behind vLLM with guided JSON decoding so the output is structurally guaranteed valid, not just usually valid.

📈 Performance

Round-trip keep-rate on 625 passages held out from training (never seen during fine-tuning): a generated question counts as "kept" if it retrieves its own source passage out of a pool of distractors (top-1), embedded with Qwen/Qwen3-Embedding-4B. This is an in-house benchmark (olaverse/qg-eval-multi-fresh), not a third-party/standardized one.

Language	NDCG-style keep-rate	Questions scored
English (en)	1.000	75/75
Spanish (es)	1.000	75/75
Portuguese (pt)	1.000	75/75
Turkish (tr)	1.000	75/75
Indonesian (id)	1.000	75/75
Afrikaans (af)	1.000	75/75
Italian (it)	0.987	74/75
Hindi (hi)	0.987	74/75
Japanese (ja)	0.987	74/75
French (fr)	0.973	73/75
German (de)	0.973	73/75
Dutch (nl)	0.973	73/75
Vietnamese (vi)	0.973	73/75
Korean (ko)	0.972	70/72
Russian (ru)	0.960	72/75
Xhosa (xh)	0.850	51/60
Swahili (sw)	0.899	62/69
Zulu (zu)	0.776	52/67
Yoruba (yo)	0.773	58/75
Hausa (ha)	0.768	53/69
Amharic (am)	0.700	21/30
Igbo (ig)	0.680	51/75
Somali (so)	0.613	46/75
Shona (sn)	0.580	40/69
Overall	0.955	1706/1786

High-resource languages cluster at 0.95–1.00; the model's weakest languages are Amharic, Somali, and Shona (0.58–0.70) — treat outputs in these three with lower confidence than the rest of the set. This gap tracks limited fine-tuning data volume (~2,000 source passages/language) more than a fixed model-capacity ceiling, and is a natural target for a future data-scaling pass.

License

Released under Apache-2.0.

Citation

@misc{mist-qg-1.5b,
  title  = {mist-qg-1.5b},
  author = {Olaverse},
  year   = {2026},
  url    = {https://huggingface.co/olaverse/mist-qg-1.5b}
}