MNN-Pocket-0.3B-RU-Chat (planned)

Status: not trained yet. This card is the plan for the second product; no weights exist. When training finishes this banner and the numbers below will be replaced.

Russian instruction-following assistant, built by SFT on top of Rahspide/MNN-Pocket-0.3B-RU.

Two models, two jobs

This repository is the assistant. Its sister, MNN-Pocket-0.3B-RU, is the continuation model — same base, same Russian, different job.

	Sister — MNN-Pocket-0.3B-RU	This model — MNN-Pocket-0.3B-RU-Chat
Job	continue text	follow instructions
Input	a prefix, often mid-sentence	a system + user message
Output	the plausible continuation	an answer
Trained with	continued pretraining on 1.2B tokens of raw text	instruction tuning on top of that model
Choose it for	keyboard suggestions, autocomplete, prefix completion	chat, Q&A, short rewrites
Do not choose it for	assistant behaviour: it will continue your question instead of answering it	autocomplete: it will answer instead of continuing

The split exists because the two tasks reward opposite behaviour. One checkpoint trained on both is mediocre at each: a continuation model that starts answering, and an assistant that leaks into completion mode.

Why start from the Russian continuation model

The base MNN-Pocket-0.3B is bilingual zh/en and has essentially no Russian; a direct SFT of the official instruct model would fight both problems at once (new language and new format). Starting from the continuation model means Russian is already paid for, and SFT only has to teach the dialogue shape.

A control run on the official taobao-mnn/MNN-Pocket-0.3B with the same SFT data is planned, so that "we lost the assistant in pretraining" can be told apart from "there was no assistant to lose".

Planned recipe

• Start point: Rahspide/MNN-Pocket-0.3B-RU (or its final checkpoint).
• Chat template: the model's own ChatML (<|im_start|> / <|im_end|>), untouched.
• Loss only on assistant tokens; system and user turns are masked context.
• Multi-turn dialogue kept as one example; packing with explicit boundaries.
• Three data volumes tried (≈5k / 20k / 50k examples) with early stopping on a held-out Russian split; one epoch per checkpoint kept, best picked on the held-out split rather than on the training loss.
• 10–20% English instruction replay from HuggingFaceH4/ultrachat_200k to keep bilingual instruction following; the exact share is tuned on an English sanity set.
• Learning-rate grid 1e-4 … 3e-4, full fine-tuning; LoRA is the fallback if full FT cannot fit.

Planned data

Dataset	Role	Licence
IlyaGusev/saiga_scored	core Russian instructions with quality scores	see dataset card
IlyaGusev/ru_turbo_alpaca	volume filler	CC-BY-4.0
HuggingFaceH4/ultrachat_200k	English replay, 10–20%	MIT

Rules: deduplicate by normalised text, drop examples shorter than ~16 tokens, exclude any dataset whose licence forbids redistribution of derivatives, and check for leakage into MERA test sets.

Planned evaluation

• Blind manual review of 50–100 Russian dialogues: instruction following, naturalness, correct refusal, 2–3 turn coherence.
• Selective MERA tasks that fit a 0.3B model's format and context — not the aggregate score.
• No regression against the continuation sister on Russian bits/char, and no collapse of English answers.

Limitations (expected)

• 0.36B parameters: short answers, weak facts, no reliable multi-step reasoning.
• Russian SFT data is mostly machine-translated or model-generated; style will be flatter than human chat.
• The sibling's Russian tokenizer cost applies here too: about 3.3 tokens per Russian word.
• Not a coding model, not a tool-calling model, no long-context dialogue.

Licence and attribution

Weights: Apache-2.0, inherited through MNN-Pocket-0.3B-RU from taobao-mnn/MNN-Pocket-0.3B-Base.