RusNanoMath-800M

English | ะ ัƒััะบะธะน


English Overview

RusNanoMath-800M is an open-weights 800M parameter bilingual (Russian / English) mathematical reasoning model. Built on a hybrid Linear Attention architecture (Qwen3.5 with Gated Delta Rule layers) and aligned via targeted multi-stage SFT and SimPO (Simple Preference Optimization), it demonstrates very strong reasoning performance across competitive mathematics benchmarks, particularly in symbolic algebra.

Benchmark Visual Leaderboards

1. MATH-500 Overall Leaderboard

MATH-500 Overall Leaderboard

2. Dedicated Algebra Breakdown (MATH-500: Algebra)

MATH-500 Algebra Leaderboard

3. GSM8K Leaderboard

GSM8K Leaderboard


Highlights

  • Symbolic Algebra (MATH-500 Algebra): In the 124 algebra competition problems, RusNanoMath-800M achieves 52.4%, closely approaching Llama-3.1-8B (53.8%) and surpassing models with 1.5Bโ€“3B parameters (Qwen2.5-1.5B at 49.5%, Llama-3.2-3B at 48.0%, Gemma-2-2B at 41.2%).
  • General Competition Math (MATH-500 Overall): Across all 500 problems, RusNanoMath-800M reaches 36.2%, outperforming sub-1B to 2B models (Gemma-2-2B at 35.0%, SmolLM2-1.7B at 21.5%, Llama-3.2-1B at 14.0%).
  • Grade School Arithmetic (GSM8K): Achieves 52.0%, handling multi-step reasoning chains and arithmetic problems.

Quickstart & Inference

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

MODEL_ID = "itismax0/RusNanoML-800M"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForImageTextToText.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model.eval()

messages = [
    {
        "role": "user",
        "content": "ะะฐะนะดะธั‚ะต ะฒัะต ะดะตะนัั‚ะฒะธั‚ะตะปัŒะฝั‹ะต ะบะพั€ะฝะธ ัƒั€ะฐะฒะฝะตะฝะธั x^4 - 5x^2 + 4 = 0. ะžั‚ะฒะตั‚ ะทะฐะฟะธัˆะธั‚ะต ะฒ \\boxed{}."
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=1024,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

Run with llama.cpp / Ollama (GGUF, full precision)

A full-precision F16 GGUF (RusNanoML-800M-F16.gguf) is included in this repo โ€” no quantization loss.

# llama.cpp
llama-cli -hf itismax0/RusNanoML-800M:RusNanoML-800M-F16.gguf --temp 0 -p "ะะฐะนะดะธั‚ะต ะฒัะต ะดะตะนัั‚ะฒะธั‚ะตะปัŒะฝั‹ะต ะบะพั€ะฝะธ ัƒั€ะฐะฒะฝะตะฝะธั x^4 - 5x^2 + 4 = 0."

Ollama (import the GGUF locally):

echo "FROM RusNanoML-800M-F16.gguf
PARAMETER temperature 0
TEMPLATE \"{{ .Prompt }}\"" > Modelfile
ollama create rusnanoml-800m -f Modelfile
ollama run rusnanoml-800m "ะะฐะนะดะธั‚ะต ะฒัะต ะดะตะนัั‚ะฒะธั‚ะตะปัŒะฝั‹ะต ะบะพั€ะฝะธ ัƒั€ะฐะฒะฝะตะฝะธั x^4 - 5x^2 + 4 = 0."

Note: the model uses the Qwen3.5 hybrid linear-attention architecture. llama.cpp builds from master (>= b12413) support it; older builds will refuse to load the file.


ะžะฑะทะพั€ ะฝะฐ ั€ัƒััะบะพะผ

RusNanoMath-800M โ€” ะพั‚ะบั€ั‹ั‚ะฐั ะดะฒัƒัะทั‹ั‡ะฝะฐั (ั€ัƒััะบะธะน ะธ ะฐะฝะณะปะธะนัะบะธะน) ะผะพะดะตะปัŒ ะฝะฐ 800 ะผะปะฝ ะฟะฐั€ะฐะผะตั‚ั€ะพะฒ ะดะปั ั€ะตัˆะตะฝะธั ะผะฐั‚ะตะผะฐั‚ะธั‡ะตัะบะธั… ะธ ะปะพะณะธั‡ะตัะบะธั… ะทะฐะดะฐั‡. ะั€ั…ะธั‚ะตะบั‚ัƒั€ะฐ ะฟะพัั‚ั€ะพะตะฝะฐ ะฝะฐ ะฑะฐะทะต Qwen3.5 ั ะณะธะฑั€ะธะดะฝั‹ะผ ะปะธะฝะตะนะฝั‹ะผ ะฒะฝะธะผะฐะฝะธะตะผ (Gated Delta Rule), ั‡ั‚ะพ ะพะฑะตัะฟะตั‡ะธะฒะฐะตั‚ ะบะพะฝัั‚ะฐะฝั‚ะฝัƒัŽ ัะปะพะถะฝะพัั‚ัŒ ะฟะฐะผัั‚ะธ ัะพัั‚ะพัะฝะธั O(1) ะฟั€ะธ ะณะตะฝะตั€ะฐั†ะธะธ. ะžะฑัƒั‡ะตะฝะธะต ะฒะบะปัŽั‡ะฐะตั‚ ัั‚ะฐะฟั‹ ั†ะตะปะตะฒะพะณะพ SFT ะฝะฐ ั†ะตะฟะพั‡ะบะฐั… ั€ะฐัััƒะถะดะตะฝะธะน (CoT) ะธ ะฒั‹ั€ะฐะฒะฝะธะฒะฐะฝะธะต ะผะตั‚ะพะดะพะผ ะฟั€ะตะดะฟะพั‡ั‚ะตะฝะธะน SimPO (Simple Preference Optimization).

ะ“ั€ะฐั„ะธะบะธ ัั€ะฐะฒะฝะตะฝะธั ั ะพั‚ะบั€ั‹ั‚ั‹ะผะธ ะผะพะดะตะปัะผะธ

1. ะžะฑั‰ะธะน ะทะฐั‡ั‘ั‚: MATH-500

MATH-500 Overall Leaderboard

2. ะ ะฐะทะดะตะป ะฐะปะณะตะฑั€ั‹: MATH-500 (Algebra)

MATH-500 Algebra Leaderboard

3. ะั€ะธั„ะผะตั‚ะธะบะฐ ะธ ั‚ะตะบัั‚ะพะฒั‹ะต ะทะฐะดะฐั‡ะธ: GSM8K

GSM8K Leaderboard


ะžัะฝะพะฒะฝั‹ะต ะฟะพะบะฐะทะฐั‚ะตะปะธ

  • ะะปะณะตะฑั€ะฐ (MATH-500 Algebra): ะ’ ะฑะปะพะบะต ะธะท 124 ะทะฐะดะฐั‡ ะฟะพ ะฐะปะณะตะฑั€ะต RusNanoMath-800M ะฟะพะบะฐะทั‹ะฒะฐะตั‚ 52.4% (65 ั€ะตัˆั‘ะฝะฝั‹ั… ะทะฐะดะฐั‡ ะธะท 124). ะœะพะดะตะปัŒ ะฒะฟะปะพั‚ะฝัƒัŽ ะฟั€ะธะฑะปะธะถะฐะตั‚ัั ะบ ะฟะพะบะฐะทะฐั‚ะตะปัะผ Llama-3.1-8B (53.8%) ะธ ะฟั€ะตะฒะพัั…ะพะดะธั‚ ะผะพะดะตะปะธ ะฑะพะปัŒัˆะตะณะพ ั€ะฐะทะผะตั€ะฐ: Qwen2.5-1.5B (49.5%), Llama-3.2-3B (48.0%) ะธ Gemma-2-2B (41.2%).
  • ะžะฑั‰ะธะน ะทะฐั‡ั‘ั‚ MATH-500: ะŸะพ ะฒัะตะผัƒ ะฝะฐะฑะพั€ัƒ ะธะท 500 ะพะปะธะผะฟะธะฐะดะฝั‹ั… ะทะฐะดะฐั‡ ะผะพะดะตะปัŒ ะฝะฐะฑะธั€ะฐะตั‚ 36.2% (181 ั€ะตัˆั‘ะฝะฝะฐั ะทะฐะดะฐั‡ะฐ), ะพะฟะตั€ะตะถะฐั Gemma-2-2B (35.0%), Mistral-7B-Instruct (28.4%), SmolLM2-1.7B (21.5%) ะธ Llama-3.2-1B (14.0%).
  • GSM8K: ะะฐ ั‚ะตัั‚ะต ะผะฝะพะณะพัˆะฐะณะพะฒะพะน ะฐั€ะธั„ะผะตั‚ะธะบะธ ะผะพะดะตะปัŒ ะฟะพะบะฐะทั‹ะฒะฐะตั‚ 52.0% (260 ั€ะตัˆั‘ะฝะฝั‹ั… ะทะฐะดะฐั‡ ะธะท 500).

ะั€ั…ะธั‚ะตะบั‚ัƒั€ะฐ ะธ ะฟะฐั€ะฐะผะตั‚ั€ั‹

  • ะŸะฐั€ะฐะผะตั‚ั€ั‹: 800 ะผะปะฝ (0.8B)
  • ะ’ะฝะธะผะฐะฝะธะต: ะ“ะธะฑั€ะธะดะฝะพะต ะปะธะฝะตะนะฝะพะต ะฒะฝะธะผะฐะฝะธะต (Gated Delta Rule) ัะพ ัะปะพะถะฝะพัั‚ัŒัŽ ะฟะฐะผัั‚ะธ ัะพัั‚ะพัะฝะธั O(1)
  • ะขั€ะตะฑะพะฒะฐะฝะธั ะบ ะฒะธะดะตะพะฟะฐะผัั‚ะธ: ะพะบะพะปะพ 1.6 ะ“ะ‘ ะฒ ั„ะพั€ะผะฐั‚ะต BF16 (ั€ะฐะฑะพั‚ะฐะตั‚ ะฝะฐ ะฟะพั‚ั€ะตะฑะธั‚ะตะปัŒัะบะธั… GPU ั 4 ะ“ะ‘ VRAM)
  • ะšะพะฝั‚ะตะบัั‚: ะดะพ 32 768 ั‚ะพะบะตะฝะพะฒ
  • ะ›ะธั†ะตะฝะทะธั: Apache 2.0
Downloads last month
178
Safetensors
Model size
0.9B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for RusMiniLM/RusNanoMath-800M

Quantizations
1 model

Evaluation results