🏆 TinyChampion-3B-R1

TinyChampion-3B-R1 is a compact, high-efficiency 3-billion parameter reasoning language model developed by MAlexC.

Fine-tuned using distilled multi-step reasoning traces from DeepSeek-R1 (671B), TinyChampion-3B-R1 brings frontier-grade chain-of-thought <think> capabilities to edge devices and consumer GPUs (runs comfortably on <6 GB VRAM).

---

📊 Benchmark Results

Benchmark	Metric	TinyChampion-3B-R1	Base Model / Baseline 3B
GSM8K (Full 1,319 test set)	Strict Match (0-shot)	89.31%	~69.60%
GSM8K (Full 1,319 test set)	Flexible Match (0-shot)	89.16%	~69.83%
MMLU (57 subjects)	Accuracy (0-shot)	61.01%	60.12%
Hellaswag	Acc Norm (0-shot)	62.23%	61.46%
TruthfulQA	MC2 (0-shot)	48.93%	48.34%

---

🧠 Reasoning Format (<think>)

The model reasons through problems before providing the final answer:

<|im_start|>user
Janet pays 0/hour for 3 hours of tennis lessons. She also buys 2 rackets for 0 each. How much did she spend in total?<|im_end|>
<|im_start|>assistant
<think>
1. Cost of tennis lessons: 3 hours * 0/hour = 20.
2. Cost of 2 rackets: 2 * 0 = 20.
3. Total expenditure: 20 + 20 = 40.
</think>

The answer is 240.<|im_end|>

---

🚀 Quickstart (Transformers)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "MAlexCCBC/TinyChampion-3B-R1"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "system", "content": "You are TinyChampion, an expert reasoning AI assistant developed by MAlexC."},
    {"role": "user", "content": "Solve: What is the sum of all prime numbers between 10 and 30?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

🛠 Training Details

• Base Architecture: 3B Parameters
• Alignment: Distilled Long Chain-of-Thought (DeepSeek-R1 style) + Multi-task SFT
• Context Length: 2,048 tokens
• Precision: Pure 16-bit (bfloat16) merged weights

Developed by MAlexC.