ftpxain/PRISM-1B-MiniCPM

PRISM-1B: Frontier-Distilled MiniCPM5-1B

PRISM-1B is a high-density, reasoning-aligned 1.0B parameter language model adapter trained by distilling frontier signals (GPT-5, Claude Mythos, Gemini 3.5, and specialized STEM tracks) into openbmb/MiniCPM5-1B.

Trained entirely within an ultra-lean memory envelope on a consumer laptop GPU (NVIDIA GeForce RTX 4050 6 GB VRAM), PRISM demonstrates that disciplined dataset filtration, deduplication, and frontier distillation enable sub-2B models to achieve substantial leaps in programming execution, mathematical reasoning, and instruction adherence.

---

Verified Empirical Benchmarks

Evaluated independently on local hardware comparing openbmb/MiniCPM5-1B (base) against PRISM-1B using EleutherAI's lm-evaluation-harness and official evaluation suites:

Benchmark Metric	Domain	Base Model (openbmb/MiniCPM5-1B)	PRISM-1B (Fine-Tuned)	Delta	Evaluation Protocol
OpenAI HumanEval	Python Code Pass@1	68.0%	72.0%	+4.0%	lm-evaluation-harness (humaneval), subprocess unit assertions
MATH-500	Competition Math Exact Match	0.0%	4.0%	+4.0% (Breakthrough)	lm-evaluation-harness (hendrycks_math500), symbolic LaTeX verification
OpenAI GSM8K	Multi-Step Math Reasoning	4.0%	32.0%	+28.0% (8x Gain)	Zero-shot Chain-of-Thought (CoT) with exact #### <number> extraction

Domain-Stratified Loss & Perplexity (Held-Out Test Split)

Capability Domain	Base Model Loss	Fine-Tuned Loss	Base Perplexity	Fine-Tuned Perplexity	PPL Reduction
Distilled Reasoning	3.4786	0.1886	32.42	1.21	-96.3%
Applied Engineering	5.7069	0.6922	300.93	2.00	-99.3%
Science & Physics	4.4634	0.6678	86.78	1.95	-97.8%
Python Coding	3.1639	0.7941	23.66	2.21	-90.7%
Instruction Following	4.3312	2.4131	76.04	11.17	-85.3%

---

Quickstart & Inference

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

base_id = "openbmb/MiniCPM5-1B"
adapter_id = "ftpxain/PRISM-1B-MiniCPM"  # or local path

tokenizer = AutoTokenizer.from_pretrained(base_id, trust_remote_code=True)

# 4-bit NF4 Quantization for consumer GPUs
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    base_id,
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
)

# Load PRISM PEFT adapter
model = PeftModel.from_pretrained(model, adapter_id)
model.eval()

# Chat format
messages = [
    {"role": "user", "content": "Write an async token bucket rate limiter in Python with type annotations."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,
        temperature=0.1,
        top_p=0.95,
        do_sample=True,
    )

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)

---

Training Hyperparameters

• Base Architecture: openbmb/MiniCPM5-1B
• Method: QLoRA (NF4 Quantization, 4-bit)
• LoRA Rank (r): 16, Alpha: 32, Dropout: 0.05
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Optimizer: Paged AdamW 8-bit
• Hardware Envelope: Single NVIDIA GeForce RTX 4050 Laptop GPU (6 GB VRAM)