Akahsizrr/boosted-v1-small

Boosted v1 Small

A LoRA adapter for DeepSeek-R1-Distill-Qwen-1.5B, trained via the iloptimus self-improvement pipeline on HumanEval v1.

Results

Benchmark	Baseline	After Training	Improvement
HumanEval v1	24.0%	70.88%	+46.88%

The model trained itself on its own benchmark traces to beat its baseline score by nearly 3x.

Training Details

• Base model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
• Backend: MLX (int4 QLoRA)
• Method: SFT (supervised fine-tuning on self-generated reasoning traces)
• LoRA config: rank=8, scale=20.0, layers=8
• LoRA targets: self_attn.q_proj, self_attn.v_proj, self_attn.o_proj
• Optimizer: AdamW
• SFT iterations: 50
• Learning rate: 2e-4
• Max sequence length: 512
• Benchmark tasks: 25 (HumanEval v1)
• Trainable parameters: 507,904
• Trained tokens: 11,890
• Final SFT loss: 0.0007

Usage

This is an MLX LoRA adapter. Load it with the base model deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B:

from mlx_lm import load, generate

model, tokenizer = load(
    "deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B",
    adapter_path="Akahsizrr/boosted-v1-small",
)

Run ID

8a9f00ec86da