RationalPursuit/Qwen3-4B-Base-Stratos-R1-Improved-SFT

Qwen3-4B-Base — R1 Stratos improved-training SFT

The final step-934 model from Moonshine's improved_training experiment: completion-only supervised fine-tuning of Qwen/Qwen3-4B-Base on 14,934 DeepSeek-R1 Stratos traces. Root files contain the merged bf16 model and its tokenizer. This repository contains the standalone merged inference export.

Training

• Exact base revision: 906bfd4b4dc7f14ee4320094d8b41684abff8539.
• Data: the R1 training split of the project's private RationalPursuit/calibration-zoo-stratos corpus, sourced from Bespoke Stratos.
• One epoch, 14,934 examples, 934 optimizer steps; effective batch size 16.
• Completion-only loss; complete responses up to 24,576 tokens, without truncation.
• LoRA rank 16, alpha 32, dropout 0.05 on all attention/MLP projections.
• Trainable tied embedding/output rows for <|im_start|>, <|im_end|>, <think> and </think>; these rows are included in the merged weights.
• Initial learning rate 2e-4, linear schedule, warmup ratio 0.03, AdamW.
• DeepSeek terminal markers were removed before Qwen tokenization. Responses supervise Qwen <|im_end|>; generation accepts both that token and native EOS.
• Held-out R1 validation loss on 64 traces: 0.76775 → 0.52240.

Training completed the original learning-rate schedule. Exact corpus revision, file hashes, training settings and merged-file checksums are recorded in metadata/. The corpus itself is not included in this model repository.

Matched MATH evaluation

A fixed 50-question subset of MATH-500, ten per difficulty level, was evaluated with one sampled completion per question. vLLM 0.10.2, bf16, temperature 0.6, top-p 0.95, top-k disabled, repetition penalty 1.0, 4,096 new tokens, both EOS IDs, and per-question seeds derived from 42. A generation stops when a block of 10–256 tokens repeats three consecutive times. All 50 requests run as one batch. Inspect scores the final boxed answer with math-verify; unfinished <think> blocks receive zero. There is no 120-second per-question deadline in this protocol.

Model	Correct / 50	Pass rate	Mean output tokens
Untouched Qwen3-4B-Base	42	84%	427
Step 900	39	78%	2,100
This model: step 934	40	80%	2,094

The final model had seven token-limit stops and two repetition stops; nine responses contained unfinished reasoning. All base responses terminated naturally. Earlier vLLM draws of the final model scored 74% and 80%, including variation despite configured per-question seeds. The original slow HF/PEFT evaluator scored 50% with half the responses hitting its 120-second deadline.

These results measure a small, repeatedly used development subset, not the complete MATH-500 benchmark. They do not demonstrate a capability improvement over the base at this generation budget. Training improves held-out teacher-response loss and produces substantially longer responses. Exact prompt matches against all 500 MATH problems were removed from training; this does not rule out paraphrases or pretraining contamination.

Inference

Tested with Transformers 4.57.6 and PyTorch 2.8.0+cu128. Load the merged root model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "RationalPursuit/Qwen3-4B-Base-Stratos-R1-Improved-SFT"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype=torch.bfloat16, device_map="auto"
).eval()
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Solve x + 3 = 7. Put your final answer in \\boxed{}."}],
    tokenize=False, add_generation_prompt=True,
)
inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=4096,
        do_sample=True, temperature=0.6, top_p=0.95, top_k=0,
        repetition_penalty=1.0,
        eos_token_id=[151645, 151643],
        pad_token_id=151643,
    )
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=False))

<|im_end|> is 151645; native <|endoftext|> is 151643. Both must terminate generation. The saved generation config is preserved from the training export and has a 2,048-token default; explicitly set 4,096 tokens to use the evaluated budget. The simple example above uses the token ceiling; the evaluation additionally uses the exact repetition guard described above. Model responses can contain <think> and </think> reasoning delimiters.

Files and provenance

• Root safetensors: merged step-934 weights for standalone inference.
• Root tokenizer/chat template and generation configuration: original final export.
• metadata/: training/data manifests, evaluation summaries and upload checksums.
• LICENSE: the Apache 2.0 license supplied with the pinned Qwen base.

This is a research model trained on reasoning traces, with no separate safety or alignment evaluation in this experiment. Reasoning can repeat or exhaust its output budget. Treat the benchmark numbers as measurements of the stated protocol.