Instructions to use chankhavu/smolmo-32b-sft-merged-proofpilot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use chankhavu/smolmo-32b-sft-merged-proofpilot with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="chankhavu/smolmo-32b-sft-merged-proofpilot")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("chankhavu/smolmo-32b-sft-merged-proofpilot")
model = AutoModelForCausalLM.from_pretrained("chankhavu/smolmo-32b-sft-merged-proofpilot", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use chankhavu/smolmo-32b-sft-merged-proofpilot with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "chankhavu/smolmo-32b-sft-merged-proofpilot"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chankhavu/smolmo-32b-sft-merged-proofpilot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/chankhavu/smolmo-32b-sft-merged-proofpilot

SGLang

How to use chankhavu/smolmo-32b-sft-merged-proofpilot with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "chankhavu/smolmo-32b-sft-merged-proofpilot" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chankhavu/smolmo-32b-sft-merged-proofpilot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "chankhavu/smolmo-32b-sft-merged-proofpilot" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "chankhavu/smolmo-32b-sft-merged-proofpilot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use chankhavu/smolmo-32b-sft-merged-proofpilot with Docker Model Runner:
```
docker model run hf.co/chankhavu/smolmo-32b-sft-merged-proofpilot
```

merged-soup8

A uniform weight-average ("model soup" / SWA) of the last 8 checkpoints of a single SFT run, created with mergekit. This is not a merge of independently fine-tuned models — it averages successive snapshots of one training trajectory.

Why this merge

The SFT run used a cosine LR schedule (5e-5 → 5e-6 floor, with warmup) targeting 24,000 steps, but was cut short at step 20,500, where the LR was still 7e-6 (i.e. mildly under-annealed relative to the 5e-6 floor).

Tail weight-averaging is used here as a proxy for the missing annealing. While the LR is nonzero, the optimizer orbits the basin minimum in a noisy distribution whose radius scales with the LR; annealing shrinks that orbit, and averaging snapshots along the trajectory estimates its center directly (the SWA / "latest weight averaging" effect). The merged checkpoints all sit in the converged, flat-loss region (training CE ≈ 0.398 from step 18,500 onward), so the basin center is stationary and the average is near-pure noise reduction rather than a blend of differently-trained weights.

Checkpoints merged

8 checkpoints, 250 steps apart, steps 18,750 → 20,500 (the converged tail):

step	LR (cosine, →5e-6 floor)
18,750	~7e-6 region
19,000
19,250
19,500
19,750
20,000
20,250
20,500	final (LR 7e-6, run cut here)

Step 18,500 was intentionally excluded to keep the window strictly within the flat-loss tail and avoid pulling in higher-LR / less-converged weights.

Merge Details

Merge Method

Linear (uniform weighted average), with normalize: true — the weighted sum is divided by the sum of weights, so equal weight: 1.0 on all 8 models yields the plain arithmetic mean (a uniform soup).

Precision

Merge math was run in bfloat16 and the output is stored in bfloat16 to match the source models. As a precision cross-check, the merge was re-run with fp32 accumulation (dtype: float32, out_dtype: bfloat16); the result was bit-for-bit identical to the bf16-accumulated merge across all 707 tensors (max |Δ| = 0.0). With only 8 terms, fp32 accumulation makes no difference once the result is cast back to bf16, so the bf16 merge is exact to the precision of the stored model.

Verification

All 14 shards present; model.safetensors.index.json consistent (707 tensors).
0 NaN / 0 Inf across all tensors.
Sampled tensors confirmed to equal the mean of the 8 sources to within one bf16 ULP (the unavoidable storage-rounding error).

Evaluation

⚠️ Note: because all source checkpoints sit at the loss floor, training CE is ~0.4 regardless and is not a useful signal for whether the soup helped. The soup's benefit (if any) is in generalization / flatness and only shows up on a held-out eval — compare merged-soup8 against the final checkpoint (step20500) there before adopting.

Configuration

The following YAML configuration was used to produce this model:

# Uniform tail soup (SWA) of the last 8 SFT checkpoints, steps 18750-20500.
# All in the converged flat-loss region; equal weights = standard SWA recipe.
models:
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step18750_20260619202843
    parameters: {weight: 1.0}
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step19000_20260619230955
    parameters: {weight: 1.0}
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step19250_20260620015122
    parameters: {weight: 1.0}
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step19500_20260620041731
    parameters: {weight: 1.0}
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step19750_20260620065846
    parameters: {weight: 1.0}
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step20000_20260620094025
    parameters: {weight: 1.0}
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step20250_20260620120632
    parameters: {weight: 1.0}
  - model: chankhavu/olmo_32b_fp8_niiattempt2_step20500_20260620144800
    parameters: {weight: 1.0}
merge_method: linear
dtype: bfloat16