Llama-3.1-8B-SPoT

Model Description

Llama-3.1-8B-SPoT is a reasoning-enhanced large language model post-trained from the meta-llama/Llama-3.1-8B base model. It is trained using the Surgical Post-Training (SPoT) paradigm, which improves reasoning capabilities while alleviating the catastrophic forgetting typically associated with standard Supervised Fine-Tuning (SFT).

This model is the Llama 3.1 8B counterpart of linius/Qwen3-8B-SPoT, introduced alongside the SPoT methodology in the paper Surgical Post-Training: Proximal On-Policy Distillation for Reasoning with Knowledge Retention.

• Code Repository: Visual-AI/SPoT

Training Details & Performance

• Training paradigm: Surgical Post-Training (SPoT).
• Base model: meta-llama/Llama-3.1-8B.
• Focus: improving mathematical and step-by-step reasoning while preserving general capabilities.
• Format: sharded safetensors checkpoint compatible with Hugging Face transformers.
• Evaluation: benchmark details for this Llama 3.1 8B checkpoint will be added separately.

Usage

You can load and generate text with this model using Hugging Face transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "linius/Llama-3.1-8B-SPoT"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

prompt = "Solve the following math problem step-by-step: ..."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **inputs,
    max_new_tokens=2048,
    temperature=0.6,
    top_p=0.9,
)
response = tokenizer.decode(
    generated_ids[0][inputs.input_ids.shape[-1]:],
    skip_special_tokens=True,
)
print(response)

Files

• config.json
• generation_config.json
• model-00001-of-00007.safetensors through model-00007-of-00007.safetensors
• model.safetensors.index.json
• tokenizer files

training_args.bin is intentionally not required for inference and may be omitted from public uploads.

Citation

If you find this model or the SPoT methodology useful in your research, please consider citing:

@article{lin2026surgical,
      title={Surgical Post-Training: Proximal On-Policy Distillation for Reasoning with Knowledge Retention},
      author={Wenye Lin and Kai Han},
      year={2026},
      journal={arXiv preprint arXiv:2603.01683}
}

License

This checkpoint uses the Llama architecture and should be used in compliance with the Llama 3.1 Community License and applicable acceptable-use terms.