tinyopsec/OctoThinker-8B-Long-Base-GGUF

OctoThinker-8B-Long-Base GGUF

GGUF quantizations of OctoThinker/OctoThinker-8B-Long-Base.

OctoThinker-8B-Long-Base is a mid-trained language model built on the Llama-3 family, designed to create a reinforcement learning–friendly base language model. It is trained using the Stable-then-Decay two-stage mid-training paradigm on large-scale mathematical datasets (MegaMath-Web-Pro-Max) and diversified Long Chain-of-Thought data, ensuring rapid convergence and improved RL stability.

For more details, see the paper.

---

Available Quantizations

File	Bits	Size (approx)	Use Case
model_f16.gguf	16	~16.1 GB	Maximum quality, reference
model_q8_0.gguf	8	~8.5 GB	Best quality, if VRAM allows
model_q6_k.gguf	6	~6.6 GB	Great quality
model_q5_k_m.gguf	5	~5.7 GB	Balanced quality/size
model_q5_k_s.gguf	5	~5.5 GB	Slightly smaller Q5
model_q4_k_m.gguf	4	~4.8 GB	Good quality, recommended
model_q4_k_s.gguf	4	~4.6 GB	Smaller Q4
model_q3_k_l.gguf	3	~4.0 GB	Low VRAM, decent quality
model_q3_k_m.gguf	3	~3.7 GB	Lower VRAM
model_q3_k_s.gguf	3	~3.5 GB	Minimum Q3
model_q2_k.gguf	2	~3.0 GB	Very low VRAM, reduced quality

---

VRAM Requirements

Quantization	VRAM (approx)
F16	~17 GB
Q8_0	~9.5 GB
Q6_K	~7.5 GB
Q5_K_M	~6.5 GB
Q4_K_M	~5.5 GB
Q3_K_M	~4.5 GB
Q2_K	~3.5 GB

---

Usage

llama.cpp

./llama-cli -m model_q4_k_m.gguf -p "Your prompt here" -n 512

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="model_q4_k_m.gguf", n_ctx=8192)
output = llm("Your prompt here", max_tokens=512)
print(output["choices"][0]["text"])

LM Studio

Download the desired .gguf file and load it directly in LM Studio.

Ollama

ollama run hf.co/tinyopsec/OctoThinker-8B-Long-Base-GGUF

---

Citation

@article{wang2025octothinker,
  title={OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling},
  author={Wang, Zengzhi and Zhou, Fan and Li, Xuefeng and Liu, Pengfei},
  year={2025},
  journal={arXiv preprint arXiv:2506.20512},
  note={Preprint}
}