Sparset/Thinklite-27B-GGUF

Thinklite-27B in a 16.5 GB Q4_K_M that keeps its efficiency. At reasoning_effort="xhigh", this Q4 file uses 48.4% less reasoning than Qwen3.8-27B quantized the same way and keeps 99.3% of its average accuracy (90.9 against 91.5), so a complete answer arrives about 1.9x sooner for one user. At reasoning_effort="low" it uses 24.8% less reasoning and keeps 96.7% of the base model's accuracy.

Check out the full-precision Thinklite-27B, which cuts 47% of reasoning tokens at 99.6% of Qwen3.8-27B's accuracy.

Model	Size	xhigh: reasoning vs base	xhigh: quality retained	xhigh: TTFO speed-up	low: reasoning vs base	low: quality retained	low: TTFO speed-up
Thinklite-27B Q4_K_M (this repo)	16.5 GB	↓ 48.4%	99.3%	1.95x	↓ 24.8%	96.7%	1.26x
Qwen3.8-27B Q4_K_M	16.5 GB	baseline	100%	1.00x	baseline	100%	1.00x

• Both files were quantized here with the identical recipe and run with the same llama.cpp build, settings and seeds, on 200 GSM8K, 30 AIME 2025, 150 MMLU-Pro and 200 IFEval questions sampled from our frozen benchmark suite, plus all 131 LiveCodeBench v6 problems and all 1,240 BFCL v4 core cases. Every number compares Thinklite-27B Q4_K_M with Qwen3.8-27B Q4_K_M.
• Reasoning vs base: change in mean reasoning tokens per answer against the base model, averaged over the six benchmarks. Quality retained: Thinklite's average accuracy as a share of the base model's. TTFO speed-up (time-to-final-output): how much sooner one user gets the complete answer, thinking included. Per benchmark it is the base model's total output tokens divided by Thinklite's, scaled by the measured single-user speed; the table shows the average over the six benchmarks. On this GPU both Q4 files decode at the same speed (69.46 vs 70.09 tokens/s for one user).

Side-by-side comparisons (full-precision Thinklite-27B)

These two videos come from the full-precision (BF16) Thinklite-27B served with vLLM, not from this Q4 file. They show what the model does; the tables on this page show what the Q4 file keeps of it.

Debugging

Full-precision Qwen3.8-27B and Thinklite-27B get the same debugging prompt (retry code plus two logs: find the duplicate charges to refund and the fix) with the same seed, each model on one NVIDIA RTX PRO 6000 serving 20 requests at once. Both answer correctly; Thinklite-27B is done in 2:36.9 with 3,381 tokens, the base model in 11:04.9 with 14,313 tokens (4.2x). The video plays at 22x speed and the clocks show the measured time. This is the largest speed-up of 20 paired runs of this prompt. Thinklite-27B was correct in 19 of the 20 runs and the base model in all 20; over the 19 correct runs the median speed-up was 2.7x.

Building a 3D physics lab

Both full-precision models get the same objective: build an interactive 3D science lab as a single HTML file in which everything has to follow realistic physics. Objects have real sizes and masses and fall under real gravity (9.81 m/s²): a hinged door swings with its momentum and stops at its frame, glassware tips over when bumped and shatters when it hits something hard, a rubber ball bounces high while a heavy steel ball barely bounces, a block tower topples and a pendulum swings. Both models run at reasoning_effort="xhigh" on the same NVIDIA RTX PRO 6000. The numbers are the total tokens and generation time each model needed to reach a working version of the lab: Thinklite-27B 87,134 tokens in 56.7 min, the base model 127,661 tokens in 87.6 min (1.5x). The video shows one working game per model; it is not a measure of how often each model succeeds.

How it works

Reasoning models spend much of their output on thinking that does not change the answer: restating the question, repeating a step, re-checking a result that was already checked. Thinklite-27B is fine-tuned from Qwen3.8-27B on its own verified-correct answers to reach the same answers by a shorter route. Only the thinking gets shorter; the visible answer keeps its full length and format.

This repository is that model quantized to 4 bits for llama.cpp, Ollama and LM Studio. The quantization uses an importance matrix: before rounding, the full-precision model is run over a broad calibration text, and the rounding then protects the weights whose inputs matter most. The file is a standard Q4_K_M; only the choice of rounded values differs.

Benchmarks

Setup:

• Both Q4_K_M files were made here with the identical recipe: llama.cpp v0.8.0 (commit 8642541b255b), BF16 GGUF from the original weights, the same importance matrix calibration, then llama-quantize --imatrix to Q4_K_M. Qwen3.8-27B: revision 1d4bf0f2ff60; Thinklite-27B: revision 52b8fbe1f025. For LiveCodeBench and BFCL the Qwen3.8-27B Q4 file was rebuilt with the same recipe on another GPU. It is not byte-identical to the file used for the other four benchmarks (its importance matrix differs in the last bits) but sits at the same distance from BF16 (mean KL divergence 0.0143 against 0.0148). Thinklite's Q4 file is this repository's file in both runs.
• Served with llama-server, q8_0 KV cache, 16 parallel requests, temperature 1.0, top_p 0.95, top_k 20, the same seed per question for both models.
• 200 GSM8K, 30 AIME 2025, 150 MMLU-Pro and 200 IFEval questions sampled from our frozen benchmark suite, plus all 131 LiveCodeBench v6 problems and all 1,240 BFCL v4 core cases, one answer per question, scored with the suite's checkers (math-verify, option letter, lm-eval IFEval prompt-level strict, the official LiveCodeBench evaluator on public and private tests, the official BFCL AST checker). Answers that hit their cap (32,768 or 49,152 tokens, 65,536 for LiveCodeBench) were rerun once with a 131,072-token cap; truncated or unparseable answers count as incorrect.

reasoning_effort="xhigh"

Model	GSM8K	AIME 2025	MMLU-Pro	IFEval	LiveCodeBench	BFCL	Average
Accuracy (%)
Thinklite-27B Q4_K_M	96.0	93.3	84.0	94.5	89.3	88.2	90.9
Qwen3.8-27B Q4_K_M	95.5	100.0	83.3	95.5	87.0	87.7	91.5
Quality retained by Thinklite-27B Q4_K_M
vs Qwen3.8-27B Q4_K_M	100.5%	93.3%	100.8%	99.0%	102.6%	100.6%	99.3%
Reasoning tokens of Thinklite-27B Q4_K_M
vs Qwen3.8-27B Q4_K_M	↓ 67.4%	↓ 49.5%	↓ 69.4%	↓ 35.6%	↓ 33.6%	↓ 34.7%	↓ 48.4%
TTFO speed-up of Thinklite-27B Q4_K_M
vs Qwen3.8-27B Q4_K_M	2.42x	1.94x	2.93x	1.42x	1.48x	1.52x	1.95x

Mean reasoning tokens, paired differences and invalid answers (Q4 files)

Benchmark	Thinklite-27B Q4	Qwen3.8-27B Q4	Thinklite - base, points (95% CI)	Only base right / only Thinklite right	Invalid answers (base / Thinklite)
GSM8K (200)	201	617	+0.5 [-1.0, +2.5]	1 / 2	none / none
AIME 2025 (30)	9,207	18,226	-6.7 [-16.7, +0.0]	2 / 0	none / none
MMLU-Pro (150)	1,165	3,805	+0.7 [-4.0, +5.3]	6 / 7	none / none
IFEval (200)	1,102	1,712	-1.0 [-4.0, +2.0]	6 / 4	none / none
LiveCodeBench (131)	17,032	25,659	+2.3 [-2.3, +7.6]	4 / 7	truncated 1 / truncated 2
BFCL (1,240)	132	202	+0.6 [-0.9, +2.0]	40 / 47	unparseable 4 / unparseable 4

reasoning_effort="low"

Model	GSM8K	AIME 2025	MMLU-Pro	IFEval	LiveCodeBench	BFCL	Average
Accuracy (%)
Thinklite-27B Q4_K_M	96.0	76.7	81.3	86.0	72.5	90.6	83.8
Qwen3.8-27B Q4_K_M	97.0	86.7	82.7	86.0	77.1	90.6	86.7
Quality retained by Thinklite-27B Q4_K_M
vs Qwen3.8-27B Q4_K_M	99.0%	88.5%	98.4%	100.0%	94.1%	100.0%	96.7%
Reasoning tokens of Thinklite-27B Q4_K_M
vs Qwen3.8-27B Q4_K_M	↓ 25.4%	↓ 28.9%	↓ 29.1%	↓ 32.9%	↓ 9.5%	↓ 22.7%	↓ 24.8%
TTFO speed-up of Thinklite-27B Q4_K_M
vs Qwen3.8-27B Q4_K_M	1.20x	1.35x	1.32x	1.33x	1.07x	1.26x	1.26x

Mean reasoning tokens, paired differences and invalid answers (Q4 files)

Benchmark	Thinklite-27B Q4	Qwen3.8-27B Q4	Thinklite - base, points (95% CI)	Only base right / only Thinklite right	Invalid answers (base / Thinklite)
GSM8K (200)	141	189	-1.0 [-2.5, +0.0]	2 / 0	none / none
AIME 2025 (30)	7,581	10,669	-10.0 [-23.3, +3.3]	4 / 1	empty 2 / none
MMLU-Pro (150)	806	1,137	-1.3 [-4.7, +2.0]	4 / 2	none / none
IFEval (200)	523	779	+0.0 [-5.0, +5.0]	13 / 13	empty 2 / empty 1
LiveCodeBench (131)	8,943	9,885	-4.6 [-13.0, +3.8]	19 / 13	unparseable 8 / unparseable 5
BFCL (1,240)	75	97	+0.0 [-1.1, +1.1]	25 / 25	unparseable 4 / unparseable 9

Usage

llama.cpp

llama-server -hf Sparset/Thinklite-27B-GGUF:Q4_K_M --jinja --reasoning-format deepseek -c 65536

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="-")
reply = client.chat.completions.create(
    model="thinklite",
    messages=[{"role": "user", "content": "How many primes are there below 100?"}],
    temperature=1.0, top_p=0.95,
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "xhigh"}},
)
print(reply.choices[0].message.content)

--jinja uses the model's own chat template, which is what reads reasoning_effort ("low", "medium" or "xhigh", exactly as with Qwen3.8-27B). --reasoning-format deepseek returns the thinking separately from the answer. Long xhigh answers on hard problems can exceed 32k tokens, so give each request enough context.

Ollama

ollama run hf.co/Sparset/Thinklite-27B-GGUF:Q4_K_M

Images: this file is the language model only. Thinklite-27B's vision encoder is byte-for-byte Qwen3.8-27B's, so the vision projector (mmproj) of a Qwen3.8-27B GGUF works with it, for example --mmproj with the projector from ggml-org/Qwen3.8-27B-GGUF.

Files

File	Size	Notes
Thinklite-27B-Q4_K_M.gguf	16.55 GB	Q4_K_M with the importance matrix; language model only (the multi-token-prediction head and vision encoder are not included)
Thinklite-27B.imatrix.gguf	27.3 MB	The importance matrix used, for making other sizes with llama-quantize --imatrix
conversion_manifest.json		Source revision, llama.cpp commit, calibration files and hashes, KL divergence and benchmark results

SHA-256 checksums

File	Bytes	SHA-256
Thinklite-27B-Q4_K_M.gguf	16,547,400,448	f708315c5bbe27b494ec263192fd4b567812a59323e8638855dbe2e7ff126cde
Thinklite-27B.imatrix.gguf	27,300,192	6eb0e6a2b86fe30090fbabdaed7b40b2ba3f1c1ab0938da8aa132928bc8c49e1

Limitations

• GSM8K, AIME 2025, MMLU-Pro and IFEval are samples of their benchmarks, one answer per question; AIME 2025 has 30 questions, so one answer moves it by 3.3 points. The 95% intervals in the details above show which differences are within noise.
• Thinklite-27B Q4 is below Qwen3.8-27B Q4 where the tables show it, most clearly on AIME 2025 and, at low effort, LiveCodeBench. See the full-precision card for the full-precision model, measured with vLLM.
• No vision encoder or multi-token-prediction head in this file; use a Qwen3.8-27B projector for images (see Usage).
• No safety tuning beyond the base model's. Check the model on your own tasks before relying on it.

License

Apache-2.0, the license of the base model Qwen/Qwen3.8-27B.