TheCacheIsALie/RidgeFill-Projector-Qwen3-8B

RidgeFill maps for Qwen3-8B: the reference study's maps at four split depths.

Fitted once in closed form. 49.6 more filled-region RULER points than the identity fill at the same split.

Data and code · All maps · llama.cpp fork · Citation

What this is

The ridge maps of the RidgeFill paper's reference study for Qwen3-8B (Qwen/Qwen3-8B, revision b968826d), plus the split-13 maps converted to GGUF for the llama.cpp fork.

Files

File	Split	Bytes	SHA256
safetensors/qwen3-8b-sweep-s9.safetensors	9 of 36	906,193,312	61715eddaf174aad918a283f3da59c4f6c2fe5847a39a295704c1834249a4302
safetensors/qwen3-8b-sweep-s13.safetensors	13 of 36 (primary)	771,942,480	945bb43c3def1349c3ddffcc29d6806898ef70fe3936ac0bbb484377bfeb8728
safetensors/qwen3-8b-sweep-s18.safetensors	18 of 36	604,128,936	f88f75a2ff937b74420ae3010f91b42a918e51d948a5c6f9d16dc20e1fc967ac
safetensors/qwen3-8b-sweep-s22.safetensors	22 of 36	469,878,096	d6420bf025bf13e3e3b6e8a8d2ee9a9fdd15aeafd36e2d565c2466f17527238e
qwen3-8b-s13-bf16.gguf	13 of 36, bf16	772,132,288	9632465ad720baa8a8816ab57e47a9f4e1c2f391c828ce37f11011050979558b
qwen3-8b-s13-q8_0.gguf	13 of 36, Q8_0	410,373,568	5b5238e19eadee904747f83b38db171947dc595b7cc8583e373fea2737d2e9d6

The safetensors files are the bf16 maps the paper's runs loaded, in the reference implementation (code/kva in the data repository). They were fitted in closed form on 845,043 rows with λ = 0.01 (0.003 at split 18).

Results

From the paper's reference study, with a 4,096-token exact tail. Qwen3-8B normalizes its keys, and its error sits in keys and values together.

Measure	Value
Filled-region RULER, RidgeFill − identity fill, split 13 (138 samples, 8K to 32K)	+49.6 [+43.3, +56.2] points
Key-token loss, RidgeFill − identity fill, split 13	−0.400 [−0.634, −0.159] nats
Novel-token loss vs exact, splits 9 / 13 / 18 / 22	+0.042 / +0.044 / +0.030 / +0.023 nats

On this model, attention-selected exact rows lower filled-region retrieval, from 93.7 to 89.9, while random rows of the same count do not; use the plain fill.

Use in llama.cpp

hf download TheCacheIsALie/RidgeFill-Projector-Qwen3-8B qwen3-8b-s13-q8_0.gguf --local-dir models
llama-server -m Qwen3-8B-Q4_K_M.gguf -c 32768 -ngl 99 -fa on \
    --ridgefill models/qwen3-8b-s13-q8_0.gguf --ridgefill-tail 8192

Build the fork as described in its RIDGEFILL.md. Only the plain fill with an exact tail applies to this model; the selection modes of the Qwen3.8-27B port do not.

Measured in llama.cpp

Earlier measurements of the split-13 GGUF, separate from the paper, from a partial run. One RTX 4090 24 GB, CUDA 12.4, flash attention on, F16 KV cache, wikitext-2 test text, T = 4096. The run stopped before the speed measurements, so prompt-processing speed for this model is not measured.

llama-perplexity (-c N; effective tail max(T, N/2)), stock → RidgeFill:

Model, maps	-c 8192	-c 16384	-c 32768
F16, bf16 maps	8.1460 → 9.2040	7.8723 → 8.5907	not run
F16, Q8_0 maps	8.1460 → 9.1869	7.8723 → 8.5772	not run
Q4_K_M, bf16 maps	8.2910 → 9.3850	8.0030 → 8.8144	8.6749 → 9.2728
Q4_K_M, Q8_0 maps	8.2910 → 9.3602	8.0030 → 8.8038	8.6749 → 9.2676

Tail NLL (last 4,095 tokens of each window), Q4_K_M, 8K, 35 windows: 2.112 → 2.236 nats per token. Agreement of this engine with the PyTorch reference implementation (bf16 GGUF, last 256 positions): KL 0.0016 at 8K (engine noise floor 0.0008) and 0.0018 at 16K (floor 0.0010). Output is not identical to stock llama.cpp.

How RidgeFill works

The lower layers run exactly up to a split. For every prompt row except the exact ones, a ridge map per later layer predicts that layer's input from the split layer's state, and the layer writes its cache from the prediction (b) instead of running in full: its own key and value projections run unchanged, and on a Gated DeltaNet layer so does its state update. The maps are fitted once, in closed form, from one pass over calibration text. The last few thousand prompt tokens, any selected exact rows and every decoding step run the full model. Nothing is trained and no token is dropped. The design comes from DeepSeek-V4.1-Flash, where the projection is trained into the model; (a) is kishida's Q3-8B-KVA-Projector, which retrofitted it onto Qwen3-8B with trained key and value heads.

Citation

The paper is RidgeFill: Training-Free Late-Layer Cache Fill for Faster Prefill by Dylan Johnston and Robert P Smith (preprint, 2026, doi:10.5281/zenodo.23890247). Cite it as:

@misc{johnston2026ridgefill,
  author    = {Johnston, Dylan and Smith, Robert P},
  title     = {{RidgeFill}: Training-Free Late-Layer Cache Fill for Faster Prefill},
  year      = {2026},
  publisher = {Zenodo},
  note      = {Preprint},
  doi       = {10.5281/zenodo.23890247},
  url       = {https://doi.org/10.5281/zenodo.23890247}
}

To cite the data and code themselves, use the data release's citation (Approximate-Science/ridgefill-data).

An earlier technical note is archived at doi:10.5281/zenodo.23179168. The paper replaces some of its numbers; SUPERSEDED.md in the data repository lists which.