Good catch, and yes: you can increase context to 256K. The 131072 limit was unfortunately baked into the GGUFs, so I had to re-upload the first shard of each quant. Nothing changed except metadata.
If you don't want to re-download the first shard, you can get the same result at runtime with the following command:
What happened: upstream's config.json ships max_position_embeddings: 131072, so that's what went into the GGUF. But inclusionAI's own model card documents a training schedule of 8K → 32K → 256K and reports benchmarks run at a 256K window. llama.cpp caps a slot to whatever context_length the GGUF declares and refuses to go past it —
the slot context (262144) exceeds the training context of the model (131072) - capping
— so any GGUF built straight from that config is hard-limited to 131072 no matter what you pass to -c.
Fixed: We've re-uploaded shard 00001 of each quant with context_length = 262144. Nothing else changed — same weights, same quantization, only that one metadata value.
Verified, not assumed: needle-in-a-haystack at depths 10/50/90% is 12/12 out to 247,583 tokens on all three quants. Worth noting the KV cache is very cheap here — only 7 of 42 layers are MLA and they're compressed — so the entire 256K window costs under 2 GiB. If you're short on VRAM, -ncmoe N (offloading expert blocks to CPU) is a much bigger lever than cutting context.
