Baekpica/Inkling-Small-GGUF

Inkling-Small — intermediate GGUF artifacts

Intermediate artifacts for Inkling-Small Mixed Quant. Independent conversion from thinkingmachines/Inkling-Small revision 8cc5877b44d343f88b92086aa1fb72897950f06a.

The MTP-BF16 sidecar is used with the separate MQ85GB main artifact in ds4-dfm-rs's DGX Spark integration. All eight draft layers have short CLI and serial HTTP MTP on/off checks. This does not qualify the 262.704 GiB Q8_0 main artifact for native serving. No llama.cpp compatibility, GB10 throughput or long-context result is claimed.

Artifact	Tensors	Shards	Size
Q8_0 reference, protected BF16 ingress/attention	888	20	262.704 GiB
MTP BF16 sidecar, all 8 dense draft layers	160	1	4.183 GiB

The Q8_0 reference uses Q8_0 for embedding/head, dense MLP matrices, shared experts and all routed experts. Attention, vision/audio, norms, convolutions, router weights/bias/scales preserve their original BF16/F32 bytes. It is a Q8 matrix reference with native anchors, not a uniformly Q8 model. It was quantized directly from the original BF16 checkpoint, without imatrix. MTP is kept separately and is not required for ordinary decoding.

general.architecture = inkling, inkling.tensor_layout = source-interleaved-v1. Every tensor keeps its original checkpoint name, shape and row order. w13 rows are [gate0, up0, gate1, up1, ...]; experts are never removed, merged or reordered. All 42 decoder layers and 256 experts per sparse layer are present. Full tokenizer/config/processor JSON and chat template are embedded in the first shard and also supplied as sidecars. This is a custom explicit contract, not an existing llama.cpp tensor-name convention.

Each artifact directory contains tensor-plan.json, audit.json and SHA256SUMS. Audits check all tensor names/shapes/types/byte counts, native payload bit equality, sampled Q8 dequantization error and every file hash. These structural checks do not establish model-output quality or runtime support.

hf download Baekpica/Inkling-Small-GGUF --include 'Q8_0/*' --local-dir ./inkling
cd inkling/Q8_0
sha256sum -c SHA256SUMS

Source: Thinking Machines Inkling-Small, Apache-2.0. GGML quantizers: Baekpica/ds4-dfm-rs, pinned 0bf398f459987c611293aa889772c80e272f045d.