simonfxr/turnsense.cpp-GGUF

TurnSense.cpp GGUF

Canonical F32 and selective Q8_0 GGUF artifacts for TurnSense.cpp, a native C/C++ end-of-utterance classifier built on official upstream ggml.

These files are converted from latishab/turnsense at immutable revision 1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63. The model architecture and weights are unchanged except for deterministic LoRA merging and, for the Q8_0 artifact, weight quantization. The production runtime has no ONNX Runtime or Python dependency.

Files

File	Purpose	Size	SHA-256
turnsense-q8_0.gguf	Recommended CPU/Vulkan runtime model	145,012,704 bytes	e978f2462e1887c2959173066e2f534f3c02fea686323bfc454aa37842970966
turnsense-f32.gguf	Canonical conversion and quantization source	540,047,168 bytes	0eb7d0ea7bccbcd2f9aa4b722ee202efbf53b4e430ba90c527548f8578d2460e

SHA256SUMS contains the same checksums in machine-readable form.

Q8_0 policy

The Q8_0 artifact is produced from the canonical F32 GGUF with the native turnsense_quantize tool and upstream ggml_quantize_chunk:

• Q8_0: token_embd.weight and all 210 transformer attention/FFN projection matrices.
• F32: all 61 RMS normalization vectors and classifier.weight.

This quantizes 211 tensors and retains 62 tensors as F32. Tensor payload drops from 513.14 MiB to 136.40 MiB, a 3.76x reduction. Repeated conversion produces a byte-identical Q8_0 file. The runtime validates this policy exactly and rejects incompatible layouts.

The dynamically quantized upstream ONNX model is not the source of this Q8_0 artifact.

Usage

Download the recommended model:

hf download simonfxr/turnsense.cpp-GGUF turnsense-q8_0.gguf \
  --local-dir models

Build and run the native runtime:

git clone --recurse-submodules https://github.com/simonfxr/turnsense.cpp.git
cd turnsense.cpp
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

./build/turnsense_cli \
  --model models/turnsense-q8_0.gguf \
  --backend cpu \
  --json "Could you send that report tomorrow?"

Use --backend vulkan for the Vulkan backend. The stable C API is documented in the source repository README.

TurnSense operates on text, not audio. A voice agent should evaluate the latest punctuated STT transcript at candidate pauses and combine prob_eou with VAD, latency, and product policy.

Validation

The runtime graph uses official ggml operations and optimized backend kernels. For Q8_0, it verifies that every quantized get_rows and mul_mat node remains on the selected CPU or Vulkan backend rather than unexpectedly falling back.

Measured maximum absolute probability differences on the bundled seven-case fixture suite:

Backend / artifact	Reference	Maximum delta	Classifications
CPU F32	FP32 ONNX Runtime	9.84e-7	identical
Vulkan F32	FP32 ONNX Runtime	5.69e-4	identical
CPU Q8_0	native CPU F32	0.04161	identical
Vulkan Q8_0	native Vulkan F32	0.01413	identical

Vulkan was validated on an AMD Radeon RX 7900 XTX. Backend floating-point accumulation is not expected to be bit-identical.

Prompt and labels

The runtime prepends the literal prompt prefix <|user|> and applies the embedded GPT-2 byte-level BPE tokenizer. Do not append <|im_end|>.

• Label 0: NON_EOU
• Label 1: EOU

Applications should normally use prob_eou with a product-specific threshold rather than treating argmax as a fixed policy.

Limitations

• The upstream model is English-focused.
• Predictions are sensitive to punctuation and STT transcript quality.
• Turn-taking decisions should also incorporate VAD, timing, and application context.
• The fixture suite validates conversion parity; it is not a broad task-quality benchmark.
• This model is not intended for safety-critical decisions.

Provenance

• Upstream source: https://github.com/latishab/turnsense
• Upstream source revision: b40a25b4da94c961b64393752e507f59295061ad
• Upstream model: https://huggingface.co/latishab/turnsense
• Upstream model revision: 1ddc8f679abf3d9a42a93373b0e709f0c9d7fe63
• Runtime source: https://github.com/simonfxr/turnsense.cpp

Full source artifact hashes and deterministic conversion instructions are in docs/model-provenance.md.

License

The model artifacts are distributed under the Apache License 2.0, matching the upstream TurnSense model. See LICENSE. The TurnSense.cpp runtime source is separately distributed under the MIT License.