AuraGo-Ling

An experimental Ling-3.0-tiny fine-tune for the native tool-call protocol of AuraGo. This is the final Q4_K_L export, not a general-purpose replacement for a larger assistant model.

Artifact

File	Bytes	SHA256
AuraGo-Ling-3.0-tiny-Q4_K_L.gguf	5,096,544,352	4c25f349d6ea6872907c6fbd827d4b90abfd420320394a8cf420ce9b60abee68

The original GGUF is published without conversion or metadata changes. Q4_K_L uses the reference mixed tensor allocation: 111 Q4_K, 30 Q5_K, 69 Q6_K, 83 Q8_0 and 233 F32 tensors. Its embedded chat template is required.

Training provenance

• Base: inclusionAI/Ling-3.0-tiny, revision b61f4338de3e68ffc9c0bc1ed5e902981a4a929e (MIT). The MoE model has approximately 7.9B total and 1.3B active parameters.
• Fresh corrective LoRA on 922 reviewed dialogues, including coverage of 57 newly identified AuraGo commands. Three epochs / 348 planned steps; the final export uses checkpoint 300.
• Training implementation: AReno revision a13b9325a4745c08d3da6c1e6ffb2ddef902dcba.
• Adapter SHA256: 0469d52493accf87b11e26cd5f8f457626b5b93a504208a61fd2fda82664c628.
• Quantization layout reference: bartowski/Ling-3.0-tiny-GGUF.

The training examples are synthetic/reviewed AuraGo protocol dialogues. This repository contains model weights and documentation; private session artifacts, credentials and raw operational logs are not part of the release.

Evaluation and limitations

The existing final Q4_K_L export evaluation scored 68/76 correction cases and 49/49 replay cases. Across the existing 140 regression/control turns, the export evaluation recorded no schema/protocol failures or truncated outputs. The unquantized adapter had scored 72/76 correction cases and 49/49 replay cases.

These are known synthetic regression cases with supplied conversation/tool-result history, not unseen benchmarks or tests that execute real tools. Nonempty general control responses do not establish factual accuracy. Known remaining failures include two display description/name cases, four premature Vercel provider choices and two German cron cases. General German knowledge about AuraGo's framework is unreliable. Validate tool arguments in the host application and retain AuraGo's permission checks and a regular fallback provider.

The reported speed of over 100 tokens/s on a GTX 1660 mobile belongs to an earlier Ling/engine experiment. It is not a measured result for this artifact. CUDA, SYCL and Vulkan runtimes remain experimental until their native Linux GPU acceptance tests have passed. Windows/WSL tests do not qualify a Linux backend.

Run with the pinned hybrid engine

Use llama-wackMall-hybrid at f37a34cd4e502284ca297e141a6c4013bd151b18 with AuraGo's startup and cache-isolation patch, or AuraGo's matching digest-pinned container. The unpatched fork can substitute cached conversation history for an unrelated API request. Example for a CUDA build:

export LLAMA_KVFLASH=0
export LLAMA_CMOE_PREFILL_BATCH=2048 LLAMA_CMOE_PREFILL_UBATCH=2048
export LLAMA_CMOE_DECODE_BATCH=64 LLAMA_CMOE_DECODE_UBATCH=64
llama-server -m AuraGo-Ling-3.0-tiny-Q4_K_L.gguf \
  --alias aurago-ling --host 127.0.0.1 --port 18081 \
  --api-key-file /run/secrets/llama-api-key \
  --ctx-size 16384 --parallel 1 --fit off \
  --n-gpu-layers all --no-cpu-moe --batch-size 64 --ubatch-size 64 \
  --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on \
  --backend-sampling --spec-type none --reasoning off --jinja \
  --chat-template-kwargs '{"enable_thinking":false}'

Create the API-key file privately before starting the server. The managed AuraGo runtime handles that through its Vault. CUDA kernel tuning specific to SM75 must not be copied to other GPUs. For SYCL/Vulkan use F16 KV, flash attention auto, batch/ubatch 512, and no CUDA phase-batch or backend-sampling settings. Use a single slot, disable Thinking/MTP/DFlash and keep KVFlash disabled. The entire 16K context must remain available; do not enable automatic --fit context reduction. 32K requires separate validation.

Hybrid runtime validation (2026-09-02)

Windows Vulkan checks used the exact published Q4_K_L, Intel Arc B580, driver 32.0.101.8991, full 16K context, F16 KV, batch/ubatch 512, Thinking off and no speculative decoding. B580 needs GGML_VK_DISABLE_F16=1: without it both Hybrid and reference b10621 produced five additional slash-only, token-limit responses. This disables F16 compute kernels, not F16 KV storage; the managed workaround is restricted to the Ling/Vulkan/B580 profile.

With the workaround, all 140 turns completed without schema/protocol failures or truncation. Hybrid retained 68/76 correction and 49/49 replay results. It introduced two failures among the eight composite knowledge probes compared with the original export: an English answer omitted memory/RAG, and a German persona/provider answer confused the roles of model and application. These persisted with prompt reuse disabled. This Windows B580 profile therefore has not passed the no-regression gate. The reference retained its existing results.

Authenticated native tool calls, two successive tool-result rounds, streaming and actual prompt-cache reuse passed on Windows. Retrieval from the beginning and middle of a 14,658-token input passed within the full 16K context on both engines. These checks do not replace native Linux GPU qualification.

Three comparable runs per engine used the same 111-token prompt, 512 generated tokens, temperature 0, seed 20260902 and prompt reuse disabled:

B580 / Windows Vulkan	Hybrid	Reference b10621
Median prompt processing, tokens/s	613	711
Median first-token latency, ms	245	204
Median streamed output, tokens/s	141.6	140.2

Output rate uses the 511 tokens after the first streamed token divided by elapsed time after that token. This small difference does not establish a speed advantage. Across the full regression/context runs, sampled dedicated GPU peaks were about 5.03 GiB (Hybrid) and 5.07 GiB (reference); a separate Hybrid API/benchmark run peaked at 5.34 GiB. These are one-second Windows process-memory samples, not a hardware power measurement.

Native Linux Vulkan

The published Vulkan image ghcr.io/antibyte/aurago-llm-vulkan@sha256:feae8bfdb9a9c6613dc3a8529acff3618cc0c5866de69a8d172c58098a1f8aea passed on an AMD Lucienne integrated GPU (RADV RENOIR), Linux 6.8, Docker 29.7, with 16K context, F16 KV, batch/ubatch 512 and no B580 workaround. Its source revision is 2eb133fb8227e0cd03bc8dfd0c5b4039bfa8e36b.

All 140 turns completed: 68/76 correction, 49/49 replay, 7/8 composite knowledge, 4/4 identity and 3/3 nonempty general controls. There were no new regressions, schema/protocol errors or truncated answers relative to the original export. Both 14,658-token retrieval cases passed, and the repeated prefix reused 1,626 tokens. Authenticated native tools with tool_choice=required, two tool-result rounds and streaming also passed. Runtime attestation confirmed the model hash, engine commit, image digest, full context and GPU/KV offload.

This result covers the tested AMD APU setup. Native Linux CUDA, SYCL and B580 qualification is still outstanding; the Windows B580 differences remain a known limitation. Automatic backend selection stays disabled pending qualification of the supported hardware profiles.

License

The model follows the base model's MIT license. The hybrid engine is separate software with its own Apache-2.0 license and attribution requirements. This model repository does not relicense the engine. The separately licensed runtime patch and notices accompany the model for reproducible builds.