Accio-Lab/occamy-1.0-GGUF

Occamy-1.0 GGUF

GGUF weights converted directly from Accio-Lab/occamy-1.0.

Files

File	Size, decimal GB	Purpose
occamy-1.0-Q8_0.gguf	36.903	8-bit quantized language model
occamy-1.0-Q4_K_M.gguf	21.167	Mixed 4-bit K-quant language model
mmproj-occamy-1.0-F16.gguf	0.899	Vision encoder/projector; use with any language-model file

Q8_0 is converted directly from the BF16 source checkpoint. Q4_K_M is quantized from the BF16 GGUF using llama-quantize, without an importance matrix. Small and sensitive tensors use the converter/quantizer's standard mixed-precision rules. File size is not the required runtime RAM or VRAM.

Hardware compatibility

Use a llama.cpp build supporting the qwen3_5_moe architecture; the exact tested revision and launch settings are listed below. These GGUF files are not NVFP4 checkpoints and do not require an NVFP4-specific backend.

Variant	Language weights	With F16 vision projector
Q4_K_M	21.167 GB	22.066 GB
Q8_0	36.903 GB	37.802 GB

These are decimal file-size totals, not measured peak memory or minimum hardware requirements. Allow additional memory for KV cache, runtime buffers, and image processing. Context length, concurrency, and CPU/GPU offload settings affect memory use. The recorded smoke configuration used an 8K context and one slot; it does not certify every device or longer-context configuration. CPU-only, Apple Silicon, and consumer-GPU compatibility have not been established by these tests.

Illustrative memory budgets (not measured)

The table below converts file bytes to GiB and adds an assumed 4 GiB or 8 GiB runtime reserve. These are transparent planning scenarios, not a calculation of the actual KV cache or a prediction that an 8K request will fit. The reserve must cover cache, compute buffers, recurrent state, and (when used) image processing; actual overhead may exceed either allowance.

Variant	Weight-size proxy, GiB	With 4 GiB reserve	With 8 GiB reserve
Q4_K_M, text only	19.71	23.71 GiB	27.71 GiB
Q4_K_M + F16 vision projector	20.55	24.55 GiB	28.55 GiB
Q8_0, text only	34.37	38.37 GiB	42.37 GiB
Q8_0 + F16 vision projector	35.21	39.21 GiB	43.21 GiB

Formula: file bytes / 2^30 + assumed runtime reserve. For fully GPU-resident execution, compare against free, not advertised, VRAM. CPU offload changes the division between host RAM and VRAM; these totals do not establish a host-RAM minimum or account for all loading-time allocations. Unified-memory machines also need room for the OS and other applications. Measure peak memory with your actual context, batch, image resolution, and runtime before relying on a configuration. These estimates exclude MTP.

Hugging Face's Add hardware for estimation configures the viewer's hardware estimate. It is separate from the measured validation results below and is not a compatibility certification.

Tokenizer compatibility

Normalize input text to NFC before tokenization to reproduce the source tokenizer's normalization. The source tokenizer has an NFC normalizer; the tested llama.cpp build does not apply it automatically. normalize_nfc.py provides this preprocessing for text input. For API clients, apply unicodedata.normalize("NFC", text) to textual prompt content before sending requests.

These GGUF files set tokenizer.ggml.pre to qwen2, matching Occamy's actual pre-tokenizer regex. The unmodified converter inferred qwen35; that inference changes tokenization for some scripts and has been corrected using llama.cpp's GGUF metadata-copy utility. Tensor data is unaffected by this metadata correction.

On the 14 recorded tokenizer checks, 13 matched the source with raw input and all 14 matched after NFC normalization. This finite test is not a proof of equivalence for every possible Unicode input. Preserve this compatibility note when redistributing the files.

Run

Use a recent compatible llama.cpp build; the tested version is recorded in Technical details.

llama-server \
  -m occamy-1.0-Q4_K_M.gguf \
  --mmproj mmproj-occamy-1.0-F16.gguf \
  -ngl 999 -c 8192 -np 1 -fa on \
  --jinja --host 127.0.0.1 --port 8080

For text-only use, omit --mmproj. The source chat template is embedded in the language-model files. The chat API accepts chat_template_kwargs: {"enable_thinking": false}; validation used thinking disabled. Do not interpret the smoke test's 8K context as a validation of the source model's full context limit.

Validation

Recorded results from VALIDATION.json; the reference here is the local BF16 GGUF, not an independent reproduction of the original Transformers benchmark suite.

Check	BF16 GGUF reference	Q4_K_M	Q8_0
Smoke responses completed with EOS	8/8	8/8	8/8
Small code-function checks	2/2	2/2	2/2
Strict JSON checks	3/4	3/4	3/4
Raw tokenizer agreement with source	13/14	13/14	13/14
Tokenizer agreement after NFC	14/14	14/14	14/14
Exact OK instruction	Failed (OK.)	Failed (OK.)	Failed (OK.)
Synthetic red-image identification	Correct	Correct	Correct

The Q4_K_M and Q8_0 variants, plus a local BF16 reference, each generated all eight fixed text/code/JSON/tool-format smoke responses without truncation. Code functionality passed two small tasks per variant. Strict JSON content checks passed 3/4 per variant: the fixed-object response was wrapped in Markdown. A native chat request for exactly OK returned OK. on all variants, so exact instruction compliance did not pass that diagnostic. These small checks do not establish general quality parity with BF16 or reproduce the source model's benchmark scores.

Synthetic image checks are recorded in VALIDATION.json. The tool-format prompts request tool calls as JSON text; they do not validate a complete agent workflow or native tool execution. Timing was collected during shared-resource conversion work and is not a controlled performance benchmark.

MTP

The source Occamy checkpoint declares zero MTP layers and contains no MTP tensors. These GGUF files therefore omit MTP. They do not contain an unaligned head borrowed from another model, and no MTP speedup is claimed.

License

Apache 2.0. Conversion settings and reproduction instructions are in Technical details.