antflydb/GLiNER2.5-Decide

Canonical Antfly layout (2026-10-17)

This repository follows the upstream Fastino checkpoint name. The previous -gguf repository URL redirects here. The paired filenames identify the Q8_0 bundle variant: the encoder matrices are Q8_0, while the auxiliary head remains FP32 despite its Q8_0 filename suffix. The weight bytes, tokenizer, and source configuration have not changed. The bundle descriptor and antfly_inference_variants.json name both required components explicitly. Historical validation reports retain the original paths and measurements; gliner_head.gguf in those reports is now gliner2-head.Q8_0.gguf.

This DeBERTa Q8 conversion supports ordinary extraction/classification compatibility. It does not qualify the public typed Decide endpoint: the exporter strips decide and typed_decisions. The source checkpoint's name and the historical browser smoke test do not confer that qualification. Use a qualified FP32 Decide checkpoint or the separate qualified 1B Q8 bundle for typed decisions. Native/Metal full-task and long-document qualification must be established separately for these converted bytes.

GLiNER2.5 Decide — Antfly Q8_0 split GGUF bundle

This is an Antfly inference bundle converted from Fastino's GLiNER2.5-Decide. The encoder is Q8_0 GGUF; the GLiNER head is a separate GGUF file. The bundle uses Antfly's gliner2_split_bundle/v1 layout and is intended for Antfly's GLiNER2 inference runtime, including its browser/WASM playground. It is not a single-file llama.cpp model. Keep the files and relative paths together.

The source is Apache-2.0 licensed. The local model.safetensors used for conversion has SHA-256 40a5a23ff860dc3dff426cecd1048cacdd29c648c96db209dad818e9686dc997, matching the file in the upstream repository at revision 5a7adf72a23b4d311abae6ce050d7f0012bb3416. Model design, intended uses, limitations, and upstream evaluation are described in Fastino's model card. Fastino did not publish this Antfly conversion.

Bundle contents

• antfly_inference_bundle.json — split-bundle receipt naming the encoder and head.
• gliner2-encoder.Q8_0.gguf — quantized encoder, SHA-256 8d9bea7df21ab4a8a17333f51f246faab5280159e5751ebbadd81a85ec96c871.
• gliner2-head.Q8_0.gguf — GLiNER head, SHA-256 52cb6197d3dcfe5ad6276ceb037a3333d2921bbcf3dfb8289aeff82fbd5693dc.
• config.json, encoder_config/config.json, special_tokens_map.json, tokenizer.json, tokenizer_config.json — configuration and tokenizer files from the source checkpoint.
• SHA256SUMS — SHA-256 hashes for all eight bundle files.

Download the repository with hf download antflydb/GLiNER2.5-Decide --local-dir ./GLiNER2.5-Decide-Q8_0, then load that directory with an Antfly runtime that supports gliner2_split_bundle/v1. For reproducible deployments, pin a repository revision instead of using main.

Validation

Antfly's focused Decide runtime test passed 3/3 cases with these exact local bundle files and the pinned browser/WASM extraction runtime. This is a compatibility smoke test, not a full quality or cross-browser evaluation. Quantization can change outputs relative to the original checkpoint.