tayi-ai/Tayi-Flash-Q4

Tayi Flash Q4

Low-bit adaptation with constrained Recovery & Protection.

MXFP4 / Q4-class · Frozen base + separate LoRA · Release v1.0

Read whitepaper v1.2 · Download PDF · Markdown · Tayi results

Overview

Tayi Flash Q4 is the custom low-bit model release developed by Paulo R. Lima, under Tayi AI / HYZIS. Its learned Tayi LoRA correction is built with constrained Recovery & Protection, using factor-consistent updates, anchored protection margins and real-candidate verification.

The Tayi model is the complete combination: the 12 MXFP4 base-weight shards plus the learned Tayi adapter. All required weight files are hosted in this repository under the Tayi distribution name. Load both components to run Tayi; loading the base alone does not apply the Tayi customization. No second model repository is required for download.

Base-model lineage: Tayi Flash Q4 inherits its architecture and frozen base weights from the pinned upstream model recorded in source-pin.json. The Tayi learned correction, adaptation method integration and release-specific evaluation are identified separately. Upstream authorship and conversion provenance are collected in the references at the end of this card. This is an adapter-trained derivative, not a claim of original backbone pretraining or full-model weight updates.

Model v1.0; document v1.2. The current whitepaper is Q4-only: it aligns the Flash identity, release method, benchmarks, specialization layers, prompt/loading contract and final upstream attribution. The adapter weights have not changed. See the revision log.

Tayi results — same-runtime comparison first

Both arms below use the same pinned MXFP4 base and runtime. The control is the Q4 reference without an effective Tayi correction; the Tayi arm enables candidate line2-59001-01. These are not comparisons against a separately executed, higher-precision upstream model.

Quality and functional qualification

Evaluation	Tayi Flash Q4	Q4 reference	Scope
MMLU generated-letter exact match — primary metric	113 / 511 · 22.11%	116 / 511 · 22.70%	Local one-token protocol
MMLU highest-scoring declared option — secondary diagnostic	119 / 511 · 23.29%	122 / 511 · 23.87%	Argmax restricted to A/B/C/D
BFCL stratified qualification	16 / 16 · 100%	16 / 16 · 100%	16 cases from a 160-case population
Previously classified diagnostic smoke	14 / 16	14 / 16	Not independent final heldout confirmation

The two MMLU rows measure different things. A correct highest-scoring option among A/B/C/D does not necessarily mean the unrestricted generated token was the expected letter. The secondary diagnostic does not replace the primary metric.

MMLU paired transitions	Preserved	Recovered	Regressed	Persistent error
Generated-letter exact match	105	8	11	387
Highest-scoring declared option	112	7	10	382

The primary result is −3 net correct decisions / −0.587 percentage points. Mean margin delta is −0.0529478 and median margin delta is −0.0367240. The local gate passed, but these observations do not support presenting the candidate as a quality-superior or lossless variant.

The reserved MMLU slice contains 512 cases. One 4,556-byte prompt exceeded the service's 4,096-byte input limit, leaving 511 matched cases. This is separate from the model session's 4,096-token context. The raw append-only file contains 1,035 observations; the audit keeps the last valid observation for each example/arm, yielding 1,022 unique example/arm records.

Sources: MMLU audit · preserved raw observations · BFCL subset · diagnostic smoke and its scope.

Local Recovery / Protection gate

Development-gate measurement	Tayi Q4
Gate composition	2 Recovery + 14 Protection cases
Mean Recovery margin before → after	−3.0116747 → −2.8021470
Mean Recovery-margin change	+0.2095277
Protection-floor violations	0
Minimum Protection slack	+0.1486052
Recovered / regressed classifications within this gate	0 / 0
Local acceptance	PASS

This is an improvement in the margin objective, not an increase in the number of correct classifications in the 16-case gate. Each protected margin must remain at least max(0, 0.75 × initial margin). The accepted trajectory coefficient is −0.000861040966293267. Source: gate summary and candidate acceptance records.

Runtime performance — exploratory, end to end

Runtime measurement	Tayi Flash Q4	Q4 reference	Observed Tayi/reference difference
Mean request latency	17.005 s	16.700 s	+1.83%
Median request latency	17.021 s	16.583 s	+2.64%
Completion tokens / total request time	1.882 tokens/s	1.916 tokens/s	−1.79%
Measured requests / output tokens	4 / 128	4 / 128	32 output tokens per call

Protocol: four identical prompts per arm, serial requests, one excluded warmup per arm, alternating paired order, temperature=0, seed=59, max_tokens=32. The resident deployment uses the 21-worker / 42-T10 configuration described below.

The rate is end-to-end: total output tokens divided by total HTTP request duration. It includes prompt processing and request handling; it is not isolated decode speed, concurrent-server throughput, TTFT, a cold-start benchmark or a quality evaluation. The observed approximately 1.8% difference in four cases does not establish a universal adapter overhead or acceleration over the original higher-precision model.

Sources: original runtime observations · metric/protocol clarification · portable reproduction script.

Parameter count shown by the Hub

The 12 frozen-base GGUF shards contain 748,494,669,424 tensor elements (748.49B, before the separate 25,600-element LoRA correction). This was recounted from every shard header with @huggingface/gguf 0.4.6; it is not an estimate from file size. See benchmarks/parameter-audit.json.

This serialized-tensor count is different from the upstream architectural backbone/activated-parameter figures. It must not be substituted for those figures. The 25,600 count belongs only to the Tayi correction, not to the full model.

The correction is stored at adapters/tayi-flash-q4.lora, still in GGUF format and with the same SHA-256. Its .lora filename distinguishes it from complete .gguf model variants during Hub indexing. Load it explicitly with --lora-scaled ./adapters/tayi-flash-q4.lora:1.0; no merge or conversion is needed. Earlier records may call the identical bytes tayi-flash-q4-lora.gguf.

Model identity and delivery

Dimension	Tayi Flash Q4 delivery	Upstream architecture reference
Architecture lineage	Same pinned upstream backbone	552B backbone parameters
Activated backbone parameters	Inherited, not independently remeasured	8B prefill / 16B decode
Serialized parameter count	Not interchangeable with backbone count	763,205,315,794 elements reported by upstream repository metadata
Context	4,096 tokens in the qualified session	Up to 1M tokens
Modality	Textual GGUF deployment	Text + image
Pretraining	No claim of repeating backbone pretraining	45T tokens reported upstream
Global KV cache	Not rebenchmarked in this release	Approximately 890 bytes/token
Representation	MXFP4 routed experts, mixed tensor types	Official upstream weights
Base files	12 GGUF shards · 403,503,056,480 bytes	Pinned GGUF conversion source below
Tayi correction	Separate 102,752-byte GGUF LoRA	Not part of upstream weights
LoRA rank / alpha	4 / 8	—
Adapter target	blk.0.attn_q_a.weight	—
Factor elements updated	25,600	Not all backbone parameters

The Q4 package is 403.50 GB decimal / 375.79 GiB, not a measurement of resident VRAM. It is not uniform Q4_K_M: the format contains MXFP4 experts and other Q8_0/F32/BF16 tensors.

Source: source pins · MXFP4 shard manifest.

Recovery & Protection method

The quantized base is frozen and the correction remains separate:

$$y={\mathrm{Linear}}_{Q_b(W)}(x)+sBAx,s=\alpha \mathrm{/}r\mathrm{.}y\; =\; \backslash operatorname\{Linear\}\_\{Q\_b(W)\}(x)\; +\; sBAx,\; \backslash qquad\; s=\backslash alpha/r.$$

For a correct answer and a competing answer, the decision margin is:

$$m_i(\theta )={\mathrm{score}}_{\theta }(y_i^{+}\mid x_i)-{\mathrm{score}}_{\theta }(y_i^{-}\mid x_i)\mathrm{.}m\_i(\backslash theta)=\backslash operatorname\{score\}\_\{\backslash theta\}(y\_i^+\backslash mid\; x\_i)-\backslash operatorname\{score\}\_\{\backslash theta\}(y\_i^-\backslash mid\; x\_i).$$

Symmetric probes estimate the objective direction and the response of protected margins:

$$g_k\approx \frac{L(\theta +\epsilon z_k)-L(\theta -\epsilon z_k)}{2\epsilon },H_{ik}\approx \frac{m_i(\theta +\epsilon z_k)-m_i(\theta -\epsilon z_k)}{2\epsilon }\mathrm{.}g\_k\; \backslash approx\; \backslash frac\{L(\backslash theta+\backslash varepsilon\; z\_k)-L(\backslash theta-\backslash varepsilon\; z\_k)\}\{2\backslash varepsilon\},\; \backslash qquad\; H\_\{ik\}\; \backslash approx\; \backslash frac\{m\_i(\backslash theta+\backslash varepsilon\; z\_k)-m\_i(\backslash theta-\backslash varepsilon\; z\_k)\}\{2\backslash varepsilon\}.$$

The method considers constrained local updates:

$$\underset{a}{\min }{g}^{T}a+\frac{\lambda }{2}\parallel a{\parallel }^2\text{subject to}{m}_i({\theta }_t)+H_{i}a\ge {\tau }_i\mathrm{.}\backslash min\_a\backslash ;\; g^Ta+\backslash frac\{\backslash lambda\}\{2\}\backslash lVert\; a\backslash rVert^2\; \backslash quad\backslash text\{subject\; to\}\backslash quad\; m\_i(\backslash theta\_t)+H\_i\; a\backslash ge\backslash tau\_i.$$

Floors are anchored to the declared reference rather than silently reset at each step. Probing and application use the same factor parameterization; the actual candidate is executed before acceptance, and failed candidates are rolled back. The general formulation and the concrete accepted trajectory are distinguished in the whitepaper v1.2.

Ground truth identifies operational errors in the Q4 run. Without a paired higher-precision model run on the same cases, it does not prove that every error was caused by quantization rather than already present in the base model.

Data provenance and evaluated scope

The preservation manifest in this release is tayi-flash-preservation-v2, which supersedes v1 after a Parquet LIST-decoding repair. Its historical materialization state is preserved; later execution evidence is reported separately.

Prepared family	Development cases	Reserved cases	Evaluation published here
IFEval	433	108	No completed paired result in this package
HumanEval + MBPP	162	420	No completed paired result in this package
BFCL robustness	640	160	16-case stratified qualification only
MMLU	512	512	511-case local paired readout, plus separately labeled gate/smoke

The separate Arandu corpus has 69 unique examples (50 quality/implementation, 19 security). The Brazilian-market corpus has 12 original pt-BR examples, split 10/2. Preparation and eligibility do not establish that those examples trained the released candidate. The closed development-gate evidence for this candidate is the 16-case MMLU slice.

Heldout observations must not be silently reused for fitting, threshold selection or coefficient search. If reserved examples are later used for adaptation, that set is no longer an independent final test and a fresh heldout evaluation is required. The 16-case smoke already carries final_heldout: false and remains diagnostic.

Sources: preservation v2 · Arandu · Brazilian market.

Runtime and loading

One separate controller coordinates 21 workers with two NVIDIA T10 GPUs each. Twenty mx.rpc processes and one mx.serve coordinator form one distributed model instance, not 42 independent copies. The qualified configuration uses a private cluster network, layer splitting and a CUDA SM75 build with 64 scheduler backends. The control plane manages authorization, persistent job state, reconciliation and cancellation; Dokploy manages deployment.

Download the complete model weights

The distribution uses Tayi-Flash-Q4-MXFP4-00001-of-00012.gguf through Tayi-Flash-Q4-MXFP4-00012-of-00012.gguf. These names identify the Tayi release; the base bytes and the learned adapter are unchanged. Historical names map to these files in provenance/filename-map.json. The original source manifest is retained under provenance/ for sealed-runtime reproduction. Existing installations pinned to an earlier commit remain reproducible at that revision; no server cache or training path was renamed.

All 12 MXFP4 base shards plus the Tayi adapter are now hosted here. The combined weight download is 403,503,159,232 bytes, approximately 403.50 GB (375.79 GiB), before documentation and local cache overhead. Reserve enough free disk space and use hardware with sufficient aggregate memory.

# Download the complete repository, including every required base shard and LoRA.
hf download tayi-ai/Tayi-Flash-Q4 --local-dir ./Tayi-Flash-Q4

# On Linux; use `shasum -a 256 -c WEIGHTS-SHA256SUMS` on macOS.
cd Tayi-Flash-Q4
sha256sum -c WEIGHTS-SHA256SUMS

The base starts at Tayi-Flash-Q4-MXFP4-00001-of-00012.gguf; all 12 shards must stay together with their published Tayi filenames. The pinned GGUF publisher specifies mx-llama.cpp, with layer splitting, for this architecture; a generic upstream llama.cpp build is not a supported substitute. See the pinned publisher instructions and runtime features. Load adapters/tayi-flash-q4.lora at runtime scale 1.0. The adapter is included but deliberately not merged or requantized: running only the base is not a Tayi run.

# LLAMA_SERVER is the compatible llama-server executable for your hardware.
# Add your own qualified GPU/RPC allocation options; no Tayi server is required.
"$LLAMA_SERVER" \
  -m ./Tayi-Flash-Q4-MXFP4-00001-of-00012.gguf \
  --lora-scaled ./adapters/tayi-flash-q4.lora:1.0 \
  --split-mode layer -c 4096 --host 127.0.0.1 --port 8080

The runtime scale is distinct from alpha/r = 2 and from the factor-update trajectory coefficient. weights-full.json lists every required weight file, its exact byte length, SHA-256 and role. The supplied GGUF base and LoRA are the same bytes used by this release; this publication step changes packaging only.

Adapter SHA-256:

8322a2660e3b68c2102171c36996df532e452c15e6a852bfcda6fe7dd0f5fc11

Do not merge and requantize the correction when reproducing the reported separate-adapter path. A generic llama.cpp binary is not asserted to support every operation in this pinned MX fork.

The measured single-turn non-thinking prompt envelope was:

<｜begin▁of▁sentence｜><｜User｜>{user_message}<｜Assistant｜></think>

For tool use, multi-turn or reasoning formats, follow the pinned upstream encoding rather than assuming this single-turn example covers them.

Reproduction and integrity

python scripts/summarize-mmlu.py benchmarks/raw/mmlu-heldout-raw.jsonl \
  --output /tmp/mmlu-audit.json

# Requires an authorized, already-running server and the resident adapter contract.
# Supply its key through TAYI_INFERENCE_API_KEY; never put credentials in this repository.
python scripts/reproduce-runtime-smoke.py \
  --base-url "$TAYI_INFERENCE_URL" \
  --confirm-resident-contract \
  --output /tmp/new-runtime-smoke.json

sha256sum -c SHA256SUMS

The benchmark script replays the documented request protocol; it does not start the GPU fleet. The smoke measurements do not guarantee a speed or quality outcome on another setup.

File or directory	Purpose
release.json	Model identity, documentation version and full weight-delivery contract
weights-full.json / WEIGHTS-SHA256SUMS	Complete base + adapter inventory and integrity checks
Tayi-Flash-Q4-MXFP4-*.gguf	All 12 required frozen base shards, hosted here
adapters/tayi-flash-q4.lora	Required Tayi correction, loaded at scale 1.0
docs/Tayi-Flash-Q4-Whitepaper-v1.2.pdf	Current Q4-only 14-page technical whitepaper
docs/Tayi-Flash-Q4-Whitepaper-v1.2.md	Markdown companion of the Q4 whitepaper
docs/corrections-v1.2.md	Q4-only documentation revision log
docs/source/	PDF builder and dependency pins
benchmarks/ and benchmarks/raw/	Summaries, audit and available original observations
scripts/	Portable audit and smoke reproduction
SHA256SUMS	SHA-256 integrity for the release files

The PDF is hosted in this repository; no external CDN is required. This model repository is public: its PDFs and weights can be downloaded without authentication. The previous PDF remains recoverable from Git history, not presented as the current technical document.

Published reference benchmarks

The following architecture and benchmark values come from the pinned upstream reference and are included as published reference values, not Tayi reproductions. Exact attribution, Technical Report and evaluation-source links are collected in the References section at the end of this card.

Published base-model results

Benchmark	Upstream Base	Shots
MMLU-Pro	74.1	5
HumanEval · Pass@1	79.4	0
BigCodeBench · Pass@1	60.6	3
GSM8K	93.0	8
MATH	61.1	4

Published instruct / agentic comparisons

Benchmark	Upstream reference	Kimi-K3	GLM-5.3	Opus-5.0	GPT-5.6 Sol
GPQA Diamond	90.9	92.9	88.1	93.4	94.1
Terminal-Bench 2.1	90.6	88.3	88.2	89.1	88.8
Terminal-Bench 3.0	30.0	17.7	28.3	43.3	34.4
DeepSWE v1.1	74.2	67.5	66.9	74.0	73.0
CyberGym	88.1	80.0	84.5	—	84.5
AutomationBench	54.8	46.7	48.8	50.3	45.8

Terminal-Bench 2.1 and 3.0 are different versions, not interchangeable scores. The upstream card reports its Instruct evaluations at maximum reasoning effort, with temperature=1.0 and top_p=0.95; code-agent evaluations use the specified harnesses and up to a 1M-token context. Benchmark-specific details remain those in the official sources. — means no score reported in that source, not zero.

A direct comparison with these results requires matching the benchmark version, task set, harness, encoding, context, decoding budget and verifier. This repository has no locally measured Tayi score for GPQA, Terminal-Bench, DeepSWE, CyberGym or AutomationBench. The local MMLU readout is not MMLU-Pro.

References and citation

DeepSeek-AI created the backbone. LoRA, QERA, MeZO and GEM retain their respective authorship. Tayi documents the specific adaptation, constrained-update workflow and evidence in this release.

DeepSeek model card · DeepSeek technical report · LoRA · QERA · MeZO · GEM.

Tayi Flash

@misc{lima2026tayiflashq4,
  title = {Tayi Flash: Low-Bit Adaptation with Constrained
           Recovery and Regression Protection},
  author = {Lima, Paulo R.},
  year = {2026},
  note = {Whitepaper v1.2; model release v1.0},
  url = {https://huggingface.co/tayi-ai/Tayi-Flash-Q4}
}

DeepSeek-V4.1-Flash

@misc{deepseekai2026deepseekv41flash,
  title = {DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression},
  author = {DeepSeek-AI},
  year = {2026}
}

Tayi AI · tayi.ai