tetracta/tetracta-znext-1b

Tetracta Z-Next 1B (v4.3)

This repository contains no model weights. It is a model information card: architecture class, measured benchmarks, measured memory/concurrency behaviour, training provenance, and an honest list of limitations. The weights are not published at this time.

Full results page (tables, fairness gates, limitations): https://www.tetracta.ai/znext.html You can, however, run the model yourself right now: → https://www.tetracta.ai/zchat

---

"So it's Mamba with a new name."

No — and we are also not claiming to have invented constant-state sequence modelling. That family exists, it is published, and Mamba and RWKV are its best-known members. The claim here is narrower and checkable: a set of measurements that, as far as we can tell, are not published elsewhere in this form.

1. A same-data transformer twin, trained by us, as a control. Architecture papers are usually compared against public checkpoints trained by other teams on other corpora with other tokenizers and recipes. We trained a deliberately ordinary transformer on the same corpus, same tokenizer, same budget, same harness — and we publish it, including the rows where it beats us.
2. Robustness to instruction tuning appears to differ by architecture. One recipe, applied unchanged to three architectures, moved them in three directions: ours +0.51 pp (95% CI [+0.04, +0.98]), our previous generation −0.43 ([−0.89, +0.04]), the transformer twin −1.39 ([−1.97, −0.79]). One run per architecture, one recipe, no seed repeats — we do not know yet whether it generalises.
3. The concurrency claim is a measurement on a real server, not an asymptotic argument. Seats were counted from real free VRAM on one consumer card running a public chat service, and we publish the caveats that cut against the number.
4. We publish where our own measurements come out negative. The limitations section is longer than the results section.

What we do not claim, and would like to fix. We have not run a budget-matched head-to-head against Mamba or RWKV. We ran state-spaces/mamba-1.4b-hf through our own harness and it scores 56.40 macro against our 45.24 — but it is a finished public release on a far larger budget, so that number answers a different question than ours. A matched-budget comparison inside the constant-state family is exactly the experiment we cannot afford alone, and it is near the top of the list of things we would run with support.

Model summary

Field	Value
Name	Tetracta Z-Next 1B, version 4.3
Developer	Tetracta (tetracta.ai)
Class	Autoregressive decoder-only language model, attention-free
Parameters	947,153,897
Pre-training budget	10.00B tokens
Pre-training wall-clock	29.97 hours on 4×H200 (single node, single continuous run)
Throughput	~92.7k tokens/s aggregate, stable across the full run
Language	English
Weights	Not released. Live demo available.
Checkpoints described here	(a) pre-trained base, (b) instruction-tuned, (c) an earlier chat-tuned build (20 Aug — the one every chat/tool measurement below was run on), (d) a newer chat-tuned build (21 Aug — one of the two builds the demo may be started with; its chat/tool acceptance gates have been measured and are reported under Limitations, but no chat-tuned build has ever been run through the classic benchmark suite)

What it is, in one paragraph

Z-Next is a decoder-only language model that does not use attention and keeps no KV cache. A conversation is carried in a fixed-size session state — the memory a session occupies does not grow with the length of the conversation or the length of the input. Architecturally it is a distant relative of the recurrent / state-space family in the sense that it carries a bounded running state rather than re-reading a growing history, but the internal mechanism differs from published models in that family and is not disclosed here. The claim of this page is not "better quality". It is a measured cost/behaviour profile at matched training budget — and the measurements, including the unflattering ones, are given below.

---

Intended use

Intended. Research into attention-free / constant-memory sequence models; evaluation of the class at 1B scale; benchmarking against budget-matched baselines; technical due diligence by potential research or commercial partners; hands-on inspection of runtime memory behaviour through the live demo.

Not intended. Production deployment, factual question answering, anything safety-critical, or any use that depends on the output being correct. This is a 1B-parameter model trained on 10B tokens. It hallucinates heavily and in a manner appropriate to that scale (see Limitations). There is no safety tuning, no RLHF, and no content filtering beyond what the pre-training corpus implies.

Out of scope. Non-English use. The corpus is English (FineWeb-Edu + English Wikipedia) and every evaluation reported here is English.

---

What is different — stated only through measurements

Three things are measured, and only these three are claimed:

1. Session memory is constant. A session's state occupies 5.227 MiB and does not change with context length. This was verified by raising the input from 952 to 31,926 tokens: the session state was byte-identical at every point (table below). This is a measurement, not a projection.
2. On one 16 GB consumer GPU, that constant cost translates into far more parked sessions than a KV-cache transformer of the same size — 727 vs 14 in the measured head-to-head configuration (≈52×). The conditions under which this ratio holds, and the conditions under which it shrinks, are spelled out below. We state the strongest counter-argument ourselves.
3. At a matched (in fact slightly unfavourable) token budget, quality is not the price paid. The macro score over 10 standard zero-shot tasks is above both budget-matched Pythia reference points with intervals clear of zero, and level with the same-data transformer baseline base-to-base (+0.62, 95% CI [−0.06, +1.28] — a tie, not a win). After the same instruction-tuning recipe the gap against that baseline is +2.51 [+1.79, +3.25], but read that one together with the fact that the recipe cost the twin 1.39 points. The defensible claim is "quality is not the price paid", not "quality is better".

Everything else on this page is context, caveat, or a negative result.

---

Benchmarks

Evaluation contract

• 10 standard zero-shot tasks, full test/validation sets, no sub-sampling (n_limit = 0): LAMBADA 5,153 · HellaSwag 10,042 · PIQA 1,838 · WinoGrande 1,267 · ARC-Easy 2,376 · ARC-Challenge 1,172 · OpenBookQA 500 · SciQ 1,000 · BoolQ 3,270 · MMLU 14,042.
• Single metric contract applied identically to every row: acc for LAMBADA, WinoGrande, ARC-Easy, BoolQ, MMLU; acc_norm for HellaSwag, PIQA, ARC-Challenge, OpenBookQA, SciQ. (Using acc for PIQA instead moves every macro by ≤0.14 p.p and changes no conclusion. We recomputed every row from the raw per-item outputs under one contract rather than copying numbers between reports.)
• Same harness, same seed (1234), pure fp32, TF32 disabled, context ceiling 1024 for all models.
• Macro = unweighted mean of the 10 task scores. Per-model 1σ on the macro ≈ 0.38 p.p. For differences between two models we report paired bootstrap 95% CIs over per-item correctness (4,000 resamples), not the single-model σ.

Main table

Model	Tokens seen	LAMB	Hella	PIQA	Wino	ARC-e	ARC-c	OBQA	SciQ	BoolQ	MMLU	Macro
Z-Next v4.3 1B — instruction-tuned	10.00B	30.5	41.8	66.9	50.5	56.6	30.6	32.8	68.8	55.6	23.4	45.8
Z-Next v4.3 1B — base	10.00B	30.5	40.1	67.0	51.6	57.2	29.0	33.0	65.9	54.5	23.5	45.2
Previous-generation Z 1B — base	10.32B	34.3	40.6	65.9	50.7	55.6	28.8	33.0	67.0	57.4	24.0	45.7
Previous-generation Z 1B — instruction-tuned	10.32B	31.3	42.2	67.0	50.5	55.1	30.2	32.8	68.5	51.4	24.0	45.3
Z-Next v4.3 1B — base @74% of budget	7.42B	29.2	39.1	66.2	49.6	54.8	29.0	33.6	64.2	57.6	23.5	44.7
Transformer twin 1B — base (same data, same budget)	10.32B	31.0	38.2	63.5	53.2	52.8	26.1	31.8	64.6	59.8	25.1	44.6
Transformer twin 1B — instruction-tuned (same recipe)	10.32B	22.5	39.9	62.5	53.7	51.2	29.4	29.2	64.7	54.2	24.9	43.2
Pythia-1.4B (step5000)	10.49B	42.2	32.5	62.2	50.6	46.4	23.9	28.0	69.8	56.1	23.2	43.5
Pythia-1B (step5000)	10.49B	36.6	31.8	62.4	53.2	44.6	22.7	26.6	69.4	53.7	23.0	42.4

Pythia rows were measured by us with the same harness, not copied from the literature (EleutherAI/pythia-1b and EleutherAI/pythia-1.4b, revision step5000 = 10.49B tokens, chosen to match our budget).

Paired differences (95% CI, paired bootstrap over per-item correctness)

Comparison	Δ macro (p.p)	95% CI	Reading
Z-Next v4.3 base vs Pythia-1B @10.49B	+2.83	[+2.10, +3.57]	ahead
Z-Next v4.3 base vs Pythia-1.4B @10.49B	+1.74	[+1.03, +2.43]	ahead
Z-Next v4.3 base vs transformer twin base	+0.62	[−0.06, +1.28]	statistically equal
Z-Next v4.3 instruction-tuned vs transformer twin base (the twin's own best checkpoint)	+1.13	[+0.42, +1.81]	ahead
Z-Next v4.3 base vs transformer twin base	+0.62	[−0.06, +1.28]	statistically equal — CI spans zero
Z-Next v4.3 instruction-tuned vs transformer twin instruction-tuned	+2.51	[+1.79, +3.25]	ahead
Z-Next v4.3 instruction-tuned vs transformer twin base (the twin's own best checkpoint)	+1.13	[+0.42, +1.81]	ahead
Z-Next v4.3 @74% budget vs finished transformer twin	+0.06	[−0.59, +0.73]	statistically equal
Z-Next v4.3 instruction-tuned vs previous-generation Z instruction-tuned	+0.45	[−0.22, +1.11]	statistically equal — we do not claim a win
Effect of the same instruction-tuning recipe on Z-Next v4.3	+0.51	[+0.04, +0.98]	small but positive
Effect of the same recipe on the transformer twin	−1.39	[−1.97, −0.79]	it cost the twin
Effect of the same recipe on previous-generation Z	−0.43	[−0.89, +0.04]	slight cost, CI touches zero

The last three rows are the honest form of a frequently over-claimed point. We do not appeal to a literature generalisation about instruction tuning lowering zero-shot benchmarks. We ran the same recipe on three architectures and report what happened: it lowered two of them and raised one.

The budget comparison is tilted against us

Model	Tokens seen	Parameters
Z-Next v4.3 1B	10.00B	947,153,897
Transformer twin 1B	10.32B (+3.2%)	936,609,792
Previous-generation Z 1B	10.32B (+3.2%)	—
Pythia-1B	10.49B (+4.9%)	1,011,781,632 (+6.8%)
Pythia-1.4B	10.49B (+4.9%)	1,414,647,808 (+49%)

Z-Next saw the fewest tokens of any model in the table. It is also smaller than both Pythia reference points. We are not claiming an equal-budget comparison; we are claiming a comparison in which our side had less budget.

Two models we measured and deliberately left out of the table

The same harness, seed, 1,024-token ceiling and gates were also run on two fully trained public releases. Under the identical metric contract:

Model (not budget-matched)	Params	Macro
state-spaces/mamba-1.4b-hf	1,372M	56.40
TinyLlama/TinyLlama_v1.1	1,100M	48.55

That is 11.2 and 3.3 points above our base model. They are excluded from the main table because their pre-training budgets are orders of magnitude beyond 10B tokens, so placing them beside a budget-matched comparison would be meaningless — but excluding them silently would be worse, so here they are.

This also answers the obvious question about family membership. Mamba is the closest published member of the constant-state family we say we belong to, and at 1B scale we have no quality argument against a version of it trained on far more data. What this card claims is a matched-budget comparison and a memory profile, not a quality win over the fully-trained field. We did not independently verify either model's token budget, so we quote no number for it.

Additionally, the 1024-token context ceiling in this suite comes from the transformer twin's positional limit and is imposed on everyone. Z-Next's own context behaviour is therefore not exercised at all by this benchmark suite.

Perplexity

WikiText-103 — word-level perplexity and bits-per-byte are tokenizer-independent, so this comparison is methodologically valid across all models including Pythia.

Model	Word PPL ↓	bits/byte ↓
Previous-generation Z 1B — base, finished	29.66	0.9142
Transformer twin 1B — base, finished	29.76	0.9151
Z-Next v4.3 1B — @94% of budget (9.39B tokens)	29.88	0.9162
Pythia-1.4B @10.49B	39.59	0.9921
Pythia-1B @10.49B	41.20	1.0029

⚠ Version caveat, stated plainly: WikiText-103 was never measured on the finished 10.00B Z-Next v4.3 checkpoint. The most advanced point we measured is 94% of the budget. It is shown here labelled as such and must not be attributed to the final model. Against the budget-matched Pythias, the gap is large and the comparison is sound; against our own twin and our own previous generation, the gap at 94% is practically closed (+0.4%) but not closed in our favour.

LAMBADA perplexity — this is a token-level PPL, so it is comparable only between models sharing our tokenizer. Pythia is deliberately excluded.

Model (shared tokenizer only)	LAMBADA PPL ↓
Previous-generation Z 1B — base	31.59
Z-Next v4.3 1B — base (finished)	42.54
Z-Next v4.3 1B — instruction-tuned	46.61
Transformer twin 1B — base	62.68
Transformer twin 1B — instruction-tuned	160.97

Note the asymmetry that runs through this whole card: on LAMBADA accuracy the model is clearly behind Pythia (30.5 vs 36.6 and 42.2), while on LAMBADA perplexity it is well ahead of the twin. Accuracy-based benchmarks do not flatter this architecture; we did not select a metric that does.

Long context — identical windows, last 256 tokens scored (loss, nats, lower is better)

Context T	Z-Next v4.3 @74% budget	Transformer twin, finished	Previous-gen Z, finished
512	2.8084	2.8558	2.7577
1,024	2.7750	2.8036	2.7156
2,048	2.7694	2.7904	2.7133
4,096	2.7656	2.7866	2.7117
8,192	2.7656	2.7873	2.7130
context gain 512→8192	−0.043 nats	−0.069 nats	−0.045 nats

Read this table honestly in both directions. Z-Next v4.3, at 74% of its budget, had lower loss than the finished transformer twin at every length. It also had higher loss than our own previous generation at every one of the five lengths. We are not dropping that column.

⚠ Two limits on this table. (i) The windows are drawn from the same corpus all three models were trained on — this is not a held-out set, so none of these numbers is a generalisation measurement. The text is identical for all three, which keeps the comparison symmetric, but the three models did not consume identical fractions of that corpus (10.00B draws for us, 10.32B for the other two). (ii) No confidence interval was computed. Each cell is 16 windows × 256 scored tokens and the differences being pointed at are 0.02–0.05 nats. The evidence is that the ordering is consistent across all five lengths, not that any single gap was interval-tested.

---

Memory and concurrency (measured, not projected)

Hardware: one NVIDIA GeForce RTX 5070 Ti. Torch reports 15,851 MiB total, 15,608 MiB free at idle. In the head-to-head configuration the measured free VRAM is split evenly between the two models (7,804 MiB each), and 350 MiB per side is reserved for transient generation working memory (peak measured transient usage: 266 MiB). Both models are fp32 — the comparison is dtype-symmetric.

	Z-Next v4.3 1B	Transformer twin 1B
Weights	3,651 MiB	3,830 MiB
Per-session cost	5.227 MiB, constant	256.0 MiB, pre-allocated (for a 1,024-token ceiling)
Session budget after weights + reserve	3,803 MiB	3,625 MiB
Concurrent parked sessions	727	14
Ratio	≈52×	—

Same card in single-model mode (whole card given to Z-Next): 2,220 sessions (reproduced in 10 separate runs).

Constant memory under growing input — the direct measurement:

Input	Prefill	Prefill rate	Transient VRAM	Session state
952 tok	1.3 s	723 tok/s	247 MiB	5.227 MiB
4,012	1.1 s	3,638	266 MiB	5.227 MiB
9,928	1.6 s	6,357	266 MiB	5.227 MiB
19,924	2.9 s	6,922	266 MiB	5.227 MiB
31,926	4.8 s	6,699	266 MiB	5.227 MiB

A 33× longer input leaves the session state bit-for-bit unchanged. The 266 MiB is shared transient working memory, not per-session.

How to read the 52× — caveats we impose on ourselves

• A seat is a memory admission limit, not throughput. 727 seats means 727 conversations can have their state resident simultaneously. It does not mean 727 concurrent token streams; generation on a single GPU is serialised.
• 727 was never literally opened. It is (measured free budget ÷ measured constant per-session cost). The highest number of Z-Next sessions actually opened live is 20, at which point VRAM linearity was confirmed to within ±1 MiB. By contrast the twin's 14 was driven into the wall end-to-end (capacity+3 sessions attempted; the excess got HTTP 503 and the declared ceiling was never exceeded).
• The single biggest lever is the twin's declared context ceiling, and the ratio scales with it almost linearly. At the same budget: 256 tok → ~13×, 1,024 tok → 52× (measured), 2,048 tok → ~104×, 4,096 tok → ~242×. Everything except the 1,024-token row is arithmetic extrapolation from the measured per-token KV cost, not a measurement.
• The strongest counter-argument, stated by us: the twin in this comparison is a straightforward implementation that allocates the entire declared ceiling at session open. Production serving stacks with paged/block KV allocate as the conversation grows, so on short conversations they will host many more sessions than 14 and the ratio will drop substantially. The constant-memory property remains, but the headline multiplier is configuration-dependent.
• Quantising only the twin's KV cache (int8/fp8) would cut the ratio by 2–4×. Both sides here are fp32.
• Capacity is read from actual free VRAM at startup, not from a nominal table; if something else is on the card, seats are reduced and a warning is printed, and if CUDA OOMs at runtime the declared capacity is lowered to the observed reality. Measured example: with 5,151 MiB of other work on the card the numbers became 258 ⇄ 4 = 64× — counter-intuitively the ratio rises on a tighter card, because the twin's 256 MiB grain rounds the remaining budget more coarsely.
• An earlier published figure of "750 ⇄ 14 = 54×" came from a pre-fix build that read the budget from the card's nominal total rather than its actual free memory. That bug is recorded in our own notes and the corrected number is 727 ⇄ 14 = 52×. We publish the corrected one.

---

Training

Pre-training

Tokens	10,000,007,168 (10.00B)
Hardware	4×NVIDIA H200, single node, rented cloud pod
Wall-clock	29.97 h (independently re-derived from telemetry as 29.95 h)
Compute	≈120 H200-GPU-hours ⇒ 83.4M tokens per GPU-hour
Throughput	~92.7k tok/s aggregate; marginal throughput 92,989 tok/s over the 20–100% span and 92,713 tok/s over the last 10% — i.e. less than 0.4% drift across 30 hours
Loss (EMA)	11.52 → ≈2.78 (last 20 records oscillate 2.750–2.784; PPL ≈ 16.2)
Run health	One continuous run. 15,258 telemetry records: zero NaN/inf, zero EMA spikes after u>20k, monotonic uninterrupted update counter, no resume, no rollback, no intervention.
Pre-flight	All 4 GPUs health-probed before launch: 744.4 / 745.2 / 745.3 / 747.3 ms per step — 0.4% spread across cards
Cost	≈$550, derived — not an invoice. (measured 29.97 h × the documented $18.36/h list rate for the 4×H200 pod at that date). A comparable earlier run of ours was estimated at $520–620 — also a projection, not an invoice. For balance: the pre-run estimate was $417 / 23 h, so the actual run exceeded its own estimate by ~30%.

Corpus

• 10.33B tokens, from two fully public sources: HuggingFaceFW/fineweb-edu (sample-100BT) and wikimedia/wikipedia (20231101.en).
• No synthetic data — deliberately excluded.
• The corpus is English; every evaluation on this page is English.
• Training windows were sampled at random with replacement. We therefore do not claim "one epoch" or "no data repetition"; expected corpus coverage is ≈62%. The defensible statement is: the corpus holds 10.33B tokens and the model consumed 10.00B, so this run does not rely on volume-scale data repetition.
• The mixture ratio between the two sources was a collection target and the realised ratio was never measured, so no ratio is quoted.

Instruction / chat tuning

Checkpoint	Examples	Tokens	Hardware	Wall-clock	Cost
Instruction-tuned (the one benchmarked above)	12,546	3.31M	1×H200	60.1 min	—
Chat-tuned (the newer build described under "Live demo")	18,005	7.45M	1×H200	69.2 min	~$12 billed (training itself ~$5.3)

The benchmarked instruction-tuned checkpoint saw instruction data equal to 0.033% of the pre-training token budget. The ~$12 figure is the only truly invoiced cost number in this card; the pre-training dollar figure above is a derivation.

---

Evaluation methodology — fairness gates

Every benchmark run records four gates alongside the scores:

Gate	What it checks	Threshold	Measured
K1 padding immunity	right-padding must not change the logits of earlier positions (i.e. batching gives no one an advantage)	< 1e-2	0.00 – 2.37e-04 across all models
K2 narrow-output identity	scoring a restricted output set must match full-vocabulary scoring	< 1e-3	1.43e-05 – 1.43e-04 across all models
K3 context ceiling	identical ceiling for every model	1,024 for all	items truncated: 0 for our models and the twin; 2 of 40,660 for Pythia; 22 for TinyLlama
K4 fast-path parity	our accelerated inference path must agree with the reference computation	< 1e-2	9.16e-05 – 2.75e-04

Honest note on K4: it only applies to models that have an accelerated path of ours. For the transformer twin and for Pythia the field is null — the question does not exist for them. So the correct statement is not "all four gates passed for all models"; it is "K1–K3 passed for every model, and K4 is a question that only exists for our own runs, where it was measured and passed."

Honest note on K3: the 1,024-token ceiling is the twin's structural limit and it is imposed on everyone including us. Also, "no item was truncated" is true for our models and the twin, but 2 Pythia items were truncated — a negligible effect, but the absolute phrasing would be wrong.

Comparison validity — three mandatory caveats

1. The twin is not shape-identical to Z-Next. It was trained on the same data with the same budget and the same recipe, but the depth/width profile is not matched one-to-one (the numbers are withheld). What was measured is therefore "architecture
   • shape" as a package, not architecture alone.
2. Not all rows come from one sitting. The five base/instruction-tuned rows were measured in a single session on 19 Aug; the twin-base and previous-generation-base rows come from a separate session on 18 Aug — same harness, pure fp32, TF32 off, same seed (1234), but not the same run.
3. Pythia was trained on a different corpus (the Pile). What is equalised is the token budget and the measurement protocol — not the data.

---

★ Limitations and honest notes

This section is deliberately as long as the results section.

1. The live demo does not run the benchmarked checkpoint. The classic benchmark suite was never run on any chat-tuned build. The 45.8 macro belongs to the instruction-tuned checkpoint, not to the model you will chat with, and the model you will chat with has no macro score at all. The demo is started with one of our two most recent chat builds (20 or 21 Aug), chosen at launch, so we cannot promise in advance which one answers you. The chat/tool numbers in items 12–13 were measured on the 20 Aug build; the 21 Aug build has since been through the identical gate suite and its numbers are given inside item 12 — they are materially better than the 20 Aug ones, and we are stating that rather than leaving the more pessimistic figure standing as if it described the current build.

2. Long-range retrieval in free-form natural language is not solved. On a format-neutral natural-language needle test, instruction tuning produced no measurable improvement at all: base 0.306 vs tuned 0.301, Δ = −0.005, p = 0.92, chance = 0.25, n = 216 per model. All three models score near chance. Broken down by depth, all three only work in roughly the last 15% of the context (0.58–0.92 there) and fall below chance at head and middle depths. We can measure a 5.5× gain on a structured retrieval curve, but it does not transfer to the neutral format. We therefore explicitly do not claim that recall is solved.

3. Retrieval decays with the gap after the fact, in the structured test too. Base model, by gap length: 280 tok → 1.000 · 1,210 → 1.000 · 2,202 → 0.583 · 3,194 → 0.292 · 7,007 → 0.083. The best tuned variant reaches 0.458 at a 7,007-token gap (chance ≈ 0.016). Real improvement, still a monotonically falling curve. The problem is not the number of stored facts but the gap after them: with 2 records, FAR placement scores 0.042 and NEAR placement scores 1.000.

4. On the live demo pipeline, needle retrieval in a natural document ends after a few thousand tokens. Measured on the 20 Aug chat-tuned build through the demo pipeline. You can reproduce this yourself:

Document	Needle retrieved?	Answer given
~500 tok	✓	correct override code quoted verbatim
~2,700 tok	✗	"5440."
~8,000 tok	✗	"5160."
~18,000 tok	✗	"5190."

Our internal reading is that this reflects a curriculum limit rather than an architectural one (the long-gap curriculum was a small fraction of the last tuning run). That reading is a hypothesis, not a measurement; the table above is the measurement.

5. Compositional / multi-hop inference is essentially absent. Two-step chained questions score 0/24 across the base and both tuned checkpoints; direct single-hop scores 1–3/24. This is an open capability gap at this scale, not a measurement artefact.

6. MMLU does not discriminate at this scale. Every 1B-class model in our table scores 22.97–25.11 against a 25.0 chance floor. It drags every macro down equally and distinguishes nobody. Read the macro with that in mind.

7. Instruction tuning made raw-corpus perplexity ~17% worse (2.7438 → 2.9013 nats; PPL 15.5 → 18.2). This is the expected distribution shift, but it belongs next to the "+0.5 macro" line, not hidden from it. The context gain over long windows was retained (−0.0497 → −0.0564 nats).

8. Two comparisons we do not get to call wins. (a) Against our own previous generation: +0.45 p.p, 95% CI [−0.22, +1.11] — equal. That generation is also ahead of us on LAMBADA perplexity (31.59 vs 42.54) and on long-context loss at all five measured lengths. (b) Base against the twin's base: +0.62 p.p, 95% CI [−0.06, +1.28] — the interval contains zero, so that is a tie too. The only architecture comparison against the twin that clears its interval is the one taken after the shared instruction-tuning recipe (+2.51), and a large part of that margin is the twin losing 1.39 points to the recipe rather than us gaining 0.51.

9. WinoGrande — a recall-signature task — is behind the twin. 51.6 vs 53.2 (−1.6 p.p) for the finished base; the gap was −3.6 p.p at the 74% checkpoint.

10. LAMBADA accuracy is well behind budget-matched Pythia (30.5 vs 36.6 and 42.2), even though the macro is ahead. Top-1 hit rate is 46.34% vs the twin's 46.75% (measured at the 88%-of-budget checkpoint, not on the finished base) — the twin is ahead on easy tokens. Accuracy-thresholded benchmarks understate this architecture and perplexity-based ones flatter it; both are shown.

11. Output-format following is a separate, real gap. A required [CODE: <answer>] output format was not produced in any of 5 seeds × 5 settings, although the correct answer still appeared in the surrounding prose in 80% of those runs. The model answers and ignores the imposed format. Measured on the 20 Aug chat-tuned build only; not re-run on the other checkpoints, so we cannot say whether it is a property of the line or of that build.

12. Mid-session tool chaining broke on one build and largely recovered on the next. In the 20 Aug chat-tuned build, end-to-end tool use scores 0.033–0.067 against a pre-registered threshold of 0.40, and 0.000 on the worst seed — while the same task in a fresh context scores 0.600. Measured root cause: a previous turn's result overrides the current argument. A related value-leak metric sits at 0.089–0.097 against a pre-registered threshold of 0.060–0.070 (down from 0.181–0.197 in the prior version — halved, still failing). Thresholds were registered before the run and were not moved afterwards.

Update, and it goes in our favour — which is exactly why it is here. The 21 Aug chat-tuned build has since been run through the identical gate suite (15 seeds, same temperature, same mixed-session harness): mid-session end-to-end tool use 0.567 (clears the 0.40 gate; worst of 15 seeds 0.500), call rate 1.000, chat recall 0.944, refusal rate 0.031. So "the session destroys the capability" was a property of one build, not of the architecture, and we are correcting it rather than letting the gloomier number stand. Still failing on that build: value leakage 0.067 against a 0.060 threshold; topic-holding in a long mixed session 0.658 against 0.680; worst-seed chat recall 0.667 against a 0.700 floor. 3 of 17 pre-registered gates fail. And the classic benchmark has still never been run on it.

13. Conversational memory is limited by format, not distance. In an ~800–920-token, 28-turn chat, recall measured 0.633–0.700, while the same distance presented in a structured format scored 1.000. The 20 Aug chat-tuned build raises chat recall to 0.933–0.967, but that too was measured at ~800 tokens of context, not at long context — and, per item 1, not on the build the demo now serves.

14. It hallucinates, heavily, in a manner appropriate to 1B/10B. Observed: the Moon's orbital period given as "21 days"; "the first three primes are 1, 2, 3"; 8102 × 47 rendered as "380,793 … 3807.680" (correct answer 380,794). Chat quality is not product-grade and we do not present it as such. The claim of this work is the cost/behaviour profile, not the quality.

15. No safety work has been done. No RLHF, no red-teaming, no content filtering, no bias evaluation. Corpus biases from FineWeb-Edu and English Wikipedia pass through unmitigated.

16. Version hygiene. Numbers measured on earlier generations of this architecture (inference-latency figures in particular) are not reproduced on this card, because they were not measured on this model. Where a number comes from an intermediate checkpoint (74% or 94% of budget), it is labelled inline. We do not substitute one version's measurement for another's.

---

The plan we are asking to fund

A compute grant is easier to judge when it buys a specific experiment rather than a direction. Three stages, each with a stop condition written before the run:

stage	run	compute	gate
1	1B on 60B tokens — 6× our current budget, everything else held fixed	≈720 GPU-h	macro ≥ 47.5, twin lead preserved, no long-context regression. If it fails, we stop.
2	3B on 200B tokens	≈7,600 GPU-h	must beat a budget-matched 3B transformer twin and move a neutral-prose needle test off chance — the one thing our 1B could not do (p=0.92)
3	7B + tool use, coding and agentic work, shipped	≈17,700 GPU-h	only if stage 2 passes

We are asking for stage 1 only: ≈720 GPU-hours. It answers one question — what six times the token budget does to this architecture — and we publish the result either way.

Compute figures are extrapolated from the measured run (947M parameters, 10.0B tokens, 29.97 h on 4×H200 ≈ 120 GPU-hours) by scaling throughput with parameter count. They are estimates, not measurements.

Live demo

→ https://www.tetracta.ai/zchat

• Runs on a single consumer GPU at home, with Z-Next and the same-data transformer twin loaded side by side on half the card each, so the seat counts above are visible live.
• CAPTCHA + proof-of-work gated; the demo is off by default and switched on manually, so it may be closed when you visit.
• The demo runs a chat-tuned checkpoint — one of our two most recent (20 or 21 Aug), chosen when the engine is started. No chat-tuned build has ever been run through the classic benchmark suite, so the 45.8 macro is not its score and it has none. Its chat and tool-use gates have been measured on both builds: the 21 Aug build clears the tool-use gate the 20 Aug build failed, and still fails 3 of 17 gates. See Limitation 1 and item 12.
• Useful things to try: paste a very long document and watch the session memory stay flat; then try to make it retrieve a fact from the middle of that document, and watch it fail past a few thousand tokens. Both behaviours are on this page.

---

Contact / collaboration

Tetracta is scaling this architecture beyond 1B and is looking for compute partners, research collaborators, and technical due-diligence conversations.

• Site: https://www.tetracta.ai — research page: https://www.tetracta.ai/research
• Live demo: https://www.tetracta.ai/zchat

What we will share under agreement: full evaluation harness details, raw per-item benchmark outputs, training telemetry, reproduction protocol for every table on this page, and the corpus construction procedure (both sources are public).

What we will not share publicly: the internal mechanism of the architecture, its shape parameters, and its inference implementation details. This card is deliberately built so that its persuasiveness comes from measurement rather than from mechanism — if the numbers above are not convincing on their own, the mechanism would not fix that.

---

Citation

@misc{tetracta_znext_1b_v43_2026,
  title  = {Tetracta Z-Next 1B (v4.3): an attention-free, KV-cache-free language model
            with constant-size session state},
  author = {Tetracta},
  year   = {2026},
  note   = {Model card and benchmarks; weights not released.
            Live demo: https://www.tetracta.ai/zchat},
  url    = {https://www.tetracta.ai/research}
}

Last updated: 2026-08-21. Every figure on this page is traceable to a measurement artefact in our internal records; figures that are derivations, extrapolations, or intermediate-checkpoint readings are labelled as such inline.