n00nehere/recursive-bitnet-ngram-102M-64K-instruct-preview

Recursive BitNet N-Gram 102M · 64K Instruct Preview

An experimental 102,282,456-parameter decoder-only language model with ternary BitLinear weights, recursive transformer blocks, and causal hashed n-gram embeddings. Six stored blocks are reused for two passes, giving twelve effective block applications. The longest training context is 65,536 tokens.

This release contains step 35,882, selected by the lowest recomputed held-out loss among ten frozen checkpoints. It retains the trained setting: two recursive passes and token 2-, 3-, and 4-gram features enabled. It has processed 4,703,125,504 input tokens, including repeated corpus visits. Its mean primary score across seven full public benchmark splits is 40.59% with the reference FP32-master/BF16-activation runtime.

The model was trained from scratch, then continued on a mixture of educational text, assistant responses, code, structured parsing, tool examples, and synthetic memory tasks. Instruct describes that training mixture; measured instruction-following reliability is limited. Both an exact FP32 master export and a compact packed ternary export are provided, with loading instructions and separate verification records below.

Architecture

Property	Value
Model	Decoder-only causal transformer, custom PyTorch implementation
Unique parameters	102,282,456
Backbone / added n-gram parameters	100,676,728 / 1,605,728
Stored transformer blocks	6
Recursive passes / effective block applications	2 / 12
Hidden width	1,024
Attention heads / KV heads / head dimension	16 / 4 / 64
Attention	Causal grouped-query attention, fused QKV projections, QK normalization
Feed-forward	1,024 → 2,816 → 1,024, squared ReLU (ReLU(x)^2)
Normalization	Pre-RMSNorm; final RMSNorm; QK scale clamp configured to 4.0
Vocabulary	49,152
Token embedding / output head	Shared FP32 master parameter; output head executes through BitLinear
Positional encoding	RoPE, theta 1,000,000, scaling factor 1.0
Configured positional ceiling	262,144
Longest context actually used in training	65,536
Dropout	0.0
Activation quantization	Disabled (activation_bits=0); mixed-precision floating-point activations
N-gram orders	2, 3, 4
N-gram tables	Three tables, each 16,384 hash buckets plus one zero/padding bucket, 32 dimensions
N-gram output projection	Shared 32 → 1,024 bias-free floating-point projection

The recurrent computation applies the complete stack in order, block_0 … block_5 → block_0 … block_5. Parameters are shared across passes; KV caches are separate for all twelve effective layers. Reusing parameters saves stored model weights but still incurs compute and cache costs on each pass.

At token position t, the n-gram branch hashes only tokens ending at t. Each order has an independent embedding table. The three embeddings are summed, scaled by 1/sqrt(3), projected to the hidden width, and added to the token embedding before the transformer stack. Incomplete windows and windows crossing an earlier EOS boundary use the padding bucket. Cached generation retains at most three preceding token IDs. The hash uses fixed integer arithmetic; collisions are expected in a bounded hash table. The new projection was initialized to zero when the branch was added, preserving backbone logits at the upgrade boundary; its parameters subsequently trained for 9,281 updates.

For each BitLinear output row, scale = mean(abs(weight)).clamp(min=1e-8) and q = round(clamp(weight / scale, -1, 1)); forward execution uses a straight-through effective weight derived from q * scale. Quantization strength is 1.0. Embedding lookup, normalization, and the n-gram branch remain floating point. The theoretical ternary alphabet size is log2(3) ≈ 1.585 bits. The packed export uses five ternary values per byte, plus row padding, scales, and floating-point tensors as detailed below. Both supplied loading paths use floating-point arithmetic; no native ternary compute speedup is claimed.

The checkpoint configuration is in config.json, and the implementation is in bitnet_core.py.

Training history and parameters

The 100.68M backbone was trained from random initialization; no external pretrained model weights were used. The pretrained Cosmo2 tokenizer was reused independently. The earlier 66M technical experiment appears in historical provenance as a reference run; it did not initialize this backbone.

The retained backbone at step 26,601 was upgraded with 1,605,728 n-gram parameters. Existing weights and named Adam state were preserved; new n-gram optimizer state began at step zero. Continuation used a separately prepared corpus and recorded data cursor. The published weights descend from that latest saved backbone, rather than the older backbone's separately selected step-22,670 export.

Parameter	Backbone phase retained in this checkpoint	N-gram continuation retained in this checkpoint
Hardware	Four NVIDIA B300 GPU ranks	One NVIDIA B300 SXM6 AC
Optimizer updates	26,601	9,281 additional
Processed input tokens	3,486,646,272	1,216,479,232 additional
Supervised target positions	2,077,629,457	703,472,548 additional
Training context	2,048, then 4,096	65,536
Effective batch	131,072 input tokens	131,072 input tokens
Per-device layout	32,768 tokens/rank across four ranks	Two 65,536-token sequences; final runtime uses microbatch 2 and one accumulation step
Optimizer	Fused AdamW	Restored fused AdamW
Adam betas / epsilon	0.9, 0.95 / 1e-8	0.9, 0.95 / 1e-8
Weight decay	0.1 on decay group; 0 on non-decay group	0.1 on decay group; 0 on non-decay group
Gradient norm clipping	1.0	1.0
Learning rate	Configured peak 3e-4, floor 3e-5, warmup 128 steps	Starts from saved 0.00012690606270625925; cosine decay over 1,073,741,824 additional tokens; floor 3e-5
LR at selected checkpoint	—	3e-5; the decay horizon has been exceeded
Seed	20261007	20261009
Master parameters / optimizer moments	FP32 / FP32	FP32 / FP32
Forward/backward activations	BF16 mixed precision	BF16 mixed precision
Ternary strength	1.0	1.0
Recorded elapsed training time	3,901.12 seconds	12,165.55 seconds across continuation invocations

Cumulative supervised positions are 2,781,102,005. Recorded input-token exposure is 1,267,466,240 at 2K, 2,219,180,032 at 4K, and 1,216,479,232 at 64K. These counters count revisits and packed input positions, including masked prompts and padding; they are not unique-data counts.

The original continuation allocation was bounded to four hours and underwent audited restarts for runtime tuning. The selected snapshot's remaining-invocation time limit is 7,085 seconds; that field is not its cumulative training duration. The later run ended at step 36,762 / 4,818,468,864 processed input tokens with status interrupt. Those later weights are not this release. The final recorded online best.pt was step 35,470 with selection loss 2.495869; the frozen step-35,882 snapshot scored lower in the corrected, common-prefix reevaluation described below.

The final training runtime used FlashAttention-4 4.0.0b34, CUTLASS DSL 4.8.0, native GQA with query-head packing disabled, compiled attention projections/QK normalization and pointwise regions, supervised-target compaction, and fused Quack vocabulary cross-entropy. Activation checkpointing was disabled. Vocabulary loss used 4,096-token chunks, with FP32 accumulation of vocabulary-weight gradients. Retunes preserved parameters, Adam state, corpus order/cursor, RNG, the full context, effective batch, and token schedule; changes in BF16 reduction order can alter rounding.

The objective is the mean next-token cross-entropy over all unmasked targets in a complete effective batch. Educational text supervises next tokens; instruction/chat/tool examples supervise assistant responses and terminators while masking prompts and observations. Dedicated memory updates supervise answer and termination tokens. There is no MTP, router, preference-optimization, or distillation term recorded for this run.

General and memory validation each use four 65,536-token sequences before training, about every five minutes or 2,500 updates, and on graceful finalization. Selection uses 0.875 * general_target_loss + 0.125 * memory_target_loss. Latest checkpoints were saved about every two minutes or 1,000 updates. Full details and retune provenance are preserved in training/training_parameters.json.

Data, formatting, and attribution

The initial backbone's packed corpus contained 300,002,086 tokens, representing 352,972 unique document identities and 252,971,655 unique document tokens before repeated technical examples. Its target mix was 40% educational knowledge, 15% chat, 30% technical/tool/code, and 15% deterministic parsing. The full source record is in training/backbone_data_manifest.json, including the legacy Hermes/ToolACE/StarCoder2 components and Nemotron-SFT-Agentic-v2 subset.

The continuation's packed training corpus contains 230,293,504 input tokens and 133,161,483 supervised positions per pass. The requested general-domain target mix was 45% knowledge, 25% instructions, and 10% each scripting, parsing, and tools. Actual packing also includes complete memory episodes, split tool families, and alignment padding:

Packed component	Input tokens per corpus pass	Fraction of packed tokens
memory_recall	28,835,840	12.52%
knowledge	90,000,004	39.08%
instruction	50,001,470	21.71%
scripting	20,000,087	8.68%
parsing	20,000,111	8.68%
tools	5,000,303	2.17%
tools_multiturn	972,868	0.42%
ui_automation	5,000,008	2.17%
web_agentic	5,000,068	2.17%
alignment_padding	5,482,745	2.38%

The saved continuation stream repeats this finite corpus. The selected checkpoint has processed approximately 5.28 corpus passes. Document identities are deduplicated/split before packing, with source hashes and pinned revisions recorded. Transformer attention may cross packed document boundaries; the n-gram branch resets its token-window features across EOS.

Approximately one in eight updates uses two complete aligned 64K synthetic memory episodes. They cover single/multiple record recall, changed values, and stored agent state over 4K, 16K, 32K, and 60K token distances. Facts, logs, and queries are inputs; answers and termination tokens carry loss. Held-out episodes use separate entities and templates. The two held-out streams each contain 1,048,576 packed tokens; checkpoint ranking evaluates the same four-sequence prefixes.

Continuation source	Selected file/component	Recorded source license
HuggingFaceTB/smoltalk2	SFT/smoltalk_smollm3_explore_instruct_rewriting_no_think-00000-of-00001.parquet	cc-by-nc-4.0
HuggingFaceTB/smoltalk2	SFT/table_gpt_no_think-00000-of-00001.parquet	mit
HuggingFaceTB/smoltalk2	SFT/tulu_3_sft_personas_instruction_following_no_think-00000-of-00001.parquet	odc-by
HuggingFaceTB/smoltalk2	SFT/smoltalk_smollm3_smol_magpie_ultra_no_think-00001-of-00006.parquet	apache-2.0
ise-uiuc/Magicoder-OSS-Instruct-75K	data-oss_instruct-decontaminated.jsonl	mit; original seed-code licenses still apply
bigcode/self-oss-instruct-sc2-exec-filter-50k	data/train-00000-of-00001.parquet	odc-by
HuggingFaceTB/smoltalk2	SFT/hermes_function_calling_v1_no_think-00000-of-00001.parquet	apache-2.0
HuggingFaceFW/finepdfs_edu_50BT-dclm_30BT-fineweb_edu_20BT-shuffled	data/train-00001-of-00100.parquet	odc-by
Team-ACE/ToolACE	data.json	apache-2.0
McGill-NLP/WebLINX	data/chat/train.json.gz	cc-by-nc-sa-4.0
rl-research/dr-tulu-sft-data	train.jsonl	odc-by
deterministic-memory64k-v1	recursive_memory_data.py	locally generated data

Some sources were scanned as fallbacks; a source manifest entry alone does not imply substantial retained training contribution. The final per-source counters, normalizers, exclusions, revisions, and file hashes are in training/continuation_data_manifest.json. Deterministic parsing and grounded-tool fixtures also contribute. WebLINX supplies text browser actions with page text/history. DR-Tulu is restricted to searcharena and openscholar source families; evaluation-derived QA families were excluded.

Source licenses apply to their respective data; the mix includes noncommercial and share-alike sources. This release preserves those attributions and does not infer a model-weight license from them. No model-weight license has been specified by the author in this release.

Tokenizer: HuggingFaceTB/cosmo2-tokenizer, revision 4ce2318a3628e77279c939ed6a9f3f03034402de, vocabulary 49,152, EOS ID 0. Fingerprint: 7e0b8f35ca7f836d92d824d9e3db5747c22d9a1e2753ccf2ec9af24a8614e979. The saved tokenizer assets retain their historical model_max_length=4096; the native runtime uses explicit context limits. No tokenizer vocabulary or token ID was changed during export.

Continuation conversations use the saved ChatML template:

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Your question here<|im_end|>
<|im_start|>assistant

The earlier backbone also saw ### Instruction / ### Response examples. Public multiple-choice benchmarks use their plain-text harness prompts with no chat template.

Checkpoint selection and 64K validation

“Best” here means lowest weighted held-out loss among the ten frozen snapshots that received the same corrected validation protocol. It does not mean every historical training step was retained or that this snapshot maximizes every public benchmark. The comparison contains five retained snapshots and five snapshots captured from complete later optimizer updates.

Rank	Step	Weighted validation loss	Seven-task mean, trained setting
1	35,882	2.489574	40.59%
2	35,451	2.499808	39.23%
3	32,965	2.500141	40.91%
4	35,601	2.501040	41.19%
5	32,622	2.509120	40.74%
6	35,742	2.512107	40.81%
7	31,495	2.527576	40.47%
8	36,024	2.547584	40.12%
9	28,926	2.601515	40.21%
10	27,208	2.835103	40.17%

Step 35,601 has the highest observed public-benchmark mean, 41.19%, but its held-out selection loss is worse (2.501040). Choosing it for that mean would select on the reported public suites. This release follows the held-out criterion and uses step 35,882.

For the released checkpoint, general target loss is 2.793742, memory target loss is 0.360400, and their weighted selection loss is 2.489574. General evaluation covers 262,144 input positions and 166,501 supervised targets; memory evaluation covers 262,144 input positions and only 160 supervised targets. Memory teacher-forced token accuracy is 93.75% (150/160), while teacher-forced exact target-sequence accuracy is 0% (0/4). Teacher forcing supplies preceding reference tokens; these measurements do not establish free-generation recall or reliable 64K retrieval.

The configured 262,144 positional ceiling is an architecture setting. The longest trained context is 65,536, and this release makes no validated 262K retrieval claim.

Rotary precision matters. Compiled training forms rotary frequencies in FP32. A prior eager evaluator formed them under BF16 autocast, changing phases substantially at long positions. The corrected sweep matches each checkpoint's training behavior without altering its parameters. For example, step 32,622's loss changes from 3.939591 with the incorrect runtime to 2.509120 with the corrected runtime, close to the recorded training value 2.509210. This release uses the corrected sweep and its loader constructs rotary frequencies outside autocast. Earlier scores obtained with the other rotary policy should not be substituted for these results.

Public benchmarks

The benchmark table and ablations below describe the original FP32 master weights with ternary forward computation and BF16 activations. The packed export has separate numerical smoke checks; its full benchmark suite has not been rerun.

Evaluation used lm-evaluation-harness 0.4.12, full available task splits, zero-shot prompts, no chat template, a 2,048-token scoring context, and likelihood ranking of answer continuations. Log probabilities are computed in FP32 and summed over continuation tokens using the harness's tokenization boundaries. acc_norm uses the corresponding harness task's length normalization. There was no context truncation in the seven released evaluations.

Runtime: NVIDIA A100-SXM4-80GB, PyTorch 2.12.0+cu130, Transformers 4.57.6, datasets 3.1.0, tokenizers 0.22.1. BitNet retains FP32 master weights and frozen exact ternary straight-through effective weights, uses BF16 activation autocast and FP32 rotary frequencies, and disables TF32. The released setting is n-grams on, two recursive passes.

Task	Questions	Primary metric	Score	Standard error
ARC-Easy	2,376	acc	40.11%	1.01 pp
ARC-Challenge	1,172	acc_norm	23.63%	1.24 pp
PIQA	1,838	acc	57.62%	1.15 pp
WinoGrande	1,267	acc	51.22%	1.40 pp
OpenBookQA	500	acc_norm	25.60%	1.95 pp
BoolQ	3,270	acc	58.65%	0.86 pp
HellaSwag	10,042	acc_norm	27.27%	0.44 pp

Unweighted mean of the seven primary task scores: 40.59%. Total questions: 20,465. This macro mean is a descriptive aggregate of different tasks/metrics, not a separately standardized benchmark. Standard errors are those reported by the harness, expressed in percentage points. All task JSON results, including both accuracy variants where available, are in evaluation/.

The likelihood benchmarks measure short-context commonsense and knowledge. They do not evaluate chat quality, instruction following, code execution, tool-use success, chain-of-thought reasoning, speculative decoding, or long-context retrieval. Scores from older 66M/100.68M checkpoints are not scores for this release.

Recursive-pass and n-gram ablations

These interventions use exactly the same step-35,882 weights and questions:

N-grams	Passes	Effective block applications	Seven-task mean
On	2	12	40.59%
On	1	6	39.19%
On	3	18	39.67%
Off	2	12	40.58%
Off	1	6	39.08%
Off	3	18	39.61%

Two passes with n-grams enabled give the highest mean for this checkpoint. The very small difference from disabling n-grams does not establish a meaningful improvement. Increasing to three passes costs additional computation and does not improve the mean here. One/three passes and n-grams-off are inference interventions; the model was trained with two passes and n-grams on.

Across ten checkpoints and six settings, 55/60 full evaluations completed, covering 1,125,575 question–configuration pairs. Five n-grams-off/three-pass runs on other checkpoints were unfinished after the Modal workspace reached its spending limit. All six settings for the released checkpoint completed. Missing evaluations are not imputed as zero. See evaluation/MATRIX.md, evaluation/scores.csv, and evaluation/sweep_audit.json. The sweep's global status remains partial; its completed rows passed the sample audit.

Question IDs, prompt/document/answer identities, full split coverage, sample-file hashes, scoring source, tokenizer, and sample-derived means were checked across the comparison. During release preparation, the selected checkpoint's seven original sample hashes and all primary means were checked again. Compact per-question outcome files are included for recomputing these means; the original sample hashes refer to the original full evaluation records.

The data pipeline checks exact duplicates and excludes exact matching released benchmark prompts. Those checks and pinned source revisions do not establish complete near-duplicate or semantic decontamination across all pretraining data. Cross-checkpoint and inference-setting comparisons are exploratory, and paired intervals in the audit are not adjusted for multiple comparisons.

Packed ternary export

Artifact	Storage	File size	Purpose
model.safetensors	Exact FP32 masters; tied embedding/head stored once	409.14 MB	Reference weights used to reproduce the published benchmarks
model_packed.safetensors	Base-3 ternary matrices; FP16 scales and other tensors	124.24 MB	Compact inference export of the same selected checkpoint

Sizes use decimal MB. The packed artifact is 3.29× smaller than the FP32 file. It contains 25 packed BitLinear matrices: attention projections, feed-forward projections, and the output head. Every group of five values from {-1, 0, +1} is encoded as five base-3 digits in one byte. Incomplete groups are padded per row. Packing/unpacking the ternary codes is exact.

The ternary matrices consume 20,153,856 bytes, their row scales 175,104 bytes, and the other FP16 tensors 103,901,616 bytes, before safetensors metadata. Matrix storage is 1.601685 bits per ternary value including row padding. The entire file uses approximately 9.72 bits per original unique parameter because it also retains floating-point token embeddings, normalization, and n-gram parameters. The token embedding and output head share a master parameter during training; the packed artifact stores an FP16 embedding lookup and a separate packed ternary output head.

This export uses the project's existing quantized_state_dict_from_model conversion. Scales and non-BitLinear tensors are rounded to FP16. The packed path applies row scales after matrix reduction; the reference path forms effective straight-through weights before multiplication. These changes can affect logits and generated text, so packed inference is not bit-for-bit identical to the reference weights.

The supplied packed loader keeps matrices as byte buffers, decodes bounded output-row chunks on demand, and uses FP32 floating-point computation on CPU or CUDA. It needs no compiler or custom extension and does not retain a full decoded weight cache. Floating-point parameters stored as FP16 are loaded into FP32 parameters, while packed scales retain their stored precision. Loaded persistent model-state tensors occupy approximately 228.13 MB in this runtime; rotary tables, KV caches, temporary decoded chunks, and allocator overhead are additional. The smaller download does not imply the same reduction in peak RAM/VRAM or an inference speedup.

Packed verification

• All ternary matrices unpack to the exact original ternary codes; all scales and other tensors match the original values rounded to FP16; serialization preserves every exported tensor exactly.
• Three fixed CPU/FP32 fixtures spanning plain text, ChatML, and JSON/code contain 95 input positions. Maximum absolute next-token cross-entropy change is 0.000147, maximum absolute logit difference is 0.017557, and top-1 logit agreement is 100.0% on those positions.
• Cached decoding agrees with a full forward within the recorded numerical tolerance. Packed chat and raw-completion commands pass on CPU; a short CUDA generation check also passes on the local NVIDIA Tesla P4.

These are bounded export/runtime checks, not packed-model benchmark scores or a general guarantee of identical generation. Full measurements, fixture texts, hashes, and storage accounting are in packed_export.json and packed_verification.json.

Load and run

This is a native PyTorch custom architecture, distributed as safetensors plus its runtime. It does not register a Transformers AutoModelForCausalLM implementation or promise Hub hosted-inference support. Use the supplied loader. A CPU can run short prompts. The default FP32-master path uses BF16 activations on supported CUDA GPUs and FP32 on CPU or older GPUs. The packed path uses FP32 computation on both CPU and CUDA. The CPU/FP32 path is a compatibility path; it does not reproduce the benchmark's BF16 arithmetic bit for bit.

hf download n00nehere/recursive-bitnet-ngram-102M-64K-instruct-preview --local-dir ./bitnet-release
cd bitnet-release
python -m pip install -r requirements.txt
python model_runtime.py --device cpu --prompt "Explain what a dictionary is in Python." --max-new-tokens 80

On a suitable CUDA GPU:

python model_runtime.py --device cuda --prompt "Return a JSON object with name and status fields." --max-new-tokens 80

To download and run only the compact weights and runtime:

hf download n00nehere/recursive-bitnet-ngram-102M-64K-instruct-preview --local-dir ./bitnet-packed --include "model_packed.safetensors" "config.json" "*.py" "tokenizer/*" "requirements.txt"
cd bitnet-packed
python -m pip install -r requirements.txt
python model_runtime.py --weights packed --device cpu --prompt "What is a Python dictionary?" --max-new-tokens 80

For the Python loader, select load_model(".", device="cpu", context_limit=2048, weights="packed"). Omit weights or pass weights="fp32" for the original master-weight export. Packed CUDA usage is --weights packed --device cuda.

For plain completion, add --raw. --passes 2 is the default; 1 and 3 expose the evaluated depth interventions. --context-limit defaults to 2,048 and can be set up to 65,536 subject to available memory. The prompt plus generation must fit the chosen limit. Chat generation stops on EOS or the saved <|im_end|> token; the script only produces text.

from contextlib import nullcontext
import torch
from model_runtime import load_model

model, tokenizer, amp_dtype = load_model(".", device="cpu", context_limit=2048)
ids = torch.tensor([tokenizer.encode("The capital of Italy is")], device=next(model.parameters()).device)
autocast = torch.autocast("cuda", dtype=amp_dtype) if amp_dtype is not None else nullcontext()
with torch.inference_mode(), autocast:
    logits, _, _ = model(ids)
print(tokenizer.decode([int(logits[0, -1].argmax())]))

Keep master weights in FP32: calling model.half() or model.bfloat16() before ternary preparation can change rounding and invalidate score comparisons. The FP32 loader restores the tied output-head alias; both formats check the tokenizer fingerprint. Inference weights are frozen; this loader is not a resumable training checkpoint loader.

Release contents and reproducibility

• model.safetensors: exact unique FP32 master tensors, with the tied head stored once. No cast, merge, or additional training occurred during export.
• model_packed.safetensors, packed_export.json, and packed_verification.json: compact weights, exact packing checks, numerical fidelity checks, and storage accounting.
• config.json, model_runtime.py, runtime dependencies, and tokenizer/: local loading and generation.
• training/: saved settings, source/data manifests, runtime notes, and training/evaluation source snapshots. Historical source files are references to the original project, not a standalone one-command reconstruction of the entire training environment.
• evaluation/: corrected validation, full task-result JSON, compact per-question scores, and the checkpoint/ablation sweep.
• release_manifest.json: source identity, file sizes, and SHA-256 hashes. verification.json records local export, reload, and generation checks.

Frozen original training-checkpoint SHA-256: 5fea986b0434049baca98e7cbbef4b7d0ef5a69ac13c81eb2b4ac3dc9ad23246. Snapshot global step: 35,882. Tokenizer fingerprint and source identities are also embedded in the configuration. The exported safetensors has its own file hash; a format conversion does not preserve the original .pt file hash.

The full original training checkpoint contains Adam moments, cursor, scheduler, and RNG state and is retained in the training archive. This inference release omits that resumable state and raw packed training data. Reconstructing optimizer state from the inference weights is not possible.

Intended use and limitations

This preview is useful for studying recursive weight sharing, ternary-weight training, hashed n-gram features, and fixed-depth inference tradeoffs at small model scale. It can generate plausible but incorrect, repetitive, or malformed text and has modest measured benchmark accuracy. Generated code and tool-like text have not been validated as safe or correct actions. The available evaluations do not establish production-grade instruction following, factual reliability, or autonomous tool use.

The data mix, repeated finite corpora, small memory-validation denominator, incomplete broader sweep, and precision sensitivity all limit generalization claims. The documented architecture and measured results define the scope of this preview.