Numbat Forensics β Reproducible Truncated Model & Weight Analysis
4,732,490,080 parameters (4.73 B) Β· 21 tensors Β· 9,464,980,352 bytes (9.46 GB) Β· BF16 Β·
1 of 114 decoder layers Β· SHA256 053a00beaf8ac0f1e14329a7e717648f275f7e641d58d65abf46a376acf452ef
This is an independently produced research artifact, not an official Anthropic model. It contains only a truncated experimental model and does not demonstrate full-checkpoint language-model capability.
Claimed identity (as stated by the uploader, not by this audit). The source repository presents itself as
numbat β described there as "Anthropic's next Mythos-class training run", a mid-pretraining checkpoint at
step 1,830,000 of a ~1.96M step run, ~14 T parameters, 772 shards / 7.72 TB, 1,224 MoE experts, quantised to
NVFP4 for evaluation. It further claims the codename appears in a March 2026 Claude Code source leak.
What this audit actually found about those weights: they are not trained. 633 norm and convolution tensors
regenerate bit-for-bit from SHA256(tensor_name), and 373,620,632 code signs across 12 expert matrices match the
same name-seeded stream, both with zero fitted parameters. The embedding table is the one genuinely trained
component. Nothing here confirms or refutes who the uploader is, or where the embeddings came from β
parameter authenticity and provenance are different questions, and only the first is answered here.
An independent audit of the public checkpoint burner22/numbat
at revision 6b7f8fc548cbafe9bec441f03af2d92a55cb4012, plus a truncated (1-of-114-layer)
experimental model built from its real tensor bytes.
The question the study answers is narrow and testable: are the parameters in that checkpoint genuinely trained, or can they be regenerated by a deterministic program?
Answer: they can be regenerated. 633 norm and convolution tensors β 11,374,080 values β reproduce
bit-for-bit from SHA256(tensor_name) with zero fitted parameters. 373,620,632 signs of non-zero
E2M1 codes across 12 expert matrices match the same name-seeded stream with zero fitted parameters.
The token embedding table is the exception: it has real learned semantic structure, and is the only component here that looks like a trained network.
1. Research overview
| Target repo | burner22/numbat (public, ungated, no license declared) |
| Pinned revision | 6b7f8fc548cbafe9bec441f03af2d92a55cb4012 |
| Checkpoint size | 772 shards, 7,720,489,342,136 bytes (7.72 TB), 1,576,410 tensors |
| Declared parameters | 6,849,378,030,158 (index metadata); README claims ~14 T |
| Method | HTTP range requests against safetensors headers; no shard downloaded; no repository code executed |
| Verified | 184.4 M parameters (11.37 M bit-exact + 173.0 M near-exact) = 0.0027 % of the checkpoint |
| Hardware used | 12 vCPU, 22 GB RAM, no GPU, 233 GB disk |
What is in this repository is the research artifact, not the checkpoint: a runnable 1-of-114-layer model, the code that rebuilds it from the source, and every measurement behind the verdict. See Β§12 for why the source weights themselves are not redistributed.
2. Original repository and pinned revision
Every number in this card comes from revision
6b7f8fc548cbafe9bec441f03af2d92a55cb4012. The revision is baked into build_artifact.py,
tensor_hashes.json and every research script; nothing resolves main.
Facts recorded at access time: created 2026-10-23, last modified 2026-10-24, public, ungated,
5,846 downloads, 225 likes, no license file, no cardData, no license tag.
3. Architecture and limitations
The source config describes a 114-layer hybrid stack:
- hidden 12,288, vocab 131,072, MLP intermediate 24,576
- Gated DeltaNet attention on layers not β‘ 3 (mod 4), MLA on the 28 layers that are
- MoE: 1,224 routed experts + 1 shared, top-48, group-limited routing (72 groups / top-8)
- NVFP4 quantisation (modelopt 0.47.0, group size 16) on MoE layers; dense layers 0, 1, 109β113 in BF16
- one MTP layer at index 114 (BF16)
This artifact keeps layer 0 only. Layer 0 is dense and GDN-type, so no NVFP4 dequantisation is involved β the bytes are used exactly as stored.
Limitations, stated plainly:
- 1 of 114 decoder layers. No MLA layer, no MoE layer, no MTP layer, no deep residual stream.
- It cannot produce coherent text and does not attempt to. The published inference output below is degenerate, and the random-weight control produces the same degenerate output.
- A truncation carries no provenance information. Do not read anything into its output.
- Peak RSS β 8 GiB; per-token latency β 12 s on 12 CPU cores, dominated by the 131,072 Γ 12,288 head.
4. Extracted parameters and tensor inventory
This artifact contains 4,732,490,080 parameters (4.73 B) in BF16 β 9,464,980,352 bytes across 21 tensors. That is 0.069 % of the checkpoint's 6.85 T parameters. It is a truncation, not a small model: an 11,072-token prompt fits, but there is one decoder layer, so the output is degenerate by construction.
tensor_hashes.json lists all 21 tensors this artifact needs, with per-tensor SHA256:
| Component | Tensors | Bytes |
|---|---|---|
model.embed_tokens.weight (131,072 Γ 12,288 BF16) |
1 | 3,221,225,472 |
| decoder layer 0 (GDN + SwiGLU MLP) | 18 | 3,023,027,936 |
model.norm.weight (12,288 BF16) |
1 | 24,576 |
lm_head.weight (131,072 Γ 12,288 BF16) |
1 | 3,221,225,472 |
| Total | 21 | 9,464,980,352 (9.465 GB) |
results/tensor_inventory.csv covers the 648 tensors examined during the audit (name, shard, dtype,
shape, storage bytes, logical parameters, offset). All 648 rows were re-checked against live
safetensors headers during verification: 0 mismatches.
4b. The safetensors file
| File | model.safetensors |
| Size | 9,464,980,352 bytes (9.46 GB / 8.82 GiB) |
| SHA256 | 053a00beaf8ac0f1e14329a7e717648f275f7e641d58d65abf46a376acf452ef (also in model.safetensors.sha256.json) |
| Format | safetensors, metadata {source_repo, source_revision, truncation, note} |
| Tensors | 21 β model.embed_tokens.weight (3.22 GB), 18 Γ model.layers.0.* (3.02 GB), model.norm.weight, lm_head.weight (3.22 GB) |
| Dtypes | 19 BF16 matrices, 2 F32 scalars (self_attn.A_log, self_attn.dt_bias) |
| Per-tensor hashes | tensor_hashes.json |
It is a single file rather than shards because it is far below the 50 GB per-file limit. Load it with
safetensors.torch.load_file, or use modeling_numbat_truncated.load_safetensors_artifact.
Verify it without downloading twice:
python build_artifact.py --verify-only # 21/21 tensors against the pinned revision
sha256sum model.safetensors # must equal the hash above
5. Forensic methodology
- Pin and inventory. Repo API + paginated tree (784 files), cross-check the repo's own numbers against each other.
- Read headers, not shards. A safetensors file is
<u64 LE header length><JSON header><data>;data_offsetsare relative to8 + header_len. Verifiedaccept-ranges: bytes/206, cached 115 headers, fetched only the byte ranges each test needed. - Test falsifiable reconstruction hypotheses, ordered from zero fitted parameters upwards.
- Run distributional controls β same-shape i.i.d. Gaussian matrices computed locally, not bare statistics.
- Look for cross-tensor relations the config denies (
weight_scale_2vsmax|w|,lm_headvsembed_tokens). - Never execute repository Python.
modeling_numbat.pywas read; the forward pass here is an independent reimplementation.
6. Reconstruction results
Recipe recovered (and independently re-derived):
seed = int.from_bytes(sha256(tensor_name.encode()).digest()[:8], "big")
g = numpy.random.Generator(numpy.random.PCG64(seed))
norms: value = bf16_rtne( exp(float32(0.06) * g.standard_normal(shape, float32)) )
convs: value = bf16_rtne( g.standard_normal(shape, float32) * float32(1/6) )
NVFP4 experts:
z1 = g.standard_normal((O, I), float32)
z2 = g.standard_normal((O, I), float32)
z3 = g.standard_normal((O,), float32)
u = bf16_rtne( z1 * exp(0.4*z2) * max(1 + 0.12*z3, 0.7)[:, None] )
w = bf16_rtne( S * u ) # S: one float32 per tensor
β NVFP4 quantise (E4M3 per-16 block scale, E2M1 RTNE codes, sign from w)
| Test | Scope | Result | Fitted parameters |
|---|---|---|---|
| P4a norms + convs | 633 tensors, 115 layers | 11,374,080 / 11,374,080 values bit-exact (100.000000 %) | 0 |
| P4b code signs | 12 expert matrices, 7 layers | 373,620,632 / 373,620,632 signs (100 %) | 0 |
P4c one tensor, exact S |
34,603,008 codes | 34,602,989 codes (99.99995 %), 2,162,683 / 2,162,688 block scales (99.9998 %) | 1 float32 |
| P4f held-out rows | 5 tensors, 173,015,040 params, S fitted on β€ 4.55 % of rows |
172,164,716 codes (99.5085 %), 10,258,382 / 10,397,696 held-out block scales (98.660 %) | 1 float32 / tensor |
The 19 residual code mismatches in P4c sit on E2M1 round-to-nearest-even tie boundaries
(0.75 / 1.75 / 3.5); the 5 block-scale mismatches sit at the E4M3 saturation edge. They are
implementation artefacts, and the P4f residual is a limit on S precision, not on the recipe:
a 0.25 % error in S moves the code match from 99.9999 % to β 99 %.
Is P4f just "quantisation reconstruction"? No. Dequantising NVFP4 is a fixed transform
(w = LUT[code] Γ scale); knowing dequantised values does not let you predict the codes. P4f predicts
99.5 % of 173 M codes and 98.7 % of 10.4 M block scales from a PRNG stream keyed on the tensor name.
The parameter-free tests are P4a and P4b β there, zero scalars touch the target.
7. Statistical evidence
| Measurement | Checkpoint | Control | Reading |
|---|---|---|---|
| Dense matrix top-1 singular variance fraction | 0.00405β0.00415 | i.i.d. Gaussian 0.00389; Gaussian + planted direction 0.1245 | indistinguishable from noise |
| Stable rank (1024 Γ 1024) | 241β247 | 257 (i.i.d.); 8.0 (planted) | no trained structure |
| E2M1 code entropy | 2.9533 / 3.0 bits | β | near-uniform, as expected from a Gaussian source |
| Skew / excess kurtosis (BF16 dense) | β0.0005 / +2.888 | β | symmetric, light-tailed |
| Embedding off-diagonal cosine std | 0.0394 | isotropic 1/βd = 0.00902 |
4.4Γ real structure |
man~woman, Paris~France, Hello~hello, cat~dog |
0.224, 0.204, 0.449, 0.181 | β 0 for noise | learned semantics |
Structural tells:
weight_scale_2 = max|w| / 2688exactly on every tensor tested β the largest block scale saturates E4M3 at 448 with the top E2M1 magnitude 6. That is a construction, not a calibration result.lm_headrows are0.8913 Γthe alignedembed_tokensrows (per-row cosine 0.8909 Β± 0.0015, 64/64 rows > 0.8) whileconfig.jsondeclarestie_word_embeddings: false. Untied embeddings in a trained model are independent.- README claims ~14 T parameters; the repo's own index metadata says 6.85 T.
8. Actual inference output
Run from the packaged model.safetensors produced by build_artifact.py. Input prompt
Hello, how are you?, greedy decoding, 24 new tokens, 1 decoder layer.
prompt ids : [9098, 19, 2110, 600, 661, 38]
prompt tokens : ['Hello', ',', 'Δ how', 'Δ are', 'Δ you', '?']
generated ids : [38] * 24
new text : "?" * 24
full text : "Hello, how are you" + "?" * 25
| Metric | Value |
|---|---|
| Decoder layers executed | 1 of 114 |
| Tensors loaded | 21 (see Β§4) |
| Peak RSS | 7.74 GiB |
| Wall time (24 tokens) | 64.6 s |
| Per-token latency | 2.67 s |
| External model or API used | none |
Correction to an earlier draft of this card. A first pass reported "Hello, how are you?" followed by
20 Γ - at 12.37 s/token. Both numbers were artefacts of the ad-hoc research harness, not of the model:
it re-fetched embedding rows over HTTP on every step (hence the inflated latency), and its row indexing
was wrong β it fed rows 19β24 for every prompt token instead of the ids actually requested. The
packaged artifact holds the full embedding table in memory and indexes it correctly; the numbers above
are from that artifact. results/inference.json and results/inference.log are the raw record.
Logit magnitudes β 46, far outside a healthy range: expected when a 114-layer stack is cut to one.
9. Control experiments
Same code, same prompt, same greedy decoding, same 24 tokens, with the layer-0 trunk replaced by
N(0, 0.02) random tensors. The embedding table, lm_head and model.norm are kept β that is the
honest control for a truncation, because replacing the head would test a different question.
generated ids : [38] * 24
new text : "?" * 24
peak RSS : 7.73 GiB
per-token : 2.51 s
Real and control produce the same degenerate output β the same token id, 24 times. The logit scale differs (β 46 real vs β 344 control), which is the only measurable difference, and it reflects the residual-stream magnitude, not any learned capability.
The experiment is therefore non-discriminating: it demonstrates that the tensors load and that the forward pass runs end-to-end, and it says nothing about provenance. Presenting a truncation's output as evidence about whether the weights were trained would be wrong, and this card does not.
10. Hardware requirements
| Task | RAM | Disk | Time |
|---|---|---|---|
| Download the artifact from this repo | β | 9.5 GiB | β |
Rebuild from source (build_artifact.py) |
~11 GiB | 9.5 GiB cache + 9.5 GiB output | ~13 min at 20 MB/s |
Run inference (examples/inference.py) |
~10 GiB | 9.5 GiB | ~5 min for 24 tokens on 12 cores |
| Re-run P4a (633 tensors) | < 2 GiB | 30 MB | ~2 min |
| Re-run P4f (5 expert matrices) | ~4 GiB | 200 MB | ~3 min |
| Full statistical suite | ~6 GiB | 10 GiB (reuses the build cache) | ~6 min |
CPU only. No GPU, no bitsandbytes, no accelerate, no CUDA kernels β NVFP4 never has to be
dequantised for this artifact because layer 0 is dense BF16.
11. Installation and reproduction
git clone https://huggingface.co/adityawakharkar/numbat-forensics
cd numbat-forensics
python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt # numpy, requests, safetensors, torch (CPU wheel)
# 1. rebuild the artifact byte-for-byte from the pinned revision (verifies SHA256 per tensor)
python build_artifact.py --out .
# 2. run the packaged model
python examples/inference.py --model model.safetensors \
--prompt "Hello, how are you?" --max-new-tokens 24 --json results/inference.json
# 3. re-derive the forensic numbers
python scripts/p4a_normconv.py # 633 tensors bit-exact
python scripts/p4b_signprobe.py # sign test
python scripts/p4f_solveS.py # held-out NVFP4 rebuild
python scripts/p8_verify.py a b c d e # verification of every headline claim
12. Licensing and redistribution
Upstream status. The source repository burner22/numbat declares no license β no LICENSE
file, no cardData, no license tag on the model page. Redistribution rights for its weights and
tokenizer are therefore unclear, and nothing here should be read as a claim that they are granted.
What this repository contains. Following the repository owner's decision, this artifact does
include model.safetensors (9.46 GB) β a byte-exact subset of 21 tensors from the pinned revision, with
per-tensor SHA256 in tensor_hashes.json. It is published on the understanding that it is a
research/quotation-sized excerpt of a publicly downloadable checkpoint, for the purpose of reproducible
verification. If a rights holder objects, the correct action is to remove model.safetensors; everything
else here keeps working, because build_artifact.py rebuilds the file from the original location and
refuses to write anything if a hash does not match.
What is not included. The tokenizer. It is fetched from the pinned revision at run time
(examples/inference.py --fetch-tokenizer, the default) rather than bundled, so this repository does not
re-host it.
Code. MIT, see LICENSE. That license covers the analysis code, the truncated loader, the build and
reproduction scripts, and the documentation β it does not purport to license the upstream tensors.
13. SHA256 verification
sha256sum -c checksums.sha256 # every file in this repository
python build_artifact.py --verify-only # every tensor against tensor_hashes.json
model.safetensors.sha256.json is written next to the artifact after a successful build, and
results/verification.json records the audit that re-derived every headline number from live bytes
(30/30 artifacts, 648/648 inventory rows, 60-tensor P4a resample, 2 previously unused tensors,
P4c recount, 21/21 cached-tensor byte comparisons).
14. Research conclusions
Confirmed observations
- 11,374,080 norm and conv values across all 115 layers regenerate bit-exactly from
SHA256(tensor_name)β PCG64 βstandard_normal, with no fitted parameters. - 373,620,632 signs of non-zero E2M1 codes across 12 expert matrices match that stream with no fitted parameters.
- 99.5085 % of 173,015,040 expert codes regenerate with one float32 per tensor.
- Dense matrices are statistically indistinguishable from i.i.d. Gaussian noise; the planted-direction control shows what trained structure would look like.
- The embedding table carries real semantic structure (4.4Γ isotropic angular spread;
man~woman0.224,Hello~hello0.449). lm_head β 0.8913 Γ embed_tokenswhile the config declares untied embeddings.weight_scale_2 = max|w| / 2688exactly.
Hypotheses (not confirmed)
- That the checkpoint is entirely fabricated. We verified 0.0027 % of parameters bit-exactly; the
families tested dominate the byte count, but attention projections, routers and the per-tensor
input_scalescalars were never rebuilt. - That the embedding table was harvested from a specific real model β plausible and consistent with the data, but the donor is unidentified.
- Anything about provenance. Nothing here shows whose weights these are, or are not. No claim about Anthropic is made or supported.
Unresolved
- The per-tensor scalar
Shas no closed form we could find:S / weight_scale_2ranges 106.2 β 143.7 across tensors. Closing it would make P4f fully independent. - The 6.85 T vs ~14 T parameter discrepancy is unexplained; both figures come from the repository.
- Untested families: MLA projections (
q_a_proj,q_b_proj,kv_a_proj_with_mqa,kv_b_proj,o_gate_proj), themlp.gaterouters, and theinput_scale/weight_scale_2scalar streams.
Recommended next experiment: recover the closed form of S (test whether it is a fourth draw from
the same PCG64 stream, another 4 bytes of the name digest, or a table indexed by layer/role), then
rebuild an MLA layer's projections end-to-end. If S falls out, reconstruction becomes parameter-free
across ~99.6 % of the checkpoint's bytes.
Author
Aditya Wakharkar
- Hugging Face: @adityawakharkar
- GitHub: codewith-aditya
- LinkedIn: aditya-wakharkar
Independent security and model-forensics research. Questions, corrections and reproduction reports are
welcome β especially if you can close the unresolved scalar S (see Β§14).
15. Attribution
- Source checkpoint:
burner22/numbat, revision6b7f8fc5β¦β no license declared by the uploader. - Analysis, truncated loader, and inference implementation: MIT, see
LICENSE. - Full methodology and per-test numbers:
reports/research_report.md. - No repository code was executed at any point; the forward pass is an independent reimplementation
of the architecture described in the source
config.json.
- Downloads last month
- 307