burner22

/

numbat

Like 318

I downloaded the full checkpoint (all 784 files, 7.02 TiB) at revision 6b7f8fc548cbafe9bec441f03af2d92a55cb4012 and audited it against the local files. Nothing from this repository was executed. Headline: the packaging is internally consistent, and 635 of its 1,576,410 named tensors reproduce bit-exactly from a function of their own names.

This extends @FloxUnder 's audit (#13), which established the recipe on 43 tensors. I applied it to every norm and conv1d tensor in the checkpoint.

Corrections to the first version of this post, all applied here: the norm total is 377, not 289 (289 excluded o_norm, enorm and hnorm); the magnitude nibbles are not from a separate stream — they correlate with |z| at +0.79 from the same seed, though the exact rule is not recovered; the file total is 7.02 TiB, not 7.01; the reproduction reads 90,072 bytes, not ~1 MB; the logical parameter count is 13.68T against a packed 6.849T (ratio 1.9975), not the ~14.5T quoted in the thread below — that figure wrongly counted the 854 GB of E4M3 block-scale tensors as parameters, which is impossible on its own terms since it exceeds 2× the packed count; and several conclusions I drew from summary statistics were stronger than the statistics support.

What is correct

Genuinely, and this matters for anyone auditing later:

So structural auditing alone will clear this repository. The problem is one layer below.

1. Layer norms — 377 / 377 bit-exact

seed = int.from_bytes(hashlib.sha256(tensor_name.encode()).digest()[:8], "big")
z    = np.random.Generator(np.random.PCG64(seed)).standard_normal(shape, dtype=np.float32)
w    = bf16(np.exp(np.float32(0.06) * z))   # bf16 = round-to-nearest-even

377 of 377 norm tensors reproduce byte-for-byte: 115 input_layernorm, 115 post_attention_layernorm, 29 q_a_layernorm, 29 kv_a_layernorm, 86 o_norm, 1 enorm, 1 hnorm, and model.norm. They live in 112 of the 772 shards — the other 660 contain only FP4 payload and scale tensors.

The 289 in the first version was a subtotal that excluded o_norm (86), enorm (1) and hnorm (1): 289 + 86 + 1 + 1 = 377.

Measured distribution across the family: mean 1.0017, std 0.0600 — at initialiser values, with no per-dimension structure.

2. Convolution kernels — 258 / 258 bit-exact

w = bf16(z * np.float32(1.0 / 6.0))

258 of 258 q_conv1d, k_conv1d, v_conv1d kernels reproduce byte-for-byte. Note the constant 1/6: the E2M1 maximum magnitude, reused as an init scale.

Together with the norms that is 635 tensors out of 1,576,410 — 0.04 % by tensor count.

3. The NVFP4 sign bits — 100.0000 %

The quantised payload is 6.21 TiB of uint8, two E2M1 nibbles per byte. Each nibble's sign bit (bit 3) equals signbit(z) for the same name-seeded stream:

• 24 tensors across 12 shards — 192,000,000 elements — 100.0000 %
• wider sweep, 13 tensors, 52,000,000 elements — 100.0000 %

This is a sampled result, not an exhaustive pass over all 6.21 TiB.

4. The magnitudes — related to z, rule not recovered

Corrected. I originally wrote that the magnitudes came from a separately-seeded stream. That was wrong.

The decoded magnitude correlates with |z| from the same name-seeded stream at +0.79. Controls break the correlation: offsetting into the same stream gives +0.0005, +0.0001, +0.0007, and a different tensor's seed gives +0.0002. Per-block rank agreement is 22.9 % against a 6.25 % chance baseline.

So the magnitudes are a function of the same z, not an independent draw — but I have not recovered the exact rule. The best fit I found, bucketize(|z|*2.5, E2M1_BOUNDS), agrees on only 38.6 % of nibbles.

What the marginal shows: entropy 2.9535 bits (uniform = 3.0000), frequencies [0.100, 0.187, 0.157, 0.126, 0.137, 0.116, 0.091, 0.087], and 10.0 % zeros. A straight quantisation of a standard normal through this repo's own E2M1_BOUNDS would give [0.599, 0.175, 0.121, 0.066, 0.034, 0.006, 0.000, 0.000] — about 60 % zeros. The observed marginal is not that. It is consistent with some other scaling applied to z, which is what I would expect given the +0.79 correlation, but I cannot state the rule.

5. The block scales have very few distinct values

weight_scale holds one E4M3 scale per 16 elements:

• shard 1: 2,162,688 entries, 45 distinct byte values (82–126)
• shard 318: 2,162,688 entries, 44 distinct byte values

I do not know what produced this. It is a measurement, not a conclusion: 45 values across 2.1 M blocks. Whether that is possible for a genuine post-training quantisation of these tensors is a question for the uploader.

6. How far the fingerprint goes — and where it stops

"Corr." above mixes three different measurements — byte equality for the first two rows, binary agreement for the sign bits, Pearson correlation for the rest. Only the first two are complete reconstruction.

So at least two generation paths exist: one seeded by tensor name, producing the norms and conv kernels, and something else producing the remainder. lm_head is Gaussian, layer-114 down_proj is heavy-tailed (excess kurtosis +3.02), and A_log shows −0.08 correlation with head index, where a trained GDN A_log decays monotonically across heads.

I am not claiming every tensor derives from its name, and I am not able to regenerate the magnitudes or the whole checkpoint.

7. The training log

training_log_tail.txt, 400 steps from 1,830,001 to 1,830,400:

• loss 1.0359 – 1.0649, mean 1.0512, std 0.0052
• least-squares trend −5.9e-6 per step
• moe_balance_loss 0.0212 – 0.0222, within 4.63 % of its mean across the window
• tokens/s 10.51–13.72 M, mfu 30.70–40.10 %

The README describes this as step 1,830,000 of a ~1.96 M step run, and the log carries lr_phase=decay.

Those are the numbers. I originally wrote that this "is not a training curve" and described it as "a random walk around a constant, with no load-balancing transients" — that was my interpretation, not something the summary statistics establish. Draw your own conclusion from the file.

ckpt_metadata.json reports MMLU 0.8849 (n=14042), ARC-c 0.7201, HellaSwag acc_norm 0.8896. Those are frontier-level numbers in a JSON file sitting next to weights that reproduce from their names.

8. The tokenizer

It is a functioning byte-level BPE — 131,072 tokens, 130,808 merges, and "The quick brown fox jumps over the lazy dog." round-trips exactly in 10 tokens. But:

• the pre-tokenizer is the GPT-2 regex
• 0 CJK tokens; 96,033 of 100,078 non-ASCII tokens are plain GPT-2 byte-glyphs
• <|role|>, <|msg|>, <|tool|>, <|toolcall|> appear in none of the 130,808 merge rules, so they are not reachable through the BPE and carry no learned merges

Anthropic's format is <antml:...> with Human:/Assistant: turns. Nothing here matches that.

9. On the Anthropic attribution

The codename is real: // @[MODEL LAUNCH]: Remove this section when we launch numbat. appears in src/constants/prompts.ts in the leaked Claude Code source. That sentence authenticates a codename. It says nothing about a checkpoint, and a known codename is not a checksum.

Minimal reproduction

The first block below reads 90,072 bytes: a 16,344-byte shard header plus a 24,576-byte norm tensor and a 49,152-byte conv tensor. The FP4 block adds 1,000,000 bytes. No model download, no repository code executed.

import hashlib, json, struct, numpy as np, requests

REV = "6b7f8fc548cbafe9bec441f03af2d92a55cb4012"
URL = f"https://huggingface.co/burner22/numbat/resolve/{REV}/model-00001-of-00772.safetensors"

def rng_read(start, end):
    r = requests.get(URL, headers={"Range": f"bytes={start}-{end}",
                                   "Accept-Encoding": "identity"}, stream=True, timeout=30)
    assert r.status_code == 206
    assert r.headers["Content-Range"].startswith(f"bytes {start}-{end}/")
    d = r.raw.read(end - start + 2)
    assert len(d) == end - start + 1
    return d

def bf16_bytes(x):
    b = x.view(np.uint32)
    return ((b + 0x7fff + ((b >> 16) & 1)) >> 16).astype("<u2").tobytes()

n = struct.unpack("<Q", rng_read(0, 7))[0]
hdr, base = json.loads(rng_read(8, 7 + n)), 8 + n

for name, recipe in [("model.layers.0.input_layernorm.weight", "norm"),
                     ("model.layers.0.self_attn.k_conv1d.weight", "conv")]:
    it   = hdr[name]
    seed = int.from_bytes(hashlib.sha256(name.encode()).digest()[:8], "big")
    z    = np.random.Generator(np.random.PCG64(seed)).standard_normal(it["shape"], dtype=np.float32)
    pred = bf16_bytes(np.exp(np.float32(0.06) * z) if recipe == "norm"
                      else z * np.float32(1/6))
    s, e = it["data_offsets"]
    assert pred == rng_read(base + s, base + e - 1), f"MISMATCH {name}"
    print(name, "EXACT")

For the FP4 sign test, on the sampled FP4 tensors:

name = "model.layers.2.mlp.experts.0.down_proj.weight"
it   = hdr[name]
s, e = it["data_offsets"]
take = min(e - s, 1_000_000)
raw  = np.frombuffer(rng_read(base + s, base + s + take - 1), dtype=np.uint8)

codes = np.empty(raw.size * 2, np.uint8)
codes[0::2] = raw & 0x0F     # element 2i   = low nibble
codes[1::2] = raw >> 4       # element 2i+1 = high nibble

seed = int.from_bytes(hashlib.sha256(name.encode()).digest()[:8], "big")
z    = np.random.Generator(np.random.PCG64(seed)).standard_normal(codes.size, dtype=np.float32)
print((((codes & 8) != 0) == np.signbit(z)).mean())   # -> 1.0

A negative control, which the first version omitted: keep the payload fixed and change one character in the string used to derive the seed. Sign agreement drops from 1.0000 to 0.5005 — exactly the chance rate for a random binary predictor. So the check is sensitive to the name rather than returning True unconditionally. (Verified: mutation returns 0.5005 on 2,000,000 elements.)

My scripts are available on request. The HTTP reproduction needs numpy and requests; the meta-device module-tree check also needs PyTorch.

Questions for @burner22

1. How are these bit-exact reproductions consistent with a trained checkpoint? If these parameters were intentionally frozen, what training configuration keeps 377 norms and 258 conv kernels at their exact initialiser values, byte for byte, across 112 shards?
2. If the NVFP4 payload is a real quantisation, what maps z to the magnitude nibble? They correlate with |z| at +0.79 but I could not recover the rule, and the marginal is not what a straightforward Gaussian quantisation produces. And why do 2.1 M block scales take only 45 distinct values?
3. Can you publish the generation and quantisation procedure? The tensor-to-shard mapping is already in the index and I verified all 1,576,410 pointers, so that part does not need publishing — the generator does.

Ask

Until those are answered, I would suggest the model card's Anthropic / "Mythos-class" attribution be removed and the repository be marked as synthetic — so nobody else spends 7.02 TiB of bandwidth and storage, as I did, expecting a model.

To anyone reading: this is not a reason to distrust the safetensors format or HF's manifest. Both did their job correctly. It is a demonstration that a convincing multi-terabyte "leaked frontier model" can be assembled from a hash function and a PRNG, and that the integrity checks I ran will pass.