I conducted a forensic inspection of burner22/numbat using HTTP Range requests to inspect the safetensors headers, scalar scales, and unquantized BF16/FP32 weight slices without needing to download the full 7.72 TB payload.

While the repository structure (model.safetensors.index.json across 772 shards and modeling_numbat.py) is carefully constructed so that load_state_dict(strict=True) succeeds, the actual numerical values inside the tensors provide conclusive mathematical proof that the weights were procedurally generated by a synthetic RNG script and never trained.

Here are the 5 smoking guns:

1. input_scale Violates Forward-Pass Causality and Follows a Synthetic LogUniform(0.05, 0.4) Distribution

In modeling_numbat.py (line 108), NVFP4Linear.forward explicitly defines input_scale as a purely activation-dependent calibration constant (amax(x) / 448 / 6). It depends only on the input tensor x and has zero dependence on the module's weight matrix W.

Yet in the checkpoint:

Identical-input modules have wildly different input_scales: Within any single routed expert (NumbatMLP), gate_proj and up_proj receive the exact same input tensor x. Even more tellingly, in Shared Experts and Attention blocks (where all tokens pass through), q_proj, k_proj, v_proj, and g_proj (in GDN layers) or q_a_proj, kv_a_proj_with_mqa, and o_gate_proj (in MLA layers) all consume the exact same self.input_layernorm(x) tensor. Yet in Shard 1 & Shard 9:
Layer 2 GDN (same input x): q_proj.input_scale = 0.2846, k_proj.input_scale = 0.1575, v_proj.input_scale = 0.2375, g_proj.input_scale = 0.3355.
Layer 3 Shared Expert (same input x): gate_proj.input_scale = 0.2998 vs. up_proj.input_scale = 0.0721 (a 4.15x discrepancy on the exact same tensor).
down_proj has the exact same activation scale distribution as gate_proj / up_proj: In real SwiGLU networks, down_proj takes silu(gate_proj(x)) * up_proj(x) (unnormalized post-activation values) rather than RMSNorm outputs, which differs in dynamic range by orders of magnitude. Yet across all 514 scalars in Shard 10:
gate_proj.input_scale range: [0.0504, 0.3990]
up_proj.input_scale range: [0.0501, 0.3864]
down_proj.input_scale range: [0.0501, 0.3978]
Exact match to LogUniform(0.05, 0.4): Mapping log(input_scale) across the 514 scalars in Shard 10 onto [0, 1] yields a sample mean of 0.4916 (theoretical Uniform(0,1) = 0.5000) and a sample standard deviation of 0.2864 (theoretical
1
/
12

0.2887
1/
12
​

=0.2887). The author literally generated every input_scale independently via torch.empty(...).uniform_(math.log(0.05), math.log(0.4)).exp().
2. Unquantized BF16 Weights Are i.i.d. Synthetic Noise With Zero Cross-Layer or Cross-Matrix Correlation

To rule out a broken quantization script on top of real weights, I inspected the unquantized BF16 layers (layers.0, layers.1, layers.2) in Shard 1 (model-00001-of-00772.safetensors):

RMSNorm weights are pure i.i.d. Gaussian noise 1.0 + N(0, 0.06²): In any genuinely trained Transformer, residual stream outlier channels cause input_layernorm and post_attention_layernorm weights to develop heavy tails (kurtosis
>
10
>10 to
50
+
50+) and strong channel-wise correlation across layers. In numbat:
layers.0.input_layernorm.weight: mean = 1.0027, std = 0.0604, kurtosis = 3.055
layers.0.post_attention_layernorm.weight: mean = 1.0020, std = 0.0607, kurtosis = 3.029
layers.1.input_layernorm.weight: mean = 1.0022, std = 0.0596, kurtosis = 3.075
layers.2.input_layernorm.weight: mean = 1.0009, std = 0.0599, kurtosis = 3.035
A kurtosis of 3.0 is the exact theoretical value of a pure Gaussian distribution. Furthermore, the Pearson correlations between these 12,288-dim vectors are zero:
Corr(layer0_in_ln, layer0_post_ln) = 0.0013
Corr(layer0_in_ln, layer1_in_ln) = 0.0160
Corr(layer1_in_ln, layer2_in_ln) = -0.0040
Zero structural coupling between gate_proj, up_proj, and RMSNorm: In a trained SwiGLU MLP, gate_proj and up_proj operate on the same normalized features, so their column L2 norms (feature sensitivity per input dimension) correlate strongly with each other and with the preceding post_attention_layernorm. In numbat Layer 0:
Corr(col_norm(gate_proj), col_norm(up_proj)) = 0.0017
Corr(post_attention_layernorm, col_norm(gate_proj)) = -0.0126
Row-wise cosine similarity between gate_proj and up_proj: mean = -0.00048, std = 0.00948 (theoretical random orthogonal vectors in
𝑅
12288
R
12288
:
1
/
12288

0.00902
1/
12288
​

=0.00902).
Flat Marchenko-Pastur Singular Value Spectrum: Computing the SVD of layers.0.self_attn.a_proj.weight (48 × 12288) yields singular values flatly distributed between 0.442 and 0.798—devoid of any learned low-rank structure. Similarly, q_conv1d, k_conv1d, and v_conv1d ([6144, 1, 4]) have mean = 0.0000, std = 0.167, and kurtosis = 2.93 / 3.02 / 3.01—pure uninitialized Gaussian noise.
3. Bit-Level Forensics of weight_scale_2 Exposes the Synthetic Generation Formula

In Shard 10, while all 514 input_scale FP32 scalars are unique, the 514 weight_scale_2 FP32 scalars contain only 143 unique values (many unrelated experts share bit-identical weight_scale_2 values like 0x37caaaab or 0x37d55555).

Why? Because in NVFP4, weight_scale_2 = amax / (448 * 6) = amax / 2688. When you multiply every weight_scale_2 in Shard 10 by 2688.0 and inspect its IEEE-754 hex representation:

0.00002416 * 2688 = 0.06494140625
→
→ 0x3d850000
0.00002525 * 2688 = 0.06787109375
→
→ 0x3d8b0000
0.00002543 * 2688 = 0.06835937500
→
→ 0x3d8c0000
0.00002616 * 2688 = 0.07031250000
→
→ 0x3d900000

Every single weight_scale_2 * 2688 has 0x0000 in its lower 16 mantissa bits! The generator script synthesized fake BF16 amax values in a tight interval ([0.064, 0.172]), where only ~150 discrete BF16 grid points exist, and divided them by 2688.0 in FP32. Furthermore, in our 64 KB sample (8,192 blocks of 16 elements) of experts.147.gate_proj.weight, 8,192 out of 8,192 blocks (100.0%) hit the maximum E2M1 code ±6.0 (nibble 7 or 15), confirming that synthetic random blocks were generated and normalized directly onto the grid.

4. Hardware & Toolchain Contradictions: NVIDIA modelopt NVFP4 on AWS Trainium2
   In the README.md FAQ, the author claims: "The eval team quantizes every async checkpoint to NVFP4 so evals run on a slice of Trainium2s instead of the full training cluster."
   Meanwhile, config.json and hf_quant_config.json specify: "quant_algo": "NVFP4", "producer": {"name": "modelopt", "version": "0.47.0"}.
   NVFP4 (16-element block size, FP8 E4M3 block scales, E2M1 data) is an NVIDIA Blackwell-specific microscaling format produced by NVIDIA TensorRT Model Optimizer (modelopt).
   AWS Trainium2 (Neuron) does not have hardware tensor cores for NVIDIA's proprietary 16-element E4M3-scaled NVFP4 format (Trainium2 supports OCP standard MXFP4, which uses a 32-element block size with an E8M0 shared power-of-two exponent). Nobody quantizes a checkpoint using NVIDIA's modelopt into an NVIDIA-proprietary Blackwell format to run evaluations on AWS Trainium2.

5. training_log_tail.txt and Architecture Naming Give Away the Franken-Model Mashup
   Absurd Eval Loop in training_log_tail.txt: The training log claims that every 50 steps (~5.5 minutes), an async job quantizes a 14T parameter model (async checkpoint in 157.8s) and runs mmlu, arc_challenge, and hellaswag in 240s–282s on a tiny 2% subset: [step 1830096] eval(step 1830050). loss=1.1908 mmlu=0.8719(n=281) arc_challenge=0.7083(n=24) hellaswag=0.8806(n=201) eval_took=282s Because n=24 for arc_challenge (
   17
   /
   24

   0.708333
   17/24=0.708333), all 8 evaluations across the 400-step log print the exact same arc_challenge=0.7083(n=24). No frontier lab monitors a 14T pretraining run every 5 minutes using a 24-question multiple-choice accuracy metric.
   Copy-Pasted DeepSeek-V3 + Qwen3-Next Naming: Anthropic's internal training stack (which isn't even based on Hugging Face transformers) does not use Hugging Face PreTrainedModel attribute names. Yet every tensor name in these 772 shards is a 1-to-1 copy of DeepSeek-V3's Hugging Face implementation (kv_a_proj_with_mqa, e_score_correction_bias, routed_scaling_factor, enorm, hnorm, eh_proj) combined with Gated DeltaNet (A_log, dt_bias, a_proj, b_proj, o_norm). The author wrote modeling_numbat.py first by stitching together open-source DeepSeek-V3 + Gated DeltaNet code, and then ran a script to populate 7.72 TB of synthetic tensors matching its state_dict.

TL;DR: Do not waste bandwidth downloading these 772 shards, and do not run trust_remote_code=True on unverified repositories. It is a well-crafted, procedurally generated hoax capitalizing on the numbat codename from the March Claude Code string leak.