leviathan-19t
A 19.0-trillion-parameter model that contains nothing.
What it demonstrates
Hugging Face computes a repository's parameter count from safetensors headers
alone. It sums prod(shape) per tensor and never reads the tensor data, so the
count is whatever the headers declare. Here they declare 4,422
tensors of shape [65536, 65536] in F4 (4 bits/param), plus one
[4294967296, 1] position-embedding tensor, across 444 shards.
That is 15.1% above the previous empty-model record of 16,501,264,351,232 parameters held by tsfrm/vacuum-16t, which is the work this repository builds on.
The headers are honest. safetensors requires tensor byte ranges to tile the
data section contiguously and requires the file length to equal
8 + header + data exactly, so overlapping tensors or truncated files are
rejected by the format. Every declared byte is really present. The bytes are
simply all 0x00.
Real cost
| Declared parameters | 18,996,640,350,208 |
| Declared bytes | 9,498,320,175,104 (9.50 TB) |
| Storage quota consumed | 9.50 TB โ quota bills declared bytes |
| Bytes actually transferred | a few hundred KB |
Xet content-defined chunking deduplicates the transfer: every block of weight data is byte-identical, hashes to one chunk, and crosses the wire once. Storage quota is not deduplicated โ it bills the logical size. The saving is in bandwidth only, which is the constraint that decides how large one of these can get.
Locally the shards cost nothing either. They are sparse files: the data section
is a filesystem hole created with ftruncate, so each shard reports its full
size to stat, reads back as zeros, and allocates no blocks.
F4 at 4 bits is the narrowest dtype safetensors supports, which fixes the
exchange rate at two parameters per byte. That, and not compute, is what bounds
a model like this.
Limitations
It has no capabilities.
- Downloads last month
- 132