HoLo_ZeRo — the input layer has 0 learned parameters

The casing is the signal: HoLoZeRo = 10101010 — a fluctuation between 0 and 1, which is also what the model eats. (Named for fun first; the meaning arrived later, as it should.)

A lean byte transformer (~25M, dim512/x6) whose front door is bytes -> frozen 27-D HSL signal features (4.6 KB LUT) -> fixed zero-pad -> attention. No tokenizer, no embedding table, no learned input projection. Trained 30k steps on one RTX 4070 over a 3-modality byte mix (EN+KO text / video windows / audio-caption windows).

Proof of operation, not quality. Read as a measured possibility — capability per FLOP and per watt — not a victory over embeddings. Model code & training harness: Woojiggun/HoLo_ZeRo - the zero door package: hsl-embedding-zero (PyPI) · demo Space: ggunio/HoLo_ZeRo_demo · parent project: HoLo 6.5.1.

DOI (software): 10.5281/zenodo.20643551 · paper: 10.5281/zenodo.20581805 Weights CC-BY-SA 4.0 (KoWiki share-alike inheritance), code MIT. Trained only on open text/video sources - FineWeb-Edu, KoWiki, Gutenberg, Blender open movies).

Use with PyTorch

pip install torch hsl-embedding huggingface_hub

import sys, torch, torch.nn.functional as F
import hsl_embedding as hsl
from huggingface_hub import hf_hub_download, snapshot_download

# the lean body (AsymDec) ships with the demo Space - reuse it as a library
space = snapshot_download("ggunio/HoLo_ZeRo_demo", repo_type="space")
sys.path.insert(0, f"{space}/core")
from hsl_asym_dec import AsymDec

class FixedPad(torch.nn.Module):              # the zero door: 0 learned input parameters
    def __init__(self, in_dim, dim):
        super().__init__(); self.p = dim - in_dim
    def forward(self, x): return F.pad(x, (0, self.p))

ck = torch.load(hf_hub_download("ggunio/HoLo_ZeRo", "holo_zero.pt"), map_location="cpu")
cfg = ck["cfg"]; K = cfg["K"]                 # dim 512 / x6 / h8 / K=8 / door="zero"
model = AsymDec(dim=cfg["dim"], dec_layers=cfg["dec_layers"], heads=cfg["heads"], K=K, with_cross=False)
model.in_proj = FixedPad(K * hsl.FEAT_DIM, cfg["dim"])
model.out_proj = FixedPad(hsl.FEAT_DIM, cfg["dim"])
model.load_state_dict(ck["model"]); model.eval()

# inputs are just bytes -> frozen HSL features (no tokenizer):
emb = hsl.Embedding()
ids = torch.tensor([list(b"The universe is ")], dtype=torch.long)
ctx = (ids.shape[1] // K) * K
fi = emb(ids[:, :ctx]).reshape(1, -1, K, hsl.FEAT_DIM)   # packed prefix slots
fo = emb(ids[:, ctx:])                                   # per-byte output stream
logits = model(fi, fo)                                   # [1, L, 258] next-byte logits
# full sampling loop (~20 lines): app.py in the demo Space