Devatri/escape-blt-entropy-student-balanced

ESCAPE: byte-balanced distilled BLT patcher (14.3M)

A follow-up arm of the distilled student of Project ESCAPE (Team Trimax, ANLP, IIIT Hyderabad, October 2026). Identical to it (14,262,144 parameters; online distillation from the BLT patcher with α = 0.1, T = 2; same data, seed and schedule) except that every domain is drawn to an equal expected share of bytes. In the unbalanced run C++ is 73% of the training bytes.

Results (from the ESCAPE final report; exploratory)

	Python	C++	English
Boundary F1 vs. teacher, held-out windows (distilled student)	0.864 (0.869)	0.860 (0.880)	0.853 (0.834)
Start-F1 margin minus the distilled student's [95% CI]	−0.002 [−0.003, −0.001]	−0.002 [−0.0035, −0.0001]	+0.010 [0.008, 0.012]

Margins were scored on the pre-registered 600-file code subset and the 240-file English test split (full-corpus extraction did not fit before the deadline), so these results are exploratory. Balancing trades code for prose. Bits/byte on held-out windows: 1.27.

Use

Clone the ESCAPE repository, then:

import torch
from huggingface_hub import hf_hub_download
from escape_distill.student_blt import StudentBLT, StudentConfig

ckpt = torch.load(hf_hub_download("Devatri/escape-blt-entropy-student-balanced", "student.pt"), map_location="cpu", weights_only=False)
model = StudentBLT(StudentConfig(**ckpt["config"])).eval()
model.load_state_dict(ckpt["model_state"])

model.safetensors holds the same weights; config.json has the model configuration and training arguments, train_log.json the loss curve, and distill_eval.json the held-out agreement with the teacher per domain.

Citation

Teacher model: Pagnoni et al., 2025, Byte Latent Transformer: Patches Scale Better Than Tokens, ACL 2025.

No licence has been chosen for this model; the training data keep their own licences.