Devatri/escape-blt-patcher-adapted

ESCAPE: BLT patcher adapted by distillation (99.5M)

The domain-adaptation arm (pre-registered as STEP 9) of Project ESCAPE (Team Trimax, ANLP, IIIT Hyderabad, October 2026). The BLT entropy patcher's own architecture (99,512,064 parameters; dim 768, 14 layers, 12 heads, untied embeddings) is initialised from itazap/blt-1b-hf at revision 91aa6b8e and trained with the same loss and data as the distilled student (α = 0.1, T = 2; two epochs, 20,580 steps, batch 16, AdamW lr 2e-4, 100 warm-up steps, seed 20260927). Like the student, it reads files through fresh 512-byte windows.

Results (from the ESCAPE final report; exploratory)

	Python	C++	English
Boundary F1 vs. the original patcher, held-out windows	0.955	0.958	0.937
Start-F1 margin minus the distilled student's [95% CI]	+0.006 [0.004, 0.008]	+0.005 [0.003, 0.007]	+0.029 [0.026, 0.032]

The pre-registered criterion (a gain larger than the width of the student's confidence interval in at least two domains) was not met (1 of 3). In code the adapted model matches the unadapted patcher read through the same windows: fine-tuning recovers the teacher rather than improving on it. Margins were scored on the pre-registered 600-file code subset and the 240-file English test split. Bits/byte on held-out windows: 1.09.

Licence

This is a fine-tune of itazap/blt-1b-hf, which is labelled Apache-2.0 and is a conversion of Meta's facebook/blt-1b, released under its own licence. Check that licence before redistributing these weights.

Use

Clone the ESCAPE repository, then:

import torch
from huggingface_hub import hf_hub_download
from escape_distill.student_blt import StudentBLT, StudentConfig

ckpt = torch.load(hf_hub_download("Devatri/escape-blt-patcher-adapted", "student.pt"), map_location="cpu", weights_only=False)
model = StudentBLT(StudentConfig(**ckpt["config"])).eval()
model.load_state_dict(ckpt["model_state"])

model.safetensors holds the same weights; config.json has the model configuration and training arguments, train_log.json the loss curve, and distill_eval.json the held-out agreement with the teacher per domain.

Citation

Teacher model: Pagnoni et al., 2025, Byte Latent Transformer: Patches Scale Better Than Tokens, ACL 2025.