hnet-smiles
Eight byte-level H-Nets (Hwang, Wang, Gu) trained on SMILES. All share one two-stage architecture (bytes to pieces to groups, Transformer stacks, cosine router) and one dataset. They differ in the training objective, and so in where they cut a molecule.
Code: DrHB/hnet-smiles. Post: How the training objective shapes learned tokens.
Models
Each folder holds model.safetensors and config.json.
| folder | objective | parameters | held-out metric |
|---|---|---|---|
next-byte-30m |
predict the next byte, causal | 30,342,784 | 0.7681 bits per byte |
fingerprint-30m |
predict the 2,048-bit Morgan fingerprint (radius 3) from the clean string | 32,901,248 | Tanimoto 0.831 |
descriptors-30m |
regress 185 RDKit descriptors from the clean string | 30,991,673 | R² 0.998 |
masked-fragments-30m |
fill in masked bytes, 25 to 45% per molecule, whole fragments first | 30,342,784 | 92.8% of masked bytes at a 35% rate |
next-byte-20m |
as above | 19,555,520 | 0.7681 bits per byte |
fingerprint-20m |
as above, head reads one position | 22,245,184 | Tanimoto 0.832 |
descriptors-20m |
as above, head reads one position | 20,335,609 | R² 0.998 |
masked-fragments-20m |
as above | 19,555,520 | 93.2% of masked bytes at a 35% rate |
The 30M round uses widths 256/384/512, AdamW and 3 epochs (101,289 steps). The 20M round uses widths 256/320/384, Muon and 5 epochs (168,815 steps). In the 30M round the fingerprint and descriptor heads read the mean and max over bytes; in the 20M round they read one position.
Where they cut
50,000 held-out molecules, first-level pieces.
| bytes per piece | distinct pieces | multi-character atoms split | preference for BRICS bonds | |
|---|---|---|---|---|
next-byte-30m |
2.75 | 8,620 | 18% | -0.04 |
fingerprint-30m |
2.59 | 4,271 | 5% | +0.16 |
descriptors-30m |
2.82 | 3,810 | 65% | +0.14 |
masked-fragments-30m |
4.33 | 14,928 | 0.4% | +0.38 |
next-byte-20m |
2.76 | 10,450 | 6% | -0.06 |
fingerprint-20m |
2.91 | 15,640 | 70% | +0.14 |
descriptors-20m |
2.84 | 3,709 | 29% | +0.13 |
masked-fragments-20m |
4.33 | 14,262 | 2% | +0.39 |
Data
90,955,563 canonical, non-stereo SMILES: PubChem (89.8M rows), COCONUT, ChEBI, LIPID MAPS and structures from public spectral libraries. 50,000 rows were held out. Input is raw bytes with BOS, EOS, PAD and MASK symbols (vocabulary 260).
Use
git clone https://github.com/DrHB/hnet-smiles && cd hnet-smiles && pip install -r requirements.txt
import torch
from data import encode, split
from model import load
model = load("Asimo/hnet-smiles/masked-fragments-30m")
smiles = "CC(C)S(=O)(=O)CCSc1cccc(C(=O)O)c1"
x, n = encode([smiles])
with torch.no_grad():
out, r = model(x, n)
print(split(smiles, r["b"][0]))
# ['CC(C)', 'S(=O)(=O)', 'CC', 'Sc', '1cccc', '(C(=O)O', ')c1']
out is byte logits for the next-byte and masked-fragment models, 2,048 fingerprint logits for the fingerprint models, and 185 descriptor z-scores for the descriptor models. r["b"] marks piece starts, r["bg"] group starts, and r["main"] is the main-network output.
Limits
Small models, one notation (canonical SMILES), Transformer outer layers where the paper uses Mamba-2. Benchmark molecules were not removed from pretraining; no benchmark label was in the corpus.
Licence
MIT.