hnet-smiles

Eight byte-level H-Nets (Hwang, Wang, Gu) trained on SMILES. All share one two-stage architecture (bytes to pieces to groups, Transformer stacks, cosine router) and one dataset. They differ in the training objective, and so in where they cut a molecule.

Code: DrHB/hnet-smiles. Post: How the training objective shapes learned tokens.

Models

Each folder holds model.safetensors and config.json.

folder objective parameters held-out metric
next-byte-30m predict the next byte, causal 30,342,784 0.7681 bits per byte
fingerprint-30m predict the 2,048-bit Morgan fingerprint (radius 3) from the clean string 32,901,248 Tanimoto 0.831
descriptors-30m regress 185 RDKit descriptors from the clean string 30,991,673 R² 0.998
masked-fragments-30m fill in masked bytes, 25 to 45% per molecule, whole fragments first 30,342,784 92.8% of masked bytes at a 35% rate
next-byte-20m as above 19,555,520 0.7681 bits per byte
fingerprint-20m as above, head reads one position 22,245,184 Tanimoto 0.832
descriptors-20m as above, head reads one position 20,335,609 R² 0.998
masked-fragments-20m as above 19,555,520 93.2% of masked bytes at a 35% rate

The 30M round uses widths 256/384/512, AdamW and 3 epochs (101,289 steps). The 20M round uses widths 256/320/384, Muon and 5 epochs (168,815 steps). In the 30M round the fingerprint and descriptor heads read the mean and max over bytes; in the 20M round they read one position.

Where they cut

50,000 held-out molecules, first-level pieces.

bytes per piece distinct pieces multi-character atoms split preference for BRICS bonds
next-byte-30m 2.75 8,620 18% -0.04
fingerprint-30m 2.59 4,271 5% +0.16
descriptors-30m 2.82 3,810 65% +0.14
masked-fragments-30m 4.33 14,928 0.4% +0.38
next-byte-20m 2.76 10,450 6% -0.06
fingerprint-20m 2.91 15,640 70% +0.14
descriptors-20m 2.84 3,709 29% +0.13
masked-fragments-20m 4.33 14,262 2% +0.39

Data

90,955,563 canonical, non-stereo SMILES: PubChem (89.8M rows), COCONUT, ChEBI, LIPID MAPS and structures from public spectral libraries. 50,000 rows were held out. Input is raw bytes with BOS, EOS, PAD and MASK symbols (vocabulary 260).

Use

git clone https://github.com/DrHB/hnet-smiles && cd hnet-smiles && pip install -r requirements.txt
import torch
from data import encode, split
from model import load

model = load("Asimo/hnet-smiles/masked-fragments-30m")
smiles = "CC(C)S(=O)(=O)CCSc1cccc(C(=O)O)c1"
x, n = encode([smiles])
with torch.no_grad():
    out, r = model(x, n)
print(split(smiles, r["b"][0]))
# ['CC(C)', 'S(=O)(=O)', 'CC', 'Sc', '1cccc', '(C(=O)O', ')c1']

out is byte logits for the next-byte and masked-fragment models, 2,048 fingerprint logits for the fingerprint models, and 185 descriptor z-scores for the descriptor models. r["b"] marks piece starts, r["bg"] group starts, and r["main"] is the main-network output.

Limits

Small models, one notation (canonical SMILES), Transformer outer layers where the paper uses Mamba-2. Benchmark molecules were not removed from pretraining; no benchmark label was in the corpus.

Licence

MIT.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for Asimo/hnet-smiles