WyFormer-LeMat-Ehull

A WyFormer generator of Wyckoff representations, conditioned on the energy above the convex hull with classifier-free guidance. Ask for genes near the hull and the model picks the space group, the composition and the Wyckoff positions.

pip install wyckoff-transformer
wyformer-generate genes.json.gz --hf-model SymmetryAdvantage/WyFormer-LeMat-Ehull \
    --condition energy_above_hull=0.05 --guidance-scale 5 \
    --initial-n-samples 2000 --firm-n-samples 1000

The repository README shows how to relax the genes into 3D structures with an MLIP.

Choosing the condition

• energy_above_hull=0.05 with --guidance-scale 5 is the recommended setting. Asking for 0 eV/atom makes the model reproduce structures it has seen, so fewer genes are novel.
• Guidance trades validity for quality. About 61% of draws at w = 5 are formally valid and 86% at w = 3. --initial-n-samples should be about twice --firm-n-samples at w = 5.
• --guidance-scale 1 samples the plain conditional model.

Training

• Data: LeMat-Bulk-fixed at revision bf8f0b31, residual force ≤ 1 eV/Å. The condition is each structure's energy_above_hull against the PBE hull of the whole dataset.
• Model: 3-layer transformer encoder with AdaLN conditioning on log1p(e_hull); the condition is dropped 10% of the time for guidance. der_tokenizer_v1 tokens.
• Optimisation: AdamW with a warmup-stable-decay schedule. W&B run ehull_adamw_wsd_5x_cfg_long-20261015-021519, best validation loss 17.23.

Evaluation

Measured with the de novo ranking protocol: genes relaxed with CrySPR and ORB v3 (orb_conserv_inf), novelty judged against LeMat-Bulk-fixed, energy_above_hull=0.05, w = 5. MetaSUN counts structures with e_hull ≤ 0.1 eV/atom that are valid, unique and novel.

per sampled gene, 1,000 genes	value
MetaSUN	0.525
metastable	0.723
SUN (e_hull ≤ 0)	0.005
gene novelty	0.699
valid and unique structures	0.939

Filtering before relaxation raises the yield. On a fully relaxed pool of 10,000 genes, MetaSUN per relaxed gene at a budget of 1,000 relaxations was:

selection	MetaSUN
the first 1,000 draws	0.513
the first 1,000 genes not in LeMat-Bulk-fixed	0.615
those, ranked by WyFormer-LeMat-Energy	0.669
those, the 30% the generator finds least surprising, then ranked	0.870