WyFormer-LeMat-Energy

A WyFormer regressor that predicts formation energy from the Wyckoff representation alone, before any atomic coordinates exist. Specifically, it predicts the lowest PBE formation energy (eV/atom) observed for any structure with the gene. Use it to rank generated genes by their predicted distance to the convex hull before spending MLIP or DFT time on them.

pip install wyckoff-transformer
wyformer-gene-screen genes.json.gz --regressor-hf-model SymmetryAdvantage/WyFormer-LeMat-Energy \
    --hf-reference kazeevn/LeMat-Bulk-fixed \
    --max-atoms 40 --unique --top 100 --out screen.csv --out-genes top100.json.gz

--max-atoms 40 drops genes with more than 40 atoms in the conventional cell before the top 100 are taken. The lowest predicted energies favour large cells, which are slow to relax.

--hf-reference builds the PBE convex hull from LeMat-Bulk-fixed. It reads about 170 MB of the dataset's columns and caches the result in ~/.cache/wyckoff_transformer/. The full pipeline is in the repository README.

Energy scale

• Target: gene_min_formation_energy_per_atom, i.e. raw PBE/PBE+U energies with no MP2020 correction. Elemental references are the lowest energy per atom of each element in LeMat-Bulk-fixed compatible_pbe at revision bf8f0b31.
• The model and the hull share one energy scale, lemat_bulk_fixed_pbe@bf8f0b31, so wyformer-gene-screen compares them without --allow-incompatible-energy. The previous version of this repository was trained on upstream LeMat-Bulk and needed that flag.

Training

• Data: LeMat-Bulk-fixed compatible_pbe at revision bf8f0b31, residual force ≤ 1 eV/Å.
• Model: 4-layer transformer encoder (8 heads) with weighted-mean pooling, trained with MSE loss, der_tokenizer_v1 tokens.
• Optimisation: AdamW with a warmup-stable-decay schedule. W&B run min_energy_adamw_wsd_h7x_xl-20261015-185731, best validation MAE 0.0246 eV/atom.

Evaluation

Used as the energy ranker in the rules-of-engagement comparison: 10,000-gene pools from WyFormer-LeMat-Ehull and WyFormer-LeMat, every gene relaxed with ORB v3 (orb_conserv_inf), novelty judged against LeMat-Bulk-fixed. Ranking the novel genes by predicted hull distance and relaxing the 1,000 best gives the MetaSUN rate (e_hull ≤ 0.1 eV/atom, stable, unique, novel) per relaxed gene below.

pool	first 1,000 novel genes	1,000 best predicted
WyFormer-LeMat-Ehull	0.615	0.669
WyFormer-LeMat	0.549	0.629

Rank only genes that are already known to be novel. Ranked on its own, the regressor's best genes are mostly ones LeMat-Bulk already holds (MetaSUN 0.348 and 0.257 on the same pools).