WyFormer-LeMat

An unconditional WyFormer generator of Wyckoff representations, trained on the near-hull part of LeMat-Bulk-fixed. It samples the space group, the composition and the Wyckoff positions.

pip install wyckoff-transformer
wyformer-generate genes.json.gz --hf-model SymmetryAdvantage/WyFormer-LeMat

The repository README shows how to relax the genes into 3D structures with an MLIP. WyFormer-LeMat-Ehull gives a higher yield of new metastable structures when sampled at energy_above_hull=0.05.

Training

• Data: LeMat-Bulk-fixed at revision bf8f0b31, residual force ≤ 1 eV/Å and e_hull ≤ 0.1 eV/atom against the PBE hull of the whole dataset.
• Model: 4-layer transformer encoder, der_tokenizer_v1 tokens.
• Optimisation: AdamW with a warmup-stable-decay schedule, peak learning rate 7.5e-4. W&B run unconditional_5x_ehull01_h3x_xl_lr7p5e-4-20261018-202012, best validation loss 18.22.

Evaluation

A fully relaxed pool of 10,000 genes: CrySPR with ORB v3 (orb_conserv_inf), novelty judged against LeMat-Bulk-fixed. MetaSUN counts structures with e_hull ≤ 0.1 eV/atom that are valid, unique and novel. Rates are per relaxed gene at a budget of 1,000 relaxations.

selection	MetaSUN	SUN
the first 1,000 draws	0.382	0.009
the first 1,000 genes not in LeMat-Bulk-fixed	0.549	0.018
those, ranked by WyFormer-LeMat-Energy	0.629	0.059
those, the 30% the generator finds least surprising, then ranked	0.758	0.047

46% of the pool's genes are not in LeMat-Bulk-fixed, so screening the samples for novelty before relaxing them matters more here than for the conditioned model.