knc6/irgpt_qwen3_4b

IRGPT (Qwen3-4B): IR spectrum to crystal structure

LoRA adapter fine-tuned from unsloth/Qwen3-4B-Base on knc6/irgpt_dft. Given a chemical formula, space group, and the main infrared peaks (cm^-1), it generates a crystal structure (lattice + fractional coordinates). Part of the AtomGPT spectroscopy suite (cf. RamanGPT, DiffractGPT).

Training

LoRA r=16, alpha=16, dropout 0 on q,k,v,o,gate,up,down; lr 2e-4 linear, effective batch 8, 2 epochs, bf16, seed 3407. ~28 min on one NVIDIA GB10.

Evaluation (481 held-out test materials, greedy decoding)

metric	value
valid-structure parse rate	100%
reduced-formula consistency	95.2%
space-group match	27.2%
lattice MAE a / b / c (A)	0.82 / 0.95 / 1.78
lattice MAE alpha / beta / gamma (deg)	8.4 / 7.8 / 10.4

Multi-sample decoding + spectral-cosine re-ranking + ALIGNN-FF relaxation (the consistency loop) improve lattice/space-group accuracy further.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "unsloth/Qwen3-4B-Base"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(base, dtype=torch.bfloat16, device_map="auto"),
    "knc6/irgpt_qwen3_4b")

prompt = (
    "### Instruction:\nBelow is a description of a material.\n### Input:\n"
    "The chemical formula is: BPO4.\nThe space group number is: 82.\n"
    "The infrared (IR) spectrum shows main peaks at: 523 cm-1(1.0), 611 cm-1(0.34).\n"
    "Generate atomic structure description with lattice lengths, angles, "
    "coordinates and atom types.\n### Output:\n"
)
ids = tok(prompt, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**ids, max_new_tokens=512)[0][ids.input_ids.shape[1]:],
                 skip_special_tokens=True))

Links

• Code / train any base LLM / inference: https://github.com/atomgptlab/irgpt
• Dataset: https://huggingface.co/datasets/knc6/irgpt_dft