knc6/ramangpt_dft_model

RamanGPT (Mistral-7B): Raman spectrum to crystal structure

LoRA adapter fine-tuned from unsloth/mistral-7b-bnb-4bit on knc6/ramangpt_dft. This is the inverse model of the RamanGPT framework: given a chemical formula and the main Raman peaks, it generates a crystal structure (lattice + fractional coordinates). The forward model (ALIGNN) predicts the 200-bin Raman spectrum (50--1000 cm$^{-1}$) from a structure, and an inverse->relax->forward loop couples the two. Part of the AtomGPT spectroscopy suite (cf. IRGPT, DiffractGPT).

Training

QLoRA (initialized from the DiffractGPT explicit-formula checkpoint), LoRA r=16, alpha=16 on q,k,v,o,gate,up,down; lr 2e-4 linear, effective batch 8, 2 epochs, 4-bit base, seed 3407.

Evaluation

• Inverse (508 held-out materials): reduced-formula consistency 90.7%, lattice-parameter MAE 1.16--2.36 Angstrom, versus 17.2% formula consistency for a formula-conditioned retrieval baseline given identical inputs.
• Forward (ALIGNN, 5,099-material Computational Raman Database): 43.4% of held-out materials reach an active-region cosine similarity >= 0.354, capturing the qualitative mode structure; it also agrees qualitatively with an experimental 1T-VSe2 spectrum absent from training.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = "unsloth/mistral-7b-bnb-4bit"
tok = AutoTokenizer.from_pretrained(base)
model = PeftModel.from_pretrained(
    AutoModelForCausalLM.from_pretrained(base, device_map="auto"),
    "knc6/ramangpt_dft_model")
prompt = ("### Instruction:\nBelow is a description of a material.\n### Input:\n"
          "The chemical formula is: MoS2.\nThe Raman spectrum shows main peaks at: "
          "383 cm-1(1.0), 408 cm-1(0.9).\nGenerate atomic structure description with "
          "lattice lengths, angles, coordinates and atom types.\n### Output:\n")
ids = tok(prompt, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**ids, max_new_tokens=512)[0][ids.input_ids.shape[1]:],
                 skip_special_tokens=True))

Links

• Dataset: https://huggingface.co/datasets/knc6/ramangpt_dft
• Code (forward + inverse, any base LLM): https://github.com/atomgptlab/ramangpt
• Web app: https://atomgpt.org/raman ; Paper: arXiv:2606.03764