LLM_Crystal_CIF — LoRA adapters for CIF generation

LoRA adapters fine-tuning Qwen2.5-7B-Instruct to emit a full CIF crystal structure from a prompt of reduced composition + target space-group number. Part of a controlled composition-sweep study (MP-20 : MPTS-52 training ratio at fixed volume/steps).

Trained with Unsloth, LoRA r=32 / α=64, lr 1e-4, 24,000 unique training crystals, pinned 4,500 steps (no early stopping), 16-bit. Adapter base: unsloth/Qwen2.5-7B-Instruct (loads fine on stock Qwen/Qwen2.5-7B-Instruct).

Adapters (by training composition, MP-20% : MPTS-52%)

Subfolder	MP-20 %	Best-of-10 match (full 8,096 MPTS-52 test)
comp_mp20_00	0 (pure MPTS-52 baseline)	30.1%
comp_mp20_25	25	30.4%
comp_mp20_50	50	29.5%
comp_mp20_75	75	28.0%
comp_mp20_100	100 (pure MP-20)	26.6%

Finding: at matched volume + steps, match declines monotonically as the MP-20 fraction rises — the gain reported for "combined" data was volume, not symmetry composition.

Each subfolder contains: model/ (LoRA adapter + adapter_config.json), tokenizer/, config.json, and training_stats.json.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct", torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/model")
tok = AutoTokenizer.from_pretrained("shehrozashoaib/LLM_Crystal_CIF", subfolder="comp_mp20_50/tokenizer")

messages = [
    {"role": "system", "content": "You are an expert in materials science and crystallography."},
    {"role": "user", "content": "Generate CIF for the given material description\n\n"
                                 "Material composition is FeCuS2. It has a space group number 122."},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device),
                     max_new_tokens=3072, do_sample=True, temperature=0.6, top_p=0.9)
print(tok.decode(out[0], skip_special_tokens=True))

Code, datasets, and full results: see the companion GitHub repo LLM_Crystal_CIF.