3D_LLM_Diffusion — Model Weights
Weights only. Code and usage documentation: https://github.com/Richardyangfan78/3D_LLM_Diffusion
hf download Yangfan78/3D_LLM_Diffusion --local-dir weights
Three independent weight sets.
generator_v41/ — Conditional crystal generator
| File | Size | Description |
|---|---|---|
best.pt |
378.9 MB | All generator weights, backbone included |
priors/train.csv |
75.2 MB | Read at sampling time: atom-count prior + allowed element set |
priors/val.csv |
25.0 MB | Validation split |
eval/ |
— | Evaluation records |
| Property | Value |
|---|---|
| Architecture ID | mattergen-spacegroup-hhi-qwen-crossattention-crystalite-v8 |
| Parameters | 94.63 M (777 tensors) |
| Dimensions | d_model 512 / 14 layers / 16 heads |
| Training step | 6000 |
| Atom type encoding | subatomic_tokenizer_pca_16 (16-d continuous) |
| Lattice representation | ltri |
| SHA-256 | cd67729bd862537be9113146ebdf48db81f86a53a4a32cd99c751bba0e06fa4a |
Eight condition channels: chemical_system, space_group, dft_band_gap, dft_bulk_modulus, dft_mag_density, energy_above_hull, hhi_score, ml_bulk_modulus.
The inference policy and sampler settings are stored inside the checkpoint and apply automatically:
guidance global 1.0 | band_gap 1.5 | e_above_hull 2.0 | element+hull joint 2.25
sampler 150 steps, sigma 0.002 to 80, rho 7, S_churn 60
text_conditioning/ — Text condition vectors and encoders
| File | Size | Description |
|---|---|---|
val_text_xrd_z.pt |
22.1 MB | 18,094 pre-encoded vectors, 256-d each |
trimodal_best.pt |
51.4 MB | Crystal-text-XRD tri-modal encoder |
qwen_latent_adapter_best.pt |
9.5 MB | Qwen hidden states to 256-d vector adapter |
val_text_xrd_z.pt.meta.json |
— | Provenance metadata |
val_text_xrd_z.pt is a dict keyed as:
mp-865981__text text modality vector
mp-865981__xrd XRD modality vector
Conditioning on these pre-encoded vectors requires no language model. Encoding new text requires Qwen3.6-35B-A3B (revision 995ad96eacd98c81ed38be0c5b274b04031597b0) together with the other two files.
Tri-modal encoder SHA-256: fa5dbe01af4d733022ab2d2a1db0e425414e2148154254931a1acb290029ea94
3D_Property/ — Property prediction (LoRA)
Adapter weights on top of Qwen3.6-35B-A3B. The base model is neither included here nor modified.
| File | Size | Description |
|---|---|---|
step_6500/lora_adapter/ |
613.5 MB | LoRA weights, PEFT standard format |
step_6500/projector.pt |
18.9 MB | Per-atom features to Qwen embedding space |
step_6500/trainer_state.pt |
1265.1 MB | Optimizer moments, LR schedule, RNG, sampler state |
step_6500/checkpoint_metadata.json |
— | Training metadata |
stage1_projector.pt |
18.9 MB | Stage-1 projector |
| Property | Value |
|---|---|
| Base model | Qwen3.6-35B-A3B @ 995ad96eacd98c81ed38be0c5b274b04031597b0 |
| LoRA | r=128, alpha=256, dropout=0.05 |
| Attached to | 250 linear layers (q/k/v/o_proj, gate/up/down_proj, in_proj_qkv, in_proj_z, out_proj) |
| Trainable parameters | 153,354,240 (~0.44% of base) |
| Progress | step 6500 / 12000 |
| Reference environment | H200 single GPU, ~105.7 GB VRAM, effective batch 256 |
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3.6-35B-A3B",
revision="995ad96eacd98c81ed38be0c5b274b04031597b0",
)
model = PeftModel.from_pretrained(base, "weights/3D_Property/step_6500/lora_adapter")
projector.pt is loaded separately to produce the per-atom soft tokens. trainer_state.pt is only needed to resume training and can be skipped for inference.
License
The denoising trunk inside generator_v41/best.pt derives from the Crystalite research backbone, MIT License, Copyright (c) 2026 Joshua Rosenthal. The 3D_Property/ LoRA weights derive from Qwen3.6-35B-A3B (Apache-2.0). Full third-party attribution is in the NOTICE file of the code repository.