SmolLM3-3B — computational-chemistry SFT (LoRA adapter)
A rsLoRA adapter from the CPT→SFT case study in https://github.com/anisiraj/llm_kickstart_repo (handbook.html → 🧪 Case Study).
- Phase: sft (completion-only loss on Q&A; prompt tokens masked to -100)
- Base model:
HuggingFaceTB/SmolLM3-3B - Data: anisiraj/comp-chem-quantum-chem
- Measured: completion ppl 39.23→6.64 (-83%), recall 18→22%; unmasked 81%
- Reproduce:
bash case_study/run.sh smollm3
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
base = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM3-3B", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "anisiraj/SmolLM3-3B-compchem-sft-lora")
tok = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM3-3B")
- Downloads last month
- 21
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support