Llama-3.2-3B-autism-qa-lora

Built with Llama.

A LoRA adapter for Llama 3.2 3B Instruct that answers questions about autism from a retrieved passage, and politely refuses requests for medical advice. It is the answer-writing step of asterly, a learning project that covers fine-tuning, retrieval, safety guardrails and deployment end to end.

Not medical advice. This is a learning project, not a medical service. Always ask a qualified professional about diagnosis, medication or treatment.

How asterly uses it

question → safety guard → FAISS search + reranker → this model writes the answer
         (crisis helplines,  (791 passages)          from the top passages
          medical refusals)                        → output check → answer

The model is trained to stay inside the passage it is given. On its own, without retrieval and the safety layers around it, it is not a reliable source.

Training

Base	Llama 3.2 3B Instruct, loaded in 4-bit with Unsloth (QLoRA)
Data	autism-qa-dataset-v1, train split: 3,960 records (Q&A, refusals, multi-turn), with a slice held out for evaluation
LoRA	r=16, alpha=32, dropout 0.05, on all attention and MLP projections
Schedule	3 epochs, 669 steps, effective batch 16, learning rate 2e-4, cosine
Hardware	One Colab T4 GPU
Eval loss	0.539 → 0.350 → 0.325 (after epochs 1, 2, 3)

Framework versions: PEFT 0.19.1, TRL 0.24.0, Transformers 5.5.0, PyTorch 2.11.

Use it

The base model is gated: accept Meta's terms on its page first. The prompt must match the training format exactly; asterly builds it in src/asterly/prompting.py (the system prompt is in src/asterly/config.py).

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-3.2-3B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.float16, device_map="cuda")
model = PeftModel.from_pretrained(model, "sivafullstackdeveloper/Llama-3.2-3B-autism-qa-lora")

from asterly.prompting import build_inference_prompt  # pip install -e . in the asterly repo

passages = [{"text": "Autism spectrum disorder is a developmental condition that affects ..."}]
prompt = build_inference_prompt("What is autism?", passages)
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Limits

• English only, and only as good as the passages it is given.
• The safety behaviour lives mostly in asterly's guard layers, not in this model.
• Not reviewed by a clinician.

License

Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. The training data is MIT-licensed (see the dataset card).