AdaptLLM Medical — Fine-tuned Gemma 3 4B
A QLoRA fine-tuned version of Gemma 3 4B on the ChatDoctor-HealthCareMagic dataset for medical Q&A.
Model Details
- Base model: google/gemma-3-4b-it
- Fine-tuning method: QLoRA (LoRA rank 16, alpha 32)
- Dataset: ChatDoctor-HealthCareMagic-100k (500 rows)
- Training: 3 epochs, ~21 minutes on RTX 4060 8GB
- Trainable parameters: 11.8M out of 4.3B (0.27%)
Training Results
| Epoch | Loss | Token Accuracy |
|---|---|---|
| 1 | 2.402 | 0.49 |
| 2 | 2.300 | 0.51 |
| 3 | 2.261 | 0.52 |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("google/gemma-3-4b-it")
model = PeftModel.from_pretrained(base, "BhuvanKrishna12/adaptllm-medical")
Example
Input: I have been taking metformin and recently started ciprofloxacin. I am experiencing hypoglycemia episodes.
Output: Ciprofloxacin is a fluoroquinolone antibiotic which can affect blood glucose levels when combined with metformin. You should consult your doctor about switching to a different antibiotic.
Disclaimer
This model is for research and educational purposes only. Not intended for actual medical diagnosis or treatment.
Built with
AdaptLLM — a local LLM fine-tuning pipeline