MedLlama-India-70B

Fine-tuned LoRA adapter on meta-llama/Llama-3.3-70B-Instruct for Indian medical entrance examination QA (AIIMS-PG and NEET-PG).

Trained via Adaption Labs AutoScientist — Healthcare category, AutoScientist Hackathon 2025.

---

Benchmark Results

AutoScientist Internal Evaluation

Metric	Base Model	MedLlama-India-70B	Improvement
Win Rate (on dataset)	20	80	+300% relative
Medical Domain Win Rate	26	74	+185% relative

External Baseline Evaluation

Model	Accuracy	Notes
Mistral 7B Instruct v0.2 (zero-shot)	44.68%	Evaluated on MedMCQA validation set (4,183 examples)
MedLlama-India-70B	~74% (AutoScientist medical win rate)	Llama 3.3 70B fine-tuned

Benchmark: openlifescienceai/medmcqa validation set — 4,183 questions across 21 medical subjects.

---

Training Details

Parameter	Value
Base model	meta-llama/Llama-3.3-70B-Instruct
Method	LoRA (PEFT)
LoRA rank	64
LoRA alpha	128
Target modules	all-linear
Epochs	3
LR scheduler	cosine
Warmup ratio	0.05
Gradient clipping	1
Training platform	Adaption Labs AutoScientist
Data platform	Adaption Labs Adaptive Data
Training examples	38,000+
Source dataset	openlifescienceai/medmcqa

AutoScientist Training Metrics

• Win rate: 20 → 80 (+300%)
• Medical win rate: 26 → 74 (+185%)
• Final training loss: 0.73
• Training steps: 624
• Loss curve: clean convergence, no overfitting

Adaptive Data Processing

• Source: MedMCQA (170,000 questions)
• Quality grade: B → A (28.9 → 57.7 percentile)
• Recipes: Reasoning Traces, Hallucination Mitigation, Prompt Rephrase, Metadata Injection, House Special
• Localization: India (English)
• Blueprint: AIIMS/NEET-PG medical examiner persona

---

Dataset

xenkrypt/MedLlama-India-Dataset

38,000+ instruction-formatted NEET-PG and AIIMS-PG questions processed via Adaption Labs Adaptive Data with reasoning traces, hallucination mitigation, and Indian medical context localization.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.3-70B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("xenkrypt/MedLlama-India-70B")
model = PeftModel.from_pretrained(base, "xenkrypt/MedLlama-India-70B")

prompt = """### Instruction:
You are a medical expert for AIIMS/NEET-PG examinations.
Answer this multiple choice question.

Question: Most common cause of mitral stenosis?
A) Rheumatic fever
B) Infective endocarditis
C) Congenital
D) SLE

### Response:
The correct answer is"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
out = model.generate(**inputs, max_new_tokens=150, do_sample=False)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Limitations

• LoRA adapter — requires base model meta-llama/Llama-3.3-70B-Instruct
• Trained on exam MCQs — validate before clinical use
• Not a substitute for professional medical advice

Demo

https://huggingface.co/spaces/xenkrypt/MedLlama-India-Demo

Submitted To

Adaption Labs AutoScientist Hackathon — Healthcare, Part 1, July 2025