NeuroQLoRA: Neurology Clinical Reasoning Assistant
A QLoRA-based research prototype for neurology-focused differential diagnosis and clinical reasoning.
Overview
NeuroQLoRA explores the adaptation of compact open-source language models for neurology-specific clinical reasoning tasks using parameter-efficient fine-tuning.
The objective of this work is not to create an autonomous diagnostic system, but to investigate whether small language models can be specialized to generate structured neurological assessments using limited computational resources.
The resulting adapter generates outputs consisting of:
- Neurological assessment
- Differential diagnoses
- Investigation recommendations
- Clinical reasoning
This repository presents a research-oriented proof-of-concept for Healthcare AI and Clinical NLP experimentation.
Why This Project?
Clinical decision support remains one of the most promising applications of domain-adapted language models. However, training and deploying large models often requires significant computational resources.
This project investigates:
- The effectiveness of QLoRA for medical domain adaptation.
- The impact of model scale on neurology-focused reasoning.
- Whether specialized clinical reasoning behavior can emerge from synthetic supervision.
- The feasibility of healthcare LLM experimentation on consumer-grade hardware.
Model Details
Developed By
Habiba Sajid
Model Type
PEFT LoRA Adapter
Language
English
License
Apache-2.0
Finetuned From
Qwen/Qwen2.5-3B-Instruct
Intended Uses
Direct Use
This model is intended for:
- Healthcare AI research
- Clinical NLP experimentation
- Educational demonstrations of QLoRA
- Exploration of neurology-focused reasoning tasks
Potential Downstream Uses
Potential future applications include:
- Clinical decision-support research
- Medical reasoning benchmarks
- Retrieval-Augmented Generation (RAG) systems
- Educational healthcare AI tools
Out-of-Scope Use
This model is not intended for:
- Clinical deployment
- Autonomous diagnosis
- Treatment recommendation
- Patient-facing medical advice
- Emergency medical decision-making
- Replacing physician judgment
Research Objective
Can compact language models efficiently specialize for neurology-focused reasoning tasks using parameter-efficient fine-tuning under constrained compute settings?
This project evaluates:
- QLoRA as a healthcare adaptation strategy
- The effect of model scale on reasoning performance
- Practical training on limited GPU resources
Base Models
| Model | Parameters |
|---|---|
| Qwen2.5-0.5B-Instruct | 0.5 Billion |
| Qwen2.5-3B-Instruct | 3 Billion |
Training Data
Synthetic neurological clinical cases were generated for research purposes.
Cases were designed to teach structured outputs involving:
- Neurological assessment
- Differential diagnosis generation
- Investigation planning
- Clinical reasoning
No real patient records or protected health information were used.
Fine-Tuning Setup
| Component | Configuration |
|---|---|
| Fine-Tuning Method | QLoRA |
| Quantization | 4-bit |
| Frameworks | Transformers + PEFT + TRL |
| Epochs | 3 |
| Hardware | Kaggle T4 GPU |
| Task | Neurological Differential Diagnosis |
| Dataset Type | Synthetic Clinical Cases |
Evaluation
Metrics
Evaluation focused primarily on:
- Validation Loss
- Mean Token Accuracy
- Comparative model-scale performance
Validation Mean Token Accuracy is reported as a measure of structured generation consistency and should not be interpreted as diagnostic accuracy.
Comparative Results
Validation Performance
| Model | Validation Mean Token Accuracy |
|---|---|
| Qwen2.5-0.5B | 0.67 |
| Qwen2.5-3B | 0.86 |
Qwen2.5-3B Training Dynamics
| Step | Validation Loss | Mean Token Accuracy |
|---|---|---|
| 50 | 1.107 | 0.728 |
| 100 | 0.650 | 0.839 |
| 150 | 0.562 | 0.862 |
| 171 (Final) | 0.561 | 0.863 |
Key Observation
The 3B parameter model demonstrated substantially stronger structured clinical reasoning consistency compared with the 0.5B model.
These findings suggest that increased model capacity improves specialization for neurology-focused reasoning tasks while remaining feasible to train using consumer-grade hardware.
Qualitative Evaluation
The following example was generated using the fine-tuned Qwen2.5-3B model on an unseen neurological emergency case.
Input
Patient: 45-year-old Male.
Duration: acute (sudden onset right-sided weakness and aphasia, <2 hours).
Severity: emergency.
Past Medical History:
- Hypertension
- Type 2 Diabetes
- Smoker
Generated Output
Neurological Assessment:
- Localisation: Left hemisphere (dominant) dominant hemisphere for motor and language.
Differential Diagnoses:
1. Ischaemic stroke (e.g., cardioembolic from AF, large vessel disease).
2. Haemorrhagic stroke (less likely initially without severe headache/confusion, but exclude with CT).
3. Seizure with post-ictal Todd's paresis and aphasia.
Recommended Investigations:
- Immediate: Non-contrast CT Head
- Urgent: Blood tests (FBC, U&E, Coags, Glucose, HbA1c, Troponins, Lactate)
- ECG (look for AF)
- CT Angiogram Neck/Cerebral vessels
- Perfusion CT
- Soon: Echocardiogram
Observations
The model demonstrates several desirable behaviors:
- Correct localization of symptoms to the dominant left cerebral hemisphere.
- Appropriate prioritization of acute ischemic stroke.
- Generation of clinically relevant differential diagnoses.
- Recommendation of evidence-based emergency investigations.
- Structured reasoning suitable for physician-facing decision support research.
How to Load the Adapter
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-3B-Instruct"
)
model = PeftModel.from_pretrained(
base_model,
"DevHabiba/Neurology-Clinical-Reasoning-QLoRA"
)
tokenizer = AutoTokenizer.from_pretrained(
"Qwen/Qwen2.5-3B-Instruct"
)
Bias, Risks, and Limitations
- Training relied exclusively on synthetic neurological cases.
- Outputs have not undergone formal neurologist validation.
- Hallucinations and factual inaccuracies may occur.
- Evaluation focused primarily on structured generation quality.
- Reported metrics do not represent clinical diagnostic accuracy.
- Generalization to real-world clinical settings remains untested.
Ethical Considerations
This repository presents a research prototype intended to advance understanding of healthcare language model adaptation.
It is not a medical device and should not be used to guide patient care or replace qualified healthcare professionals.
Users are responsible for ensuring appropriate oversight and validation in any downstream experimentation.
Future Work
Potential future directions include:
- Retrieval-Augmented Generation (RAG) using neurology textbooks
- Expert neurologist evaluation of generated outputs
- Benchmark-based evaluation
- Hallucination reduction strategies
- Expansion to additional neurology tasks
- Comparison with alternative PEFT methods
Citation
If you use this work in research or educational settings, please cite:
@misc{sajid2026neuroqlora,
title={NeuroQLoRA: Neurology Clinical Reasoning Assistant},
author={Habiba Sajid},
year={2026},
howpublished={Hugging Face Repository}
}
Contact
Habiba Sajid Email: habibasajid752@gmail.com linkedin : www.linkedin.com/in/habiba-sajid
Healthcare AI • Clinical NLP • Medical LLMs • Parameter-Efficient Fine-Tuning
Hugging Face: https://huggingface.co/DevHabiba