sajjadhadi/Disease-Diagnosis-Mistral-7B-v0.3

Disclaimer

Important Notice: This fine-tuned disease diagnosis model is intended STRICTLY for research and educational purposes. It is NOT CERTIFIED for clinical use and should NOT be used for real medical diagnosis. The model may generate inaccurate or incomplete suggestions based on its training data.

Medical Disclaimer: This AI system is not a substitute for professional medical advice, diagnosis, or treatment. Always seek the guidance of qualified healthcare providers with any questions regarding medical conditions. The developers disclaim all responsibility for any decisions made based on this model's outputs.

Model Description

This model is a LoRA-adapted version of Mistral-7B-v0.3, specifically fine-tuned for disease diagnosis research. Key features:

• Architecture: 7B parameter transformer with LoRA adapters (r=32, alpha=64)
• Training Approach: 4-bit quantized training with NF4 quantization
• Capabilities: Symptom analysis and differential diagnosis suggestion
• Context Window: 8k tokens for comprehensive case analysis

Metrics

Metric	Accuracy
Top-1	87%
Top-3	92%

Check out this Notebook for more information about test results.

Training Configuration

Hyperparameters

Parameter	Value
Learning Rate	1e-4
Batch Size	64
Sequence Length	93 tokens
Optimizer	Paged AdamW 32-bit
Scheduler	Cosine with 3% Warmup
LoRA Rank	32
LoRA Alpha	64
Epochs	1

Technical Stack

• PEFT: 0.14.0
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Datasets: 3.3.2
• Bitsandbytes: 0.43.0
• TRL: 0.8.6

Intended Use

Research Applications:

• Medical education simulations
• Symptom pattern analysis
• Diagnostic decision support systems research

Prohibited Uses:

• Clinical diagnosis
• Patient treatment decisions
• Self-diagnosis tools

Ethical Considerations

• Bias Mitigation: Regularization techniques applied during training
• Privacy: No real patient data used in training
• Transparency: Full architecture details available in configuration