RezaAskari/GPT2-optimal_insulin_calculator

GPT-2 Insulin Titration Fine-Tuning

A specialized fine-tuning project for GPT-2 to provide insulin dosage adjustment recommendations based on 7-day fasting blood glucose (FBG) measurements.

🎯 Project Overview

This project fine-tunes GPT-2 language models to analyze patient blood glucose patterns and recommend insulin dosage adjustments. The system implements clinical decision rules for insulin titration based on average FBG levels and variability patterns.

Decision Logic

• High FBG (>180 mg/dL): Increase insulin by 20 units
• Low FBG (<100 mg/dL): Decrease insulin by 20 units
• Optimal FBG (100-180 mg/dL): No change in dosage
• High variability (std >40): Consider smaller adjustments

📁 Project Structure

fine_tuning/
├── fine_tune.py              # Standard GPT-2 fine-tuning
├── fine_tune_lora.py         # LoRA (Low-Rank Adaptation) fine-tuning
├── fine_tune_qlora.py        # QLoRA (Quantized LoRA) fine-tuning
├── push_to_hub.py            # Script to upload models to HuggingFace Hub
├── gpt2_insulin_titration/   # Standard fine-tuned model
├── gpt2_insulin_lora/        # LoRA fine-tuned model
└── gpt2_insulin_qlora/       # QLoRA fine-tuned model

🚀 Quick Start

Prerequisites

pip install torch transformers datasets peft accelerate bitsandbytes huggingface-hub

Basic Fine-Tuning

Run standard GPT-2 fine-tuning:

python fine_tune.py

Advanced Fine-Tuning Options

LoRA Fine-Tuning (Memory Efficient)

python fine_tune_lora.py

QLoRA Fine-Tuning (Most Memory Efficient)

python fine_tune_qlora.py

💻 System Requirements

Hardware

• Minimum: 8GB RAM (CPU training)
• Recommended: NVIDIA GPU with 8GB+ VRAM
• Supported: Apple Silicon (M1/M2/M3) with MPS acceleration

Software

• Python 3.8+
• PyTorch 2.0+
• Transformers 4.30+
• CUDA 11.8+ (for NVIDIA GPUs)

🔧 Configuration

Training Parameters

Parameter	Standard	LoRA	QLoRA
Epochs	5	3	3
Batch Size	4	8	8
Learning Rate	5e-5	3e-4	2e-4
Warmup Steps	50	50	50
Max Length	128	128	128

LoRA Configuration

• Rank (r): 16
• Alpha: 32
• Dropout: 0.1
• Target Modules: c_attn, c_proj

QLoRA Configuration

• 4-bit quantization
• Double quantization enabled
• NF4 quantization type
• FP16 compute dtype

📊 Dataset

The project generates synthetic training data with 100 samples covering four scenarios:

1. High Glucose: FBG consistently >180 mg/dL
2. Low Glucose: FBG consistently <100 mg/dL
3. Optimal Range: FBG between 100-180 mg/dL
4. Variable: Mixed high and low readings

Each training example follows the format:

Patient's fasting blood glucose over 7 days (mg/dL): [values]. 
Insulin adjustment recommendation: [INCREASE/DECREASE/NO CHANGE]

🧪 Testing

The fine-tuned models are tested with three scenarios:

• High FBG pattern (requires increase)
• Low FBG pattern (requires decrease)
• Optimal FBG pattern (no change needed)

📤 Deployment

Local Usage

from transformers import GPT2LMHeadModel, GPT2Tokenizer

# Load model
model = GPT2LMHeadModel.from_pretrained("./gpt2_insulin_titration")
tokenizer = GPT2Tokenizer.from_pretrained("./gpt2_insulin_titration")

# Make prediction
fbg_values = "210, 195, 220, 205, 190, 215, 200"
prompt = f"Patient's fasting blood glucose over 7 days (mg/dL): {fbg_values}. Insulin adjustment recommendation:"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=20)
recommendation = tokenizer.decode(outputs[0], skip_special_tokens=True)

HuggingFace Hub

Push your model to HuggingFace Hub:

# Login to HuggingFace
huggingface-cli login

# Push model
python push_to_hub.py

Or use the model directly from the Hub:

model = GPT2LMHeadModel.from_pretrained("MohammadReza99/GPT2-optimal_insulin_calculator")

📈 Performance Comparison

Method	Training Time	Memory Usage	Model Size	Accuracy
Standard	~10 min	4-6 GB	548 MB	Baseline
LoRA	~5 min	2-3 GB	3.2 MB (adapter)	~95% of baseline
QLoRA	~7 min	1-2 GB	3.2 MB (adapter)	~93% of baseline

⚠️ Important Notes

Medical Disclaimer

This is a demonstration project for educational purposes. The model should NOT be used for actual medical decisions without proper validation and regulatory approval.

Limitations

• Synthetic training data may not capture all real-world scenarios
• Model outputs should be validated by healthcare professionals
• Does not account for individual patient factors beyond FBG

🔬 Future Improvements

• Incorporate additional patient factors (weight, age, insulin type)
• Add hypoglycemia risk assessment
• Implement continuous glucose monitoring (CGM) data support
• Expand training data with real clinical cases
• Add uncertainty quantification for recommendations
• Implement multi-language support

📚 References

• GPT-2 Paper
• LoRA: Low-Rank Adaptation
• QLoRA: Efficient Finetuning
• HuggingFace Transformers

📝 License

This project is for educational and research purposes. Please consult appropriate medical guidelines and regulations before any clinical application.

🤝 Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues for improvements.

📧 Contact

For questions or collaboration opportunities, please reach out through the HuggingFace Hub repository: MohammadReza99/GPT2-optimal_insulin_calculator