🧬 MedGemma-Chromosome-VLM

A parameter-efficient Vision-Language Model for image-grounded cytogenetic report generation

Chromosome spread image in → structured, uncertainty-aware cytogenetics report out.

---

📋 Overview

MedGemma-Chromosome-VLM is a domain-adapted vision-language model for automated chromosome spread interpretation. Given a chromosome spread image, it generates an image-grounded structured report covering chromosome count, morphology, overlap status, image quality, findings, interpretation, and explicit uncertainty statements.

This repository ships LoRA adapter weights only, fine-tuned on top of Google's MedGemma-4B.

⚠️ Research use only. Not validated for clinical diagnosis, treatment decisions, prenatal diagnosis, or genetic counseling. All outputs require review by a qualified professional.

---

🗂️ Table of Contents

• Model Details
• Intended Uses
• Training Details
• Evaluation
• Hardware & Environmental Impact
• Biases & Limitations
• Usage
• Citation
• Acknowledgements

---

🔍 Model Details

Field	Value
Developed by	Mukilan A.M.
Base model	unsloth/medgemma-4b-it-unsloth-bnb-4bit
Model type	Vision-Language Model (VLM)
Architecture	MedGemma-4B + LoRA
Fine-tuning method	Parameter-Efficient Fine-Tuning (LoRA, rank 32)
Frameworks	Transformers · PEFT · TRL · Unsloth
Language	English
Domain	Medical AI · Cytogenetics · Biomedical Image Analysis

The model is fine-tuned with LoRA using completion-only supervised fine-tuning (SFT) — only assistant response tokens contribute to the loss. The visual encoder is frozen; only language projection layers are updated. The training objective favors conservative, image-grounded reports that avoid hallucinating abnormalities.

---

🎯 Intended Uses

✅ Direct Use Chromosome spread interpretation Automated cytogenetics report generation Educational demonstrations Biomedical AI research Medical VLM benchmarking Input: chromosome spread image Output: structured report — count, morphology, overlap, image quality, findings, impression, uncertainty

❌ Out-of-Scope Clinical diagnosis Patient treatment decisions Disease confirmation Prenatal diagnosis Genetic counseling Generated reports are for research purposes only.

Downstream applications: digital pathology workflows, laboratory decision-support systems, chromosome visualization tools, medical AI assistants, biomedical VLM research.

---

🏋️ Training Details

Dataset

Split	Images
Train	~4,200
Validation	300
Test	300

Each sample pairs a chromosome spread image (resized to 448 × 448) with a structured annotation: cytogenetic description, morphology, chromosome count, findings, interpretation, and uncertainty statement.

Image preprocessing: EXIF correction → RGB conversion → CLAHE enhancement → auto contrast → bicubic resizing → white letterbox padding.

LoRA Configuration

Parameter	Value
Rank (r)	32
Alpha	64
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Vision encoder	Frozen
Language layers	Trainable
RSLoRA	Enabled

Hyperparameters

Parameter	Value		Parameter	Value
Epochs	6		Warmup ratio	0.05
Learning rate	8e-5		Max sequence length	2048
Optimizer	paged_adamw_8bit		Image size	448
Weight decay	0.01		Seed	3407
Batch size	1		Grad. accumulation	16
Effective batch size	16

Quantization

4-bit NF4 · Double Quantization · BF16 compute · Flash Attention 2 (when available)

---

📊 Evaluation

Model selection was performed on validation loss.

Metric	Value
Best checkpoint	checkpoint-250
Validation loss	≈ 0.430

Additional evaluation axes: BLEU, ROUGE-L, token-level F1, and medical consistency evaluation.

---

⚙️ Hardware & Environmental Impact

GPU	NVIDIA L4
Precision	BF16
Quantization	4-bit
Training time	~2.5–3 hours

---

⚠️ Biases & Limitations

• Inherits limitations from base MedGemma and the fine-tuning dataset
• Dataset skews toward normal chromosome count (46), limiting rare-abnormality generalization
• Cannot replace expert cytogenetic interpretation
• Possible degradation on low-quality input images
• Conservative reporting style may omit subtle abnormalities

Ethical considerations: intended solely for research and education. Must not be used as an autonomous diagnostic system — all outputs require review by qualified medical professionals.

---

💻 Usage

pip install unsloth peft transformers

from unsloth import FastVisionModel
from peft import PeftModel

base_model = FastVisionModel.from_pretrained(
    "unsloth/medgemma-4b-it-unsloth-bnb-4bit"
)

model = PeftModel.from_pretrained(
    base_model,
    "Mukilan06/MedGemma-Chromosome-VLM"
)

---

📖 Citation

If you use this model in academic work, please cite:

@misc{mukilan2026medgemmachromosomevlm,
  title={MedGemma-Chromosome-VLM: A Parameter-Efficient Vision Language Model for Image-Grounded Cytogenetic Report Generation},
  author={Mukilan A. M.},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/Mukilan06/MedGemma-Chromosome-VLM}}
}

---

🙏 Acknowledgements

This work builds upon Google MedGemma, Gemma, Unsloth, Hugging Face Transformers, PEFT, and TRL. Special thanks to the open-source AI community for enabling efficient medical vision-language model development.

---

Made with 🧬 by Mukilan A.M.