OneScience-Sugon/Medgemma

MedGemma

Model Overview

MedGemma is an open-source medical multimodal large language model from Google. It is based on the Gemma 3 architecture and is trained for medical text and medical image understanding. MedGemma provides two variants:

• MedGemma 4B: a multimodal model that supports joint input of medical text and medical images.
• MedGemma 27B: a text-only model focused on medical text understanding and question answering.

Model Description

MedGemma 4B uses an image encoder and has been pretrained on multiple types of de-identified medical data, including chest X-rays (CXR), dermatology images, ophthalmology images, and histopathology slides. Its language model component has been trained on radiology images, pathology images, ophthalmology images, dermatology images, and medical text.

Weights and datasets are not available at the moment. They will be uploaded to Hugging Face soon, and command-line downloads will be supported later.

Use Cases

Use case	Description
Medical question answering	Evaluates the model's question-answering capability on medical knowledge benchmarks such as MedQA.
Medical image analysis	Supports tasks such as anatomical localization on chest X-rays (CXR) and longitudinal comparison of multi-timepoint images.
Domain fine-tuning	Performs parameter-efficient fine-tuning with LoRA on datasets such as NCT colon histopathology images.
Unified inference	Provides interactive and batch-file inference through MedicalInferenceRunner.

Use and Limitations

MedGemma is a foundation model for developing healthcare AI applications and is intended to serve as a starting point for downstream development, adaptation, and validation. Developers should fully validate, adapt, and meaningfully modify the model for their specific use case.

Outputs generated by MedGemma are not intended to directly inform clinical diagnosis, patient management decisions, treatment recommendations, or any other direct clinical practice. All model outputs should be considered preliminary and require independent verification, clinical correlation, and further investigation.

Usage

1. Using OneCode

You can try intelligent one-click AI4S programming through the OneCode online environment:

Try intelligent one-click AI4S programming

2. Manual Installation and Usage

Hardware Requirements

• Running on a GPU or DCU is recommended.
• CPU can be used for connectivity checks, but it is relatively slow.
• DCU users need to install DTK in advance. DTK 25.04.2 or later is recommended, or the OneScience-recommended version that matches the current cluster.

Software Requirements

DCU users who want to learn more about adaptation details can contact liubiao@sugon.com.

Environment Checks

• NVIDIA GPU:

nvidia-smi

• Hygon DCU:

hy-smi

Environment Preparation

1. Check the botocore version. If the version is too old, upgrade it:

   pip install --upgrade boto3==1.43.36 botocore==1.43.36
   

2. Check the transformers version. If the version is too old, upgrade it:

   pip install --upgrade transformers==5.12.1
   

Quick Start

1. Install the Runtime Environment

conda create -n onescience311 python=3.11 -y
conda activate onescience311
pip install onescience[bio] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai

If the following code cannot find required libraries at runtime, activate CUDA as shown below.

source ${ROCM_PATH}/cuda/env.sh
export LD_LIBRARY_PATH="$CONDA_PREFIX/lib:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="$CONDA_PREFIX/lib/python3.11/site-packages/fastpt/torch/lib:$LD_LIBRARY_PATH"

2. Download the Model Package

# By default, the package is downloaded to the model folder under the current path. To change this, adjust the path after local_dir.
hf download --model OneScience-Sugon/Medgemma --local-dir ./model
cd model

Training Weights and Datasets

Weights and datasets are not provided at the moment. They will be uploaded to Hugging Face soon, and command-line downloads will be supported later.

3. Usage Modes

Data and Model Weights

By default, the scripts load the model from:

${ONESCIENCE_DATASETS_DIR}/medgemma/modelscope/google/medgemma-1.5-4b-it

By default, the scripts load datasets from the following paths:

Task	Data	Default path
MedQA evaluation	MedQA parquet data	${ONESCIENCE_DATASETS_DIR}/medgemma/medqa
Chest X-ray anatomical localization	Chest X-ray images	${ONESCIENCE_DATASETS_DIR}/medgemma/Chest_Xray/...
Chest X-ray longitudinal comparison	Two chest X-rays from before and after	${ONESCIENCE_DATASETS_DIR}/medgemma/test_compare/...
Pathology image fine-tuning	NCT-CRC-HE-100K / CRC-VAL-HE-7K	${ONESCIENCE_DATASETS_DIR}/medgemma/nct/...

Download the model in advance and place it in this directory, or override it through the model_path environment variable.

Detailed Usage

1. Integration Test

Verify whether MedGemma modules, configurations, data adapters, and image processing components can be imported correctly in OneScience:

python tests/test_integration.py

---

2. Medical Question Answering Evaluation (run_evaluate_on_medqa.sh)

Evaluate the model's medical question-answering capability on the MedQA dataset. By default, 10 samples are processed for quick validation.

bash scripts/run_evaluate_on_medqa.sh

Output:

• scripts/medqa_results/medqa_results.json: detailed results for each sample
• scripts/medqa_results/summary.txt: summary metrics such as accuracy

---

3. Chest X-Ray Anatomical Localization (run_cxr_anatomy.sh)

Perform anatomical localization on one or more chest X-rays. The script runs both single-image mode and batch mode internally:

bash scripts/run_cxr_anatomy.sh

Output:

• scripts/outputs/result_*.json: localization coordinates and labels
• scripts/outputs/result_*.png: visualization images with bounding-box annotations
• scripts/outputs/batch_summary.json: summary results for batch mode

---

4. Chest X-Ray Longitudinal Comparison (run_cxr_longitudinal_comparison.sh)

Compare two chest X-rays from the same patient taken at different timepoints:

bash scripts/run_cxr_longitudinal_comparison.sh

Output:

• scripts/compare_outputs/compare_<image1>_vs_<image2>.txt: text comparison report
• scripts/compare_outputs/compare_<image1>_vs_<image2>.json: structured JSON results

---

5. Pathology Image LoRA Fine-Tuning (run_fine_tune.sh)

Perform LoRA fine-tuning on the NCT colon histopathology image dataset:

bash scripts/run_fine_tune.sh

Output:

• scripts/medgemma-nct-lora/: LoRA weights, training logs, and evaluation results

Note: The script automatically checks and fixes the boto3==1.43.36 and botocore==1.43.36 versions to avoid dependency conflicts.

---

6. Using the Inference Runner

runner/medical_inference_runner.py provides a unified inference entry point and supports both interactive and batch-file inference.

Interactive Inference

export PYTHONPATH=../../../src:$PYTHONPATH
python runner/medical_inference_runner.py \
    --config configs/inference_config.yaml \
    --interactive

Batch-File Inference

export PYTHONPATH=../../../src:$PYTHONPATH
python runner/medical_inference_runner.py \
    --config configs/inference_config.yaml \
    --input data/example_input.json

Notes

• Make sure the ONESCIENCE_DATASETS_DIR environment variable is correctly set before running the scripts.
• The scripts use HIP_VISIBLE_DEVICES=0 by default and can run directly on Hygon DCU platforms. On CUDA platforms, replace it with CUDA_VISIBLE_DEVICES=0 or adjust it according to the available devices.
• To use vLLM for accelerated inference, make sure the corresponding version of vLLM is installed and configure use_vllm: true.
• The chest X-ray anatomical localization and pathology fine-tuning scripts automatically fix boto3 / botocore versions to avoid dependency conflicts.
• The 4B multimodal model requires substantial GPU memory for inference. At least 24 GB of memory on a single device is recommended. Multi-device execution can be controlled through num_gpus or an external CUDA_VISIBLE_DEVICES setting.

Official OneScience Information

Platform	OneScience main repository	Skills repository
Gitee	https://gitee.com/onescience-ai/onescience	https://gitee.com/onescience-ai/oneskills
GitHub	https://github.com/onescience-ai/OneScience	https://github.com/onescience-ai/oneskills

Citation and License

The MedGemma model is licensed under the Health AI Developer Foundations License, and the example code in this repository is licensed under Apache 2.0.

For more information, see:

• Developer documentation
• Model card
• Community guidelines
• Hugging Face
• Google Model Garden