Access MedGemma on Hugging Face

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

To access MedGemma on Hugging Face, you're required to review and agree to Health AI Developer Foundations' terms of use. To do this, please ensure you're logged in to Hugging Face and click below. Requests are processed immediately.

MedGemma 1.5 4B IT — LiteRT-LM (.litertlm)

On-device conversion of google/medgemma-1.5-4b-it to the .litertlm format for LiteRT-LM, targeting offline-first health-Q&A use on low-connectivity, low-RAM Android phones.

Converted by TobiasLogic, as requested in litert-community/gemma-4-E2B-it-litert-lm · discussion #39.

⚠️ Medical disclaimer — read before use

MedGemma is a developer foundation model for decision support and education. It is not a medical device, not a diagnostic tool, and not a substitute for professional medical advice, diagnosis, or treatment. Outputs may be incorrect, incomplete, or unsafe if acted on without independent clinical verification. Developers must validate this model for their specific use case — including the effects of the int4 quantization applied here — before any real-world deployment, and must comply with the Health AI Developer Foundations terms of use. MedGemma is prompt-sensitive and not tuned for multi-turn conversational use; phrase each query as a single, self-contained question.

License and gating

This model is a derivative of MedGemma and remains governed by the Health AI Developer Foundations (HAI-DEF) terms of use — it is not Apache-2.0. This repository preserves the same gated access as the source model: you must review and accept the HAI-DEF terms before downloading. Source model: google/medgemma-1.5-4b-it.

Files

File	Variant	Size (bytes)	Size
`medgemma-1.5-4b-it_q4_block32_ekv2048.litertlm`	Text-only, int4 (block-32) weights	2,583,871,056	2.41 GiB
`medgemma-1.5-4b-it_q4_block32_vision_ekv2048.litertlm`	Text + SigLIP vision encoder (int8 weight-only)	3,023,069,488	2.82 GiB

Conversion details

Source model: google/medgemma-1.5-4b-it (Gemma 3–based, Gemma3ForConditionalGeneration; text decoder: 34 layers, hidden 2560, vocab 262,208, 128K context, sliding window 1024; vision: SigLIP, 27 layers, 896×896)
Tools: litert-torch-nightly v0.10.0.dev20260711, litert-lm v0.14.0 (installed via uv tool install), Python 3.11 (tool venv)
Text-only export command:

litert-torch export_hf \
  --model=google/medgemma-1.5-4b-it \
  --output_dir=/tmp/medgemma15_4b_text \
  --externalize_embedder \
  --quantization_recipe=dynamic_wi4b32_afp32 \
  --prefill_lengths=32,128,512,1024 \
  --cache_length=2048

Quantization: dynamic_wi4b32_afp32 — int4 weights, block size 32, dynamic-range fp32 activations. Embedder externalized and quantized.
KV cache: exported with cache_length=2048 (ekv2048) to bound runtime memory on low-RAM devices; prefill signatures at 32/128/512/1024 tokens.
Chat template: the source model's Jinja chat template is bundled in the .litertlm; litert-lm applies it automatically. No template override was needed for coherent output.
Multimodal variant: exported with --task=image_text_to_text --vision_encoder_quantization_recipe=weight_only_wi8_afp32 (int8 weight-only for the SigLIP tower, per exporter guidance for CPU execution of conv ops). Adds the SigLIP-400M vision encoder (27 layers, 896×896 input, 256 image tokens) plus a small vision adapter (2.87 MiB). Size increase: +0.41 GiB (2.41 → 2.82 GiB), ~17%.

Full multimodal export command:
```
litert-torch export_hf \
  --model=google/medgemma-1.5-4b-it \
  --output_dir=/tmp/medgemma15_4b_mm \
  --task=image_text_to_text \
  --externalize_embedder \
  --quantization_recipe=dynamic_wi4b32_afp32 \
  --vision_encoder_quantization_recipe=weight_only_wi8_afp32 \
  --prefill_lengths=32,128,512,1024 \
  --cache_length=2048
```
GPU/NPU realistically needed for the multimodal variant on low-end phones. The SigLIP encoder is convolution/matmul-heavy and turns each 896×896 image into 256 tokens that then prefill the decoder. On a CPU-only low-end device this makes time-to-first-token for an image query painfully long; the exporter itself distinguishes CPU vs GPU vision recipes for this reason. Pass --vision-backend gpu (or NPU where available) and keep text-only queries on CPU. If the target fleet is low-RAM / CPU-only, ship the text-only variant and treat multimodal as an opt-in for capable devices.

Multimodal smoke test (CPU vision backend) passed: prompted with a test image and "Describe what you see in this image," the model returned an accurate description of the image contents — the vision encoder → decoder path works end-to-end in the .litertlm.

Quality verification

Sanity check (text-only artifact, CPU backend): prompt "How do you differentiate bacterial from viral pneumonia?" produced a coherent, well-structured, clinically reasonable answer with correct Gemma chat formatting and clean termination.

Eval parity (ai-edge-eval 0.0.1, lm-eval harness)

Task: mmlu_clinical_knowledge_generative (MMLU clinical knowledge, generative protocol — the model writes the answer; chat template applied on both sides), first 50 questions.

Model	Runner / backend	Metric	Score
`.litertlm` int4 (this repo)	litert-lm, CPU	acc (letter extraction)	0.58 (29/50)
`google/medgemma-1.5-4b-it` bf16	HF transformers, CUDA	acc (letter extraction)	0.58 (29/50)
`google/medgemma-1.5-4b-it` bf16 (reference, classic MC loglikelihood, no chat template)	HF transformers, CUDA	acc	0.54

Result: the int4 on-device model matched the bf16 source model exactly on this 50-question set (0.58 vs 0.58). The int4 run produced 0 server errors and 5 unparseable answers (vs 4 for bf16) — i.e. essentially identical behavior. This is a smoke-test level of parity, not a guarantee; run a larger, task-specific eval before deployment.

The generative task's stock get_response filter does not parse MedGemma's prose answers ("The correct answer is B…"), so accuracy was computed from the harness-recorded sample outputs with an identical answer-letter extractor for both models (unparseable responses counted as wrong).

Note: 50-question samples give a coarse parity signal (±~7 pp stderr), not a leaderboard number. Validate on your own task before deployment.

Benchmark (indicative only — server CPU, not a phone)

litert-lm benchmark, CPU backend, AMD EPYC 7K62 (shared cloud slice), prefill 256 / decode 128 tokens:

Metric	Value
Prefill speed	12.04 tokens/s
Decode speed	5.74 tokens/s
Engine init time	19.9 s
Time to first token	21.4 s

Expect different (often better) numbers on modern Arm big cores with XNNPack; measure on your target devices. ~2.6 GB of weights must fit in memory/mmap: devices with <4 GB free RAM will struggle; the text-only variant is the right choice for low-RAM phones.

Running it

# after accepting the HAI-DEF terms on this repo's page
litert-lm run --from-huggingface-repo litert-community/MedGemma-1.5-4B-IT \
  medgemma-1.5-4b-it_q4_block32_ekv2048.litertlm \
  --prompt "How do you differentiate bacterial from viral pneumonia?"

On Android, use the LiteRT-LM engine (or MediaPipe LLM Inference API) and load the .litertlm from local storage. GPU/NPU backends help the multimodal variant substantially; the text-only variant runs acceptably on CPU.

Intended use & limitations

Offline health question answering, patient education, clinical decision support — always with a human professional in the loop.
Single-turn prompts; the model is not optimized for multi-turn chat.
English-centric training; validate for other languages.
int4 quantization can shift factual recall and calibration relative to the bf16 source model — the parity eval above is a smoke test, not a clearance.

Citation

Source model and documentation: MedGemma on HAI-DEF Converted and packaged with LiteRT-Torch / LiteRT-LM.

Downloads last month: 3

Model tree for litert-community/MedGemma-1.5-4B-IT

Base model

google/medgemma-1.5-4b-it

Finetuned

(82)

this model