Nekodeus/gemma3-4b-it-openvino

Gemma-3-4B-it OpenVINO (INT8)

OpenVINO IR conversion of google/gemma-3-4b-it with weight-only INT8 quantization, produced on CPU-only Kaggle with optimum-cli (fully offline, token-free).

• Base model: google/gemma-3-4b-it (Gemma3ForConditionalGeneration, 4.3B params, Gemma license)
• Task: image-text-to-text (multimodal: vision encoder + text embeddings + language model)
• Precision: weights INT8 (NNCF), activations FP16/FP32 as traced
• Converted with: optimum-intel 2.2.0, openvino 2026.4.1, transformers 5.16.1 (MAX_TRANSFORMERS_VERSION gate bypassed)
• 128k context, 17 supported languages, built-in image understanding

Usage

optimum-intel

from optimum.intel import OVModelForVisualCausalLM
from transformers import AutoProcessor

model = OVModelForVisualCausalLM.from_pretrained("Nekodeus/gemma3-4b-it-openvino")
processor = AutoProcessor.from_pretrained("Nekodeus/gemma3-4b-it-openvino")
inputs = processor("What is in this image?", images=image, return_tensors="pt")
print(processor.decode(model.generate(**inputs, max_new_tokens=64)[0]))

OpenVINO GenAI

ov::genai::VLMPipeline pipe("path/to/Nekodeus/gemma3-4b-it-openvino", "CPU");

Reproduce

optimum-cli export openvino \
  --model google/gemma-3-4b-it \
  --task image-text-to-text \
  --weight-format int8 \
  gemma3-4b-it-openvino

License

Gemma Terms of Use (gated on HF; this conversion is un-gated).