CodeGemma-7B-it OpenVINO (INT8)

OpenVINO IR conversion of google/codegemma-7b-it with weight-only INT8 quantization, produced on CPU-only Kaggle with optimum-cli (fully offline, token-free).

Base model: google/codegemma-7b-it (GemmaForCausalLM, 8.5B params, Gemma license)
Task: text-generation
Precision: weights INT8 (NNCF), activations FP16/FP32 as traced
Converted with: optimum-intel 2.2.0, openvino 2026.4.1, transformers 5.16.1
Instruction-tuned code model (Gemma-1B coder head on Gemma-7B base), 8k context, Python/program synthesis

Usage

optimum-intel

from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer

model = OVModelForCausalLM.from_pretrained("Nekodeus/codegemma-7b-it-openvino")
tok = AutoTokenizer.from_pretrained("Nekodeus/codegemma-7b-it-openvino")
inputs = tok("<start_of_turn>user\nWrite a Python function to reverse a string<end_of_turn>\n<start_of_turn>model\n", return_tensors="pt")
print(tok.decode(model.generate(**inputs, max_new_tokens=256)[0]))

OpenVINO GenAI

ov::genai::LLMPipeline pipe("path/to/Nekodeus/codegemma-7b-it-openvino", "CPU");

Reproduce

optimum-cli export openvino \
  --model google/codegemma-7b-it \
  --task text-generation \
  --weight-format int8 \
  codegemma-7b-it-openvino

License

Gemma Terms of Use (gated on HF; this conversion is un-gated).

Downloads last month: -