Nekodeus/phi-3.5-vision-instruct-openvino

Phi-3.5-Vision-Instruct OpenVINO (INT8)

OpenVINO IR conversion of microsoft/Phi-3.5-vision-instruct with weight-only INT8 quantization, produced on CPU-only Kaggle with optimum-cli (fully offline, token-free).

• Base model: microsoft/Phi-3.5-vision-instruct (Phi3VForCausalLM, 4.2B params, MIT)
• Task: image-text-to-text (multimodal: CLIP vision tower + projection + text embeddings + Phi-3 Mini language model)
• Precision: weights INT8 (NNCF), activations FP16/FP32 as traced
• Converted with: optimum-intel 2.2.0, openvino 2026.4.1, transformers 5.16.1 (remote-code + legacy cache APIs patched for transformers 5)
• 128k context, high-res dynamic resolution, chart/table/document understanding
• 6 IR submodels: language_model, vision_embeddings, vision_projection, text_embeddings, tokenizer, detokenizer

Usage

optimum-intel

from optimum.intel import OVModelForVisualCausalLM
from transformers import AutoProcessor

model = OVModelForVisualCausalLM.from_pretrained("Nekodeus/phi-3.5-vision-instruct-openvino")
processor = AutoProcessor.from_pretrained("Nekodeus/phi-3.5-vision-instruct-openvino")
inputs = processor("Describe this image", images=image, return_tensors="pt")
print(processor.decode(model.generate(**inputs, max_new_tokens=64)[0]))

OpenVINO GenAI

ov::genai::VLMPipeline pipe("path/to/Nekodeus/phi-3.5-vision-instruct-openvino", "CPU");

Reproduce

optimum-cli export openvino \
  --model microsoft/Phi-3.5-vision-instruct \
  --task image-text-to-text \
  --weight-format int8 \
  --trust-remote-code \
  phi-3.5-vision-instruct-openvino