Nekodeus/internlm25-7b-chat-openvino

InternLM2.5-7B-Chat OpenVINO (INT8)

OpenVINO IR conversion of Shanghai_AI_Laboratory/InternLM2_5-7B-Chat with weight-only INT8 quantization, produced on CPU-only Kaggle with optimum-cli (fully offline, token-free).

• Base model: Shanghai_AI_Laboratory/InternLM2_5-7B-Chat (InternLM2ForCausalLM, 7.7B params, Apache-2.0)
• Task: text-generation
• Precision: weights INT8 (NNCF), activations FP16/FP32 as traced
• Converted with: optimum-intel 2.2.0, openvino 2026.4.1, transformers 5.16.1 (optimum's MAX_TRANSFORMERS_VERSION gate bypassed; export verified working)
• Requires trust_remote_code=True on the PyTorch side (InternLM2 custom code); the OV repo ships the tokenizer/config needed by optimum-intel

Usage

optimum-intel

from optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer

model = OVModelForCausalLM.from_pretrained("Nekodeus/internlm25-7b-chat-openvino")
tok = AutoTokenizer.from_pretrained("Nekodeus/internlm25-7b-chat-openvino", trust_remote_code=True)
inputs = tok("<|BOT|>What is OpenVINO?<|EOT|>", return_tensors="pt")
print(tok.decode(model.generate(**inputs, max_new_tokens=128)[0]))

OpenVINO GenAI

ov::genai::LLMPipeline pipe("path/to/Nekodeus/internlm25-7b-chat-openvino", "CPU");

Reproduce

optimum-cli export openvino \
  --model Shanghai_AI_Laboratory/InternLM2_5-7B-Chat \
  --task text-generation \
  --weight-format int8 \
  --trust-remote-code \
  internlm25-7b-chat-openvino