Azaz666/Ovis2-4B-PYTORCH-INT4

AIDC-AI__Ovis2-4B__pytorch_int4

This is a PyTorch INT4 (custom) (4-bit) quantized version of AIDC-AI/Ovis2-4B.

Quantization Details

• Method: PyTorch INT4 (custom)
• Bits: 4
• Base model: AIDC-AI/Ovis2-4B
• Group size: 128
• Format: Per-group asymmetric, packed uint8 (2x int4 per byte)
• Vision encoder: FP16

Usage

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "{REPO_ID}",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("{REPO_ID}", trust_remote_code=True)

Replace {REPO_ID} with the repo ID of this model.

Original Model

See AIDC-AI/Ovis2-4B for the original FP16 model.