LFM3-VL-2B-int4-ov

• Model creator: LiquidAI
• Original model: LFM3-VL-2B

Description

This is LFM3-VL-2B model converted to the OpenVINO™ IR (Intermediate Representation) format with weights compressed to INT4 by NNCF.

Quantization Parameters

Weight compression was performed using optimum-cli export openvino (nncf.compress_weights) with the following parameters:

• mode: INT4_ASYM
• ratio: 1.0
• group_size: 64

The parameters above apply to the language model. The remaining sub-models are compressed with the Optimum Intel defaults for this export, see openvino_config.json.

For more information on quantization, check the OpenVINO model optimization guide.

Conversion

optimum-cli export openvino --model LiquidAI/LFM3-VL-2B --task image-text-to-text --trust-remote-code --weight-format int4 --group-size 64 --ratio 1.0 LFM3-VL-2B-int4-ov

Compatibility

The provided OpenVINO™ IR model is compatible with:

• OpenVINO version 2026.5.0 (currently available as nightly builds) and higher
• Optimum Intel with LFM3 support from huggingface/optimum-intel#2087 (until it is merged and released)

Running Model Inference with Optimum Intel

1. Install packages required for using Optimum Intel integration with the OpenVINO backend:

pip install "git+https://github.com/huggingface/optimum-intel.git@refs/pull/2087/head" --extra-index-url https://download.pytorch.org/whl/cpu
pip install -U --pre openvino openvino-tokenizers --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install "transformers==5.20.0"
pip install torchvision pillow --extra-index-url https://download.pytorch.org/whl/cpu

2. Run model inference on an image:

from PIL import Image
from transformers import AutoProcessor
from optimum.intel.openvino import OVModelForVisualCausalLM

model_id = "OpenVINO/LFM3-VL-2B-int4-ov"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = OVModelForVisualCausalLM.from_pretrained(model_id, trust_remote_code=True)

image = Image.open("image.jpg").convert("RGB")
inputs = model.preprocess_inputs(text="Describe this image.", image=image, processor=processor)
outputs = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(processor.decode(outputs[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True))

3. Run model inference on a video:

inputs = model.preprocess_inputs(text="Describe this video.", video="video.mp4", processor=processor)
outputs = model.generate(**inputs, max_new_tokens=200, do_sample=False)
print(processor.decode(outputs[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True))

Video decoding requires a video backend supported by Transformers, for example pip install torchcodec or pip install av.

For more examples and possible optimizations, refer to the Inference with Optimum Intel.

You can find more detailed usage examples in OpenVINO Notebooks:

• LFM3
• VLM notebooks

Running Model with OpenAI client and OpenVINO Model Server

Use an OVMS build that includes OpenVINO and OpenVINO GenAI 2026.5.0 or higher and explicitly supports LFM3-VL in the visual language pipeline. The commands below are deployment examples, not confirmation of end-to-end OVMS validation for this artifact. A latest or weekly image is not sufficient unless it includes the required model support. For NPU, the runtime and driver caveats in Compatibility also apply.

1a. Deploy on Windows using the binary package:

mkdir models
curl.exe -L -o ovms.zip https://storage.openvinotoolkit.org/repositories/openvino_model_server/packages/weekly/latest/ovms_windows.zip
tar -xf ovms.zip
ovms.exe --rest_port 8000 --source_model OpenVINO/LFM3-VL-2B-int4-ov --model_repository_path models --target_device CPU

1b. Deploy in a Docker container on Linux:

mkdir -p "$HOME/models"
docker run --rm --user $(id -u):$(id -g) -p 8000:8000 -v "$HOME/models:/models" openvino/model_server:latest --rest_port 8000 --source_model OpenVINO/LFM3-VL-2B-int4-ov --model_repository_path /models --target_device CPU

For NPU deployment, prefer a supported Windows binary package. The Docker example requires device access and permissions appropriate for the selected target; the command as shown is ready for CPU, not a complete NPU container setup. OVMS detects the model task from the source model metadata.

2. Install the client library:

pip install openai

3. Wait for the model to load, then send a local JPEG image with the OpenAI client:

import base64
from pathlib import Path
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
image_base64 = base64.b64encode(Path("image.jpg").read_bytes()).decode("ascii")
stream = client.chat.completions.create(
    model="OpenVINO/LFM3-VL-2B-int4-ov",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Describe this image."},
            {"type": "image_url", "image_url": {
                "url": f"data:image/jpeg;base64,{image_base64}",
            }},
        ],
    }],
    max_tokens=200,
    stream=True,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)
print()

Reasoning is enabled by default in the model chat template. Remove extra_body to let the model think before answering. The image is embedded in the request; no remote image URL allowlist is needed.

See the OVMS VLM deployment guide for supported builds, device configuration, and API examples.

Limitations

Check the original model card for limitations.

Legal information

The original model is distributed under the LFM Open License v1.0. More details can be found in LFM3-VL-2B.

Disclaimer

Intel is committed to respecting human rights and avoiding causing or contributing to adverse impacts on human rights. See Intel's Global Human Rights Principles. Intel's products and software are intended only to be used in applications that do not cause or contribute to adverse impacts on human rights.