gatilin/lfm3-vl-2b-vision-encoder

LFM3-VL-2B vision tower and projector

The vision tower and multimodal projector extracted from LiquidAI/LFM3-VL-2B, source revision 01f2d64a885266638807097e1c571065df4cca2f. This package does not contain language-model weights.

Original BF16 tensor names, shapes and values are preserved in model.safetensors: model.vision_tower.* and model.multi_modal_projector.*. The vision tower has 14 layers: 8 convolutional and 6 bidirectional attention layers. The projector merges 2×2 visual patches into language-sized features. tensor_mapping.json records the original offsets; extraction.json records source-range byte checks and SHA-256.

Usage

Install PyTorch, transformers>=5.21,<5.22, accelerate, safetensors, Pillow and compatible torchvision. The retained upstream Python files implement the original vision tower and image processor. component.py loads only the vision tower and projector with strict state matching:

import importlib
from component import load_model, native

root = '/path/to/this/package'
model = load_model(root)
native(root)
image_module = importlib.import_module('_lfm3_vl_vision_native.image_processing_lfm3_vl')
# Configure Lfm3VlImageProcessor from processor_config.json["image_processor"].
# Pass its pixel_values and image_grid_thw to model(pixel_values, image_grid_thw).

The processor configuration in processor_config.json includes original normalization, dynamic resize, and patch ordering. See validation.json for the real-image CPU smoke test, including the exact tested software version, shapes, and limits. The test does not establish parity with the complete original language model or GPU execution.

License

Distributed under the original LFM Open License v1.0; it includes commercial-use conditions and a revenue threshold. Review the original terms before use or redistribution. Upstream source-code attribution is retained in the included files.