gatilin/internvlu_vision_bridge

Native InternVL-U InternViT encoder, pixel-shuffle bridge and special-token embeddings.

Extracts the understanding encoder, its 2x2 pixel-shuffle MLP bridge and special-token embedding. A 448x448 image yields 256 projected 2048-d tokens. Generation decoder, QwenImage VAE and language backbone are separate components; this bundle does not perform image generation.

Source and conversion

Model/weights: InternVL-U/InternVL-U, fixed revision f012d760e69712bb47f7d3d09a24280f346cee01. Native code: OpenGVLab/InternVL-U, fixed revision 018a59c1b69690a33698234128aa04e9b3f46c71.

All selected tensor names, shapes, dtypes and payload bytes are preserved, except the explicit IBQ EMA shadow-name mapping. See extraction.json, provenance.json and tensor/conversion mappings. This repository contains component weights and a local native loader, not an AutoModel.from_pretrained checkpoint.

Load

Install requirements.txt in a compatible environment, then:

from huggingface_hub import snapshot_download
import sys
import torch
root = snapshot_download("gatilin/internvlu_vision_bridge")
sys.path.insert(0, root)
from component import load_model
model = load_model(root, dtype=torch.float32, device="cpu")

The model is returned in evaluation mode. Run inference under torch.inference_mode(). Inputs are native tensors/patches, not arbitrary PIL images; inspect component.py and the included validation script for the exact layout. CPU float32 was tested; CUDA/MPS and full end-to-end benchmarks were not tested.

Validation

See validation.json for the executed component tests, shapes and limitations. Source payload equality and native strict loading are required before release. Component tests do not establish full-model generation or understanding quality. Original stochastic decoding settings and normalization conventions must be retained.

License and attribution

The upstream code is MIT (LICENSE). The upstream HF card does not specify a separate weight license; no broader weight-use rights are asserted here. Preserve the original source attribution and consult upstream terms.