Instructions to use omlab/VLX-Seek-1.5-10B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use omlab/VLX-Seek-1.5-10B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="omlab/VLX-Seek-1.5-10B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("omlab/VLX-Seek-1.5-10B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use omlab/VLX-Seek-1.5-10B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "omlab/VLX-Seek-1.5-10B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "omlab/VLX-Seek-1.5-10B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/omlab/VLX-Seek-1.5-10B

SGLang

How to use omlab/VLX-Seek-1.5-10B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "omlab/VLX-Seek-1.5-10B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "omlab/VLX-Seek-1.5-10B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "omlab/VLX-Seek-1.5-10B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "omlab/VLX-Seek-1.5-10B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use omlab/VLX-Seek-1.5-10B with Docker Model Runner:
```
docker model run hf.co/omlab/VLX-Seek-1.5-10B
```

VLX-Seek-1.5-10B

VLX-Seek-1.5-10B is the open-source 10B model in the VLX-Seek 1.5 family, designed for fine-grained perception and visual grounding in embodied scenarios. It targets practical settings such as drones, robots, robotic dogs, surveillance cameras, inspection systems, and other edge-side visual intelligence applications where a model must identify what is present, localize the right instance, and avoid grounding objects that are absent.

Unlike coordinate-generation-based VLMs that directly decode bounding-box numbers, VLX-Seek reformulates localization as region retrieval and region reference. Candidate visual regions are represented as addressable entities, and the model answers by selecting, comparing, and referring to those regions. This makes localization more aligned with the strengths of language models: comparison, selection, reference, and reasoning.

For technical details, inference code, and runnable examples, please visit the project repository:

GitHub: om-ai-lab/VLX-Seek

Model Highlights

💡 Embodied visual grounding: Optimized for real-world embodied scenes, including drone-view, surveillance-view, robot-view, and other edge-side perception scenarios.
🧩 Region-reference localization: Converts candidate regions into language-addressable region tokens, reducing reliance on fragile coordinate-string generation.
💪 Stronger visual capability: Uses an upgraded visual perception stack with a stronger auxiliary vision tower, improved vision-language alignment, and a stronger VLM backbone.
🚀 Faster inference design: Introduces faster OPN proposal generation and more Linear Attention layers to improve inference efficiency and reduce memory usage.
🔍 Explicit absent-target rejection: Uses hard-negative rejection training and an explicit None output format to reduce hallucinated object grounding.
📦 Multi-scale family: VLX-Seek 1.5 is planned in 0.6B, 3B, and 10B sizes. This model card describes the 10B checkpoint.

Intended Use

VLX-Seek-1.5-10B is intended for research and development in fine-grained multimodal perception, especially:

Open-vocabulary object detection and localization.
Referring expression comprehension.
Multi-object visual grounding.
Object counting with region-level evidence.
Drone-view perception with small objects and dense layouts.
Robot-centric and embodied spatial reasoning.
Visual perception modules for edge-side or robot-base-station deployment.
Experiments that require explicit rejection of absent targets.

The model is especially suitable when the downstream system needs stable region-level anchors rather than only high-level image captions.

How to Use

VLX-Seek uses a custom region-reference inference pipeline, including candidate-region proposal generation, region-token construction, prompt formatting, and output post-processing. We provide the full inference code and detailed runnable examples in the official repository:

https://github.com/om-ai-lab/VLX-Seek

Please follow the repository instructions for installation, model loading, inference scripts, prompt templates, and visualization examples.

Evaluation

General Recognition

Table 1: Results on common object detection benchmarks.

Table 2: Results on referring expression comprehension benchmarks.

Table 3: Results on REC and counting benchmarks.

Drone Scenarios

Table 4: Results on drone-view benchmarks.

Embodied Robot Scenarios

Table 5: Results on embodied spatial reasoning benchmarks.

Object Hallucination Evaluation

Table 6: Object hallucination comparison on HumanRef, VisDrone, and RefDrone.

For full comparison tables and qualitative examples, see the VLX-Seek 1.5 blog:

VLX-Seek 1.5: Enhanced Fine-grained Perception for Embodied Scenarios

Limitations

VLX-Seek relies on candidate regions. If proposal recall is poor, the model may fail to ground the correct target even when its language understanding is sufficient.
Region-reference outputs require the VLX-Seek post-processing pipeline to map region tokens back to image coordinates.
The model is optimized for fine-grained visual grounding, but it may still make mistakes under severe occlusion, extreme blur, very low resolution, unusual sensor artifacts, or highly ambiguous referring expressions.
Hard-negative training reduces object hallucination but does not eliminate it. Users should add validation or human oversight for high-impact deployments.

Ethical Considerations

VLX-Seek-1.5-10B can be used in embodied and monitoring scenarios, including cameras, drones, and robots. Such applications may involve privacy, safety, and fairness risks. Users are responsible for complying with applicable laws and institutional policies, obtaining appropriate consent where needed, and avoiding deployments that enable unlawful surveillance, discrimination, or unsafe autonomous actions.

Citation

If you use VLX-Seek-1.5-10B, please cite the project:

@misc{vlxseek15_2026,
  title = {VLX-Seek 1.5: Enhanced Fine-grained Perception for Embodied Scenarios},
  author = {Om AI Lab},
  year = {2026},
  url = {https://om-ai-lab.github.io/2026_07_06_vlx_seek_1_5_en.html}
}

Contact

For technical details, inference examples, and updates, please use the official project repository:

om-ai-lab/VLX-Seek

Downloads last month: -

Safetensors

Model size

10B params

Tensor type

I64

BF16

Collection including omlab/VLX-Seek-1.5-10B

VLX-Seek 1.5-Models

Collection

1 item • Updated about 19 hours ago • 2