Instructions to use webbrain-one/Laguna-XS-2.1-Vision-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use webbrain-one/Laguna-XS-2.1-Vision-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="webbrain-one/Laguna-XS-2.1-Vision-NVFP4", trust_remote_code=True)
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("webbrain-one/Laguna-XS-2.1-Vision-NVFP4", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("webbrain-one/Laguna-XS-2.1-Vision-NVFP4", trust_remote_code=True, device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use webbrain-one/Laguna-XS-2.1-Vision-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webbrain-one/Laguna-XS-2.1-Vision-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbrain-one/Laguna-XS-2.1-Vision-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/webbrain-one/Laguna-XS-2.1-Vision-NVFP4

SGLang

How to use webbrain-one/Laguna-XS-2.1-Vision-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "webbrain-one/Laguna-XS-2.1-Vision-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbrain-one/Laguna-XS-2.1-Vision-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "webbrain-one/Laguna-XS-2.1-Vision-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbrain-one/Laguna-XS-2.1-Vision-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use webbrain-one/Laguna-XS-2.1-Vision-NVFP4 with Docker Model Runner:
```
docker model run hf.co/webbrain-one/Laguna-XS-2.1-Vision-NVFP4
```

Laguna XS 2.1 Vision (NVFP4)

Laguna XS 2.1 with sight. A vision-language development checkpoint that connects Poolside's agentic coding model to the MoonViT vision encoder from Kimi-K2.6 through a trained PatchMerger projector.

The text backbone and vision tower remain frozen. The only newly trained parameters are the 30,679,808-parameter projector that merges each 2x2 group of MoonViT patches and maps the resulting 4608-dimensional representation into Laguna's 2048-dimensional token space.

Why vision at WebBrain

At WebBrain, we build browser agents that need to understand the visual state of the web—not just extracted text. Screenshots, charts, dashboards, rich editors, and the location and appearance of controls are part of real browser work, so vision is a practical product requirement.

In our American–Chinese open-model frontier benchmark, Laguna XS 2.1 stood out as a strong model in its size class, but the upstream checkpoint is text-only. This project adds a basic MoonViT vision bridge while keeping both the language backbone and vision tower frozen.

This repository is still being assembled. The pinned NVFP4 text backbone is complete and verified, and the frozen MoonViT tower plus final 100K-example projector are packaged and fingerprint-verified. Multimodal model code, processor/serving integration, NVFP4 equivalence, and final image-inference validation are still pending, so this is not yet a stock ready-to-serve checkpoint.

Component	Detail
Text backbone	Laguna XS 2.1, 33B total / 3B active MoE, 2048 hidden size — frozen
Packaged text weights	NVFP4 from `poolside/Laguna-XS-2.1-NVFP4` — exact pinned copy
Vision tower	MoonViT-3d from Kimi-K2.6, 27 layers, 1152-dimensional patch features — frozen
Projector	`LayerNorm -> 2x2 merge -> Linear(4608, 4608) -> GELU -> Linear(4608, 2048)` — trained in BF16
Projector size	30,679,808 trainable parameters
Training envelope	Up to 512 merged image tokens inside 2,048-token training sequences
Backbone context	262,144 tokens, inherited from Laguna XS 2.1
Target hardware	Blackwell; final single-RTX PRO 6000 96 GB serving recipe pending validation

Build status

Pin the upstream Laguna XS 2.1 NVFP4 revision.
Copy and fingerprint-verify all 16 backbone files (21.60 GB).
Pass backward and 2,048-token memory gates before the full run.
Cache 100,000 MoonViT training examples.
Finish the 100,000-example projector run.
Add and fingerprint-verify the frozen MoonViT tower and final projector.
Assemble multimodal configuration, processor, and serving integration.
Pass final single-GPU loading, image inference, and regression gates.

Included vision artifacts

File	Contents	Size	SHA-256
`vision_tower.safetensors`	Frozen MoonViT-3d tower, 329 tensors, all BF16	833,765,768 bytes	`befe801bd7dfe8bf5630fef56a7f53c2235065599ca9eea4d995040e2e6fd183`
`mm_projector.safetensors`	Final step-782 PatchMerger projector, 6 tensors, all BF16	61,360,104 bytes	`7837384f18be69a4f875ca44a8ed69ec186501d70896daf4b305fa77547974de`

Machine-readable provenance is in VISION_ADAPTER_MANIFEST.json.

Provenance

The packaged text backbone is copied from poolside/Laguna-XS-2.1-NVFP4 at immutable revision d32afde8b09af1539b49ff96ff5551c674485f8e. Every copied file was checked against its upstream Git blob or LFS SHA-256 fingerprint, then independently rechecked after upload.

Projector training used the frozen BF16 Laguna XS 2.1 backbone and frozen MoonViT-3d features. This repository now pairs the final trained projector and frozen tower with the verified NVFP4 backbone above. NVFP4 equivalence and end-to-end behavior will be validated after the remaining multimodal integration is assembled.

Usage

A serving command is intentionally not published yet. The current repository contains the complete text backbone, frozen vision tower, and final projector, but not the multimodal processor and serving assembly that connects them. A tested quickstart will be added after integration passes the final gates.

Method credit

The overall construction and model-card approach was inspired by Baseten's GLM-5.2-Vision-NVFP4: keep the text backbone and MoonViT tower frozen, train a compact PatchMerger projector between them, and publish the provenance and hardware constraints explicitly. Credit to the Baseten team for demonstrating this practical recipe. This repository does not reuse Baseten model weights, benchmark results, or deployment artifacts.

License

The redistributed Laguna XS 2.1 NVFP4 backbone remains subject to the included OpenMDW-1.1 license and Poolside's source notices. The included MoonViT tower remains subject to the included Kimi-K2.6 Modified MIT terms. The newly trained projector is documented in the manifest above. Downstream users remain responsible for complying with both upstream licenses.

Acknowledgements

Built on Poolside's Laguna XS 2.1 NVFP4 and Moonshot AI's Kimi-K2.6, with the vision-attachment method inspired by Baseten's GLM-5.2-Vision-NVFP4. These teams were not involved in this development checkpoint; please do not direct issues with this repository to them.

Want this model on your inference provider?

Ask your inference provider—such as OpenRouter or another OpenAI-compatible managed service—to deploy this exact repository with its multimodal processor and serving plugin. Deploying only the upstream text backbone will not enable image input.

Experimental status, roadmap, and get involved

Experimental vision package. The MoonViT adapter and its serving package are experimental. Live end-to-end validation of the NVFP4 package on an RTX 5090 is still provisioning: the pinned SGLang container image is being fetched, so neither the text nor image smoke test has passed yet. Fine-grained OCR, small-object and control identification, GUI grounding, and hallucination calibration remain limited. The roadmap is to expand and diversify the training data, add more high-resolution OCR and UI examples, and pursue broader parameter-efficient tuning if community interest warrants the investment. To contribute evaluation or training data, sponsor compute, or explore a design partnership, use the community interest form. Do not use this package as the sole decision source for safety-critical automation.

Downloads last month: -

Safetensors

Model size

33B params

Tensor type

F32

BF16

Model tree for webbrain-one/Laguna-XS-2.1-Vision-NVFP4

Base model

moonshotai/Kimi-K2.6

Quantized

(45)

this model