Instructions to use webbrain-one/webbrain-browser-vision-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use webbrain-one/webbrain-browser-vision-tiny with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="webbrain-one/webbrain-browser-vision-tiny")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("webbrain-one/webbrain-browser-vision-tiny")
model = AutoModelForMultimodalLM.from_pretrained("webbrain-one/webbrain-browser-vision-tiny", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use webbrain-one/webbrain-browser-vision-tiny with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webbrain-one/webbrain-browser-vision-tiny"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbrain-one/webbrain-browser-vision-tiny",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/webbrain-one/webbrain-browser-vision-tiny

SGLang

How to use webbrain-one/webbrain-browser-vision-tiny with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "webbrain-one/webbrain-browser-vision-tiny" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbrain-one/webbrain-browser-vision-tiny",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "webbrain-one/webbrain-browser-vision-tiny" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webbrain-one/webbrain-browser-vision-tiny",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use webbrain-one/webbrain-browser-vision-tiny with Docker Model Runner:
```
docker model run hf.co/webbrain-one/webbrain-browser-vision-tiny
```

WebBrain Browser Vision Tiny

WebBrain Browser Vision Tiny is a browser-GUI specialist derived from LiquidAI/LFM2.5-VL-450M. It converts a browser viewport screenshot into the six-section observation format consumed by WebBrain's planning agent. This repository contains the merged Transformers model, not only a LoRA adapter.

Intended use

Browser screenshot description for a downstream automation planner.
Visible UI text, control, form-state, dialog, error, loading, consent, CAPTCHA, and blocker detection.
A small local fallback when a larger vision model is unavailable.

This is not a general replacement for a frontier VLM. It must not be trusted to read passwords, payment details, or other sensitive information, and its observations must not be treated as authorization to perform an action.

Training

Item	Value
Base model	`LiquidAI/LFM2.5-VL-450M`
Method	Supervised fine-tuning with LoRA, then merged
LoRA	rank 16, alpha 32, dropout 0.05
Epochs	1
Effective batch size	16
Learning rate	1e-4, cosine schedule, 3% warmup
Maximum image tokens	256
Precision	BF16 with gradient checkpointing
Teacher	`qwen/qwen3.6-35b-a3b` through OpenRouter
Hardware	One NVIDIA RTX 4090 24 GB
Runtime	6,134 seconds (102.2 minutes)
Peak VRAM	21,954 MiB (about 21.4 GiB)
Trainable parameters	4,128,768 / 452,847,616 (0.9117%)

The final rolling train loss was 0.401. Periodic held-out validation loss improved from 0.624 at step 250 to 0.597 at step 500 and 0.583 at step 750. No NaN, OOM, or worker restart occurred.

Data

The public training corpus is available as webbrain-one/webbrain-browser-vision-tiny-dataset. It contains 15,348 train and 1,298 validation rows (16,646 unique images after deduplication), with image bytes embedded in 18 Parquet shards.

Source	Version/commit	License	Train	Validation	Transformation
HuggingFaceM4/WebSight	v0.1 @ `b11f817`	CC-BY-4.0	9,067	494	Screenshot plus HTML-grounded teacher label
Farama MiniWoB++	`7fd85d7`	MIT	3,910	682	Locally rendered state plus accessibility-grounded teacher label
docling-project/ScreenParse	`bc0113f`	CC-BY-4.0	2,371	122	Screenshot plus dense-annotation-grounded teacher label

The release pipeline removed 51 duplicate images, quarantined 968 rows during the automated public-data privacy review, and validated the six-section format for all 16,646 released rows. A deterministic source-stratified manual audit of 15 examples passed. The teacher accepted 17,665 rows before public filtering.

The immutable local training snapshot ID is f721ca83b8b6308c7c386435325ef2b884d975d349bf91d239e618a2f40a727c. The dataset manifest and SHA256SUMS file record the public shard hashes.

Leakage controls

WebBrain's 100-case test/vision suite was excluded from training and used only for final evaluation. The release checks found:

zero train/validation overlap;
zero exact-image overlap with the WebBrain suite;
zero perceptual-image overlap with the WebBrain suite;
prompt parity between training and production evaluation.

Evaluation

The table reports the unchanged 100-case WebBrain browser-vision suite using the production six-section prompt. “Passes” require the rubric threshold and all critical checks. Difficulty columns are mean rubric scores.

Model	Passes / 100	Mean rubric	Easy	Basic	Intermediate	Advanced	Challenging
Base LFM2.5-VL-450M	0	4.17%	5.49%	6.38%	5.13%	2.36%	1.50%
WebBrain Browser Vision Tiny	30	70.06%	81.99%	74.13%	70.32%	54.94%	68.94%

Both runs completed all 100 cases with zero inference errors. Mean latency was 561 ms for the base model and 2,395 ms for the fine-tuned model on the same RTX 4090 Transformers server. Timing is implementation-specific and is not a hardware-independent speed claim. Machine-readable summaries are included in eval/base-summary.json and eval/final-summary.json.

Representative weak cases

The lowest-scoring held-out cases show where more targeted data is needed:

Case	Category	Screen	Score	Main failure
057	multilingual OCR	Japanese payment	18.2%	Exact heading, action, input, and status text
097	multilingual OCR	Arabic security settings	18.2%	Exact RTL heading, action, input, and status text
067	loading state	Reconnecting	22.2%	Readiness, heading, and spinner state
064	form validation	Turkish tax details	30.0%	Invalid value, exact error, and blocker
068	consent banner	Turkish cookie preferences	30.0%	Overlay title and consent state
069	dashboard	Regional performance	33.3%	Target metric and alert
044	form validation	Invite teammate	40.0%	Invalid input, error, and blocker
084	form validation	Publish dataset	40.0%	Invalid input, error, and blocker
041	authentication	Administrator login	41.7%	Button state and blocker
071	data table	Turkish invoice list	42.9%	Exact target-row status

General-image regression has not yet been measured. This omission matters: browser specialization can improve the target prompt while degrading broad image understanding. Evaluate on a separate general VLM suite before using this checkpoint outside browser screenshots.

Limitations

Small, low-contrast, occluded, or non-Latin text can be missed.
UI labels can be hallucinated, especially around overlays.
Focus, disabled state, form errors, and readiness signals require visual corroboration.
The model inherits teacher errors and may imitate the teacher's response style.
Site styles and languages outside the corpus may regress.
A valid six-section response does not make downstream automation safe.

License and attribution

This derivative is distributed under the LFM Open License v1.0, subject to its redistribution conditions and commercial-use threshold. The complete license text is included in LICENSE. The base model is LiquidAI/LFM2.5-VL-450M; retain its attribution and required notices.

Dataset components retain their upstream licenses: WebSight and ScreenParse are CC-BY-4.0, and MiniWoB++ is MIT. Labels were generated by the disclosed teacher model and then filtered, validated, and manually sampled. The dataset repository uses license: other because it contains multiple upstream licenses.

Reproducibility

Training and dataset preparation code, frozen-source metadata, validation, deduplication, leakage checks, and the 24 GB LoRA configuration are published at esokullu/lfm-finetune.

Downloads last month: -

Safetensors

Model size

0.4B params

Tensor type

BF16

Model tree for webbrain-one/webbrain-browser-vision-tiny

Base model

LiquidAI/LFM2.5-350M-Base

Finetuned

LiquidAI/LFM2.5-350M

Finetuned

LiquidAI/LFM2.5-VL-450M

Finetuned

(31)

this model

webbrain-one
/

webbrain-browser-vision-tiny

WebBrain Browser Vision Tiny

Intended use

Training

Data

Leakage controls

Evaluation

Representative weak cases

Limitations

License and attribution

Reproducibility

Model tree for webbrain-one/webbrain-browser-vision-tiny

Dataset used to train webbrain-one/webbrain-browser-vision-tiny