Instructions to use prithivMLmods/Fara1.5-9B-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use prithivMLmods/Fara1.5-9B-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="prithivMLmods/Fara1.5-9B-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("prithivMLmods/Fara1.5-9B-FP8")
model = AutoModelForMultimodalLM.from_pretrained("prithivMLmods/Fara1.5-9B-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use prithivMLmods/Fara1.5-9B-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "prithivMLmods/Fara1.5-9B-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/Fara1.5-9B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/prithivMLmods/Fara1.5-9B-FP8

SGLang

How to use prithivMLmods/Fara1.5-9B-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "prithivMLmods/Fara1.5-9B-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/Fara1.5-9B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "prithivMLmods/Fara1.5-9B-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "prithivMLmods/Fara1.5-9B-FP8",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use prithivMLmods/Fara1.5-9B-FP8 with Docker Model Runner:
```
docker model run hf.co/prithivMLmods/Fara1.5-9B-FP8
```

Fara1.5-9B-FP8

Fara1.5-9B-FP8 is a FP8 dynamic quantized version of microsoft/Fara1.5-9B, a multimodal computer use agent (CUA) for web browsers developed by Microsoft Research AI Frontiers. This quantization reduces model size and memory footprint while preserving the model's action-grounding and web-task-completion capabilities, making deployment more accessible on smaller GPUs.

Fara1.5-9B is supervised fine-tuned from Qwen3.5-9B on synthetic web-task trajectories generated by FaraGen1.5. It observes the browser through screenshots and emits structured tool calls (click, type, scroll, visit URL, web search, and so on) to complete tasks end-to-end. It is vision-only at perception time, with coordinate-grounded actions and a critical-points safety design that pauses before personal info entry, payments, submissions, sign-ins, and other irreversible actions.

Quantization Details

Quantization was performed using llmcompressor with the following recipe:

default_stage:
  default_modifiers:
    QuantizationModifier:
      targets: [Linear]
      ignore: ['re:.*lm_head', 're:.*embed_tokens$', 're:.*visual.*', 're:.*model.visual.*',
        're:.*linear_attn.*']
      scheme: FP8_DYNAMIC
      bypass_divisibility_checks: false

Linear layers are quantized to FP8 with dynamic per-tensor activation scaling. The lm_head, embedding table, all vision-tower (visual) components, and linear_attn layers are excluded from quantization and remain at full precision to preserve grounding accuracy and multimodal fidelity.


Base model	microsoft/Fara1.5-9B
Quantization scheme	FP8_DYNAMIC (Linear layers only)
Format	compressed-tensors
Original model size	~18.8 GB
Compressed model size	~13.5 GB
Size reduction	~28%
Excluded from quantization	lm_head, embed_tokens, visual encoder, linear_attn

Model Details


Developer (base model)	Microsoft Research AI Frontiers
Quantized by	prithivMLmods
Architecture	Multimodal decoder-only LM (image + text to text)
Parameters	9B
Context length	262,144 tokens
Inputs	User goal (text), current screenshot(s), prior agent thoughts and actions
Outputs	Chain-of-thought block followed by a tool-call block (XML-tagged)
License	MIT
Base model	Qwen3.5-9B (via Fara1.5-9B)

Use with vLLM

Fara1.5-9B-FP8 is served through vLLM with native support for compressed-tensors FP8 checkpoints.

Requirements

torch >= 2.11.0
vllm >= 0.19.1
A GPU with FP8 support recommended (Hopper or Blackwell class) for best throughput; also runs on Ampere with FP8 dequantized on the fly.

Serve

vllm serve prithivMLmods/Fara1.5-9B-FP8 \
  --max-model-len 262144 \
  --limit-mm-per-prompt image=10

Client request

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

screenshot_b64 = capture_browser_screenshot()  # base64 PNG

messages = [
    {"role": "system", "content": FARA_SYSTEM_PROMPT},
    {"role": "user", "content": [
        {"type": "text", "text": "Book a table for 2 at a sushi place in Sunnyvale for Friday 7pm."},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{screenshot_b64}"}},
    ]},
]

response = client.chat.completions.create(
    model="prithivMLmods/Fara1.5-9B-FP8",
    messages=messages,
    temperature=0.0,
    max_tokens=2048,
)

print(response.choices[0].message.content)

The assistant reply contains chain-of-thought text followed by a <tool_call>{"name": "computer_use", "arguments": {...}}</tool_call> block. Execute the action in a sandboxed browser, capture the new screenshot, append the assistant turn and a tool/observation turn, and loop until the model emits action="terminate". Keep only the most recent 3 screenshots in the chat history. The screen resolution Fara is most commonly trained with is 1440x900; match this in your sandbox for the most reliable grounding.

System prompt

Use the original Fara system prompt verbatim for best results, as documented in the base model card.

Recommended deployment

As with the base model, the safest way to run Fara1.5-9B-FP8 is inside a sandboxed environment such as MagenticLite, with Docker isolation, domain allow-lists, watch-mode monitoring, and an immediate pause control. Do not run this model with unrestricted browser access on a machine holding sensitive credentials or files.

Intended Use and Limitations

Intended use, known limitations, safety considerations, and responsible AI guidance are unchanged from the base model. See the Fara1.5-9B model card for full details on critical-points safety design, evaluation results, training data, out-of-scope use, and responsible AI considerations. Quantization to FP8 may introduce minor numerical differences relative to the bf16 base model; downstream accuracy should be validated for production use cases.