Fara1.5-4B-8bit

8-bit MLX quantization of microsoft/Fara1.5-4B, a 4.5B computer-use / web-agent VLM built on Qwen3.5-4B. It reads screenshots and acts on interfaces. Converted with mlx-vlm 0.6.8 (mlx 0.32.0) for inference on Apple Silicon.

Every prior MLX conversion of this model is 4-bit, so 8-bit was the open gap. Fidelity was measured against the unquantized bf16 source, not assumed.

pip install mlx-vlm
python -m mlx_vlm generate \
    --model mlx-community/Fara1.5-4B-8bit \
    --image screenshot.png \
    --prompt "Describe the buttons and the total shown in this screenshot." \
    --max-tokens 256

Image input needs mlx-vlm. Stock mlx-lm loads the text path only, which for a computer-use model is not a smaller model but a broken one.

Quantization

Bits	8
Group size	64
Mode	affine
Effective bits/weight	9.053
Size on disk	4.8 GB
Quantized tensors	249 (language model)
Unquantized tensors	297 (vision tower, bf16)

Effective bits per weight exceeds the nominal 8 because mlx-vlm quantizes only the language model and leaves the vision tower in bf16 by design. The vision path is preserved, which for a computer-use model is the whole point: verified at tensor level, of 297 vision tensors zero carry a .scales entry.

A conversion trap worth knowing about

This model declares dtype: bfloat16 at the top level of its config but float32 in both text_config and vision_config. mlx_vlm.convert honours the sub-configs, so converting without an explicit --dtype bfloat16 silently keeps the scales, biases and the entire vision tower in fp32:

	default	--dtype bfloat16
Effective bpw	10.695	9.053
Size on disk	5.7 GB	4.8 GB
Peak RAM	7.44 GB	6.23 GB
Weight SNR vs bf16	45.22 dB	42.70 dB

This repo is the --dtype bfloat16 build. The higher SNR of the fp32 build is not better fidelity: it is a float32 reconstruction being compared against a bf16 source, bought with 0.9 GB of precision that has no business in an 8-bit repo. Rounding those scales to bf16 accounts for the entire 2.5 dB difference.

Measured fidelity

Against the unquantized bf16 source, greedy decoding, on an M2 Pro / 32 GB.

Metric	bf16 (source)	8-bit
Weight SNR	n/a	42.70 dB
Relative L2	n/a	0.73%
Cosine similarity	n/a	0.999973
Worst single element	n/a	0.003906
Perplexity	3.4974	3.4775
Top-1 agreement	n/a	0.9853
KL (nats/token)	0	0.00086
Task accuracy	7/8	7/8
BLEU vs bf16	n/a	81.30
Decode tok/s	18.5	35.4
Peak RAM	9.23 GB	6.15 GB

Weight-level metrics stream the source tensor by tensor from disk over all 4,204,789,760 quantized parameters. No prompts, no sampling, hardware-independent.

42.70 dB is the expected value, and that is the point. Six 8-bit affine conversions at group size 64 across six different architectures (AREX-Turbo, Fara-7B, Fara1.5-4B, GLM-4.1V, MagenticBrain, OvisOCR2) all land on 42.66-42.70 dB. A reading outside that band is a defect signal, which is exactly how the fp32-scales issue above was caught.

Highest-error tensors, all in the linear-attention input projections:

rel_l2=0.00841  snr=41.50 dB  language_model.model.layers.25.linear_attn.in_proj_a
rel_l2=0.00839  snr=41.53 dB  language_model.model.layers.28.linear_attn.in_proj_a
rel_l2=0.00834  snr=41.57 dB  language_model.model.layers.26.linear_attn.in_proj_a

1.9x faster, 1.5x less memory than bf16.

Vision path, verified working rather than merely present

Given a synthetic UI screenshot, the 8-bit model read both button labels (SubmitOrder, Cancel) and the total (42.50 USD) correctly.

On the perplexity aggregate

The harness reports aggregate perplexity as exp(sum_nll/sum_tokens) over five passages, and flags it as unreliable here: the bf16 source itself spans 10.3x across those passages, so a single out-of-domain passage can dominate. The per-passage ratios are reported instead, and they agree with the aggregate rather than contradict it, all within 1.8% of the bf16 reference:

Passage	bf16 PPL	8-bit PPL	Ratio
prose	4.66	4.69	1.007
python-code	1.23	1.23	0.999
legal	8.82	8.70	0.986
biology	6.62	6.64	1.002
meta	12.74	12.51	0.982

BLEU of 81.30 measures drift, not quality: it treats the bf16 output as ground truth, so it penalises valid paraphrase and would penalise a better answer.

What was not measured

• No judged quality. In the open-ended set 11/18 (bf16) and 14/18 (8-bit) answers hit the 640-token cap, so grading them would have measured truncation rather than quality. This model emits a <think> block, which consumes budget.
• No agentic evaluation. No WebArena, no ScreenSpot, no WebVoyager, no end-to-end computer-use task. This model's actual job is multi-step browser automation and none of that is scored here.
• No standard benchmarks (MMLU, GSM8K). The accuracy layer is 8 short verifiable prompts, not a benchmark.
• The vision check is one screenshot, not a dataset, and it was scored for correctness by reading it, not by a metric.
• Throughput figures are for one M2 Pro and do not transfer to other chips.

Related conversions

Model	Variant	Note
Fara1.5-9B-8bit	8.86 bpw	larger sibling, top-1 agreement 1.000
Fara-7B-8bit	9.11 bpw	previous generation
Fara-7B-4bit	5.44 bpw	previous generation

Credits

All credit for the underlying model belongs to Microsoft. This is a format conversion and quantization; no training or fine-tuning was performed. See the source model card for license, intended use and limitations.