Instructions to use aboufa/Bonsai-2-Clicky with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use aboufa/Bonsai-2-Clicky with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("aboufa/Bonsai-2-Clicky") config = load_config("aboufa/Bonsai-2-Clicky") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use aboufa/Bonsai-2-Clicky with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf aboufa/Bonsai-2-Clicky:Q2_0 # Run inference directly in the terminal: llama cli -hf aboufa/Bonsai-2-Clicky:Q2_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf aboufa/Bonsai-2-Clicky:Q2_0 # Run inference directly in the terminal: llama cli -hf aboufa/Bonsai-2-Clicky:Q2_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf aboufa/Bonsai-2-Clicky:Q2_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf aboufa/Bonsai-2-Clicky:Q2_0
Use Docker
docker model run hf.co/aboufa/Bonsai-2-Clicky:Q2_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use aboufa/Bonsai-2-Clicky with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "aboufa/Bonsai-2-Clicky" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "aboufa/Bonsai-2-Clicky", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/aboufa/Bonsai-2-Clicky:Q2_0
- Ollama
How to use aboufa/Bonsai-2-Clicky with Ollama:
ollama run hf.co/aboufa/Bonsai-2-Clicky:Q2_0
- Unsloth Desktop
- Pi
How to use aboufa/Bonsai-2-Clicky with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "aboufa/Bonsai-2-Clicky"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "aboufa/Bonsai-2-Clicky" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use aboufa/Bonsai-2-Clicky with Docker Model Runner:
docker model run hf.co/aboufa/Bonsai-2-Clicky:Q2_0
- Lemonade
How to use aboufa/Bonsai-2-Clicky with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull aboufa/Bonsai-2-Clicky:Q2_0
Run and chat with the model
lemonade run user.Bonsai-2-Clicky-Q2_0
List all available models
lemonade list
- Hermes Agent
How to use aboufa/Bonsai-2-Clicky with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "aboufa/Bonsai-2-Clicky"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default aboufa/Bonsai-2-Clicky
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use aboufa/Bonsai-2-Clicky with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "aboufa/Bonsai-2-Clicky"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "aboufa/Bonsai-2-Clicky" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Bonsai 2 for computer use on a Mac (Clicky)
Created using Bonsai by Prism ML.
These are tuned versions of PrismML's Ternary Bonsai 2 9B and 4B for operating a Mac: finding a control on a screenshot, and carrying a task out one step at a time. They were made on 14 and 15 November 2026 for a hackathon entry called Clicky, a menu-bar assistant that runs entirely on a MacBook.
The weights stay ternary. The files keep PrismML's format, size and speed; the only additions are two small pointing decoders in full precision.
Code, notes and every measurement: https://github.com/aboufama/hackathon-clicky
What is here
| File | Size | What it is |
|---|---|---|
Bonsai-2-9B-planpoint-v1-u1500-Q2_0.gguf |
2.56 GB | 9B planner, the model inside Clicky. A complete model file for LM Studio or llama.cpp. It plans a task step by step, points, and writes tool calls. |
patches/bonsai2-9b-planpoint-v1-u1500.patch.safetensors |
245 MB | The same 9B planner as a patch. |
patches/bonsai2-9b-joint-nb-l7-best-trunk.patch.safetensors and decoders/bonsai2-9b-joint-nb-l7-best-dec2x768-L20.safetensors |
272 MB and 111 MB | 9B pointer. A pair: they only work together. |
patches/bonsai2-4b-joint-h1-trunk-s12500.patch.safetensors and decoders/bonsai2-4b-joint-h1-dec2x768-L20-s12500.safetensors |
129 MB and 101 MB | 4B pointer. A pair: they only work together. |
patches/bonsai2-9b-point-v1.patch.safetensors |
221 MB | Multipliers-only repair of the 9B. The first and smallest change; see below. |
A patch holds only the tensors that differ from PrismML's original MLX 2-bit pack (prism-ml/Ternary-Bonsai-2-9B-mlx-2bit or prism-ml/Ternary-Bonsai-2-4B-mlx-2bit). It is applied to that pack in memory.
Compared with the stock models
On every computer-use measure that was run, the tuned models are far ahead of Ternary Bonsai 2 as shipped, at the same size and speed. They were not shown to be better in general, because that was not measured.
"As shipped" is PrismML's original model, measured with the same harness. One look is a single answer; a second look enlarges the area around the first answer and asks again.
| Measure | Bonsai 2 as shipped | This repository | Qwen3.5-9B parent, full precision |
|---|---|---|---|
| Long tasks, 9B planner (120 simulator episodes on tasks never trained on) | 1 of 120 | 27 of 120 | 9 of 120 |
| Pointing, ScreenSpot-v2, all 1,272 cases, 9B planner (written answer) | 43.3% | 84.0% | 91.7% |
| The same, 9B pointer pair | 43.3% | 85.5% with one look, 86.4% with two | 91.7% |
| The same, 4B pointer pair | 12.2% | 82.4% with one look, 85.2% with two | |
| Pointing, macOS app windows, 2,106 cases, 9B pointer pair | 13.2% | 46.8% with one look, 57.5% with two | 63.8% |
| The same, 4B pointer pair | 3.7% | 46.4% with one look, 55.3% with two | |
| Guided multi-step tasks, 75 tasks, 4B with its pointing decoder (an earlier checkpoint, see below) | 3% | 71% |
Limits to keep in mind
- Pointing is still behind the full-precision parent in every row that has one, and further behind UI-Venus-2-9B (95.8% on ScreenSpot-v2, 68.4% on macOS app windows).
- The long-task result is from a simulator. The planner beats the parent there (27 against 9), but the planner was trained on other tasks of the same simulator and the parent was never trained on this format. 93 of 120 episodes still fail.
- General skills were only spot-checked. A 50-question check held steady (see "General skills and tool calls"). PrismML's own general benchmarks were not rerun on the tuned models.
- Tool calling has no like-for-like comparison with the stock model. The numbers that exist are below; they are not a claim of a gain.
Benchmarks
All pointing numbers use greedy decoding with thinking off and the screenshot's long side at 1280 px, which is what the assistant captures. A hit means the point lies inside the target's box. Brackets are 95% bootstrap intervals over cases.
Pointing
| Model | ScreenSpot-v2 (1,272 cases) | macOS app windows (2,106 cases) | Small controls with close neighbours (600 of the macOS cases) |
|---|---|---|---|
| Ternary Bonsai 2 9B, as shipped | 43.3 (40.6โ46.0) | 13.2 (11.7โ14.6) | 14.2 (11.5โ17.0) |
| 9B planner, written answer | 84.0 (81.9โ85.9) | 44.7 (42.6โ46.8) | 51.3 (47.5โ55.3) |
| 9B pointer pair, one look | 85.5 (83.5โ87.3) | 46.8 (44.6โ48.8) | 54.7 (50.7โ58.7) |
| 9B pointer pair, two looks | 86.4 (84.5โ88.3) | 57.5 (55.3โ59.5) | 65.7 (61.8โ69.5) |
| Ternary Bonsai 2 4B, as shipped | 12.2 (10.5โ14.0) | 3.7 (2.9โ4.5) | not measured |
| 4B pointer pair, one look | 82.4 (80.3โ84.4) | 46.4 (44.2โ48.5) | 50.8 (46.8โ54.8) |
| 4B pointer pair, two looks | 85.2 (83.2โ87.2) | 55.3 (53.1โ57.4) | 61.5 (57.5โ65.5) |
| Qwen3.5-9B, the 9B's parent, full precision | 91.7 (90.2โ93.2) | 63.8 (61.8โ65.9) | 65.3 (61.5โ69.2) |
| UI-Venus-2-9B, full precision | 95.8 (94.7โ96.9) | 68.4 (66.4โ70.3) | 72.7 |
| Qwen3.5-4B, 4-bit | 89.9 (88.2โ91.4) | 58.8 (56.7โ60.9) | not measured |
- Near misses. Counting an answer on the target or within 20 px of its box, ScreenSpot-v2 gives 90.6% for the planner, 90.0% for the 9B pair and 88.1% for the 4B pair (one look).
- The macOS set is Screen2AX-Task restricted to 27 apps that no training run saw. Its labels are noisy: of 150 labels checked by eye, 114 were right, 17 wrong and 19 unclear, which lowers every model's score.
- ScreenSpot-v2 was never trained on. One eighth of it was used to choose between training runs, and eight desktop cases share their screens with public macOS training data. For an earlier checkpoint, leaving both out moved the score by under a point.
- The second look costs a second read of the screen. It adds 9 to 11 points on the macOS sets and 1 to 3 on ScreenSpot-v2.
Long tasks (9B planner)
The model gets a request and a numbered list of the front window's controls, and chooses every step itself. An episode counts when the app's end state is right and the model said done. The simulator has ten Mac-style apps; three apps and 14 task templates were held out of training. Three seeds, 120 episodes a model.
| Model | New tasks in apps trained on (42) | Apps never trained on (78) | All (120) |
|---|---|---|---|
| Ternary Bonsai 2 9B, as shipped | 1 | ||
| Qwen3.5-9B parent | 4 (95% interval 4% to 22%) | 5 (3% to 14%) | 9 |
| The planner's starting checkpoint, before training on simulator tasks | 1 | 2 | 3 |
| 9B planner | 12 (17% to 44%) | 15 (12% to 29%) | 27 |
| 9B planner inside the Clicky app's rules | 34 |
How the planner's 93 failed episodes ended: 37 said done with a wrong result, 33 repeated a step that had no effect, 20 never said done, 3 ran out of steps. Further training rounds gave 24 to 28 of 120, which is no clear gain.
Guided tasks (4B with its pointing decoder)
Each step is given in words and the model only points. 75 tasks in the same simulator, run in MLX on a Mac mini (M6, 16 GB).
These rows were measured with the checkpoint at step 11,000 of the 4B run. The pair published here is step 12,500 of the same run, which was not rerun on these tasks; on a 158-screen slice of ScreenSpot-v2 it points better (83.5% against 79.7%).
| Model | Input | Aids | Tasks done |
|---|---|---|---|
| Ternary Bonsai 2 4B, as shipped, written answers | 1280 px | none | 2 of 75 (3%) |
| 4B with pointing decoder | 1280 px | none | 45 of 75 (60%) |
| 4B with pointing decoder | 1600 px | none | 53 of 75 (71%, interval 60% to 81%) |
| 4B with pointing decoder | 1600 px | click snapped to an element, second look when nothing changed | 55 of 75 (73%) |
For reference, in separate runs of the guided tasks the full-precision 9B parent finished 92% and UI-Venus-2-9B 100%.
The multipliers-only patch (bonsai2-9b-point-v1)
This patch changes the multipliers of 176 matrices and nothing else. The score is the probability that every digit of the written answer is acceptable, on two separate sets of 159 ScreenSpot-v2 screens.
| Model | First set | Second set |
|---|---|---|
| Ternary Bonsai 2 9B, as shipped | 0.327 | 0.372 |
With bonsai2-9b-point-v1 |
0.698 | 0.704 |
| Qwen3.5-9B parent | 0.848 | 0.843 |
| UI-Venus-2-9B | 0.932 | 0.939 |
With generated answers on the second set it hits 65.4% with one look and 74.2% with a second look.
General skills and tool calls
- General skills, spot check. 50 short questions in seven skills; the score is the share of questions on which every answer token is the model's first choice. The 9B as shipped scores 0.86 and the 9B planner 0.92. On 50 questions that difference is noise; it shows no loss, not a gain.
- Tool calls, strict check. On 300 tool-call cases of the project's own benchmark, the planner gets every token of the expected call right on 80.0%. A pointing-only checkpoint scored 42.0% on the same check before any tool-call training. The stock model was not measured with this check.
- Tool calls, generated answers. The stock 9B scores 84.8% on the project's 400-case tool benchmark (its parent 89.0%). The tuned models were not run on it.
- Not run on the tuned models: PrismML's general benchmarks, BFCL, ScreenSpot-Pro and OSWorld-G.
Speed
The tuning does not change the model's size or speed. Measured figures:
| Where | What | Time |
|---|---|---|
| Mac mini (M6, 16 GB), MLX | One pointing answer at 1280 px | 2.46 s with the 9B, 1.43 s with the 4B |
| Mac mini (M6, 16 GB), MLX | A further request about a screenshot already read, through the 4B decoder | 76 ms |
| MacBook Air M4 (16 GB), LM Studio | One planner step from a list of controls | 4 to 10 s |
| DGX Spark, llama.cpp | One planner step | 0.6 s (about 85 tokens a second generated) |
How it was done
- Multipliers retuned, ternary values kept. A Bonsai layer is a grid of -1, 0 and +1 with one multiplier for every group of 128 weights. Comparing Bonsai with its parent layer by layer showed that the loss of pointing sits in layers 4 to 19. Retraining only the multipliers recovers most of it.
bonsai2-9b-point-v1shows how small that change is. No ternary value changed, and each multiplier moved by 0.7% on average. The later patches also train the small full-precision parts of the layers. The 4B patch changes no ternary value; the planner patch and the 9B pointer patch each change 12 ternary values in 2 matrices.- A pointing decoder after layer 20. The decoder has two layers and about 25 million parameters. It reads the model's state after layer 20 of 32, scores every image cell against the request, and returns a point without writing any token. The last 12 layers do not run for pointing.
- A planner. The 9B was trained to carry out a task from a list of the front window's controls: each turn it gets the request, the steps so far and the numbered controls, and answers with one tool call.
What was tried and did not help is in docs/TRIALS.md of the code repository: flipping ternary values, transplanting single layers from the parent, pulling internal states toward a teacher, side paths of low rank, and dropping blank image areas.
Using the files
The GGUF. Load it in LM Studio or in PrismML's llama.cpp fork, with thinking off and temperature 0. For screenshots it needs the vision file Ternary-Bonsai-2-9B-mmproj-BF16.gguf from prism-ml/Ternary-Bonsai-2-9B-gguf; this work did not change the vision weights, so that file is used as it is.
The planner expects this system text and one tool named next_step:
You operate a Mac for the user, one step at a time. Reply with exactly one next_step call. Prefer open_app, open_url, key and menu over clicking. To press or fill a control from the list, give its id. Use look only when the control you need is not in the list. When the task is finished, call done with one sentence for the user.
The user message has the lines Task, Apps, Front app | Window, Done so far and Controls. The tool's action is one of open_app, open_url, click, double_click, right_click, type, key, menu, scroll, drag, look, done. The exact tool definition is in weightlab/planner.py of the code repository, and the Swift app in assistant/ shows it in use.
For pointing in writing, the model answers [x, y] on a 0 to 1000 scale to the Qwen3.5 family's grounding prompt, with the screenshot's long side at 1280 px.
The patches and decoders. These need the code repository:
- PyTorch:
weightlab/state.py(apply_patch) andweightlab/head.py(the decoder). - MLX on a Mac:
playground/server.py(get_model) andtaskenv/mlx_head.py.
The decoders do not run in LM Studio or llama.cpp.
Training data and teachers
Training used screens drawn by the repository's own generator and public macOS screenshot sets (listed in docs/BENCHMARK.md, section "Contamination", and docs/MAC_DATA.md). Qwen3.5-9B and UI-Venus-2-9B served as references and teachers, and some training requests were written by Claude Fable.
The planner was tested on real macOS apps only inside the Clicky app, where most of the gain came from the app's own rules, not from the model.
Licence and credits
Apache-2.0, the licence of the models this is built from. LICENSE and NOTICE.txt carry PrismML's notices.
- Ternary Bonsai 2 by Prism ML, Inc.: https://huggingface.co/prism-ml
- Qwen3.5 by Alibaba Cloud, the parent of Bonsai 2.
- Downloads last month
- 72
Model tree for aboufa/Bonsai-2-Clicky
Base model
Qwen/Qwen3.5-4B-Base