sebpost/TinyClick-ONNX

TinyClick (ONNX, for the browser)

ONNX export of TinyClick (Interspeech 2025), a 0.27B single-turn GUI agent built on Florence-2-base. Given a screenshot and a command such as "open settings", it returns where to click.

Converted to ONNX and quantized, in the layout transformers.js expects, so the model runs fully in the browser (WASM or WebGPU).

Where it comes from

TinyClick was released by Samsung R&D Poland under the MIT license. Samsung's original repositories are no longer online; this export starts from the MIT-licensed mirror at Krystianz/TinyClick (revision 0e1356f0b7cfb416099207121f6a766818ab8a66), which is listed as the base model. The MIT notice is kept in LICENSE.

Files

Part	fp32	fp16	q8	q4
vision_encoder	349 MB	175 MB	89 MB	77 MB
embed_tokens	150 MB	75 MB	38 MB	–
encoder_model	165 MB	83 MB	42 MB	29 MB
decoder_model_merged	370 MB	–	93 MB	61 MB

Recommended for the web: q8 vision encoder and embeddings, q4 encoder and decoder (about 217 MB). On phones, the q4 vision encoder brings it to about 190 MB.

Usage (transformers.js)

import { Florence2ForConditionalGeneration, AutoProcessor, RawImage } from "@huggingface/transformers";

const id = "sebpost/TinyClick-ONNX";
const model = await Florence2ForConditionalGeneration.from_pretrained(id, {
  dtype: { vision_encoder: "q8", embed_tokens: "q8", encoder_model: "q4", decoder_model_merged: "q4" },
});
const processor = await AutoProcessor.from_pretrained(id);

const image = await RawImage.fromURL("screenshot.png");
const prompt = "What to do to execute the command? " + "open settings".trim().toLowerCase();
const inputs = await processor(image, prompt);
const ids = await model.generate({ ...inputs, max_new_tokens: 20 });
const text = processor.batch_decode(ids, { skip_special_tokens: false })[0];
// "click <loc_X><loc_Y>": X and Y are in 0–999, relative to the image width and height
const [x, y] = [...text.matchAll(/<loc_(\d+)>/g)].map(m => (Number(m[1]) + 0.5) / 1000);

Credits and license

• TinyClick by Samsung R&D Poland (paper), MIT license. Weights from the Krystianz/TinyClick mirror.
• Built on Florence-2 by Microsoft, MIT license.
• Export recipe adapted from Xenova's Florence-2 exporter.
• The ONNX conversion and quantized files are released under the same MIT license; see LICENSE.

This repository is independent: it is not affiliated with, endorsed by or maintained by Samsung.