Instructions to use Srinjoy99/tadpole-0.8b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Srinjoy99/tadpole-0.8b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Srinjoy99/tadpole-0.8b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Srinjoy99/tadpole-0.8b")
model = AutoModelForMultimodalLM.from_pretrained("Srinjoy99/tadpole-0.8b", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Srinjoy99/tadpole-0.8b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Srinjoy99/tadpole-0.8b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Srinjoy99/tadpole-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Srinjoy99/tadpole-0.8b

SGLang

How to use Srinjoy99/tadpole-0.8b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Srinjoy99/tadpole-0.8b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Srinjoy99/tadpole-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Srinjoy99/tadpole-0.8b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Srinjoy99/tadpole-0.8b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Srinjoy99/tadpole-0.8b with Docker Model Runner:
```
docker model run hf.co/Srinjoy99/tadpole-0.8b
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

🐸 Tadpole 0.8B

A tiny model that reads diagrams. Show Tadpole a flowchart, a UML diagram, a BPMN process or a logic circuit, whether it is hand-drawn on a whiteboard, scanned or exported from a tool. It hands back every box, every label and every arrow as structured text that converts losslessly to JSON, YAML or XML.

Why "Tadpole"? A tadpole is a dot with a tail, which is one node and one arrow: the smallest diagram there is.

Code, docs and training details: github.com/GhoshSrinjoy/Diagram-OCR
Files for llama.cpp (CPU or GPU, 0.73 GB): Srinjoy99/tadpole-0.8b-GGUF
Smaller sibling: Tadpole Nano 258M is half the download and about 1.5× faster on a CPU, but gets only about half the arrows right (edge F1 0.444 vs 0.846).

Example

A hand-drawn UML activity diagram from the CAS2UML test split (CC BY 4.0). The model never saw this image in training.

Tadpole's answer, exactly right:

diagram activity dir=TB
node n1 initial
node n2 action "Sign Contract"
node n3 fork
node n4 action "IT: Provision Laptop"
node n5 action "HR: Setup Benefits"
node n6 action "Legal: NDA Signature"
node n7 join
node n8 action "First Day Orientation"
node n9 final
edge n1 n2
edge n2 n3
edge n3 n4
edge n3 n5
edge n3 n6
edge n4 n7
edge n5 n7
edge n6 n7
edge n7 n8
edge n8 n9

Ask for boxes and every element also gets a position (0–1000 coordinates). Here they are drawn back onto a photo of a hand-drawn BPMN diagram (hdBPMN test split, CC BY 4.0):

Boxes predicted by Tadpole on a hand-drawn BPMN photo

What it reads

Flowcharts, UML activity, sequence, class and state diagrams, BPMN processes, ER diagrams, logic circuits (gates and wires), and generic node-link graphs such as mind maps and architecture diagrams.

The output names each element's kind, such as decision, fork, interface or nand, and its text. It also gives every arrow with its direction and label, and containers such as lanes, pools, packages and composite states. Boxes are optional.

Results

These are the test-set results on 580 images from 12 sources. "Never trained" marks sources kept out of training, so their style is completely new to the model.

Source	Style	Edge F1	Node F1
FlowVQA	rendered flowcharts	0.988	0.998
CAS2UML	hand-drawn UML	0.958	0.992
Image-to-Mermaid v3	rendered, 6 diagram types	0.946	0.994
rakitha (never trained)	rendered flowcharts	0.921	0.955
DiDi	hand-drawn ink flowcharts	0.917	0.968
FlowGen	4 renderers, scanned copies	0.899	0.981
Flowchart-to-Mermaid	rendered flowcharts	0.899	0.980
The Stack v2 PlantUML	real-world UML	0.790	0.842
UML-in-the-Wild (never trained)	real-world UML	0.746	0.711
Handwritten FC/FA (never trained)	hand-drawn scans	0.730	0.836
hdBPMN	hand-drawn BPMN photos	0.694	0.932
LogicBench-1K	hand-drawn logic circuits	0.525	0.867
All 580 images		0.846	0.930

Edge F1 counts arrows with the right endpoints and direction.
Node F1 counts the elements.
Valid output (it parses without errors) is 99.3%. 36% of answers are exactly right, character for character.

Use it with transformers

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "Srinjoy99/tadpole-0.8b"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

image = Image.open("diagram.png").convert("RGB")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "Convert this diagram to DGM."}]}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False)
inputs = processor(text=[prompt], images=[image], return_tensors="pt",
                   images_kwargs={"size": {"shortest_edge": 65536, "longest_edge": 1254400}}).to(model.device)

output = model.generate(**inputs, max_new_tokens=1536, do_sample=False)
dgm = processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(dgm)

Required settings:
- enable_thinking=False: the model was trained with an empty think block.
- Greedy decoding.
- The image size limits shown: 64 to 1,225 image tokens, as in training.
Boxes: use the prompt "Convert this diagram to DGM with boxes.".
Transparent PNGs: flatten them onto white first, so black lines stay visible.

JSON, YAML or XML

The answer is DGM, the compact line format of the Diagram IR. The diagram_ir package in the GitHub repo converts it losslessly:

python -m diagram_ir convert diagram.dgm --to json,yaml,xml -o exports/

Or, from Python:

from diagram_ir import dump, parse

diagram, problems = parse(dgm)  # lenient: a cut-off answer still keeps every complete line
print(dump(diagram, "json"))

The repo also has a one-line CLI (python -m deploy.predict diagram.png --to json) and a demo notebook. The format is defined in SPEC.md.

How it was trained

Tadpole is a full fine-tune of Qwen3.5-0.8B on one RTX 3090, in about 24 GPU-hours:

three supervised stages;
two rounds of reinforcement learning (DAPO-style GRPO) whose reward is graph F1;
weight merging.

The training data comes from 12 sources with about 137,000 image–target pairs, including 26,000 synthetic diagrams generated for the project. Every target comes from the source's own structured ground truth, never from OCR. The full story, including a metric bug the RL run exploited and how it was fixed, is in TRAINING.md. The datasets and cleaning are in DATASETS.md.

Training data:

Apache-2.0: FlowGen
MIT: Image-to-Mermaid v3, Flowchart-to-Mermaid, FlowVQA
CC BY 4.0: DiDi, hdBPMN, LogicBench-1K, CAS2UML
Permissively licensed files only: The Stack v2 PlantUML
Synthetic circuits, BPMN and PlantUML diagrams, generated for this project

Sets whose licenses do not allow training were used for testing only.

Limitations

Big circuits. Circuits with 11 or more gates are mostly wrong (edge F1 0.27). Small ones do much better (0.71 for up to 6 gates).
Hand-drawn BPMN. Arrows into collapsed pools are often missing.
Handwriting. On never-trained handwritten flowcharts, 59% of node texts are exact.
Very large diagrams. Answers are capped at 1,536 tokens.
Boxes. They are reliable for BPMN and circuits, the styles that had box annotations in training.
Out of scope. Charts, plots and tables.

License

Apache-2.0, like the base model. The CC BY 4.0 training sets are credited above.

Downloads last month: 25

Safetensors

Model size

0.9B params

Tensor type

BF16

Model tree for Srinjoy99/tadpole-0.8b

Base model

Qwen/Qwen3.5-0.8B-Base

Finetuned

Qwen/Qwen3.5-0.8B

Finetuned

(524)

this model

Quantizations

2 models

Srinjoy99
/

tadpole-0.8b