Srinjoy99/tadpole-nano-258m

🐸 Tadpole Nano 258M

The baby tadpole: a very small model that reads diagrams. It is the little sibling of Tadpole 0.8B. It was trained on the same data for the same job: turning a picture of a flowchart, UML diagram, BPMN process or logic circuit into structured text that converts to JSON, YAML or XML.

Read this first. Nano is smaller and faster, but much less accurate on arrows. It reads boxes and their text almost as well as Tadpole 0.8B, but gets only about half the arrows right. Use it when size or speed matters more than getting every connection right. Otherwise, use Tadpole 0.8B.

	Tadpole 0.8B	Tadpole Nano 258M
Arrows correct (edge F1, 580 test images)	0.846	0.444
Boxes correct (node F1)	0.930	0.834
Valid output	99.3%	97.8%
Download (llama.cpp, smallest file)	734 MB	368 MB
CPU time per diagram (16 threads, 60 images)	7.2 s	4.8 s

• Code, docs and training details: github.com/GhoshSrinjoy/Diagram-OCR
• Files for llama.cpp: Srinjoy99/tadpole-nano-258m-GGUF

Example

A hand-drawn UML activity diagram from the CAS2UML test split (CC BY 4.0). Neither model saw this image in training.

Nano's answer:

diagram activity dir=TB
node n1 initial
node n2 action "Sign Contract"
node n3 fork
node n4 action "HR: Setup Benefits"
node n5 action "Legal: NDA Signature"
node n6 join
node n7 action "First Day Orientation"
node n8 final
edge n1 n2
edge n2 n3
edge n3 n4
edge n3 n5
edge n4 n6
edge n5 n6
edge n6 n7
edge n7 n8

This is almost right: it missed one of the three parallel branches ("IT: Provision Laptop"), for edge F1 0.89. Tadpole 0.8B gets this one exactly right. It's a typical Nano mistake: the text is read well, but part of the structure goes missing.

Results per source

Edge F1 on the same 580 test images. "Never trained" marks sources kept out of training for both models.

Source	Style	Tadpole 0.8B	Nano 258M	Nano node F1
FlowVQA	rendered flowcharts	0.988	0.683	0.984
CAS2UML	hand-drawn UML	0.958	0.672	0.936
DiDi	hand-drawn ink flowcharts	0.917	0.698	0.968
rakitha (never trained)	rendered flowcharts	0.921	0.639	0.906
Image-to-Mermaid v3	rendered, 6 diagram types	0.946	0.515	0.855
Flowchart-to-Mermaid	rendered flowcharts	0.899	0.446	0.911
Handwritten FC/FA (never trained)	hand-drawn scans	0.730	0.418	0.740
The Stack v2 PlantUML	real-world UML	0.790	0.410	0.757
LogicBench-1K	hand-drawn logic circuits	0.525	0.376	0.551
UML-in-the-Wild (never trained)	real-world UML	0.746	0.351	0.637
FlowGen	4 renderers, scanned copies	0.899	0.271	0.916
hdBPMN	hand-drawn BPMN photos	0.694	0.250	0.790
All 580 images		0.846	0.444	0.834

Use it with transformers

import torch
from PIL import Image
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "Srinjoy99/tadpole-nano-258m"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto")

image = Image.open("diagram.png").convert("RGB")
messages = [{"role": "user", "content": [{"type": "image"}, {"type": "text", "text": "Convert this diagram to DGM."}]}]
prompt = processor.apply_chat_template(messages, add_generation_prompt=True)
inputs = processor(text=[prompt], images=[[image]], return_tensors="pt").to(model.device)

output = model.generate(**inputs, max_new_tokens=1536, do_sample=False)
dgm = processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()
print(dgm)

• Images: pass them as one list per sample (images=[[image]]). The image size the model was trained with (512-px tiles, longest edge 1536 px) is stored in the processor.
• Boxes: use the prompt "Convert this diagram to DGM with boxes.".
• JSON, YAML or XML: convert the answer with the diagram_ir package from the GitHub repo, with python -m diagram_ir convert diagram.dgm --to json. The repo's python -m deploy.predict diagram.png --checkpoint Srinjoy99/tadpole-nano-258m --to json does it all in one step.

How it was made

Base model. Granite-Docling-258M (Apache-2.0), a small document-conversion model with a SigLIP2 vision encoder (86M parameters) and a language model of about 170M parameters.

Choosing it. It beat SmolVLM-256M in an 800-step pilot on the same data:

At step 800	Granite-Docling	SmolVLM-256M
Validation edge F1	0.193	0.150
Node F1	0.582	0.469
Valid output	91%	81%

Both ran at the same speed.

Image size. The model reads images as 512-px tiles. At its native 2048-px size it would barely be faster than Tadpole 0.8B on a CPU, because image reading dominates. So it was trained at 1536 px: at most 10 tiles, about the same detail Tadpole 0.8B sees.

Training. One supervised run on the same data as Tadpole 0.8B:

• every real training source plus the synthetic circuits, BPMN and PlantUML;
• 68,600 samples per epoch, 2 epochs (8,580 steps);
• learning rate 4e-5;
• about 9 hours on one RTX 3090.

There was no reinforcement learning. Validation edge F1 levelled off around 0.49 over the last quarter of training.

Why the arrows are weaker. Probably two reasons:

• the language model is 4× smaller than Tadpole 0.8B's;
• the image is read as separate tiles, so arrows that cross tile edges are harder to follow.

The training data, cleaning and evaluation are documented in DATASETS.md and TRAINING.md. Training data and attribution are listed on the Tadpole 0.8B card.

Limitations

• Arrows. About half are wrong. The worst cases are large rendered flowcharts (FlowGen, 0.27) and hand-drawn BPMN (0.25).
• Missing parts. Big diagrams can lose whole branches.
• Quantization. Q4 quantization breaks this small model: edge F1 drops from 0.42 to 0.27. Use the Q8_0 or bf16 GGUF.
• Inherited from Tadpole: answers are capped at 1,536 tokens, and charts, plots and tables are out of scope.

License

Apache-2.0, like the base model. The CC BY 4.0 training sets are credited on the Tadpole 0.8B card.