Hero0963/threadgrid-qwen35-4b-p4c-gguf

threadgrid-qwen35-4b-p4c (GGUF)

Qwen3.5-4B fine-tuned to read a screenshot of a grid path puzzle (a square grid with numbered waypoints and walls between cells) and reply with the board as JSON. Exported to GGUF f16 for Ollama.

It is the vision half of the side project thread-the-grid: the model reads the board, and an exact solver draws the route.

Inspired by the Zip puzzle games found online. The model was trained only on images drawn by the project's own renderer; no screenshots from any game were used for training.

Files

File	Bytes	SHA-256
threadgrid-qwen35-4b-p4c-text-f16.gguf (language model)	8,424,393,344	28ec51a414244f306fdd6078325789e94da20968308695e49f94af9db3cdb773
threadgrid-qwen35-4b-p4c-mmproj-f16.gguf (vision projector)	672,423,040	4c2081c33cf7258f9cbc31ed06d522b35080c2e6c1dd6ceffc0b07d0b74a5e18
adapter/adapter_model.safetensors (the LoRA, before merging)	155,126,928	92331a9eb8dbec587b09abf7c197864a496fd420a6a089e7ebeea2ca531ca889
adapter/adapter_config.json	1,659
Modelfile		two FROM lines, nothing else
LICENSE		Apache-2.0, copied from the base model

You need both GGUF files: the first is the language model, the second is the vision encoder. The adapter is only for people who want to merge or continue training themselves (PEFT 0.19.1, r=16, alpha=16, trained against unsloth/Qwen3.5-4B, a copy of the base model).

Use it with Ollama

Validated on Ollama 0.32.13 with an NVIDIA GPU (16 GB).

hf download Hero0963/threadgrid-qwen35-4b-p4c-gguf --local-dir threadgrid-qwen35-4b-p4c-gguf
cd threadgrid-qwen35-4b-p4c-gguf
ollama create threadgrid-qwen35-4b-p4c:f16 -f Modelfile
ollama show threadgrid-qwen35-4b-p4c:f16      # capabilities should list "vision"

Keep the Modelfile exactly as it is: it has no TEMPLATE line, and that is the setup that was validated. Pulling with ollama run hf.co/... directly is not equivalent, because it chooses a template of its own.

Send the image with this exact instruction, word for word (it still says "Zip" because that is the text the model was trained with) (prompt_variants.py):

Read this Zip puzzle screenshot and reply with ONLY a JSON object.
"layout" is a 2D array of two-character strings: "  " for an empty cell, "xx" for a blocked cell, and a zero-padded number such as "01" for a waypoint.
"walls" is a list of {"cell1": [row, col], "cell2": [row, col]} objects, one per thick black bar drawn on a grid line between two neighbouring cells. Report every wall you can see and do not invent any.

The project's app sends it to Ollama's OpenAI-compatible endpoint, /v1/chat/completions, with temperature 0, seed 42 and reasoning turned off (backends.py); the worked example below shows the exact request.

Worked example

One board, sent through the project's own reading path on 2026-09-19. It is a harder case than the training data: the project's editor draws walls in red, and every training image had black ones. One image shows that the model can do this, not that it always will.

Input: a 615 × 616 PNG, sent as it is (no resizing).

Request, as captured on the wire (the image's base64 is shortened here):

POST /v1/chat/completions
{
  "model": "threadgrid-qwen35-4b-p4c:f16",
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "<the instruction above, word for word>"},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgo... (38,038 chars)"}}
  ]}],
  "temperature": 0.0,
  "seed": 42,
  "reasoning_effort": "none",
  "stream": false
}

Reply, verbatim (6.7 s with the model already loaded):

{
  "layout": [
    ["  ", "02", "  ", "  ", "  ", "07"],
    ["  ", "01", "  ", "  ", "  ", "  "],
    ["  ", "  ", "  ", "05", "06", "  "],
    ["  ", "  ", "  ", "  ", "  ", "  "],
    ["  ", "03", "04", "  ", "  ", "  "],
    ["10", "  ", "  ", "  ", "09", "08"]
  ],
  "walls": [
    {"cell1": [0, 3], "cell2": [0, 4]},
    {"cell1": [2, 2], "cell2": [2, 3]},
    {"cell1": [2, 4], "cell2": [2, 5]},
    {"cell1": [3, 0], "cell2": [3, 1]},
    {"cell1": [3, 2], "cell2": [4, 2]}
  ]
}

Every waypoint and all five walls match the picture. An exact solver (CP-SAT) then finds the route through all 36 cells, and an independent checker confirms that it solves the board. The green numbers are the order of the steps:

The script that produced this, and its full output, are in the project repository under ai-collab/reports/artifacts/vlm-walkthrough/.

Run the whole app

The easiest way to run the whole thing, solver and UI included, is the project itself: python start.py from the repository, with these files placed as its model-weights guide describes.

How it was trained

Data	8,000 synthetic 6×6 boards drawn by the project's renderer: 7,800 for training, the last 200 held out. 0 to 12 walls; light theme, or dark 25% of the time; cells of 72 to 132 px; 30% slightly rotated (±2°) and 35% JPEG-compressed
Method	bf16 LoRA (r=16, alpha=16) on the language and vision blocks, with Unsloth. 1 epoch, 975 steps, about 1.6 hours on one Colab L4
Export	LoRA merged into the base, converted to GGUF f16 with llama.cpp, imported into Ollama

Evaluation

Test	Result
200 held-out synthetic boards (2026-08-22)	200/200 exactly right: layout, every waypoint and every wall
Re-check at the project wrap-up (2026-09-19): 6 newly rendered boards (2 to 12 walls) and 4 held-out boards, served from Ollama	10/10
6 real screenshots (2026-08-29, not a target of this model)	5/6 exactly right; wall F1 0.972 (the base model scored 2/6 and 0.438)

Limitations

• It was trained on 6×6 boards from one renderer. It has learned that renderer's pictures, not screenshots in general: six real screenshots are far too few to claim more.
• The held-out score is already perfect, so it can no longer tell a better model from a worse one. A harder test set would be needed before changing anything.
• Always check the answer. The project's API reports solvable: false when the board it read has no solution, which means the reading was wrong.
• f16 needs about 9.1 GB of disk and a GPU with enough memory; it was not tested on CPU.

License

Apache-2.0, the same as the base model. This is a modified version of Qwen3.5-4B: a LoRA fine-tune merged into the weights and converted to GGUF. The base model's license text is in LICENSE.