Pinhole-M

A 34.4M-parameter text-to-image model for pixel art, flat icons and stickers. Small enough for laptops and, next, the browser. Part of Darkroom, a project to build tiny image generators that run where you are.

Parameters	34.4M (plus CLIP ViT-B/32 text encoder, TAESD decoder)
Weights	69 MB (fp16)
Output	256x256 RGB
Speed, T4 GPU	225 ms per image (20 steps, guidance 4)
Speed, CPU (Kaggle)	3,661 ms per image
CLIP score, held-out prompts	30.36 (teacher PixArt-LCM: 32.64)

Samples

The same held-out prompts, never seen in training, from the 0.6B teacher and from each Pinhole size:

Run it in your browser

No install needed: open the live demo. The model runs on your device with ONNX Runtime Web (WebAssembly or WebGPU), so prompts and images never leave the browser.

Browser file	Size
onnx/dit.onnx (Pinhole-M)	138 MB
onnx/text_encoder.onnx (CLIP ViT-B/32, fp32)	253 MB
onnx/taesd_decoder.onnx	4.9 MB
Total first download (cached afterwards)	396 MB

Parity: the ONNX-only pipeline matches PyTorch to within a mean of 0.00/255 per pixel (same noise, 20 steps). The browser's tokenizer is checked against Python by the demo's Self-test button.

import * as ort from "https://cdn.jsdelivr.net/npm/onnxruntime-web@1.20.1/dist/ort.all.min.mjs";
import { AutoTokenizer } from "https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.8.1";
import { PinholeWeb } from "./pinhole-web.js";      // web/pinhole-web.js in this repo
const pinhole = await PinholeWeb.load("nuhmanpk/pinhole-m", { ort, AutoTokenizer });
const { rgba, width, height } = await pinhole.generate("sticker of a happy red fox", { steps: 20, cfg: 4, seed: 7 });
canvas.getContext("2d").putImageData(new ImageData(rgba, width, height), 0, 0);

Quick start

pip install torch diffusers transformers safetensors huggingface_hub

from huggingface_hub import snapshot_download
import sys
path = snapshot_download("nuhmanpk/pinhole-m"); sys.path.insert(0, path)
from pinhole import Pinhole

pipe = Pinhole.from_pretrained(path)            # GPU if available, else CPU
images = pipe("sticker of a happy red fox", n=4, steps=20, cfg=4.0, seed=0)
images[0].save("fox.png")

Prompting

Start with one of the three styles, then describe a simple subject: <style> of a <adjective> <colour> <subject>.

Style	Example
pixel art	pixel art of a retro blue rocket
flat icon	flat icon of a cozy green house
sticker	sticker of a smiling pink cupcake

Tips:

• Single, simple subjects work best: animals, food, objects, plants, vehicles and fantasy items.
• Adjust guidance with cfg: 3–6 is the useful range; higher means stronger prompt following and less variety.
• Adjust steps with steps: 12 is faster; 20 is the default.

The Pinhole family

Model	Params	Weights	CLIP score	T4 GPU	CPU
pinhole-s	13.1M	26 MB	30.28	187 ms	1,844 ms
pinhole-m	34.4M	69 MB	30.36	225 ms	3,661 ms
pinhole-l	70.8M	142 MB	30.57	258 ms	6,764 ms
PixArt-LCM teacher (reference)	600M + 4.7B text encoder	~11 GB	32.64

How it works

• Generator: a small diffusion transformer (adaLN-Zero blocks with cross-attention to text) predicts 4x32x32 latents with patch size 2 (256 tokens).
• Text: frozen CLIP ViT-B/32 text features (77 tokens plus the pooled embedding).
• Pixels: the TAESD decoder turns latents into 256x256 RGB.
• Training objective: rectified-flow matching (x_t = (1 - t) x0 + t * noise, predict noise - x0) with logit-normal timesteps and 10% caption dropout for classifier-free guidance.
• Weights: an EMA of the training weights is shipped.

Training

• Data: nuhmanpk/darkroom-pinhole-18k, 18,000 images made by PixArt-alpha/PixArt-LCM-XL-2-1024-MS (4 steps) for short captions in 3 styles.
• Schedule: 30,000 steps, global batch 128, AdamW (lr 0.0003), warmup plus cosine, fp16, EMA 0.999, horizontal-flip augmentation.
• Hardware: Kaggle 2x T4, 152 minutes. Final training loss 0.3350.

Limitations

• Low resolution: 256x256, best for icons, sprites and stickers, not photographs.
• Simple subjects: it learned single, centered subjects on plain backgrounds. Complex scenes, counting, text in images and people are not supported.
• Limited by the teacher and a small dataset. Expect some off-prompt or blobby results, especially from pinhole-s. The CLIP-score gap to the teacher is shown above.
• English captions only.

License

CreativeML Open RAIL++-M, inherited from the PixArt-LCM teacher, including its use-based restrictions (see LICENSE.md). pinhole.py is Apache-2.0.

Credits

Built by Nuhman PK. Teacher: PixArt-α / PixArt-LCM. Text encoder: OpenAI CLIP ViT-B/32. Decoder: TAESD by Ollin Boer Bohan. Training ideas from DiT and rectified flow.