TexelDiff-16-62M

Text-to-texture diffusion for 16×16 Minecraft-style block and item textures.

A small (~62M parameter) pixel-space diffusion model that generates 16×16 RGBA textures from a text prompt. Not affiliated with or endorsed by Mojang or Microsoft.

• Architecture: convolutional UNet with self- and cross-attention, conditioned on frozen CLIP text features (openai/clip-vit-base-patch32) and on texture resolution. Trained with DDPM (cosine schedule, noise prediction), sampled with DDIM and classifier-free guidance.
• Output: exactly 16×16 pixels with an alpha channel.

Usage

pip install torch transformers safetensors huggingface_hub pillow numpy

Download model.py, model.safetensors and config.json from this repo, then:

from model import TextureGenerator

gen = TextureGenerator("ArshPathan/TexelDiff-16-62M")        # or a local folder
imgs = gen("minecraft item texture: gold ingot", res=16, n=8, guidance=4.0)
imgs[0].resize((256, 256), 0).save("ingot.png")           # 0 = nearest neighbour for crisp pixels
blocks = gen("minecraft block texture: stone", res=16, n=8, opaque=True)

Prompts

Prompts work best in the training format: minecraft block texture: <name> or minecraft item texture: <name> (e.g. iron sword, oak planks, redstone ore). Free-form prompts that look nothing like the training captions mostly produce a blend of the nearest known textures.

Tips

• guidance 3–5 and steps 50 are good defaults. Lower guidance gives more variety.
• Use opaque=True for full blocks; leave it off for items and sprites.
• Many ore textures in the training data are overlays (gems on a transparent background). Generate the overlay without opaque and composite it over a generated base block (stone, deepslate, ...).
• Reducing the palette (Image.quantize) gives a cleaner pixel-art look.

Samples

Limitations

• 16×16 only; other resolutions are not supported by these weights.
• Little text understanding beyond the training vocabulary (CLIP features are used, but the UNet was trained from scratch on a narrow caption style).
• Common items have many near-identical training variants, so outputs for those can look alike, and some outputs may closely resemble training textures.
• Style reflects mod textures, which differ from the vanilla look.

Training data

Trained on a subset of NathMen12/16xModdedMinecraft-TextToImage (textures from Minecraft mods, each with its own license). Only rows with permissive licenses (MIT, Apache, BSD, CC0, CC-BY and similar) were used; "All Rights Reserved", non-commercial and GPL rows were excluded. Up to 50,000 textures were kept after removing duplicates, near-empty and flat-color images, and capping near-identical captions.

TODO before publishing: credit the dataset and its sources, state your final license choice (CC-BY and similar source licenses require attribution), and confirm the dataset's access conditions allow releasing a derived model.

Training

40 epochs (~31k steps), AdamW, lr 2e-4 with warm-up and cosine decay, EMA 0.999, mixed precision, 10% text dropout for classifier-free guidance, 1× T4 on Kaggle (about 9.5 hours). Released weights are the EMA weights.