irotem98/Codex-Pets-Sprite-Sheet-Generator

Codex Pets Sprite Sheet Generator

Give the model one transparent character image and it creates a complete V2 sprite sheet with 11 animation rows. The included script handles image preparation, all 11 diffusion calls, row packing, final assembly, and binary alpha cleanup.

It is built around the V2 animation layout used by Codex Pets.

Created by irotem98 on Codex Pets, using the irotem98/codex-pets-sprite-sheets training dataset.

The example above was generated from only Jigglypuff's initial image.

Quick start

git clone https://huggingface.co/irotem98/Codex-Pets-Sprite-Sheet-Generator
cd Codex-Pets-Sprite-Sheet-Generator
pip install -r requirements.txt

python generate_sprite_sheet.py character.png --output full-sprite-sheet.png

That is the complete inference command. On first use, the script automatically downloads the original Boogu/Boogu-Image-0.1-Edit base model. A transparent PNG containing one centered character works best.

Generation uses a row batch size of 1 by default. If your GPU has additional memory, increase it with --row-batch-size 2, 4, or up to 11 to generate multiple animation rows together. The underlying model is 10B parameters, so a CUDA GPU with substantial VRAM is recommended.

Output layout

The output is a 1536 × 2288 RGBA PNG: eight columns, eleven rows, and 192 × 208 pixels per sprite cell.

Row	Animation	Frames
1	idle	6
2	run right	8
3	run left	8
4	wave	4
5	jump	5
6	failed	8
7	waiting	6
8	running	6
9	review	6
10	look directions 1–8	8
11	look directions 9–16	8

Unused cells at the end of shorter rows remain transparent.

How the model was trained

This release uses a two-stage fine-tuning pipeline.

1. Alpha-compatible RGBA VAE

The original Boogu VAE accepts and reconstructs RGB. It was expanded to RGBA while preserving the pretrained RGB path:

• the original RGB VAE weights were frozen
• a trainable one-channel alpha input adapter was added to the encoder
• a trainable one-channel alpha output head was added to the decoder
• RGB reconstruction, alpha reconstruction, foreground alpha, transparent RGB, and KL losses were weighted separately
• after training, the adapters were merged into a standard four-channel AutoencoderKL

The released VAE is checkpoint 2,000, trained on 3,015 sprite sheets from irotem98/codex-pets-sprite-sheets, represented as 27,691 training rows and 540 validation rows. It is stored under vae/ and can be loaded directly with Diffusers.

2. Boogu Image Edit sprite-sheet LoRA

The RGBA VAE and Qwen3-VL instruction encoder were frozen. A rank-8 LoRA was trained on the Boogu diffusion transformer's to_q, to_k, to_v, and to_out.0 attention projections.

For every animation row:

• input: a balanced 3×3 grid where every occupied cell repeats the first sprite from the sheet's first row
• target: the real animation frames packed into the same occupied cells
• instruction: an explicit prompt naming the animation, frame count, ordering, identity preservation, transparent background, and occupied-cell layout

The released LoRA is checkpoint 21,000 and is stored under lora/. The diffusion objective was flow matching. During training, ordinary 50-step diffusion samples were logged every 50 optimizer steps.

One-image generation pipeline

generate_sprite_sheet.py performs the full workflow:

1. Read the input as RGBA.
2. Remove only padding where alpha is zero.
3. Resize proportionally to fit one 192 × 208 cell without cropping visible pixels.
4. Repeat that same first sprite in the correct 3×3 occupied-cell pattern for each animation row.
5. Generate all 11 row grids with row-specific prompts and 50 diffusion steps.
6. Unpack the row grids into the final eight-column V2 sprite sheet.
7. Clean the alpha channel, keep only the main connected opaque character region in each sprite cell, set every pixel in its two-pixel outer edge band to exact black, and save the finished RGBA PNG.

Included files

• vae/ — deployable four-channel alpha-compatible VAE
• lora/ — rank-8 Boogu Image Edit sprite-sheet adapter
• generate_sprite_sheet.py — one-image to full-sheet command
• sprite_sheet_config.json — dimensions, row names, frame counts, and inference defaults
• boogu/ — the Apache-2.0 Boogu inference modules needed by the script

Limitations

• Best results come from a single pixel-art character on a transparent background.
• RGB/JPEG input is accepted, but its entire rectangular background is treated as visible; transparent PNG is strongly recommended.
• Motion quality, facing consistency, and identity preservation can vary between rows.
• The layout is fixed to the eleven-row V2 format above.
• This repository contains the fine-tuned VAE and LoRA, but still downloads and requires the original 10B Boogu base model.

Base model and license

This is a fine-tune of Boogu/Boogu-Image-0.1-Edit. The base model and included Boogu inference code are Apache-2.0 licensed. Review the upstream model card for its responsible-use guidance and broader limitations.