Gepo MC Skin Pixel EDM

English text → native 64×64 RGBA Minecraft skin

Pixel-space EDM · UV-aware conditioning · Alex/Steve geometry · no VAE

Model summary

Gepo MC Skin Pixel EDM is a research model that maps an English description to a native 64×64 RGBA Minecraft Java Edition skin atlas. It performs diffusion directly in pixel space and does not use a VAE. A frozen CLIP ViT-L/14 text encoder conditions a custom U-Net through cross-attention, while fixed spatial channels encode the Minecraft UV layout, base/overlay regions, body parts, and Alex/Steve geometry.

This is a custom pipeline. The repository includes Diffusers-format U-Net weights plus the project inference code; it is not loadable as a stock Stable Diffusion pipeline.

How it works

English text is encoded by a frozen CLIP ViT-L/14 and injected through cross-attention. Ten fixed spatial channels encode UV validity, base and overlay layers, body-part identity, and arm geometry. The 217.8M-parameter U-Net denoises RGBA pixels directly; a 40-step Heun sampler and deterministic UV/alpha constraints produce the final game-ready PNG while preserving the raw output for research evaluation.

Model details

Output: 64×64 RGBA PNG atlas.
Denoiser: Diffusers UNet2DConditionModel.
Trainable U-Net parameters: 217,760,644.
Text encoder: frozen openai/clip-vit-large-patch14 at revision 32bd64288804d66eefd0ccbe215aa642df71cc41.
Text length: at most 77 CLIP tokens.
Structural input: ten fixed UV conditioning channels.
Geometry label: Alex or Steve.
Objective: weighted EDM pixel-space denoising.
Selected checkpoint: EMA at step 100,000.
Selection metric: validation EMA loss 0.04206468362826854.
Selected weight SHA-256: 7674a690e80cbbc214e18744918db2062b7914fb9a906d35d0517a8c81095436.

Training data

The frozen source collection contains 105,000 text/image pairs. A deterministic validity pass accepted 100,440 images and rejected 4,560 images with transparent pixels in required base-layer UV regions. Exact SHA-256 grouping produced:

97,428 training rows;
1,506 validation rows;
1,506 held-out test rows.

All targets remain native 64×64 PNG files. No resize, crop, flip, color jitter, or lossy image transformation is used. Captions are English and explicitly include the Alex/Steve model type.

Data access notice: the training dataset is not publicly distributed. The model weights and bundled software are released under the MIT License; this does not grant access to, or a license for, the private training dataset. Source-level provenance, license, and takedown records are not distributed in this repository. Users are responsible for ensuring that their own data, prompts, outputs, and uses comply with applicable rights and terms.

Intended uses

Research on text-conditioned structured pixel-art generation.
Research on RGBA diffusion, overlay prediction, and fixed UV conditioning.
Generation of native 64×64 Minecraft Java Edition skin atlases.
Initialization for future research fine-tuning on appropriately licensed data.

Out-of-scope uses

Natural-image generation.
Bedrock geometry, capes, or nonstandard high-definition skin formats.
Impersonation, deceptive identity use, or ownership claims over third-party skins, characters, logos, or trademarks.

Inference

Install the pinned dependencies, then run the bundled sampler:

python -m pip install -r requirements.txt
python sample.py \
  --model-dir . \
  --prompt "Minecraft skin, alex model, forest ranger, green coat, brown boots" \
  --model-type alex \
  --seed 42 \
  --output forest-ranger.png

The sampler saves both the raw output and the constrained game-ready PNG. Generation defaults to 40 Heun steps with classifier-free guidance scale 5.0.

Evaluation

Step 100,000 achieved the lowest recorded fixed-split EMA validation loss: 0.04206468362826854. In the 16-prompt qualitative evaluation, constrained outputs are valid 64×64 RGBA atlases. Before constraints, mean base-nonopaque rate was 0.047078, so the hard UV/alpha constraint remains a required part of the operational generator.

These checks establish file and geometry validity, not comprehensive perceptual quality or prompt adherence. A paper claim should also include raw-output metrics, fixed-seed 3D render evaluation, diversity, blinded human preference, confidence intervals, required ablations, and a current latent-diffusion baseline.

The qualitative evaluation uses 16 captions written in the training dataset's tag style, eight Alex and eight Steve prompts, and fixed seed 20260830. It includes one Spider-Man prompt and contains no exact caption duplicate from the frozen splits. Generated metadata.jsonl follows the training schema. The evaluation prompts were defined after training completed and therefore were not used to select the checkpoint. The complete prompt list, constrained skins, raw outputs, render gallery, generation records, and structural metrics are stored under evaluation/.

Limitations

English only.
CLIP truncates inputs beyond 77 tokens.
Rare concepts and long compositional prompts may be weak.
A valid UV atlas can still contain weak faces, seams, front/back confusion, excessive color micro-variation, or poor prompt alignment.
The hard output constraint guarantees format validity but cannot create missing semantic detail.
The dataset contains more Alex than Steve skins.
Character names or branded visual concepts in a prompt can raise independent intellectual-property concerns.

Fine-tuning

Use the training repository's --init-from-unet option to initialize a fresh optimizer, schedule, and EMA from ema_unet/. Optimizer, scheduler, RNG, and intermediate checkpoint state from the original run are not publicly released.

Reproducibility

The release contains the training YAML, runtime manifest, selected-weight checksum, tokenizer, inference code, and fixed-seed qualitative evaluation. The private dataset and optimizer-level checkpoint state are not included. Random seed: 20260824; effective batch size: 64; precision: BF16.

License and attribution

The bundled source code and model weights are released under the MIT License. The private training dataset is not included and receives no license through the model release. The MIT License does not grant rights to Minecraft or Mojang marks, third-party skins, characters, logos, or other content.

This independent research project is not affiliated with or endorsed by Mojang Studios or Microsoft.

Downloads last month: -