Ukiyo-e Diffusion

If you make something using these models, you're welcome to mention me @thegenerativegeneration

Named by dataset used. Current and best version is models/ukiyoe-all/v1/ema_0.9999_056000.pt

Current Plans

clean dataset
- remove borders
- remove some of the samples with text in them

Models

Ukiyo-e-all

v1

models/ukiyoe-all/v1/ema_0.9999_056000.pt

Model configuration is:

model_config = {
    'attention_resolutions': '32, 16, 8',
    'class_cond': False,
    'image_size': 256,
    'learn_sigma': True,
    'rescale_timesteps': True,
    'noise_schedule': 'linear',
    'num_channels': 128,
    'num_heads': 4,
    'num_res_blocks': 2,
    'resblock_updown': True,
    'use_checkpoint': True,
    'use_fp16': True,
    'use_scale_shift_norm': True,
}

Tips

Results closest to original training data are achieved by turning off the secondary model in Disco Diffusion.
Turning secondary model on can lead to very creative results
It is not necessary to specify Ukiyo-e as artstyle to get ukiyo-e-like images.

Examples

If you make something nice using these models, I would like to link your image.

Secondary Off

Secondary On

About

Trained from scratch on a ~170000 images corpus of ukiyo-e.org filtered by colorfulness >= 5.

(Deprecated) Ukiyo-e-few

models/ukiyoe-few/v1/ukiyoe_diffusion_256_022000.pt

Finetuned on 5224 images from Wikiart (1168) and ? ().

Model configuration is

model_config = {
    'attention_resolutions': '16',
    'class_cond': False,
    'diffusion_steps': 1000,
    'rescale_timesteps': True,
    'timestep_respacing': 'ddim100',
    'image_size': 256,
    'learn_sigma': True,
    'noise_schedule': 'linear',
    'num_channels': 128,
    'num_heads': 1,
    'num_res_blocks': 2,
    'use_checkpoint': True,
    'use_scale_shift_norm': False
}

Trained using a fork of guided-diffusion-sxela. Added random crop which did not lead to good results.