Edit model card

Ukeiyo-style Diffusion

This is the fine-tuned Stable Diffusion model trained on traditional Japanese Ukeiyo-style images. Use the tokens ukeiyoddim style in your prompts for the effect. The model repo also contains a ckpt file , so that you can use the model with your own implementation of stable diffusion.

🧨 Diffusers

This model can be used just like any other Stable Diffusion model. For more information, please have a look at the Stable Diffusion.

You can also export the model to ONNX, MPS and/or FLAX/JAX.

#!pip install diffusers transformers scipy torch
from diffusers import StableDiffusionPipeline
import torch
model_id = "salmonhumorous/ukeiyo-style-diffusion"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")
prompt = "illustration of ukeiyoddim style landscape"
image = pipe(prompt).images[0]

Training procedure and data

The training for this model was done using a RTX 3090. The training was completed in 28 minutes for a total of 2000 steps. A total of 33 instance images (Images of the style I was aiming for) and 1k Regularization images was used. Regularization images dataset used by ProGamerGov.

Training notebook used by Shivam Shrirao.

Training hyperparameters

The following hyperparameters were used during training:

  • number of steps : 2000
  • learning_rate: 1e-6
  • train_batch_size: 1
  • scheduler_type: DDIM
  • number of instance images : 33
  • number of regularization images : 1000
  • lr_scheduler : constant
  • gradient_checkpointing


Below are the sample results for different training steps : img

Sample images by model trained for 2000 steps :

prompt = "landscape" img prompt = "ukeiyoddim style landscape" img prompt = " illustration of ukeiyoddim style landscape" img



Many thanks to nitrosocke, for inspiration and for the guide. Also thanks, to all the amazing people making stable diffusion easily accessible for everyone.

Downloads last month

Dataset used to train salmonhumorous/ukeiyo-style-diffusion