🐟 Evo-Nishikie-v1

🤗 Models | 📝 Blog | 🐦 Twitter

Evo-Nishikie-v1 is an experimental education-purpose Ukiyoe colorization model. The model is a ControlNet trained with Evo-Ukiyoe. The dataset used to train the model came from Ukiyoe images belonged to Ritsumeikan University, Art Research Center. The sample data is belonged to the Pre-Modern Japanese Text dataset, owned by National Institute of Japanese Literature and curated by ROIS-DS Center for Open Data in the Humanities.

Please refer to our blog for more details.

Usage

Use the code below to get started with the model.

Click to expand

Git clone this model card

git clone https://huggingface.co/SakanaAI/Evo-Nishikie-v1

Install git-lfs if you don't have it yet.

sudo apt install git-lfs
git lfs install

Create conda env

conda create -n evo-nishikie python=3.11
conda activate evo-nishikie

Install packages

cd Evo-Nishikie-v1
pip install -r requirements.txt

Run

import torch
from io import BytesIO
from PIL import Image
import requests
from evo_nishikie_v1 import load_evo_nishikie

#Get image from URL
#url = "https://huggingface.co/spaces/SakanaAI/Evo-Nishikie/resolve/main/sample2.jpg"
#original_image = Image.open(BytesIO(requests.get(url).content))

#Use local image
original_image = Image.open('test.jpg')

# Generate
device = "cuda"
pipe, processor = load_evo_nishikie(device)
images = pipe(
     prompt="着物を着た女性が、赤ん坊を抱え、もう一人の子どもが手押し車を引いています。背景には木があります。最高品質の輻の浮世絵。超詳細。",
     negative_prompt="暗い",
     image=processor(original_image),
     guidance_scale=7.0,
     controlnet_conditioning_scale=0.8,
     num_inference_steps=35,
     num_images_per_prompt=1,
     output_type="pil",
).images
images[0].save("out.png")

Model Details

Developed by: Sakana AI
Model type: Diffusion-based image-to-image generative model
Language(s): Japanese
Blog: https://sakana.ai/evo-ukiyoe

License

The Python script included in this repository is licensed under the Apache License 2.0. Please note that the license for the model/pipeline generated by this script is inherited from the source models. The sample images used in the code has CC BY SA 4.0 license and is belonged to the Pre-Modern Japanese Text dataset, owned by National Institute of Japanese Literature and curated by ROIS-DS Center for Open Data in the Humanities.

Uses

This model is provided for research and development purposes only and should be considered as an experimental prototype. It is not intended for commercial use or deployment in mission-critical environments. Use of this model is at the user's own risk, and its performance and outcomes are not guaranteed. Sakana AI shall not be liable for any direct, indirect, special, incidental, or consequential damages, or any loss arising from the use of this model, regardless of the results obtained. Users must fully understand the risks associated with the use of this model and use it at their own discretion.

Acknowledgement

Evo-Nishikie was trained based on Evo-Ukiyoe and Evo-Ukiyoe was trained based on Evo-SDXL-JP. We would like to thank the developers of Evo-SDXL-JP source models for their contributions and for making their work available.

Citation

@misc{Evo-Ukiyoe, 
url    = {[https://huggingface.co/SakanaAI/Evo-Nishikie-v1](https://huggingface.co/SakanaAI/Evo-Nishikie-v1)}, 
title  = {Evo-Nishikie}, 
author = {Clanuwat, Tarin and Shing, Makoto and Imajuku, Yuki and Kitamoto, Asanobu and Akama, Ryo}
}

SakanaAI
/

Evo-Nishikie-v1