|
# Cool Japan Diffusion 2.1.1 Beta Model Card |
|
|
|
![eyecatch](eyecatch.jpg) |
|
|
|
# Introduction |
|
Cool Japan Diffusion (for learning) is the latent diffusion model created from Stable Diffsion. |
|
Cool Japan Diffusion is suitable for genetrating Cool Japan images such as Anime, Manga, and Game. |
|
|
|
# Legal and ethical information |
|
We create this model legally. |
|
However, we think that this model have ethical problems. |
|
Therefore, we cannot use the model for commercially except for news reporting. |
|
|
|
TBA. |
|
|
|
# Usage |
|
You can try the model by our [Space](https://huggingface.co/spaces/alfredplpl/cool-japan-diffusion-2-1-0). |
|
I recommend to use the model by Web UI. |
|
You can download the model [here](https://huggingface.co/aipicasso/cool-japan-diffusion-2-1-0/resolve/main/v2-1-0.ckpt). |
|
|
|
## Model Details |
|
- **Developed by:** Robin Rombach, Patrick Esser, Alfred Increment |
|
- **Model type:** Diffusion-based text-to-image generation model |
|
- **Language(s):** English |
|
- **License:** [CreativeML Open RAIL++-M-NC License](https://huggingface.co/aipicasso/cool-japan-diffusion-2-1-1-beta/blob/main/MODEL-LICENSE) |
|
- **Model Description:** This is a model that can be used to generate and modify images based on text prompts. It is a [Latent Diffusion Model](https://arxiv.org/abs/2112.10752) that uses a fixed, pretrained text encoder ([OpenCLIP-ViT/H](https://github.com/mlfoundations/open_clip)). |
|
- **Resources for more information:** [GitHub Repository](https://github.com/Stability-AI/). |
|
- **Cite as:** |
|
|
|
@InProceedings{Rombach_2022_CVPR, |
|
author = {Rombach, Robin and Blattmann, Andreas and Lorenz, Dominik and Esser, Patrick and Ommer, Bj\"orn}, |
|
title = {High-Resolution Image Synthesis With Latent Diffusion Models}, |
|
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, |
|
month = {June}, |
|
year = {2022}, |
|
pages = {10684-10695} |
|
} |
|
|
|
## Examples |
|
|
|
- Web UI |
|
- Diffusers |
|
|
|
## Web UI |
|
Download the model [here](). |
|
Then, install [Web UI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) by AUTIMATIC1111. |
|
|
|
## Diffusers |
|
|
|
Using the [🤗's Diffusers library](https://github.com/huggingface/diffusers) to run Cool Japan Diffusion 2.1.1 Beta in a simple and efficient manner. |
|
|
|
```bash |
|
pip install --upgrade git+https://github.com/huggingface/diffusers.git transformers accelerate scipy |
|
``` |
|
|
|
Running the pipeline (if you don't swap the scheduler it will run with the default DDIM, in this example we are swapping it to EulerDiscreteScheduler): |
|
|
|
```python |
|
from diffusers import StableDiffusionPipeline, EulerDiscreteScheduler |
|
import torch |
|
|
|
model_id = "aipicasso/cool-japan-diffusion-2-1-1-beta" |
|
|
|
scheduler = EulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler") |
|
pipe = StableDiffusionPipeline.from_pretrained(model_id, scheduler=scheduler, torch_dtype=torch.float16) |
|
pipe = pipe.to("cuda") |
|
|
|
prompt = "anime, a portrait of a girl with black short hair and red eyes, kimono, full color illustration, official art, 4k, detailed" |
|
negative_prompt="low quality, bad face, bad anatomy, bad hand, lowres, jpeg artifacts, 2d, 3d, cg, text" |
|
image = pipe(prompt,negative_prompt=negative_prompt).images[0] |
|
|
|
image.save("girl.png") |
|
|
|
``` |
|
|
|
**Notes**: |
|
- Despite not being a dependency, we highly recommend you to install [xformers](https://github.com/facebookresearch/xformers) for memory efficient attention (better performance) |
|
- If you have low GPU RAM available, make sure to add a `pipe.enable_attention_slicing()` after sending it to `cuda` for less VRAM usage (to the cost of speed) |
|
|
|
|
|
*This model card was written by: Alfred Increment and is based on the [Stable Diffusion v2](https://huggingface.co/stabilityai/stable-diffusion-2/raw/main/README.md) |