|
--- |
|
language: |
|
- en |
|
thumbnail: "" |
|
tags: |
|
- Asian |
|
- China |
|
- Character |
|
- stable-diffusion |
|
- aiart |
|
license: creativeml-openrail-m |
|
--- |
|
|
|
<center><img src="https://huggingface.co/DGSpitzer/Guan-Yu-Diffusion/resolve/main/img/0.jpg" width="768" height="768"/></center> |
|
|
|
![visitors](https://visitor-badge.glitch.me/badge?page_id=Guan_Yu_Diffusion) |
|
|
|
# Guan Yu Diffusion |
|
|
|
There is few model that focus on elements of Asian culture and heritage, so I manage to make my own one! |
|
Hope you like it! XD |
|
|
|
Guan Yu Diffusion is an AI model that generates traditional oriental hero Guan Yu, from Three Kindoms Era in China during 220 to 280 AD! |
|
|
|
As an ancient hero "Saint of War" and the Door God that ward off evil spirits, Guan Yu plays a big role in Asian culture, by chance you can find his statues and mini temples in a lot of Asian restaurants and markets all over the world. |
|
|
|
It also comes with an additional support prompt to create Wu Xia - traditional Chinese martial art characters from movie and TV shows. |
|
|
|
Based of Stable Diffusion V1.5, training in Fast Dreambooth |
|
|
|
**For research purposes only!** This is an experiment I did to generate consistent character with clothing using AI - I used a series of photo of an **1/6 Guan Yu action figure model from Inflames Toy**,with still photos from **original Three Kingdom TV Show in 1994** as the dataset, therefore the model probably gonna overfit quite a lot by using only the default prompt keyword! (but you can then adding more prompts to reduce this issue) |
|
|
|
by [DGSpitzer](https://www.youtube.com/channel/UCzzsYBF4qwtMwJaPJZ5SuPg) |
|
|
|
分享一个最近自己做的小实验,通过finetune实验训练了一个具有较高服装一致性的关羽AI生成模型Guan Yu Diffusion |
|
|
|
原版的Stable Diffusion由于数据集的原因,对中国文化相关的人物生成效果比较薄弱,所以我试着训练了关羽人物的模型进行补充! |
|
|
|
基于Diffusion模型的特性,于是还可以生成出各种经典的关羽形象,也可以与现代其他的英雄角色进行融合,实现时空穿越、东汉末年的自拍等等很有意思的效果。 |
|
|
|
为了能够保持AI生成人物服装统一,这次我采用的方法是通过导入同一个手办不同角度的照片,可以保证较高的服装连贯性。 |
|
|
|
训练用的数据集大部分来自INFLAMES TOYS 1/6神勇关云长兵人模型。因此使用初始词条生成的角色图片会非常接近手办本身(该模型请勿商用,仅作学习研究目的使用),可以通过加入其他的prompt词条,后置主词条的方式来弱化该模型过拟合的问题。 |
|
|
|
|
|
### 🧨 Diffusers |
|
|
|
This repo contains both .ckpt and Diffuser model files. It's compatible to be used as any Stable Diffusion model, using standard [Stable Diffusion Pipelines](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion). |
|
|
|
You can convert this model to [ONNX](https://huggingface.co/docs/diffusers/optimization/onnx), [MPS](https://huggingface.co/docs/diffusers/optimization/mps) and/or [FLAX/JAX](https://huggingface.co/blog/stable_diffusion_jax). |
|
|
|
```python example for loading the Diffuser |
|
#!pip install diffusers transformers scipy torch |
|
from diffusers import StableDiffusionPipeline |
|
import torch |
|
|
|
model_id = "DGSpitzer/Guan-Yu-Diffusion" |
|
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16) |
|
pipe = pipe.to("cuda") |
|
|
|
prompt = "Portrait of guanyu walking in ancient battlefield, close up shot" |
|
image = pipe(prompt).images[0] |
|
|
|
image.save("./guanyu.png") |
|
``` |
|
|
|
# Online Demo |
|
|
|
You can try the Online Web UI demo build with [Gradio](https://github.com/gradio-app/gradio), or use Colab Notebook at here: |
|
|
|
*My Online Space Demo* |
|
[![Open In Spaces](https://camo.githubusercontent.com/00380c35e60d6b04be65d3d94a58332be5cc93779f630bcdfc18ab9a3a7d3388/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f25463025394625413425393725323048756767696e67253230466163652d5370616365732d626c7565)](https://huggingface.co/spaces/DGSpitzer/DGS-Diffusion-Space) |
|
|
|
|
|
*Buy me a coffee if you like this project ;P ♥* |
|
[![Buy me a coffee](https://badgen.net/badge/icon/Buy%20Me%20A%20Coffee?icon=buymeacoffee&label)](https://www.buymeacoffee.com/dgspitzer) |
|
|
|
<center><img src="https://huggingface.co/DGSpitzer/Guan-Yu-Diffusion/resolve/main/img/2.jpg" width="768" height="600"/></center> |
|
|
|
<center><img src="https://huggingface.co/DGSpitzer/Guan-Yu-Diffusion/resolve/main/img/1.jpg" width="768" height="768"/></center> |
|
|
|
# **👇Model👇** |
|
|
|
AI Model Weights available at huggingface: https://huggingface.co/DGSpitzer/Guan-Yu-Diffusion/ |
|
|
|
# Usage |
|
|
|
After model loaded, use keyword **guanyu** in your prompt. |
|
|
|
For sampler, use **Euler A** for the best result (**DDIM** kinda works too), CFG Scale 7, steps 20 should be fine |
|
|
|
**Example 1:** |
|
|
|
``` |
|
Portrait of guanyu walking in ancient battlefield, close up shot |
|
``` |
|
|
|
You can merge the Guan Yu's outfit with other famous characters: |
|
|
|
**Example 2:** |
|
|
|
``` |
|
batman, portrait of fancy superhero guanyu, golden spiderman, mech, robot, high tech, shining core, intricate details, 4k |
|
``` |
|
|
|
**Example 3 (Additional keywork Wuxia):** |
|
|
|
There is another hidden keyword you can use in this model **wuxia**, to generate traditional Chinese martial art character: |
|
|
|
``` |
|
a gorgeous wuxia girl standing in the palace |
|
``` |
|
|
|
--- |
|
|
|
**NOTE: usage of this model implies accpetance of stable diffusion's [CreativeML Open RAIL-M license](LICENSE)** |
|
|
|
--- |
|
|