wuxiaojun commited on
Commit
86b3d2e
1 Parent(s): b3b5101

init commit

Browse files
Files changed (3) hide show
  1. README.md +11 -15
  2. result_examples/boy.png +3 -0
  3. result_examples/girl.png +3 -0
README.md CHANGED
@@ -53,22 +53,15 @@ The first open source Chinese Stable diffusion Anime model, which was trained on
53
 
54
  ## 模型信息 Model Information
55
 
56
- 我们将[Noah-Wukong](https://wukong-dataset.github.io/wukong-dataset/)数据集(100M)和[Zero](https://zero.so.com/)数据集(23M)用作预训练的数据集,先用[IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese](https://huggingface.co/IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese)对这两个数据集的图文对相似性进行打分,取CLIP Score大于0.2的图文对作为我们的训练集。 我们使用[IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese](https://huggingface.co/IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese)作为初始化的text encoder,冻住[stable-diffusion-v1-4](https://huggingface.co/CompVis/stable-diffusion-v1-4)([论文](https://arxiv.org/abs/2112.10752))模型的其他部分,只训练text encoder,以便保留原始模型的生成能力且实现中文概念的对齐。该模型目前在0.2亿图文对上训练了一个epoch。 我们在 32 x A100 训练了大约100小时。该版本只是一个初步的版本,我们将持续优化并开源后续模型,欢迎交流。
57
 
58
- We use [Noah-Wukong](https://wukong-dataset.github.io/wukong-dataset/)(100M) [Zero](https://zero.so.com/)(23M) as our dataset, and take the image and text pairs with CLIP Score (based on [IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese](https://huggingface.co/IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese)) greater than 0.2 as our Training set. We use [IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese](https://huggingface.co/IDEA-CCNL/Taiyi-CLIP-RoBERTa-102M-ViT-L-Chinese) as our init text encoder. To keep the powerful generative capability of stable diffusion and align Chinese concepts with the images, We only train the text encoder and freeze other part of the [stable-diffusion-v1-4](https://huggingface.co/CompVis/stable-diffusion-v1-4)([paper](https://arxiv.org/abs/2112.10752)) model. It takes 100 hours to train this model based on 32 x A100. This model is a preliminary version and we will update this model continuously and open sourse. Welcome to exchange!
59
 
60
  ### Result
61
- Basic Prompt
62
 
63
- | 铁马冰河入梦来,3D绘画。 | 飞流直下三千尺,油画。 | 女孩背影,日落,唯美插画。 |
64
- | ---- | ---- | ---- |
65
- | ![](result_examples/tiema.png) | ![](result_examples/feiliu.png) | ![](result_examples/nvhai.jpg) |
66
-
67
- Advanced Prompt
68
-
69
- | 铁马冰河入梦来,概念画,科幻,玄幻,3D | 中国海边城市,科幻,未来感,唯美,插画。 | 那人却在灯火阑珊处,色彩艳丽,古风,资深插画师作品,桌面高清壁纸。 |
70
- | ---- | ---- | ---- |
71
- | ![](result_examples/tiema2.jpg) | ![](result_examples/chengshi.jpg) | ![](result_examples/naren.jpg) |
72
 
73
 
74
  ## 使用 Usage
@@ -102,6 +95,12 @@ prompt = '1个女孩,美丽,可爱'
102
  image = pipe(prompt, guidance_scale=7.5).images[0]
103
  image.save("1个女孩.png")
104
  ```
 
 
 
 
 
 
105
 
106
  ### 使用手册 Handbook for Taiyi
107
 
@@ -111,9 +110,6 @@ https://github.com/IDEA-CCNL/Fengshenbang-LM/blob/main/fengshen/examples/stable_
111
 
112
  https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen/examples/finetune_taiyi_stable_diffusion
113
 
114
- ### webui配置 Configure webui
115
-
116
- https://github.com/IDEA-CCNL/stable-diffusion-webui/blob/master/README.md
117
 
118
  ### DreamBooth
119
 
53
 
54
  ## 模型信息 Model Information
55
 
56
+ 我们将两份动漫数据集(100万低质量数据和1万高质量数据),基于[IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-v0.1](https://huggingface.co/IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-v0.1) 模型进行了两阶段的微调训练,计算开销是4 x A100 训练了大约100小时。该版本只是一个初步的版本,我们将持续优化并开源后续模型,欢迎交流。
57
 
58
+ We use two anime dataset(1 million low-quality data and 10k high-qualty data) for two-staged training the chinese anime model based our pretrained model[IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-v0.1](https://huggingface.co/IDEA-CCNL/Taiyi-Stable-Diffusion-1B-Chinese-v0.1. It takes 100 hours to train this model based on 4 x A100. This model is a preliminary version and we will update this model continuously and open sourse. Welcome to exchange!
59
 
60
  ### Result
 
61
 
62
+ | 1个女孩,绿色头发,毛衣,看向阅图者,上半身,帽子,户外,下雪,高领毛衣 | 1个男生,帅气,微笑,看着阅图者,简单背景,白皙皮肤,衬衫 |
63
+ | ---- | ---- |
64
+ | ![](result_examples/girl.png) | ![](result_examples/boy.png) |
 
 
 
 
 
 
65
 
66
 
67
  ## 使用 Usage
95
  image = pipe(prompt, guidance_scale=7.5).images[0]
96
  image.save("1个女孩.png")
97
  ```
98
+ ### webui配置 Configure webui
99
+ 非常推荐使用webui的方式使用本模型,webui提供了可视化的界面加上一些高级修图功能。
100
+
101
+ It is highly recommended to use this model in a webui way. webui provides a visual interface plus some advanced retouching features.
102
+
103
+ https://github.com/IDEA-CCNL/stable-diffusion-webui/blob/master/README.md
104
 
105
  ### 使用手册 Handbook for Taiyi
106
 
110
 
111
  https://github.com/IDEA-CCNL/Fengshenbang-LM/tree/main/fengshen/examples/finetune_taiyi_stable_diffusion
112
 
 
 
 
113
 
114
  ### DreamBooth
115
 
result_examples/boy.png ADDED

Git LFS Details

  • SHA256: dc0d345b20453a5f835c78ac724f511e415f3746b5d25b08e360bf940c67222c
  • Pointer size: 131 Bytes
  • Size of remote file: 364 kB
result_examples/girl.png ADDED

Git LFS Details

  • SHA256: c70f6ac936cedc81fb49fde3d96dbde954f26c6fbb6279652900fc303e77cf7f
  • Pointer size: 131 Bytes
  • Size of remote file: 369 kB