FredZhang7 commited on
Commit
7fbb07e
1 Parent(s): 8752403

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +27 -9
README.md CHANGED
@@ -14,17 +14,36 @@ library_name: diffusers
14
  ---
15
  ## Paint Journey V2 is [Paint Journey V1](https://huggingface.co/FredZhang7/paint-journey-v1) fine-tuned on 768x768 oil paintings by Midjourney, Open Journey V2, and Disco Diffusion
16
 
17
- *Redoing the examples because I recently discovered the endless possibilities with Paint Journey V2.
18
- Paint Journey V2 crafts more stunning masterpieces with more descriptive positive and negative prompts,
19
- although the model can also generate beautiful landscapes with short prompts.*
20
-
21
  Begin the prompt with **((oil painting))** to add the oil paint effect. For digital and other painting styles, enter similar prompts as you would for Midjourney (with some tweaks), Stable Diffusion v1.5 (add more styles), Open Journey V2, or Disco Diffusion.
22
 
23
  Paint Journey V2's paintings are closer to human-drawn art than Open Journey V2.
24
  Compared to models like Dreamlike Diffusion 1.0, this model tends to generate 768x768 or higher resolution images with reduced noise levels.
25
  This model is also capable of generating stunning portraits at 768x1144 resolution without duplicated faces (with [Camenduru's WebUI](https://github.com/camenduru/stable-diffusion-webui)), a difficult task to models like DreamShaper 3.3.
26
 
27
- [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/AMLA-UBC/100-Exploring-the-World-of-Modern-Machine-Learning/blob/main/assets/PaintJourneyV2.ipynb)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  ## Training
30
  Instead of solely fine-tuning its Unet, Paint Journey V2 focuses on fine-tuning its text encoder with a diverse range of prompts.
@@ -37,9 +56,8 @@ To further improve the resolution and reduce noise in generated images, especial
37
  For example, a Automatic1111's WebUI user can add both files to the `./stable-diffusion-webui/models/Stable-diffusion` folder.
38
 
39
 
40
- ## Examples
41
-
42
- *Releasing soon*
43
 
44
 
45
  ## Automatic1111's WebUI
@@ -48,6 +66,7 @@ git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
48
  ```
49
  Download [paint_journey_v2.ckpt](./paint_journey_v2.ckpt) to the `./stable-diffusion-webui/models/Stable-diffusion` folder. Run `webui-user.bat`.
50
 
 
51
  ## Diffusers
52
  ```bash
53
  pip install --upgrade diffusers
@@ -59,7 +78,6 @@ import torch
59
  pipe = StableDiffusionPipeline.from_pretrained("FredZhang7/paint-journey-v2")
60
  pipe = pipe.to("cuda")
61
 
62
- # Use Prompt Hero for ideas of descriptive (positive) prompts
63
  prompt = "((oil painting)), a boat sailing, night sky, high resolution, uhd, 4 k wallpaper"
64
  image = pipe(prompt).images[0]
65
 
 
14
  ---
15
  ## Paint Journey V2 is [Paint Journey V1](https://huggingface.co/FredZhang7/paint-journey-v1) fine-tuned on 768x768 oil paintings by Midjourney, Open Journey V2, and Disco Diffusion
16
 
 
 
 
 
17
  Begin the prompt with **((oil painting))** to add the oil paint effect. For digital and other painting styles, enter similar prompts as you would for Midjourney (with some tweaks), Stable Diffusion v1.5 (add more styles), Open Journey V2, or Disco Diffusion.
18
 
19
  Paint Journey V2's paintings are closer to human-drawn art than Open Journey V2.
20
  Compared to models like Dreamlike Diffusion 1.0, this model tends to generate 768x768 or higher resolution images with reduced noise levels.
21
  This model is also capable of generating stunning portraits at 768x1144 resolution without duplicated faces (with [Camenduru's WebUI](https://github.com/camenduru/stable-diffusion-webui)), a difficult task to models like DreamShaper 3.3.
22
 
23
+ This model crafts more stunning masterpieces with more descriptive positive and negative prompts, though it can also generate beautiful landscapes with short prompts.
24
+
25
+ [![Open with Camenduru's WebUI in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/AMLA-UBC/100-Exploring-the-World-of-Modern-Machine-Learning/blob/main/assets/PaintJourneyV2.ipynb)
26
+
27
+ ## Examples
28
+
29
+ *All examples were generated using Camenduru's WebUI (see the Colab file)*
30
+
31
+ ![](./assets/characters.png)
32
+
33
+ *768x1136 portraits, generated with descriptive prompts and without face restoration, [generation parameters](https://huggingface.co/FredZhang7/paint-journey-v2/raw/main/assets/character_settings.txt)*
34
+
35
+ ![](./assets/nature.png)
36
+
37
+ *1280x768 (mostly) natural landscapes, generated with shorter prompts, [generation parameters](https://huggingface.co/FredZhang7/paint-journey-v2/raw/main/assets/nature_settings.txt)*
38
+
39
+ ![](./assets/outerspace.png)
40
+
41
+ *1152x768 outerspace landscapes, generated with descriptive prompts, [generation parameters](https://huggingface.co/FredZhang7/paint-journey-v2/raw/main/assets/outerspace_settings.txt)*
42
+
43
+ ![](./assets/lamborghini.png)
44
+
45
+ *1152x768 lamborghini, [generation parameters](https://huggingface.co/FredZhang7/paint-journey-v2/raw/main/assets/lamborghini_settings.txt)*
46
+
47
 
48
  ## Training
49
  Instead of solely fine-tuning its Unet, Paint Journey V2 focuses on fine-tuning its text encoder with a diverse range of prompts.
 
56
  For example, a Automatic1111's WebUI user can add both files to the `./stable-diffusion-webui/models/Stable-diffusion` folder.
57
 
58
 
59
+ ## Running out of prompts?
60
+ Useful resources: [Lexica.art](https://lexica.art/), [Fast GPT PromptGen](https://huggingface.co/FredZhang7/distilgpt2-stable-diffusion-v2), [Prompt Hero](https://prompthero.com/)
 
61
 
62
 
63
  ## Automatic1111's WebUI
 
66
  ```
67
  Download [paint_journey_v2.ckpt](./paint_journey_v2.ckpt) to the `./stable-diffusion-webui/models/Stable-diffusion` folder. Run `webui-user.bat`.
68
 
69
+
70
  ## Diffusers
71
  ```bash
72
  pip install --upgrade diffusers
 
78
  pipe = StableDiffusionPipeline.from_pretrained("FredZhang7/paint-journey-v2")
79
  pipe = pipe.to("cuda")
80
 
 
81
  prompt = "((oil painting)), a boat sailing, night sky, high resolution, uhd, 4 k wallpaper"
82
  image = pipe(prompt).images[0]
83