YiYiXu commited on
Commit
1df06eb
1 Parent(s): 48ea15d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +21 -12
README.md CHANGED
@@ -26,34 +26,29 @@ pip install diffusers transformers accelerate
26
  ### Text Guided Inpainting Generation
27
 
28
  ```python
29
- from diffusers import KandinskyV22InpaintPipeline, KandinskyV22PriorPipeline
30
  from diffusers.utils import load_image
31
  import torch
32
  import numpy as np
33
 
34
- pipe_prior = KandinskyV22PriorPipeline.from_pretrained(
35
- "kandinsky-community/kandinsky-2-2-prior", torch_dtype=torch.float16
36
- )
37
- pipe_prior.to("cuda")
38
 
39
  prompt = "a hat"
40
- prior_output = pipe_prior(prompt)
41
-
42
- pipe = KandinskyV22InpaintPipeline.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16)
43
- pipe.to("cuda")
44
 
45
  init_image = load_image(
46
  "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main" "/kandinsky/cat.png"
47
  )
48
 
49
- mask = np.ones((768, 768), dtype=np.float32)
50
  # Let's mask out an area above the cat's head
51
- mask[:250, 250:-250] = 0
 
52
 
53
  out = pipe(
 
54
  image=init_image,
55
  mask_image=mask,
56
- **prior_output,
57
  height=768,
58
  width=768,
59
  num_inference_steps=150,
@@ -64,6 +59,20 @@ image.save("cat_with_hat.png")
64
  ```
65
  ![img](https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinskyv22/cat_with_hat.png)
66
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
67
 
68
  ## Model Architecture
69
 
 
26
  ### Text Guided Inpainting Generation
27
 
28
  ```python
29
+ from diffusers import AutoPipelineForInpainting
30
  from diffusers.utils import load_image
31
  import torch
32
  import numpy as np
33
 
34
+ pipe = AutoPipelineForInpainting.from_pretrained("kandinsky-community/kandinsky-2-2-decoder-inpaint", torch_dtype=torch.float16)
35
+ pipe.enable_model_cpu_offload()
 
 
36
 
37
  prompt = "a hat"
 
 
 
 
38
 
39
  init_image = load_image(
40
  "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main" "/kandinsky/cat.png"
41
  )
42
 
43
+ mask = np.zeros((768, 768), dtype=np.float32)
44
  # Let's mask out an area above the cat's head
45
+ mask[:250, 250:-250] = 1
46
+
47
 
48
  out = pipe(
49
+ prompt=prompt,
50
  image=init_image,
51
  mask_image=mask,
 
52
  height=768,
53
  width=768,
54
  num_inference_steps=150,
 
59
  ```
60
  ![img](https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main/kandinskyv22/cat_with_hat.png)
61
 
62
+ __<font color=red>Breaking change on the mask input:</font>__
63
+
64
+ We introduced a breaking change for Kandinsky inpainting pipeline in the following pull request: https://github.com/huggingface/diffusers/pull/4207. Previously we accepted a mask format where black pixels represent the masked-out area. We have changed to use white pixels to represent masks instead in order to have a unified mask format across all our pipelines.
65
+ Please upgrade your inpainting code to follow the above. If you are using Kandinsky Inpaint in production. You now need to change the mask to:
66
+
67
+ ```python
68
+ # For PIL input
69
+ import PIL.ImageOps
70
+ mask = PIL.ImageOps.invert(mask)
71
+
72
+ # For PyTorch and Numpy input
73
+ mask = 1 - mask
74
+ ```
75
+
76
 
77
  ## Model Architecture
78