h94 commited on
Commit
3f6cbb7
1 Parent(s): 309dc56

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +53 -0
README.md CHANGED
@@ -122,6 +122,58 @@ images = ip_model.generate(
122
  prompt=prompt, negative_prompt=negative_prompt, faceid_embeds=faceid_embeds, num_samples=4, width=512, height=768, num_inference_steps=30, seed=2023
123
  )
124
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
125
  ```
126
 
127
  ### IP-Adapter-FaceID-SDXL
@@ -188,6 +240,7 @@ images = ip_model.generate(
188
 
189
  ```
190
 
 
191
  ### IP-Adapter-FaceID-Plus
192
 
193
  Firstly, you should use [insightface](https://github.com/deepinsight/insightface) to extract face ID embedding and face image:
 
122
  prompt=prompt, negative_prompt=negative_prompt, faceid_embeds=faceid_embeds, num_samples=4, width=512, height=768, num_inference_steps=30, seed=2023
123
  )
124
 
125
+ ```
126
+
127
+ you can also use a normal IP-Adapter and a normal LoRA to load model:
128
+
129
+ ```python
130
+ import torch
131
+ from diffusers import StableDiffusionPipeline, DDIMScheduler, AutoencoderKL
132
+ from PIL import Image
133
+
134
+ from ip_adapter.ip_adapter_faceid_separate import IPAdapterFaceID
135
+
136
+ base_model_path = "SG161222/Realistic_Vision_V4.0_noVAE"
137
+ vae_model_path = "stabilityai/sd-vae-ft-mse"
138
+ ip_ckpt = "ip-adapter-faceid_sd15.bin"
139
+ lora_ckpt = "ip-adapter-faceid_sd15_lora.safetensors"
140
+ device = "cuda"
141
+
142
+ noise_scheduler = DDIMScheduler(
143
+ num_train_timesteps=1000,
144
+ beta_start=0.00085,
145
+ beta_end=0.012,
146
+ beta_schedule="scaled_linear",
147
+ clip_sample=False,
148
+ set_alpha_to_one=False,
149
+ steps_offset=1,
150
+ )
151
+ vae = AutoencoderKL.from_pretrained(vae_model_path).to(dtype=torch.float16)
152
+ pipe = StableDiffusionPipeline.from_pretrained(
153
+ base_model_path,
154
+ torch_dtype=torch.float16,
155
+ scheduler=noise_scheduler,
156
+ vae=vae,
157
+ feature_extractor=None,
158
+ safety_checker=None
159
+ )
160
+
161
+ # load lora and fuse
162
+ pipe.load_lora_weights(lora_ckpt)
163
+ pipe.fuse_lora()
164
+
165
+ # load ip-adapter
166
+ ip_model = IPAdapterFaceID(pipe, ip_ckpt, device)
167
+
168
+ # generate image
169
+ prompt = "photo of a woman in red dress in a garden"
170
+ negative_prompt = "monochrome, lowres, bad anatomy, worst quality, low quality, blurry"
171
+
172
+ images = ip_model.generate(
173
+ prompt=prompt, negative_prompt=negative_prompt, faceid_embeds=faceid_embeds, num_samples=4, width=512, height=768, num_inference_steps=30, seed=2023
174
+ )
175
+
176
+
177
  ```
178
 
179
  ### IP-Adapter-FaceID-SDXL
 
240
 
241
  ```
242
 
243
+
244
  ### IP-Adapter-FaceID-Plus
245
 
246
  Firstly, you should use [insightface](https://github.com/deepinsight/insightface) to extract face ID embedding and face image: