patrickvonplaten
commited on
Commit
•
de54b51
1
Parent(s):
c7d2cd7
Add code snippet for Diffusers
Browse filesHey
@cerspense
,
Amazing work on adding such a cool model that is not polluted by watermarks! Just adding a quick code snippet here for `diffusers` :-)
Best,
Patrick
README.md
CHANGED
@@ -19,6 +19,42 @@ zeroscope_v2_576w uses 7.9gb of vram when rendering 30 frames at 576x320
|
|
19 |
|
20 |
For upscaling, it's recommended to use [zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) via vid2vid in the 1111 extension. It works best at 1024x576 with a denoise strength between 0.66 and 0.85. Remember to use the same prompt that was used to generate the original clip. <br />
|
21 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
22 |
|
23 |
### Known issues
|
24 |
|
|
|
19 |
|
20 |
For upscaling, it's recommended to use [zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) via vid2vid in the 1111 extension. It works best at 1024x576 with a denoise strength between 0.66 and 0.85. Remember to use the same prompt that was used to generate the original clip. <br />
|
21 |
|
22 |
+
### Usage in 🧨 Diffusers
|
23 |
+
|
24 |
+
Let's first install the libraries required:
|
25 |
+
|
26 |
+
```bash
|
27 |
+
$ pip install diffusers transformers accelerate torch
|
28 |
+
```
|
29 |
+
|
30 |
+
Now, generate a video:
|
31 |
+
|
32 |
+
```py
|
33 |
+
import torch
|
34 |
+
from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
|
35 |
+
from diffusers.utils import export_to_video
|
36 |
+
|
37 |
+
pipe = DiffusionPipeline.from_pretrained("cerspense/zeroscope_v2_576w", torch_dtype=torch.float16)
|
38 |
+
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
|
39 |
+
pipe.enable_model_cpu_offload()
|
40 |
+
|
41 |
+
prompt = "Darth Vader is surfing on waves"
|
42 |
+
video_frames = pipe(prompt, num_inference_steps=40, height=320, width=576, num_frames=24).frames
|
43 |
+
video_path = export_to_video(video_frames)
|
44 |
+
```
|
45 |
+
|
46 |
+
Here are some results:
|
47 |
+
|
48 |
+
<table>
|
49 |
+
<tr>
|
50 |
+
Darth vader is surfing on waves.
|
51 |
+
<br>
|
52 |
+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/darthvader_cerpense.gif"
|
53 |
+
alt="Darth vader surfing in waves."
|
54 |
+
style="width: 576;" />
|
55 |
+
</center></td>
|
56 |
+
</tr>
|
57 |
+
</table>
|
58 |
|
59 |
### Known issues
|
60 |
|