stabilityai

/

stable-video-diffusion-img2vid-xt

like 2.31k

it should be used this way (linux):

To set up and use the Stable Video Diffusion XT model (stable-video-diffusion-img2vid-xt) from Stability AI, you can follow these steps:

Prerequisites:

The setup is confirmed to work on Ubuntu 22.04.3 LTS with Python version 3.10.12.
An NVIDIA GPU is required.
Ensure sufficient storage space as model files are around 10GB each.
Clone the Generative Models Repository:

Clone Stability AI's generative-models repository and navigate to it:

git clone https://github.com/Stability-AI/generative-models.git
cd generative-models
Set Up Python Environment:

Install python3.10-venv and PyTorch 2.0:

sudo apt install python3.10-venv
python3 -m venv .pt2
source .pt2/bin/activate
pip3 install -r requirements/pt2.txt
pip3 install .
Modify the Streamlit Helpers File (optional):

Edit the file scripts/demo/streamlit_helpers.py and set lowvram_mode to True if you have limited VRAM.
Download Model Weights:

Create a checkpoints directory and download the required model files from Hugging Face into this directory:

mkdir checkpoints
wget -O ./checkpoints/svd_xt.safetensors 'https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/resolve/main/svd_xt.safetensors?download=true'
wget -O ./checkpoints/svd_xt_image_decoder.safetensors 'https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/resolve/main/svd_xt_image_decoder.safetensors?download=true'
Running the Demo Script:

Run the demo script using Streamlit:

PYTHONPATH=. ./.pt2/bin/streamlit run scripts/demo/video_sampling.py
Select the desired model version, upload your image, and adjust settings as needed. Finally, click 'Sample' to start generating the video.
Output:

The generated video will be saved in the ./outputs/demo/vid/svd/samples/ directory. You can set up a temporary HTTP server to access the files easily using Docker:

sudo docker run -p 80:80 -v ./outputs/demo/vid/svd/samples:/usr/local/apache

I could proceed until the line to run:

PYTHONPATH=. ./.pt2/bin/streamlit run scripts/demo/video_sampling.py

streamlit is not in my pt2/bin/folder. Can anyone give any hint? Any info will be really appreciated.

pip install streamlit

Thank you for your comment, peterxing.
'stremlit' is already installed as the "pip3 install -r requirements/pt2.txt" includes it.
But, still I do not have 'streamlit' in the .pt2/bin directory. So, 'PYTHONPATH=. ./.pt2/bin/streamlit run' generates an error.

Are you in the generative-models (root of the cloned Depot) folder on your terminal when you are running this line ?

PYTHONPATH=. ./.pt2/bin/streamlit run scripts/demo/video_sampling.py

Thank you very much for your comments and info people. Yes, now I can start it, but has error as below.

Traceback (most recent call last):
File "/content/generative-models/scripts/demo/video_sampling.py", line 5, in
from scripts.demo.streamlit_helpers import *
ModuleNotFoundError: No module named 'scripts'

I can see "generative-models/scripts/demo/streamlit_helpers.py" though.

did you think of reactivating the venv before running the streamlit command ?

in the generative-models folder :

source .pt2/bin/activate

and are you sure you are not running the pythonpath commandline from the /content folder?
Make sure you are in the generative-models folder

PYTHONPATH=. ./.pt2/bin/streamlit run scripts/demo/video_sampling.py

Yo when i run this "PYTHONPATH=. ./.pt2/bin/streamlit run scripts/demo/video_sampling.py" This Error comes up "'PYTHONPATH' is not recognized as an internal or external command,
operable program or batch file."

Python is installed and it's in the environment variable (Whatever it's called). Stable Diffusion is running fine that means python is INSTALLED.

I am getting this error can anyone please help:-

warning: build failed, waiting for other jobs to finish...
      error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module --crate-type cdylib --` failed with code 101
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
  Building wheel for clip (setup.py) ... done
  Created wheel for clip: filename=clip-1.0-py3-none-any.whl size=1369499 sha256=9af0f995c082d79cad9ebd70e649f8ff9a69d337be0fc0becbe819386abee02f
  Stored in directory: /tmp/pip-ephem-wheel-cache-2gc8onh0/wheels/3f/7c/a4/9b490845988bf7a4db33674d52f709f088f64392063872eb9a
Successfully built clip
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects