+
+## Installation
+```
+conda env create -f environment.yml
+conda activate lavie
+```
+
+## Download Pre-Trained models
+Download [pre-trained models](https://huggingface.co/YaohuiW/LaVie/tree/main), [stable diffusion 1.4](https://huggingface.co/CompVis/stable-diffusion-v1-4/tree/main), [stable-diffusion-x4-upscaler](https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler/tree/main) to `./pretrained_models`. You should be able to see the following:
+```
+βββ pretrained_models
+β βββ lavie_base.pt
+β βββ lavie_interpolation.pt
+β βββ lavie_vsr.pt
+β βββ stable-diffusion-v1-4
+β β βββ ...
+βββ βββ stable-diffusion-x4-upscaler
+ βββ ...
+```
+
+## Inference
+The inference contains **Base T2V**, **Video Interpolation** and **Video Super-Resolution** three steps. We provide several options to generate videos:
+* **Step1**: 320 x 512 resolution, 16 frames
+* **Step1+Step2**: 320 x 512 resolution, 61 frames
+* **Step1+Step3**: 1280 x 2048 resolution, 16 frames
+* **Step1+Step2+Step3**: 1280 x 2048 resolution, 61 frames
+
+Feel free to try different options:)
+
+
+### Step1. Base T2V
+Run following command to generate videos from base T2V model.
+```
+cd base
+python pipelines/sample.py --config configs/sample.yaml
+```
+Edit `text_prompt` in `configs/sample.yaml` to change prompt, results will be saved under `./res/base`.
+
+### Step2 (optional). Video Interpolation
+Run following command to conduct video interpolation.
+```
+cd interpolation
+python sample.py --config configs/sample.yaml
+```
+The default input video path is `./res/base`, results will be saved under `./res/interpolation`. In `configs/sample.yaml`, you could modify default `input_folder` with `YOUR_INPUT_FOLDER` in `configs/sample.yaml`. Input videos should be named as `prompt1.mp4`, `prompt2.mp4`, ... and put under `YOUR_INPUT_FOLDER`. Launching the code will process all the input videos in `input_folder`.
+
+
+### Step3 (optional). Video Super-Resolution
+Run following command to conduct video super-resolution.
+```
+cd vsr
+python sample.py --config configs/sample.yaml
+```
+The default input video path is `./res/base` and results will be saved under `./res/vsr`. You could modify default `input_path` with `YOUR_INPUT_FOLDER` in `configs/sample.yaml`. Smiliar to Step2, input videos should be named as `prompt1.mp4`, `prompt2.mp4`, ... and put under `YOUR_INPUT_FOLDER`. Launching the code will process all the input videos in `input_folder`.
+
+
+## BibTex
+```bibtex
+@article{wang2023lavie,
+ title={LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models},
+ author={Wang, Yaohui and Chen, Xinyuan and Ma, Xin and Zhou, Shangchen and Huang, Ziqi and Wang, Yi and Yang, Ceyuan and He, Yinan and Yu, Jiashuo and Yang, Peiqing and others},
+ journal={arXiv preprint arXiv:2309.15103},
+ year={2023}
+}
+```
+
+## Acknowledgements
+The code is buit upon [diffusers](https://github.com/huggingface/diffusers) and [Stable Diffusion](https://github.com/CompVis/stable-diffusion), we thank all the contributors for open-sourcing.
+
+
+## License
+The code is licensed under Apache-2.0, model weights are fully open for academic research and also allow **free** commercial usage. To apply for a commercial license, please fill in the [application form]().
diff --git a/base/__pycache__/download.cpython-311.pyc b/base/__pycache__/download.cpython-311.pyc
new file mode 100644
index 0000000000000000000000000000000000000000..8f9de5e7eb4b7a4bfd816fc575ea4888229239de
Binary files /dev/null and b/base/__pycache__/download.cpython-311.pyc differ
diff --git a/base/app.py b/base/app.py
new file mode 100644
index 0000000000000000000000000000000000000000..14c66df82b058e77c83c9cd1106e5df64aa939c0
--- /dev/null
+++ b/base/app.py
@@ -0,0 +1,175 @@
+import gradio as gr
+from text_to_video import model_t2v_fun,setup_seed
+from omegaconf import OmegaConf
+import torch
+import imageio
+import os
+import cv2
+import pandas as pd
+import torchvision
+import random
+config_path = "/mnt/petrelfs/zhouyan/project/lavie-release/base/configs/sample.yaml"
+args = OmegaConf.load("/mnt/petrelfs/zhouyan/project/lavie-release/base/configs/sample.yaml")
+device = "cuda" if torch.cuda.is_available() else "cpu"
+# ------- get model ---------------
+model_t2V = model_t2v_fun(args)
+model_t2V.to(device)
+if device == "cuda":
+ model_t2V.enable_xformers_memory_efficient_attention()
+
+# model_t2V.enable_xformers_memory_efficient_attention()
+css = """
+h1 {
+ text-align: center;
+}
+#component-0 {
+ max-width: 730px;
+ margin: auto;
+}
+"""
+
+def infer(prompt, seed_inp, ddim_steps,cfg):
+ if seed_inp!=-1:
+ setup_seed(seed_inp)
+ else:
+ seed_inp = random.choice(range(10000000))
+ setup_seed(seed_inp)
+ videos = model_t2V(prompt, video_length=16, height = 320, width= 512, num_inference_steps=ddim_steps, guidance_scale=cfg).video
+ print(videos[0].shape)
+ if not os.path.exists(args.output_folder):
+ os.mkdir(args.output_folder)
+ torchvision.io.write_video(args.output_folder + prompt[0:30].replace(' ', '_') + '-'+str(seed_inp)+'-'+str(ddim_steps)+'-'+str(cfg)+ '-.mp4', videos[0], fps=8)
+ # imageio.mimwrite(args.output_folder + prompt.replace(' ', '_') + '.mp4', videos[0], fps=8)
+ # video = cv2.VideoCapture(args.output_folder + prompt.replace(' ', '_') + '.mp4')
+ # video = imageio.get_reader(args.output_folder + prompt.replace(' ', '_') + '.mp4', 'ffmpeg')
+
+
+ # video = model_t2V(prompt, seed_inp, ddim_steps)
+
+ return args.output_folder + prompt[0:30].replace(' ', '_') + '-'+str(seed_inp)+'-'+str(ddim_steps)+'-'+str(cfg)+ '-.mp4'
+
+print(1)
+
+# def clean():
+# return gr.Image.update(value=None, visible=False), gr.Video.update(value=None)
+def clean():
+ return gr.Video.update(value=None)
+
+title = """
+ + Apply InternΒ·Vchitect to generate a video +
+