README.md

metadata

title: vdpm
app_file: gradio_demo.py
sdk: gradio
sdk_version: 5.17.1

V-DPM Low-VRAM Multi-View Inference

Multi-view modification of V-DPM with low-vram modification to improve inference speed on consumer graphics cards and allow longer sequences to finish. Tested on 3070Ti, H200, PRO 6000,

First, clone the repository and setup a virtual environment with uv:

Original model's aggregator stores intermediate outputs of all 24 attention blocks, but only 4 of them is used by prediction heads. Made it return only that 4. del unused intermediate tensors to free memory for subsequent code @torch.compile some functions (e.g. MLP with GELU, LayerNorm) torch.cuda.empty_cache() when helpful

git clone git@github.com:eldar/vdpm.git
cd vdpm
uv venv --python 3.12
. .venv/bin/activate

# Install PyTorch with CUDA 11.8 first
uv pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu118

# Then install remaining dependencies
uv pip install -r requirements.txt

Gradio demo

python gradio_demo.py