FL4N: Feed-forward Latent Model for 4D Novel View Synthesis

[arXiv] | [Project Page] | [Code]

FL4N is a feed-forward latent 4D novel view synthesis model that renders dynamic scenes from unseen viewpoints and timestamps without explicit 3D representations. It models temporal dynamics as residuals on top of a pretrained static latent prior, enabling effective 4D scene modeling with limited dynamic training data.

Quick Start

git clone https://github.com/cvlab-kaist/FL4N.git
cd FL4N
conda create -n fl4n python=3.11 -y && conda activate fl4n
pip install torch torchvision && pip install -r requirements.txt

# Download the checkpoint
python -c "from huggingface_hub import hf_hub_download; hf_hub_download('eunbeen-hong/FL4N', 'fl4n.ckpt', local_dir='checkpoints')"

# Render one NVIDIA Dynamic Scenes scene (data under $FL4N_DATA/NVIDIA)
FL4N_DATA=/path/to/data python benchmark/run_eval.py --models fl4n --protocols ctvs --datasets nvidia --scenes Balloon1 --gpus 0

Citation

(TBA)

Acknowledgements

Built upon LagerNVS, and VGGT.