Hypernerf practice reproduction instruction
before getting started
Python 3.8 and Python 3.9 has been tested to work flawlessly. Python 3.10 will fail.
Colab
Important Information: in May 2023, colab is updated to python 3.10, in order for this reproduction to work, select tools- command palatte - use fallback runtime.
This will change the runtime back to python 3.9.
The fallback runtime is supported until mid May 2023, after that, we could only use other tricks to change the colab python version.
Colab TPU doesnot work here anymore, GPU runs flawlessly.
local machine
Start with python 3.8 or python 3.9
Install the following:
sudo apt-get install libilmbase-dev libopenexr-dev ffmpeg
Follow official colmap project to install with cuda support, or use default package manager to install colmap, but might only has cpu support.
Then install pip requirements:
wget https://raw.githubusercontent.com/xieyizheng/hypernerf/main/requirements.txt
pip install -r requirements.txt
With this, everything is set! The Processing, Training and Rendering notebooks should now work!
using the notebooks
There are three notebooks: Processing, Training and Rendering.
- Processing turn raw video into Dataset
- Training turn Dataset into a experiment/checkpoint
- Rendering load a checkpoint and render desired video
For our custom scenes, we can start at any notebook, just set the correct data_dir and experiment_dir at the beginning.
For rendering from an existing experiment/checkpoint, also edit the config.gin and set the correct data_dir before rendering.
Important hyperparameters: image_scale & batch_size
While rendering: setting the "ambient dimension"/"time axis" coordinates correctly to get the desired result. This was not documented in the original notebook.
Dataset&checkpoint Download:
Separately:
https://huggingface.co/datasets/xieyizheng/hypernerf_custom_scenes/tree/main