FrugalNeRF: Fast Convergence for Few-shot Novel View Synthesis without
  Learned Priors
Authors: name: Chin-Yang Lin; name: Chung-Ho Wu; name: Chang-Han Yeh; name: Shih-Han Yen; name: Cheng Sun; name: Yu-Lun Liu
Published: 2024-10-21 17:59:53

Abstract: Neural Radiance Fields (NeRF) face significant challenges in few-shot
scenarios, primarily due to overfitting and long training times for
high-fidelity rendering. Existing methods, such as FreeNeRF and SparseNeRF, use
frequency regularization or pre-trained priors but struggle with complex
scheduling and bias. We introduce FrugalNeRF, a novel few-shot NeRF framework
that leverages weight-sharing voxels across multiple scales to efficiently
represent scene details. Our key contribution is a cross-scale geometric
adaptation scheme that selects pseudo ground truth depth based on reprojection
errors across scales. This guides training without relying on externally
learned priors, enabling full utilization of the training data. It can also
integrate pre-trained priors, enhancing quality without slowing convergence.
Experiments on LLFF, DTU, and RealEstate-10K show that FrugalNeRF outperforms
other few-shot NeRF methods while significantly reducing training time, making
it a practical solution for efficient and accurate 3D scene reconstruction.