FastSR-NeRF: Improving NeRF Efficiency on Consumer Devices with A Simple
  Super-Resolution Pipeline
Authors: name: Chien-Yu Lin; name: Qichen Fu; name: Thomas Merth; name: Karren Yang; name: Anurag Ranjan
Published: 2023-12-15 21:02:23

Abstract: Super-resolution (SR) techniques have recently been proposed to upscale the
outputs of neural radiance fields (NeRF) and generate high-quality images with
enhanced inference speeds. However, existing NeRF+SR methods increase training
overhead by using extra input features, loss functions, and/or expensive
training procedures such as knowledge distillation. In this paper, we aim to
leverage SR for efficiency gains without costly training or architectural
changes. Specifically, we build a simple NeRF+SR pipeline that directly
combines existing modules, and we propose a lightweight augmentation technique,
random patch sampling, for training. Compared to existing NeRF+SR methods, our
pipeline mitigates the SR computing overhead and can be trained up to 23x
faster, making it feasible to run on consumer devices such as the Apple
MacBook. Experiments show our pipeline can upscale NeRF outputs by 2-4x while
maintaining high quality, increasing inference speeds by up to 18x on an NVIDIA
V100 GPU and 12.8x on an M1 Pro chip. We conclude that SR can be a simple but
effective technique for improving the efficiency of NeRF models for consumer
devices.