Package: nerf-pytorch
Neural Radiance Fields (Re-Implementation)This repository implements a minimal training and inference package around Neural Radiance Fields (NeRF). See the original paper "NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis," by Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng at ECCV 2020.InstallationThe package may be installed via pip from my PyPI package:pipinstallnerf-pytorchRenderingThe NeRF class implements all core functionality for volume rendering of a neural radiance field. You can initialize your own NeRF model and render an image using your (untrained) model with the following snippet.fromnerf.modelimportNeRFimporttorch# build the NeRF model with default parametersmodel=NeRF(normalize_position=6.0).cuda()# select a pose for the camera in homogeneous coordinatespose_o=torch.zeros(1,3).unsqueeze(0)pose_d=torch.eye(3).unsqueeze(0)# settings for the pinhole cameraimage_h=100image_w=100focal_length=130.0# settings for ray sampling and pose normalizationnear=2.0far=6.0# number of points sampled along the raynum_samples_per_ray=64max_chunk_size=1024# whether to sample points randomlyrandomly_sample=True# standard deviation of gaussian noise added to density headdensity_noise_std=1.0# render the image using volume renderingwithtorch.no_grad():image=model.render_image(pose_o.cuda(),pose_d.cuda(),image_h,image_w,focal_length,near,far,num_samples_per_ray,max_chunk_size=max_chunk_size,randomly_sample=randomly_sample,density_noise_std=density_noise_std)TrainingThis package provides several helpful utilities for training a NeRF model with supervised learning given image pose pairs. You can build a dataset for training a NeRF model using the provided PixelRayDataset class.fromnerf.datasetimportPixelRayDatasetfromnerf.modelimportNeRFimporttorch.utils.dataasdata# build the NeRF model with default parametersmodel=NeRF(normalize_position=6.0).cuda()# distanced of the clipping planes along the cameta z axisnear=2.0far=6.0# settings for the pinhole camerafocal_length=130.0# number of points sampled along the raynum_samples_per_ray=64# whether to sample points randomlyrandomly_sample=True# standard deviation of gaussian noise added to density headdensity_noise_std=1.0# create a dataset of pixels and corresponding rays for NeRFdataset=PixelRayDataset(images,poses,focal_length)data_loader=data.DataLoader(dataset,batch_size=1024,shuffle=True)fortarget,rays_o,rays_dindata_loader:# render a pixel for each ray using NeRFpixels=model.render_rays(rays_o,rays_d,near,far,num_samples_per_ray,randomly_sample=randomly_sample,density_noise_std=density_noise_std)# mean squared error in pixelsloss=((pixels-target)**2).mean()CitationCredit for the idea of NeRF goes to the original authors in their 2020 paper.@misc{mildenhall2020nerf,
    title={NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis},
    author={Ben Mildenhall and Pratul P. Srinivasan and Matthew Tancik and Jonathan T. Barron and Ravi Ramamoorthi and Ren Ng},
    year={2020},
    eprint={2003.08934},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}However, if you find this package helpful, please consider citing it!@misc{trabucco2021nerf,
  title={NeRF},
  author={Trabucco, Brandon},
  howpublished={\url{https://github.com/brandontrabucco/nerf}},
  year={2021}
}