MM-NeRF: Large-Scale Scene Representation with Multi-Resolution Hash Grid and Multi-View Priors Features

Authors:
raw_author_name: Bo Dong
raw_affiliation_string: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China; Key Laboratory of Network Information System Technology (NIST), Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China; University of Chinese Academy of Sciences, Beijing 100190, China
raw_author_name: Kaiqiang Chen
raw_affiliation_string: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China; Key Laboratory of Network Information System Technology (NIST), Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China
raw_author_name: Zhirui Wang
raw_affiliation_string: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China; Key Laboratory of Network Information System Technology (NIST), Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China
raw_author_name: Menglong Yan
raw_affiliation_string: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China; Cyber Intelligent Technology (Shandong) Co., Ltd., Jinan 250100, China; Jigang Defence Technology Company, Ltd., Jinan 250132, China; Key Laboratory of Network Information System Technology (NIST), Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China
raw_author_name: Jiaojiao Gu
raw_affiliation_string: Coastal Defense College, Naval Aeronautical University, Yantai 264001, China
raw_author_name: Xian Sun
raw_affiliation_string: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China; Key Laboratory of Network Information System Technology (NIST), Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China

Abstract: Reconstructing large-scale scenes using Neural Radiance Fields (NeRFs) is a research hotspot in 3D computer vision. Existing MLP (multi-layer perception)-based methods often suffer from issues of underfitting and a lack of fine details in rendering large-scale scenes. Popular solutions are to divide the scene into small areas for separate modeling or to increase the layer scale of the MLP network. However, the subsequent problem is that the training cost increases. Moreover, reconstructing large scenes, unlike object-scale reconstruction, involves a geometrically considerable increase in the quantity of view data if the prior information of the scene is not effectively utilized. In this paper, we propose an innovative method named MM-NeRF, which integrates efficient hybrid features into the NeRF framework to enhance the reconstruction of large-scale scenes. We propose employing a dual-branch feature capture structure, comprising a multi-resolution 3D hash grid feature branch and a multi-view 2D prior feature branch. The 3D hash grid feature models geometric details, while the 2D prior feature supplements local texture information. Our experimental results show that such integration is sufficient to render realistic novel views with fine details, forming a more accurate geometric representation. Compared with representative methods in the field, our method significantly improves the PSNR (Peak Signal-to-Noise Ratio) by approximately 5%. This remarkable progress underscores the outstanding contribution of our method in the field of large-scene radiance field reconstruction.