The 3D reconstruction of scenes and objects from 2D images is an extremely important part of many tasks, such as robot navigation, scene understanding and surveying. Mobile platforms with a single camera can easily overwhelm systems that attempt exhaustive optimisations over all images. Reconstruction algorithms that are capable of selecting data that maximises performance, while reducing computational time are necessary to perform reconstruction in the real world. This work proposes an approach to intelligently filter large amounts of data for 3D reconstructions of unknown scenes using monocular cameras. Figure 1 shows how the reconstruction progresses with a limited number of views. We can achieve state-of-the-art results using as little as 3.8% of the views on the Middlebury dataset. Furthermore, view selection is efficient, taking only 1.1ms per pose pair. We first present a novel criterion for Next-Best View (NBV) optimisation based on a compromise between the competing objectives of coverage and accuracy. The coverage objective will drive the system to collect views of previously unobserved parts of the scene (e.g., due to restrictions on the field of view or occlusion), whereas the accuracy objective will drive the system to choose the next pose to reduce the point cloud's uncertainty. These two criteria are optimised jointly, making use of an octree structure and a dense point cloud. The octree allows for quick and efficient calculations on scene geometry, while the dense cloud (and covariances) allow for more detailed calculations about scene noise and and viewing angle. The NBV is calculated as follows: Given a Configuration Space (CS) of sensor poses, the cost of each pose can be estimated by casting a set S r of random rays from the camera centre through the image plane. Each ray will continue until it hits either an occupied (V o ) or unobserved (V u ) voxel, ignoring empty (V e ) voxels. When a ray r ∈ S r intersects with an occupied voxel v ∈ V o , we can estimate a cost for each point p ∈ P v as φ (r, p) = e −||λ p e p ×r|| , where λ p and e p are the largest eigenvalue and eigenvector, respectively, of the covariance Σ p . Consequently, the cost of a voxel is defined as the average point cost The NBV cost of a particular pose x is defined as In this equation, γ is a parameter that can encourage or discourage exploration. As shown in Figure 2 γ of 1 will give the highest cost to unobserved voxels, prefering to reduce the uncertainty of observed voxels, while 0 will give them the lowest, preferring exploratory behaviour.
Recently Mendez et al. REF propose an efficient method to compute, among a set of candidate poses, both the NBV by considering only a part of the scene, and a method to add a further camera that forms a good stereo pairs with the NBV.