Scene Coordinate Reconstruction: Posing of Image Collections via
  Incremental Learning of a Relocalizer
Authors: name: Eric Brachmann; name: Jamie Wynn; name: Shuai Chen; name: Tommaso Cavallari; name: Áron Monszpart; name: Daniyar Turmukhambetov; name: Victor Adrian Prisacariu
Published: 2024-04-22 17:02:33

Abstract: We address the task of estimating camera parameters from a set of images
depicting a scene. Popular feature-based structure-from-motion (SfM) tools
solve this task by incremental reconstruction: they repeat triangulation of
sparse 3D points and registration of more camera views to the sparse point
cloud. We re-interpret incremental structure-from-motion as an iterated
application and refinement of a visual relocalizer, that is, of a method that
registers new views to the current state of the reconstruction. This
perspective allows us to investigate alternative visual relocalizers that are
not rooted in local feature matching. We show that scene coordinate regression,
a learning-based relocalization approach, allows us to build implicit, neural
scene representations from unposed images. Different from other learning-based
reconstruction methods, we do not require pose priors nor sequential inputs,
and we optimize efficiently over thousands of images. Our method, ACE0 (ACE
Zero), estimates camera poses to an accuracy comparable to feature-based SfM,
as demonstrated by novel view synthesis. Project page:
https://nianticlabs.github.io/acezero/