DepthFM: Fast Monocular Depth Estimation with Flow Matching
Authors: name: Ming Gui; name: Johannes S. Fischer; name: Ulrich Prestel; name: Pingchuan Ma; name: Dmytro Kotovenko; name: Olga Grebenkova; name: Stefan Andreas Baumann; name: Vincent Tao Hu; name: Björn Ommer
Published: 2024-03-20 17:51:53

Abstract: Monocular depth estimation is crucial for numerous downstream vision tasks
and applications. Current discriminative approaches to this problem are limited
due to blurry artifacts, while state-of-the-art generative methods suffer from
slow sampling due to their SDE nature. Rather than starting from noise, we seek
a direct mapping from input image to depth map. We observe that this can be
effectively framed using flow matching, since its straight trajectories through
solution space offer efficiency and high quality. Our study demonstrates that a
pre-trained image diffusion model can serve as an adequate prior for a flow
matching depth model, allowing efficient training on only synthetic data to
generalize to real images. We find that an auxiliary surface normals loss
further improves the depth estimates. Due to the generative nature of our
approach, our model reliably predicts the confidence of its depth estimates. On
standard benchmarks of complex natural scenes, our lightweight approach
exhibits state-of-the-art performance at favorable low computational cost
despite only being trained on little synthetic data.