Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
Authors: name: Aleksei Bochkovskii; name: Amaël Delaunoy; name: Hugo Germain; name: Marcel Santos; name: Yichao Zhou; name: Stephan R. Richter; name: Vladlen Koltun
Published: 2024-10-02 22:42:20

Abstract: We present a foundation model for zero-shot metric monocular depth
estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with
unparalleled sharpness and high-frequency details. The predictions are metric,
with absolute scale, without relying on the availability of metadata such as
camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map
in 0.3 seconds on a standard GPU. These characteristics are enabled by a number
of technical contributions, including an efficient multi-scale vision
transformer for dense prediction, a training protocol that combines real and
synthetic datasets to achieve high metric accuracy alongside fine boundary
tracing, dedicated evaluation metrics for boundary accuracy in estimated depth
maps, and state-of-the-art focal length estimation from a single image.
Extensive experiments analyze specific design choices and demonstrate that
Depth Pro outperforms prior work along multiple dimensions. We release code and
weights at https://github.com/apple/ml-depth-pro