Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance
  for Self-supervised Monocular Depth Estimation
Authors: name: Wenyao Zhang, name: Hongsi Liu, name: Bohan Li, name: Jiawei He, name: Zekun Qi, name: Yunnan Wang, name: Shengyang Zhao, name: Xinqiang Yu, name: Wenjun Zeng, name: Xin Jin

Current self-supervised monocular depth estimation (MDE) approaches encounter
performance limitations due to insufficient semantic-spatial knowledge
extraction. To address this challenge, we propose Hybrid-depth, a novel
framework that systematically integrates foundation models (e.g., CLIP and
DINO) to extract visual priors and acquire sufficient contextual information
for MDE. Our approach introduces a coarse-to-fine progressive learning
framework: 1) Firstly, we aggregate multi-grained features from CLIP (global
semantics) and DINO (local spatial details) under contrastive language
guidance. A proxy task comparing close-distant image patches is designed to
enforce depth-aware feature alignment using text prompts; 2) Next, building on
the coarse features, we integrate camera pose information and pixel-wise
language alignment to refine depth predictions. This module seamlessly
integrates with existing self-supervised MDE pipelines (e.g., Monodepth2,
ManyDepth) as a plug-and-play depth encoder, enhancing continuous depth
estimation. By aggregating CLIP's semantic context and DINO's spatial details
through language guidance, our method effectively addresses feature granularity
mismatches. Extensive experiments on the KITTI benchmark demonstrate that our
method significantly outperforms SOTA methods across all metrics, which also
indeed benefits downstream tasks like BEV perception. Code is available at
https://github.com/Zhangwenyao1/Hybrid-depth.