###### Abstract  
Depth maps captured by traditional consumer-grade depth cameras are often noisy and low-resolution. Especially when upsampling low-resolution depth maps with large upsampling factors, the resulting depth maps tend to suffer from vague edges. To address these issues, we propose a multi-channel progressive attention fusion network that utilizes a pyramid structure to progressively recover high-resolution depth maps. The inputs of the network are the low-resolution depth image and its corresponding color image. The color image is used as prior information in this network to fill in the missing high-frequency information of the depth image. Then, an attention-based multi-branch feature fusion module is employed to mitigate the texture replication issue caused by incorrect guidance from the color image and inconsistencies between the color image and the depth map. This module restores the HR depth map by effectively integrating the information from both inputs. Extensive experimental results demonstrate that our proposed method outperforms existing methods.  
depth map super-resolution; residual learning; deep convolutional neural network; attention mechanism +
Footnote †: journal: Applied  
0000-0001-999-7  
super-resolution. [PERSON] [9] proposes an end-to-end network model SRCNN, which applies deep learning to image super-resolution for the first time, demonstrating that deep learning can surpass traditional methods and achieve higher performance in super-resolution reconstruction. In recent years, color image super-resolution algorithms based on convolutional neural networks [10; 11; 12] have achieved great success, and many researchers have begun to apply this method to depth image super-resolution reconstruction [13; 14; 15]. However, unlike color images that contain abundant texture information, depth maps only display the depth information of objects. This presents a greater challenge for depth map super-resolution as the task involves recovering finer details and enhancing the resolution of the depth map.  
This paper proposes a novel multi-channel progressive attention fusion network based on color image guidance to address the aforementioned challenges. It utilizes the rich texture information of color images to fill in the missing information in LR depth images. The network is guided by the color image corresponding to the depth image and uses a pyramid structure combined with residual learning to progressively restore high-quality depth maps. The network mainly consists of two parts: the color image guidance branch and the depth image fusion upsampling branch. For the depth image branch, we propose an attention-based multi-branch feature fusion module, which can adaptively select the texture information consistent with the LR depth image in the HR color image for effective fusion, while ignoring the abundant foreground information present in the HR color image. As a result, it effectively avoids the generation of depth artifacts and preserves the quality of the depth map. Afterwards, the fused feature map is optimized through a multi-scale feature optimization residual module to capture and enhance the features of different levels, which strengthens the multi-scale expression ability of the network. Extensive results show that our method outperforms the current state-of-the-art methods.  
Our contributions are as follows:  
1. A novel multi-channel progressive attention fusion network is proposed, incorporating local residual connections and global residual connections to predict the high-frequency residual information of the image, which is more conducive to the recovery of depth map details.
2. The multi-scale feature optimization residual module is utilized to enhance the network's multi-scale feature representation capability through multiple intertwined paths.
3. We propose an attention-based multi-branch feature fusion module that can adaptively fuse features from both the depth and color images.