## 3 Proposed Method
### Attention-Based Multi-Branch Feature Fusion Module  
Generally, the fusion operation involves simply concatenating or weighted summing of features. However, in this task, simple feature fusion cannot achieve effective guidance for depth map reconstruction. Instead, it may lead to depth artifacts due to the rich texture edges present in the color image. Therefore, we designed a feature fusion module based on the attention mechanism, which can learn the feature consistency between the color image  
Figure 1: Multi-scale progressive attention fusion network with an upsampling factor of 8. The network consists of two main branches: the color-guided branch and the depth fusion and upsampling branch. The AMBFF module combines the color image features and the depth image features in an adaptive manner, taking into account their respective importance and relevance to the depth map reconstruction task. The MSFB module is used to capture the multi-scale information of feature maps and enhance the multi-scale representation ability of the network. and the depth image, explore useful information in the color image, and fully exert the guidance role of the color image in depth map super-resolution. As shown in Figure 2, the inputs of this module are the feature maps extracted from the two branches. To address the issue of unnecessary texture information in the color image and noise in the depth image, it is necessary to enhance the extracted features to reduce redundant and irrelevant information in the features so that the network can obtain richer and more representative features. The features from both branches are processed through convolution operations. Then, the PReLU and Sigmoid activation functions are applied to the features separately. By element-wise multiplication, the network adaptively extracts useful information from both inputs while filtering out irrelevant details. This process enhances the features of the images, facilitating effective guidance for depth map reconstruction. The specific operations are as follows:  
\[H_{c1} = \sigma(W_{c1}*h_{c}+b_{c1}) \tag{1}\] \[H_{d1} = \sigma(W_{d1}*h_{d}+b_{d1})\] (2) \[H_{c2} = \varepsilon(W_{c2}*h_{c}+b_{c2})\] (3) \[H_{d2} = \varepsilon(W_{d2}*h_{d}+b_{d2}) \tag{4}\]  
In the equation shown, \(h_{c}\) and \(h_{d}\) represent the outputs of the branches, \(W_{c1}\), \(W_{c2}\), \(W_{d1}\), and \(W_{d2}\) represent the two convolution operations performed on the color and depth image features, \(b_{c1}\), \(b_{c2}\), \(b_{d1}\), and \(b_{d2}\) represent the corresponding bias terms. \(\sigma\) represents the PReLU activation function, and \(\varepsilon\) represents the Sigmoid activation function. \[H_{sc} =H_{c1}*H_{c2} \tag{5}\] \[H_{sd} =H_{d1}*H_{d2} \tag{6}\]  
\(H_{sc},H_{sd}\) represent the enhanced color feature map and depth feature map, respectively, and \(*\) represents element-wise multiplication. Since the useful edge information in the color image can guide the depth map reconstruction, the useless information may lead to texture duplication or edge discontinuity. Therefore, an attention mechanism is needed to minimize the negative effects of texture replication and select the features that are consistent and significant with the depth image. The convolution operation is inspired by the Inception [29] network, which is divided into two branches, performing 3 \(\times\) 3 convolution and 5 \(\times\) 5 convolution, respectively, to learn feature information of different scales. The specific operations are as follows:  
\[H_{b1}=maxpol(H_{sc})\]  
Figure 2: Attention-based multi-branch feature fusion module. This module consists of two components: feature enhancement and attention fusion. The feature enhancement component helps the network better understand image features and improves its expressive power. The attention fusion component alleviates the negative effects caused by depth replication, allowing the network to focus on important features.