# Correction Compensation and Adaptive Cost Aggregation for Deep Laparoscopic Stereo Matching
## 3 Materials and Methods
### Correction Compensation Module  
Errors in camera calibration can be represented by a homography matrix \(H\in\mathbb{R}^{3\times 3}\)[29], resulting in a corrected practical image denoted as \(I_{\text{imperfect}}=I_{\text{perfect}}(\mathcal{W}(p,Hp))\), where \(p\) denotes the image coordinate and \(\mathcal{W}\) represents the warp operation. It is natural to recognize that imperfect alignment includes both vertical and horizontal shifts. This was also observed by [15], but they only addressed vertical offsets. In this paper, we propose a correction compensation module to cope with both vertical and horizontal shifts caused by imperfect calibration.  
Our correction compensation module consists of a feature compensation part and an image compensation part. As depicted in Figure 2, within the feature correction part, three convolution combinations are utilized to learn the offsets of features at 1/4, 1/8, and 1/16 scales, respectively. Each combination employs a 3 \(\times\) 3 convolution to learn the offset information of pixels in the feature maps as well as a 1 \(\times\) 1 convolution to integrate the information into a two-channel output. In the image compensation part, we use an hourglass network to mine the offset information from the shallow feature at 1/4 scale and the feature offset map at the same scale.  
To facilitate the proper learning of offsets by the correction compensation module, we introduce additional constraints to the network based on prior knowledge of correction targets. Initially, according to the general camera imaging and image correction outcomes, we do not encourage large offsets. Consequently, we impose exponential constraints on the absolute value of the offset, which can be formulated as follows:  
\[L_{\text{mag}}=\sum_{s\in\{1,2,3,4\}}\frac{\lambda_{s}}{N}(\|e^{|\mathbf{O}_{ \text{f}}|}+e^{|\mathbf{O}_{\text{f}}|}\|_{1}), \tag{1}\]  
where \(s\in\{1,2,3,4\}\) denotes \(1/16\), \(1/8\), \(1/4\), and full scale, respectively. \(\mathbf{O}_{\text{f}}^{s}\) and \(\mathbf{O}_{\text{r}}^{s}\) denote the left and right offset maps at \(s\) scale, respectively. \(\lambda_{s}\) denotes the weight coefficient of the magnitude loss at \(s\) scale. \(N\) denotes the number of pixels involved in the loss calculation. Note that in the formulas, the values in the offset maps are normalized; however, the offsets are measured in pixels during the compensation process. We limit the maximum offset for features at the \(1/16\) scale to be \(m\), which is a hyperparameter, and the maximum offset for features at other scales is a multiple of \(m\), specifically determined by the ratio of their resolutions.  
Moreover, to ensure consistency among offset maps of different scales, we apply a scale constraint to the absolute differences between offset maps at adjacent scales. This constraint is defined as follows:  
\[L_{\text{scale}}=\sum_{s\in\{1,2,3\}}\frac{\mu_{s}}{N}\|\mathbf{O}^{s}- \texttt{interp}(\mathbf{O}^{s+1})\|_{1}, \tag{2}\]  
where \(s\in\{1,2,3\}\) denotes \(1/16\), \(1/8\), and \(1/4\) scales, respectively. \(\mathbf{O}^{s}\) denotes the left or right offset map at \(s\) scale. \(\mu_{s}\) denotes the weight coefficient of the scale loss at \(s\) scale. And \(\texttt{interp}\) denotes the bilinear interpolation operation, where the scale factor is 0.5 or 0.25 depending on the ratio of their resolutions.  
In addition, for a pair of matching points, the offset from the left image to the right image and that from the right image to the left image should be inversely correlated. Therefore, we utilize the left-right consistency loss to constrain the absolute value of their sum, which is formulated as follows:  
\[L_{\text{Ir}}=\sum_{s\in\{1,2,3,4\}}\frac{\
u_{s}}{N}\|\mathbf{O}_{l}^{s}+ \mathbf{O}_{r}^{s}\|_{1}, \tag{3}\]  
where \(\
u_{s}\) denotes the weight coefficient of the left-right consistency loss at \(s\) scale. Finally, the loss of our correction compensation module can be expressed as  
\[L_{\text{C}}=L_{\text{mag}}+L_{\text{scale}}+L_{\text{Ir}}. \tag{4}\]