# Semi-Global Stereo Matching Algorithm Based on Multi-Scale Information Fusion
## 2 SGM-MSIF Algorithm
### Improved Cross-Scale Multi-Path Cost Aggregation  
The initial cost volume obtained by the above method is very sensitive to noise since only the correlation of local pixels is considered [26]. To improve the robustness of the algorithm, the idea of solving the energy function of SGM algorithm is introduced in this paper. The dynamic programming problem in two-dimensional (2D) space is simplified into a series of one-direction dynamic programming problems in one-dimensional (1D) space. The multi-path cost aggregation algorithm based on the one-direction dynamic programming is used to optimize the initial matching cost volume at each scale. It will greatly improve the processing efficiency of the algorithm while ensuring the matching accuracy. Then, the optimized cost volumes at each scale are fused through cross-scale cost aggregation to make up for the deficiency of stereo matching algorithm in low-texture regions at a single scale. Figure 2: Schematic diagram of census transform. For the multi-path cost aggregation algorithm based on one-direction dynamic programming, the disparity corresponding to each pixel is obtained by minimizing its energy function. In SGM [11], the energy function \(E(D)\) is defined as:  
\[E(D)=\sum_{m}C(m,D_{m})+\sum_{n\in N_{m}}P_{1}T[|D_{m}-D_{n}|=1]+\sum_{n\in N_{ m}}P_{2}T[|D_{m}-D_{n}|>1] \tag{4}\]  
where the first term represents the sum of the initial matching cost values of all pixels when the disparity map is \(D\). The second and third terms are smoothing terms, indicating that all pixels in the \(N_{m}\) neighborhood of pixel \(m\) are punished. \(P_{1}\) and \(P_{2}\) (\(P_{2}>P_{1}\)) are penalty coefficients. \(T\) is the truncation function to ensure that the second term is executed when the absolute difference between the disparity \(D_{m}\) of pixels \(m\) and \(D_{n}\) of pixels \(n\) is equal to 1 and the third term is executed when the absolute difference is larger than 1. The one-dimensional cost aggregation on all paths around each pixel is performed to obtain the cost values of each path. Then, the cost values under all paths are added to get the matched cost value of the pixel after aggregation. The cost aggregation formula of pixel \(p\) along path \(r\) under disparity \(d\) is shown in Equation (5):  
\[L_{r}(p,d)=C(p,d)+\min \begin{cases}L_{r}(p-r,d)\\ L_{r}(p-r,d-1)+P_{1}\\ L_{r}(p-r,d+1)+P_{1}\\ \min_{t}L_{r}(p-r,t)+P_{2}\end{cases} -\min_{t}L_{r}(p-r,t) \tag{5}\]  
where \(L_{r}\) represents the aggregated cost value on path \(r\), \(p-r\) represents the last pixel of pixel \(p\) on path \(r\), and \(\min_{t}L_{r}(p-r,t)\) represents the minimum cost value of pixel \(p-r\) under all disparity. The penalty coefficients \(P_{1}\) and \(P_{2}\) are determined by the color difference \(\beta_{l}\) and \(\beta_{r}\) of the pixels. \(\beta_{l}\) is the color difference between \(p\) and \(p-r\) in the left image, and \(\beta_{r}\) is the color difference between \(q\) and \(q-r\) in the right image, as shown in Equation (6):  
\[\left\{ \begin{array}{ll}P_{1}=\frac{\Pi_{1}}{10},\ P_{2}=\frac{\Pi_{2}}{10}, &\text{if }\beta_{l}>\tau,\ \beta_{r}>\tau\\ P_{1}=\Pi_{1},\ P_{2}=\Pi_{2},&\text{if }\beta_{l}<\tau,\ \beta_{r}<\tau\\ P_{1}=\frac{\Pi_{1}}{4},\ P_{2}=\frac{\Pi_{2}}{4},&\text{otherwise}\end{array} \right. \tag{6}\]  
where \(\Pi_{1}\) and \(\Pi_{2}\) are two constants, and \(\tau\) represents the threshold to determine the color difference. The aggregated cost value of each path is added to obtain the multi-path cost aggregation result \(\widetilde{C}(p,d)\) of pixel \(p\) under disparity \(d\) as:  
\[\widetilde{C}(p,d)=\sum_{r}L_{r}(p,d) \tag{7}\]  
In multi-path cost aggregation, increasing the number of paths can effectively reduce the EMR of stereo matching algorithm. However, with the increase of the path, the processing efficiency of the algorithm will be significantly reduced.