# A Method for Adapting Stereo Matching Algorithms to Real Environments
## 3 Materials and Methods
### The Method of Auxiliary Calibration
\begin{split}\mathbf{p_{0}}&=\mathbf{A_{0}}[ \mathbf{R_{0}}|\mathbf{t_{0}}]\mathbf{P_{w0}}\\ \mathbf{p_{1}^{\prime}}+\mathbf{v_{1}}&=\mathbf{A_{1 }}[\mathbf{R_{1}}|\mathbf{t_{1}}]\mathbf{P_{w1}}\end{split}  \tag{3}\]  
The concept of the \(\mathbf{v_{1}}\) vector is derived from the bundle adjustment process; however, in this paper, it will be applied to a single stereo camera instead of a large number of images taken from different points of view. \(\mathbf{v_{1}}\) reflects the inaccuracy of mapping point \(\mathbf{P_{w1}}\) to 2D plane cameras. In Equation (3), \(\mathbf{p_{1}^{\prime}}\) denotes the location of a point after a perfectly performed calibration. The \(\mathbf{v_{1}}\) vector is included only in the equation for the side camera because adjusting a side image to a reference image is sufficient to obtain extrinsic calibration. This is sufficient because, in the ACS module, only a rotation and a translation in the \(y\)-axis are considered. Not including the vector \(\mathbf{v_{0}}\) in the equation for a reference image is even more important because it does not interfere with the process of transforming disparity maps into depth maps. Stereo matching algorithms produce a disparity map. However, in real-world applications, a disparity map is not needed; it is a depth map that determines the distances to objects that are visible to an image sensor. The calibration parameters of the reference camera make it possible to convert a disparity map into a depth map. If the \(\mathbf{v_{0}}\) vector is added to the calibration formula of a reference image, then it influences these transformations. Due to the exclusion of \(\mathbf{v_{0}}\), this problem is avoided. The process of auxiliary calibration aims to minimize vector \(\mathbf{v_{1}}\). In order to achieve this, the author added an additional transformation \(\mathbf{C}\), which is a result of executing the ACS module presented in Figure 1. Transformation \(\mathbf{C}\) does not modify matrices \(\mathbf{A_{1}}\), \(\mathbf{R_{1}}\), and \(\mathbf{t_{1}}\); it only further processes images. After adding the ACS module, Equation (3) is converted into Equation (4). \[ \begin{split}\mathbf{p_{0}}&=\mathbf{A_{0}}[ \mathbf{R_{0}}|\mathbf{t_{0}}]\mathbf{P_{w0}}\\ \mathbf{p_{1}^{\prime}}+\mathbf{v_{1}^{\prime}}&= \mathbf{C}\mathbf{A_{1}}[\mathbf{R_{1}}|\mathbf{t_{1}}]\mathbf{P_{w1}}\end{split}  \tag{4}\]  
where \(\mathbf{v_{1}^{\prime}}\) is the result of minimizing \(\mathbf{v_{1}}\). Vector \(\mathbf{v_{1}^{\prime}}\) reflects a discrepancy from a perfect calibration, even after executing the ACS module. The \(\mathbf{C}\) matrix contains transformation parameters corresponding to the transformation model applied in the ACS module. These transformations include translating the side image in the \(y\)-axis and rotating this image. The aim of the ACS is to reduce these extrinsic distortions. The side image is shifted by the number of points equal to \(c\). It is also rotated by the angle \(\theta\). Therefore, the transformation \(\mathbf{C}\) has the form presented in Equation (5). \[\mathbf{C}= \begin{bmatrix}cos(\theta)&-sin(\theta)&sin(\theta)c\\ sin(\theta)&-cos(\theta)&cos(\theta)c\\ 0&0&1\end{bmatrix}  \tag{5}\]The process of estimating the value of parameters \(c\) and \(\theta\) is similar to bundle adjustment; however, it is performed only on the basis of two images. It is described in the next section.