# A Method for Adapting Stereo Matching Algorithms to Real Environments
## 3 Materials and Methods
### The Method of Auxiliary Calibration  
In this paper, a two-stage stereo camera calibrating method is proposed. The first stage uses regular pattern-based calibration, and the second stage uses additional calibration based on bundle adjustment and self-calibration. In this paper, this second stage is called the auxiliary calibration step (ACS). It is executed right before the stereo matching algorithm. In fact, it forms a part of the stereo matching algorithm. As described in the introduction, commonly used stereo camera calibration methods do not guarantee calibrating cameras to the extent achieved for the purpose of preparing testbeds. Stereo matching algorithms are frequently designed for processing images calibrated for high quality. The auxiliarycalibration step aims to additionally transform images obtained from stereo cameras used in real environments in order to improve their calibration quality. The process of obtaining disparity maps using the ACS is presented in Figure 1.  
The first step required to use the ACS is the preparation of a calibration pattern in order to obtain transformation parameters for pattern-based camera calibration. Uncalibrated images from a stereo camera are then transformed into calibrated ones. The images after these transformations are typically used to obtain disparity maps using a stereo matching algorithm. However, with the use of the ACS module, the images are further calibrated. This module is integrated into a stereo matching algorithm because it is applied to every pair of images processed by a stereo matching algorithm; this is unlike regular calibration, which aims to generally identify distortions created by stereo cameras. The main purpose of adding the ACS module is to reduce discrepancies in the y-axis. This kind of distortion severely affects the possibility of obtaining a proper disparity map. As presented in Section 2.2, [PERSON] et al. used bundle adjustment after pattern-based calibration in order to handle distortions in the y-axis [27]. The work described in [27] aimed to prepare a testbed. However, the ACS module is expected to work in real time for every pair of images processed by a stereo matching algorithm. Therefore, it is not acceptable for the ACS to be considered part of an iterative algorithm, as is the case for bundle adjustment. This paper proposes a solution based on bundle adjustment, but it focuses only on distortions in the y-axis. The functions of the ACS module are presented in this paper using the same notations as in Section 2.1. If pattern-based calibration is performed according to the transformations presented in Equation (1), then this calibration applies to both of the cameras included in a stereo camera. Therefore, Equation (1) takes the form presented in Equation (2). \[ \begin{split}\mathbf{p_{0}}&=\mathbf{A_{0}}[ \mathbf{R_{0}}|\mathbf{t_{0}}]\mathbf{P_{w0}}\\ \mathbf{p_{1}}&=\mathbf{A_{1}}[\mathbf{R_{1}}| \mathbf{t_{1}}]\mathbf{P_{w1}}\end{split}  \tag{2}\]  
where index 0 refers to the left camera, which is assumed to be a reference camera. The reference camera performs the function of being a point of view in a stereo camera. Index  
Figure 1: The process of using the ACS module. 1 in Equation (1) denotes the right camera, which is a side camera. An image from this camera is used to determine disparities in the points visible in an image taken from a reference camera. Equation (1) presented in Section 2.1 applies to homogeneous coordinates. Therefore, the parameter \(\mathbf{s}\) is excluded from further calculations presented in this paper, and it is not present in Equation (2). Equation (2) sufficiently describes the transformations required to obtain a calibrated pair of images from a stereo camera if a perfect pattern-based calibration exists. However, inaccuracies remain after calibration. These inaccuracies are modeled by vector \(\mathbf{v_{1}}\), which is included in Equation (3). \[