## Materials and Methods
### Image Acquisition  
A sequence of image pairs is recorded synchronously with the two cameras of the stereo setup.  
**2. Rectification.** We rectify the camera images, that is, we apply a geometric transformation to the images to simulate two cameras with parallel image planes. The rectification algorithm also corrects lens distortion. After rectification, the images have the important property that an object point, which is visible in a specific image row \(r\) of the first camera's images, also appears in the  
Figure 1: General outline of the 3D reconstruction.  
identical row \(r\) in the other camera's images. This means that the epipolar lines [32; 33] are parallel to the image rows.
3. **Coarse Correspondence Search.** We search corresponding pixels between the cameras. For each pixel in the left camera at position \((c_{L},r)\), we search a pixel in the right camera at position \((c_{R},r)\) which shows the same object point. Due to rectification (step 2), this pixel can be found on the same image row \(r\). The column \(c_{R}\) has to be determined. In the coarse correspondence search, we consider any \(c_{R}\) with a distance of up to two pixels from the real correspondence correct. The difference of the column numbers \(d=c_{L}-c_{R}\) is called \"disparity\". The result of the coarse correspondence search is the coarse disparity map.
4. **Correspondence Refinement.** We refine the result by searching an interval around the coarse correspondence. We then interpolate between pixels in the right image and find the best match amongst the interpolated sub-pixels. The result is the refined disparity map.
5. **Calculation of 3D points.** For each pixel in the disparity map, we calculate a 3D point by triangulation.  
Open-source code which facilitates writing algorithms based on this approach is readily available [34; 35]. As we work with statistical patterns, we do not have a-priori information about the projected patterns. This means that, in the coarse correspondence search (step 3), the best matching pixel needs to be searched amongst all candidates. In practice, this is the most time consuming part of the algorithm. We therefore focus on the coarse correspondence search in this paper. Its goal is to find the approximate position of the correspondence up to an accuracy of a few pixels. All further correspondence refinement (step 4) happens afterward in a separate algorithmic step where sub-pixel accuracy is achieved.  
Starting with a pixel in the left camera's rectified image sequence, we search a set of candidate pixels in the same row of the right camera's rectified image sequence. The set of candidate pixels is determined by the sensor geometry and the size of the measurement volume. This means that only a restricted interval of disparities have to be searched.  
To compare two pixels, a measure of similarity is required. This is where the algorithms differ from each other (Sections 2.2 and 2.3). The pixel which is the most similar according to the measure is picked amongst the candidates.  
When the matching pixel has been found (in the right camera's image sequence), we validate it by reverse searching the best match for it in the left camera image sequence. Only if the so found reverse match is equal (or at a maximum distance of 2) to the original pixel, the match is accepted.