# Evaluation of RGB-D Multi-Camera Pose Estimation for 3D Reconstruction
## 1 Introduction  
The idea and use of 3D imaging dates back to the 19 th century, and laser scanning to the 1960s [1], but only recently has it been capable of revolutionising the interaction between robots, the environment and humans. Many advances in computational power, sensor precision and affordability have made this possible [2, 3, 4]. The recent development of RGB-D cameras has provided visual sensor devices capable of generating pixel-wise depth information, together with a colour image. The technology behind these cameras has been constantly improving, with developers working to reduce noise and increase precision, e.g., Microsoft Kinect Azure and Intel RealSense L515 and D455 [3]. The depth information from those devices can be used to generate a three-dimensional projection of the captured object. Understanding the well-known pinhole camera model [5] is important to understand how the reprojection works, and how it is affected by noise in the depth data. The model describes the transformation from the 3D world to the 2D image plane, as shown in Figure 1 and in the Equation (1) [6]. It can also be used to calculate the inverse path for reprojecting from 2D to 3D. Equation (1) has two matrices: one for the intrinsic parameters and another for the extrinsic parameters. The first contains the camera's internal parameters, which are constant for each camera. The second describes where the camera is in the world, i.e., the pose of the camera in relation to an origin coordinate system. The direction vector of the ray from the camera projection centre can be found using these parameters, but the length of the vector cannot. This information is lost in the conversion from 3D to 2D. However, when using an RGB-D camera this information is saved, as a depth that determines where the point lies in the world. The set of points reprojected from these data is called a point cloud. Other parameters that are important for reprojection are the distortion coefficients which are used to correct the radial and tangential distortions of the lens [7]. In this work, the FRAMOS Industrial Depth Camera D415e, which was built with Intel(r) RealSense(tm) technology, was used. The Intel(r) RealSense module claims to have no lens distortions [8; 9]. \[s \begin{bmatrix}u\\ v\\ 1\end{bmatrix} =\underbrace{ \begin{bmatrix}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix} }_{\text{intrinsic}}\underbrace{ \begin{bmatrix}r_{11}&r_{12}& r_{13}&t_{1}\\ r_{21}&r_{22}&r_{23}&t_{2}\\ r_{31}&r_{32}&r_{33}&t_{3}\end{bmatrix} }_{\text{extrinsic}} \begin{bmatrix} X\\ Y\\ 1\end{bmatrix}  \tag{1}\]  
The intrinsic parameters of a camera are normally represented as a 3 \(\times\) 3 matrix, as shown in Equation (2). \[K= \begin{bmatrix}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix}  \tag{2}\]  
where \(f_{x}\) and \(f_{y}\) are the focal lengths in the \(x\) and \(y\) directions, respectively. Furthermore, \(c_{x}\) and \(c_{y}\) are the optical centres of the image plane, shown as a solid red line in Figure 1.  
As illustrated in Figure 1, the focal length is the distance from the camera lens to the image plane; since the pixel is not necessarily square, the focal length is divided by the pixel size in \(x\) and \(y\), resulting in the variables \(f_{x}\) and \(f_{y}\), respectively, expressing the values in pixels. The extrinsic parameters are typically represented by a homogeneous transformation matrix, shown in Equation (3), and this was well explained by [PERSON] and [PERSON] (2015) [10]. This contains the rotation matrix \(R_{3\times 3}\) and the translation vector \(T_{3\times 1}\), representing the  
Figure 1: Pinhole camera’s projective geometry. camera's transformation in relation to the origin of the reference coordinate system in the desired scene. \[D= \begin{bmatrix}R_{3\times 3}&T_{3\times 1}\\ 0_{1\times 3}&1\end{bmatrix}  \tag{3}\]  
The distortion coefficients are the parameters used to describe the radial and the tangential distortion. They are represented as \(k_{n}\) and \(p_{n}\), respectively.