# Evaluation of RGB-D Multi-Camera Pose Estimation for 3D Reconstruction
## 1 Introduction
### Related Work  
The problem of scene reconstruction has been well studied in different scenarios. To solve the challenge of registering two or more point clouds together, the six degrees of freedom (DoF) transformation between them has to be found. This can be calculated using the point clouds by either selecting the matching points manually or using algorithms to find possible matching points. Another method is to calculate a known position for the sensors.  
[PERSON] and [PERSON] [16] and [PERSON] and [PERSON] [17] proposed one of the most widely used algorithms for the registration of 3D shapes, the iterative closest point (ICP). It tries to find the best match between the point clouds by finding the closest points from one point cloud to the other, then it determines the transformation matrix that minimises the distance between the points, and finally, it iterates until it converges. Due to fact that it relieson the data association of the points between point clouds, a good overlap is necessary for convergence.  
[PERSON] and [PERSON] [18] proposed a replacement of ICP called the deep closest point (DCP) method, which uses a three-step approach, first embedding the point clouds into a common space, then capturing co-contextual information in an attention-based module and finally using singular value decomposition to find the transformation matrix.  
[PERSON] et al. [19] used the Pointnet [20] method with the Lukas and Kanade (LK) [21] algorithm to solve the registration problem.  
Other methods include that of [PERSON] et al. [22], who used a three-dimensional normal distribution transform representation of the distance between the models, followed by a Newton optimisation, and that of [PERSON] et al. [23], who proposed an algorithm based on normal vector and particle swarm optimisation. These methods all rely on having a sufficient overlap between the point clouds to solve the problem. Moreover, these methods tend to be slow.  
Performing an estimation of the position of the sensorprovides a more reliable transformation for the point clouds. Initially, work on camera calibration was focused on finding the intrinsic parameters of a single camera. [PERSON] [24] was the first to propose the solution to this challenge using a chessboard pattern, i.e., a planar target, through least-square approximation.  
With the intrinsic parameters, in theory, it would be possible to calculate the pose of the camera with four coplanar points that are not collinear, but [PERSON] and [PERSON] [25], discussing pose ambiguities, proved that two local minima exist, and proposed an algorithm to solve this problem.  
Fiducial markers became popular for camera pose estimations, and several markers have been proposed, including point-based [26], circle-based [27], and square-based [28; 29] markers, which can determine the pose using the four corners and have the ID in the middle of the marker.  
[PERSON] et al. [28] proposed a system with configurable marker dictionaries, specially designed for camera localization. The authors developed a marker generator, as well as an automatic detection algorithm. These ArUco markers form the basis of the charuco board used in this work.  
Other publications have proposed ways to solve the problem of pose estimation using an arbitrary scene with texture [30; 31; 32]. These are not relevant to this project, since it was designed in a contained environment.  
In this work, we propose and carry out a novel evaluation method for multiple camera perspectives. We used a charuco to identify and label the points on which the metrics were based, and calculated the errors of three different methods in order to carry out the transformation of the cameras to the global coordinate system.