## 2 Research Methodology
### Point Cloud Contrustion and Pre-Processing
#### 2.2.1 Stereo Calibration Principle  
Two ToF depth cameras will generate two distinguished point clouds. Combining these two point clouds will create a point cloud with more details so the data for the location system will be more accurate. Calibration for combining two point clouds is performed using the stereo calibration principle [10]. Stereo calibration will help the study to find out the transformation matrix rotation, **R**, and translation, **t**, between two RGB cameras of the Azure Kinect. Firstly, to use stereo calibration, the parameters of the transformation between an object in 3D space and the 2D image observed by the camera from visual information have to be determined. The pinhole camera model used to calibrate a camera is described in Figure 2.  
Considering that \(\textbf{{P}}(X,Y,Z)\) is the 3D world coordinates point of the object, the 3D coordinate of the same point in the camera frame, \(\textbf{{P}}_{\textit{cam}}\), is:  
\[\textbf{{P}}_{\textit{cam}}=\big{[} \begin{array}{cc}\textbf{{R}}&\textbf{{t }}\end{array} \big{]}\textbf{{P}} \tag{1}\]  
where \(\textbf{{R}}\) is the 3 \(\times\) 3 rotation matrix and \(\textbf{{t}}\) is the 3 \(\times\) 1 translation matrix. Let \(\textbf{{p}}(x,y)\) be the position of the 3D point in the image coordinate, this will result in the 3D to 2D mapping; here, \(\textbf{{K}}\) is the intrinsic matrix of the camera:  
\[\textbf{{p}}=\textbf{{K}}\big{[} \begin{array}{cc}\textbf{{R}}&\textbf{{t}} \end{array} \big{]}\textbf{{P}} \tag{2}\]  
Figure 1: Experiment device setup of stereo vision scanning for dynamic workpiece. Figure 2: Calibration model of a camera in the dynamic workpiece modeling application. The definition of the intrinsic matrix of the camera is as follows:  
\[\mathbf{K}=\left[ \begin{array}{ccc}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{array} \right] \tag{3}\]  
where \(f_{x}\) is the \(X\)-axis focal length and \(f_{y}\) is the \(Y\)-axis focal length of the camera, and \((c_{x},c_{y})\) is the coordinate of the principal point. The camera calibration's purpose is to find out the intrinsic parameter, \(\mathbf{K}\) matrix, and extrinsic parameters, \(\mathbf{R}\) and \(\mathbf{t}\) matrices. The calculation uses linear algebra to find all the parameters. To achieve these parameters, multiple images of a checkerboard with a fixed square size will be taken and all the calibration patterns (the cross-point of the black/white consecutive squares) in each image will be found. These calibration patterns in the image correspond to some 3D points in the world. These point-to-point correspondences will be stored, and after that, the non-linear algorithm is used to solve the calibration parameters. After the intrinsic matrices \(\mathbf{K_{1}}\) and \(\mathbf{K_{2}}\) of the two cameras are known, the next problem is figuring out the rotation, \(\mathbf{R_{12}}\), and translation, \(\mathbf{t_{12}}\), between camera 1 and camera 2, which will contribute to finding point correspondences in the left and right image planes. The schematic diagram of the stereo calibration of the two cameras is described in Figure 3.  
Let \(\mathbf{p_{1}}\) and \(\mathbf{p_{2}}\) be a point in the camera 1 and camera 2 image coordinates respectively, which is the mapping of world coordinate \(\mathbf{P}\) in 3D space (Figure 3). The fundamental matrix \(\mathbf{F}\) is defined as a mapping from a point in an image plane to an epipolar line in the other image. Therefore, the following equation can be obtained:  
\[\mathbf{l_{2}}=\mathbf{F}\mathbf{p_{1}} \tag{4}\]  
The form of the fundamental matrix in terms of the two camera projection matrices: \(\mathbf{M_{1}}\) and \(\mathbf{M_{2}}\), may be derived algebraically.