# Camera Calibration in High-Speed Robotic Assembly Operations
## 3 Materials and Methods  
The study develops an algorithm for camera calibration in applications that implement robotic vision. The main objective of the proposed algorithm is to mathematically determine the measurement plane for the vision camera. The coordinates of the determined plane are then expressed in relation to robot coordinates. The required data are collected through experimental measurement. The experimental setup uses an xArm (UFACTORY, Guangdong, China) collaborative robot with 5 DOF. A video processing unit (O3R222 from IFM, Essen, Germany) with 2 camera types was mounted on the robot using a custom mechanical interface. The video unit is illustrated in Figure 2.  
The processing unit is based on a Dual-Core NVIDIA Denver 2 64-Bit; ARM Cortex A57 with a NVIDIA Pascal 256 CUDA Cores (1.3 TFLOPs) GPU and an Nvidia Jetson TX2 4 GB Module. Both cameras can capture 2D and 3D images with different fields of view. Using a camera with 2D and 3D capabilities is required for the procedure since the 2D feature is used to capture the image itself, while the 3D feature is used to determine the distance to the measurement points. The 3D camera uses a PMD 3D chip which uses the ToF (Time of Flight) technology to measure the distances. The ToF sensor emits an infrared-modulated light that is reflected by surrounding surfaces. By measuring the time difference between the emitted and the response light, a distance is determined with high accuracy. The representation of a 3D camera consists of a point cloud in which each pixel's color represents the distance to the first object encountered, as shown in Figure 3.  
Figure 2: O3R222 3D/2D camera. Legend: 1—2D camera; 2—3D camera; 3—Light source. The accuracy of this camera model is highly dependent on the distance to the target. At distances under 1 m, the repeatability is less than 7 mm. Between 1 and 2 m, the accuracy decreases to 15 mm, and at distances beyond 4 m, it can reach up to 51 mm. Measurement repeatability can be significantly improved by using light filters. For camera calibration, a repeatability of less than 10 mm is considered acceptable. The control software for the robot and the camera software were installed on separate laptops for more convenient operation. The experimental setup is illustrated in Figure 4. The purpose of the experimental setup is to determine the position and orientation of the camera relative to the measuring plane. To achieve this goal, a blank sheet of paper was used as a marker for the measuring area. The main purpose of the study is to determine the position and orientation of a mobile camera relative to the plane on which the detected objects are placed. Once the position and orientation are determined, the calibration error due to perspective deformation can be compensated. Considering these aspects as well as the main goal of the research, the experimental procedures were conducted without any work objects placed in the testing area. Figure 4: Experimental setup illustrating the camera mounted on the xArm 5 robot and the control software running. Figure 3: Differences between the 2D and 3D image of an object. Legend: 1—2D image; 2—3D image processed by the camera software. * The distance from the camera to each measurement point was determined using the camera software. The first set of measurement points (the two pairs positioned at a distance of 20 pixels from the origin along the X and Y axes) and the camera position relative to the measurement plane are illustrated in Figure 4. The Xc and Yc axes belong to the camera reference frame. The Xm and Ym axes belong to the measurement plane reference frame. * This approach is based on the law of cosines in order to determine the camera rotation angles around the X and Y axes relative to the measurement plane. The law of cosines determines a triangle when all the sides or two sides and the included angle are known.