# Global Measurement Method for Large-Scale Components Based on a Multiple Field of View Combination

Authors: Yang Zhang; Wei Liu; Zhiguang Lan; Zhiyuan Zhang; Fan Ye; Haiyang Zhao; Xiaodong Li; Zhenyuan Jia
Journal: Journal of Sensors
 (2017)

---

## Abstract

Considering the limited measurement range of a machine vision method for the three-dimensional (3D) surface measurement of large-scale components, a noncontact and flexible global measurement method combining a multiple field of view (FOV) is proposed in this paper. The measurement system consists of two theodolites and a binocular vision system with a transfer mark. The process of multiple FOV combinations is described, and a new global calibration method is proposed to solve the coordinate system unification issue of different instruments in the measurement system. In addition, a high-precision image acquisition method, which is based on laser stripe scanning and centre line extraction, is discussed to guarantee the measurement efficiency. With the measured 3D data, surface reconstruction of large-scale components is accomplished by data integration. Experiments are also conducted to verify the precision and effectiveness of the global measurement method.


---

## Body

## 1. Introduction

In the field of modern manufacturing and assembly, the accuracy of large-scale components of dimensions is required to guarantee automatic assembly of high-end equipment. To ensure that the components are manufactured as designed, the dimensions of the components have to be accurately measured [1, 2]. However, the measurement range for large components is too large to rapidly complete onsite measurements using a single instrument. For large high-precision parts that are used in aerospace and aviation, mark points are not allowed to be pasted on the surfaces of the components; that is, the measuring methods that only work with the marks pasted on the surfaces are not suitable. Large-scale components, such as wings of airplanes and large antenna radomes, are usually clamped onto a bracket to prevent deformation prior to assembly. As a result, the surfaces of these components are partly blocked from view, and the area of interest for the measurement instrument is invisible from a specific viewing angle. Thus, studies of noncontact and flexible global measurement methods for 3D surface measurement of large-scale components are important for high-quality assembly in aerospace and aviation.Machine vision measurement has such advantages as noncontact, high efficiency, and high accuracy, so it has been extensively applied to industry [3–5]. Recently, researchers have proposed large-scale component measurement methods that are based on machine vision [6–8]. There are three main machine vision methods using in the measurement of large aviation components; they are photogrammetry [9–11], structured light method, and binocular vision [12–15]. Photogrammetry is a method to collect quantity of reflective marks pasted on the surface of a part using a single camera and reconstruct the marks to represent the surface of the part. A system called V-STARS adopted this method to measure the lay-up tool of Boeing B320 aircraft [16]. The system consists of a high resolution camera and reflective marks which need to be affixed onto the surface of the measured part. The marking points can be reconstructed by the images captured by the camera. The system has the advantages of good portability. However, pasting the marking points is time-consuming, and it is difficult to measure the entire surface using marking points, especially the boundary of the surface; structured light method is to reconstruct the projecting light by building the triangulation between the monocular camera and the active projecting light (laser/grating). During the measurement, the camera and the position of the projecting light have to remain relatively still, which brings a problem that, to guarantee the measurement accuracy, the FOVs for measurement are a few. To measure parts using large FOV, measurement equipment needs to be driven by highly accurate movement mechanisms to scan parts, as a novel 3D scanning measurement technique for large components proposed by Jiang et al. [17]. However, in field of aviation assembly, limited to complex environments, the method that uses large and highly accurate movement mechanisms to achieve on site measurement for large components is impractical. A German company Gom measures large components with structured lights and photogrammetry and has developed a measurement system called ATOS based on profilometry with grating and another system called TRITOP based on photogrammetry with reflective marks, respectively. NASA combined the ATOS system with the TRITOP system to measure X-38 aircraft [18], whereas too many moves of the method are needed, and quantity of mark points has to be stuck on the surfaces of parts ahead of measurement. Binocular vision is a method that uses binocular cameras to collect features projected onto surface to be measured and reconstruct the surface; a light probe based large FOV 3D vision measurement system proposed by Feng and Wei, which is based on binocular vision, can measure the curved surface of a large-scale object [19]. However, keeping the aided target steady during the measuring process is difficult, and only the coordinates of the points on the curved surface can be measured using this method. The measuring process is complex, and the measurement accuracy is unstable. Therefore, a novel global measurement method for 3D surfaces of large-scale components, which is based on a multiple field of view (FOV) combination, is proposed in this paper.First, the principle and measurement process of the proposed method are introduced. Second, the calibration method of the measurement system is presented. Third, the algorithms of laser stripe extraction, image matching, and reconstruction are described. Last, the flexibility and precision of the measurement method are verified by experiments, as discussed in Section5.


## 2. Measurement Principle

The measurement system is a binocular vision system that consists of two theodolites and a transfer mark that is designed for the transition of views. In the measuring process, two theodolites that are placed in the back are employed as the global control station for field combination and data integration. The binocular vision system with the transfer mark is placed in the front to capture an image of the component surface. The global measurement coordinate system is established on the left theodolite to ensure that the binocular vision system can be placed in any position in the measurement range of the theodolites. By transferring the surface information that is captured by cameras in different positions into the global coordinate system, high-precision data integration can be achieved. The measurement principle of the system is shown in Figure1.Figure 1
Schematic of the measurement system.As the measurement system consists of different types of instruments, the measurement coordinate systems, respectively, established on these instruments are different as well. To reconstruct an entire surface, the coordinates that are measured by these instruments must be unified in the global coordinate system. TheCraig expression method of coordinate transformation is adopted in this paper to express the transformation relation [20]. The coordinate vector of any point P in the coordinate system F is expressed as PF(1)PF=OP→F=xyz⟺OP→=xi+yj+zk,where (x,y,z) is the coordinate of point P. Two theodolites are employed as the global measurement control station of the measurement system. The global coordinate system oGxGyGzG is established on the left theodolite.The binocular vision system consists of two industry complementary metal oxide semiconductor (CMOS) cameras. The coordinate system of the left camera—oCLxCLyCLzCL—is established on the left camera, with the origin on the optical centre of the camera, and the coordinate axis is in the same directions as the axis of the image sensor. Similarly, the right camera coordinate system—oCRxCRyCRzCR—is established on the right camera.In the process of measurement, the large component to be measured must be divided into multiple fields of view and measured several times; thus, the cameras must be moved to different positions. To connect the mobile camera coordinate systems with the fixed global coordinate system, a transfer mark is taken into use of the binocular vision system. The relative position between the mark and the cameras remains unchanged during the measurement. By measuring the feature points on the mark, two theodolites in the back can obtain the position and the direction of the binocular vision system. The coordinate system of the transfer marksoMxMyMzM and HGM is the transformation matrix from the transfer mark coordinate system to the global coordinate system.The measurement process of the proposed system is as follows: first, the system is calibrated, which includes the calibration of two theodolites, the binocular cameras, and their transformation relation which is calibrated using the transfer mark. Second, the surface of the large-scale component is artificially divided into several parts and separately measured by the binocular cameras at different positions; based on binocular vision method with assisted laser, the 3D point cloud data of measured part is obtained [21]. Third, feature points on the transfer mark are measured by two theodolites for coordinate system transformation. Last, the data of every measured part obtained by the cameras at different positions are integrated into the global coordinate system. With fusion of the obtained data, the component with overall size is measured. With the data obtained, the entire surface of the large component can be reconstructed.


## 3. Calibration of the Measurement System

The calibration of the measurement system is the foundation of the 3D measurements. The proposed measurement system consists of different instruments. Thus, the high-precision calibration of all instruments is essential for ensuring the accuracy of large field measurements.To measure the large surface of a component, the binocular vision system must be moved several times to acquire regional characteristic information. Due to the changed measuring position, the spatial location of the camera coordinate system varies in the meantime. Thus, an accurate transformation relationship between the camera coordinate system and the global coordinate system must be established.In this paper, the global coordinate system is established on the left theodolite. The two-theodolite system is calibrated according to the two-theodolite spatial three-dimensional (3D) measurement model. The binocular cameras are calibrated using Zhang’s calibration method [22]. By measuring the coordinates of the feature points on the transfer mark with the theodolites, the relation among the theodolites, the binocular cameras, and the transfer mark can be obtained. In this manner, global calibration is accomplished.


### 3.1. Calibration of Two Theodolites

The global coordinate system is established on the left theodolite according to the perspective projection model [23, 24]. As shown in Figure 2, the 3D coordinate system oTLxTLyTLzTL of the left theodolite is constructed with the origin on the observation centre oTL. The two-dimensional (2D) image coordinate system OTLXTLYTL of the left theodolite is established at zTL=1 in oTLxTLyTLzTL. The relation between OTLXTLYTL and oTLxTLyTLzTL can be calculated based on the perspective projection model as(2)XTLYTL1=λxTLyTLzTL=cot⁡αTLtan⁡φTLsin⁡αTL1,where λ is the scale factor, αTL is the horizontal angle, and φTL is the vertical angle obtained by the theodolite. For ensuring the accuracy of measurement, the degree of the horizontal angle and the vertical angle should not be about 0 degrees or 90 degrees. XTL,YTL and xTL,yTL,zTL are the coordinates in the 2D image coordinate system of the left theodolite and the coordinates in the 3D coordinate system of the left theodolite, respectively.Figure 2
Coordinate systems for the two theodolites.Similarly, the 3D coordinate systemoTRxTRyTRzTR and the 2D image coordinate system OTRXTRYTR of the right theodolite are established, and the relationship between these two systems is calculated.The coordinate vector of pointPin oTLxTLyTLzTL is expressed as PTL, and PTR is the coordinate vector of pointPin oTRxTRyTRzTR. The relationship between PTL and PTR is expressed as(3)PTR1=HTRTLPTL1=RTRTLtTRTL0T1PTL1,where RTRTL=r1r2r3r4r5r6r7r8r9, tTRTL=t1t2t3T, RTRTL is the coordinate rotation matrix, and tTRTL is the coordinate translation matrix.According to (2) and (3), we can obtain(4)t1-XTRt3r7XTLYTR+r8YTLYTR+r9YTR-r4XTL-r5YTL-r6=t2-YTRt3r7XTLXTR+r8XTRYTL+r9XTR-r1XTL-r2YTL-r3.By measuring a target of certain length thrice using the two theodolites, the rotation matrixRTRTL and translation matrix tTRTL are obtained.In the global coordinate system, the coordinate vector of any pointP in the FOV is expressed as(5)PG=cot⁡αTL·zGtan⁡βTLsin⁡αTL·zGt1-cot⁡αTRcot⁡αTRr7cot⁡αTL+r8tan⁡φTL/sin⁡αTL+r9-r1cot⁡αTL+r2tan⁡φTL/sin⁡αTL+r3,where αTL is the horizontal angle, φTL is the vertical angle obtained by the left theodolite, and αTR is the horizontal angle obtained by the right theodolite.


### 3.2. Calibration of Binocular Cameras

Two industrial cameras with high resolution are employed in the 3D surface measurement system. To measure the surface of the large-scale component that is blocked, the binocular cameras must be moved to different positions.The pixel coordinates vector of pointP in the image coordinate system is pC=uPC,vPCT, and the coordinates vector in the World Coordinate System is PW=xPW,yPW,zPWT. After the calibration of the cameras, the relation between pC and PW can be described by(6)pC1=K·H·PW1=K·Rt0T1·PW1,where K is the intrinsic matrix of the binocular vision system and H is the extrinsic matrix. R is the coordinate rotation matrix and t is the coordinate translation matrix. The coordinate system of the left camera is considered as the global coordinate system, so the transformation matrix, the rotation matrix, and the translation matrix from the coordinate system of the right camera to the global coordinate system are set to be HCLCR, RCLCR, and tCLCR, respectively.


### 3.3. Calibration of the Relationship between the Binocular Cameras and the Transfer Mark

In the process of measurement, the transfer mark is essential for the transformation from the camera coordinate system to the global coordinate system.As the transfer mark is not in the opposite direction to the binocular cameras, the feature points on the mark cannot be captured by the cameras. The transformation relation between the camera coordinate system and the transfer mark coordinate system cannot be directly established. As a result, the calibration process must be divided into the following two steps.(1) Transformation from the Transfer Mark Coordinate System to the Global Coordinate System. To guarantee the transformation precision, a high-precision checkerboard is employed as the transfer mark. The checkerboard is placed in the opposite direction to the binocular cameras on the bracket, as shown in Figure 3.Figure 3
Binocular-camera measurement system with the transfer mark.The coordinate system of the transfer markoMxMyMzM is established with the origin at the centre point of the checkerboard OM. The directions of the XM axis and the YM axis are defined, as shown in Figure 3. The direction of ZM axis is defined by the coordinate right-hand rule.By measuring the feature points on the transfer mark with two theodolites, the transformation matrixHGM between the transfer mark coordinate system and the global coordinate system is acquired. The transformation relation can be expressed as(7)PG1=HGMPM1=RGMtGM0T1PM1,where RGM is the rotation matrix from the transfer mark coordinate system to the global coordinate system and tGM is the translation matrix.(2) Transformation from the Binocular-Camera Coordinate System to the Global Coordinate System. The binocular-camera coordinate system is constructed on the left camera. The xCL axis on the image plane is horizontal, and the yCL axis on the image plane is vertical, as shown in Figure 3. First, feature points on the transfer mark are measured by two theodolites and reconstructed in the global coordinate system. Second, the same points are measured by binocular cameras and reconstructed in the binocular-camera coordinate system. With the two groups of coordinates, the transformation relation HGC between the two coordinate systems can be written as follows:(8)PG1=HGCPC1=RGCtGC0T1PC1,where RGC is the rotation matrix from the binocular-camera coordinate system to the global coordinate system and tGC is the translation matrix.(3) Transformation from the Binocular-Camera Coordinate System to the Transfer Mark Coordinate System. As HGM and HGC were determined in the preceding section, the transformation matrix HMC from the binocular-camera coordinate system to the transfer mark coordinate system can be obtained based on the invariance of the spatial vector. The transformation relation can be expressed as follows:(9)PM1=HMCPC1=H-1GMHGCPC1.After the calibration, the relative position between the binocular cameras and the transfer mark remains unchanged. In the process of measurement, data transformation from the binocular-camera coordinate system to the global coordinate system is achieved by calibratingHGM with the two theodolites.


## 3.1. Calibration of Two Theodolites

The global coordinate system is established on the left theodolite according to the perspective projection model [23, 24]. As shown in Figure 2, the 3D coordinate system oTLxTLyTLzTL of the left theodolite is constructed with the origin on the observation centre oTL. The two-dimensional (2D) image coordinate system OTLXTLYTL of the left theodolite is established at zTL=1 in oTLxTLyTLzTL. The relation between OTLXTLYTL and oTLxTLyTLzTL can be calculated based on the perspective projection model as(2)XTLYTL1=λxTLyTLzTL=cot⁡αTLtan⁡φTLsin⁡αTL1,where λ is the scale factor, αTL is the horizontal angle, and φTL is the vertical angle obtained by the theodolite. For ensuring the accuracy of measurement, the degree of the horizontal angle and the vertical angle should not be about 0 degrees or 90 degrees. XTL,YTL and xTL,yTL,zTL are the coordinates in the 2D image coordinate system of the left theodolite and the coordinates in the 3D coordinate system of the left theodolite, respectively.Figure 2
Coordinate systems for the two theodolites.Similarly, the 3D coordinate systemoTRxTRyTRzTR and the 2D image coordinate system OTRXTRYTR of the right theodolite are established, and the relationship between these two systems is calculated.The coordinate vector of pointPin oTLxTLyTLzTL is expressed as PTL, and PTR is the coordinate vector of pointPin oTRxTRyTRzTR. The relationship between PTL and PTR is expressed as(3)PTR1=HTRTLPTL1=RTRTLtTRTL0T1PTL1,where RTRTL=r1r2r3r4r5r6r7r8r9, tTRTL=t1t2t3T, RTRTL is the coordinate rotation matrix, and tTRTL is the coordinate translation matrix.According to (2) and (3), we can obtain(4)t1-XTRt3r7XTLYTR+r8YTLYTR+r9YTR-r4XTL-r5YTL-r6=t2-YTRt3r7XTLXTR+r8XTRYTL+r9XTR-r1XTL-r2YTL-r3.By measuring a target of certain length thrice using the two theodolites, the rotation matrixRTRTL and translation matrix tTRTL are obtained.In the global coordinate system, the coordinate vector of any pointP in the FOV is expressed as(5)PG=cot⁡αTL·zGtan⁡βTLsin⁡αTL·zGt1-cot⁡αTRcot⁡αTRr7cot⁡αTL+r8tan⁡φTL/sin⁡αTL+r9-r1cot⁡αTL+r2tan⁡φTL/sin⁡αTL+r3,where αTL is the horizontal angle, φTL is the vertical angle obtained by the left theodolite, and αTR is the horizontal angle obtained by the right theodolite.


## 3.2. Calibration of Binocular Cameras

Two industrial cameras with high resolution are employed in the 3D surface measurement system. To measure the surface of the large-scale component that is blocked, the binocular cameras must be moved to different positions.The pixel coordinates vector of pointP in the image coordinate system is pC=uPC,vPCT, and the coordinates vector in the World Coordinate System is PW=xPW,yPW,zPWT. After the calibration of the cameras, the relation between pC and PW can be described by(6)pC1=K·H·PW1=K·Rt0T1·PW1,where K is the intrinsic matrix of the binocular vision system and H is the extrinsic matrix. R is the coordinate rotation matrix and t is the coordinate translation matrix. The coordinate system of the left camera is considered as the global coordinate system, so the transformation matrix, the rotation matrix, and the translation matrix from the coordinate system of the right camera to the global coordinate system are set to be HCLCR, RCLCR, and tCLCR, respectively.


## 3.3. Calibration of the Relationship between the Binocular Cameras and the Transfer Mark

In the process of measurement, the transfer mark is essential for the transformation from the camera coordinate system to the global coordinate system.As the transfer mark is not in the opposite direction to the binocular cameras, the feature points on the mark cannot be captured by the cameras. The transformation relation between the camera coordinate system and the transfer mark coordinate system cannot be directly established. As a result, the calibration process must be divided into the following two steps.(1) Transformation from the Transfer Mark Coordinate System to the Global Coordinate System. To guarantee the transformation precision, a high-precision checkerboard is employed as the transfer mark. The checkerboard is placed in the opposite direction to the binocular cameras on the bracket, as shown in Figure 3.Figure 3
Binocular-camera measurement system with the transfer mark.The coordinate system of the transfer markoMxMyMzM is established with the origin at the centre point of the checkerboard OM. The directions of the XM axis and the YM axis are defined, as shown in Figure 3. The direction of ZM axis is defined by the coordinate right-hand rule.By measuring the feature points on the transfer mark with two theodolites, the transformation matrixHGM between the transfer mark coordinate system and the global coordinate system is acquired. The transformation relation can be expressed as(7)PG1=HGMPM1=RGMtGM0T1PM1,where RGM is the rotation matrix from the transfer mark coordinate system to the global coordinate system and tGM is the translation matrix.(2) Transformation from the Binocular-Camera Coordinate System to the Global Coordinate System. The binocular-camera coordinate system is constructed on the left camera. The xCL axis on the image plane is horizontal, and the yCL axis on the image plane is vertical, as shown in Figure 3. First, feature points on the transfer mark are measured by two theodolites and reconstructed in the global coordinate system. Second, the same points are measured by binocular cameras and reconstructed in the binocular-camera coordinate system. With the two groups of coordinates, the transformation relation HGC between the two coordinate systems can be written as follows:(8)PG1=HGCPC1=RGCtGC0T1PC1,where RGC is the rotation matrix from the binocular-camera coordinate system to the global coordinate system and tGC is the translation matrix.(3) Transformation from the Binocular-Camera Coordinate System to the Transfer Mark Coordinate System. As HGM and HGC were determined in the preceding section, the transformation matrix HMC from the binocular-camera coordinate system to the transfer mark coordinate system can be obtained based on the invariance of the spatial vector. The transformation relation can be expressed as follows:(9)PM1=HMCPC1=H-1GMHGCPC1.After the calibration, the relative position between the binocular cameras and the transfer mark remains unchanged. In the process of measurement, data transformation from the binocular-camera coordinate system to the global coordinate system is achieved by calibratingHGM with the two theodolites.


## 4. Acquisition and Reconstruction of a Large-Scale Three-Dimensional Surface of a Component

### 4.1. Local Image Processing and Reconstruction

In a single FOV, the feature information of the component is measured using a laser scanning method. The assisted laser stripes that are projected on the surface of the component are captured by a binocular vision measurement system.Prior to reconstructing the laser stripes, the centres of the stripes are extracted using a grey centroid method [21]. The corresponding centre points of the left and right images are matched by the polar constraint of the cameras of the binocular vision system. The matching equation is expressed as(10)xlTFxr=0,where xl is the coordinate vector of the laser stripe centre point in the left image and xr is the coordinate vector of the laser stripe centre point in the right image. The fundamental matrix F can be calculated using two calibration target points with an exact distance [25].After matching the feature points of the left and right images, reconstruction of the image can be realized according to the binocular reconstruction principle. The coordinate vector of any pointP in the camera coordinate system can be expressed as(11)PC=zCXPCLf1zCYPCLf1f1f2t2′-YPt3′CRYPCRr7′XPCL+r8′YPCL+r9′f1-f2r4′XPCL+r5′YPCL+r6′f1,where RCRCL=r1′r2′r3′r4′r5′r6′r7′r8′r9′ and tCRCL=t1′t2′t3′T. RCRCL is the rotation matrix from the right camera relative to the left camera, and tCRCL is the translation matrix from the right camera to the left camera; they can be obtained by the calibration of the binocular cameras. XPCL,YPCL are the coordinates of point P in the image coordinate system of the left camera; XPCR,YPCR are the coordinates of point P in the image coordinate system of the right camera. f1 and f2 are the effective focal length of the left camera and the effective focal length of the right camera, respectively.


### 4.2. Reconstruction of a 3D Surface of a Large-Scale Component

A binocular vision system is utilized to capture the large-scale component in different positions. Measuring the spatial positions of the feature points on the transfer mark with the two theodolites,HGM in different positions can be calculated. With the intrinsic parameters that are calibrated in the laboratory and the extrinsic parameters that are calibrated onsite, the local coordinates are transferred to global coordinates according to (8) and (10). The global reconstruction of a 3D surface of a large-scale component can be realized.


## 4.1. Local Image Processing and Reconstruction

In a single FOV, the feature information of the component is measured using a laser scanning method. The assisted laser stripes that are projected on the surface of the component are captured by a binocular vision measurement system.Prior to reconstructing the laser stripes, the centres of the stripes are extracted using a grey centroid method [21]. The corresponding centre points of the left and right images are matched by the polar constraint of the cameras of the binocular vision system. The matching equation is expressed as(10)xlTFxr=0,where xl is the coordinate vector of the laser stripe centre point in the left image and xr is the coordinate vector of the laser stripe centre point in the right image. The fundamental matrix F can be calculated using two calibration target points with an exact distance [25].After matching the feature points of the left and right images, reconstruction of the image can be realized according to the binocular reconstruction principle. The coordinate vector of any pointP in the camera coordinate system can be expressed as(11)PC=zCXPCLf1zCYPCLf1f1f2t2′-YPt3′CRYPCRr7′XPCL+r8′YPCL+r9′f1-f2r4′XPCL+r5′YPCL+r6′f1,where RCRCL=r1′r2′r3′r4′r5′r6′r7′r8′r9′ and tCRCL=t1′t2′t3′T. RCRCL is the rotation matrix from the right camera relative to the left camera, and tCRCL is the translation matrix from the right camera to the left camera; they can be obtained by the calibration of the binocular cameras. XPCL,YPCL are the coordinates of point P in the image coordinate system of the left camera; XPCR,YPCR are the coordinates of point P in the image coordinate system of the right camera. f1 and f2 are the effective focal length of the left camera and the effective focal length of the right camera, respectively.


## 4.2. Reconstruction of a 3D Surface of a Large-Scale Component

A binocular vision system is utilized to capture the large-scale component in different positions. Measuring the spatial positions of the feature points on the transfer mark with the two theodolites,HGM in different positions can be calculated. With the intrinsic parameters that are calibrated in the laboratory and the extrinsic parameters that are calibrated onsite, the local coordinates are transferred to global coordinates according to (8) and (10). The global reconstruction of a 3D surface of a large-scale component can be realized.


## 5. Measurement Experiments

The global measurement system for the 3D surface of a large-scale component was set up in a laboratory, as shown in Figure4. The binocular vision system in the front consists of two CMOS cameras (Vieworks, VC-12 MC-M/C 65 with 35 mm lenses and a resolution of 3072×4096, the principled limitation of the resolution is about 0.5 mm) and a transfer mark (checkerboard of 12×9 checkers with an interval of 30 mm and an accuracy of 0.5 μm,). Two theodolites (Kolida, accuracy of 1′′) are set in the back. According to the presented global calibration method, the theodolites and the relationship between the mark and the binocular cameras are precisely calibrated in the laboratory. The reconstruction experiment is conducted with this measurement system.Figure 4
Global measurement system for 3D surface measurement.


### 5.1. Calibration of the Global Measurement System and Accuracy Evaluation

According to the calibration method proposed in Section3, the calibration experiments of the measurement system are conducted in the laboratory.(1) Calibration of the System Parameters. The transformation matrix HMC is calibrated. Based on the coordinates of six spatial feature points and the orthogonal constraint condition of the rotation matrices, the rotation matrix and the translation matrix of the two-theodolite system are obtained as follows:(12)RTLTR=0.8493-0.00010.52790.00910.9999-0.0144-0.52780.01700.8492,tTLTR=-1576.7-17.3496.3.The global coordinate system is constructed on the left theodolite.The calibration results of the binocular cameras are shown as follows:The intrinsic parameters of the left and right cameras are(13)KCL=6216.1202120.7306222.631578.84001,KCR=6199.8302036.8806202.821536.43001,where KCL and KCR are the intrinsic parameters matrices of the left camera and the right camera, respectively.The extrinsic parameters of the binocular cameras are(14)RCLCR=0.8760.0070.4820.0200.998-0.050-0.4820.0540.875,tCLCR=-905.922.94204.82.According to the calibration method that was discussed in Section3.2, HMC is obtained as follows:(15)HMC=0.9890-0.1422-0.2400107.020.12920.97720.1939239.120.07130.16060.9626278.800001.(2) Calibration of Global Parameters. Based on the calibration principle of the two theodolites, the global measurement system is calibrated on one spot. Feature points on the transfer mark are measured using the two theodolites. Thus, HGM can be obtained as follows:(16)HGM=0.01530.97680.2135-408.420.9987-0.0048-0.0499-465.07-0.04770.2140-0.97573905.850001.WithHMC and HGM, any point in the FOV can be reconstructed in the global coordinate system.To evaluate the measurement accuracy, a long one-dimensional (1D) target with two characteristic points (the distance between the two points is 1225.0214 mm) is considered. The measurement FOV is divided into two parts. The evaluation process is as follows: first, the target is placed in front of the cameras, as shown in Figure5. Second, we used the cameras to measure the left characteristic point of the target and then moved the cameras to the second position to measure the right characteristic point of the target. Last, the coordinates of the two characteristic points that were measured by the binocular cameras at two positions are integrated into the global coordinate system. To guarantee repeatability, the 1D target is placed at three different positions and the evaluation of this process is repeated three times. The targets that were reconstructed in the global coordinate system are shown in Figure 6. To evaluate the accuracy of the measurement system, the deviation between the real length and the measured length of the 1D target is calculated and listed in Table 1. The results indicate a maximum global measurement deviation of 0.103% for the proposed method.Table 1
Evaluation of the measurement accuracy (mm).
Position
x/mm
y/mm
z/mm
Measured value/mm
Standard value/mm
Deviation/%
1
375.4370
−234.7837
166.1296
1224.63
1225.0214
0.032%
−185.4221
852.2557
107.0487
2
−203.9730
−205.5046
131.8487
1223.753
0.103%
437.1248
834.3868
203.9114
3
−79.5582
−312.6632
−114.0780
1224.81
0.017%
283.6261
801.1597
233.7018Figure 5
Target measurement experiment. (a) 1D long target with two feature points. (b) First measurement position of the cameras.
(a)
(b)Figure 6
Reconstruction of the target at three different positions.


### 5.2. 3D Measurement of the Global Measurement System

To verify the feasibility of the proposed method, a standard flat part with the size of 600 mm × 800 mm is measured in the laboratory. Before the experiments, the standard flat part was measured using a three-coordinate measuring machine (Zeiss Prismo Navigator) to obtain the measuring reference data. By comparing the reconstruction results with the actual value, the construction accuracy of the global measurement system is validated. The experimental system is shown in Figure7. According to the calibration calculation of global system, the calibration results of binocular cameras and global system are illustrated in Tables 2 and 3, respectively.Table 2
The calibration results of binocular cameras.
Left camera
Right camera
Intrinsic matrix
K
=
5051.8
0
2093.7
0
5049.7
1589.0
0
0
1
K
=
4999.1
0
2065.0
0
4998.6
1539.5
0
0
1
Transformation matrix
H
C
R
C
L
=
0.8863
-
0.0795
0.4562
-
569.14
0.0694
0.9968
0.0389
-
23.749
-
0.4579
-
0.0028
0.8890
103.70
0
0
0
1Table 3
The calibration results of global system.
H
T
R
T
L
H
M
C
Transformation matrix
H
T
R
T
L
=
0.9997
0.0020
-
0.0252
1392.6
-
0.0020
1.0000
-
0.0001
-
14.512
0.0252
0.0002
0.9997
42.612
0
0
0
1
H
M
C
=
0.9618
-
0.0248
0.2726
-
254.48
0.0005
-
0.9957
-
0.0925
-
10.851
0.2738
0.0891
-
0.9577
-
218.60
0
0
0
1Figure 7
Reconstruction of the target at different positions.In the experiment, the measurement FOV is divided into four parts based on the size of the plate. Next, the four parts of the board are measured by the respective cameras at different positions. Images of the laser stripes, which are projected on the board, are captured by the cameras. The centres of laser stripes are extracted and matched based on image processing method in Section4.1. By monitoring the transfer mark with the two theodolites, the local 3D data from two measurements are united with the global coordinate system. The results of feature extraction and transformation matrix of different positions are shown in Table 4. The reconstructed results are shown in Figure 8. The 3D reconstruction points of measured part are compared with the ideal standard plane. The experimental results reveal the construction deviation is 0.14% with the corresponding measurement field.Table 4
The results of feature extraction and transformation in different positions.
Position
Results of feature extraction
Transformation matrixHGM
Left camera
Right camera
1
H
1
G
M
=
0.9900
0.0209
-
0.1398
922.80
0.0185
-
0.9997
-
0.0178
-
609.54
-
0.1401
0.0151
-
0.9900
3258.3
0
0
0
1
2
H
2
G
M
=
0.9926
0.0230
-
0.1190
926.98
0.0211
-
0.9996
-
0.0177
-
752.53
-
0.1194
0.0150
-
0.9927
3262.8
0
0
0
1
3
H
3
M
C
=
0.9997
0.0187
-
0.0156
1480.4
0.0184
-
0.9996
-
0.0211
-
751.85
-
0.0160
0.0208
-
0.9997
3256.4
0
0
0
1
4
H
4
M
C
=
0.9996
0.0162
-
0.0222
1475.8
0.0157
-
0.9996
-
0.0237
-
608.88
-
0.0226
0.0233
-
0.9995
3251.7
0
0
0
1Figure 8
Reconstruction results of the large plate in the global coordinate system.To verify the validity of the system, a curved composite part is measured. The part is shown in Figure9(a) and the reconstruction result is shown in Figure 9(b). The results indicate that the proposed system can effectively measure curved components.Figure 9
Experimental results of a curved composite part in the global coordinate system. (a) The measured curved composite part. (b) Reconstruction results.
(a)
(b)


## 5.1. Calibration of the Global Measurement System and Accuracy Evaluation

According to the calibration method proposed in Section3, the calibration experiments of the measurement system are conducted in the laboratory.(1) Calibration of the System Parameters. The transformation matrix HMC is calibrated. Based on the coordinates of six spatial feature points and the orthogonal constraint condition of the rotation matrices, the rotation matrix and the translation matrix of the two-theodolite system are obtained as follows:(12)RTLTR=0.8493-0.00010.52790.00910.9999-0.0144-0.52780.01700.8492,tTLTR=-1576.7-17.3496.3.The global coordinate system is constructed on the left theodolite.The calibration results of the binocular cameras are shown as follows:The intrinsic parameters of the left and right cameras are(13)KCL=6216.1202120.7306222.631578.84001,KCR=6199.8302036.8806202.821536.43001,where KCL and KCR are the intrinsic parameters matrices of the left camera and the right camera, respectively.The extrinsic parameters of the binocular cameras are(14)RCLCR=0.8760.0070.4820.0200.998-0.050-0.4820.0540.875,tCLCR=-905.922.94204.82.According to the calibration method that was discussed in Section3.2, HMC is obtained as follows:(15)HMC=0.9890-0.1422-0.2400107.020.12920.97720.1939239.120.07130.16060.9626278.800001.(2) Calibration of Global Parameters. Based on the calibration principle of the two theodolites, the global measurement system is calibrated on one spot. Feature points on the transfer mark are measured using the two theodolites. Thus, HGM can be obtained as follows:(16)HGM=0.01530.97680.2135-408.420.9987-0.0048-0.0499-465.07-0.04770.2140-0.97573905.850001.WithHMC and HGM, any point in the FOV can be reconstructed in the global coordinate system.To evaluate the measurement accuracy, a long one-dimensional (1D) target with two characteristic points (the distance between the two points is 1225.0214 mm) is considered. The measurement FOV is divided into two parts. The evaluation process is as follows: first, the target is placed in front of the cameras, as shown in Figure5. Second, we used the cameras to measure the left characteristic point of the target and then moved the cameras to the second position to measure the right characteristic point of the target. Last, the coordinates of the two characteristic points that were measured by the binocular cameras at two positions are integrated into the global coordinate system. To guarantee repeatability, the 1D target is placed at three different positions and the evaluation of this process is repeated three times. The targets that were reconstructed in the global coordinate system are shown in Figure 6. To evaluate the accuracy of the measurement system, the deviation between the real length and the measured length of the 1D target is calculated and listed in Table 1. The results indicate a maximum global measurement deviation of 0.103% for the proposed method.Table 1
Evaluation of the measurement accuracy (mm).
Position
x/mm
y/mm
z/mm
Measured value/mm
Standard value/mm
Deviation/%
1
375.4370
−234.7837
166.1296
1224.63
1225.0214
0.032%
−185.4221
852.2557
107.0487
2
−203.9730
−205.5046
131.8487
1223.753
0.103%
437.1248
834.3868
203.9114
3
−79.5582
−312.6632
−114.0780
1224.81
0.017%
283.6261
801.1597
233.7018Figure 5
Target measurement experiment. (a) 1D long target with two feature points. (b) First measurement position of the cameras.
(a)
(b)Figure 6
Reconstruction of the target at three different positions.


## 5.2. 3D Measurement of the Global Measurement System

To verify the feasibility of the proposed method, a standard flat part with the size of 600 mm × 800 mm is measured in the laboratory. Before the experiments, the standard flat part was measured using a three-coordinate measuring machine (Zeiss Prismo Navigator) to obtain the measuring reference data. By comparing the reconstruction results with the actual value, the construction accuracy of the global measurement system is validated. The experimental system is shown in Figure7. According to the calibration calculation of global system, the calibration results of binocular cameras and global system are illustrated in Tables 2 and 3, respectively.Table 2
The calibration results of binocular cameras.
Left camera
Right camera
Intrinsic matrix
K
=
5051.8
0
2093.7
0
5049.7
1589.0
0
0
1
K
=
4999.1
0
2065.0
0
4998.6
1539.5
0
0
1
Transformation matrix
H
C
R
C
L
=
0.8863
-
0.0795
0.4562
-
569.14
0.0694
0.9968
0.0389
-
23.749
-
0.4579
-
0.0028
0.8890
103.70
0
0
0
1Table 3
The calibration results of global system.
H
T
R
T
L
H
M
C
Transformation matrix
H
T
R
T
L
=
0.9997
0.0020
-
0.0252
1392.6
-
0.0020
1.0000
-
0.0001
-
14.512
0.0252
0.0002
0.9997
42.612
0
0
0
1
H
M
C
=
0.9618
-
0.0248
0.2726
-
254.48
0.0005
-
0.9957
-
0.0925
-
10.851
0.2738
0.0891
-
0.9577
-
218.60
0
0
0
1Figure 7
Reconstruction of the target at different positions.In the experiment, the measurement FOV is divided into four parts based on the size of the plate. Next, the four parts of the board are measured by the respective cameras at different positions. Images of the laser stripes, which are projected on the board, are captured by the cameras. The centres of laser stripes are extracted and matched based on image processing method in Section4.1. By monitoring the transfer mark with the two theodolites, the local 3D data from two measurements are united with the global coordinate system. The results of feature extraction and transformation matrix of different positions are shown in Table 4. The reconstructed results are shown in Figure 8. The 3D reconstruction points of measured part are compared with the ideal standard plane. The experimental results reveal the construction deviation is 0.14% with the corresponding measurement field.Table 4
The results of feature extraction and transformation in different positions.
Position
Results of feature extraction
Transformation matrixHGM
Left camera
Right camera
1
H
1
G
M
=
0.9900
0.0209
-
0.1398
922.80
0.0185
-
0.9997
-
0.0178
-
609.54
-
0.1401
0.0151
-
0.9900
3258.3
0
0
0
1
2
H
2
G
M
=
0.9926
0.0230
-
0.1190
926.98
0.0211
-
0.9996
-
0.0177
-
752.53
-
0.1194
0.0150
-
0.9927
3262.8
0
0
0
1
3
H
3
M
C
=
0.9997
0.0187
-
0.0156
1480.4
0.0184
-
0.9996
-
0.0211
-
751.85
-
0.0160
0.0208
-
0.9997
3256.4
0
0
0
1
4
H
4
M
C
=
0.9996
0.0162
-
0.0222
1475.8
0.0157
-
0.9996
-
0.0237
-
608.88
-
0.0226
0.0233
-
0.9995
3251.7
0
0
0
1Figure 8
Reconstruction results of the large plate in the global coordinate system.To verify the validity of the system, a curved composite part is measured. The part is shown in Figure9(a) and the reconstruction result is shown in Figure 9(b). The results indicate that the proposed system can effectively measure curved components.Figure 9
Experimental results of a curved composite part in the global coordinate system. (a) The measured curved composite part. (b) Reconstruction results.
(a)
(b)


## 6. Conclusions

In this paper, a global measurement method that is based on a multiple FOV combination was proposed for measuring the 3D surface of a large-scale component. Compared with existing methods, the proposed method has the advantage of high efficiency and no requirement for pasting a target mark on the component. As the cameras with the transfer mark can be placed at any position in the measurement range of theodolites, the measurement system is more flexible for large-scale component measurement. The experimental results in the laboratory indicate that a maximum accuracy of the measurement system of 0.103% can be attained when the length of the 1D target is approximately 1.225 m. For the large board measurement experiment, the reconstruction accuracy is less than 0.14%. Thus, the proposed method is practicable and suitable for measuring the 3D surface of a large-scale component in an industrial site. The measurement system can be employed in the assembly process of large-scale industry components. Additional research is suggested to improve the global measurement accuracy by optimizing the calibration process and improving the calibration precision.



---

*Source: 8765450-2017-09-17.xml*