# Automatic Assembly Technology of Dense Small Screws for Flat Panel Parts
## 2 Methods
### Definition of Coordinate System
#### 2.2.2 Calibration of Vision Systems Based on Nine-Point Calibration  
An industrial camera collects the image information of the workpiece, and the pixel coordinates of the key points can be obtained through image processing. However, the pixel coordinates cannot be directly used to guide the movement of the manipulator. Thus, they must be calibrated and transformed into \(\{F\}\) coordinates to guide the camera's movement. Therefore, the nine-point calibration method is used to solve the position and attitude relationship between \(\{p\}\) and \(\{F\}\). The solution process mainly involves the basic (\(\{B\}\)), flange (\(\{f\}\)), camera (\(\{C\}\)), pixel (\(\{p\}\)), and workpiece (\(\{F\}  
The \(Z_{f}\) axis of \(\{f\}\) is perpendicular to the \(\{F\}\) plane. The optical axis of the camera is set parallel to the \(Z_{f}\) axis of the flange (i.e., the camera optical axis is perpendicular to the workpiece surface) through fixture design. This means that the pixel coordinate system is parallel to the workpiece surface. Because the manipulator carries the camera and screw-locking device to perform movement at the same height relative to \(\{F\}\) at a fixed attitude, considering the \(Z_{F}\) axis is unnecessary for \(\{F\}\). Thus, the problem is simplified to solving the corresponding relationship between the target pixel set \((u,v)\) of the plane image and coordinate point set \((x,y)\) of the manipulator in \(\{F\}\). The pixel points, \((u,v)\), in the image can be transformed into points \((x,y)\) in \(\{F\}\) through two-dimensional affine transformation. In two-dimensional affine transformation, when a figure on one plane is translated to another, the parallel lines remain parallel before and after the translation. Each affine transformation can be represented by a combination of translation, rotation, shearing, and scaling [20]. The composition of the two-dimensional homogeneous affine matrix is given by Equation (5):  
\[H_{\rm 2D}= \begin{bmatrix}1&0&T_{\rm X}\\ 0&1&T_{\rm Y}\\ 0&0&1\end{bmatrix} \cdot \begin{bmatrix}\cos\alpha&-\sin\alpha&0\\ \sin\alpha&\cos\alpha&0\\ 0&0&1\end{bmatrix} \cdot \begin{bmatrix}1&-\sin\theta&0\\ 0&\cos\theta&0\\ 0&0&1\end{bmatrix} \cdot \begin{bmatrix}S_{\rm X}&0&0\\ 0&S_{\rm Y}&0\\ 0&0&1\end{bmatrix}  \tag{5}\]  
where  
\(S_{\rm X}\) is the scaling of the \(X\) axis of the original coordinate during the conversion;  
\(S_{\rm Y}\) is the scaling of the \(Y\) axis of the original coordinate during the conversion;  
\(\theta\) is the shear coefficient of the transformed coordinate system;  
\(\alpha\) is the rotation angle around the origin of the coordinate system;  
\(T_{\rm X}\) is the translation distance in the \(X\) axis direction of two coordinate systems;  
\(T_{\rm Y}\) is the translation distance in the \(Y\) axis direction of two coordinate systems. The affine transformation between the pixel and workpiece coordinate systems is represented by a two-dimensional homogeneous transformation matrix obtained from Equation (6):  
\[ \begin{bmatrix}x\\ y\\ 1\end{bmatrix} = \begin{bmatrix}a_{11}&a_{12}&a_{13}\\ a_{21}&a_{22}&a_{23}\\ 0&0&1\end{bmatrix}\begin{bmatrix}u\\ v\\ 1\end{bmatrix}  \tag{6}\]  
Figure 5: Coordinate system of manipulator. where  
\(u\) is the abscissa of \(\{p\}\);  
\(v\) is the vertical coordinate of \(\{p\}\);  
\(x\) is the abscissa of \(\{F\}\); and  
\(y\) is the vertical coordinate of \(\{F\}\). Calculating the affine matrix requires at least three sets of corresponding image pixels and camera coordinates in \(\{F\}\). To improve the calibration accuracy, nine datasets are generally used. This operation is overdetermined, the resulting affine matrix minimizes the distance between the input and transformed point sets, as given by Equation (7):  
\[\sum_{i}\left\| \begin{bmatrix}x\\ y\\ 1\end{bmatrix} -H_{\text{2D}}\cdot \begin{bmatrix}u\\ v\\ 1\end{bmatrix} \right\|^{2}=\min \tag{7}\]  
where \(H_{\text{2D}}\) is the affine matrix.