# A Robotic Automatic Assembly System Based on Vision
## 3 Proposed Method
### Multi-Module Hybrid Calibration  
Multi-module hybrid calibration mainly includes a robot base coordinate system \(\{B\}\), an industrial camera coordinate system \(\{C\}\), and a conveyor coordinate system \(\{T\}\). Each coordinate system consists of an x axis; y axis; horizontal rotation angle \(\theta\) of a workpiece; and the conveyor coordinate system as the calibration link to realize the transformation and unification of the coordinate system, as shown in Figure 5.  
First the vision system uses the calibration plate, combines the HALCON calibration operator, takes [PERSON]'s calibration [34] method to obtain the camera parameters, and then adopts the nine-point method [35] to obtain the camera's external parameters. The pose data of the industrial camera coordinate system \(\{C\}\) are represented by \(\frac{T}{C}P\) in the conveyor coordinate system \(\{T\}\), and their expression is as follows:  
Figure 4: Assembly production of phone shell. Figure 5: Relative relations between coordinate systems. \[ \begin{array}{c}T_{C}P{=}\left[\begin{array}{cc}T_{C}&R_{C}\\ 0&1\end{array}\right]\end{array}  \tag{1}\]  
\(R_{C}\) denotes a rotation vector rotated about the \(Z\) axis of the conveyor coordinate system, and \(T_{C}\) represents the translation vector along the \(X\) axis and \(Y\) axis of the conveyor belt coordinate system. The calibration of the vision system and the belt is completed through the coordinate values in the pixel coordinate system converting into coordinate values in the conveyor coordinate system. Then the marker is placed on the conveyor belt. The initial pose \( \begin{smallmatrix}B_{1}\\ T\end{smallmatrix} P\) of the marker is recorded, as are the pose \( \begin{smallmatrix}B_{2}\\ T\end{smallmatrix} P\) after the belt has moved a distance and the number of pulses of the encoder \(L_{1}\),\(L_{2}\). The relationship between the two positions is as follows:  
\[ \begin{array}{c}\begin{smallmatrix}B_{2}\\ T\end{smallmatrix}P{=}\begin{smallmatrix}B_{2}\\ B_{1}\end{smallmatrix}P\times\begin{smallmatrix}B_{1}\\ T\end{smallmatrix}P\end{array}  \tag{2}\]  
As the process of moving the workpiece on the conveyor belt is only x axis translation, with no rotation, the matrix expression of \( \begin{smallmatrix}B_{2}\\ B_{1}\end{smallmatrix} P\) is defined as follows:  
\[ \begin{array}{c}\begin{smallmatrix}B_{2}\\ B_{1}\end{smallmatrix}P{=}\left[\begin{array}{ccc}1&0&x_{B_{1}B_{2}}\\ 0&1&0\\ 0&0&1\end{array}\right]\end{array}  \tag{3}\]  
where \(x_{B_{1}B_{2}}\) represents the translation distance of the workpiece along the X axis of the conveyor belt coordinate system. The calculation method is as follows:  
\[x_{B_{1}B_{2}}=M\cdot L \tag{4}\]  
where \(M\) indicates the change of \(X\) axis value of an encoder pulse corresponding to the conveyor belt coordinate system. \(L{=}L_{2}-L_{1}\) represents the sum of pulses turned by the encoder during the workpiece's movement. When the workpiece is detected by the industrial camera, with the advance of the conveyor belt, its position and pose data in the robot base coordinate system \( \begin{smallmatrix}B_{2}\\ C\end{smallmatrix} P\) of the mechanical arm are expressed as follows:  
\[ \begin{array}{c}\begin{smallmatrix}B_{2}\\ C\end{smallmatrix}P{=}\begin{smallmatrix}B_{2}\\ B_{1}\end{smallmatrix}P\times\begin{smallmatrix}B_{1}\\ T\end{smallmatrix}P\times\begin{smallmatrix}T\\ C\end{smallmatrix}P\\ =\left[\begin{array}{ccc}1&0&M\cdot L\\ 0&1&0\\ 0&0&1\end{array}\right]\left[\begin{array}{ccc}R_{B}&T_{B}\\ 0&1\end{array}\right]\left[\begin{array}{ccc}R_{C}&T_{C}\\ 0&1\end{array}\right].\end{array}  \tag{5}\]  
According to the rotation translation relationship of the coordinate system and the speed of the conveyor belt, the calibration of the conveyor and the robot is completed. As shown in Figure 6, \(B\) is the fixed base coordinate system of the robot arm, E is the end coordinate system of the arm, and \(C\) is the camera coordinate system. \( \begin{smallmatrix}B\\ H_{W}\end{smallmatrix}