# Intelligent Alignment Monitoring Method for Tortilla Processing Based on Machine Vision
## 3 Method
### Three-Dimensional Deformation Measurement Algorithm
#### 3.2.1 Displacement Measurement Algorithm  
For the displacement measurement of the structure measurement point position, in order to make the measurement results easier to understand and intuitively represent the displacement information of the structure, this paper uses the 2D checkerboard grid target to establish the \(xoy\) plane of the world coordinate system \(o-xyz\) on the structure plane, and the specific establishment method is as follows.  
1. Based on the known internal and external parameters of the camera of the binocular stereo vision structure deformation measurement system, the 2D checkerboard target is placed at the desired plane, and a single frame is acquired by the left camera of the measurement system.  
2. Extract the image coordinates of each corner point of the checkerboard grid in the checkerboard grid image by the Hough transformation algorithm, as shown in Figure 4.  
Figure 3: Binocular stereo vision 3D coordinate monitoring.  
3. Assign the corresponding real 3D coordinates to each corner point, where the Z-axis coordinates of each corner point are assumed to be zero, and the X-axis and Y-axis coordinates are the real coordinates of the checkerboard grid relative to the set coordinate origin O, respectively.  
4. Solve Equation (5) to obtain the stiffness transformation matrix [R, T] of the left camera coordinate system \(O_{l}-x_{l}y_{l}z_{l}\) of the measurement system with respect to the established world coordinate system \(o-xyz\).  
\[s\left[\begin{array}{ccc}f_{x}&\lambda&u\\ 0&f_{y}&v\\ 0&0&1\end{array}\right]\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}r_{11}&r_{12}&r_{13}&t_{x}\\ r_{21}&r_{22}&r_{23}&t_{y}\\ r_{31}&r_{32}&r_{33}&t_{z}\end{array}\right]\left[\begin{array}{c}x\\ y\\ z\\ 1\end{array}\right]=\left[R,T\right]\left[\begin{array}{c}x\\ z\\ 1\end{array}\right] \tag{5}\]  
where \(s\left[\begin{array}{ccc}f_{x}&\lambda&u\\ 0&f_{y}&v\\ 0&0&1\end{array}\right]\) is a known internal parameter of the left camera, \(s\) is a scale factor, and \([u,v,1]^{T}\) is the chi-square form of the angular image coordinates.  
5. Using the stiffness conversion matrix obtained in the previous step, the 3D coordinates in the coordinate system of the left camera calculated by the measurement point Equation (4) are converted to the world coordinate system o, as shown in Equation (6).  
\[\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=R^{-1}(\left[\begin{array}{c}x_{l}\\ y_{l}\\ z_{l}\end{array}\right]-T) \tag{6}\]  
The in-plane and out-of-plane displacements of the structure are calculated from the 3D coordinates of the measurement points in the established world coordinate system. Assume that the 3D coordinates of the measurement point P at time t are \((x_{0},y_{0},z_{0})\). Due to the deformation of the structure, the 3D coordinates of the measurement point P at time t are \((x_{i},y_{i},z_{i})\), as shown in Figure 5.  
Figure 4: Establishing the world coordinate system.  
Then:  
\[\left\{\begin{array}{c}d_{i}=\sqrt{\left(x_{i}-x_{0}\right)^{2}+\left(y_{i}-y_{0} \right)^{2}}\times sign(\frac{x_{i}-x_{0}}{\sqrt{\left(x_{i}-x_{0}\right)^{2}+ \left(y_{i}-y_{0}\right)^{2}}})\\ d_{i}=z_{i}-z_{0}\end{array}\right. \tag{7}\]