# Autonomous Operation Method of Multi-DOF Robotic Arm Based on Binocular Vision
## 2 Materials and Methods
### Target Object Extraction  
The target object is a cube with a side length of 40 mm, and its front plane is perpendicular to the optical axis of the camera. The positional relationship between the target object and the camera is shown in Figure 2. It can be seen from the figure the position of the centroid of the target object is the moving target position of the end effector of the robotic arm. Figure 1: Schematic of the binocular vision positioning principle. Point \(M\) is the front plane center of the target object, whereas point \(N\) is the centroid of the target object; the coordinates of the points \(M\) and \(N\) in the left camera coordinate system are (\(X\), \(Y\), \(Z\)) and (\(X_{0}\), \(Y_{0}\), \(Z_{0}\)). The relationship between \(M\) and \(N\) is shown in Equation (2), where \(l\) is the side length of the target object. \[\left\{ \begin{array}{l}X_{0}=X\\ Y_{0}=Y\\ Z_{0}=Z+l/2\end{array} \right. \tag{2}\]  
In accordance with the above analysis, the positioning of the centroid of the target object can be transformed into the positioning of point \(M\). Point \(M\) is set as the target feature point of the study. The gray scale distribution of the image near the target feature point is shown in Figure 3.  
The gray value matrix of the image in the red frame in Figure 4 is \(K\), which is defined as follows:  
\[K=\left[ \begin{array}{cccccccc}A&AB&B\\ AC&O&BD\\ C&CD&D\end{array} \right]=\left[ \begin{array}{cccccccc}a_{11}&\cdots&a_{1n} &ab_{11}&\cdots&ab_{1m}&b_{11}&\cdots&b_{1n}\\ \vdots&\ddots&\vdots&\vdots&\ddots&\vdots&\vdots&\ddots&\vdots\\ a_{n1}&\cdots&a_{nm}&ab_{n1}&\cdots&ab_{nm}&b_{n1}&\cdots&b_{nn}\\ ac_{11}&\cdots&ac_{1n}&0_{11}&\cdots&0_{1m}&bd_{11}&\cdots&bd_{1n}\\ \vdots&\ddots&\vdots&\vdots&\ddots&\vdots&\vdots&\ddots&\vdots\\ ac_{m1}&\cdots&ac_{mn}&0_{m1}&\cdots&0_{mm}&bd_{m1}&\cdots&bd_{mm}\\ c_{11}&\cdots&c_{1n}&cd_{11}&\cdots&cd_{1m}&d_{11}&\cdots&d_{1n}\\ \vdots&\ddots&\vdots&\vdots&\ddots&\vdots&\vdots&\ddots&\vdots\\ c_{n1}&\cdots&c_{m}&cd_{n1}&\cdots&cd_{nm}&d_{n1}&\cdots&d_{nn}\end{array}  \right]. \tag{3}\]  
In Equation (3), the matrix \(AB\), \(AC\), \(BD\), and \(CD\) is the gray value matrix of the black and white boundary in the image, where \(m\) is much smaller than \(n\). In the ideal case, the values of the elements in matrix \(A\) and matrix \(D\) are 0, whereas those in matrix \(B\) and matrix \(C\) are 255. The gray value of the pixel changes uniformly at the black and white boundary in the image. The values of the elements in matrices \(AC\) and \(BD\) gradually increase from top to bottom, whereas those in matrices \(AB\) and \(BD\) gradually increase from left to right. The target feature point is located in the matrix \(O\). It should be noted that, if the extracted target feature point is located in the region \(O\), then the extraction can be considered successful because \(m\) is a small positive integer. Image convolution operation is used to enlarge the feature point feature to achieve target point extraction. The convolution process is as follows:  
Figure 3: Distribution of gray values near the target point. Figure 2: Schematic diagram of target grabbing object placement. \[d(x,y)=\sum_{s=1}^{2b+1}\sum_{t=1}^{2b+1}w(s,t)y(x-s+b+1,y-t+b+1). \tag{4}\]  
In Equation (4), \(w\) (\(s,t\)) is the convolution kernel of the image convolution operation and is defined as Equation (5), and \(b\) is a positive integer. \(y\) (\(x,y\)) is the gray matrix of the input image, and \(d\) (\(x,y\)) is the convolution result output matrix, which is the same size as the input image gray matrix. \[w(s,t)=\left[ \begin{array}{ccccc}E&EF&F\\ EG&I&FH\\ G&GH&H\end{array} \right]=\left[ \begin{array}{ccccc}e_{11}&\cdots&e_{1b}&e_{ f11}&f_{11}&\cdots&f_{1b}\\ \vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots\\ e_{b1}&\cdots&e_{bb}&e_{f1b}&f_{b1}&\cdots&f_{bb}\\ e_{g11}&\cdots&e_{g1b}&i&fh_{11}&\cdots&fh_{1b}\\ g_{11}&\cdots&b_{bb}&gh_{11}&h_{11}&\cdots&h_{1b}\\ \vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots\\ g_{b1}&\cdots&g_{bb}&gh_{b1}&h_{b1}&\cdots&h_{bb}\end{array}