# Vision-Guided Six-Legged Walking of Little Crabster Using a Kinect Sensor
## 3 Camera-Robot Coordinate Transformation
### Derivation of Transformation Matrix Using Tsai Algorithm
#### 3.2.2 Coordinate Transformation  
In order to use the obtained 3D coordinates for the robot's walking, it is first necessary to calculate a homogeneous transformation matrix between the robot-fixed coordinate frame and the camera-fixed coordinate frame; the homogeneous transformation matrix was then derived using the Tsai algorithm [21]. Figure 5a shows the experimental setup for this process. First, homogeneous transformation matrices \(A_{1}\) and \(A_{2}\), which indicate the relationships of the two different camera positions to a known target ball position, were derived. Second, homogeneous transformation matrices \(B_{1}\) and \(B_{2}\) from the robot-fixed coordinate frame to the two camera holder positions were obtained from the CAD information. Finally, a homogeneous transformation matrix \(X\), which shows the relationship between the camera holder position and the Kinect camera, was derived using the \(A\) and \(B\) matrices indicated in Figure 5a, and a homogeneous transformation between the robot-fixed coordinate frame and the camera-fixed coordinate frame was obtained. More specifically, we collected 3D coordinates of the many target ball positions with respect to both the camera-fixed coordinate and the robot-fixed coordinate frames for two different Kinect sensor positions, as shown in Figure 5b. The target ball positions were accurately changed using the developed calibration tool. In contrast to the conventional Tsai algorithm, optimized matrices \(A_{1}\) and \(A_{2}\) were derived from the pseudo inverse of Equations (2) and (3). In Equations (2) and (3), \(D_{\text{1i}}\) and \(D_{\text{2i}}\) are the two 3D camera-fixed  
Figure 4: Image acquisition: (**a**) Barrel distortion effect; (**b**) Calculation of the 3D coordinate of ground point. Figure 5: Coordinate transformation: (**a**) Experimental setup of Tsai algorithm; (**b**) Calculation of \(A_{1}\) and \(A_{2}\) matrices using pseudo inverse optimization. coordinates of \(n\) positions of the target ball for two different camera positions, and the values of \(P_{\text{i}}\) are the known robot-fixed coordinates of \(n\) positions of the target ball. \[A_{1}D_{1}=P;\quad A_{1}\left[ \begin{array}{cccc}D_{11x}&D_{12x}&\cdots&D_{1 nx} \\ D_{11y}&D_{12y}&\cdots&D_{1 ny}\\ D_{11z}&D_{12z}&\cdots&D_{1 nz}\\ 1&1&1&1\end{array} \right]=\left[ \begin{array}{cccc}P_{1x}&P_{2x}&\cdots&P_{nx }\\ P_{1y}&P_{2y}&\cdots&P_{ny}\\ P_{1z}&P_{2z}&\cdots&P_{nz}\\ 1&1&\cdots&1\end{array} \right] \tag{2}\]  
\[A_{2}D_{2}=P;\quad A_{2}\left[ \begin{array}{cccc}D_{21x}&D_{22x}&\cdots&D_{2 nx}\\ D_{21y}&D_{22y}&\cdots&D_{2 ny}\\ D_{21z}&D_{22z}&\cdots&D_{2 nz}\\ 1&1&1&1\end{array} \right]=\left[ \begin{array}{cccc}P_{1x}&P_{2x}&\cdots&P_{ nx}\\ P_{1y}&P_{2y}&\cdots&P_{ny}\\ P_{1z}&P_{2z}&\cdots&P_{nz}\\ 1&1&\cdots&1\end{array} \right] \tag{3}\]  
Matrix \(A\) was then calculated from matrices \(A_{1}\) and \(A_{2}\), as in Equation (4), and matrix \(B\) was also calculated from matrices \(B_{1}\) and \(B_{2}\), as in Equation (5) in the same manner. \[A=A_{1}A_{2}{}^{-1}=\left[ \begin{array}{cccc}1.08&-0.15&0.02&2.69\\ -0.08&0.98&-0.07&77.54\\ -0.10&-0.09&1.05&0.60\\ 0&0&0&1\end{array} \right] \tag{4}\]  
where \(A_{1}=\left(PD_{1}^{\text{T}}\right)\left(D_{1}D_{1}^{\text{T}}\right)^{-1}\), \(A_{2}=\left(PD_{2}^{\text{T}}\right)\left(D_{2}D_{2}^{\text{T}}\right)^{-1}\)  
\[B=B_{1}{}^{-1}B_{2}=\left[ \begin{array}{cccc}1.00&0.00&0.00&-100.00\\ 0.00&1.00&0.00&0.00\\ 0.00&0.00&1.00&50.00\\ 0&0&0&1\end{array} \right] \tag{5}\]  
In Figure 5a, the relationship among matrices \(A\), \(B\), and \(X\) can be derived as in Equation (6), and matrix \(X\) was finally calculated as in Equation (7) by solving the relationship expressed in Equation (6). \[XA=BX \tag{6}\]  
\[X=\left[ \begin{array}{cccc}0.75&0.02&0.66&-125.43\\ 0.02&1.00&-0.05&-15.03\\ -0.66&0.05&0.75&94.88\\ 0&0&0&1\end{array}