# Design of an Eye-in-Hand Smart Gripper for Visual and Mechanical Adaptation in Grasping
## 5 Visual Adaptivity by Eye-in-Hand System
### Motion Planning
#### 5.3.1 Conversion of Image Coordinates
\right] \tag{18}\]  
where \(r_{1}\ =\ K^{-1}h_{1}\) and \(r_{2}\ =\ K^{-1}h_{2}\). As \(r_{1}\) and \(r_{2}\) are orthogonal, therefore:  
\[r_{1}r_{2}\ =\ 0\ \Rightarrow h_{1}^{\mathrm{T}}K^{-T}K^{-1}h_{1}\ =\ 0 \tag{19}\]  
\[|r_{1}|\ =\ |r_{2}|\ =\ 1\Rightarrow h_{1}^{\mathrm{T}}K^{-T}K^{-1}h_{1}-h_{2}^{ \mathrm{T}}K^{-T}K^{-1}h_{2}\ =\ 0 \tag{20}\]  
_Matrix B_ is defined as the inverse and transpose of the intrinsic matrix multiplied by the inverse of the intrinsic matrix. That is, \(B\ =\ K^{-T}K^{-1}\). This \(B\) matrix is in fact a Hermite matrix, which can be decomposed into the definiteness of a matrix multiplied by the Hermitian transpose matrix to determine the intrinsic parameters. As conversion errors exist because of manufacturing and assembling imperfections, it is necessary to find the corresponding intrinsic matrix to minimize the errors, and make the \(r_{1}\) and \(r_{2}\) vectors orthogonal. Until now, we have not considered lens distortion of the camera. The lens distortion parameter matrix \(K\) can be calculated using the OpenCV image processing library to find the ideal pixel of the image acting as the target object center. We define (\(u\), \(v\)) as the ideal pixel image coordinates and (\(\vec{u}\), \(\vec{v}\)) as the corresponding real observed image coordinates, where (\(x\), \(y\)) and (\(\vec{x}\), \(\vec{y}\)) are their world coordinates, respectively. Owing to the pinhole camera model, the ideal position versus the distance of the target equals the pixel versus the focal length. That is, we can express this relationship as:  
\[u\ (pixel\ position)\text{:}f\ ( focal\ length)\ =\ x\ (global\ position)\ \text{and}\ z\ (target\ distance) \tag{21}\]  
Figure 25: Transformation from world coordinate system to image coordinate system. Assume \(\vec{x}^{\prime}~{}=~{}\frac{\vec{x}}{\hat{z}}~{},\vec{y}^{\prime}~{}=~{}\frac{ \vec{y}}{\hat{z}}~{},x^{\prime}~{}=~{}\frac{x}{\hat{z}}~{},\) and \(y^{\prime}~{}=~{}\frac{y}{\hat{z}}\), where \(z\) is the height of the target object, as expressed in Equation (25). \(\vec{x}^{\prime}\) and \(\vec{y}^{\prime}\) can then be rewritten as:  
\[\vec{x}^{\prime}~{}=~{}\frac{\vec{u}-u_{0}}{\alpha},\,\vec{y}^{\prime}~{}=~{} \frac{\vec{v}-v_{0}}{\beta}. \tag{22}\]  
Assuming that \(r^{2}~{}=~{}\big{(}\vec{x}^{\prime 2}+\vec{y}^{\prime 2}\big{)}\), the ideal world coordinates are given by:  
\[ \begin{array}{l}x^{\prime}~{}=~{}\vec{x}^{\prime}\big{(}1+k_{1}r^{2}+k_{2}r^ {4}+k_{3}r^{6}\big{)}+2P_{1}\vec{x}^{\prime}\vec{y}^{\prime}+P_{2}\big{(}r^{2 }+2\vec{x}^{\prime 2}\big{)}\\ y^{\prime}~{}=~{}\vec{y}^{\prime}\big{(}1+k_{1}r^{2}+k_{2}r^{4}+k_{3}r^{6} \big{)}+2P_{2}\vec{x}^{\prime}\vec{y}^{\prime}+P_{1}\big{(}r^{2}+2\vec{y}^{ \prime 2}\big{)}\end{array}  \tag{23}\]  
where \(k_{1}\), \(k_{2}\), \(k_{3}\), \(P_{1}\), and \(P_{2}\) are distortion parameters. Finally, the ideal pixel (\(u\), \(v\)) can be expressed as:  
\[u~{}=~{}\alpha x^{\prime}+u_{0},~{}v~{}=~{}\beta y^{\prime}+v_{0} \tag{24}\]