## 4 Model Development  
### Stereo Camera Model  
Depth between the objects to the camera plane is either approximated or estimated in the IBVS for generating the interaction matrix [15]. Using a stereo camera system in IBVS eliminates the inaccuracies associated with monocular depth estimation, as it directly measures depth by leveraging the disparity between the left and right images. The stereo camera model is illustrated in Figure 6. A stereo camera consists of two lenses separated by a fixed baseline \(b\). Each lens has a focal length \(F\) (measured in mm) which is the distance from the image plane to the focal point. Assuming the camera is calibrated, the intrinsic parameters: \(b\), \(F\) is accurately estimated. A scene point \(I\) is measured in the 3D coordinate frame {C} centered at the middle of the baseline with its coordinates as \(\left[{X^{C}},\;{Y^{C}},{Z^{C}}\right]^{T}\). The stereo camera model maps the 3D coordinates of this point to its 2D coordinates projected on the left and right image plane as \(\left[{u_{l}},{v}\right]^{T}\) and \(\left[{u_{r}},{v}\right]^{T}\), respectively. The full camera projection map, incorporating both intrinsic and extrinsic parameters, is given by:  
\[s\cdot p_{image}=K\cdot\left[{R|T}\right]\cdot P_{C} \tag{5}\]  
where \(p_{image}\) are the image coordinates of the point and \(P_{C}\) are the 3D coordinates measured in the camera frame {C}. \(s\) is the scale factor that ensures correct projection between 2D and 3D features. \(K\) is the intrinsic matrix with a size of 3 \(\times\) 3, and the mathematical expression is presented as:  
\[K= \begin{bmatrix}F&k&u_{0}\\ 0&F&v_{0}\\ 0&0&1\end{bmatrix}  \tag{6}\]  
where \(k\) is the skew factor, which represents the angle between the image axes (\(u\) and \(v\) axis). \(u_{0}\) and \(v_{0}\) are coordinates offsets in image planes. In Equation (5), \(R\) is the rotational matrix from camera frame {C} to each image coordinate frame, and \(T\) is the translation matrix from camera frame {C} to each camera lens center. Since there is no rotation between the camera frame {C} and image frames but only a translation along the \(X_{C}\) axis occurs, the transformation matrices for the left and right image planes are expressed as:  
\[\left[R\middle|T\right]_{Left}= \begin{bmatrix}1&0&0&-b/2\\ 0&1&0&|&0\\ 0&0&1&0\end{bmatrix}  \tag{7}\]  
\[\left[R\middle|T\right]_{Right}= \begin{bmatrix}1&0&0&b/2\\ 0&1&0&|&0\\ 0&0&1&0\end{bmatrix}  \tag{8}\]  
Assume the \(u\) and \(v\) axis are perfectly perpendicular (take \(k=0\)), and there are no offsets in the image coordinates (take \(u_{0}=v_{0}=0\)) for both lens. Also, set factor \(s=Z_{I}^{C}\) accounts for perspective depth scaling. The projection equations for the left and right image planes can be rewritten in homogeneous coordinates as:  
\[Z_{I}^{C}\cdot \begin{bmatrix}u_{I}\\ v\\ 1\end{bmatrix} = \begin{bmatrix}F&0&0\\ 0&F&0\\ 0&0&1\end{bmatrix} \cdot \begin{bmatrix}1&0&0&-b/2\\ 0&1&0&|&0\\ 0&0&1&0\end{bmatrix} \cdot \begin{bmatrix}X_{I}^{C}\\ Y_{I}^{C}\\ Z_{I}^{C}\\ 1\end{bmatrix}  \tag{9}\]  
\[Z_{I}^{C}\cdot \begin{bmatrix}u_{r}\\ v\\ 1\end{bmatrix} = \begin{bmatrix}F&0&0\\ 0&F&0\\ 0&0&1\end{bmatrix} \cdot \begin{bmatrix}1&0&0&b/2\\ 0&1&0&|&0\\ 0&0&1&0\end{bmatrix} \cdot \begin{bmatrix}X_{I}^{C}\\ Y_{I}^{C}\\ Z_{I}^{C}\\ 1\end{bmatrix}  \tag{10}\]  
Equations (9) and (10) establish the mathematical relationship between the 3D coordinates of a point in the camera frame {C} and its 2D projections on the left and right image  
Figure 6: The projection of a scene object on the stereo camera’s image planes. Note: The v-coordinate on each image plane is not displayed in this plot but is measured along the axis that is perpendicular to and pointing out of the plot. planes. The pixel value along the \(v\)-axis remains the same for both images.