<figure><img src="image_1.png" /></figure>

HAL
open science

# Positioning a Coarse-Calibrated Camera with Respect to an Unknown Planar Object by 2D 1/2 Visual Servoing

Ezio Malis, François Chaumette, Sylvie Boudet

► To cite this version:

Ezio Malis, François Chaumette, Sylvie Boudet. Positioning a Coarse-Calibrated Camera with Respect to an Unknown Planar Object by 2D 1/2 Visual Servoing. FAC Symposium on Robot Control (SYROCO), Sep 1997, Nantes, France. pp.497-503, 10.1016/S1474-6670(17)44310-2. hal-04655038

HAL Id: hal-04655038

https://hal.science/hal-04655038v1

Submitted on 20 Jul 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
POSITIONING A COARSE-CALIBRATED CAMERA
WITH RESPECT TO AN UNKNOWN PLANAR
OBJECT BY 2D 1/2 VISUAL SERVOING

Ezio MALIS * François CHAUMETTE *
Sylvie BOUDET **

* IRISA/INRIA, Campus de Beaulieu, 35042 Rennes, France

** EDF-DER, 6 Quai Watier, 78401 Chatou, France

**Abstract:** In this paper we propose a new vision-based robot control approach halfway between the classical position-based and image-based visual servoings. It allows to avoid their respective disadvantages. The homography between the feature points extracted from two images (corresponding to the current and desired camera poses) is computed at each iteration. Then, an approximate partial-pose, where the translational term is known only up to a scale factor, is deduced, from which can be designed a closed-loop control law controlling the six camera d.o.f. Contrarily to the position-based visual servoing, our scheme does not need any geometric 3D model of the object. Furthermore and contrarily to the image-based visual servoing, our approach ensures the convergence of the control law in all the task space.

**Keywords:** Robot Control, Computer Vision, Visual Servoing, Homography.

# 1. INTRODUCTION

Vision-based robot control using an eye-in-hand system is classified into two groups (Weiss et al., 1987; Hashimoto, editor, 1993; Hutchinson et al., 1996): *position-based* and *image-based* control systems. In a *position-based* control system, the control error function is computed in the 3D Cartesian space (Wilson et al., 1996) (for this reason this approach can be called *3D visual servoing*). The pose of the target with respect to the camera, which describes its 3D position and 3D orientation, is estimated from image features corresponding to the perspective projection of the target in the image. Numerous methods exist to recover the pose of an object (see (Dementhon and Davis, 1995) for example). They are all based on the knowledge of a perfect geometric model of the object and necessitate a calibrated camera to obtain unbiased results. The main advantage of 3D visual servoing is that it controls the camera trajectory in the Cartesian space, which allows to easily combine the visual positioning task with obstacles avoidance and/or joint limits and singu-

larities avoidance. On the other hand, in an *image-based* control system, the control error function is computed in the 2D image space (for this reason this approach can be called *2D visual servoing*). This local approach is based on the use of an image Jacobian (also called interaction matrix (Espiau et al., 1992)) and the control laws provide at each iteration the camera velocity for minimizing the observed error in the image. This approach is known to be very robust with respect to camera and robot calibration errors (Espiau, 1993). However, the convergence is ensured only in a region (quite impossible to determine) around the desired position. Furthermore, the Cartesian trajectory of the camera is uncontrolled, and some image features may get out of the camera field of view during the servoing, which generally gives rise to its failure.

The purpose of this paper is to design a new visual servoing system which combines the advantages of 2D and 3D visual servoings and avoids their drawbacks. We point out our attention to one of the typical applications of visual servoing:
positioning a camera mounted on a robot end-effector relative to a target, for a grasping task for instance. The positioning task is divided into two steps. In the first off-line learning step, the camera is moved to its desired position. The image of the target corresponding to this position is acquired and the extracted desired features are stored. In the second on-line step, after the camera and/or the target have been moved, the camera is commanded so that the current features reach their desired position in the image. In such application, the target geometry is not always precisely known. Furthermore, the convergence has to be obtained in all the task space.

Our approach can be called *2D 1/2 visual servoing* since the control error function is computed in part in the 3D Cartesian space and in part in the 2D image space. More precisely, the homography between the feature points extracted from two images (corresponding to the current and desired camera poses) is computed at each iteration. From the homography, the rotation of the camera between the two views is estimated. Consequently, the rotational control loop can be decoupled from the translational one. A such decoupled system allows to obtain the convergence in all the task space. It must be emphasized that the use of an homography does not need neither a 3D model of the target (the method we actually use to estimate the homography is however only valid for planar objects) nor a precisely calibrated camera. As far as the camera translational displacement is concerned, it can only be estimated up to a scale factor. It is therefore controlled using 2D image features and the ratio, obtained from the homography, of the unknown desired and current distances between the camera and the target. Finally, in spite of the partial-pose estimation, experiments show that the approach is robust to calibration errors.

The paper is organized as follows. In Section 2 and Section 3 we briefly recall 3D and 2D visual servos- ings respectively. In Section 4 we present the 2D 1/2 visual servoing approach. The experimental results are given in Section 5.

## 2. 3D VISUAL SERVOING

Let $F_0$ be the coordinate frame attached to the target, $F_1$ and $F_2$ be the coordinate frames attached to the camera in its desired and current position respectively (see Figure 1). Knowing the coordinates, expressed in $F_0$, of at least four points of the target (Dementhon and Davis, 1995), it is possible from their projection to compute the desired camera pose (represented in Figure 1 by the transformation matrix $^1T_0$) and the current camera pose (represented in Figure 1 by the transformation matrix $^2T_0$). The camera displacement to

<figure><img src="image_3.png" /><figcaption>Fig. 1. Modelisation of camera displacement for 3D visual servoing</figcaption></figure>

reach the desired position is thus easily obtained, and can be performed either in open loop or, more robustly, in closed loop. Finally, the control law performing the computed displacement has to be designed in order that the image features used in the pose estimation always appear in the camera field of view. Let us also note that if the camera is not calibrated, the current and desired camera poses will not be accurately estimated. However, this is not a drawback if a closed-loop scheme is performed. The corresponding block diagram of the 3D visual servoing approach is given in Figure 2.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Block diagram of the 3D visual servoing</figcaption></figure>

## 3. 2D VISUAL SERVOING

The control error function is now expressed di- rectly in the 2D image space (see Figure 3).

<figure><img src="image_5.png" /><figcaption>Fig. 3. Modelisation of camera displacement for 2D visual servoing</figcaption></figure>
Let **s** be the current value of visual features observed by the camera and **s**\* be the desired value of **s** to be reached in the image. The time variation of **s** is related to camera velocity by (Espiau *et al.*, 1992):

$$ \dot{\mathbf{s}} = \mathbf{L}(\mathbf{s}, Z)\mathbf{T} \qquad (1) $$

where $\mathbf{T} = [\mathbf{V} \ \mathbf{\Omega}]$ is the camera velocity screw and $\mathbf{L}(\mathbf{s}, Z)$ is the interaction matrix related to $\mathbf{s}$. For example, if the chosen features are the coordinates $\mathbf{s} = [u \ v]^T = [X/Z \ Y/Z]^T$ in the image of a 3D point $\mathbf{P}$ of coordinates $[X \ Y \ Z]^T$ in the camera frame, the interaction matrix related to $u$ and $v$ is given by:

$$ \mathbf{L}(\mathbf{s}, Z) = \left[ \frac{1}{Z} \mathbf{L}_v(\mathbf{s}) \ \mathbf{L}_\omega(\mathbf{s}) \right] \qquad (2) $$

where:

$$ \mathbf{L}_v(\mathbf{s}) = \begin{bmatrix} -1 & 0 & u \\ 0 & -1 & v \end{bmatrix} \qquad (3) $$

$$ \mathbf{L}_{\omega}(\mathbf{s}) = \begin{bmatrix} uv & -(1+u^2) & v \\ (1+v^2) & -uv & -u \end{bmatrix} \qquad (4) $$

The interaction matrix for more complex image features can be found in (Espiau *et al.*, 1992). The vision-based task **e** (to be regulated to 0), corresponding to the regulation of **s** to **s**\*, is defined by:

$$ \mathbf{e} = \mathbf{C}(\mathbf{s} - \mathbf{s}^*) \qquad (5) $$

where $\mathbf{C}$ is a matrix which has to be selected such that $\mathbf{C}\mathbf{L}(\mathbf{s}, Z) > 0$ in order to ensure the convergence of the control law. The optimal choice is to consider $\mathbf{C}$ as the pseudo-inverse $\mathbf{L}(\mathbf{s}, Z)^+$ of the interaction matrix. However, it depends on the depth $Z$ of each target point used in the visual servoing. An estimation of the depth can be obtained using, as in 3D visual servoing, a pose determination algorithm (if a 3D target model is available), or using a structure from known motion algorithm (if the camera motion can be measured).

Another choice is to consider $\mathbf{C}$ as a constant matrix equal to $\mathbf{L}(\mathbf{s}^*, Z^*)^+$, the pseudo-inverse of the interaction matrix computed for $\mathbf{s} = \mathbf{s}^*$ and $Z = Z^*$, where $Z^*$ is an approximate value of $Z$ at the desired camera position. In this simple case, the condition for convergence is however only satisfied in the neighborhood of the desired position, which means that the convergence may not be ensured if the initial camera position is too far away from the desired one.

If an exponential convergence is desired ($\dot{\mathbf{e}} = -\lambda\mathbf{e}$ where $\lambda$ is a positive scalar), a simple control law is given by (Espiau *et al.*, 1992):

$$ \mathbf{T} = -\lambda\mathbf{e} = -\lambda\mathbf{C}(\mathbf{s} - \mathbf{s}^*) \qquad (6) $$

The block diagram of the 2D visual servoing approach is given in Figure 4.

<figure><img src="image_6.png" /><figcaption>Fig. 4. Block diagram of the 2D visual servoing</figcaption></figure>

## 4. 2D 1/2 VISUAL SERVOING

If $n$ 3D points $\mathbf{P}_j$ on a planar target are used, it is well known that each image point $\mathbf{p}_j$, of coordinates $[u \ v \ 1]^T$ in the current camera frame $F_2$, is related to the corresponding image point $\mathbf{p}_j^*$, of coordinates $[u^* \ v^* \ 1]^T$ in the desired camera frame $F_1$, by an homography such that (Faugeras and Lustman, 1988):

$$ \alpha_j \mathbf{p}_j = \mathbf{H} \mathbf{p}_j^* \quad \{j = 0, 1, 2, \dots, n-1\} \qquad (7) $$

where $\alpha_j$ is a positive scalar and $\mathbf{H}$ is a $(3 \times 3)$ homography matrix. More precisely, $\mathbf{H}$ can be written (Faugeras and Lustman, 1988):

$$ \mathbf{H} = \mathbf{R} + \mathbf{n}^* \frac{\mathbf{t}^T}{d^*} \qquad (8) $$

where $\mathbf{R}$ and $\mathbf{t}$ are the rotational matrix and the translational vector between the current camera frame $F_2$ and the desired camera frame $F_1$ respectively (see Figure 5), $\mathbf{n}^* = [n_x^* \ n_y^* \ n_z^*]^T$ is the unit vector normal to the target plane $\pi$ expressed in $F_1$ and $d^*$ is the distance between the origin of $F_1$ and $\pi$. Let us remark that $\mathbf{H}$ is defined up

<figure><img src="image_7.png" /><figcaption>Fig. 5. Modelisation of camera displacement for 2D 1/2 visual servoing</figcaption></figure>

to a scalar factor, therefore one of the unknown $\alpha_j$ can be set to 1 without loss of generality (for
example $\alpha_0 = 1$). Equation (7) is available for each feature point in the image, thus, for n points, we have to solve a linear system of $3n$ equations and $n+8$ unknowns: that is $n-1$ unknowns $\alpha_j$ ($\{j=1, 2, ..., n-1\}$ since $\alpha_0 = 1$), and 9 unknown elements of $\mathbf{H}$. A minimum of four points is needed to perform a linear estimation. Let us emphasize that the estimation of $\mathbf{H}$ does not necessitate the knowledge of the 3D position of the target points on plane $\pi$, which makes unnecessary a 3D model of the target.

After matrix $\mathbf{H}$ is computed, $\mathbf{R}$, $\mathbf{t}/d^*$ and $\mathbf{n}^*$ can be estimated (Faugeras and Lustman, 1988). Unfortunately, in the most general case we have two different solutions. The indetermination is eliminated by choosing the solution which is such that the vector $\mathbf{n}^*$ is as co-linear as possible with the desired orientation of the camera optical axis. Let us note that the translational vector $\mathbf{t}$ is estimated only up to a scale factor since the desired distance $d^*$ is unknown. 3D visual servoing can thus not be employed. We remark that the current normal vector $\mathbf{n}$ to the plane $\pi$, expressed in $F_2$, and the current distance $d$ between frame $F_2$ and $\pi$ can easily be written in function of vector $\mathbf{n}^*$ and distance $d^*$:

$$ \mathbf{n} = \mathbf{R}\mathbf{n}^* \quad (9) $$

$$ d = d^* + \mathbf{n}^T \mathbf{t} \quad (10) $$

The distances $d$ and $d^*$ are unknown but the ratio $r = d/d^*$ can be estimated. Indeed, from equations (9) and (10) we get:

$$ r = \frac{d}{d^*} = 1 + \mathbf{n}^T \frac{\mathbf{t}}{d^*} = 1 + \mathbf{n}^{*T} \mathbf{R}^T \frac{\mathbf{t}}{d^*} \quad (11) $$

The block diagram of the 2D 1/2 visual servoing approach is given in Figure 6.

<figure><img src="image_8.png" /><figcaption>Fig. 6. Block diagram of the 2D 1/2 visual servoing</figcaption></figure>

We now describe its different parts. Our control scheme is based on the 3D estimated rotation between $F_2$ and $F_1$ (which has to reach the identity matrix), and the ratio $r$ which controls the depth between the camera and the target (and which has to reach the desired value $r^* = 1$). In order to control the 6 camera d.o.f and to maintain the target in the camera field of view, we also introduce the use of two independent visual features, such as the

image coordinates of a target point, which can be controlled by using classical 2D visual servoing.

We first consider the control of the camera orientation. Let $\mathbf{u}$ be the rotation axis and $\theta$ the rotation angle obtained from the previous partial-pose estimation. The rotational velocity of the camera $\mathbf{\Omega}$ can be expressed in function of the angular velocity $\dot{\theta}$ around the axis of rotation $\mathbf{u}$:

$$ \mathbf{u} \dot{\theta} = \mathbf{\Omega} \quad (12) $$

The translational degrees of freedom are used to maintain the target in the image and to control the ratio $r$ between the current distance $d$ and the desired distance $d^*$. A point $\mathbf{p}^*$ is chosen in the desired image, such as for example the nearest from the image center. As done for the 2D visual servoing, the chosen features are its coordinates $\mathbf{s} = [u \ v]^T$ in the image. By using equations (1) and (2) the relationship between the time variation of the chosen features and the velocity of the camera is:

$$ \dot{\mathbf{s}} = \frac{1}{Z} \mathbf{L}_v(\mathbf{s})\mathbf{V} + \mathbf{L}_\omega(\mathbf{s})\mathbf{\Omega} \quad (13) $$

where $Z$ is the depth of the corresponding 3D point. Since the 3D point belongs to the target plane $\pi$, $Z$ can be written as:

$$ Z = \frac{d}{\mathbf{n}^T \mathbf{p}} \quad (14) $$

and from equation (11), we get:

$$ \frac{1}{Z} = \frac{\mathbf{n}^T \mathbf{p}}{r d^*} = \frac{\gamma}{d^*} \quad (15) $$

where $\gamma = \frac{\mathbf{n}^T \mathbf{p}}{r}$ can be computed from image measurement and the partial-pose algorithm. Furthermore, by derivating equation (11) with respect to time, we get:

$$ \dot{\mathbf{r}} = \frac{\mathbf{n}^{*T} \mathbf{R}^T}{d^*} (\mathbf{R} \dot{\mathbf{R}}^T + \dot{\mathbf{t}}) = -\frac{\mathbf{n}^T}{d^*} \mathbf{V} \quad (16) $$

Combining this equation to equation (12) and equation (13), we get the following system:

$$ \begin{bmatrix} \dot{\mathbf{s}} \\ \dot{\mathbf{r}} \\ \mathbf{u}\dot{\theta} \end{bmatrix} = \begin{bmatrix} \frac{1}{d^*} \mathbf{M}_v & \mathbf{M}_\omega \\ \mathbf{0} & \mathbf{I} \end{bmatrix} \begin{bmatrix} \mathbf{V} \\ \mathbf{\Omega} \end{bmatrix} \quad (17) $$

where:

$$ \mathbf{M}_v = \begin{bmatrix} -\gamma & 0 & \gamma u \\ 0 & -\gamma & \gamma v \\ -n_x & -n_y & -n_z \end{bmatrix} \quad (18) $$

$$ \mathbf{M}_\omega = \begin{bmatrix} uv & -(1+u^2) & v \\ (1+v^2) & uv & -u \\ 0 & 0 & 0 \end{bmatrix} \quad (19) $$
Equation (17) is similar to equation (1). The task function approach (Espiau et al., 1992) can thus be applied. If we desire an exponential convergence of s towards s*, of r towards r* = 1 and uθ towards uθ* = 0 (with decreasing velocity λ), we obtain:

$$ \begin{bmatrix} \dot{s} \\ \dot{r} \\ \mathbf{u}\dot{\theta} \end{bmatrix} = -\lambda \begin{bmatrix} \mathbf{s} - \mathbf{s}^* \\ r - 1 \\ \mathbf{u}\theta \end{bmatrix} \quad (20) $$

From equations (17) and (20), the control law is given by:

$$ \begin{bmatrix} \mathbf{v} \\ \mathbf{\Omega} \end{bmatrix} = -\lambda \begin{bmatrix} d^* \mathbf{M}_v^{-1} & -d^* \mathbf{M}_v^{-1} \mathbf{M}_\omega \\ \mathbf{0} & \mathbf{I} \end{bmatrix} \begin{bmatrix} \mathbf{s} - \mathbf{s}^* \\ r - 1 \\ \mathbf{u}\theta \end{bmatrix} \quad (21) $$

where $\mathbf{M}_v^{-1}$, the inverse matrix of $\mathbf{M}_v$, is given by:

$$ \mathbf{M}_v^{-1} = \frac{-1}{\mathbf{n}^T \mathbf{p}} \begin{bmatrix} \frac{n_z + n_y v}{\gamma} & -\frac{n_y u}{\gamma} & \frac{u}{\gamma} \\ -\frac{n_x v}{\gamma} & \frac{n_z + n_x u}{\gamma} & \frac{v}{\gamma} \\ -n_x & -n_y & 1 \end{bmatrix} \quad (22) $$

The determinant of matrix $\mathbf{M}_v$ is $-\gamma^2(\mathbf{n}^T \mathbf{p})$. This determinant is null only if the optical axis of the camera belongs to plane $\pi$ (in this singular case, all the image points are collinear), which ensures the stability of the system in all the task space (more precisely, the half space in front of the target plane) if $\theta$ and $\mathbf{n}$ are correctly estimated.

We note that the positive scalar $\gamma$ only influence the time-to-convergence of the translational control loop. Indeed, if $\gamma$ is estimated as the positive scalar $\bar{\gamma}$ and vector $\mathbf{n}$ is well-estimated, the matrix $\mathbf{M}_v^{-1}(\bar{\gamma})\mathbf{M}_v(\gamma) = \frac{\bar{\gamma}}{\gamma}\mathbf{I}$ is always definite positive. We can also note that the camera translational velocity is proportional to the desired distance $d^*$ between $F_2$ and $\pi$. An approximate value has thus to be chosen during the off-line learning stage. However, this value has not to be precisely determined (by hand in the following experiments) since it does not influence the stability of the system, but only the time-to-convergence of the translational velocity and the amplitude of the possible tracking error due to a wrong compensation of the rotational motion. As far as the tracking error is concerned, it is proportional to the rotational velocity $\mathbf{\Omega}$ and thus disappears when the camera is correctly oriented.

## 5. EXPERIMENTAL RESULTS

The control law has been tested on a seven d.o.f. industrial robot MITSUBISHI PA10 (at EDF DER Chatou) and a six d.o.f. cartesian robot AFMA (at IRISA). The camera was mounted on the end-effector of the robot. The target is a black board with seven white marks (see Figure 7). The

extracted visual features are the image coordinates of the center of gravity of each mark. As far as camera calibration is concerned, we have used the pixel and focal lengths given by the constructor in order to compute the image coordinates u and v from their measured values in the image. The center of the image has been used for the projection of the optical axis in the image. The images corresponding to the desired and initial camera position are plotted in Figure 7a and 7b respectively.

<figure><img src="image_9.png" /><figcaption>Fig. 7. Images of the target for the desired (a) and the initial (b) camera position</figcaption></figure>

Classical 2D visual servoing has first been tested, but without success due to the too much important displacement between the initial and desired positions. Matrix C of equation (5) was chosen always equal to $\mathbf{L}^+(\mathbf{s}^*, \mathbf{Z}^*)$ because Z cannot be calculated without a geometric model of the target. The errors on the coordinates of the center of gravity of each mark are given in Figure 8. The corresponding trajectory in the image is given in Figure 9.

<figure><img src="image_10.png" /><figcaption>Fig. 8. Error on the coordinates of the image features (pixels) versus iteration number</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>Fig. 9. Trajectory of target points in the image</figcaption></figure>
On the obtained plots, we can note the divergence of the visual features, which induces the getting out of the target outside the camera field of view, and thus the failure of the 2D visual servoing. The corresponding translational and rotational velocity are given together in Figure 10.

<figure><img src="image_12.png" /><figcaption>Fig. 10. Translational velocity V (cm/s), rotational velocity Ω (dg/s) versus iteration number</figcaption></figure>

We now present the results obtained using our new 2D 1/2 visual servoing scheme. In the presented experiment, $d^*$ is set to 20 cm. From the estimated homography, we get a partial estimation of the camera displacement. For example, the estimated rotational displacement, using the initial and desired images, was $\bar{r}_x = 29.6dg$, $\bar{r}_y = -32.5dg$, $\bar{r}_z = 95.9dg$ (while the real displacement was $r_x = 28.1dg$, $r_y = -34.0dg$, $r_z = 96.1dg$). Similarly, the estimated direction of translation was $\frac{\bar{r}_x}{\|\bar{r}_x\|} = -0.19$, $\frac{\bar{r}_y}{\|\bar{r}_y\|} = 0.95$, $\frac{\bar{r}_z}{\|\bar{r}_z\|} = -0.23$ (while the real displacement was $t_x = -3.8cm$, $t_y = 23.6cm$, $t_z = -5.9cm$ which implies $\frac{t_x}{\|t_x\|} = -0.15$, $\frac{t_y}{\|t_y\|} = 0.95$, $\frac{t_z}{\|t_z\|} = -0.23$. The used algorithm is thus quite precise (maximal rotational error is around 2 dg, as well as the angle error on the direction of translation) in despite of the coarse calibration which has been used.

The exponential decreasing of the estimated rotation, of ratio $r = d/d^*$ and of the coordinates $u$ and $v$ of the target point in the image are given in Figure 11, Figure 12 and Figure 13 respectively.

<figure><img src="image_13.png" /><figcaption>Fig. 11. Rotation $u\theta$ (dg) versus iteration number</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Fig. 12. Ratio $r = d/d^*$ versus iteration number</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 13. Error on the coordinates of the target center of gravity (pixels) versus iteration number</figcaption></figure>

As can be seen on the plots, the obtained results are particularly stable and robust. In this experiment, $\lambda$ is set to 0.1. A smaller value just implies a slower time-to-convergence. Higher values could also be chosen. However, the system (whose rate is 10 Hz) becomes unstable when $\lambda > 2.5$.

A good approximation of $d^*$ reduces the amplitude of the tracking error due to the rotational movement. A such small perturbation can be observed in Figure 13 from iteration 120 to 250 where the tracking error is approximately 3 pixels. Let us note that the tracking error completely disappears (after iteration 250) when the rotational motion has no more influence.

The outputs of the control law, **V** and **Ω** are given in Figure 14.

<figure><img src="image_16.png" /><figcaption>Fig. 14. Translational velocity V (cm/s), rotational velocity Ω (dg/s) versus iteration number</figcaption></figure>
Once again, we can note the stability and the robustness of the control law. Finally, the error on the image coordinates of each target point is given in Figure 15 and the corresponding trajectory in the image is given in Figure 16.

<figure><img src="image_17.png" /><figcaption>Fig. 15. Error on the coordinates of the image features (pixels) versus iteration number</figcaption></figure>

We can note the convergence of the coordinates to their desired values (the control scheme is stopped when maximal error is less than 0.5 pixels), which demonstrates the correct realization of the positioning task. Let us recall that these image coordinates are only used to compute the homography between initial and final camera positions.

<figure><img src="image_18.png" /><figcaption>Fig. 16. Trajectory of target points in the image</figcaption></figure>

## 6. CONCLUSION

In this paper we have proposed a new approach to vision-based robot control. Contrarily to the pose-based approach, the proposed control law does not need any geometric model of the target. Furthermore, the domain of convergence for the 2D 1/2 visual servoing (the half space in front of the target) is larger than for the 2D visual servoing. Finally, experiments show that a precise camera calibration is not needed. This is due to the fact that the homography estimation is iteratively performed in conjunction with a closed-loop control law. The main hypothesis of our actual scheme is that the image points used to estimate the homography correspond to coplanar 3D points. Future work will thus concern the

generalization of homography estimation in the case of any non-planar target using algorithms such as those presented in (Couapel and Bainian, 1995) and (Boufama and Mohr, 1995).

## ACKNOWLEDGEMENTS

This work was supported by the national French Company of Electricity Power: EDF. We are grateful to the team manager and the researchers of the Teleoperation/Robotics group, at DER Chatou, for their participation and help.

## 7. REFERENCES

* Boufama, B. and R. Mohr (1995). Epipole and fundamental matrix estimation using the virtual parallax property. In: *IEEE International Conference on Computer Vision*. Cambridge, USA. pp. 1030–1036.

* Couapel, B. and K. Bainian (1995). Stereo vision with the use of a virtual plane in the space. *Chinese Journal of Electronics* **4**(2), 32–39.

* Dementhon, D and L. S. Davis (1995). Model-based object pose in 25 lines of code. *International Journal of Computer Vision* **15**(1/2), 123–141.

* Espiau, B. (1993). Effect of camera calibration errors on visual servoing in robotics. In: *3rd International Symposium on Experimental Robotics*. Kyoto, Japan.

* Espiau, B., F. Chaumette and P. Rives (1992). A new approach to visual servoing in robotics. *IEEE Trans. on Robotics and Automation* **8**(3), 313–326.

* Faugeras, O. and F. Lustman (1988). Motion and structure from motion in a piecewise planar environment. *International Journal of Pattern Recognition and Artificial Intelligence* **2**(3), 485–508.

* Hashimoto, editor, K. (1993). *Visual Servoing: Real Time Control of Robot manipulators based on visual sensory feedback*. Vol. 7 of *World Scientific Series in Robotics and Automated Systems*. World Scientific Press. Singapore.

* Hutchinson, S., G. D. Hager and P. I. Corke (1996). A tutorial on visual servo control. *IEEE Trans. on Robotics and Automation* **12**(5), 651–670.

* Weiss, L. E., A. C. Sanderson and C. P. Neuman (1987). Dynamic sensor-based control of robots with visual feedback. *IEEE Journal of Robotics and Automation* **3**(5), 404–417.

* Wilson, W. J., C. C. W. Hulls and G. S. Bell (1996). Relative end-effector control using cartesian position-based visual servoing. *IEEE Trans. on Robotics and Automation* **12**(5), 684–696.