# Modelling 3D Unknown Object by Range Finder and Video Camera and Updating of a 3D Database by a Single Camera View

Camille Edie Nzi, Jean Triboulet, Malik Mallem, Florent Chavand

► To cite this version:

Camille Edie Nzi, Jean Triboulet, Malik Mallem, Florent Chavand. Modelling 3D Unknown Object by Range Finder and Video Camera and Updating of a 3D Database by a Single Camera View. Global Journal of Pure and Applied Sciences, 2005, 11 (1), pp.153-163. 10.4314/gjpas.v11i1.16476 . lirmm-00105362

HAL Id: lirmm-00105362

https://hal-lirmm.ccsd.cnrs.fr/lirmm-00105362

Submitted on 12 Apr 2018

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# MODELING 3D UNKNOWN OBJECT BY RANGE FINDER AND VIDEO CAMERA AND UPDATING OF A 3D DATABASE BY A SINGLE CAMERA VIEW.

C. NZIE, J. TRIBOULET, MALIK MALLEM and F. CHAVAND

(Received 24 June 2003; Revision Accepted 6 July 2004)

## ABSTRACT

The device consists of a camera which gives the HO an indirect view of a scene (real world); proprioceptive and exteroceptive sensors allowing the recreating of the 3D geometric database of an environment (virtual world). The virtual world is projected onto a video display terminal (VDT). Computer-generated and video images are superimposed. The man-machine interface functions deal mainly with on line building of graphic aids to improve perception, updating the geometric database of the robotic site, and video control of the robot. The superimposition of the real and virtual worlds is carried out through a calibration of the multisensor system.

First, the sensing system and several methods used for modeling an environment's geometric database in telerobotics have been presented. The operator models unknown objects (of cylindrical or polyhedral pattern) using a video camera(VC) image and a range finder(RF). When the quality of images is poor, the range finder, which is mounted on a site and azimuth rotation turret, brings an indispensable complement by measures of depth. The improvement offered by the cooperation VC/RF has been shown for small depths (below 1.6m). In some particular cases we propose to achieve the fusion of multisensor redundant data, in order to reduce the uncertainty.

Then the updating of environment's 3D geometric database in telerobotics by single camera view is presented. It concerns the pose determination of known objects using 2D clues obtained through video images. A two step algorithm is implemented and assessed. The first step is available in case of object large motion, it is a geometric algorithm, which doesn't use redundant data; it provides a first estimate of the pose. The second step is available in case of object small motion. A linear approach is carried out to determine the pose. It allows the use of redundant data, so it improves the accuracy achieved after the first step.

**KEY WORDS:** Teleoperation, Telerobotics, Range finder; Video camera; Sensor modeling; 3D database updating, Image matching.

## INTRODUCTION

Our laboratory is designing a Multimedia Control Interface for Teleoperation (MCIT) whose purpose is to provide the human operator with a visual aid for the perception of remote scenes and for robot control in cases of indirect viewing via a video camera(VC). The device consists of a camera which gives the HO an indirect view of a scene (real world); proprioceptive and exteroceptive sensors allowing the recreating of the 3D geometric database of an environment (virtual world). The virtual world is projected onto a video display terminal (VDT). Computer-generated and video images are superimposed. The man-machine interface functions deal mainly with on line building of graphic aids to improve perception, updating the geometric database of the robotic site, and video control of the robot. This functional architecture is described in (Malle, Colle And Chavand, 1993). The superimposition of the real and virtual worlds is carried out through a calibration of the multisensor system; models of the sensors and several calibration methods are presented in (Chekhar, Colle, Louki And Chavand, 1994).

This paper deals with the modeling of unknown 3D object by Range Finder(RF) and VC and updating of a 3D database(3DDB) by a single camera view. The first part present the modeling of unknown cylindrical and polyhedral objects. The sensing system is presented first; then we present methods used to model unknown objects, and we carry out an error analysis leading to the introduction of the second part: The updating of the
geometric database. This second part present the principle of 3D object reconstruction using 2D video clues. A two step algorithm is implemented. The first step is used in case of large motion amplitude and the second in case of small motion of an object. The method of object large motion is a geometric method The translation and the rotation of the object is thus determined with little accuracy. The method of object small motion uses redundant data in order to improve accuracy.

The representation of objects is the surface boundary representation (B-Rep) : objects are defined by their boundaries ( vertices, edges, faces ). The objects' location is defined by the position-orientation of an object-referenced coordinate frame with respect to the work reference coordinate system.

II. THE SENSING SYSTEM

The slave site is equipped with exteroceptive sensors (one VC coupled with a 3D sensing device) and proprioceptive sensors (on-robot sensors measuring its own current configuration). The VC gives the operator an indirect view of the scene. Computer generated images (virtual world) are built according to the object's model, which is located with respect to a coordinate frame Ro chosen in the slave site, and usually linked to one robot's joint. Figure II.1 shows sensors and coordinate frames distribution.

(a) Video Camera

We use a black and white VC with a 8.5 mm focal length, coupled with a graphic card providing a 512x256
pixels resolution. We work in the screen area presenting less than 1 pixel of geometric distortion. The graphic
card allows the operator to point out the relevant areas on the screen, or to superimpose a computer
generated image on the video image. The VC model defines the transformation matrix C'₃x₄ associating to
each 3D point M(x₀, y₀, z₀, 1) in R₀ its 2D image m(u, v) in the screen plane. The u and v coordinates
are expressed in pixels : (u:s v:s s') = C₃x₄(x₀ y₀ z₀ 1)ₖ₀

where the parameter s is proportional to the distance VC/point. The VC calibration leads to the determination
of the matrix C₃x₄ and of these parameters (Chekhar, Colle, Loukil and Chavand, 1994).

(b) 3D system

A time of flight RF (PS10) with a λ = 905 nm measures distances between 0.6 and 10 m. The RF is mounted on a two degrees of freedom orientable turret (θx, θy) allowing the spherical coordinates of a measured point to be expressed. The geometric model of the RF is represented by the homogeneous matrix expressing the transposition between Ro and the coordinate frame Rr linked to the RF, and in which the measure is done
(x₀ y₀ z₀ 1)ₖ₀ = Tₚₐ⁻¹(0 0 -ρ 1)ₖr where Tₚₐ is determined by calibration (LOUKIL, 1993).

III. MODELING UNKNOWN OBJECTS

This function achieves the fusion of data provided by the operator and by sensors in order to create and set a
model of an unknown object in the geometric 3DDB. Qualitative information is given by the operator while
quantitative information is obtained from sensors (KOIZUMI and TOMITA, 1988, P-681).

té data provided by the operator:

- semantic: general shape (cylinder, polyhedron, pyramid, etc.)

- geometric: vertical, oblique, etc.

- topological: situation of object in environment ("vertical cylinder lies on the table").

These data allow the building of an embodying volume for unknown object.

té data provided by the exteroceptive sensors:

These numerical data provided by the VC and the RF are used to determine the size and location of the
embodying volume. Methods concerning the modeling of cylindrical and pyramidal volumes are presented. The
video image allows to determine the interpretation planes tangent to the object and running through the optical
center (see Fig. IV.1). With the RF, the geometric characteristics of the pattern contained in the volume
bounded by the interpretation planes, are determined.

III.1. Modeling polyhedral volume

The general idea to model such a volume is to determine the faces' characteristics. For example, let's model
the ABCD face : the VC allows the determination of the interpretation planes(P₁,...,P₄) which contain the
edges AB, BC, CD, DE relating to this face. By pointing a set of points belonging to the face, the RF allows the
determination of the equation of the face plane. The intersections of the face plane with the interpretation
planes give the edge equations in the 3D space(Fig III.1)
<figure><img src="image_2.png" /><figcaption>Figure III.1 : Polyhedral modelisation</figcaption></figure>

For a face which is not parallel to the z axis, the relating plane equation is : $ax+by+z+d=0$. So for n points we also obtain a linear equation which can be solved using least squares method :

$$A \cdot X = B;$$

with:

$$A = \begin{bmatrix} x_1 & y_1 & 1 \\ x_2 & y_2 & 1 \\ x_n & y_n & 1 \end{bmatrix}, \quad X = \begin{bmatrix} a \\ b \\ d \end{bmatrix} \quad \text{and} \quad B = \begin{bmatrix} -z_1 \\ -z_2 \\ -z_n \end{bmatrix}$$

The same process is used to model the other faces, but the fact that an edge belongs to two faces must be taken into account. Results are given in table III.1 and errors are represented in (Fig III.2). In the results presented, errors in height, width, and length are respectively smaller than 5%, 7% and 6%. Error in length is greater because the angle between the normal to the side face and the RF beam is too large. For the small sensor/object distance(1m) errors are greater because of the VC/RF parallax.

<table><caption>Table III.1: Results obtained with the different interpretation plane</caption><thead><tr><th>Dist. object/<br>sensor(in m)</th><th>width(in cm)</th><th>Height<br>(in cm)</th><th>length<br>(in cm)</th><th>Volume<br>(in dm<sup>3</sup>)</th></tr></thead><tbody><tr><td>1.3</td><td>0.67</td><td>0.73</td><td>0.52</td><td>0.20</td></tr><tr><td>1.4</td><td>0.66</td><td>0.58</td><td>0.53</td><td>0.19</td></tr><tr><td>1.5</td><td>0.55</td><td>0.53</td><td>0.48</td><td>0.14</td></tr><tr><td>1.6</td><td>0.37</td><td>0.38</td><td>0.47</td><td>0.13</td></tr></tbody></table>

Object dimensions : $w \times h \times l = 9cm \times 14cm \times 9.4cm$

## III.2. Modeling cylindrical volume

Here are presented modeling methods using either a sole sensor (RF) or the two sensors(VC and RF) in cooperation. We determine the position/orientation of the object's axis, as well as two radius (in the case of conic objects).

### (a) V C/RF cooperation : tangential planes method

The VC allows the determination of the equations of the two interpretation planes $P_1$ and $P_2$ relative to the lateral generating lines (AD) and (BC) of the cylinder (Fig. III.4). Figure III.3 shows the cylinder profile as obtained by the RF which gives interpretation planes(P3, P4, P5) (Fig III.4). Two horizontal scannings done with the RF and the interpretation planes determine the cylinder. For our object, errors on radius and height remain less than 4mm.

<figure><img src="image_3.png" /><figcaption>Figure III.3 Cylinder profile</figcaption></figure>
Object dimensions : r = 5.1cm, h = 16.6cm

<figure><img src="image_4.png" /><figcaption>Figure III.4 Tangential planes method</figcaption></figure>

**(b) VC/RF cooperation : bisecting planes method**

The interpretation planes $P_{c1}$ and $P_{c2}$ provided by the VC are used to determine the bisecting plane $P_{A1}$ (Fig. III.5). A complete profile measure is done with the RF, for two different heights. Tangential lines to the object are used to determine two interpretation planes proper to the RF, $P_{RF1}$ and $P_{RF2}$. These two new planes lead to the equation setting of a second bisecting plane $P_{MIRF}$ (Fig. III.5). The intersection of the two planes $P_{A1}$ and $P_{MIRF}$ determines the equation of the object's axis. Then, the determination of the cylinder is done with $P_{RF1}$ and $P_{RF2}$. The errors on the radius and on the height are greater with this approach : 1.2 cm(± 5mm).

<figure><img src="image_5.png" /><figcaption>Figure III.5 Bisecting planes method</figcaption></figure>

**(c) Range finder alone : tangential planes method**

This approach uses again the process presented in the tangential planes method, but the set of the three planes needed is obtained by ranging measures only. We use the tangential lines lateral to the object to define two lateral planes $P_{RF1}$ and $P_{RF2}$. The third one $P_3$ is shown in Fig. III.4. The axis equation is obtained expressing the equidistant. The radius and height are determined by projection on the interpretation planes. Errors on radius and height are comparable to those of precedent methods.

**(d) CONCLUSION**

Table III.2 and FigIII.6 gives results in depth for the three methods. The one using interpretation planes by VC/RF cooperation seems to have the best results: Error is less than 5mm. The error in case of bisecting plane using VC and RF decrease when the distance object/sensors increase. It is the contrary in case of tangential plane with RF only, where the error increase with the distance object/sensor(FigIII.6).
<figure><img src="image_6.png" /><figcaption>Figure. III.6 Depth error for the three methods</figcaption></figure>

Cylinder dimensions: R = 5.1cm, h = 16.6cm, Depth=5.1cm

<table><caption>Table III.2: Result in Depth for the three methods</caption><thead><tr><th>Distance<br>(in m)</th><th>tangential<br>planes(Vc+RF)<br>(in cm)</th><th>bissecting<br>planes(VC+RF)</th><th>tangential<br>planes(RF)<br>(in cm)</th></tr></thead><tbody><tr><td>1</td><td>4.8314</td><td>3.5098</td><td>5.6332</td></tr><tr><td>1.2</td><td>4.7523</td><td>3.5512</td><td>5.6775</td></tr><tr><td>1.4</td><td>4.6765</td><td>4.1416</td><td>4.1501</td></tr><tr><td>1.6</td><td>4.6861</td><td>4.3974</td><td>4.0068</td></tr></tbody></table>

IV. THE UPDATING 3DDB BY A SINGLE CAMERA VIEW

First we present the problem of modeling known objects. In this process, the HO uses 2D data provided by the video image. The data regarding the object's shape are known. The HO has only to determine the correct object's pose. He himself makes the matching of the video and computer-generated images of the object. The equation of each edge Li of the object is known with respect to a coordinate frame (Ro) attached to the object. The process must determine the position-orientation (Rm) of (Ro) in the environment i.e. the rotation matrix R and the translation vector T defining the coordinate transformation between (Ro) and (Rm) (Fig IV.1). The visual marks used by the HO are the image li of the edge Li.

<figure><img src="image_7.png" /><figcaption>Figure IV.1 : Position-Orientation of a known object</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Figure IV.2: The interpretation plane</figcaption></figure>

The rotation R and the translation T (Fig IV.2) are determined with the equation (1) and (2).

$$ \vec{Ni}(R \cdot \vec{Voi}) = 0 \quad (1) $$

$$ \vec{Ni}(T + RMoi) = 0 \quad (2) $$

where (·) is the dot product

Three edges Li (i = I, 2,3) are used to solve the equation(1) to determine R, then to find T using the equation(2). Many solutions have been proposed to solve this problem: (Dhome, Richetin, Lapreste and Rives, 1989), (Horaud, 1987) have a geometric approach; (Yuan, 1989), (Lowe, 1987), (Horaud, Monga, 1993) have a numerical approach. All these approaches are heavy in processing time. The direct resolution of the orientation from (Rm) to (Ro) is complex. In order to reduce computation complexity, we introduce intermediary coordinate frames. In the next sections, we present the two steps of the method, the first one is used in case of object large motion and the second one in case of object small motion. Both these algorithms can be used successively, the first one provides an estimate of the pose and the second one improves the accuracy obtained in the first case.

## IV.1. CASE OF LARGE AMPLITUDE

### b) Contribution of our approach

The method consists in finding the rotation R and the translation T from (Ro) to (Rm) through a geometrical approach. It is based on DHOME'S work (Dhome, Richetin, Lapreste and Rives, 1989). The direct resolution introduces the three Euler angles and gives complex equations which we can't solve. In order to reduce complexity, Dhome introduces two coordinate frames Rcl and Rml (Fig IV.3), in such a way that Rclml depends on two angles only: So it is possible to determine the solution.

<figure><img src="image_9.png" /><figcaption>Figure IV.3: The different reference coordinate frames</figcaption></figure>

The method of large amplitude is based on DHOME'S works. However DHOME works with the internal and external camera models separately. His method is based on internal parameters of the camera and after each object's motion calculates the external camera model (transformation matrix from (Rm) to (Rc)). Therefore separating the external and internal creates a drawback (problem of stability) (LOUKIL A., 1993).

The originality of our approach consists in working with the global camera model. This way we avoid the instability constraint. It both reduces the number of calculation and improves the accuracy of the method.

b). The different reference coordinate systems and the determination of the rotation R from (Ro) to (Rm)
First let us defined the used matrices

### Roc1, rotation matrix from (Ro) to (Rcl)

Camera global model calibration $C_{3x4}$, expresses the relationship between the 2D coordinates $p(u,v)$ of the image of a given 3D point $P(\begin{bmatrix} X \\ Y \\ Z \end{bmatrix})$ in the reference (Ro), we have: $\begin{bmatrix} x^*u \\ x^*v \\ x \end{bmatrix} = C_{3x4} \begin{bmatrix} X \\ Y \\ Z \end{bmatrix}$ (3): with:

$$C_{3x4} = \begin{bmatrix} C'11 & C'12 & C'13 & C'14 \\ C'21 & C'22 & C'23 & C'24 \\ C'31 & C'32 & C'33 & C'34 \end{bmatrix}$$

The development of relation (3) gives two independent equations:

$$\begin{cases} mu.P + av = 0 \\ mv.P + av = 0 \end{cases} \quad (4)$$

Where

$$P(\begin{bmatrix} X \\ Y \\ Z \end{bmatrix})$$

$$mu = \begin{bmatrix} C'11 - u^*C'31 \\ C'12 - u^*C'32 \\ C'13 - u^*C'33 \end{bmatrix}$$

$$mv = \begin{bmatrix} C'21 - v^*C'31 \\ C'22 - v^*C'32 \\ C'23 - v^*C'33 \end{bmatrix}$$

$$mu = (C'14 - u^*C'34)$$

and

$$av = (C'24 - v^*C'34)$$

Relation (4) represents two plane equations. The cross product of the normal vectors to the two planes(nu and nv) gives the visual ray running through p. Visual ray is a line defined by camera optical center(c) and 2D pointed out point(p). The HO uses the image li(i= 1 to 3) of the edge Li(i= 1 to 3) as visual marks. The visual ray of the first 2D point of li defines the $\vec{x}$ axis of (Rcl). So, the $\vec{x}$ axis of (Rcl) is in the interpretation plane of li. The normal vector of the interpretation plane given by li1 defines the $\vec{y}$ axis of (Rcl). We define $\vec{z}$ axis by the cross product of $\vec{x}$ and: The coordinates of ($\vec{x}, \vec{y}, \vec{z}$) form the columns of the matrix Roc1 or the lines of the inverse matrix Roc10.

**Note:** Only li image of Li is used to determine Roc1.

### Rmlm, rotation matrix from (Rm) to (Rml)

Li(i= 1 to 3) are the edges of the current position of the object. The initial coordinates of Li in (Ro) are the same as the coordinate of Li in (Rm). The $\vec{x}$ axis of (Rml) is carried by Li. So, the $\vec{x}$ axis of (Rml) is in the interpretation plane of li1. The $\vec{z}$ axis of (Rml) is given by the cross product of $\vec{i1}$ and $\vec{i2}$ respectively carried by L1 and L2. We define the $\vec{y}$ axis by the cross product of $\vec{z}$ and $\vec{x}$. The coordinate of ($\vec{x}, \vec{y}, \vec{z}$) form the columns of the matrix Rmlm or the lines of the inverse matrix Rmlm.

**Note:** Only L1 and L2 are used to define Rmlm.

### Rclml, rotation matrix from (Rcl) to (Rml)

We have seen, that both (Rcl) and (Rml) $\vec{x}$ axis are in the interpretation plane of li. Also, the $\vec{y}$ axis of (Rcl) is the normal vector of the interpretation plane of li1. Therefore, the rotation matrix from (Rcl) to (Rml) depends on two rotations only. The rotation $(\vec{y}, \beta_1)$ so that the two $\vec{x}$ axis become parallel and the rotation $(\vec{x}, \alpha)$ so that the two $\vec{y}$ axis become parallel. If these correspondences are achieved, the two $\vec{z}$ axis become automatically parallel. Now the rotation matrix from (Ro) to (Rm) is given by: $R = Roc1 Rclml Rmlm$

So, the determination of R consists in finding Rclml which depends on unknown angles $(\alpha, \beta)$. We obtain a polynomial equation of degree 8 (see Appendix). The computation gives $\alpha$ and $\beta$ and thus Rclml. Equations are set in form with symbolic calculation software, whose syntax is compatible with C language. It permits the integration into our program. Polynomial equation of degree 8 is obtained. It is computed with a numerical method based on the determination of the eigenvalue of the companion matrix (William, 1992).

### Determination of the translation T from (Ro) to (Rm)
The translation is given by equation:

$$ \vec{N_i}_{Ro} \cdot C \vec{M_i}_{Ro} = 0 \quad \Leftrightarrow \quad \vec{N_i}_{Ro} \cdot (T + R \vec{M_i}_{Ro} - C_{Ro}) = 0 \quad (5) $$

where: $M_i_{Ro}$ is a 3D point of Li, $C_{Ro}$ is the Center of (Ro)

Note: The calculation of the translation T depends on the rotation R. The translation includes three unknown parameters(a, b, c). Edges Li(i=1 to 3) and points Mi(i=1 to 3) are used for translation determination ( where each point Mi belongs to the edge Li).

The edges Li(i= 1 to 3) used to determine the rotation R are triangular. This configuration gives 4 solutions in rotation. The solutions (1') and (2') are rejected because they are not in the direction of the camera view. A test of distances between 2D points of image obtained and initial image allows the choice of the good solution (1) (FigIV. 4). However in the case of the translation T, we must have three no-coplanar edges Li(i= 1 to 3).

<figure><img src="image_10.png" /><figcaption>Figure IV.4: The possible solutions
Figure IV.4: The possible solutions</figcaption></figure>

## IV.2. CASE OF SMALL AMPLITUDE

In this case, we have implemented and tested an original method. With this method, $\vec{r} = 0$ is expressed as a function of the vector rotation $\vec{r}$, which represents the axis and angle 0 of the rotation. This method consists in linearizing the motion with respect to the three coordinates of $\vec{r}$. It is available to rectify the orientation of an object in order to get the computer-generated and video images rigorously superimposed.

For small rotations: $\vec{r} = \vec{r} + \vec{R}_x \vec{r}$ Where $\vec{r} = \vec{r} + d\vec{r}$ with $d\vec{r} = \vec{R}_x \vec{r}$ (6)

$\vec{r}$: vector director of a 3D edge of the object in initial position, $\vec{r}'$ vector director of the same edge after motion. In equation (6) the translation doesn't appear, so we introduce the translation with reducing the expression to points, we have: $d\vec{P} = \vec{T} + \vec{R}_x \vec{P}$; O is the center of (Ro), so: $d\vec{P} = \vec{T} + \vec{R}_x \vec{P}$ (7)

The derivation of the equations (4) (case of small motion: rotation and translation) gives: $\begin{cases} d\vec{u}_P + d\vec{u}_P + d\vec{u}_P = 0 \\ d\vec{v}_P + d\vec{v}_P + d\vec{v}_P = 0 \end{cases}$ (8)

$d\vec{P}$ is given by equation (7), we introduce the transformation: $\begin{cases} d\vec{u}_P(\vec{R}_x \vec{P}) = (\vec{P} \cdot \vec{u}_P) \cdot \vec{R} \\ d\vec{v}_P(\vec{R}_x \vec{P}) = (\vec{P} \cdot \vec{v}_P) \cdot \vec{R} \end{cases}$ we obtain: $\begin{cases} d\vec{u}_P \cdot (\vec{R}_x \vec{P}) = (\vec{P} \cdot \vec{u}_P) \cdot \vec{R} \\ d\vec{v}_P \cdot (\vec{R}_x \vec{P}) = (\vec{P} \cdot \vec{v}_P) \cdot \vec{R} \end{cases}$ (9) which can be written as matrix: $A \cdot \vec{X} = \vec{B}$ (10)

where: $\vec{X} = (a, b, c, Rx, Ry, Rz)'$ and $\vec{T} = (a, b, c)'$, $\vec{R} = (Rx, Ry, Rz)'$

The equation (10) can be solved by using the least squares method, if more than three points P are taken.
IV.3. RESULTS

The algorithm assessment is carried out first with a simulation process and then with measurements on real objects.

Object tested : parallelepiped (height = 13.2 cm, width=23.1 cm, length = 35cm). Camera : focal: 8.5mm. Resolution 512x256 pixels. Average distance-camera/object: 2m.

We have two kinds of tests: In the first test the 2D coordinates are given by the computer. In this test there are no noise measurements so the result gives the accuracy of the method. In the case of large amplitude the algorithm works with 10⁻⁶ errors for the rotation and 10⁻⁶ m errors in translation (fig V.1a and V.1b). In the case of small amplitude, we have tested the validity domain of the small motion. The algorithm is accurate for a rotation which is not greater than 18° and the translation less than 5cm. The algorithm is considered accurate when the error in the rotation angle value is less than 0.5°. In the second test the HO points out the 2D points on the video image. We have applied successively the two algorithms. The results are presented (Figure V.2a and V.2b). The algorithm of large amplitude gives error less than 0.7° in rotation, but the error in translation is not small, up to 4cm. The method of small amplitude gives an error less than 0.3° in rotation and less than 1cm in translation. The use of redundant data in the second step provides a better accuracy and reduces standard deviation of the error(Table V.1). The algorithm of large amplitude gives 1.4cm and 1.3cm of mean value and standard deviation of error in translation and 0.23° and 0.19° in rotation. Therefore the method of small amplitude gives 6mm and 2mm of mean value and standard deviation of error in translation and 0.1° and 0.1° in rotation. (g: Large motion, p: Small motion)

<figure><img src="image_11.png" /><figcaption>Figure V.1a: Error in translation(large amplitude)</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Figure V.1b: Error in rotation(large amplitude)</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Figure V.2a: Error in translation for the two methods</figcaption></figure>

<figure><img src="image_14.png" /><figcaption>Figure V.2b: Error in rotation for the two methods</figcaption></figure>

<table><caption>Table V.1: Mean value and standard deviation of the errors</caption><thead><tr><th rowspan="2"></th><th colspan="2">Mean</th><th colspan="2">Standard deviation</th></tr><tr><th>large motion</th><th>small motion</th><th>large motion</th><th>small motion</th></tr></thead><tbody><tr><td>tz(m)</td><td>0.01378</td><td>0.00563</td><td>0.01268</td><td>0.00233</td></tr><tr><td>R(°)</td><td>0.23342</td><td>0.09810</td><td>0.19033</td><td>0.11136</td></tr></tbody></table>
V CONCLUSION

In the first part, the sensing system and methods used for updating an environment's geometric database in Telerobotics have been presented in this paper. The operator models unknown objects (of cylindrical or polyhedral pattern) using a video image and a RF. The time of flight RF presents the smallest error over the largest measure domain. The improvement offered by the cooperation VC/RF has been shown for small depths (below 1.6 m). In the case of cylindrical objects several modeling methods have been presented and assessed, and in some particular cases we propose to achieve the fusion of multisensor redundant data, in order to reduce the uncertainty.

In the second part, method for updating an environment's 3D geometric data base in Telerobotics has been described. The method was tested when the shape of objects is known. It's composed of two steps which are applied successively. In the first step, when the motion of the object is large, a geometric method is used. An approximate solution is obtained. In the second step, the case of small motion of an object, we use an algorithm which improves the accuracy obtained in the first step. The tests for the algorithm accuracy were carried out first by simulation and then in real case. For the simulation, we don't take the HO into account. For the small amplitude method, the simulation tests the limit of the algorithm validity. The rotation amplitude must be less than 18°. Below 18°, the error in the rotation angle value is less than 0.5°. For the large amplitude method, the simulation tests the validity of the algorithm. Accuracy obtained are (10⁻⁶)" for the rotation and 10⁻⁶m in translation. For the experiment on real object the 2D image coordinates are pointed out by the HO. The noise measurement reduces the accuracy of the large amplitude method. The error in rotation is however less than 0.7", but the error in translation is not small, up to 4cm. Applying the method of small amplitude, the error in rotation becomes less than 0.3° and in translation less than 1 cm. These methods are tested on a real Telerobotics environment. Results show that accuracy obtained is sufficient for Telerobotics task performing.

REFERENCES

Chekhar Y., Colle, E., Loukil A., and Chavand F., 1994. Calibration multicapteur d'un système d'aide à la téléopération. APII, Hermès, France, 28(–n°6,): 599623.

Dhome, M., Richetin, M., Lapreste, J. T., and Rives G., 1989, Determination of the attitude of 3D objects from a single perspective view. IEEE Transactions on Pattern Analysis and Machine Intelligence. II(12): 1265-1278.

Horaud, R., 1987, New methods for Matching; 3D objects with single perspective views. IEEE Transactions on Pattern Analysis and Machine Intelligence. PAMI-9.(3): 401-412.

Horaud, R. and Monga, O., 1993, Vision par Ordinateur, Outils Fondamentaux. Hermes, Paris, France.

Koizumi, M. and Tomita, F., 1988. Qualitative and quantitative of solid models and images of 3D objects. Proc. 9th Int. Conf. on Pattern Recognition. pp. 681.

Loukil, A., 1993. Interface Homme-Machine de contrôle-commande en robotique téléopérée. Thèse de doctorat, Université. Evry(France).

Lowe D., 1987, Three-dimensional Object Recognition from Single Two-dimensional Images. Artificial Intelligence, 31: 355-395.

Mallem M., Colle E., and Chavand, F., 1993. A Multimedia Control Interface in Teleoperation. IEEE/SMC'93 Conference. System Engineering in the Service of Humans. Le Touquet, France.

Yuan J. S. C., 1989. A general photogrammetric method for determining object position and orientation. IEEE Transaction on Robotics and Automation, 5(2): 129-142.

William H., Press et al, 1992. Numerical Recipes i1: C, CAMBRIDGE University.
APPENDIX

**Determination of the polynomial equation:** The visual mark Li gives the normal vector $\vec{N_i}$ of the interpretation plane which contains the edge Li and the associated vector $\vec{i}$. We have: $\vec{N_i}_{R_{i1}} \cdot \vec{i}_{R_{i1}} = 0$ (i= 1 to 3) (A1)

However we know $\vec{N_i}_{R_{i1}}$ and $\vec{i}_{R_{i1}}$,

so we use the different transformation matrices, we obtain: $$ [R_{C1} \vec{N_i}_{R_{i1}}] [R_{C1m1} R_{m1m} \vec{i}_{R_{i1}}] = 0 \quad (A2) $$

where: $$ R_{i,1m1} = R_i R_{i1} = \begin{bmatrix} \cos \beta & 0 & \sin \beta \\ 0 & 1 & 0 \\ -\sin \beta & 0 & \cos \beta \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & \cos \alpha & -\sin \alpha \\ 0 & \sin \alpha & \cos \alpha \end{bmatrix} $$

The development of the dot product in the equation (A2) for (i=1 to 3) gives three equations:

$$ G_i(\alpha, \beta) = 0 \quad (i= 1 \text{ to } 3) $$

We use the two last equations:

$$ G_2: s1(\alpha) * \cos(\beta) + s2(\alpha) * \sin(\beta) = s3(\alpha) \quad (A3) $$

$$ G_3: s4(\alpha) * \cos(\beta) + s5(\alpha) * \sin(\beta) = s6(\alpha) $$

**Note:**

Below, we present conditions needed to solve equation (A3).

- Two equations are necessary to find $\alpha$ and $\beta$

- Both Li(i= 1 to 3) and their 2D images must be different

- Equation G1 isn't used because it's invariable with respect to $\alpha$

- Edges Li(i=2 to 3) must not be parallel to L1

Equation, (A3) gives $\cos \beta$ and $\sin \beta$ expressed as a function of $\alpha$.

We introduce: $\cos^2 \beta + \sin^2 \beta = 1$ and then obtain an equation:

$$ F(\alpha) = 0 \quad (A4); \quad \text{We introduce } \cos \alpha = \frac{1-t^2}{1+t^2} \text{ and } \sin \alpha = 2 \cdot \frac{t}{1+t^2} $$

We obtain a polynomial equation of degree 8. The computation gives $\alpha$ and $\beta$ and thus $R_{C1m1}$.