The Epipolar Geometry Toolbox : multiple view geometry and visual servoing for MATLAB

The Epipolar Geometry Toolbox (EGT) was realized to provide a MATLAB user with an extensible framework for the creation and visualization of multi-camera scenarios a nd the manipulation of the visual information and the geometry between them. Functions provided, for both pin-hole and panoramic vision sensors, include camera placement and visualizatio n, computation and estimation of epipolar geometry entities a nd many others. The compatibility of EGT with the Robotics Toolbox [7] allows to address general vision-based control issues. Two applications of EGT to visual servoing tasks are here provided. This article introduces the Toolbox in tutorial form. Examples are provided to show its capabilities. The complet toolbox, the detailed manual and demo examples are freely available on the EGT web site [21]. I. I NTRODUCTION The Epipolar Geometry Toolbox (EGT) is a toolbox designed for MATLAB [29]. MATLAB is a software environment, available for a wide range of platforms, designed around linear algebra principles and graphical presentati ons also for large datasets. Its core functionalities are exten ded by the use of many additional toolboxes. Combined with interactive MATLAB environment and advanced graphical functions, EGT provides a wide set of functions to approach computer vision problems especially with multiple views. The Epipolar Geometry Toolbox allows to design visionbased control systems for both pin-hole and central panoram ic cameras. EGT is fully compatible with the well known Robotics Toolbox by Corke [7]. The increasing interest in robotic visual servoing for both6DOF kinematic chains and mobile robots equipped with pin-hole or panoramic cameras fixed to the workspace or to the robot, motivated the development of EGT. Several authors, such as [4], [9], [18], [20], [24], have proposed new visual servoing strategies based on the geometry relating multiple views acquired from different camer a configurations, i.e. the Epipolar Geometry [14]. In these years we have observed the necessity to develop a software environment that could help researchers to rapid ly create a multiple camera setup, use visual data and design new visual servoing algorithms. With EGT we provide a wide set of easy-to-use and completely customizable functions t design general multi-camera scenarios and manipulate the visual information between them. Let us emphasize that EGT can also be successfully employed in many other contexts when single and multiple view geometry is involved as, for example, in visual odometry and structure from motion applications [23] [22]. For example i n the first work an interesting “visual odometry” approach for robot SLAM is proposed in which the multiple view geometry is used to estimate the camera motion from pairs of images without requiring the knowledge of the observed scene. EGT, as the Robotics Toolbox, is a simulation environment, but the EGT functions can be easily embedded by the user in Simulink models. In this way, thanks to the MATLAB RealTime Workshop, the user can generate and execute stand-alon e C code for many off-line and real-time applications. A distinguishable remark of EGT is that it can be used to create and manipulate visual data provided by both pinhole and panoramic cameras. Catadioptric cameras, due to their wide field of view, has been recently applied in visual servoing [32]. The second motivation lead to the development of EGT was the increasing distribution of “free” software in the lates t years, on the basis of the Free Software Foundation [10] principles . In this way users are allowed, and also encouraged, to adapt and improve the program as dictated by their needs. Examples of programs that follow these principles include for instan ce the Robotics Toolbox [7], for the creation of simulations in robotics, and the Intel’s OpenCVC++ libraries for the implementation of computer vision algorithms, such as imag e processing and object recognition [1]. The third important motivation for EGT was the availability and increasing sophistication of MATLAB. EGT could have been written in other languages, such as C, C++ and this would have freed it from dependency on other software. However these low-level languages are not so conducive to rapid program development as MATLAB. This tutorial assumes the reader has familiarity with MATLAB and presents the basic EGT functions, after short theory recalls, together with intuitive examples. In this tutoria l we also present two applications of EGT to visual servoing. Section 2 presents the basic vector notation in EGT, while in Section 3 the pin-hole and omnidirectional camera models ogether with EGT basic functions are presented. In Section 4 we present the setup for multiple camera geometry (Epipolar