Omnidirectional vision algorithms in robotics

Omnidirectional vision is becoming more and more popular in those robotics applications in which an autonomous robot is required to react to visual stimuli that may come from any direction at any moment of its activity, or to plan its behaviour according to them. This paper discusses some interesting opportunities provided by omnidirectional vision sensors in robotics, introduces the Inverse Perspective Transform and its implementation for use with omnidirectional catadioptric sensors, and finally describes two algorithms based on such a transform and omnidirectional sensors. The algorithms have been devised within a general framework in which, with some limitations, most existing algorithm designed for traditional cameras can be adapted for use with omnidirectional sensors. 1 Omnidirectional vision sensors The need for robotic sensory systems that provide a global description of the surrounding environment is increasing. This has stimulated growing interest in omnidirectional vision systems (see [1] for a recent collection of articles on the subject). Effective self-localization strategies can be based on that kind of sensors. For example, in [2] a system is presented which is composed of a scanning laser with a field of view to measure the azimuth angle to wall-mounted reference reflectors. Notice that angular information is sufficient to solve triangulation problems; usually angular information is much easier to be retrieved than the euclidean distance, both with standard vision systems and, even more so, with omnidirectional sensors. Omnidirectional vision systems are often based on a catadioptric sensor, consisting of an upwards-oriented camera that acquires the image reflected on a convex mirror hanging above it. A number of such sensors are available and, consequently, a large number of approaches to omnidirectional vision are described in literature [1]. Besides providing the widest possible field of view, omnidirectional vision systems can obviate the need for active cameras, that require complex control strategies. However, they suffer from relevant limitations, such as having the near field partially obstructed by the reflection of the camera on the mirror or accumulating camera and system distortions such that a precise compensation law is often impossible to find. This problem can often be overcome with empirical calibration procedures, or by accurate design of mirrors that produce a nondistorted image of a reference plane [4, 5]. However, the latter solution imposes constraints on the design of the mirror that often conflict with other requirements, such as the extension of the field of view or the local distortion or resolution in the relevant part of the image. Recent robotics and surveillance applications in which omnidirectional sensors have been used effectively, as the only vision sensor or jointly with other higher-resolution non omnidirectional ones, are described in [6, 7, 8]. In this paper we propose a framework within which algorithms for omnidirectional sensors can be easily developed, with almost no extra overhead with respect to traditional camerabased algorithms. This is the case (that includes many real-world applications) when it is possible to find regions of interest where the image acquired through the omnidirectional sensor can be remapped on a new undistorted image. The calibration process needed to do so can be easily automated. In the following sections we first describe a calibration algorithm for catadioptric omnidirectional sensors, discussing some points related to its implementation. Then, we describe two algorithms for omnidirectional sensors devised within the framework described previously.