Chapter 2 Empirical Characterization of Convergence Properties for Kernel-based Visual Servoing

Visual servoing typically involves separate feature tracking and control processes. Feature tracking remains an art, and is generally treated as independent of the underlying controller. Kernel-based visual servoing (KBVS) is a categorically different approach that eliminates explicit feature tracking. This chapter presents an experimental assessment of the convergence properties (domain of attraction and steady-state error) of the proposed approach. Using smooth weighting functions (the kernels) and Lyapunov theory, we analyze the controllers as they act on images acquired in controlled environments. We ascertain the domain of attraction by finding the largest positive invariant set of the Lyapunov function, inside which its time derivative is negative definite. Our experiments show that KBVS attains a maximum pixel error of one pixel and is commonly on the order of one tenth of a pixel. 2.1 Featureless Visual Servoing Typically, visual servoing involves tracking image features and controlling a robot based on the motions of these image features. Usually this involves tuning the feature tracking algorithm and controller independently, with no clear notion of how to co-optimize tracking and control. KBVS is distinguished from traditional visual servoing techniques in two respects, namely the lack of explicit feature tracking at each image frame and the inherent combination of tracking and control. These John P. Swensen and Noah J. Cowan Department of Mechanical Engineering, Johns Hopkins University, 3400 North Charles Street, Baltimore, MD 21218, USA e-mail: {jpswensen,ncowan}@jhu.edu Vinutha Kallem GRASP Laboratory, University of Pennsylvania, 3330 Walnut Street, Philadelphia, PA 19104, USA e-mail: vkallem@seas.upenn. edu G. Chesi & K. Hashimoto (Eds.): Visual Servoing via Advan. Numer. Methods, LNCIS 401, pp. 23–38. springerlink.com c © Springer-Verlag Berlin Heidelberg 2010 24 J.P. Swensen, V. Kallem, and N.J. Cowan characteristics may provide a benefit over traditional position-based and imagebased visual servoing [10], 2 1/2 D visual servoing [2], and other advanced visual servo algorithms [1]. “Featureless” visual servoing methods including KBVS take advantage of the rich set of visual data present in images without reducing the image to feature points. This reduces computation because extracting image features usually requires more computation than the image measurements used in featureless methods. Also, this confers robustness to region or feature occlusions in the images. Nayar et al. [15] present one of the earliest works in featureless visual servoing. The authors select patches of the goal image that form a high dimensional measurement. They then generate a training data set of images acquired at known robot poses. The leading principal components of the training images form a low dimensional approximation of the so-called appearance manifold. Images during control are projected onto this manifold and control is performed on this low dimensional space. The sampling of the camera workspace becomes prohibitively time consuming as the number of degrees of freedom increases and has to be evaluated for each scene. Deguchi [6] extends the method to 6 degrees of freedom and automates patch selection to improve the Jacobian. These papers do not address formal guarantees of convergence. More recently, Tahri and Chaumette [16] use moments for determining camera motions with respect to a goal. They compute the Jacobian relating changes in loworder image moments to the velocity of the moving camera. The image moments provide a similar measurement as the KBVS sampling kernel in that they provide a scalar measurement of the entire image for each type of moment calculation. Formally, with an appropriately designed family of kernels, it may be possible to conceive of moment-based visual servoing as a special case of KBVS. The advantages of KBVS are that selection of kernels with compact (finite) support will minimize edge effects, and moreover the KBVS approach embraces gray scale images. However, the image moment solution provides for 6 degree of freedom visual servoing, a problem not yet solved for KBVS. A related result by Collewet et al. [3] notes that for a static scene, the timevarying image is simply a transformed version of the original image, assuming no occlusions. They develop the Jacobian relating the motion of every pixel in the image to the motion of the camera, thus allowing derivatives of the image signal to be calculated directly. They then use the time derivative of the image to minimize a cost function, which in similar to the Lyapunov function as discussed later in this chapter, if the image is a continuous signal. This approach is analogous to placing a kernel at every pixel. KBVS presents a new kind of featureless visual servoing and this chapter presents the first in-depth empirical evaluation of KBVS. Section 2.2 presents the KBVS algorithm and its conceptual and theoretical underpinnings. Section 2.3 describes the experiments conducted to evaluate the method and characterize the convergence properties for several degrees of freedom. We conclude by discussing the outstanding issues that need to be addressed to make KBVS robust and to rationalize kernel selection. 2 Empirical Characterization of Convergence Properties for KBVS 25 2.2 Kernel-based Visual Servoing To aid the reader in understanding the basics of KBVS, we describe the details of the algorithm in a simple two dimensional case [12] and refer the reader to Kallem et al. [13] for the details of the SO(3) case, roll about the optical axis, translation along the optical axis, and combined motion derivations. The kernel-projected value in KBVS, derived from the kernel-based tracking literature [4, 9, 8, 7, 17], is a weighted measurement of an image based on the a sampling kernel. A Lyapunov function is formed from the vector of kernel-projected values and the KBVS control input ensures Lyapunov stability, as shown below. A key concept in the demonstration of stability is the equivalence of the kernelprojected value under a change of coordinates: ideally there is no difference between the kernel-projected value under a transformation of the image or the inverse transformation of the kernel. In the two dimensional case discussed below, this transformation is simply translation parallel to the image plane of the camera. Throughout the analytical derivation of KBVS, several assumptions are made about the scene and the signals that appear in the computation. First, we assume that the image plane is continuous (rather than discrete) and infinite, namely the image plane is a copy of R2. Second, we assume the scene to be planar. Finally, we require that pixels are constantly illuminated across all image frames. We assume the robot, as seen in Figure 2.1, is the kinematic plant ẋ = u (2.1) where the configuration of the robot is described as x = [x,y]T and u∈R2 drives the robot parallel to the image plane of the camera. Let the image be represented as a Fig. 2.1 The robot used for the experiments done in this chapter 26 J.P. Swensen, V. Kallem, and N.J. Cowan signal I(w,x(t)) ∈ R. The spatial indexing parameter of the 2D image is w ∈I = R 2. The signal is represented as a function of the spatial indexing and the position which is a function of time. Although one might expect to see the signal written only as a function of time, I(w,t), we will be more explicit and write it as a function of the robot configuration which is in turn a function of time, I(w,x(t)). For the sake of notational simplicity and without loss of generality we assume that the image plane is a unit distance away from the scene. Using the modified notation, the signal at any translated position is related to the signal at the goal I(w,x(t)) = I(w−x(t),0) = I0(w−x(t)). (2.2) In the two dimensional case, the kernel-projected value is a function ξ : I → Rn where ξ = ∫