Gradient-Based Learning for Object Detection , Segmentation andRecognitionYann

Finding an appropriate set of features is an essential problem in the design of visual recognition systems. This paper attempts to show that for recognizing simple deformable objects with high shape variability such as handwritten characters, it is possible, and even advantageous, to feed the system directly with minimally processed images and to rely on learning to extract the right set of features. Convolutional Neural Networks are shown to be particularly well suited to this task. We also show that these networks can be used to recognize multiple objects without requiring explicit segmentation of the objects from their surrounding. The second part of the paper presents the Graph Transformer Network model which extends the applicability of gradient-based learning to systems that use graphs to represents features, objects, and their combinations. 1 Gradient-Based Learning in Complex Systems Until recently, the applicability of machine learning techniques to vision was been limited for three main reasons. First, vision tasks involve multiple steps such as detection, segmentation, feature extraction, and recognition, that require di erent strategies, not all of which are easily learnable. Second, the recognition of complex objects often involve combinatorial searches that are not easily learned. Third, objects are often represented by structured objects (such as valued graphs) or high-dimensional vectors that most traditional learning algorithms have a hard time dealing with. Learning algorithms are generally restricted to a particular type of architecture. This often limits their applicability, since di erent perceptual tasks often require di erent recognition architectures. When learning techniques must be combined with prior knowledge to construct a recognition system, the prior knowledge is generally used to design an appropriate preprocessor or feature extractor, whose output is fed to a \generic" trainable recognizer. A more satisfying method would be to rst construct an architecture that is appropriate to the task from a set of predetermined building blocks, and then to train this entire architecture by simultaneously optimizing the parameters of all the building blocks so as to maximize an overall performance measure. More precisely, the most commonly accepted model of pattern recognition, is composed of a segmenter whose role is to extract objects of interest from their background, a hand-crafted feature extractor that gathers relevant information from the input and eliminates irrelevant variabilities, and a classi er which categorizes the resulting feature representations (generally vectors or strings of symbols) into categories. There are three major methods for classi cation: template matching ((Yuille et al. 1992)) matches the feature representation to a set of class templates; generative methods use a probability density model for each class, and pick the class with the highest likelihood of generating the feature representation; discriminative models compute a discriminant function that directly produces a score for each class. Generative ((Moghaddam and Pentland 1995)) and discriminative models ((Sung and Poggio 1994); (Papageorgiou et al. 1998)) are often estimated (learned) from training samples. In all of these approaches, the overall performance of the system is largely determined by the quality of the feature extractor and the segmentation technique. The feature extractor usually relies on simplifying assumptions about the input data and can rarely take into account all the variability of the real world. The segmentation technique is either exhaustive, exploring a range of possible scale and positions, or hand-crafted, again using simplifying assumptions to restrict the number of candidate segments. Because simplifying assumptions are likely sources of unrecoverable losses of information, it may be preferable to feed the entire system with minimally processed inputs (e.g. \raw" pixel images), and train it from data so as to minimize an overall loss function (which maximizes a given performance measure). Keeping the preprocessing to a minimum ensures that no unrealistic assumption is made about the data. Unfortunately, that also requires to come up with a suitable learning architecture that can handle the high dimension of the input (number 3 of pixels), the high degree of variability due to pose variations or geometric distortions among other things, and the necessarily complex non-linear relation between the input and the output. What is proposed here is a method for learning the features and the classi cation function simultaneously in a supervised way. Many authors, most recently Zhu and Mumford ((Zhu and Mumford)), have proposed unsupervised methods for learning appropriate features for natural images. These methods have the advantage of producing generic features, but the disadvantage of not being able to tune the features to the particular task at hand. Gradient-Based Learning provides a framework in which to build such a system. The learning machine computes a function Y p = F (Zp;W ) where Zp is the p-th input pattern, and W represents the collection of adjustable parameters in the system. The output Y p contains scores or probabilities for each category. A loss function Ep = D(Dp; F (W;Zp)), measures the discrepancy between Dp, the \correct" output for pattern Zp, and the output produced by the system. The average loss function Etrain(W ) is the average of the errors Ep over a set of labeled examples called the training set f(Z1;D1); ::::(ZP ;DP )g. In the simplest setting, the learning problem consists in nding the value of W that minimizes Etrain(W ). Making the loss function di erentiable with respect to W ensures that e cient, gradient-based non-linear optimization methods can be used to nd a minimum. To ensure global di erentiability, the system is built as a feed-forward network of modules. In the simplest case, each module computes a function Xn = Fn(Wn;Xn 1), where Xn is an object (a vector in the simplest case) representing the output of the module, Wn is the vector of tunable (trainable) parameters in the module (a subset of W ), and Xn 1 is the module's input (as well as the previous module's output). The input X0 to the rst module is the system's input pattern Zp. The main idea of Gradient-Based Learning, which is a simple extension of the well-known backpropagation neural network learning algorithm, is that the objective function can be e ciently minimized through gradient descent (or other more sophisticated non-linear optimization methods) because the gradient of E with respect to W can be e ciently computed with a backward recurrence through the network of modules. If the partial derivative of Ep with respect to Xn is known, then the partial derivatives of Ep with respect to Wn and Xn 1 can be computed using the following backward recurrence: @Ep @Wn = @Fn @W (Wn;Xn 1) @Ep @Xn @Ep @Xn 1 = @Fn @X (Wn;Xn 1) @Ep @Xn (1) where @Fn @W (Wn;Xn 1) is the Jacobian of Fn with respect to W evaluated at the point (Wn;Xn 1), and @Fn @X (Wn;Xn 1) is the Jacobian of Fn with respect to X. The rst equation computes some terms of the gradient of Ep(W ), while the second equation propagates the partial gradients backward. The idea can be trivially extended to any network of functional modules. A completely rigorous derivation of the gradient propagation procedure in the general case can be done using Lagrange functions ((LeCun 1988); (Bottou and Gallinari 1991)). As we show in the following, a large variety of modules have been devised that allow Gradient-Based Learning to be applied to invariant object recognition, multiple object recognition, and recognition of objects that are best represented as graphs. 2 Shape Recognition with Convolutional Neural Networks Traditional multi-layer neural networks are a special case of the above where the states Xn are xedsized vectors, and where the modules are alternated layers of matrix multiplications (the weights) 4