Hierarchical, generic to specific multi-class object recognition
Authors: Florin Cutu
Venue:  (2006)

Human beings can perform complex (2D) object recognition tasks easily and over different levels of generalization. A dog can not only be thought of as a dog, but also a boxer, a quadruped, or in general an animate being. Human beings can seamlessly transition from the highly general term quadruped to the more generic term dog and with some further knowledge also disambiguate between different species of dogs. Early vision systems could perform specific object recognition reasonably well but did not fare as well on identifying the natural class of an object. Recent advances have led to systems that can learn a representation for different object classes and achieve good generic object class recognition. However, these systems are generally unable to perform the fine distinctions required for specific object identification. It seems that the two approaches are in contrast with each other. While looking at multiple levels of object categorization rather than individual classes, another desired property of Computer Vision systems is that objects from the same generic category, such as quadrupeds, are classified as more similar to each other than objects from other generic categories. Further, if it is not possible to correctly identify the specific object (say a dog), we would like that it be at least assigned a correct generic label (i.e., quadruped), thus providing graceful degradation. Most existing 2D object recognition systems require that an input object be compared to models of all of the existing examples/classes. In particular if a recognition system has been modeled for n classes, the number of models that have to be evaluated at every region of the image is of the order of O(n). This makes it difficult for systems to include models for a large number of objects. We would ideally like that as models of different objects are added to the system, the search complexity increases at a slower rate. Previous recognition systems try to include all views of the object over the viewing sphere into a single model. While it seems clear that systems need to be trained with all views of the objects, the front view of some objects is not similar to their side views and vice versa. For example the front view of a horse is quite different from the side view. It might seem beneficial to create different models for similar views of an object, thus leading to multiple models per object. This however leads to a much larger search as comparisons have to be performed with multiple models for each trained object. This dissertation outlines the development of a system that is able to accommodate multiple categories of labels, from the generic to the specific. The idea is to learn a hierarchical system of object categorization and recognition that assigns labels to an input image region, starting from the most generic labels at the top of the tree to the most specific labels at the leaves of the tree. In fact, some of the nodes at the top levels of the tree need not carry a label at all, they may be just formed by the grouping of visually similar objects. The creation of this tree structure in the space of images may be viewed as a recursive partitioning of the image space. The tree structure thus formed is very similar to classical decision trees. However unlike in a decision tree node where a single informative attribute of the feature vector is used for partitioning the space and performing the decision of whether to move to the left child or the right child (for a binary tree), in our formulation the entire set of features is used for performing the decision. The idea is to perform a nearest neighbor comparison at each node of the tree, where each node represents a hyper-ellipsoid cluster in the feature space. Each node in our treestructure is represented by a center and an elliptical metric that defines the varying attraction of that center along different directions of the image space (i.e., for different pixels). To estimate the centers and the elliptical metrics an algorithm called Elliptical Nearest Neighbors (ENN), has been developed. This algorithm was originally conceived while trying to infer the internal templates used by humans in a simple classification experiment. The ENN algorithm generates prototypes that are all at the same level, there is no hierarchical structure to the prototypes. The ENN algorithm is a nearest neighbor prototype reduction algorithm and it belongs to the class of "prototype creation" algorithms. We provide results of the use of the ENN algorithm and compare the ENN algorithm to other prototype reduction algorithms. To impose a hierarchical structure on the prototypes and to enable generic to specific classification, we introduce the Hierarchical Nearest Neighbors (HNN) algorithm. The HNN algorithm is based on a recursive use of the ENN algorithm, yielding a decision tree structure. We experimentally illustrate the working of the HNN algorithm and compare the results with other decision tree algorithms. Finally the HNN algorithm is applied to the problem of generic to specific object recognition and results on real images are reported.