Image Feature Extraction Techniques and Their Applications for CBIR and Biometrics Systems

In CBIR (Content-Based Image Retrieval), visual features such as shape, color and texture are extracted to characterize images. Each of the features is represented using one or more feature descriptors. During the retrieval, features and descriptors of the query are compared to those of the images in the database in order to rank each indexed image according to its distance to the query. In biometrics systems images used as patterns (e.g. fingerprint, iris, hand etc.) are also represente d by feature vectors. The candidates patterns are then retrieved f rom database by comparing the distance of their feature vectors. Th e feature extraction methods for this applications are discussed. CBIR, Biometrics, Feature extraction I. I NTRODUCTION In various computer vision applications widely used is the process of retrieving desired images from a large collectio n on the basis of features that can be automatically extracted fr om the images themselves. These systems called CBIR (ContentBased Image Retrieval) have received intensive attention i n the literature of image information retrieval since this ar ea was started years ago, and consequently a broad range of techniques has been proposed. The algorithms used in these systems are commonly divided into three tasks: extraction, selection, and classification. The extraction task transforms rich content of images into various content features. Feature extraction is the proces s of generating features to be used in the selection and classific ation tasks. Feature selection reduces the number of feature s provided to the classification task. Those features which ar e likely to assist in discrimination are selected and used in the classification task. Features which are not selected are discarded . Of these three activities, feature extraction is most criti cal because the particular features made available for discrim ination directly influence the efficacy of the classification tas k. The end result of the extraction task is a set of features, commonly called a feature vector, which constitutes a representation of the image. In the last few years, a number of above mentioned systems using image content feature extraction technologies prove d reliable enough for professional applications in industri al Manuscript received March 10, 2007; Revised June 2, 2007 Ryszard S. Chorá s is with the University of Technology & Life Sciences, Institute of Telecommunications, Image Processing Group, 85796 Bydgoszcz, S. Kaliskiego 7, Poland,  Diagram of the image retrieval process. automation, biomedicine, social security, biometric auth en ication and crime prevention . Fig. 1 shows the architecture of a typical CBIR system. For each image in the image database, its features are extracted and the obtained feature space (or vector) is stored in the featu r database. When a query image comes in, its feature space will be compared with those in the feature database one by one and the similar images with the smallest feature distance will b e retrieved. CBIR can be divided in the following stages: •  The image is first processed in order to extract the features, which describe its contents. The processing involves filtering, normalization, segmentation, and object identification. The output of this stage is a set of significant regions and objects. •  Features such as shape, texture, color, etc. are used to describe the content of the image. Image features can be classified into primitives. CBIR combines high-tech elements such as: multimedia, signal and image processing, pattern recognition, human-computer interaction, human perception information sciences. In Pattern Recognition we extract ”relevant” information about an object via experiments and use these measurements (= features) to classify an object. CBIR and direct object recognition, although similar in principle, using many of the same image analysis/statistical tools, are very differ ent operations. Object recognition works with an existing data b se of objects and is primarily a statistical matching problem. One can argue that object recognition is a particularly nice ly defined sub-set of CBIR.  2007 6 In CBIR, human factors play the fundamental role. Another distinction between recognition and retrieval is evident i less specialized domains these applications are inherent ly concerned with ranking (i.e., re-ordering database images according to their measured similarity to a query example) rather than classification (i.e., deciding process whether or not an observed object matches a model), as the result of similarity-based retrieval. First generation CBIR systems were based on manual textual annotation to represent image content. This techni que can only be applied to small data volumes and, to be truly effective, annotation must be limited to very narrow visual domains. In content-based image retrieval, images are automaticall y indexed by generating a feature vector (stored as an index in feature databases) describing the content of the image. The similarity of the feature vectors of the query and database images is measured to retrieve the image. Let {F (x, y);x = 1, 2, . . . , X, y = 1, 2, . . . , Y } be a two-dimensional image pixel array. For color images F (x, y) denotes the color value at pixel (x, y) i.e., F (x, y) = {FR(x, y), FG(x, y), FB(x, y)}. For black and white images, F (x, y) denotes the grayscale intensity value of pixel (x, y). The problem of retrieval is following: For a query image Q , we find imageT from the image database, such that distance between corresponding feature vectors is less than specifie d threshold, i.e., D(Feature(Q), F eature(T ) ≤ t (1) II.  The feature is defined as a function of one or more measurements, each of which specifies some quantifiable propert y of an object, and is computed such that it quantifies some significant characteristics of the object. We classify the various features currently employed as follows: •  Application independent features such as color, texture, and shape. According to the abstraction level, they can be further divided into: Pixel-level features: Features calculated at each pixel, e.g. color, location.  Features calculated over the results of subdivision of the image band on image segmentation or edge detection.  Features calculated over the entire image or just regular sub-area of an image. • Domain-specific features: Application dependent features such as human faces, fingerprints, and conceptual features. These features are often a synthesis of low-level features for a specific domain. On the other hand, all features can be coarsely classified into low-level features and high-level features. Low-leve l features can be extracted directed from the original images , whereas high-level feature extraction must be based on lowlevel features . A. Color The color feature is one of the most widely used visual features in image retrieval. Images characterized by color features have many advantages: • Robustness. The color histogram is invariant to rotation of the image on the view axis, and changes in small steps when rotated otherwise or scaled . It is also insensitive to changes in image and histogram resolution and occlusion. • Effectiveness. There is high percentage of relevance between the query image and the extracted matching images. • Implementation simplicity. The construction of the color histogram is a straightforward process, including scanning the image, assigning color values to the resolution of the histogram, and building the histogram using color components as indices. • Computational simplicity. The histogram computation has O(X,Y ) complexity for images of sizeX × Y . The complexity for a single image match is linear, O(n), where n represents the number of different colors, or resolution of the histogram. • Low storage requirements. The color histogram size is significantly smaller than the image itself, assuming color quantisation. Typically, the color of an image is represented through some color model. There exist various color model to describ e color information. A color model is specified in terms of 3-D coordinate system and a subspace within that system where each color is represented by a single point. The more commonly used color models are RGB (red, green, blue), HSV (hue, saturation, value) and Y,Cb, Cr (luminance and chrominance). Thus the color content is characterized by 3-channels from some color model. One representation of color content of the image is by using color histogram. Statistically, it denotes the joint probability of the inte sities of the three color channels. Color is perceived by humans as a combination of three color stimuli: Red, Green, Blue, which forms a color space . This model has both a physiological foundation and a hardware related one. RGBcolors are called primary colors and are additive. By varying their combinations, other colo rs can be obtained. The representation of the HSV space  is derived from theRGB space cube, with the main diagonal of the RGBmodel, as the vertical axis in HSV. As saturation varies form 0.0 to 1.0, the colors vary from unsaturated (gra y) to saturated (no white component). Hue ranges from 0 to 360 degrees, with variation beginning with red, going throu gh yellow, green, cyan, blue and magenta and back to red. These color spaces are intuitively corresponding to the RGB model from which they can be derived through linear or non-linear transformations.  TheRGBcolor space and the HSV color space Fig. 3. Original image