Chapter 2 Feature Learning for Facial Kinship Verification

In this chapter, we discuss feature learning techniques for facial kinship verification.Wefirst review twowell-knownhand-crafted facial descriptors including local binary patterns (LBP) and the Gabor feature. Then, we introduce a compact binary face descriptor (CBFD) method which learns face descriptors directly from raw pixels. Unlike LBP which samples small-size neighboring pixels and computes binary codes with a fixed coding strategy, CBFD samples large-size neighboring pixels and learn a feature filter to obtain binary codes automatically. Subsequently, we present a prototype-based discriminative feature learning(PDFL) method to learn mid-level discriminative features with low-level descriptor for kinship verification. Unlikemost existingprototype-based feature learningmethodswhich learn themodel with a strongly labeled training set, this approach works on a large unsupervised generic set combined with a small labeled training set. To better use multiple lowlevel features for mid-level feature learning, a multiview PDFL (MPDFL) method is further proposed to learn multiple mid-level features to improve the verification performance. 2.1 Conventional Face Descriptors 2.1. 1 Local Binary Patterns Local binary pattern (LBP) is a popular texture descriptor for feature representation. Inspired by its success in texture classification, Ahonen et al. [1] proposed a novel LBP feature representation method for face recognition. The basic idea is as follows: for each pixel, its 8-neighborhood pixels are thresholded into 1 or 0 by comparing themwith the center pixel. Then the binary sequence of the 8-neighborhoods is transferred into a decimal number (bit pattern states with upper left corner moving clockwise around center pixel), and the histogram with 256 bins of the processed image is used as the texture descriptor. To capture the dominant features, Ahonen et al. [1] extended LBP to a parametric form (P,R), which indicates that P gray-scale values are equally distributed in a circle with radiusR to form circularly symmetric neighbor sets. To better characterize the property of texture information, uniform pattern is © The Author(s) 2017 H. Yan and J. Lu, Facial Kinship Verification, SpringerBriefs in Computer Science, DOI 10.1007/978-981-10-4484-7_2 7 8 2 Feature Learning for Facial Kinship Verification Fig. 2.1 The LBP and ILBP operators where a Original image patch b Results of LBP operator c Results of ILBP operator defined according to the number of spatial transitions that are bitwise 0/1 changes in the calculated binary values. If there are at most two bitwise transitions from 0 to 1 or vice versa, the binary value is called a uniform pattern. Recently, Jin et al. [22] developed an improved LBP (ILBP) method. Different fromLBP, ILBPmakes better use of the central pixel in the original LBP and takes the mean of all gray values of elements as the threshold value. For the newly generated binary value of the central pixel, it was added to the most left position of the original binary string. The corresponding decimal value range would be changed from [0, 255] to [0, 510] for a 3×3 operator. Figure2.1 shows the basic ideas of LBP and ILBP. Since LBP and ILBP are robust to illumination changes, they have been widely used in many semantic understanding tasks such as face recognition and facial expression recognition. 2.1. 2 Gabor Feature Representation Gabor wavelet is a popular feature extractionmethod for visual signal representation, and discriminative information is extracted by convoluting the original image with a set of Gabor kernels with different scales and orientations. A 2-D Gabor wavelet kernel is the product of an elliptical Gaussian envelope and a complex plane wave, defined as [31]: