nils-herrmann/clean_scirepeval huggingface dataset:
A preliminary investigation on the sensitivity of COTS face recognition systems to forensic analyst-style face processing for occlusions

Modern day law enforcement banks heavily on the use of commercial off-the-shelf (COTS) face recognition systems (FRS) as a tool for biometric evaluation and identification. However, in many real-world scenarios, when the face of an individual is occluded or degraded in some way, commercial recognition systems fail to accept the face for evaluation or simply return unusable matched faces. In these kinds of cases, forensic experts rely on image processing techniques and tools, to make the face fit to be processed by the commercial recognition systems (e.g. use partial face images from another subject to fill in the occluded parts of the face of interest, or have a tight crop around the face). In this study, we evaluate the sensitivity of commercial recognition systems to such forensic techniques. More specifically, we study the change in the rank-1 identification result that is caused by forensic processing of faces-of-interest that are unusable by the commercial recognition systems. Further, forensic processing of such faces is more of an art and it is extremely difficult to process faces consistently such that there is a predictable effect on the rank-n identification result. This study is meant to serve as an evaluation of the effect of a few forensic techniques intended to allow commercial recognition systems to process and match face images that were otherwise unusable. Our results indicate that COTS FRS can be sensitive to the subjectivity in facial part swapping and cropping, resulting in inconsistencies in the identification rankings and similarity scores.
Face identification using novel frequency-domain representation of facial asymmetry

Face recognition is a challenging task. This paper introduces a novel set of biometrics, defined in the frequency domain and representing a form of "facial asymmetry." A comparison with existing spatial asymmetry measures suggests that the frequency-domain representation provides an efficient approach for performing human identification in the presence of severe expressions and for expression classification. Error rates of less than 5% are observed for human identification and around 25% for expression classification on a database of 55 individuals. Feature analysis indicates that asymmetry of the different face parts helps in these two apparently conflicting classification problems. An interesting connection between asymmetry and the Fourier domain phase spectra is then established. Finally, a compact one-bit frequency-domain representation of asymmetry is introduced, and a simplistic Hamming distance classifier is shown to be more efficient than traditional classifiers from storage and the computation point of view, while producing equivalent human identification results. In addition, the application of these compact measures to verification and a statistical analysis are presented
Contourlet appearance model for facial age estimation

In this paper we propose a novel Contourlet Appearance Model (CAM) that is more accurate and faster at localizing facial landmarks than Active Appearance Models (AAMs). Our CAM also has the ability to not only extract holistic texture information, as AAMs do, but can also extract local texture information using the Nonsubsampled Contourlet Transform (NSCT). We demonstrate the efficiency of our method by applying it to the problem of facial age estimation. Compared to previously published age estimation techniques, our approach yields more accurate results when tested on various face aging databases.
Partial & Holistic Face Recognition on FRGC-II data using Support Vector Machine

In this paper we investigate how to perform face recognition on the hardest experiment (Exp4) in Face Recognition Grand Challenge(FRGC) phase-II data which deals with subjects captured under uncontrolled conditions such as harsh overhead illumination, some pose variations and facial expressions in both indoor and outdoor environments. Other variations include the presence and absence of eye-glasses. The database consists of a generic dataset of 12,776 images for training a generic face subspace, a target set of 16,028 images and a query set of 8,014 images are given for matching. We propose to use our novel face recognition algorithm using Kernel Correlation Feature Analysis for dimensionality reduction (222 features) coupled with Support Vector Machine discriminative training in the Target KCFA feature set for providing a similarity distance measure of the probe to each target subject. We show that this algorithm configuration yields the best verification rate at 0.1% FAR (87.5%) compared to PCA+SVM, GSLDA+SVM, SVM+SVM, KDA+SVM. Thus we explore with our proposed algorithm which facial regions provide the best discrimination ability, we analyze performing partial face recognition using the eye-region, nose region and mouth region. We empirically find that the eye-region is the most discriminative feature of the faces in FRGC data and yields a verification rate closest to the holistic face recognition of 83.5% @ 0.1% FAR compared to 87.5%. We use Support Vector Machines for fusing these two to boost the performance to ~90@0.1 % FAR on the first large-scale face database such as the FRGC dataset.
Incremental updating of advanced correlation filters for biometric authentication systems

In this paper we show mathematical formulation of incrementally building advanced correlation filters used in authentication systems that are based on face and fingerprint images as biometrics for verification. This method is crucial for incorporating such algorithms on small devices with limited memory and computational resources. We also present results that show that these correlation filters perform well for face and fingerprint images. We used the PIE (pose, illumination and expression) database from CMY to test the verification performance using face images. Similarly, for fingerprint images we used the NIST special database 24 to evaluate verification performance.
Distributed class dependent feature analysis — A big data approach

Big data has been becoming ubiquitous and applied in numerous fields recently. The challenges to solve a large-scale machine learning problem in big data scenario generally lie in three aspects. Firstly, a proposed machine learning algorithm has to be appropriated for the distributed optimization problem. Secondly, it needs a platform for the distributed implementation. Finally, the communication delays different machines may cause problems in convergence even though the non-distributed algorithm shows a good convergence rate. In order to solve these challenges, we propose a new machine learning approach named Distributed Class-dependent Feature Analysis (DCFA), to combine the advantages of sparse representation in an over-complete dictionary. The classifier is based on the estimation of class-specific optimal filters, by solving an l1-norm optimization problem. We demonstrate how this problem is solved using the Alternating Direction Method of Multipliers and also explore relevant convergency details. More importantly, our proposed framework can be efficiently implemented on a robust distributed framework. Thus, it improves both accuracy and computational time in large-scale databases. Our method achieves very high classification accuracies in face recognition in the presence of occlusions on AR database. It also outperforms the state of the art methods in object recognition on two challenging large-scale object databases, i.e. Caltech101 and Caltech256. It hence shows its applicability to general computer vision and pattern recognition problems. In addition, computational time experiments show our distributed method achieves high speedup of 7.85x on Caltech256 databases with just 10 machine nodes compared to the non-distributed version and can gain even more with more computing resources.
Gait-ID on the move: Pace independent human identification using cell phone accelerometer dynamics

In this paper, we have proposed a robust, acceleration based, pace independent gait recognition framework using Android smartphones. From our extensive experiments using cyclostationarity and continuous wavelet transform spectrogram analysis on our gait acceleration database with both normal and fast paced data, our proposed algorithm has outperformed the state-of-the-art by a great margin. To be more specific, for normal to normal pace matching, we are able to achieve 99.4% verification rate (VR) at 0.1% false accept rate (FAR); for fast vs. fast, we are able to achieve 96.8% VR at 0.1% FAR; for the challenging normal vs. fast, we are still able to achieve 61.1% VR at 0.1% FAR. The findings have laid the foundation of pace independent gait recognition using mobile devices with high accuracy.
Spatial frequency domain image processing for biometric recognition

Biometric recognition refers to the process of matching an input biometric to stored biometric information. In particular, biometric verification refers to matching the live biometric input from an individual to the stored biometric template about that individual. Examples of biometrics include face images, fingerprint images, iris images, retinal scans, etc. Thus, image processing techniques prove useful in the biometric recognition. We discuss spatial frequency domain image processing methods useful for biometric recognition.
Encoding and decoding local binary patterns for harsh face illumination normalization

In this work, we propose a new illumination normalization technique based on a simple, yet widely used descriptor: local binary patterns (LBP). We capitalize on the fact that LBP retains tolerance to illumination changes and use the LBP mapping to remove illumination variations cast on face images. Through learning a reverse mapping from the LBP domain to the pixel domain, we are able to recover the illumination normalized face with high fidelity. The reverse mapping step is made possible via a joint dictionary learning framework between the LBP domain and the pixel domain. The illumination normalized faces using our proposed LBP encoding and decoding method not only exhibit very high fidelity against neutrally illuminated face, but also allow for a significant improvement in face verification experiments using even the simplest nearest-neighbor classifier. These conclusions are drawn after benchmarking our algorithm against 22 prevailing illumination normalization techniques on Extended YaleB database which has been widely adopted for challenging face illumination problems.
Comparison of Feature Space Methods for Face Recognition

Many feature space methods have been investigated for appearance-based face recognition. In this paper we compare a new feature space face recognition method - the class-dependence feature analysis (CFA) with three other popular methods, namely, the principal component analysis (PCA), the linear discriminant analysis (LDA) and the independent component analysis (ICA), for appearance-based 2-D face recognition. The numerical results on the face recognition grand challenge (FRGC) show that the CFA outperforms the other three method
Driver cell phone usage detection on Strategic Highway Research Program (SHRP2) face view videos

The harmful effects of cell phone usage on driver behavior have been well investigated and the growing problem has motivated several several research efforts aimed at developing automated cell phone usage detection systems. Computer vision based approaches for dealing with this problem have only emerged in recent years. In this paper, we present a vision based method to automatically determine if a driver is holding a cell phone close to one of his/her ears (thus keeping only one hand on the steering wheel) and quantitatively demonstrate the method's efficacy on challenging Strategic Highway Research Program (SHRP2) face view videos from the head pose validation data that was acquired to monitor driver head pose variation under naturalistic driving conditions. To the best of our knowledge, this is the first such evaluation carried out using this relatively new data. Our approach utilizes the Supervised Descent Method (SDM) based facial landmark tracking algorithm to track the locations of facial landmarks in order to extract a crop of the region of interest. Following this, features are extracted from the crop and are classified using previously trained classifiers in order to determine if a driver is holding a cell phone. We adopt a through approach and benchmark the performance obtained using raw pixels and Histogram of Oriented Gradients (HOG) features in combination with various classifiers.
Quaternion correlation filters for face recognition in wavelet domain

A new frequency domain face recognition method using wavelet decomposition and quaternion correlation filters is proposed. The wavelet decomposition of the face image leads to a wavelet subband representation, which contains four subband images corresponding to four orthogonal channels. These four subbands can be encoded into a 2D quaternion number array. The quaternion correlation filter method is developed to perform pattern recognition jointly on multichannel 2D signals. The proposed method has been shown to achieve significant improvement in face recognition results compared to the traditional advanced correlation filter method for handling illumination variations of face images. We experimented with the CMU PIE database consisting of 65 people with 21 illumination variations per person, showing that our method can achieve close to 100% recognition accuracy using just a single training image of a person under neutral frontal lighting and testing on all other unseen harsh illumination conditions.
Biometric data hiding: A 3 factor authentication approach to verify identity with a single image using steganography, encryption and matching

Digital Steganography exploits the use of host data to hide a piece of information in such a way that it is imperceptible to a human observer. Its main objectives are imperceptibility, robustness and high payload. DCT Domain message embedding in Spread Spectrum Steganography describes a novel method of using redundancy in DCT coefficients. We improved upon the method of DCT embedding by using the sign of the DCT coefficients to get better accuracy of retrieved data and more robustness under channel attacks like channel noise and JPEG compression artifacts while maintaining the visual imperceptibility of cover image, and even extending the method further to obtain higher payloads. We also apply this method for secure biometric data hiding, transmission and recovery. We hide iris code templates and fingerprints in the host image which can be any arbitrary image, such as face biometric modality and transmit the so formed imperceptible Stego-Image securely and robustly for authentication, and yet obtain perfect reconstruction and classification of iris codes and retrieval of fingerprints at the receiving end without any knowledge of the cover image i.e. a blind method of steganography, which in this case is used to hide biometric template in another biometric modality.
Cancelable biometric filters for face recognition

In this paper, we address the issue of producing cancelable biometric templates; a necessary feature in the deployment of any biometric authentication system. We propose a novel scheme that encrypts the training images used to synthesize the single minimum average correlation energy filter for biometric authentication. We show theoretically that convolving the training images with any random convolution kernel prior to building the biometric filter does not change the resulting correlation output peak-to-sidelobe ratios, thus preserving the authentication performance. However, different templates can be obtained from the same biometric by varying the convolution kernels thus enabling the cancelability of the templates. We evaluate the proposed method using the illumination subset of the CMU pose, illumination, and expressions (PIE) face dataset. Our proposed method is very interesting from a pattern recognition theory point of view, as we are able to 'encrypt' the data and perform recognition in the encrypted domain that performs as well as the unencrypted case, regardless of the encryption kernel used; we show analytically that the recognition performance remains invariant to the proposed encryption scheme, while retaining the desired shift-invariance property of correlation filters.
How Low Can You Go? Low Resolution Face Recognition Study Using Kernel Correlation Feature Analysis on the FRGCv2 dataset

In this paper we investigate the effect of image resolution of the face recognition grand challenge (FRGC) dataset on the kernel class-dependence feature analysis (KCFA) method. Good performance on low-resolution image data is important for any face recognition system using low- resolution imagery, such as in surveillance footage. We show that KCFA works reliably even at very low resolutions on the FRGC dataset Experiment 4 using the one-to-one matching protocol (greater than 70% verification rate (VR) at 0.1% false accept rate (FAR)). We observe reasonable performance at resolution as low as 16x16. However performance of KCFA degrades significantly below this resolution, but still outperforms the PCA baseline algorithm with 12% VR at 0.1% FAR.
Single face image super-resolution via solo dictionary learning

In this work, we have proposed a single face image super-resolution approach based on solo dictionary learning. The core idea of the proposed method is to recast the super-resolution task as a missing pixel problem, where the low-resolution image is considered as its high-resolution counterpart with many pixels missing in a structured manner. A single dictionary is therefore sufficient for recovering the super-resolved image by filling the missing pixels. In order to fill in 93.75% of the missing pixels when super-resolving a 16 × 16 low-resolution image to a 64 × 64 one, we adopt a whole image-based solo dictionary learning scheme. The proposed procedure can be easily extended to low-resolution input images with arbitrary dimensions, as well as high-resolution recovery images of arbitrary dimensions. Also, for a fixed desired super-resolution dimension, there is no need to retrain the dictionary when the input low-resolution image has arbitrary zooming factors. Based on a large-scale fidelity experiment on the FRGC ver2 database, our proposed method has outperformed other well established interpolation methods as well as the coupled dictionary learning approach.
An exploration of gender identification using only the periocular region

The periocular region, the region of the face surrounding the eyes, has gained increasing attention in biometrics in recent years. This region of the face is of particular interest when trying to identify a person whose face is partially occluded. We propose the novel idea of applying the information obtained from the periocular region to identify the gender of a person, which is a type of soft biométrie recognition. We gradually narrow the region of interest of the face to explore the feasibility of using smaller, eye-centered regions for building a robust gender classifier around the periocular region alone. Our experimental results show that at least an 85% classification rate is still obtainable using only the periocular region with a database of 936 low resolution images collected from the web.
Efficient design of advanced correlation filters for robust distortion-tolerant face recognition

The paper summarizes new research in performing face recognition using advanced correlation filters. We examine the performance of such filters in the area of biometrics for face authentication. We also compare results when the filters are applied to face identification. Our results are based on the illumination subsets of the CMU PIE database. We also present methods that reduce the memory requirements of these filters to run on limited computational resources, including computationally efficient methods of synthesizing these filters. Finally, we describe an online training algorithm implemented on a face verification system for synthesizing correlation filters from a video stream to handle pose/scale variations. The system also uses an efficient scheme to perform face localization within the current framework during the authentication stage.
The multifactor extension of Grassmann manifolds for face recognition

We propose the use of a multifactor model that extends Grassmann manifold to multiple factor frameworks. Both manifold learning algorithms and multifactor analysis are state-of-the-art dimension reduction techniques that are suitable to model variations of face images. In this paper, we demonstrate that Grassmann manifold can be extended to Mul-tifactor Grassmann manifold when used in conjunction with Multilinear PCA (MPCA). Indeed, the multifactor manifold learning algorithm proposed in this paper can be interpreted as MPCA's kernel-based extension using a kernel function that is defined in terms of geodesic distance. As a result, we first propose the use of Multifactor Grassmann manifold, which can learn both a multifactor structure and an underlying manifold in a given set of face images. We then demonstrate that our proposed method, Multifactor Grassmann manifold, produces more reliable results in the context of face recognition than the traditional dimension reduction techniques.
Using Feature Combination and Statistical Resampling for Accurate Face Recognition Based on Frequency Domain Representation of Facial Asymmetry

This paper explores the efficiency of facial asymmetry in face identification tasks using a frequency domain representation. Satisfactory results are obtained for two different tasks, namely, human identification under extreme expression variations and expression classification, using a PCA-type classifier on a database with 55 individuals, which establishes the robustness of these measures to intra-personal distortions. Furthermore, we demonstrate that it is possible to improve upon these results significantly by simple means such as feature set combination and statistical resampling methods like bagging and random subspace method (RSM) using the same PCA-type base classifier. This even succeeds in attaining perfect classification results with 100% accuracy in some cases. Moreover, both these methods require few additional resources (computing time and power), hence they are useful for practical applications as well and help establish the effectiveness of frequency domain representation of facial asymmetry in automatic identification tasks
Palmprint Recognition with Multiple Correlation Filters Using Edge Detection for Class-Specific Segmentation

We present a new series of results that show the competitive performance of advanced correlation filter classifiers for palmprint recognition. We design multiple correlation filters in subregions of the palmprint for each class. We propose a segmentation stage that selects palmprint subregions to train the filters in a class-by-class basis using different edge-detection operators. This effectively guides the filter training process to rely on regions that have a stronger line content, increasing between-class separation of the palm-prints. We evaluate the proposed algorithm in a large palmprint database of 385 classes. Our preliminary results show that most classes can be perfectly separated and the average equal error rates are as low as 0.0003% for regions of interest of size 64 times 64 pixels.