nils-herrmann/clean_scirepeval huggingface dataset:
3-D Generic Elastic Models for Fast and Texture Preserving 2-D Novel Pose Synthesis

This paper provides an in-depth analysis on face shape alignment for pose insensitive face recognition. The dissimilarity between two face images can be modeled as the difference in intensity between these two images, obtained by warping these faces onto the same shape. In order to achieve this, we must first align both face images independently to obtain a sparse 2-D shape representation. We achieve this by using a Combination of ASMs and AAMs (CASAAMs). We then exchange these two shapes and obtain new intensity (texture) faces based on these exchanged shapes. This allows us to align the two faces with increased pixel-level correspondence while simultaneously achieving a certain degree of pose correction. In order to account for large pose variation, it becomes necessary to model the underlying 3-D face structure for the synthesis of novel 2-D poses. However, in many real-world scenarios, only a single image of the subject is provided and acquisition of the 3-D model is not always feasible. To tackle this common real-world scenario, we propose a novel approach for modeling faces, called 3D Generic Elastic Model (3D-GEM), which can be deformed from a single 2-D image. Our analysis shows that 3-D depth information of human faces does not dramatically change across people, indicating that precise depth information of a person is not needed to generate useful novel 2-D poses. This is a significant discovery that makes our method feasible. We thus demonstrate that our 3-D face model can be efficiently produced by using a generic depth model which can be elastically deformed based on input facial features. This face model can then be rotated in 3-D in order to synthesize any arbitrary 2-D facial pose. Experimental results show that 3-D faces modeled by our proposed work effectively handle large 3-D pose changes in face alignment and they can be used for achieving pose tolerant face recognition. We also provide comparative results of face synthesis obtained by an actual 3-D face scanner and our approach, showing our proposed modeling approach is both effective and efficient.
Robust modified Active Shape Model for automatic facial landmark annotation of frontal faces

In this paper we present an improved method for locating facial landmarks in images containing frontal faces using a modified Active Shape Model. Our main contributions include the use of an optimal number of facial landmark points, better profiling methods during the fitting stage and the development of a more suitable optimization metric to determine the best location of the landmarks compared to the simplistic minimum Mahalanobis distance criteria used to date. We build a subspace to model variations of appearance around each facial landmark and use this subspace to enhance the accuracy of the fitting process around each landmark. This enhancement provides a significant improvement in fitting and simultaneously determines which points were poorly fitted using reconstruction error, thus allowing for automatic correction or interpolation of any poorly fitted points. Our implementation, with the above mentioned improvements, leads to extremely accurate results even when dealing with faces with expressions, slight pose variations and in-plane rotations. Experiments conducted on test sets drawn from three databases (NIST Multiple Biometric Grand Challenge-2008 (MBGC-2008), CMU Multi-PIE and the Japanese Female Facial Expression (JAFFE) database) show that our proposed approach leads to far better performance compared to the classical Active Shape Model of Cootes et al. and other traditional methods and provides a robust automatic facial landmark annotation which is the first critical step in face registration, pose correction and face recognition.
Towards a Unified Framework for Pose, Expression, and Occlusion Tolerant Automatic Facial Alignment

We propose a facial alignment algorithm that is able to jointly deal with the presence of facial pose variation, partial occlusion of the face, and varying illumination and expressions. Our approach proceeds from sparse to dense landmarking steps using a set of specific models trained to best account for the shape and texture variation manifested by facial landmarks and facial shapes across pose and various expressions. We also propose the use of a novel <inline-formula><tex-math notation="LaTeX">$\ell _1$</tex-math><alternatives> <inline-graphic xlink:type="simple" xlink:href="seshadri-ieq1-2505301.gif"/></alternatives></inline-formula>-regularized least squares approach that we incorporate into our shape model, which is an improvement over the shape model used by several prior Active Shape Model (ASM) based facial landmark localization algorithms. Our approach is compared against several state-of-the-art methods on many challenging test datasets and exhibits a higher fitting accuracy on all of them.
Discriminative Invariant Kernel Features: A Bells-and-Whistles-Free Approach to Unsupervised Face Recognition and Pose Estimation

We propose an explicitly discriminative and 'simple' approach to generate invariance to nuisance transformations modeled as unitary. In practice, the approach works well to handle non-unitary transformations as well. Our theoretical results extend the reach of a recent theory of invariance to discriminative and kernelized features based on unitary kernels. As a special case, a single common framework can be used to generate subject-specific pose-invariant features for face recognition and vice-versa for pose estimation. We show that our main proposed method (DIKF) can perform well under very challenging large-scale semisynthetic face matching and pose estimation protocols with unaligned faces using no landmarking whatsoever. We additionally benchmark on CMU MPIE and outperform previous work in almost all cases on off-angle face matching while we are on par with the previous state-of-the-art on the LFW unsupervised and image-restricted protocols, without any low-level image descriptors other than raw-pixels.
In between 3D Active Appearance Models and 3D Morphable Models

In this paper we propose a novel method of generating 3D morphable models (3DMMs) from 2D images. We develop algorithms of 3D face reconstruction from a sparse set of points acquired from 2D images. In order to establish correspondence between images precisely, we combined active shape models (ASMs) and active appearance models (AAMs)(CASAAMs) in an intelligent way, showing improved performance on pixel-level accuracy and generalization to unseen faces. The CASAAMs are applied to the images of different views of the same person to extract facial shapes across pose. These 2D shapes are combined for reconstructing a sparse 3D model. The point density of the model is increased by the loop subdivision method, which generates new vertices by a weighted sum of the existing vertices. Then, the depth of the dense 3D model is modified with an average 3D depth-map in order to preserve facial structure more realistically. Finally, all 249 3D models with expression changes are combined to generate a 3DMM for a compact representation. The first session of the multi-PIE database, consisting of 249 persons with expression and illumination changes, is used for the modeling. Unlike typical 3DMMs, our model can generate 3D human faces more realistically and efficiently (2-3 seconds on P4 machine) under diverse illumination conditions.
Automatic iris mask refinement for high performance iris recognition

How to estimate artifacts on an iris image in polar domain is an important question for any iris recognition system which pursues high recognition rate as its goal. In literature, there are many different existing algorithm that estimate iris occlusion in either Cartesian or polar coordinate. In this paper, our goal is not to propose another new method to compete with existing method. Rather, our goal is to propose a new algorithm which can take any iris mask estimated by existing algorithm, and refine it into a much more accurate mask. In this way, our proposed method could co-work with any other existing algorithm and improve iris recognition performance. Experimental results show our proposed method can improve iris recognition rate by a great lead compared to the performance of the system using the unrefined iris masks.
An extension of multifactor analysis for face recognition based on submanifold learning

Lately, Multilinear Principal Component Analysis (MPCA) has been successfully applied to face recognition since MPCA provides analysis of multiple factors of face images such as people's identities, viewpoints, and lighting conditions. MPCA employees multiple linear subspaces constructed by varying factors. In this paper, we propose nonlinear submanifold analysis, which can represent the variation of each factor more accurately than the conventional multilinear subspace analysis. Based on submani-fold learning, we propose an extension of the multiple factor analysis. This paper proposes the kernel-based extension of MPCA whose definition of a kernel function and neighbors of each sample is robust for submanifold learning. The experimental results in this paper demonstrate that the proposed methods produce a synergetic advantage for face recognition. This is because our method offers the combined virtues of both multifactor analysis and manifold learning.
An analysis of facial shape and texture for recognition: A large scale evaluation on FRGC ver2.0

Traditional approaches to face recognition have utilized aligned facial images containing both shape and texture information. This paper analyzes the contributions of the individual facial shape and texture components to face recognition. These two components are evaluated independently and we investigate methods to combine the information gained from each of them to enhance face recognition performance. The contributions of this paper are the following: (1) to the best of our knowledge, it is the first large-scale study of how face recognition is influenced by shape and texture as all of our results are benchmarked against traditional approaches on the challenging NIST FRGC ver2.0 experiment 4 dataset, (2) we empirically show that shape information is reasonably discriminative, (3) we demonstrate significant improvement in performance by registering texture with dense shape information, and finally (4) show that fusing shape and texture information consistently boosts recognition results across different subspace-based algorithms.
Robust hand detection in Vehicles

The problems of hand detection have been widely addressed in many areas, e.g. human computer interaction environment, driver behaviors monitoring, etc. However, the detection accuracy in recent hand detection systems are still far away from the demands in practice due to a number of challenges, e.g. hand variations, highly occlusions, low-resolution and strong lighting conditions. This paper presents the Multiple Scale Faster Region-based Convolutional Neural Network (MS-FRCNN) to handle the problems of hand detection in given digital images collected under challenging conditions. Our proposed method introduces a multiple scale deep feature extraction approach in order to handle the challenging factors to provide a robust hand detection algorithm. The method is evaluated on the challenging hand database, i.e. the Vision for Intelligent Vehicles and Applications (VIVA) Challenge, and compared against various recent hand detection methods. Our proposed method achieves the state-of-the-art results with 20% of the detection accuracy higher than the second best one in the VIVA challenge.
Improved Human Face Identification Using Frequency Domain Representation of Facial Asymmetry

This paper explores the role of facial asymmetry in identification tasks using a frequency domain representation. Satisfactory results are obtained for two different tasks, namely, human identification under extreme expression variations and expression classification, using a PCA-type classifier which establishes the robustness of these measures to intra-personal distortions. We next demonstrate that it is possible to even improve upon these results by simple means. In particular, we use two methods, namely, feature set combination and statistical resampling methods like bagging, which attains perfect classification results (0% error rate) in some cases. Both these methods require very few additional resources in terms of computing power, hence they are useful for practical applications as well
A novel eyebrow segmentation and eyebrow shape-based identification

Recent studies in biometrics have shown that the periocular region of the face is sufficiently discriminative for robust recognition, and particularly effective in certain scenarios such as extreme occlusions, and illumination variations where traditional face recognition systems are unreliable. In this paper, we first propose a fully automatic, robust and fast graph-cut based eyebrow segmentation technique to extract the eyebrow shape from a given face image. We then propose an eyebrow shape-based identification system for periocular face recognition. Our experiments have been conducted over large datasets from the MBGC and AR databases and the resilience of the proposed approach has been evaluated under varying data conditions. The experimental results show that the proposed eyebrow segmentation achieves high accuracy with an F-Measure of 99.4% and the identification system achieves rates of 76.0% on the AR database and 85.0% on the MBGC database.
CUDA accelerated illumination preprocessing on GPUs

In this paper we develop a parallelized implementation of the anisotropic diffusion image preprocessing algorithm for illumination invariant face recognition proposed by Gross and Brajovic. Our implementation employs Red-Black Gauss-Seidel relaxation running on inexpensive Graphics Processing Units (GPUs) programmed with Nvidia's CUDA framework. We are able to achieve a 20X speedup over a multithreaded implementation running on a quadcore CPU. Additionally a comparison to an open-source implementation of anisotropic diffusion in the Torch3vision library is performed, demonstrating a GPU speedup of greater than 900X over this commonly used machine vision library.
Long range iris acquisition system for stationary and mobile subjects

Most iris based biometric systems require a lot of cooperation from the users so that iris images of acceptable quality may be acquired. Features from these may then be used for recognition purposes. Relatively fewer works in literature address the question of less cooperative iris acquisition systems in order to reduce constraints on users. In this paper, we describe our ongoing work in designing and developing such a system. It is capable of capturing images of the iris up to distances of 8 meters with a resolution of 200 pixels across the diameter. If the resolution requirement is decreased to 150 pixels, then the same system may be used to capture images from up to 12 meters. We have incorporated velocity estimation and focus tracking modules so that images may be acquired from subjects on the move as well. We describe the various components that make up the system, including the lenses used, the imaging sensor, our auto-focus function and velocity estimation module. All the hardware components are Commercial Off The Shelf (COTS) with little or no modifications. We also present preliminary iris acquisition results using our system for both stationary and mobile subjects.
NIR-VIS heterogeneous face recognition via cross-spectral joint dictionary learning and reconstruction

A lot of real-world data is spread across multiple domains. Handling such data has been a challenging task. Heterogeneous face biometrics has begun to receive attention in recent years. In real-world scenarios, many surveillance cameras capture data in the NIR (near infrared) spectrum. However, most datasets accessible to law enforcement have been collected in the VIS (visible light) domain. Thus, there exists a need to match NIR to VIS face images. In this paper, we approach the problem by developing a method to reconstruct VIS images in the NIR domain and vice-versa. This approach is more applicable to real-world scenarios since it does not involve having to project millions of VIS database images into learned common subspace for subsequent matching. We present a cross-spectral joint ℓ0 minimization based dictionary learning approach to learn a mapping function between the two domains. One can then use the function to reconstruct facial images between the domains. Our method is open set and can reconstruct any face not present in the training data. We present results on the CASIA NIR-VIS v2.0 database and report state-of-the-art results.
Compressed Submanifold Multifactor Analysis

Although widely used, Multilinear PCA (MPCA), one of the leading multilinear analysis methods, still suffers from four major drawbacks. First, it is very sensitive to outliers and noise. Second, it is unable to cope with missing values. Third, it is computationally expensive since MPCA deals with large multi-dimensional datasets. Finally, it is unable to maintain the local geometrical structures due to the averaging process. This paper proposes a novel approach named Compressed Submanifold Multifactor Analysis (CSMA) to solve the four problems mentioned above. Our approach can deal with the problem of missing values and outliers via SVD-L1. The Random Projection method is used to obtain the fast low-rank approximation of a given multifactor dataset. In addition, it is able to preserve the geometry of the original data. Our CSMA method can be used efficiently for multiple purposes, e.g., noise and outlier removal, estimation of missing values, biometric applications. We show that CSMA method can achieve good results and is very efficient in the inpainting problem as compared to  ,  . Our method also achieves higher face recognition rates compared to LRTC, SPMA, MPCA and some other methods, i.e., PCA, LDA and LPP, on three challenging face databases, i.e., CMU-MPIE, CMU-PIE and Extended YALE-B.
Multiple Scale Faster-RCNN Approach to Driver’s Cell-Phone Usage and Hands on Steering Wheel Detection

In this paper, we present an advanced deep learning based approach to automatically determine whether a driver is using a cell-phone as well as detect if his/her hands are on the steering wheel (i.e. counting the number of hands on the wheel). To robustly detect small objects such as hands, we propose Multiple Scale Faster-RCNN (MSFRCNN) approach that uses a standard Region Proposal Network (RPN) generation and incorporates feature maps from shallower convolution feature maps, i.e. conv3 and conv4, for ROI pooling. In our driver distraction detection framework, we first make use of the proposed MS-FRCNN to detect individual objects, namely, a hand, a cell-phone, and a steering wheel. Then, the geometric information is extracted to determine if a cell-phone is being used or how many hands are on the wheel. The proposed approach is demonstrated and evaluated on the Vision for Intelligent Vehicles and Applications (VIVA) Challenge database and the challenging Strategic Highway Research Program (SHRP-2) face view videos that was acquired to monitor drivers under naturalistic driving conditions. The experimental results show that our method archives better performance than Faster R-CNN on both hands on wheel detection and cell-phone usage detection while remaining at similar testing cost. Compare to the state-of-the-art cell-phone usage detection, our approach obtains higher accuracy, is less time consuming and is independent to landmarking. The groundtruth database will be publicly available.
Analyzing asymmetry biometric in the frequency domain for face recognition

The paper introduces a novel set of facial biometrics based on quantified facial asymmetry measures in the frequency domain. In particular, we show that these biometrics work well for images showing expression variations. A comparison of the recognition rates with those obtained from spatial domain asymmetry measures based on raw intensity values suggests that the frequency domain representation is more robust to intra-personal distortions and, indeed, provides an efficient approach for performing classification or recognition. The role of asymmetry of the different regions (e.g., eyes, mouth, nose) of the face is investigated to determine which regions provide the maximum discrimination among individuals in the presence of different expressions for better classification results in such a scenario.
Spartans: Single-Sample Periocular-Based Alignment-Robust Recognition Technique Applied to Non-Frontal Scenarios

In this paper, we investigate a single-sample periocular-based alignment-robust face recognition technique that is pose-tolerant under unconstrained face matching scenarios. Our Spartans framework starts by utilizing one single sample per subject class, and generate new face images under a wide range of 3D rotations using the 3D generic elastic model which is both accurate and computationally economic. Then, we focus on the periocular region where the most stable and discriminant features on human faces are retained, and marginalize out the regions beyond the periocular region since they are more susceptible to expression variations and occlusions. A novel facial descriptor, high-dimensional Walsh local binary patterns, is uniformly sampled on facial images with robustness toward alignment. During the learning stage, subject-dependent advanced correlation filters are learned for pose-tolerant non-linear subspace modeling in kernel feature space followed by a coupled max-pooling mechanism which further improve the performance. Given any unconstrained unseen face image, the Spartans can produce a highly discriminative matching score, thus achieving high verification rate. We have evaluated our method on the challenging Labeled Faces in the Wild database and solidly outperformed the state-of-the-art algorithms under four evaluation protocols with a high accuracy of 89.69%, a top score among image-restricted and unsupervised protocols. The advancement of Spartans is also proven in the Face Recognition Grand Challenge and Multi-PIE databases. In addition, our learning method based on advanced correlation filters is much more effective, in terms of learning subject-dependent pose-tolerant subspaces, compared with many well-established subspace methods in both linear and non-linear cases.
Generic 3D face pose estimation using facial shapes

Generic 3D face pose estimation from a single 2D facial image is an extremely crucial requirement for face-related research areas. To meet with the remaining challenges for face pose estimation, suggested Murphy-Chutorian et al. , we believe that the first step is to create a large corpus of a 3D facial shape database in which the statistical relationship between projected 2D shapes and corresponding pose parameters can be easily observed. Because facial geometry provides the most essential information for facial pose, understanding the effect of pose parameters in 2D facial shapes is a key step toward solving the remaining challenges. In this paper, we present necessary tasks to reconstruct 3D facial shapes from multiple 2D images and then explain how to generate 2D projected shapes at any rotation interval. To deal with self occlusions, a novel hidden points removal (HPR) algorithm is also proposed. By flexibly changing the number of points in 2D shapes, we evaluate the performance of two different approaches for achieving generic 3D pose estimation in both coarse and fine levels and analyze the importance of facial shapes toward generic 3D pose estimation.