nils-herrmann/clean_scirepeval huggingface dataset:
Report on the BTAS 2016 Video Person Recognition Evaluation

This report presents results from the Video Person Recognition Evaluation held in conjunction with the 8th IEEE International Conference on Biometrics: Theory, Applications, and Systems (BTAS). Two experiments required algorithms to recognize people in videos from the Point-and-Shoot Face Recognition Challenge Problem (PaSC). The first consisted of videos from a tripod mounted high quality video camera. The second contained videos acquired from 5 different handheld video cameras. There were 1,401 videos in each experiment of 265 subjects. The subjects, the scenes, and the actions carried out by the people are the same in both experiments. An additional experiment required algorithms to recognize people in videos from the Video Database of Moving Faces and People (VDMFP). There were 958 videos in this experiment of 297 subjects. Four groups from around the world participated in the evaluation. The top verification rate for PaSC from this evaluation is 0.98 at a false accept rate of 0.01 - a remarkable advancement in performance from the competition held at FG 2015.
Unsupervised segmentation of color images

A novel technique for unsupervised learning in feature space is presented. The feature space is considered as composed of two distinct sources, "mode" and "valley", in the point of view of information theory. An entropy-based thresholding is taken to distinguish the discretized cells in the feature space. The cells labeled as "mode" are then chained to form mode areas. Thereafter a modified Akaike's information criterion is proposed to solve the cluster validity problem. After all the parameters are estimated, a labeling algorithm is developed based on the majority game theory. The method is applied to color image segmentation. The segmentation process is completely autonomous.
Facial Expression Recognition Influenced by Human Aging

Facial expression recognition (FER) is an active research topic in computer vision. However, there is no study yet to discover whether FER is affected by human aging, from a computational perspective. We perform a computational study of FER within and across age groups and compare the FER accuracies. Two databases from the psychology society are introduced to the computer vision community and used for our study. We found that the FER is influenced significantly by human aging, and we analyze the influence and interpret it from a computational viewpoint. Next, we propose some schemes to reduce the influence of aging on FER and evaluate the effectiveness in dealing with lifespan FER.
Human age estimation: What is the influence across race and gender?

In this paper we study some problems important for large-scale human age estimation. First, we study age estimation performance under variations across race and gender. Through a large number of age estimation experiments, significant differences are observed for age estimation between “no crossing” and “crossing.” Our study discovers that crossing race and gender can result in significant error increases for age estimation. This finding provides a guide for age estimation in practice, especially for cross-database experiments. Second, we propose a complete framework for crossing race and gender age estimation, based on our findings. Third, age estimation is performed on the large database of MORPH-II with more than 55,000 images. A small MAE of 4.45 years is obtained based on our proposed framework, which is much smaller than a recently reported MAE of 8.60 years on MORPH-II.
A study on human age estimation under facial expression changes

In this paper, we study human age estimation in face images under significant expression changes. We will address two issues: (1) Is age estimation affected by facial expression changes and how significant is the influence? (2) How to develop a robust method to perform age estimation undergoing various facial expression changes? This systematic study will not only discover the relation between age estimation and expression changes, but also contribute a robust solution to solve the problem of cross-expression age estimation. This study is an important step towards developing a practical and robust age estimation system that allows users to present their faces naturally (with various expressions) rather than constrained to the neutral expression only. Two databases originally captured in the Psychology community are introduced to Computer Vision, to quantitatively demonstrate the influence of expression changes on age estimation, and evaluate the proposed framework and corresponding methods for cross-expression age estimation.
Visually Interpretable Representation Learning for Depression Recognition from Facial Images

Recent evidence in mental health assessment have demonstrated that facial appearance could be highly indicative of depressive disorder. While previous methods based on the facial analysis promise to advance clinical diagnosis of depressive disorder in a more efficient and objective manner, challenges in visual representation of complex depression pattern prevent widespread practice of automated depression diagnosis. In this paper, we present a deep regression network termed DepressNet to learn a depression representation with visual explanation. Specifically, a deep convolutional neural network equipped with a global average pooling layer is first trained with facial depression data, which allows for identifying salient regions of input image in terms of its severity score based on the generated depression activation map (DAM). We then propose a multi-region DepressNet, with which multiple local deep regression models for different face regions are jointly leaned and their responses are fused to improve the overall recognition performance. We evaluate our method on two benchmark datasets, and the results show that our method significantly boosts state-of-the-art performance of the visual-based depression recognition. Most importantly, the DAM induced by our learned deep model may help reveal the visual depression pattern on faces and understand the insights of automated depression diagnosis.
A Study on Visible to Infrared Action Recognition

Human action recognition is important in image and video processing with many applications. With the development of sensor technology, different cameras can be used for action acquisition, e.g., infrared cameras. Is it possible to adapt the visible light action recognizers to a new modality or domain? In this paper, we study the feasibility to adapt the action recognizer learned from visible light spectrum to infrared. A preliminary result is obtained on a large database based on an adaptive learning method, demonstrating the potential to perform cross-spectral action recognition.
Cross-Age Face Recognition on a Very Large Database: The Performance versus Age Intervals and Improvement Using Soft Biometric Traits

Facial aging can degrade the face recognition performance dramatically. Traditional face recognition studies focus on dealing with pose, illumination, and expression (PIE) changes. Considering a large span of age difference, the influence of facial aging could be very significant compared to the PIE variations. How big the aging influence could be? What is the relation between recognition accuracy and age intervals? Can soft biometrics be used to improve the face recognition performance under age variations? In this paper we address all these issues. First, we investigate the face recognition performance degradation with respect to age intervals between the probe and gallery images on a very large database which contains about 55,000 face images of more than 13,000 individuals. Second, we study if soft biometric traits, e.g., race, gender, height, and weight, could be used to improve the cross-age face recognition accuracies, and how useful each of them could be.
Head pose estimation: Classification or regression?

Head pose estimation has many useful applications in practice. How to estimate the head pose automatically and robustly is still a challenging problem. In pose estimation, different pose angles can be used as regression values or viewed as different class labels. Thus a question is raised in our study: which is proper for pose estimation - classification or regression? We investigate representative classification and regression methods on the same problem to see any difference. A method that combines regression and classification approaches is also examined. Preliminary experiments show some interesting results which might prompt further exploration of related issues in pose estimation.
Still-to-Video Face Matching Using Multiple Geodesic Flows

Still-to-video (S2V) face recognition has recently attracted attention from researchers because of its great applications in real-world scenarios. In S2V FR, still images are usually of high quality, captured from cooperative users under controlled environment, such as mugshots, while video clips may be acquired with low resolutions and low quality, from non-cooperative users under uncontrolled environment. Because of those significant differences, we interpret the S2V FR as a heterogeneous matching problem, and propose an approach aiming at building multiple “bridges” between those two heterogeneous face modalities. Considering the unbalanced distributions and large diversities between two modalities, we propose to exploit a Grassmann manifold learning method to construct subspaces in between to find connections (or transitions) between the still images and video clips. Multiple geodesic flows are generated connecting the subspace of still images and the clustered subspace centers of videos, which are representative and robust to characterize the relationship between still images and video frames. Extensive experiments are conducted on two large scale benchmark databases, COX-S2V and PaSC, with different recognition tasks: face identification and verification. The experimental results show that the proposed approach outperforms the state-of-the-art methods under the same experimental settings.
Unsupervised segmentation of SAR images

A novel technique for unsupervised learning in feature space is presented. The features are derived by Gabor filters, and the feature space is considered as composed of two distinct sources, "mode" and "valley", in the point of view of information theory. An entropy-based thresholding is taken to distinguish the discretized cells in the feature space. The cells labeled as "mode" are then chained to form mode areas. Thereafter a modified Akaike information criterion is proposed to solve the cluster validity problem. After all the parameters are estimated, a labeling algorithm is developed based on the majority game theory. The method is applied to synthetic aperture radar (SAR) image segmentation. The segmentation process is completely autonomous.
A New Approach for 2D-3D Heterogeneous Face Recognition

This paper proposes a novel scheme for face recognition from visible images to depth images. In our proposed technique, we adopt Partial Least Square (PLS) to handle correlation mapping between 2D to 3D. A considerable performance improvement is observed compared to using Canonical Correlation Analysis (CCA). To further improve the performance, a fusion scheme based on PLS and CCA is advocated. We evaluate the advocated approach on a popular face dataset-FRGCV2.0. Experimental results demonstrate that the proposed scheme is an effective approach to perform 2D-3D face recognition.
A Probabilistic Fusion Approach to human age prediction

Human age prediction is useful for many applications. The age information could be used as a kind of semantic knowledge for multimedia content analysis and understanding. In this paper we propose a probabilistic fusion approach (PFA) that produces a high performance estimator for human age prediction. The PFA framework fuses a regressor and a classifier. We derive the predictor based on Bayespsila rule without the mutual independence assumption that is very common for traditional classifier combination methods. Using a sequential fusion strategy, the predictor reduces age estimation errors significantly. Experiments on the large UIUC-IFP-Y aging database and the FG-NET aging database show the merit of the proposed approach to human age prediction.
Efficient Group-n Encoding and Decoding for Facial Age Estimation

Different ages are closely related especially among the adjacent ages because aging is a slow and extremely non-stationary process with much randomness. To explore the relationship between the real age and its adjacent ages, an age group-n encoding (AGEn) method is proposed in this paper. In our model, adjacent ages are grouped into the same group and each age corresponds to n groups. The ages grouped into the same group would be regarded as an independent class in the training stage. On this basis, the original age estimation problem can be transformed into a series of binary classification sub-problems. And a deep Convolutional Neural Networks (CNN) with multiple classifiers is designed to cope with such sub-problems. Later, a Local Age Decoding (LAD) strategy is further presented to accelerate the prediction process, which locally decodes the estimated age value from ordinal classifiers. Besides, to alleviate the imbalance data learning problem of each classifier, a penalty factor is inserted into the unified objective function to favor the minority class. To compare with state-of-the-art methods, we evaluate the proposed method on FG-NET, MORPH II, CACD and Chalearn LAP 2015 databases and it achieves the best performance.
What Is the Challenge for Deep Learning in Unconstrained Face Recognition?

Recently deep learning has become dominant in face recognition and many other artificial intelligence areas. We raise a question: Can deep learning truly solve the face recognition problem? If not, what is the challenge for deep learning methods in face recognition? We think that the face image quality issue might be one of the challenges for deep learning, especially in unconstrained face recognition. To investigate the problem, we partition face images into different qualities, and evaluate the recognition performance, using the state-of-the-art deep networks. Some interesting results are obtained, and our studies can show directions to promote the deep learning methods towards high-accuracy and practical use in solving the hard problem of unconstrained face recognition.
Boosting for fast face recognition

We propose to use the AdaBoost algorithm for face recognition. AdaBoost is a kind of large margin classifiers and is efficient for online learning. In order to adapt the AdaBoost algorithm to fast face recognition, the original AdaBoost which uses all given features is compared with the boosting feature dimensions. The comparable results assure the use of the latter, which is faster for classification. The AdaBoost is typically a classification between two classes. To solve the multi-class recognition problem, we propose to use a constrained majority voting strategy to largely reduce the number of pairwise comparisons, without losing the recognition accuracy. Experimental results on a large face database of 1079 faces of 137 individuals show the feasibility of our approach for fast face recognition.
A New Approach to Teeth Segmentation

Teeth segmentation is one of the important components in building an Automated Dental Identification System (ADIS). The extraction of the teeth from their corresponding dental radiographs is called teeth segmentation. Dental radiographs may suffer from poor teeth image quality, low contrast and uneven exposure that complicate the task of teeth segmentation. To achieve a good performance in segmentation, the teeth images are preprocessed by a two-step thresholding technique, which starts with an iterative thresholding followed by an adaptive thresholding to binarize the teeth images. Then, we propose to adapt the seam carving technique on the binary images, using both horizontal and vertical seams, to separate each individual tooth. The proposed method is evaluated experimentally and compared to other algorithms. The results show that our new approach achieves the lowest failure rate among all existing methods, and the highest optimality among all of the fully automated approaches reported in the literature.
Face Authentication With Makeup Changes

Recent studies have shown that facial cosmetics have an impact on face recognition. To develop a face recognition system that is robust to facial makeup, we propose performing correlation mapping between makeup and nonmakeup faces on features extracted from local patches. Three methods are explored to learn the correlations. We also study the problem of makeup detection. Four categories of features are proposed to characterize cosmetics, including skin color tone, skin smoothness, texture, and highlight. A patch selection scheme and discriminative mapping are presented to enhance the performance of makeup detection. A complete system is then developed for face verification utilizing the makeup detection result. Experimental results show that our system is robust to cosmetics in face authentication. An accuracy of about 80.0% can be achieved on a database of about 500 pairs of makeup and nonmakeup face images.
Kinship Measurement on Salient Facial Features

Humans have the capability to recognize family members. Phrases such as “John has his father's nose” or “Joe has his mother's eyes” are quite common. Motivated by this, we consider the following question: Is it possible to develop a method to extract the salient familial traits in face images for kinship recognition? If this idea works, an instrument may be invented to measure familial relationships. This computational kinship measurement might have a large impact in real applications, such as child adoptions, trafficking/smuggling of children, and finding missing children. The novel problem is related to but very different from traditional face recognition. It is more challenging than a typical face recognition problem since we need to find subtle features that are reliable across a large span of ages (e.g., grandfather and grandson) and sex difference (e.g., mother and son). A recently developed descriptor, i.e., DAISY, is adapted to our problem to represent the salient features, and a dynamic scheme is developed to stochastically combine familial traits. Experiments are performed on a database to show that our new approach can perform reasonably well for kinship verification. The encouraging result may inspire further research on this emerging problem.
Relating Diagnosability, Strong Diagnosability and Conditional Diagnosability of Strong Networks

An interconnection network's diagnosability is an important measure of its self-diagnostic capability. Based on the classical notion of diagnosability, strong diagnosability and conditional diagnosability were proposed later to better reflect the networks' self-diagnostic capability under more realistic assumptions. In this paper, we study a class of interconnection networks called strong networks, which are n-regular, (n - 1)-connected, and with cn-number no more than n - 3. We build a relationship among the three diagnosability measures for strong networks. Under both PMC and MM* models, given a strong network G with diagnosability t, we prove that G is strongly t-diagnosable if and only if G's conditional diagnosability is greater than t. A simple check can show that almost all well-known regular interconnection networks are strong networks. The significance of this paper's result is that it reveals an important relationship between strong and conditional diagnosabilities, and the proof of strong diagnosability for many interconnection networks under MM* or PMC model is not necessary if their conditional diagnosability can be shown to be strictly larger than their diagnosability.
A joint optimization scheme to combine different levels of features for face recognition with makeup changes

We address the problem of makeup face recognition. Our main idea is to incorporate different levels of features into a joint optimization framework. Specifically, we combine both mid-level (e.g. attributes) and low-level features to obtain a new representation for a better matching between makeup and non-makeup faces. Previous studies have discovered the influence of cosmetics on face recognition, and some methods have been developed to address makeup face recognition. However, it is for the first time to integrate different levels of features to address this challenging problem. To minimize the differences between makeup and non-makeup faces, transformations are learned in a joint optimization framework which combines different levels of features. Experimental results demonstrate that our proposed approach outperforms the state-of-the-art methods.
Attributes in Multiple Facial Images

Facial attribute recognition is conventionally computed from a single image. In practice, each subject may have multiple face images. Taking the eye size as an example, it should not change, but it may have different estimation in multiple images, which would make a negative impact on face recognition. Thus, how to compute these attributes corresponding to each subject rather than each single image is a profound work. To address this question, we deploy deep training for facial attributes prediction, and we explore the inconsistency issue among the attributes computed from each single image. Then, we develop two approaches to address the inconsistency issue. Experimental results show that the proposed methods can handle facial attribute estimation on either multiple still images or video frames, and can correct the incorrectly annotated labels. The experiments are conducted on two large public databases with annotations of facial attributes.
Face recognition robust to head pose changes based on the RGB-D sensor

Face recognition is still a great challenge in biometrics research, because of the large variations of facial appearance, caused by head pose, lighting, facial expression, aging, etc. Among all possible variations, the biggest change of facial appearance in 2-dimensional (2D) face images probably comes from the three-dimensional (3D) head rotations. With the sensor technology advances, e.g., the recent RGB-D cameras, we study the advantage of using RGB-D images for face recognition, focusing on the challenge of 3D head pose variations. We propose an approach to face recognition robust to head rotations utilizing the RGB-D face images. Unlike the traditional 3D morphable model, our method does not need to learn a generic face model or take a complicated 3D to 2D face registration. We study what is the appropriate scheme to deal with pose variations in order to develop a robust system towards pose-invariant face recognition. Experiments on a public database show that our approach is effective and efficient for face recognition under significant pose changes. Our preliminary result demonstrates the advantages of using the RGB-D sensor for face recognition robust to large pose variations.