nils-herrmann/clean_scirepeval huggingface dataset:
Multi-modal biometrics for mobile authentication

User authentication in the context of a secure transaction needs to be continuously evaluated for the risks associated with the transaction authorization. The situation becomes even more critical when there are regulatory compliance requirements. Need for such systems have grown dramatically with the introduction of smart mobile devices which make it far easier for the user to complete such transaction quickly but with a huge exposure to risk. Biometrics can play a very significant role in addressing such problems as a key indicator of the user identity and thus reducing the risk of fraud. While unimodal biometrics authentication systems are being increasingly experimented by mainstream mobile system manufacturers (e.g., fingerprint in iOS), we explore various opportunities of reducing risk in a multimodal biometrics system. The multimodal system is based on fusion of several biometrics combined with a policy manager. A new biometric modality: chirography which is based on user writing on multi-touch screens using their finger is introduced. Coupling with chirography, we also use two other biometrics: face and voice. Our fusion strategy is based on inter-modality score level fusion that takes into account a voice quality measure. The proposed system has been evaluated on an in-house database that reflects the latest smart mobile devices. On this database, we demonstrate a very high accuracy multi-modal authentication system reaching an EER of 0.1% in an office environment and an EER of 0.5% in challenging noisy environments.
Learning face recognition from limited training data using deep neural networks

Often deep learning methods are associated with huge amounts of training data. The deeper the network gets, the larger is the need for training data. A large amount of labeled data helps the network learn about the variations it needs to handle in the prediction stage. It is not easy for everyone to get access to huge amounts of labeled data leaving a few to have the luxury to design very deep networks. In this paper, we propose to flatten the disparity by using the modeling methods to minimize the need for huge amounts of data for training a deep network. Using face recognition as an example, we demonstrate how limited labeled data can be leveraged to obtain near state of the art performance with generalization capability across multiple databases. In addition, we show that the normalization in the overall network can improve the speed and resource requirement for the prediction/inferencing stage.
Sectored Random Projections for Cancelable Iris Biometrics

Privacy and security are essential requirements in practical biometric systems. In order to prevent the theft of biometric patterns, it is desired to modify them through revocable and non invertible transformations called Cancelable Biometrics. In this paper, we propose an efficient algorithm for generating a Cancelable Iris Biometric based on Sectored Random Projections. Our algorithm can generate a new pattern if the existing one is stolen, retain the original recognition performance and prevent extraction of useful information from the transformed patterns. Our method also addresses some of the drawbacks of existing techniques and is robust to degradations due to eyelids and eyelashes.
An efficient, two-stage iris recognition system

There have been claims of very high information content in iris texture, higher even than in fingerprints. This makes iris attractive for large scale identification systems with possibly millions of people. However, some systems operate by performing N 1:1 matches of the probe against the database. This can get prohibitively expensive in terms of computation as N grows large. Note that for identification systems the per-match time dominants system performance, unlike verification where feature extraction time is the primary component. In this paper we show how to use a short-length iris code to pre-screen a large database and thereby reduce the number of full comparisons needed to a fraction of the total. Since the screening code is much smaller than the full iris code, the time to process the whole database is greatly reduced. As an added benefit, we show that we can use the alignment inferred from the short code to greatly restrict the range of alignments searched for the full code, which further speeds up the system. As we demonstrate in experiments, the two stage approach can reduce the cost and/or time needed by an order of magnitude with very little impact on identification performance.
Median Filter Based Iris Encoding Technique

Iris-based human recognition is very attractive because of the high accuracy achievable. However, existing encoding methods are unable to handle iris images acquired when the ambient lighting is non-uniform. In this paper we propose a novel encoding technique which can handle images acquired under such conditions. The method is based on 2D median filters, which are commonly used as de-noising tools in image processing domain, and uses their nonlinear characteristics to generate binary codes from gray scale iris images. The application of this method for robust iris recognition is discussed and compared with two more traditional iris encoding techniques: the 2D Gabor filter based method and the 1D log-Gabor method. Our algorithm is usable for the whole range of conditions encountered in the MMU1 iris database, performs well in a synthetic uneven illumination study, and gets comparable results on other tests.
Secure one-time biometrie tokens for non-repudiable multi-party transactions

Numerous applications require users to interact with a multitude of entities in order to avail a service. In such applications, the identity of the user is typically verified through physical or digital tokens, which are prone to both identity theft (lost or stolen tokens) and repudiation claims by malicious users. The use of biometrics can provide non-repudiability by ensuring that a purchaser is the bonafide user of the service. However, in applications entailing multiple stakeholders, there may be privacy issues with sharing user's biometrics data. While this concern can be addressed by storing the user's biometric data on the token itself, strong mechanisms are required to ensure that token is both secure and tamper-proof. In this paper, we propose a single use biometric token that relies on Shamir's secret sharing algorithm and blockchain technology to ensure that the encrypted biometric template contained in the token is secure, tamper-proof, and any attempt to use the issued token is irrefutably logged to prove subsequently that the user indeed availed the service. We also analyze issues related to the system security, user privacy, and usability of the proposed solution.
Iris individuality: a partial iris model

Biometrics-based personal authentication systems are becoming popular with increased demand on security. A biometrics is expected to have significant amount of discriminatory information in representing uniqueness of a person. This discriminatory information about the biometric is loosely defined as the individuality. Even though some individuality studies have been carried out for fingerprints, face and handwriting, formal analysis of iris individuality has not been carried out. We propose a partial model for iris individuality and show its usefulness in predicting the empirical performance results.
Multilayer perceptrons on Splash 2

Multilayer perceptrons (MLPs) are one of the most popular neural network models for solving pattern classification and image classification problems. Because of their ability to learn complex decision boundaries, MLPs are used in many practical computer vision applications involving classification (or supervised segmentation). Once the connection weights in a MLP have been learnt, the network can be used repeatedly for classification of new input patterns. Several special-purpose architectures have been described in the literature for neural networks as they are slow on a conventional uniprocessor. In this paper, we describe mapping of MLPs onto Splash 2-a "custom computing machine". The main features of the proposed mapping are: (i) the number of nodes in a layer is not fixed; (ii) the number of layers in the network is not fixed; (iii) it is based on a set of reprogrammable FPGAs and a programmable crossbar; and (iv) it has a significant speedup over a uniprocessor. The mapping has been used for implementing a 3-layer MLP for page segmentation application with an appreciable speedup of approximately 150 over a SPARCstation 20 for one million pattern vectors with 20 features per pattern.
Cancelable Biometrics: A review

Recent years have seen an exponential growth in the use of various biometric technologies for trusted automatic recognition of humans. With the rapid adaptation of biometric systems, there is a growing concern that biometric technologies may compromise the privacy and anonymity of individuals. Unlike credit cards and passwords, which can be revoked and reissued when compromised, biometrics are permanently associated with a user and cannot be replaced. To prevent the theft of biometric patterns, it is desirable to modify them through revocable and noninvertible transformations to produce cancelable biometric templates. In this article, we provide an overview of various cancelable biometric schemes for biometric template protection. We discuss the merits and drawbacks of available cancelable biometric systems and identify promising avenues of research in this rapidly evolving field.
A distributed edge detection and surface reconstruction algorithm

A scalable parallel algorithm for edge detection and surface reconstruction is presented. The algorithm is based on fitting a weak membrane to the pixel gray valves by minimizing the associated energy functional. The edge detection process is modeled as a line process and used as a constraint in minimizing the energy functional of the image. The optimal edge assignment cannot be obtained directly as the energy function is non-convex. Using graduated non-convexity (GNC) approach, the energy is minimized. The proposed parallel algorithm has been implemented on a cluster of workstations using the PVM communication library. The results of parallel implementation on synthetic and natural images are presented. The speedup is observed to be near-linear, thus providing scalability with the problem size. The parallel processing approach presented here can be extended to solve similar problems (e.g., image restoration, and image compression) which use regularization techniques.
Cancelable Biometrics: A Case Study in Fingerprints

Biometrics offers usability advantages over traditional token and password based authentication schemes, but raises privacy and security concerns. When compromised, credit cards and passwords can be revoked or replaced while biometrics are permanently associated with a user and cannot be replaced. Cancelable biometrics attempts to solve this by constructing revocable biometric templates. We present several constructs for cancelable templates using feature domain transformations and empirically examine their efficacy. We also present a method for accurate registration which is a key step in building cancelable transforms. The overall approach has been tested using large databases and our results demonstrate that without losing much accuracy, we can build a large number of cancelable transforms for fingerprints
Structure in errors: a case study in fingerprint verification

Measuring the accuracy of biometrics systems is important. Accuracy estimates depend very much on the quality of the test data that are used: including poor quality data will degrade the accuracy estimates. Factors that determine the good quality data and poor quality data can not be revealed by simple accuracy estimates. We propose a novel methodology to analyze how the overall accuracy estimate of a system relates to the specific quality of biometrics samples. Using a large collection of fingerprint samples, we present an analysis of system accuracy, which suggests that a significant part of the error is due to few fingers.
Gradient based Textural Characterization of Fingerprints

Though a lot of research has been done to match fingerprints, most existing approaches rely on locations of minutiae features for matching tasks. Relatively, little effort has gone into utilizing textural information present in fingerprints as distinguishing characteristic. In this paper, we propose a novel gradient-based approach to characterize textural information in fingerprints for the task of biometric matching. In particular, the proposed approach uses histograms of oriented gradients (HOGs) to represent minutiae neighborhoods. The minutiae neighborhoods are divided into several regions to make the computed histograms distinguishing and robust at the same time. Experimental results are provided to show the efficacy of the proposed characterization.
A Multimodal Database for Affect Recognition and Implicit Tagging

MAHNOB-HCI is a multimodal database recorded in response to affective stimuli with the goal of emotion recognition and implicit tagging research. A multimodal setup was arranged for synchronized recording of face videos, audio signals, eye gaze data, and peripheral/central nervous system physiological signals. Twenty-seven participants from both genders and different cultural backgrounds participated in two experiments. In the first experiment, they watched 20 emotional videos and self-reported their felt emotions using arousal, valence, dominance, and predictability as well as emotional keywords. In the second experiment, short videos and images were shown once without any tag and then with correct or incorrect tags. Agreement or disagreement with the displayed tags was assessed by the participants. The recorded videos and bodily responses were segmented and stored in a database. The database is made available to the academic community via a web-based system. The collected data were analyzed and single modality and modality fusion results for both emotion recognition and implicit tagging experiments are reported. These results show the potential uses of the recorded modalities and the significance of the emotion elicitation protocol.
Continuous Prediction of Spontaneous Affect from Multiple Cues and Modalities in Valence-Arousal Space

Past research in analysis of human affect has focused on recognition of prototypic expressions of six basic emotions based on posed data acquired in laboratory settings. Recently, there has been a shift toward subtle, continuous, and context-specific interpretations of affective displays recorded in naturalistic and real-world settings, and toward multimodal analysis and recognition of human affect. Converging with this shift, this paper presents, to the best of our knowledge, the first approach in the literature that: 1) fuses facial expression, shoulder gesture, and audio cues for dimensional and continuous prediction of emotions in valence and arousal space, 2) compares the performance of two state-of-the-art machine learning techniques applied to the target problem, the bidirectional Long Short-Term Memory neural networks (BLSTM-NNs), and Support Vector Machines for Regression (SVR), and 3) proposes an output-associative fusion framework that incorporates correlations and covariances between the emotion dimensions. Evaluation of the proposed approach has been done using the spontaneous SAL data from four subjects and subject-dependent leave-one-sequence-out cross validation. The experimental results obtained show that: 1) on average, BLSTM-NNs outperform SVR due to their ability to learn past and future context, 2) the proposed output-associative fusion framework outperforms feature-level and model-level fusion by modeling and learning correlations and patterns between the valence and arousal dimensions, and 3) the proposed system is well able to reproduce the valence and arousal ground truth obtained from human coders.
A demonstration of audiovisual sensitive artificial listeners

Sensitive artificial listeners (SAL) are virtual dialogue partners who, despite their very limited verbal understanding, intend to engage the user in a conversation by paying attention to the user's emotions and non-verbal expressions. The SAL characters have their own emotionally defined personality, and attempt to drag the user towards their dominant emotion, through a combination of verbal and non-verbal expression. The demonstrator shows an early version of the fully autonomous SAL system based on audiovisual analysis and synthesis.
Automatic visual mimicry expression analysis in interpersonal interaction

Mimicry occurs in conversations both when people agree with each other and when they do not. However, it has been reported that there is more mimicry when people agree than when they disagree: when people want to express shared opinions and attitudes, they do so by displaying behavior that is similar to their interlocutors' behavior. In a conversation, mimicry occurs in order to gain acceptance from an interaction partner by conforming to that person's attitudes, opinions, and behavior. In this paper we describe how visual behavioral information expressed between two interlocutors can be used to detect and identify visual mimicry. We extract and encode visual features that are expected to represent mimicry in a useful way. In order to show that mimicry has indeed occurred, we calculate correlations between visual features extracted from the interactants and compare these against each other and against a baseline. We show that it is possible to visualize the occurrence of visual mimicry during the progress of a conversation which allows us to research in which situations and to what extent mimicry occurs.
Audiovisual Detection of Behavioural Mimicry

Human mimicry is a behavioural cue occurring during social interaction that can inform us about the participants' inter-personal states and attitudes. It occurs when a participant in an interaction exhibits some behaviour as a result of a co-participants prior display of that signal, and occurs on both short and long time-scales. To develop a detection method for such behaviour, we use a method based on feature prediction, where we train an ensemble of regression models from one subject's features to the co-subject's features, for each class. The ensemble of models with lowest reconstruction error is used to detect mimicry and non-mimicry, using continuous audiovisual streams. As mimicry events are dynamical phenomena, we use a temporal regression model (long short-term memory neural networks) to capture sequential dependencies in the data. On a data set of ten 12-minute dyadic interaction episodes, our method gave average positive and negative recall rates of 77.5% and 60.0% respectively, on data with significant class imbalances, due to the relative sparsity of mimicry samples when doing continuous detection.
Continuous emotion detection using EEG signals and facial expressions

Emotions play an important role in how we select and consume multimedia. Recent advances on affect detection are focused on detecting emotions continuously. In this paper, for the first time, we continuously detect valence from electroencephalogram (EEG) signals and facial expressions in response to videos. Multiple annotators provided valence levels continuously by watching the frontal facial videos of participants who watched short emotional videos. Power spectral features from EEG signals as well as facial fiducial points are used as features to detect valence levels for each frame continuously. We study the correlation between features from EEG and facial expressions with continuous valence. We have also verified our model's performance for the emotional highlight detection using emotion recognition from EEG signals. Finally the results of multimodal fusion between facial expression and EEG signals are presented. Having such models we will be able to detect spontaneous and subtle affective responses over time and use them for video highlight detection.
Detecting facial actions and their temporal segments in nearly frontal-view face image sequences

The recognition of facial expressions in image sequences is a difficult problem with many applications in human-machine interaction. Facial expression analyzers achieve good recognition rates, but virtually all of them deal only with prototypic facial expressions of emotions and cannot handle temporal dynamics of facial displays. The method presented here attempts to handle a large range of human facial behavior by recognizing facial action units (AUs) and their temporal segments (i.e., onset, apex, offset) that produce expressions. We exploit particle filtering to track 20 facial points in an input face video and we introduce AU-dynamics recognition using temporal rules. When tested on Cohn-Kanade and MMI facial expression databases, the proposed method achieved a recognition rate of 90% when detecting 27 AUs occurring alone or in a combination in an input face image sequence.
Automatic Analysis of Facial Actions: A Survey

As one of the most comprehensive and objective ways to describe facial expressions, the Facial Action Coding System (FACS) has recently received significant attention. Over the past 30 years, extensive research has been conducted by psychologists and neuroscientists on various aspects of facial expression analysis using FACS. Automating FACS coding would make this research faster and more widely applicable, opening up new avenues to understanding how we communicate through facial expressions. Such an automated process can also potentially increase the reliability, precision and temporal resolution of coding. This paper provides a comprehensive survey of research into machine analysis of facial actions. We systematically review all components of such systems: pre-processing, feature extraction and machine coding of facial actions. In addition, the existing FACS-coded facial expression databases are summarised. Finally, challenges that have to be addressed to make automatic facial action analysis applicable in real-life situations are extensively discussed. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the future of machine recognition of facial actions: what are the challenges and opportunities that researchers in the field face.
Introduction to the "Best of ACII 2013" Special Section

The papers in this special section were presented at the 6th Biannual International Conference on Affective Computing and Intelligent Interaction (ACII 2013), held from September 2-5, 2013, in Geneva, Switzerland. These papers reflect the diversity of affective computing research, ranging from physiological models for affect detection, methodological issues, computational models of affect, and human perceptions of affective virtual agents.