nils-herrmann/clean_scirepeval huggingface dataset:
Recognising spontaneous facial micro-expressions

Facial micro-expressions are rapid involuntary facial expressions which reveal suppressed affect. To the best knowledge of the authors, there is no previous work that successfully recognises spontaneous facial micro-expressions. In this paper we show how a temporal interpolation model together with the first comprehensive spontaneous micro-expression corpus enable us to accurately recognise these very short expressions. We designed an induced emotion suppression experiment to collect the new corpus using a high-speed camera. The system is the first to recognise spontaneous facial micro-expressions and achieves very promising results that compare favourably with the human micro-expression detection accuracy.
Context based face anti-spoofing

The face recognition community has finally started paying more attention to the long-neglected problem of spoofing attacks and the number of countermeasures is gradually increasing. Fairly good results have been reported on the publicly available databases but it is reasonable to assume that there exists no superior anti-spoofing technique due to the varying nature of attack scenarios and acquisition conditions. Therefore, we propose to approach the problem of face spoofing as a set of attack-specific subproblems that are solvable with a proper combination of complementary countermeasures. Inspired by how we humans can perform reliable spoofing detection only based on the available scene and context information, this work provides the first investigation in research literature that attempts to detect the presence of spoofing medium in the observed scene. We experiment with two publicly available databases consisting of several fake face attacks of different nature under varying conditions and imaging qualities. The experiments show excellent results beyond the state of the art. More importantly, our cross-database evaluation depicts that the proposed approach has promising generalization capabilities.
Experiments with Facial Expression Recognition using Spatiotemporal Local Binary Patterns

In this paper, the recently introduced method for facial expression recognition using spatiotemporal local binary patterns is reviewed and experiments are carried out to investigate the robustness of the approach. In experiments with the Cohn-Kanade facial expression database, our results from the cross-validation with low resolutions and low frame rates are promising. Advantages of our approach include local processing, robustness to low quality of videos and simple computation.
Computer vision for face-to-face human-computer interaction

computer vision will play an important role in future human-computer interactions. In order to achieve natural human-computer interaction, there is a need for the computer to be able to interact with the user similar to the way human-human interaction takes place. In recent years we have been investigating various methods for different tasks of face-to-face interaction. Many of these are using image and video descriptors based on local binary patterns (LBP). This talk overviews our recent progress in LBP descriptors, face recognition and biometrics, recognition of facial expressions and emotions, visual speech recognition, synthesis of a talking face, and recognition of human actions and gestures. Results of our research on an experimental system for intelligent humanrobot interaction are also discussed.
Remote Heart Rate Measurement from Face Videos under Realistic Situations

Heart rate is an important indicator of people's physiological state. Recently, several papers reported methods to measure heart rate remotely from face videos. Those methods work well on stationary subjects under well controlled conditions, but their performance significantly degrades if the videos are recorded under more challenging conditions, specifically when subjects' motions and illumination variations are involved. We propose a framework which utilizes face tracking and Normalized Least Mean Square adaptive filtering methods to counter their influences. We test our framework on a large difficult and public database MAHNOB-HCI and demonstrate that our method substantially outperforms all previous methods. We also use our method for long term heart rate monitoring in a game evaluation scenario and achieve promising results.
Towards a practical lipreading system

A practical lipreading system can be considered either as subject dependent (SD) or subject-independent (SI). An SD system is user-specific, i.e., customized for some particular user while an SI system has to cope with a large number of users. These two types of systems pose variant challenges and have to be treated differently. In this paper, we propose a simple deterministic model to tackle the problem. The model first seeks a low-dimensional manifold where visual features extracted from the frames of a video can be projected onto a continuous deterministic curve embedded in a path graph. Moreover, it can map arbitrary points on the curve back into the image space, making it suitable for temporal interpolation. Based on the model, we develop two separate strategies for SD and SI lipreading. The former is turned into a simple curve-matching problem while for the latter, we propose a video-normalization scheme to improve the system developed by Zhao et al. We evaluated our system on the OuluVS database and achieved recognition rates more than 20% higher than the ones reported by Zhao et al. in both SD and SI testing scenarios.
Complementary countermeasures for detecting scenic face spoofing attacks

The face recognition community has finally started paying more attention to the long-neglected problem of spoofing attacks. The number of countermeasures is gradually increasing and fairly good results have been reported on the publicly available databases. There exists no superior antispoofing technique due to the varying nature of attack scenarios and acquisition conditions. Therefore, it is important to find out complementary countermeasures and study how they should be combined in order to construct an easily extensible anti-spoofing framework. In this paper, we address this issue by studying fusion of motion and texture based countermeasures under several types of scenic face attacks. We provide an intuitive way to explore the fusion potential of different visual cues and show that the performance of the individual methods can be vastly improved by performing fusion at score level. The Half-Total Error Rate (HTER) of the best individual countermeasure was decreased from 11.2% to 5.1% on the Replay Attack Database. More importantly, we question the idea of using complex classification schemes in individual countermeasures, since nearly same fusion performance is obtained by replacing them with a simple linear one. In this manner, the computational efficiency and also probably the generalization ability of the resulting anti-spoofing framework are increased.
Multi-Object Tracking Using Color, Texture and Motion

In this paper, we introduce a novel real-time tracker based on color, texture and motion information. RGB color histogram and correlogram (autocorrelogram) are exploited as color cues and texture properties are represented by local binary patterns (LBP). Object's motion is taken into account through location and trajectory. After extraction, these features are used to build a unifying distance measure. The measure is utilized in tracking and in the classification event, in which an object is leaving a group. The initial object detection is done by a texture-based background subtraction algorithm. The experiments on indoor and outdoor surveillance videos show that a unified system works better than the versions based on single features. It also copes well with low illumination conditions and low frame rates which are common in large scale surveillance systems.
Encoding Local Binary Patterns using the re-parametrization of the second order Gaussian jet

In object recognition a robust feature set is considered as an important component in almost all the approaches proposed in the literature. In facial analysis, one of the best known feature set is based in Local Binary Patterns (LBP) which extracts the information contained in the image using comparisons between pixels in a region, finally such comparisons are encoded in form of histogram. We argue that this kind of encoding is statistically non-stable and can lead to errors during the recognition process, specially in noisy and low-resolution images, where the information contained in the image is not enough to generate a statistically robust histogram. In this paper, we propose a new method to encode the Local Binary Patterns using an re-parametrization of the second local order Gaussian Jet which generates more robust and reliable histograms suitable for different facial analysis tasks. We show that our method can be used for recognizing micro-expressions with competitive performances on the Spontaneous Micro-expression Corpus (SMIC) and the YORK Deception Detection Test.
Dynamic Facial Expression Recognition With Atlas Construction and Sparse Representation

In this paper, a new dynamic facial expression recognition method is proposed. Dynamic facial expression recognition is formulated as a longitudinal groupwise registration problem. The main contributions of this method lie in the following aspects: 1) subject-specific facial feature movements of different expressions are described by a diffeomorphic growth model; 2) salient longitudinal facial expression atlas is built for each expression by a sparse groupwise image registration method, which can describe the overall facial feature changes among the whole population and can suppress the bias due to large intersubject facial variations; and 3) both the image appearance information in spatial domain and topological evolution information in temporal domain are used to guide recognition by a sparse representation method. The proposed framework has been extensively evaluated on five databases for different applications: the extended Cohn-Kanade, MMI, FERA, and AFEW databases for dynamic facial expression recognition, and UNBC-McMaster database for spontaneous pain expression monitoring. This framework is also compared with several state-of-the-art dynamic facial expression recognition methods. The experimental results demonstrate that the recognition rates of the new method are consistently higher than other methods under comparison.
Face recognition based on the appearance of local regions

Recently, we proposed a novel facial representation for face recognition based on the local binary pattern (LBP) features. We obtained excellent results when dividing the face images into several regions from which the LBP features are extracted and concatenated into an enhanced feature vector as a face descriptor. However, it was unclear whether the obtained results were due to the use of local regions (instead of a holistic approach) or to the discriminative power of LBP. In this work, we investigated this issue by adopting and comparing four different texture features when using the appearances of local regions. The experimental results clearly showed and confirmed the validity of using LBP for face description.
Rotation Invariant Local Binary Convolution Neural Networks

Convolutional neural networks (CNNs) have achieved unprecedented successes in computer vision fields, but they remain challenged by the problem about how to effectively process the orientation transformation of objects with fewer parameters. In this paper, we propose a new convolutional module, local binary orientation module (LBoM), which takes advantages of both local binary convolutional and active rotating filters to effectively deal with the rotation variations with fewer parameters. LBoM can be naturally inserted to popular CNN models and upgrade them to be rotation invariant local binary CNNs (RI-LBCNNs). RI-LBCNNs can be learned with off-the-shelf optimization approaches in an end-to-end manner and fulfill image classification tasks. Extensive experiments on four benchmarks show that RI-LBCNNs can perform image classification with fewer network parameters and significantly outperform the baseline LBCNN when processing images with large rotation variations.
A 3-D scene interpreter for indoor navigation

A 3D scene interpreter for visual navigation of a mobile robot is presented. The interpreter provides a partial understanding of scene contents. With the help of a priori knowledge and world constraints, it can be used for route planning and obstacle avoidance. The interpreter consists of data-driven low and intermediate-level stages and a model-driven high-level analysis stage. The high-level analysis interprets the scene contents by matching the description of the scene provided by the lower-level stages to semantic models stored in the database. Experimental results are presented to verify the performance of the interpreter.<<ETX>>
A Spontaneous Micro-expression Database: Inducement, collection and baseline

Micro-expressions are short, involuntary facial expressions which reveal hidden emotions. Micro-expressions are important for understanding humans' deceitful behavior. Psychologists have been studying them since the 1960's. Currently the attention is elevated in both academic fields and in media. However, while general facial expression recognition (FER) has been intensively studied for years in computer vision, little research has been done in automatically analyzing micro-expressions. The biggest obstacle to date has been the lack of a suitable database. In this paper we present a novel Spontaneous Micro-expression Database SMIC, which includes 164 micro-expression video clips elicited from 16 participants. Micro-expression detection and recognition performance are provided as baselines. SMIC provides sufficient source material for comprehensive testing of automatic systems for analyzing micro-expressions, which has not been possible with any previously published database.
Generalized face anti-spoofing by detecting pulse from face videos

Face biometric systems are vulnerable to spoofing attacks. Such attacks can be performed in many ways, including presenting a falsified image, video or 3D mask of a valid user. A widely used approach for differentiating genuine faces from fake ones has been to capture their inherent differences in (2D or 3D) texture using local descriptors. One limitation of these methods is that they may fail if an unseen attack type, e.g. a highly realistic 3D mask which resembles real skin texture, is used in spoofing. Here we propose a robust anti-spoofing method by detecting pulse from face videos. Based on the fact that a pulse signal exists in a real living face but not in any mask or print material, the method could be a generalized solution for face liveness detection. The proposed method is evaluated first on a 3D mask spoofing database 3DMAD to demonstrate its effectiveness in detecting 3D mask attacks. More importantly, our cross-database experiment with high quality REAL-F masks shows that the pulse based method is able to detect even the previously unseen mask type whereas texture based methods fail to generalize beyond the development data. Finally, we propose a robust cascade system combining two complementary attack-specific spoof detectors, i.e. utilize pulse detection against print attacks and color texture analysis against video attacks.
Experiments with texture classification using averages of local pattern matches

Laws has introduced a class of texture features based on average degrees of match of the pixel neighbourhoods with a set of standard masks. These features yield better texture classification than standard features based on pairs of pixels. Simplifications of these features are investigated. Their performance is not greatly affected by their exact form and also appears to remain the same if only local match maxima are used. An alternative definition of such features is also presented, based on sums and differences of Gaussian convolutions.
A document management interface utilizing page decomposition and content-based compression

Page decomposition is very important in order to achieve an efficient interface for electronic (re)usability of documents. This paper proposes an approach in which the page is first segmented and classified into background text and image areas. These areas are then manipulated separately with a dedicated compression method applied to each class and represented in a way which provides an easy and open interface; for example, to send, manage or archive selected or all parts of the document. We call this a document management interface (DMI).
Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns

Presents a theoretically very simple, yet efficient, multiresolution approach to gray-scale and rotation invariant texture classification based on local binary patterns and nonparametric discrimination of sample and prototype distributions. The method is based on recognizing that certain local binary patterns, termed "uniform," are fundamental properties of local image texture and their occurrence histogram is proven to be a very powerful texture feature. We derive a generalized gray-scale and rotation invariant operator presentation that allows for detecting the "uniform" patterns for any quantization of the angular space and for any spatial resolution and presents a method for combining multiple operators for multiresolution analysis. The proposed approach is very robust in terms of gray-scale variations since the operator is, by definition, invariant against any monotonic transformation of the gray scale. Another advantage is computational simplicity as the operator can be realized with a few operations in a small neighborhood and a lookup table. Experimental results demonstrate that good discrimination can be achieved with the occurrence statistics of simple rotation invariant local binary patterns.
Gait Recognition Using Fractal Scale and Wavelet Moments

Video-based gait recognition is a challenging problem in computer vision. In this paper, fractal scale wavelet analysis is applied to describe and automatically recognize gait. Fractal scale based on wavelet analysis represents the self-similarity of signals, and improves the flexibility of wavelet moments. Optimal wavelets based on generalized multi-resolution analysis are used to improve the recognition rate. Descriptors of fractal scale are translation, scale and rotation invariant. Moreover, a combination of fractal scale and wavelet moments improves the recognition rate. Experiments show that the proposed descriptor is efficient for gait recognition
Automatic ground-truth generation for skew-tolerance evaluation of document layout analysis methods

Generation of ground-truths is of great importance for unbiased performance evaluation of document layout analysis methods. This is especially necessary because many methods are claimed to be skew-tolerant. However, experimental evaluation of this fact is often based only on human subjective judgement and restricted to a few experiments. The main obstacle for obtaining human-independent and more automated performance evaluation is that usually there are only ground-truths for upright images, i.e., images with no skew of text lines, because currently available ground-truthing techniques are too time-consuming. We propose a methodology of automatic generation of ground-truths for skewed images by using the ground-truths available for upright images. This methodology is simple and quite fast because processing is done at the level of small square blocks, but not at pixel level.
Robust texture classification by subsets of local binary patterns

Recently, a nonparametric approach to texture analysis has been developed, in which the distributions of simple texture measures based on local binary patterns (LBP) are used for texture description. The basic LBP encodes 256 simple feature detectors in a single 3/spl times/3 operator. This paper shows that a properly selected subset of patterns encoded in LBP forms an efficient and robust texture description which can achieve better classification rates in comparison with the whole LBP histogram. Experiments on classification of textures from the Columbia-Utrecht (CURET) database demonstrate the robustness of the approach.
Pose Estimation via Complex-Frequency Domain Analysis of Image Gradient Orientations

Head Pose Estimation (HPE) has recently attracted a lot of interests in various computer vision applications. One challenging problem for accurate HPE is to model the intrinsic variations among poses, and suppress the extraneous variations derived from other factors, such as the illumination changes, outliers, and noise. To this end, this paper proposes a simple and efficient facial description for head pose estimation from images. To handle the illumination changes, we characterize each image pixel by its image gradient orientation (IGO), rather than the intensity, which is sensitive to illumination changes. We then carry out complex-frequency domain analysis of the IGO image via the two-dimensional image transform, such as the 2D Discrete Cosine Transform (DCT2), to encode the spatial configuration of image gradient orientations. The proposed facial description is called IGO-DCT2. It is robust to illumination changes, outliers, and noise. In addition, it is learning free and computationally efficient. Finally, the fine-grain head pose estimation is regarded as a regression problem and off-the-shelf non-linear regression models are used to learn the mapping from the feature space to the continuous pose labels. Experimental results show the proposed facial description achieves highly competitive results on the publicly available FacePix dataset.
Robust Facial Expression Recognition Using Revised Canonical Correlation

The poor alignment and large variations in the temporal sale of facial expressions are two crucial problems for facial expression recognition (FER). Canonical correlation (CC) has recently received increasing attention in surveillance face recognition because of its robustness to variations of alignment. But it could not suit well to FER, because the facial expression variations and temporal information are ignored for canonical subspace. This paper proposes the revised canonical correlation method to address the two above described issues for making FER be robust to false detection or mis-alignment. Firstly, this paper presents the local binary pattern to describe the appearance features for enhancing the spatial variations of facial expression. Secondly, this paper proposes the temporal orthogonal locality preserved projection for building a canonical subspace of a video clip, where it mostly captures the motion changes of facial expressions. Then this paper presents the discriminative CC to model the low-dimensional feature space, which increases robustness to imprecise alignment and strengthens discrimination for facial expressions. Extensive experimental results on Extended Cohn-Kanade and MAHNOB-HCI databases demonstrate that the proposed method achieves the best results in recognizing facial expressions and performs robustly with ordinary on general face detection and eye detection.
A Compact Representation of Visual Speech Data Using Latent Variables

The problem of visual speech recognition involves the decoding of the video dynamics of a talking mouth in a high-dimensional visual space. In this paper, we propose a generative latent variable model to provide a compact representation of visual speech data. The model uses latent variables to separately represent the inter-speaker variations of visual appearances and those caused by uttering, and incorporates the structural information of the observed visual data within an utterance through modelling the structure using a path graph and placing variables' priors along its embedded curve.