nils-herrmann/clean_scirepeval huggingface dataset:
Outex - new framework for empirical evaluation of texture analysis algorithms

This paper presents the current status of a new initiative aimed at developing a versatile framework and image database for empirical evaluation of texture analysis algorithms. The proposed Outex framework contains a large collection of surface textures captured under different conditions, which facilitates construction of a wide range of texture analysis problems. The problems are encapsulated into test suites, for which baseline results obtained with algorithms from literature are provided. The rich functionality of the framework is demonstrated with examples in texture classification, segmentation and retrieval. The framework has a web site for public dissemination of the database and comparative results obtained by research groups world wide.
Face spoofing detection from single images using texture and local shape analysis

Current face biometric systems are vulnerable to spoofing attacks. A spoofing attack occurs when a person tries to masquerade as someone else by falsifying data and thereby gaining illegitimate access. Inspired by image quality assessment, characterisation of printing artefacts and differences in light reflection, the authors propose to approach the problem of spoofing detection from texture analysis point of view. Indeed, face prints usually contain printing quality defects that can be well detected using texture and local shape features. Hence, the authors present a novel approach based on analysing facial image for detecting whether there is a live person in front of the camera or a face print. The proposed approach analyses the texture and gradient structures of the facial images using a set of low-level feature descriptors, fast linear classification scheme and score level fusion. Compared to many previous works, the authors proposed approach is robust and does not require user-cooperation. In addition, the texture features that are used for spoofing detection can also be used for face recognition. This provides a unique feature space for coupling spoofing detection and face recognition. Extensive experimental analysis on three publicly available databases showed excellent results compared to existing works.
Edge-based texture measures for surface inspection

Pietikainen and Rosenfeld (1982) introduced a class of texture measures based on first-order statistics derived from edges in an image. the objective of this paper is to evaluate the performance of these measures and some new edge-based texture measures using two different types of data sets: images taken from the Brodatz album and images from a practical wood surface inspection problem. The results obtained for edge-based measures are compared to those obtained by popular second-order texture measures and tonal features. The role of the classifier on the performance is also studied by comparing the results obtained for three parametric classifiers and for a nonparametric k-nearest neighbor classifier. The results indicate that edge-based approaches are very promising for surface inspection problems, because they are relatively simple to compute and have performed very well in experiments.<<ETX>>
CS-3DLBP and geometry based person independent 3D facial action unit detection

Face is the key component in understanding emotions which play significant roles in many areas from security and entertainment to psychology and education. In this paper, we propose a method to detect facial action units in 3D face data by combining novel geometric properties and a new descriptor based on the Local Binary Pattern (LBP) methodology. The proposed method enables person and gender independent facial action unit detection. The decision level fusion is used by employing the Random Forests classifiers to combine geometric and LBP based features. Unlike the previous methods which suffer from the diversity among different persons and normalize features utilizing neutral faces, our method extracts features on a single 3D face data. Besides, we show that orientation based 3D LBP descriptor can be implemented efficiently in terms of size and time without degrading the performance. We tested our method on the Bosphorus database and present comparative results with the existing methods. Our results outperform those of existing methods, achieving a mean receiver operating characteristic area under curve of 97.7%.
Color-based face detection using skin locus model and hierarchical filtering

Introduces an architecture for face detection in color images. Based on skin locus and successive detectors, the method allows high efficiency, under drastically varying illumination conditions. While most color-based approaches work only if the camera was calibrated and detect faces if the features (eyes, mouth, eyebrows...) are visible and detectable, our system detects faces even when these features are not well defined. The detection is based on a robust modeling of skin color, called skin locus, which is used to extract the skin-like regions. After orientation normalization and based on verifying a set of criteria (face symmetry, presence of some facial features, variance of pixel intensities and connected component arrangement), only facial regions are selected. We show that our system can detect faces and deal with different conditions (size, orientation, illumination, and complex background).
Face spoofing detection from single images using micro-texture analysis

Current face biometric systems are vulnerable to spoofing attacks. A spoofing attack occurs when a person tries to masquerade as someone else by falsifying data and thereby gaining illegitimate access. Inspired by image quality assessment, characterization of printing artifacts, and differences in light reflection, we propose to approach the problem of spoofing detection from texture analysis point of view. Indeed, face prints usually contain printing quality defects that can be well detected using texture features. Hence, we present a novel approach based on analyzing facial image textures for detecting whether there is a live person in front of the camera or a face print. The proposed approach analyzes the texture of the facial images using multi-scale local binary patterns (LBP). Compared to many previous works, our proposed approach is robust, computationally fast and does not require user-cooperation. In addition, the texture features that are used for spoofing detection can also be used for face recognition. This provides a unique feature space for coupling spoofing detection and face recognition. Extensive experimental analysis on a publicly available database showed excellent results compared to existing works.
Selecting models from videos for appearance-based face recognition

We propose an unsupervised approach to select representative face samples (models) from raw videos and build an appearance-based face recognition system. The approach is based on representing the face manifold in a low-dimensional space using the locally linear embedding (LLE) algorithm and then performing K-means clustering. We define the face models as the cluster centers. Our strategy is motivated by the efficiency of LLE to recover meaningful low-dimensional structures hidden in complex and high dimensional data such as face images. Two other well-known unsupervised learning algorithms (Isomap and SOM) are also considered. We compare and assess the efficiency of these different schemes on the CMU MoBo database which contains 96 face sequences of 24 subjects. The results clearly show significant performance enhancements over traditional methods such as the PCA-based one.
Combining motion and appearance for gender classification from video sequences

We investigate whether combining appearance (face structure) and motion (the way a person is talking and moving his/her facial features) boosts gender classification from face sequences. We propose and compare different schemes based on appearance only, motion only, and combination of appearance and motion. Experiments on various face video datasets of persons uttering phrases or expressing emotions show that combination of motion and appearance is useful for gender analysis of familiar faces, yielding in classification accuracy of 100%. However, for unfamiliar faces, motion seems to not provide additional discriminative information as the best performance (96.3%) is obtained using an appearance based approach with Local Binary Pattern (LBP) features and Support Vector Machines (SVMs).
An In-depth Examination of Local Binary Descriptors in Unconstrained Face Recognition

Automatic face recognition in unconstrained conditions is a difficult task which has recently attained increasing attention. In this domain, face verification methods have significantly improved since the release of the Labeled Faces in the Wild database, but the related problem of face identification, is still lacking considerations, which is partly because of the shortage of representative databases. Only recently, two new datasets called Remote Face and Point-and-Shoot Challenge were published providing appropriate benchmarks for the research community to investigate the problem of face recognition in challenging imaging conditions, in both, verification and identification modes. In this paper we provide an in-depth examination of three local binary description methods in unconstrained face recognition evaluating them on these two recently published datasets. In detail, we investigate three well established methods separately and fusing them at rank- and score-levels. We are using a well-defined evaluation protocol allowing a fair comparison of our results for future examinations.
Unsupervised learning using locally linear embedding: experiments with face pose analysis

This paper considers a recently proposed method for unsupervised learning and dimensionality reduction, locally linear embedding (LLE). LLE computes a compact representation of high-dimensional data combining the major advantages of linear methods (computational efficiency, global optimality, and flexible asymptotic convergence guarantees) with the advantages of non-linear approaches (flexibility to learn a broad class on non-linear manifolds). We assess the performance of the LLE algorithm on real-world data (face images in different poses) and compare the results with those obtained with two different approaches (PCA and SOM). Extensions to the original LLE algorithm are proposed and applied to the problem of pose estimation.
Unsupervised dynamic texture segmentation using local spatiotemporal descriptors

Dynamic texture (DT) is an extension of texture to the temporal domain. In this paper, we address the problem of segmenting DT into disjoint regions in an unsupervised way. Each region is characterized by histograms of local binary patterns and contrast in a spatiotemporal mode. It combines the motion and appearance of DT together. Experimental results show that our method is effective in segmenting regions that differ in their dynamics.
Detection of skin color under changing illumination: a comparative study

Faces and hands recorded under natural environments are frequently subject to illumination variations which affect their color appearance. This is a problem when the color cue is used to detect skin candidates at pixel level. Traditionally, color constancy has been suggested for correction, but after a lot of effort no good solution suitable for machine vision has emerged. However, many approaches have been proposed for general skin detection, but they are typically tested under mild changes in illumination chromaticity or do not define the variation range. This makes it difficult to evaluate their applicability for objects under varying illumination. The paper compares four state-of-the-art skin detection schemes under realistic conditions with drastic chromaticity change.
Dynamic texture synthesis using a spatial temporal descriptor

Dynamic textures are image sequences with visual pattern repetition in time and space, such as smoke, flames, moving objects and so on. Dynamic texture synthesis is to provide a continuous and infinitely varying stream of images by doing operations on dynamic textures. Considering that the previous video texture method provides high-quality visual results, but its representation does not well explore the temporal correlation among frames, we develop a novel spatial temporal descriptor for frame description accompanied with a similarity measure on the basis of the video texture method. Compared with the previous one, our method considers both the spatial and temporal domains of video sequences in representation; moreover, combines the local and global description on each spatial-temporal plane. From experimental results, the proposed method achieves better performance in both the syntheses of natural scene and human motion. Especially, it has the characteristic to be robust to noise in remodeling videos into infinite time domain.
Towards Reading Hidden Emotions: A Comparative Study of Spontaneous Micro-Expression Spotting and Recognition Methods

Micro-expressions (MEs) are rapid, involuntary facial expressions which reveal emotions that people do not intend to show. Studying MEs is valuable as recognizing them has many important applications, particularly in forensic science and psychotherapy. However, analyzing spontaneous MEs is very challenging due to their short duration and low intensity. Automatic ME analysis includes two tasks: ME spotting and ME recognition. For ME spotting, previous studies have focused on posed rather than spontaneous videos. For ME recognition, the performance of previous studies is low. To address these challenges, we make the following contributions: (i) We propose the first method for spotting spontaneous MEs in long videos (by exploiting feature difference contrast). This method is training free and works on arbitrary unseen videos. (ii) We present an advanced ME recognition framework, which outperforms previous work by a large margin on two challenging spontaneous ME databases (SMIC and CASMEII). (iii) We propose the first automatic ME analysis system (MESR), which can spot and recognize MEs from spontaneous video data. Finally, we show our method outperforms humans in the ME recognition task by a large margin, and achieves comparable performance to humans at the very challenging task of spotting and then recognizing spontaneous MEs.
Facial 3D Shape Estimation from Images for Visual Speech Animation

In this paper we describe the first version of our system for estimating 3D shape sequences from images of the frontal face. This approach is developed with 3D Visual Speech Animation (VSA) as the target application. In particular, the focus is on the usability of an existing state-of-the-art image-based VSA system and subsequent on-line estimation of the corresponding 3D facial shape sequence from its output. This has the added advantage of a 3D visual speech, which is mainly render ability of the face in different poses and illumination conditions. The idea is based on the detection of landmarks from the facial image which are then used to determine the pose and shape. The method belongs to the category of methods which use a prior 3D Morph able Models (3D-MM) trained using 3D facial data. For the time being it is developed for a person-specific domain, i.e. the 3D-MM and the 2D facial landmark detector are trained using the data of a single person and tested with the same person-specific data.
An improved local descriptor and threshold learning for unsupervised dynamic texture segmentation

Dynamic texture (DT) is an extension of texture to the temporal domain. How to segment DTs is a challenging problem. In this paper, we propose significant improvements to a recently published DT segmentation method. We employ a new spatiotemporal local texture descriptor which combines local binary patterns with a differential excitation measure. We also address the important problem of threshold selection by proposing a method for determining thresholds for the segmentation method by statistical learning. An improved criterion for merging adjacent regions is also introduced. Experimental results show that our approach provides very good segmentation results compared to state-of-the-art methods.
Lipreading: A Graph Embedding Approach

In this paper, we propose a novel graph embedding method for the problem of lipreading. To characterize the temporal connections among video frames of the same utterance, a new distance metric is defined on a pair of frames and graphs are constructed to represent the video dynamics based on the distances between frames. Audio information is used to assist in calculating such distances. For each utterance, a subspace of the visual feature space is learned from a well-defined intrinsic and penalty graph within a graph-embedding framework. Video dynamics are found to be well preserved along some dimensions of the subspace. Discriminatory cues are then decoded from curves of the projected visual features to classify different utterances.
Automatic Dynamic Texture Segmentation Using Local Descriptors and Optical Flow

A dynamic texture (DT) is an extension of the texture to the temporal domain. How to segment a DT is a challenging problem. In this paper, we address the problem of segmenting a DT into disjoint regions. A DT might be different from its spatial mode (i.e., appearance) and/or temporal mode (i.e., motion field). To this end, we develop a framework based on the appearance and motion modes. For the appearance mode, we use a new local spatial texture descriptor to describe the spatial mode of the DT; for the motion mode, we use the optical flow and the local temporal texture descriptor to represent the temporal variations of the DT. In addition, for the optical flow, we use the histogram of oriented optical flow (HOOF) to organize them. To compute the distance between two HOOFs, we develop a simple effective and efficient distance measure based on Weber's law. Furthermore, we also address the problem of threshold selection by proposing a method for determining thresholds for the segmentation method by an offline supervised statistical learning. The experimental results show that our method provides very good segmentation results compared to the state-of-the-art methods in segmenting regions that differ in their dynamics.
Combining sparse and dense descriptors with temporal semantic structures for robust human action recognition

Automatic categorization of human actions in the real world is very challenging due to the great intra-class differences. In this paper, we present a new method for robust recognition of human actions. We first cluster each video in the training set into temporal semantic segments by a dense descriptor. Each segment in the training set is represented by a concatenated histogram of sparse and dense descriptors. These histograms of segments are used to train a classifier. In the recognition stage, a query video is also divided into temporal semantic segments by clustering. Each segment will obtain a confidence evaluated by the trained classifier. Combining the confidence of each segment, we classify this query video. To evaluate our approach, we perform experiments on two challenging datasets, i.e., the Olympic Sports Dataset (OSD) and Hollywood Human Action dataset (HOHA). We also test our method on the benchmark KTH human action dataset. Experimental results confirm that our algorithm performs better than the state-of-the-art methods.
Lipreading With Local Spatiotemporal Descriptors

Visual speech information plays an important role in lipreading under noisy conditions or for listeners with a hearing impairment. In this paper, we present local spatiotemporal descriptors to represent and recognize spoken isolated phrases based solely on visual input. Spatiotemporal local binary patterns extracted from mouth regions are used for describing isolated phrase sequences. In our experiments with 817 sequences from ten phrases and 20 speakers, promising accuracies of 62% and 70% were obtained in speaker-independent and speaker-dependent recognition, respectively. In comparison with other methods on AVLetters database, the accuracy, 62.8%, of our method clearly outperforms the others. Analysis of the confusion matrix for 26 English letters shows the good clustering characteristics of visemes for the proposed descriptors. The advantages of our approach include local processing and robustness to monotonic gray-scale changes. Moreover, no error prone segmentation of moving lips is needed.
Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework

In this paper we propose the first method known to the authors that successfully differentiates spontaneous from posed facial expressions using a realistic training corpus. We propose a new spatiotemporal local texture descriptor (CLBP-TOP) that outperforms other descriptors. We demonstrate that our temporal interpolation and visual/near-infrared fusion methods improve the differentiation performance. Finally, we propose a new generic facial expression recognition framework that subdivides the facial expression recognition problem into a cascade of smaller tasks that are simpler to tackle. The system is the first to differentiate spontaneous from posed facial expressions with a realistic corpus and achieves promising results.
Robust skew estimation on low-resolution document images

Presents a new technique for estimating an arbitrary skew angle using a simple and efficient text row accumulation based on first- and second-order statistics. In contrast to other works, where an image resolution of 100-300 dpi is necessary, a lower resolution (50 dpi) is enough for our method to obtain a correct result. The performance of the method was evaluated on the database UW-1 of the University of Washington and compared with an advanced Hough transform-based method. A significantly lower error rate for our method confirms its advantages and benefits for automated document processing.
A hybrid computer architecture for machine vision

A hybrid computer architecture for machine vision which combines the useful properties of different types of architectures is introduced. HYBRID, an experimental hybrid system consisting of specialized Datacube-compatible processors and a transputer network, has been developed in a Sun-3 environment. The VLSI implementation of an edge-preserving smoothing operator for the low-level vision system is described, and the performance of transputer-based systems for higher-level vision is evaluated. Methods for analyzing and optimizing the performance of a hybrid architecture are discussed.<<ETX>>
Face and Eye Detection for Person Authentication in Mobile Phones

Computer vision applications for mobile phones are gaining increasing attention due to several practical needs resulting from the popularity of digital cameras in today's mobile phones. In this work, we consider the task of face detection and authentication in mobile phones and experimentally analyze a face authentication scheme using Haar-like features with Ad-aBoost for face and eye detection, and local binary pattern (LBP) approach for face authentication. For comparison, another approach to face detection using skin color for fast processing is also considered and implemented. Despite the limited CPU and memory capabilities of today's mobile phones, our experimental results show good face detection performance and average authentication rates of 82% for small-sized faces (40times40 pixels) and 96% for faces of 80times80 pixels. The system is running at 2 frames per second for images of 320times240 pixels. The obtained results are very promising and assess the feasibility of face authentication in mobile phones. Directions for further enhancing the performance of the system are also discussed.