nils-herrmann/clean_scirepeval huggingface dataset:
Isomorphic Manifold Inference for hair segmentation

Hair segmentation is challenging due to the diverse appearance, irregular region boundary and the influence of complex background. To deal with this problem, we propose a novel method, named Isomorphic Manifold Inference (IMI). Given a head-shoulder image, a Coarse Hair Probability Map (Coarse HPM), each element of which represents the probability of the pixel being hair, is initially calculated by exploring hair location and color priors. Then, based on an observation that similar Coarse HPMs imply similar segmentations, we formulate Coarse HPM and corresponding ground segmentation (Optimal HPM) as a pair of isomorphic manifolds. Under this formulation, final hair segmentation is inferred from the Coarse HPM with regression techniques. In this way, the IMI implicitly exploits the hair-specific prior embodied in the training set. Extensive experimental comparisons are conducted and the results strongly encourage the method. The generality of IMI to other class-specific image segmentation is also discussed.
Relative Forest for Visual Attribute Prediction

Accurate prediction of the visual attributes is significant in various recognition tasks. For many visual attributes, while it is very difficult to describe the exact degrees of their presences, by comparing the pairs of samples, the relative ordering of presences may be easily figured out. Based on this observation, instead of considering such attribute as binary attribute, the relative attribute method learns a ranking function for each attribute to provide more accurate and informative prediction results. In this paper, we also explore pairwise ranking for visual attribute prediction and propose to improve the relative attribute method in two aspects. First, we propose a relative tree method, which can achieve more accurate ranking in case of nonlinearly distributed visual data. Second, by resorting to randomization and ensemble learning, the relative tree method is extended to the relative forest method to further boost the accuracy and simultaneously reduce the computational cost. To validate the effectiveness of the proposed methods, we conduct extensive experiments on four databases: PubFig, OSR, FGNET, and WebFace. The results show that the proposed relative forest method not only outperforms the original relative attribute method, but also achieve the state-of-the-art accuracy for ordinal visual attribute prediction.
Unified Principal Component Analysis with generalized Covariance Matrix for face recognition

Recently, 2DPCA and its variants have attracted much attention in face recognition area. In this paper, some efforts are made to discover the underlying fundaments of these methods, and a novel framework called unified principal component analysis (UPCA) is proposed. First, we introduce a novel concept, named generalized covariance matrix (GCM), which is naturally derived from the traditional covariance matrix (CM). Each element of GCM is a generalized covariance of two random vectors rather than two scalar variables in CM. Based on GCM, the UPCA framework is proposed, from which the traditional PCA and its 2D counterparts can be deduced as special cases. Furthermore, under the UPCA framework, we not only revisit the existing 2D PCA methods and their limitations, but also propose two new methods: the grid-sampling method (GridPCA) and the intra-group correlation reduction method. Extensive experimental results on the FERET face database support the theoretical analysis and validate the feasibility of the proposed methods.
Directional Binary Pattern (DBP): A novel object representation approach for gaze estimation

This paper presents a novel object descriptor, Directional Binary Pattern (DBP), for robust gaze estimation. In DBP, the local directional derivations are proposed to encode the binary patterns in the given orientations. As an object descriptor, DBP has many advantages: noise restrain, robustness to illumination, contrast enhancement on the boundary, and local texture encoded as the directional information. DBP features of eyes are finally fed into Support Vector Regression (SVR) to match the gaze mapping function, which is then used to predict the gaze direction with respect to the camera coordinate system. In our experiments, an eye gaze dataset includes 4089 training samples of 11 persons. Experimental results show that our method can achieve an accuracy of less than 2°.
Temporally multiple dynamic textures synthesis using piecewise linear dynamic systems

Real-world nonlinear dynamic textures (DTs) usually consist of temporally multiple linear DTs which cannot be correctly modeled by previous works. In this paper, we propose piecewise linear dynamic systems (PLDS) to model temporally multiple DTs. PLDS simultaneously decides the temporal segmentation, models each DT segment with an LDS and the whole DT by switching between the LDS'. Experimental results verify that PLDS can capture the stochastic and dynamic nature of temporally multiple DTs and it synthesizes nonlinear DTs without decay or divergence. An EM-like algorithm iterating between sequence division and LDS' fitting is adopted to learn the model parameters.
Local Regression Model for Automatic Face Sketch Generation

As one of the important artistic styles of portrait, sketch portrait has wide applications for both digital entertainment and law enforcement. In this paper, an automatic face sketch generation approach is presented by learning from photo-sketch pair examples. Specifically, the relationship between a face photo and its corresponding face sketch is learned on image patch level. By applying this relationship to the input face photo patch, we can infer the output face sketch patch by exploiting some regression techniques such as kNN, the Lasso and so on. Via our local regression model, we can synthesize an appealing sketch portrait from a given face photo in a few minutes. Experiments conducted on CUHK database have shown that our results are more compelling than previous methods especially in two respects: (1) our synthesized sketches preserve more identity information of the original face photo, (2) our synthesized sketches presents more pencil sketch texture.
Bi-Shifting Auto-Encoder for Unsupervised Domain Adaptation

In many real-world applications, the domain of model learning (referred as source domain) is usually inconsistent with or even different from the domain of testing (referred as target domain), which makes the learnt model degenerate in target domain, i.e., the test domain. To alleviate the discrepancy between source and target domains, we propose a domain adaptation method, named as Bi-shifting Auto-Encoder network (BAE). The proposed BAE attempts to shift source domain samples to target domain, and also shift the target domain samples to source domain. The non-linear transformation of BAE ensures the feasibility of shifting between domains, and the distribution consistency between the shifted domain and the desirable domain is constrained by sparse reconstruction between them. As a result, the shifted source domain is supervised and follows similar distribution as target domain. Therefore, any supervised method can be applied on the shifted source domain to train a classifier for classification in target domain. The proposed method is evaluated on three domain adaptation scenarios of face recognition, i.e., domain adaptation across view angle, ethnicity, and imaging sensor, and the promising results demonstrate that our proposed BAE can shift samples between domains and thus effectively deal with the domain discrepancy.
Unsupervised person re-identification with locality-constrained Earth Mover's distance

The difficult acquisition of labeled data and the misalignment of local matching are major obstacles to apply person re-identification in real scenarios. To alleviate these problems, we propose an unsupervised method, called locality-constrained Earth Mover's Distance (LC-EMD), to learn the optimal measure between image pairs. Specifically, Gaussian mixture models (GMMs) are learned as signatures. By imposing locality constraints, LC-EMD can naturally achieve partial matching between Gaussian components. Moreover, LC-EMD has the analytical solution which can be efficiently computed. Experiments on two public datasets demonstrate LC-EMD is robust to misalignment and performs better than other unsupervised methods.
Deep Multi-Task Learning for Joint Prediction of Heterogeneous Face Attributes

Face attribute prediction has important applications in video surveillance, face retrieval, and social media. While a number of methods have been proposed for face attribute prediction, most of them did not explicitly consider the attribute correlation and heterogeneity during feature learning. In this paper, we propose a Deep Multi-Task Learning (DMTL) network to jointly learn multiple models; each addresses the prediction of one category of homogenous attributes. Specifically, we group the heterogeneous face attributes into two categories (i.e., nominal and ordinal), and design corresponding prediction models. At the same time, we use a convolutional neural network (CNN) for early stage feature learning, which is shared by all the attributes. Experiments on the publicdomain MORPH II, CelebA, and LFWA databases show that the proposed approach outperforms the state of the art in joint face attribute prediction, and has good generalization ability.
Hierarchical hybrid statistic based video binary code and its application to face retrieval in TV-series

We address the problem of video face retrieval in TV-Series, which searches video clips based on the presence of particular character, given one video clip of his/hers. This is tremendously challenging because on one hand, faces in TV-Series are captured in largely uncontrolled conditions with complex appearance variations, and on the other hand retrieval task typically needs highly efficient representation with low time and space complexity. To handle such problems, we propose a compact and discriminative binary representation for the huge body of video data based on a novel hierarchical hybrid statistic. Our method, named Hierarchical Hybrid Statistic based Video Binary Code (HHSVBC), first utilizes different parameterized Fisher Vectors (FVs) as frame representation that can encode multi-granularity low level variation information within the frame, and then models the video by its frame covariance matrix to capture high level variation information among video frames. To incorporate discriminative information and obtain more compact video signature, the high-dimensional video representation is further encoded to a much lower-dimensional binary vector, which finally yields the proposed HHSVBC. Specifically, each bit of the code, is produced via supervised learning in a max margin framework, which aims to make a trade-off between code discriminability and stability. Face retrieval experiments on two challenging large scale TV-Series video databases demonstrate the competitiveness of the proposed HHSVBC over state-of-the-art retrieval methods.
Efficient human action detection: a coarse-to-fine strategy

This paper proposes a coarse-to-fine strategy to detect human actions in the realistic videos given a single example of such action. The proposed method is learning-free and doesn't require any prior knowledge. Input video is separated into a batch of spatio-temporal volumes based on chi-square distance measure of the volumetric features and further identified by contextual motion information. Instead of the exhaustive search, query action is localized by matching local salient geometric features only between itself and the pruned spatio-temporal volumes. The competitive results obtained from the evaluation on a collection of challenging action data indicate the effectiveness and the computational efficiency of our method.
Multi-View Discriminant Analysis

In many computer vision systems, the same object can be observed at varying viewpoints or even by different sensors, which brings in the challenging demand for recognizing objects from distinct even heterogeneous views. In this work we propose a Multi-view Discriminant Analysis (MvDA) approach, which seeks for a single discriminant common space for multiple views in a non-pairwise manner by jointly learning multiple view-specific linear transforms. Specifically, our MvDA is formulated to jointly solve the multiple linear transforms by optimizing a generalized Rayleigh quotient, i.e., maximizing the between-class variations and minimizing the within-class variations from both intra-view and inter-view in the common space. By reformulating this problem as a ratio trace problem, the multiple linear transforms are achieved analytically and simultaneously through generalized eigenvalue decomposition. Furthermore, inspired by the observation that different views share similar data structures, a constraint is introduced to enforce the view-consistency of the multiple linear transforms. The proposed method is evaluated on three tasks: face recognition across pose, photo versus. sketch face recognition, and visual light image versus near infrared image face recognition on Multi-PIE, CUFSF and HFB databases respectively. Extensive experiments show that our MvDA achieves significant improvements compared with the best known results.
Learning Expressionlets on Spatio-temporal Manifold for Dynamic Facial Expression Recognition

Facial expression is temporally dynamic event which can be decomposed into a set of muscle motions occurring in different facial regions over various time intervals. For dynamic expression recognition, two key issues, temporal alignment and semantics-aware dynamic representation, must be taken into account. In this paper, we attempt to solve both problems via manifold modeling of videos based on a novel mid-level representation, i.e. expressionlet. Specifically, our method contains three key components: 1) each expression video clip is modeled as a spatio-temporal manifold (STM) formed by dense low-level features, 2) a Universal Manifold Model (UMM) is learned over all low-level features and represented as a set of local ST modes to statistically unify all the STMs. 3) the local modes on each STM can be instantiated by fitting to UMM, and the corresponding expressionlet is constructed by modeling the variations in each local ST mode. With above strategy, expression videos are naturally aligned both spatially and temporally. To enhance the discriminative power, the expressionlet-based STM representation is further processed with discriminant embedding. Our method is evaluated on four public expression databases, CK+, MMI, Oulu-CASIA, and AFEW. In all cases, our method reports results better than the known state-of-the-art.
Human facial expression recognition based on learning subspace method

The learning subspace method (LSM) is one of the most important methods for pattern recognition and it has been successfully used in many practical applications. We propose to use the LSM for human facial expression recognition. Seven expression subspaces are built for expression models. The idea of recognizing facial expression through a single static image is realized and the recognition rate as high as 89.5% is achieved. In order to make these expression subspaces more adaptive we can gradually learn them by using the averaged learning subspace method (ALSM). Experimental results also indicate that the recognition rate is over 90%. The dynamic characteristics of the projection vector sequence on these facial expression subspaces are also discussed.
Fusing Local Patterns of Gabor Magnitude and Phase for Face Recognition

Gabor features have been known to be effective for face recognition. However, only a few approaches utilize phase feature and they usually perform worse than those using magnitude feature. To investigate the potential of Gabor phase and its fusion with magnitude for face recognition, in this paper, we first propose local Gabor XOR patterns (LGXP), which encodes the Gabor phase by using the local XOR pattern (LXP) operator. Then, we introduce block-based Fisher's linear discriminant (BFLD) to reduce the dimensionality of the proposed descriptor and at the same time enhance its discriminative power. Finally, by using BFLD, we fuse local patterns of Gabor magnitude and phase for face recognition. We evaluate our approach on FERET and FRGC 2.0 databases. In particular, we perform comparative experimental studies of different local Gabor patterns. We also make a detailed comparison of their combinations with BFLD, as well as the fusion of different descriptors by using BFLD. Extensive experimental results verify the effectiveness of our LGXP descriptor and also show that our fusion approach outperforms most of the state-of-the-art approaches.
Spatiotemporal Local Monogenic Binary Patterns for Facial Expression Recognition

Feature representation is an important research topic in facial expression recognition from video sequences. In this letter, we propose to use spatiotemporal monogenic binary patterns to describe both appearance and motion information of the dynamic sequences. Firstly, we use monogenic signals analysis to extract the magnitude, the real picture and the imaginary picture of the orientation of each frame, since the magnitude can provide much appearance information and the orientation can provide complementary information. Secondly, the phase-quadrant encoding method and the local bit exclusive operator are utilized to encode the real and imaginary pictures from orientation in three orthogonal planes, and the local binary pattern operator is used to capture the texture and motion information from the magnitude through three orthogonal planes. Finally, both concatenation method and multiple kernel learning method are respectively exploited to handle the feature fusion. The experimental results on the Extended Cohn-Kanade and Oulu-CASIA facial expression databases demonstrate that the proposed methods perform better than the state-of-the-art methods, and are robust to illumination variations.
Thorax disease diagnosis using deep convolutional neural network.

Computer aided diagnosis (CAD) is an important issue, which can significantly improve the efficiency of doctors. In this paper, we propose a deep convolutional neural network (CNN) based method for thorax disease diagnosis. We firstly align the images by matching the interest points between the images, and then enlarge the dataset by using Gaussian scale space theory. After that we use the enlarged dataset to train a deep CNN model and apply the obtained model for the diagnosis of new test data. Our experimental results show our method achieves very promising results.
Unsupervised learning of overcomplete face descriptors

The current state-of-the-art indicates that a very discriminative unsupervised face representation can be constructed by encoding overlapping multi-scale face image patches at facial landmarks. If fixed as such, there are even suggestions (albeit subtle) that the underlying features may no longer have as much meaning. In spite of the effectiveness of this strategy, we argue that one may still afford to improve especially at the feature level. In this paper, we investigate the role of overcompleteness in features for building unsupervised face representations. In our approach, we first learn an overcomplete basis from a set of sampled face image patches. Then, we use this basis to produce features that are further encoded using the Bag-of-Features (BoF) approach. Using our method, without an extensive use of facial landmarks, one is able to construct a single-scale representation reaching state-of-the-art performance in face recognition and age estimation following the protocols of LFW, FERET, and Adience benchmarks. Furthermore, we make several interesting findings related, for example, to the positive impact of applying soft feature encoding scheme preceding standard dimensionality reduction. To this end, making the encoding faster, we propose a novel method for approximative soft-assignment which we show to perform better than its hard-assigned counterpart.
Robust text detection from binarized document images

Many document images are rich in color and have complex background. To detect text from them, a standard approach utilizes both color and binary information. This often leads to time-consuming processing and requires a lot of parameters to be tuned. In contrast, we propose a new method for text detection using a binary image alone. The main virtues of our method include detection of both normal and inverted text and robustness to various font types, styles and sizes and small skew angles, combined with a moderate number of free parameters.
Generalized textured contact lens detection by extracting BSIF description from Cartesian iris images

Textured contact lenses cause severe problems for iris biometric systems because they can be used to alter the appearance of iris texture in order to deliberately increase the false positive and, especially, false negative match rates. Many texture analysis based techniques have been proposed for detecting the presence of cosmetic contact lenses. However, it has been shown recently that the generalization capability of the existing approaches is not sufficient because they have been developed for detecting specific lens texture patterns and evaluated only on those same lens types seen during development phase. This scenario does not apply in unpredictable practical applications because unseen lens patterns will be definitely experienced in operation. In this paper, we address this issue by studying the effect of different iris image preprocessing techniques and introducing a novel approach formore generalized cosmetic contact lens detection using binarized statistical image features (BSIF).Our extensive experimental analysis on benchmark datasets shows that the BSIF description extracted from preprocessed Cartesian iris texture images yields to promising generalization capabilities across unseen texture patterns and different iris sensors with mean equal error rate of 0.14%and 0.88%, respectively. The findings support the intuition that the textural differences between genuine iris texture and fake ones are best described by preserving the regular structure of different printing signatures without transforming the iris images into polar coordinate system.
A Hybrid Approach to Face Detection under Unconstrained Environments

To detect faces in natural and unconstrained environments, we propose an approach which combines the advantages of both color and gray scale based methods. The idea consists of first preprocessing the images using a state-of-the-art approach for skin modeling in order to determine the potential skin regions. Thus, a scanning of the whole image when searching for faces is avoided. Then, in contrast to the existing methods, we consider the fact that the skin detection step still may produce unsatisfactory results or even fail and therefore we apply an exhaustive search in and around the detected skin regions using a new gray scale based approach. The experimental results show that the proposed approach inherits the speed from the color based methods and the efficiency from the gray scale based ones
Evaluating quality of surface description using robust methods

In previous work (Koivunen and Pietikainen, 1991) the authors presented a segmentation method that combines useful properties of edge and region-based segmentation. The least squares estimation used gives good results when pixels in the neighborhood are from one statistical population, and the noise is Gaussian distributed. To be able to deal with very deviant pixel values, the authors applied an iterative reweighting least squares method and a least trimmed squares method for surface description. This paper presents the improvements on the robustness of the surface description, and a quantitative analysis of the quality of the description. The validity of the assumptions used is also evaluated quantitatively. Both synthetic and real range images are used for test images.<<ETX>>