nils-herrmann/clean_scirepeval huggingface dataset:
Text similarity computing based on sememe Vector Space

Vector Space Model (VSM) is a classic text presentation model in natural language processing. However the assumption that text terms are pairwise orthogonal is not suitable. General Vector Space Model (GVSM) was proposed to improve the VSM by using term similarity to overcome the pairwise orthogonal term assumption. In this paper, based on GVSM a new approach using HowNet sememe similarity to calculate text similarity in sememe space was proposed and verified by experiment.
LDF-Net: Learning a Displacement Field Network for Face Recognition across Pose

Face recognition is an important problem in computer vision, however, it is still challenging due to a few wild factors, such as large variations caused by pose, expression, lighting, etc. In this work, we mainly focus on dealing with the pose variations for face recognition. The proposed method attempts to directly transform a non-frontal face image into frontal one by Learning a Displacement Field network (LDFNet) and then recognizes with the transformed images. The existing methods, that follow the same scheme of transforming non-frontal faces into frontal ones, either transform by using 3D-model (3D methods) or transform by using 2D reconstructive methods (2D methods). The 3D methods may lead to the invisibility of some pixels in the transformed frontal images, while the 2D methods may lead to difference between the pixels in the transformed frontal images and the original non-frontal images. Our proposed LDF-Net method can handle these two problems by learning a morphable displacement field for each pixel in the transformed frontal image. Therefore, LDF-Net can achieve a frontal image where all pixels are from the original non-frontal image pixels and no invisible pixels exist, so as to maintain the informative information from the non-frontal images as much as possible. The experiments on MultiPIE dataset show that the proposed LDF-Net achieves state-of-theart performance for face recognition across pose, especially for those large poses.
AU-aware Deep Networks for facial expression recognition

In this paper, we propose to construct a deep architecture, AU-aware Deep Networks (AUDN), for facial expression recognition by elaborately utilizing the prior knowledge that the appearance variations caused by expression can be decomposed into a batch of local facial Action Units (AUs). The proposed AUDN is composed of three sequential modules: the first module consists of two layers, i.e., a convolution layer and a max-pooling layer, which aim to generate an over-complete representation encoding all expression-specific appearance variations over all possible locations; In the second module, an AU-aware receptive field layer is designed to search subsets of the over-complete representation, each of which aims at best simulating the combination of AUs; In the last module, multilayer Restricted Boltzmann Machines (RBM) are exploited to learn hierarchical features, which are then concatenated for final expression recognition. Experiments on three expression databases CK+, MMI and SFEW demonstrate the effectiveness of AUDN in both lab-controlled and wild environments. All our results are better than or at least competitive to the best known results.
Leveraging Datasets with Varying Annotations for Face Alignment via Deep Regression Network

Facial landmark detection, as a vital topic in computer vision, has been studied for many decades and lots of datasets have been collected for evaluation. These datasets usually have different annotations, e.g., 68-landmark markup for LFPW dataset, while 74-landmark markup for GTAV dataset. Intuitively, it is meaningful to fuse all the datasets to predict a union of all types of landmarks from multiple datasets (i.e., transfer the annotations of each dataset to all other datasets), but this problem is nontrivial due to the distribution discrepancy between datasets and incomplete annotations of all types for each dataset. In this work, we propose a deep regression network coupled with sparse shape regression (DRN-SSR) to predict the union of all types of landmarks by leveraging datasets with varying annotations, each dataset with one type of annotation. Specifically, the deep regression network intends to predict the union of all landmarks, and the sparse shape regression attempts to approximate those undefined landmarks on each dataset so as to guide the learning of the deep regression network for face alignment. Extensive experiments on two challenging datasets, IBUG and GLF, demonstrate that our method can effectively leverage the multiple datasets with different annotations to predict the union of all types of landmarks.
Locality preserving constraints for super-resolution with neighbor embedding

In this paper, we revisit the manifold assumption which has been widely adopted in the learning-based image super-resolution. The assumption states that point-pairs from the high-resolution manifold share the local geometry with the corresponding low-resolution manifold. However, the assumption does not hold always, since the one-to-multiple mapping from LR to HR makes neighbor reconstruction ambiguous and results in blurring and artifacts. To minimize the ambiguous, we utilize Locality Preserving Constraints (LPC) to avoid confusions through emphasizing the consistency of localities on both manifolds explicitly. The LPC are combined with a MAP framework, and realized by building a set of cell-pairs on the coupled manifolds. Finally, we propose an energy minimization algorithm for the MAP with LPC which can reconstruct high quality images compared with previous methods. Experimental results show the effectiveness of our method.
Sparsely Encoded Local Descriptor for face recognition

In this paper, a novel Sparsely Encoded Local Descriptor (SELD) is proposed for face recognition. Compared with K-means or Random-projection tree based previous methods, sparsity constraint is introduced in our dictionary learning and sequent image encoding, which implies more stable and discriminative face representation. Sparse coding also leads to an image descriptor of summation of sparse coefficient vectors, which is quite different from existing code-words appearance frequency(/histogram)-based descriptors. Extensive experiments on both FERET and challenging LFW database show the effectiveness of the proposed SELD method. Especially on the LFW dataset, recognition accuracy comparable to the best known results is achieved.
Learning Discriminative Latent Attributes for Zero-Shot Classification

Zero-shot learning (ZSL) aims to transfer knowledge from observed classes to the unseen classes, based on the assumption that both the seen and unseen classes share a common semantic space, among which attributes enjoy a great popularity. However, few works study whether the human-designed semantic attributes are discriminative enough to recognize different classes. Moreover, attributes are often correlated with each other, which makes it less desirable to learn each attribute independently. In this paper, we propose to learn a latent attribute space, which is not only discriminative but also semantic-preserving, to perform the ZSL task. Specifically, a dictionary learning framework is exploited to connect the latent attribute space with attribute space and similarity space. Extensive experiments on four benchmark datasets show the effectiveness of the proposed approach.
Strip Features for Fast Object Detection

This paper presents a set of effective and efficient features, namely strip features, for detecting objects in real-scene images. Although shapes of a specific class usually have large intraclass variance, some basic local shape elements are relatively stable. Based on this observation, we propose a set of strip features to describe the appearances of those shape elements. Strip features capture object shapes with edgelike and ridgelike strip patterns, which significantly enrich the efficient features such as Haar-like and edgelet features. The proposed features can be efficiently calculated via two kinds of approaches. Moreover, the proposed features can be extended to a perturbed version (namely, perturbed strip features) to alleviate the misalignment caused by deformations. We utilize strip features for object detection under an improved boosting framework, which adopts a complexity-aware criterion to balance the discriminability and efficiency for feature selection. We evaluate the proposed approach for object detection on the public data sets, and the experimental results show the effectiveness and efficiency of the proposed approach.
Projection Metric Learning on Grassmann Manifold with Application to Video based Face Recognition

In video based face recognition, great success has been made by representing videos as linear subspaces, which typically lie in a special type of non-Euclidean space known as Grassmann manifold. To leverage the kernel-based methods developed for Euclidean space, several recent methods have been proposed to embed the Grassmann manifold into a high dimensional Hilbert space by exploiting the well established Project Metric, which can approximate the Riemannian geometry of Grassmann manifold. Nevertheless, they inevitably introduce the drawbacks from traditional kernel-based methods such as implicit map and high computational cost to the Grassmann manifold. To overcome such limitations, we propose a novel method to learn the Projection Metric directly on Grassmann manifold rather than in Hilbert space. From the perspective of manifold learning, our method can be regarded as performing a geometry-aware dimensionality reduction from the original Grassmann manifold to a lower-dimensional, more discriminative Grassmann manifold where more favorable classification can be achieved. Experiments on several real-world video face datasets demonstrate that the proposed method yields competitive performance compared with the state-of-the-art algorithms.
Discriminant analysis on Riemannian manifold of Gaussian distributions for face recognition with image sets

This paper presents a method named Discriminant Analysis on Riemannian manifold of Gaussian distributions (DARG) to solve the problem of face recognition with image sets. Our goal is to capture the underlying data distribution in each set and thus facilitate more robust classification. To this end, we represent image set as Gaussian Mixture Model (GMM) comprising a number of Gaussian components with prior probabilities and seek to discriminate Gaussian components from different classes. In the light of information geometry, the Gaussians lie on a specific Riemannian manifold. To encode such Riemannian geometry properly, we investigate several distances between Gaussians and further derive a series of provably positive definite probabilistic kernels. Through these kernels, a weighted Kernel Discriminant Analysis is finally devised which treats the Gaussians in GMMs as samples and their prior probabilities as sample weights. The proposed method is evaluated by face identification and verification tasks on four most challenging and largest databases, YouTube Celebrities, COX, YouTube Face DB and Point-and-Shoot Challenge, to demonstrate its superiority over the state-of-the-art.
RGB-D Face Recognition via Deep Complementary and Common Feature Learning

RGB-D face recognition has attracted increasing attentions in recent years because of its robustness in unconstrained environment. However, existing approaches either handle individual modalities using completely separate pipelines or treat all the modalities equally using the same pipeline. Such approaches did not adequately consider the modality differences and exploit the modality correlations. We propose a novel approach for RGB-D face recognition that is able to learn complementary features from multiple modalities and common features between different modalities. Specifically, we introduce a joint loss taking activation from both modality-specific feature learning networks, and enforcing the features to be learned in a complementary way. We further extend the capability of this multi-modality (e.g., RGB-D vs. RGB-D) matcher into cross-modality (e.g., RGB vs. RGB-D) scenarios by learning a common feature transformation mapping different modalities into the same feature space. Experimental results on a number of public RGB-D face databases (e.g., EURECOM, VAP, IIIT-D, and BUAA), and a large RGB-D database we collected, show the impressive performance of the proposed approach.
Manifold Discriminant Analysis

This paper presents a novel discriminative learning method, called manifold discriminant analysis (MDA), to solve the problem of image set classification. By modeling each image set as a manifold, we formulate the problem as classification-oriented multi-manifolds learning. Aiming at maximizing “manifold margin”, MDA seeks to learn an embedding space, where manifolds with different class labels are better separated, and local data compactness within each manifold is enhanced. As a result, new testing manifold can be more reliably classified in the learned embedding space. The proposed method is evaluated on the tasks of object recognition with image sets, including face recognition and object categorization. Comprehensive comparisons and extensive experiments demonstrate the effectiveness of our method.
Face Alignment across Large Pose via MT-CNN Based 3D Shape Reconstruction

Face alignment plays an important role for robust face recognition and analysis applications in the wild. While a number of face alignment methods are available, large-pose face alignment remains a very challenging problem due to the ambiguity of facial keypoints in 2D face images. Recent attempts to solve this problem via 3D model fitting show more robustness against large poses and 2D ambiguity, but their accuracy and speed are still limited. We propose a 3D reconstruction based method to quickly and accurately detect 2D facial landmarks and estimate their visibilities. By designing a cascaded multi-task CNN model, we can efficiently reconstruct the 3D face shape, together with pose estimation as an auxiliary task. Finally, the landmarks on 3D shape are projected to the 2D face image to get the 2D landmarks and their visibilities. Experimental results on the challenging 300W-LP, AFLW2000-3D, and AFLW databases show that the proposed approach can be comparable with the state-of-the-art methods and is able to run in real time (32ms per image) on 3.4 GHz CPU.
A robust approach for recognition of text embedded in natural scenes

In this paper, we propose a robust approach for recognition of text embedded in natural scenes. Instead of using binary information as most other OCR systems do, we extract features from intensity of an image directly. We utilize a local intensity normalization method to effectively handle lighting variations. We then employ Gabor transform to obtain local features, and use the linear discriminant analysis (LDA) for selection and classification of features. The proposed method has been applied to a Chinese sign recognition task. The system can recognize a vocabulary of 3755 level I Chinese characters in the Chinese national standard character set GB2312-80 with various print fonts. We tested the system on 1630 test characters in sign images captured from the natural scenes, and the recognition accuracy was 92.46%. We have integrated the system into our automatic Chinese sign translation system.
Shape-based web image clustering for unsupervised object detection?

Automatic object detection for an arbitrary class is an important but very challenging problem, due to the countless kinds of objects in the world and the large amount of labeling work for each object. In this work, we target at solving the problem of automatic object detection for an arbitrary class without the laborious human effort. Motivated by the explosive growth of Web images and the phenomenal success of search techniques, we develop an unsupervised object detection framework by automatically training the object detector on the top returns of certain image search engine queried by the name of the object class. In order to automatically isolate the objects from the Web images for training, only clipart images with simple background are used, which preserve most of the shape information of the objects. A two-stage shape-based clustering algorithm is proposed to mine typical shapes of the object, in which the inner-class variance of object shapes is considered and undesired images are filtered out. In order to reduce the gap between clipart images and real-world images, we introduce an efficient algorithm to synthesize the real-world images from clipart images, and only shape feature is used in the detector training part. Finally, the synthetic images could be used to train object detectors by an off-the-shelf discriminative algorithm, e.g., boosting or SVM. Extensive experiments show the effectiveness of the proposed framework on objects with simple and representative shapes, and the proposed framework could be considered as a good beginning of solving this challenging problem.
Continuous Gesture Recognition with Hand-Oriented Spatiotemporal Feature

In this paper, an efficient spotting-recognition framework is proposed to tackle the large scale continuous gesture recognition problem with the RGB-D data input. Concretely, continuous gestures are firstly segmented into isolated gestures based on the accurate hand positions obtained by two streams Faster R-CNN hand detector In the subsequent recognition stage, firstly, towards the gesture representation, a specific hand-oriented spatiotemporal (ST) feature is extracted for each isolated gesture video by 3D convolutional network (C3D). In this feature, only the hand regions and face location are considered, which can effectively block the negative influence of the distractors, such as the background, cloth and the body and so on. Next, the extracted features from calibrated RGB and depth channels are fused to boost the representative power and the final classification is achieved by using the simple linear SVM. Extensive experiments are conducted on the validation and testing sets of the Continuous Gesture Datasets (ConGD) to validate the effectiveness of the proposed recognition framework. Our method achieves the promising performance with the mean Jaccard Index of 0.6103 and outperforms other results in the ChaLearn LAP Large-scale Continuous Gesture Recognition Challenge.
Prototype Discriminative Learning for Image Set Classification

This letter presents a prototype discriminative learning (PDL) method for image set classification. We aim to simultaneously learn prototypes and a linear discriminative projection to drive that in the target subspace each image set can be discriminated with its nearest neighbor prototype. To reveal the unseen appearance variations implicitly in an image set, the prototypes are actually “virtual,” which do not certainly appear in the set but are searched in the corresponding affine hull. Moreover, to enhance the stability and robustness of the learned target subspace, an orthogonality constraint is imposed on the projection. Thus, to optimize the prototypes and the projection jointly, we design a specific gradient descent mechanism by updating the projection on Stiefel manifold and the prototypes in Euclidean space in an alternative optimization manner. Experimental results on four challenging databases demonstrate the superiority of the proposed PDL method.
Learning Multifunctional Binary Codes for Both Category and Attribute Oriented Retrieval Tasks

In this paper we propose a unified framework to address multiple realistic image retrieval tasks concerning both category and attributes. Considering the scale of modern datasets, hashing is favorable for its low complexity. However, most existing hashing methods are designed to preserve one single kind of similarity, thus incapable of dealing with the different tasks simultaneously. To overcome this limitation, we propose a new hashing method, named Dual Purpose Hashing (DPH), which jointly preserves the category and attribute similarities by exploiting the convolutional networks (CNN) to hierarchically capture the correlations between category and attributes. Since images with both category and attribute labels are scarce, our method is designed to take the abundant partially labelled images on the Internet as training inputs. With such a framework, the binary codes of new-coming images can be readily obtained by quantizing the network outputs of a binary-like layer, and the attributes can be recovered from the codes easily. Experiments on two large-scale datasets show that our dual purpose hash codes can achieve comparable or even better performance than those state-of-the-art methods specifically designed for each individual retrieval task, while being more compact than the compared methods.
Face video retrieval with image query via hashing across Euclidean space and Riemannian manifold

Retrieving videos of a specific person given his/her face image as query becomes more and more appealing for applications like smart movie fast-forwards and suspect searching. It also forms an interesting but challenging computer vision task, as the visual data to match, i.e., still image and video clip are usually represented quite differently. Typically, face image is represented as point (i.e., vector) in Euclidean space, while video clip is seemingly modeled as a point (e.g., covariance matrix) on some particular Riemannian manifold in the light of its recent promising success. It thus incurs a new hashing-based retrieval problem of matching two heterogeneous representations, respectively in Euclidean space and Riemannian manifold. This work makes the first attempt to embed the two heterogeneous spaces into a common discriminant Hamming space. Specifically, we propose Hashing across Euclidean space and Riemannian manifold (HER) by deriving a unified framework to firstly embed the two spaces into corresponding reproducing kernel Hilbert spaces, and then iteratively optimize the intra- and inter-space Hamming distances in a max-margin framework to learn the hash functions for the two spaces. Extensive experiments demonstrate the impressive superiority of our method over the state-of-the-art competitive hash learning methods.
A Unified Multiplicative Framework for Attribute Learning

Attributes are mid-level semantic properties of objects. Recent research has shown that visual attributes can benefit many traditional learning problems in computer vision community. However, attribute learning is still a challenging problem as the attributes may not always be predictable directly from input images and the variation of visual attributes is sometimes large across categories. In this paper, we propose a unified multiplicative framework for attribute learning, which tackles the key problems. Specifically, images and category information are jointly projected into a shared feature space, where the latent factors are disentangled and multiplied for attribute prediction. The resulting attribute classifier is category-specific instead of being shared by all categories. Moreover, our method can leverage auxiliary data to enhance the predictive ability of attribute classifiers, reducing the effort of instance-level attribute annotation to some extent. Experimental results show that our method achieves superior performance on both instance-level and category-level attribute prediction. For zero-shot learning based on attributes, our method significantly improves the state-of-the-art performance on AwA dataset and achieves comparable performance on CUB dataset.