nils-herrmann/clean_scirepeval huggingface dataset:
Learning Coupled Feature Spaces for Cross-Modal Matching

Cross-modal matching has recently drawn much attention due to the widespread existence of multimodal data. It aims to match data from different modalities, and generally involves two basic problems: the measure of relevance and coupled feature selection. Most previous works mainly focus on solving the first problem. In this paper, we propose a novel coupled linear regression framework to deal with both problems. Our method learns two projection matrices to map multimodal data into a common feature space, in which cross-modal data matching can be performed. And in the learning procedure, the ell_21-norm penalties are imposed on the two projection matrices separately, which leads to select relevant and discriminative features from coupled feature spaces simultaneously. A trace norm is further imposed on the projected data as a low-rank constraint, which enhances the relevance of different modal data with connections. We also present an iterative algorithm based on half-quadratic minimization to solve the proposed regularized linear regression problem. The experimental results on two challenging cross-modal datasets demonstrate that the proposed method outperforms the state-of-the-art approaches.
Encyclopedia enhanced semantic embedding for zero-shot learning

There are tremendous object categories in the real world besides those in image datasets. Zero-shot learning aims to recognize image categories which are unseen in the training set. A large number of previous zero-shot learning models use word vectors of the class labels directly as category prototypes in the semantic embedding space. But word vectors cannot obtain the global knowledge of an image category sufficiently. In this paper, we propose a new encyclopedia enhanced semantic embedding model to promote the discriminative capability of word vector prototypes with the global knowledge of each image category. The proposed model extracts the TF-IDF key words from encyclopedia articles to acquire the global knowledge of each category. The convex combination of the key words' word vectors acts as the prototypes of the object categories. The prototypes of seen and unseen classes build up the embedding space where the nearest neighbour search is implemented to recognize the unseen images. The experiments show that the proposed method achieves the state-of-the-art performance on the challenging ImageNet Fall 2011 1k2hop dataset.
Feature Selection Based on Structured Sparsity: A Comprehensive Study.

Feature selection (FS) is an important component of many pattern recognition tasks. In these tasks, one is often confronted with very high-dimensional data. FS algorithms are designed to identify the relevant feature subset from the original features, which can facilitate subsequent analysis, such as clustering and classification. Structured sparsity-inducing feature selection (SSFS) methods have been widely studied in the last few years, and a number of algorithms have been proposed. However, there is no comprehensive study concerning the connections between different SSFS methods, and how they have evolved. In this paper, we attempt to provide a survey on various SSFS methods, including their motivations and mathematical representations. We then explore the relationship among different formulations and propose a taxonomy to elucidate their evolution. We group the existing SSFS methods into two categories, i.e., vector-based feature selection (feature selection based on lasso) and matrix-based feature selection (feature selection based on lr,p-norm). Furthermore, FS has been combined with other machine learning algorithms for specific applications, such as multitask learning, multilabel learning, multiview learning, classification, and clustering. This paper not only compares the differences and commonalities of these methods based on regression and regularization strategies, but also provides useful guidelines to practitioners working in related fields to guide them how to do feature selection.
Palmprint recognition using coarse-to-fine statistical image representation

Recent literatures have revealed that statistics of local texture measures can provide accurate descriptions of palmprint appearances. In this framework, one palmprint image is divided into local blocks with multiple spatial resolutions. The statistical texture descriptions of each block are then concatenated to form a multi-scale image representation. However, resultant high-dimensional statistical features lead to increasing of computational cost. In this paper, we tackle this problem by performing a coarse-to-fine cascade scheme, which makes use of information redundancy of statistical texture descriptions between different spatial scales. In contrast with non-cascade strategies, the proposed method reduces most of computational burden and achieves accurate classification simultaneously.
Robust steganalysis based on training set construction and ensemble classifiers weighting

The cover source mismatch problem in steganalysis is a serious problem which keeps current steganalysis from practical use. It is mainly because of the high intra-class variation of cover and stego samples in the feature space, since current steganalytic features are inevitably affected much by the image content, size, quality and many other factors. Small training set often reflects only part of the real data distribution, hence the classifier (steganalyzer) may be undertrained and lack of robustness. In this paper, we propose a scheme to efficiently construct large representative training set for steganalysis. We also scheme out weighted ensemble classifiers which can be adaptive to testing data. Experimental results show that our method can improve the performance and robustness of ste-ganalysis under high intra-class variation.
Biometric personal identification based on handwriting

In this paper, we describe a new method to identify the writer of Chinese handwritten documents. There are many methods for signature verification or writer identification, but most of them require segmentation or connected component analysis. They are content dependent identification methods, as signature verification requires the writer to write the same text (e.g. his name). In our new method, we take the handwriting as an image containing some special texture, and writer identification is regarded as texture identification. This is a content independent method. We apply the well-established 2D Gabor filtering technique to extract features of such textures and a weighted Euclidean distance classifier to fulfil the identification task. Experiments are made using Chinese handwritings from 17 different people and very promising results were achieved.
Automatic detection of 3D lighting inconsistencies via a facial landmark based morphable model

Existing 3D lighting consistency based forensic methods have some practical problems. They usually require additional images and human labor to reconstruct the 3D face model for lighting estimation, and furthermore, they cannot deal with expressional faces effectively. These drawbacks make them unusable in many practical cases. In this paper, we propose a more practical 3D lighting based forensic method by incorporating a facial landmark based 3D morphable model to efficiently fit the face shape. We also introduce a residual error based algorithm to automatically exclude outliers in lighting estimation. Our proposed method is fully automatic and very efficient compared to previous ones. Also, it does not depend on additional images and has better performance for expressional faces. Experiments on a realistic face dataset with variational lighting conditions indicate the efficacy and superiority of our method.
Written language recognition based on texture analysis

Numerous techniques have been reported for optical character recognition (OCR). Almost all such techniques make an implicit assumption that the language of the document to be processed is known. We attempt to eliminate this assumption by presenting a novel algorithm for automatic written language recognition. Given that different languages are often visually distinctive in written form, we take a global approach based on texture analysis, where each language is regarded as a different texture. In principle this allows us to apply any standard texture recognition algorithm for the task. Experiments with six languages clearly demonstrate the great potential of the proposed global approach.
Silhouette Analysis-Based Gait Recognition for Human Identification

Human identification at a distance has recently gained growing interest from computer vision researchers. Gait recognition aims essentially to address this problem by identifying people based on the way they walk. In this paper, a simple but efficient gait recognition algorithm using spatial-temporal silhouette analysis is proposed. For each image sequence, a background subtraction algorithm and a simple correspondence procedure are first used to segment and track the moving silhouettes of a walking figure. Then, eigenspace transformation based on principal component analysis (PCA) is applied to time-varying distance signals derived from a sequence of silhouette images to reduce the dimensionality of the input feature space. Supervised pattern classification techniques are finally performed in the lower-dimensional eigenspace for recognition. This method implicitly captures the structural and transitional characteristics of gait. Extensive experimental results on outdoor image sequences demonstrate that the proposed algorithm has an encouraging recognition performance with relatively low computational cost.
A brief review on integrated audio-visual processing for personal identification

Personal identification has potential applications in numerous areas where the identity of a person needs to be determined or validated. The use of human identification in personal data systems used by organisations is remarkably little-discussed in the information systems literature. This paper presents a brief review on speaker identification, face recognition and integrated audio-visual systems for personal identification. A brief literature review on existing systems based on voice, face and integrated audio-visual cues is given, together with remarks on the advantages of using integrated audio-visual systems. It appears that a lot of work has been done on speaker identification and face recognition but not much on combining the two approaches in one system. In general, the voice and face of a person are sources of information presumed to uniquely identify a person. It is well accepted that the likelihood for the voices or faces of two persons to be identical is extremely small. The integration of audio and visual data for personal identification has the advantages of higher recognition performance than speaker recognition or face recognition systems operating separately. Recognition performance is expected to improve in terms of accuracy and robustness against variation of either facial or vocal information. (6 pages)
Robust direction estimation of gradient vector field for iris recognition

As a reliable personal identification method, iris recognition has been receiving increasing attention. Based on the theory of robust statistics, a novel geometry-driven method for iris recognition is presented in this paper. An iris image is considered as a 3D surface of piecewise smooth patches. The direction of the 2D vector, which is the planar projection of the normal vector of image surface, is illumination insensitive and opposite to the direction of gradient vector. So the directional information of iris image's gradient vector field (GVF) is used to represent iris pattern. Robust direction estimation, direction diffusion followed by vector directional filtering, is performed on the GVF to extract stable iris feature. Extensive experimental results demonstrate that the recognition performance of the proposed algorithm is comparable with the best method in the open literature.
Ordinal Feature Selection for Iris and Palmprint Recognition

Ordinal measures have been demonstrated as an effective feature representation model for iris and palmprint recognition. However, ordinal measures are a general concept of image analysis and numerous variants with different parameter settings, such as location, scale, orientation, and so on, can be derived to construct a huge feature space. This paper proposes a novel optimization formulation for ordinal feature selection with successful applications to both iris and palmprint recognition. The objective function of the proposed feature selection method has two parts, i.e., misclassification error of intra and interclass matching samples and weighted sparsity of ordinal feature descriptors. Therefore, the feature selection aims to achieve an accurate and sparse representation of ordinal measures. And, the optimization subjects to a number of linear inequality constraints, which require that all intra and interclass matching pairs are well separated with a large margin. Ordinal feature selection is formulated as a linear programming (LP) problem so that a solution can be efficiently obtained even on a large-scale feature pool and training database. Extensive experimental results demonstrate that the proposed LP formulation is advantageous over existing feature selection methods, such as mRMR, ReliefF, Boosting, and Lasso for biometric recognition, reporting state-of-the-art accuracy on CASIA and PolyU databases.
Semi-supervised Learning for Cross-Device Visual Location Recognition

The aim of this work is to localize a query mobile photograph by utilizing surveillance images, which naturally provide location information. We cast this cross-device visual localization problem as a classification task. By exploiting the surveillance network to collect reference images, the data acquisition process is significantly facilitated. However, the discrepancy between mobile images and surveillance images makes the training samples difficult to be used directly, and the scarcity of training samples caused by the immobility of surveillance cameras further degrades the performance. In contrast to most traditional domain adaptation problems and semi-supervised problems, the scarce labeled data and plentiful unlabeled data exist in different domains. Our location recognition method first exploits the unsupervised subspace alignment to weaken the discrepancy between the two domains, and then adopts the semi-supervised Laplacian SVM to reinforce the discriminant information utilizing the unlabeled mobile images. Experimental results show that our location recognition method significantly outperforms other related methods.
Deep semantic ranking based hashing for multi-label image retrieval

With the rapid growth of web images, hashing has received increasing interests in large scale image retrieval. Research efforts have been devoted to learning compact binary codes that preserve semantic similarity based on labels. However, most of these hashing methods are designed to handle simple binary similarity. The complex multi-level semantic structure of images associated with multiple labels have not yet been well explored. Here we propose a deep semantic ranking based method for learning hash functions that preserve multilevel semantic similarity between multi-label images. In our approach, deep convolutional neural network is incorporated into hash functions to jointly learn feature representations and mappings from them to hash codes, which avoids the limitation of semantic representation power of hand-crafted features. Meanwhile, a ranking list that encodes the multilevel similarity information is employed to guide the learning of such deep hash functions. An effective scheme based on surrogate loss is used to solve the intractable optimization problem of nonsmooth and multivariate ranking measures involved in the learning procedure. Experimental results show the superiority of our proposed approach over several state-of-the-art hashing methods in term of ranking evaluation metrics when tested on multi-label image datasets.
CASIA Image Tampering Detection Evaluation Database

Image forensics has now raised the anxiety of justice as increasing cases of abusing tampered images in newspapers and court for evidence are reported recently. With the goal of verifying image content authenticity, passive-blind image tampering detection is called for. More realistic open benchmark databases are also needed to assist the techniques. Recently, we collect a natural color image database with realistic tampering operations. The database is made publicly available for researchers to compare and evaluate their proposed tampering detection techniques. We call this database CASI-A Image Tampering Detection Evaluation Database. We describe the purpose, the design criterion, the organization and self-evaluation of this database in this paper.
An optimal set of code words and correntropy for rotated least squares regression

This paper presents a robust feature extraction method for face recognition based on least squares regression (LSR). Our focus is to enhance the robustness and discriminability of the LSR. First, an optimal set of code words is introduced in LSR. Compared to the traditional set of code words, this new set uses less number of code words. Furthermore, it can make the distance of the regression targets of different classes as large as possible. Then, correntropy is integrated into the LSR model for better robustness. Furthermore, considering the commonly used distance metrics such as Euclidean distance and Cosine distance in the subspace are invariant to rotation transformation, rotation is introduced as additional freedom to promote flexibility without sacrificing accuracy. Our objective function is optimized using half-quadratic (HQ) optimization, which facilitates algorithm development and convergence study. Experimental results show that our method outperforms several subspace methods for face recognition, which indicates the validity of the proposed method.
Agent orientated annotation in model based visual surveillance

The paper presents an agent based surveillance system for use in monitoring scenes involving both pedestrians and vehicles. The system supplies textual descriptions for the dynamic activity occurring in the 3D world. These are derived by means of dynamic and probabilistic inference based on geometric information provided by a vision system that tracks vehicles and pedestrians. The symbolic scene annotation is given at two major levels of description: the object level and the inter-object level. At object level, each tracked pedestrian or vehicle is assigned a behaviour agent which uses a Bayesian network to infer the fundamental features of the objects' trajectory, and continuously updates its textual description. The inter-object interaction level is interpreted by a situation agent which is created dynamically when two objects are in close proximity. In the work included here the situation agent can describe a two-object interaction in terms of basic textual annotations, to summarise the dynamics of the local action.
IAPR keynote lecture I: Large-scale visual computing: Challenges and opportunities

The widespread deployment of visual sensors such as surveillance cameras leads to the explosion of visual data. The timely processing and understanding of such massive information presents a clear challenge as well as a great opportunity for computer vision. This talk starts with a brief introduction to the concept of large-scale visual computing and outlines the status quo of the field as well as the main research challenges. It focuses on discussing some of the key issues in large-scale visual computing such as large-scale feature representation and large-scale modelling. It also discusses some promising directions for future research.
Depth-embedded multiple pooling for image classification

Most existing methods of image classification ignore the role of depth information hidden in 2-D images. However, the depth information is important for visual perception, especially when the appearance information does not perform well. In this paper, we propose to embed depth information within multiple pooling into the classic platform of image classification, namely bag-of-features. The proposed method quantifies depth diversity by projecting objects to their nearby depth planes, resulting pooling features in the 3-D space indirectly. Experimental results on the MIT Indoor Scene database demonstrate that our proposed depth-embedded multiple pooling is effective to enhance the accuracy of image classification, especially when the appearance features alone are not so discriminative.
Multi-task ConvNet for blind face inpainting with application to face verification

Face verification between ID photos and life photos (FVBIL) is gaining traction with the rapid development of the Internet. However, ID photos provided by the Chinese administration center are often corrupted with wavy lines to prevent misuse, which poses great difficulty to accurate FVBIL. Therefore, this paper tries to improve the verification performance by studying a new problem, i.e. blind face inpainting, where we aim at restoring clean face images from the corrupted ID photos. The term blind indicates that the locations of corruptions are not known in advance. We formulate blind face inpainting as a joint detection and reconstruction problem. A multi-task ConvNet is accordingly developed to facilitate end to end network training for accurate and fast inpainting. The ConvNet is used to (i) regress the residual values between the clean/corrupted ID photo pairs and (ii) predict the positions of residual regions. Moreover, to achieve better inpainting results, we employ a skip connection to fuse information in the intermediate layer. To enable training of our ConvNet, we collect a dataset of synthetic clean/corrupted ID photo pairs with 500 thousand samples from around 10 thousand individuals. Experiments demonstrate that our multi-task ConvNet achieves superior performance in terms of reconstruction errors, convergence speed and verification accuracy.
Detection of computer generated faces in videos based on pulse signal

The rapid development in the field of computer graphics (CG) makes it quite easy to create photo-realistic images and videos. This brings forward an emergent requirement for techniques that can distinguish CG from real contents. In this paper, we propose a method that leverages human pulse signal to distinguish between CG and real videos that include human faces. We use a robust tracking method to locate a patch of skin on the face. Then, a chrominance-based algorithm is employed to robustly extract pulse signal. By checking the frequency waveform of the extracted pulse signal, we can tell CG and real videos apart. The experiment shows encouraging results, which demonstrate the efficiency of our method.
A hierarchical model for the evaluation of biometric sample quality

The evaluation of biometric sample quality is of great importance in the evaluation of biometric algorithms. In this paper, we propose a novel hierarchical model to compute the sample quality at three levels. This model is developed on the basis of three types of influencing factors: global factors, subjective factors and variable factors. We adopt different strategies to compute the corresponding three level qualities: database level quality, class level quality and image level quality. The database level quality is estimated by experience. Then, we compute the mean value of variable number of normalized genuine scores, the quantiles of which are used to determine the class level quality. On the image level quality evaluation, a novel concept of subset frequency is proposed.