nils-herrmann/clean_scirepeval huggingface dataset:
Partial Person Re-Identification

We address a new partial person re-identification (re-id) problem, where only a partial observation of a person is available for matching across different non-overlapping camera views. This differs significantly from the conventional person re-id setting where it is assumed that the full body of a person is detected and aligned. To solve this more challenging and realistic re-id problem without the implicit assumption of manual body-parts alignment, we propose a matching framework consisting of 1) a local patch-level matching model based on a novel sparse representation classification formulation with explicit patch ambiguity modelling, and 2) a global part-based matching model providing complementary spatial layout information. Our framework is evaluated on a new partial person re-id dataset as well as two existing datasets modified to include partial person images. The results show that the proposed method outperforms significantly existing re-id methods as well as other partial visual matching methods.
Weighted Multi-view On-Line Competitive Clustering

Multi-view clustering has become a popular clustering technique in recent years due to its ability to analyze data collected from multiple sources or represented by multiple views. In this paper, we propose a novel multi-view clustering approach termed weighted multi-view online competitive clustering (WMLCC). We simultaneously exploit the variable weighting strategy and the online competitive learning in our approach and cast the multi-view clustering problem into an optimization problem. The multi-view clustering result can be obtained by optimizing a new objective function. We conduct Multi-view clustering has become a popular clustering technique in recent years due to its ability to analyze data collected from multiple sources or represented by multiple views. In this paper, we propose a novel multi-view clustering approach termed weighted multi-view online competitive clustering (WMLCC). We simultaneously exploit the variable weighting strategy and the online competitive learning in our approach and cast the multi-view clustering problem into an optimization problem. The multi-view clustering result can be obtained by optimizing a new objective function. We conduct experiments on two real-world multi-view datasets. Experimental results demonstrate the effectiveness and efficiency of our approach.
Logarithm Gradient Histogram: A general illumination invariant descriptor for face recognition

In the last decade, illumination problem has been the bottleneck of robust face recognition system. Extracting illumination invariant features becomes more and more significant to solve this issue. However, existing works in this field only consider the variations caused by lighting direction or magnitude (denoted as homogeneous lighting), while the spectral wavelength is always ignored in most of the developed illumination invariant descriptors. In this paper, we claim that the spectral wavelength is important, and we propose a novel gradient based descriptor, namely Logarithm Gradient Histogram (LGH), which takes the illumination direction, magnitude and even the spectral wavelength together into consideration (denoted as heterogeneous lighting). Our proposal contributes in the following three-folds: (1) we incorporate homogeneous filtering to alleviate the illumination effect for each image and extract two illumination invariant components, namely logarithm gradient orientation (LGO) and logarithm gradient magnitude (LGM); (2) we propose an effective postprocessing strategy to guarantee the fault-tolerant ability and generate a histogram representation to integrate both LGO and LGM; (3) we present thorough theoretical analysis on the illumination invariant properties for our proposed method. Experimental results on CMU-PIE, Extended YaleB and HFB databases are reported to verify the effectiveness of our proposed method.
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition

In this paper, we focus on heterogeneous features learning for RGB-D activity recognition. We find that features from different channels (RGB, depth) could share some similar hidden structures, and then propose a joint learning model to simultaneously explore the shared and feature-specific components as an instance of heterogeneous multi-task learning. The proposed model formed in a unified framework is capable of: 1) jointly mining a set of subspaces with the same dimensionality to exploit latent shared features across different feature channels, 2) meanwhile, quantifying the shared and feature-specific components of features in the subspaces, and 3) transferring feature-specific intermediate transforms (i-transforms) for learning fusion of heterogeneous features across datasets. To efficiently train the joint model, a three-step iterative optimization algorithm is proposed, followed by a simple inference model. Extensive experimental results on four activity datasets have demonstrated the efficacy of the proposed method. A new RGB-D activity dataset focusing on human-object interaction is further contributed, which presents more challenges for RGB-D activity benchmarking.
Learning Person–Person Interaction in Collective Activity Recognition

Collective activity is a collection of atomic activities (individual person's activity) and can hardly be distinguished by an atomic activity in isolation. The interactions among people are important cues for recognizing collective activity. In this paper, we concentrate on modeling the person-person interactions for collective activity recognition. Rather than relying on hand-craft description of the person-person interaction, we propose a novel learning-based approach that is capable of computing the class-specific person-person interaction patterns. In particular, we model each class of collective activity by an interaction matrix, which is designed to measure the connection between any pair of atomic activities in a collective activity instance. We then formulate an interaction response (IR) model by assembling all these measurements and make the IR class specific and distinct from each other. A multitask IR is further proposed to jointly learn different person-person interaction patterns simultaneously in order to learn the relation between different person-person interactions and keep more distinct activity-specific factor for each interaction at the same time. Our model is able to exploit discriminative low-rank representation of person-person interaction. Experimental results on two challenging data sets demonstrate our proposed model is comparable with the state-of-the-art models and show that learning person-person interactions plays a critical role in collective activity recognition.
An enhanced deep feature representation for person re-identification

Feature representation and metric learning are two critical components in person re-identification models. In this paper, we focus on the feature representation and claim that hand-crafted histogram features can be complementary to Convolutional Neural Network (CNN) features. We propose a novel feature extraction model called Feature Fusion Net (FFN) for pedestrian image representation. In FFN, back propagation makes CNN features constrained by the handcrafted features. Utilizing color histogram features (RGB, HSV, YCbCr, Lab and YIQ) and texture features (multi-scale and multi-orientation Gabor features), we get a new deep feature representation that is more discriminative and compact. Experiments on three challenging datasets (VIPeR, CUHK01, PRID450s) validates the effectiveness of our proposal.
Exemplar-Based Recognition of Human–Object Interactions

Human action can be recognized from a single still image by modeling human-object interactions (HOIs), which infers the mutual spatial structure information between human and the manipulated object as well as their appearance. Existing approaches rely heavily on accurate detection of human and object and estimation of human pose; they are thus sensitive to large variations of human poses, occlusion, and unsatisfactory detection of small size objects. To overcome this limitation, a novel exemplar-based approach is proposed in this paper. Our approach learns a set of spatial pose-object interaction exemplars, which are probabilistic density functions describing spatially how a person is interacting with a manipulated object for different activities. Specifically, a new framework consisting of an exemplar-based HOI descriptor and an associated matching model is formulated for robust human action recognition in still images. In addition, the framework is extended to perform HOI recognition in videos, where the proposed exemplar representation is used for implicit frame selection to negate irrelevant or noisy frames by temporal structured HOI modeling. Extensive experiments are carried out on two image action datasets and two video action datasets. The results demonstrate the effectiveness of our proposed methods and show that our approach is able to achieve state-of-the-art performance, compared with several recently proposed competitors.
PersonRank: Detecting Important People in Images

Always, some individuals in images are more important/attractive than the others in some events such as presentation, basketball game or speech. However, it is challenging to ?nd important people among all individuals in an image directly based on their spatial or appearance information due to the existence of diverse variations of pose, action, appearance of persons an various changes of occasions. We overcome this challenge by constructing a multiple HyperInteraction Graph that treats each individual in an image as a node and inferring the most active node from the interactions estimated by using various types of cues. We model a pairwise interaction between people as an edge message communicated between nodes, resulting in a bidirectional pairwise-interaction graph. To enrich the person-person interaction estimation, we further introduce a unidirectional hyper-interaction graph that models the consensus of interactions between a focal person and any person in his/her local region around. Finally, we modify the PageRank algorithm to infer the activeness of people on the multiple Hybrid-Interaction Graph (HIG), the union of the pairwise-interaction and hyper-interaction graphs, and we call our algorithm the PersonRank. In order to provide publicable datasets for evaluation, we have contributed a new dataset called Multi-scene Important People Image Dataset and gathered a NCAA Basketball Image Dataset from sports game sequences. We have demonstrated that the proposed PersonRank outperforms related methods clearly and substantially. Our code and datasets are available at https://weihonglee.github.io/Projects/PersonRank.htm.
Cross-View Asymmetric Metric Learning for Unsupervised Person Re-Identification

While metric learning is important for Person reidentification (RE-ID), a significant problem in visual surveillance for cross-view pedestrian matching, existing metric models for RE-ID are mostly based on supervised learning that requires quantities of labeled samples in all pairs of camera views for training. However, this limits their scalabilities to realistic applications, in which a large amount of data over multiple disjoint camera views is available but not labelled. To overcome the problem, we propose unsupervised asymmetric metric learning for unsupervised RE-ID. Our model aims to learn an asymmetric metric, i.e., specific projection for each view, based on asymmetric clustering on cross-view person images. Our model finds a shared space where view-specific bias is alleviated and thus better matching performance can be achieved. Extensive experiments have been conducted on a baseline and five large-scale RE-ID datasets to demonstrate the effectiveness of the proposed model. Through the comparison, we show that our model works much more suitable for unsupervised RE-ID compared to classical unsupervised metric learning models. We also compare with existing unsupervised REID methods, and our model outperforms them with notable margins. Specifically, we report the results on large-scale unlabelled RE-ID dataset, which is important but unfortunately less concerned in literatures.
HEp-2 specimen classification via deep CNNs and pattern histogram

Automatic classification of Human Epithelial Type-2 (HEp-2) specimen patterns is an important yet challenging problem in medical image analysis. Most prior works have primarily focused on cells images classification problem which is one of the early essential steps in the system pipeline, while less attention has been paid to the classification of whole-specimen ones. In this work, a specimen pattern recognition system combining convolutional neural networks (CNNs) and pattern histogram was proposed. The pattern histograms were obtained based on the prediction of each single cell inside the specimens. Two strategies were designed to predicted the pattern of a whole specimen: 1) the most dominant cell pattern in pattern histogram was represented as the specimen pattern, 2) the pattern histograms were employed as bags of patterns and then were trained and predicted separately by a SVM classifier. Experimental results show that the proposed system is effective and achieves high classification accuracy on public benchmark datasets. We further evaluate the robustness of the proposed framework by testing trained CNNs on another different dataset, demonstrating that the system is robust to inter-lab data.
Semi-Supervised Multi-View Discrete Hashing for Fast Image Search

Hashing is an important method for fast neighbor search on large scale dataset in Hamming space. While most research on hash models are focusing on single-view data, recently the multi-view approaches with a majority of unsupervised multi-view hash models have been considered. Despite of existence of millions of unlabeled data samples, it is believed that labeling a handful of data will remarkably improve the searching performance. In this paper, we propose a semi-supervised multi-view hash model. Besides incorporating a portion of label information into the model, the proposed multi-view model differs from existing multi-view hash models in three-fold: 1) a composite discrete hash learning modeling that is able to minimize the loss jointly on multi-view features when using relaxation on learning hashing codes; 2) exploring statistically uncorrelated multi-view features for generating hash codes; and 3) a composite locality preserving modeling for locally compact coding. Extensive experiments have been conducted to show the effectiveness of the proposed semi-supervised multi-view hash model as compared with related multi-view hash models and semi-supervised hash models.
Maximum Correntropy Criterion for Robust Face Recognition

In this paper, we present a sparse correntropy framework for computing robust sparse representations of face images for recognition. Compared with the state-of-the-art l1norm-based sparse representation classifier (SRC), which assumes that noise also has a sparse representation, our sparse algorithm is developed based on the maximum correntropy criterion, which is much more insensitive to outliers. In order to develop a more tractable and practical approach, we in particular impose nonnegativity constraint on the variables in the maximum correntropy criterion and develop a half-quadratic optimization technique to approximately maximize the objective function in an alternating way so that the complex optimization problem is reduced to learning a sparse representation through a weighted linear least squares problem with nonnegativity constraint at each iteration. Our extensive experiments demonstrate that the proposed method is more robust and efficient in dealing with the occlusion and corruption problems in face recognition as compared to the related state-of-the-art methods. In particular, it shows that the proposed method can improve both recognition accuracy and receiver operator characteristic (ROC) curves, while the computational cost is much lower than the SRC algorithms.
Fast Open-World Person Re-Identification

Existing person re-identification (re-id) methods typically assume that: 1) any probe person is guaranteed to appear in the gallery target population during deployment (i.e., closed-world) and 2) the probe set contains only a limited number of people (i.e., small search scale). Both assumptions are artificial and breached in real-world applications, since the probe population in target people search can be extremely vast in practice due to the ambiguity of probe search space boundary. Therefore, it is unrealistic that any probe person is assumed as one target people, and a large-scale search in person images is inherently demanded. In this paper, we introduce a new person re-id search setting, called large scale open-world (LSOW) re-id, characterized by huge size probe images and open person population in search thus more close to practical deployments. Under LSOW, the under-studied problem of person re-id efficiency is essential in addition to that of commonly studied re-id accuracy. We, therefore, develop a novel fast person re-id method, called Cross-view Identity Correlation and vErification (X-ICE) hashing, for joint learning of cross-view identity representation binarisation and discrimination in a unified manner. Extensive comparative experiments on three large-scale benchmarks have been conducted to validate the superiority and advantages of the proposed X-ICE method over a wide range of the state-of-the-art hashing models, person re-id methods, and their combinations.
Transfer re-identification: From person to set-based verification

Solving the person re-identification problem has become important for understanding people's behaviours in a multicamera network of non-overlapping views. In this work, we address the problem of re-identification from a set-based verification perspective. More specifically, we have a small set of target people on a watch list (a set) and we aim to verify whether a query image of a person is on this watch list. This differs from the existing person re-identification problem in that the probe is verified against a small set of known people but requires much higher degree of verification accuracy with very limited sampling data for each candidate in the set. That is, rather than recognising everybody in the scene, we consider identifying a small set of target people against non-target people when there is only a limited number of target training samples and a large number of unlabelled (unknown) non-target samples available. To this end, we formulate a transfer learning framework for mining discriminant information from non-target people data to solve the watch list set verification problem. Based on the proposed approach, we introduce the concepts of multi-shot and one-shot verifications. We also design new criteria for evaluating the performance of the proposed transfer learning method against the i-LIDS and ETHZ data sets.
Facial skin beautification via sparse representation over learned layer dictionary

In this paper, we propose a facial skin beautification framework to remove facial spots based on layer dictionary learning and sparse representation. More precisely, we first decompose the face image into three layers: lighting layer, detail layer and color layer. The corresponding detail layer dictionary are learned by using 60 thousands beauty images collected from the Internet. Thereafter, the detail layer of the image is reconstructed by using sparse representation. Moreover, a binary mask obtained from the learned layer is used to transform detail information from original detail layer to the learned one. The experiment results demonstrate that the proposed method is more effective in eliminating moles, flaws and wrinkles in face image compared with representative commercial systems like PicTreat, Portrait+, Portraitrue and MeituPic.
Multi-Scale Learning for Low-Resolution Person Re-Identification

In real world person re-identification (re-id), images of people captured at very different resolutions from different locations need be matched. Existing re-id models typically normalise all person images to the same size. However, a low-resolution (LR) image contains much less information about a person, and direct image scaling and simple size normalisation as done in conventional re-id methods cannot compensate for the loss of information. To solve this LR person re-id problem, we propose a novel joint multi-scale learning framework, termed joint multi-scale discriminant component analysis (JUDEA). The key component of this framework is a heterogeneous class mean discrepancy (HCMD) criterion for cross-scale image domain alignment, which is optimised simultaneously with discriminant modelling across multiple scales in the joint learning framework. Our experiments show that the proposed JUDEA framework outperforms existing representative re-id methods as well as other related LR visual matching models applied for the LR person re-id problem.
Person Re-Identification by Camera Correlation Aware Feature Augmentation

The challenge of person re-identification (re-id) is to match individual images of the same person captured by different non-overlapping camera views against significant and unknown cross-view feature distortion. While a large number of distance metric/subspace learning models have been developed for re-id, the cross-view transformations they learned are view-generic and thus potentially less effective in quantifying the feature distortion inherent to each camera view. Learning view-specific feature transformations for re-id (i.e., view-specific re-id), an under-studied approach, becomes an alternative resort for this problem. In this work, we formulate a novel view-specific person re-identification framework from the feature augmentation point of view, called Camera coR relation Aware Feature augmenTation (CRAFT). Specifically, CRAFT performs cross-view adaptation by automatically measuring camera correlation from cross-view visual data distribution and adaptively conducting feature augmentation to transform the original features into a new adaptive space. Through our augmentation framework, view-generic learning algorithms can be readily generalized to learn and optimize view-specific sub-models whilst simultaneously modelling view-generic discrimination information. Therefore, our framework not only inherits the strength of view-generic model learning but also provides an effective way to take into account view specific characteristics. Our CRAFT framework can be extended to jointly learn view-specific feature transformations for person re-id across a large network with more than two cameras, a largely under-investigated but realistic re-id setting. Additionally, we present a domain-generic deep person appearance representation which is designed particularly to be towards view invariant for facilitating cross-view adaptation by CRAFT. We conducted extensively comparative experiments to validate the superiority and advantages of our proposed framework over state-of-the-art competitors on contemporary challenging person re-id datasets.