nils-herrmann/clean_scirepeval huggingface dataset:
Face Hallucination Through KPCA

This paper demonstrates how Kernel Principal Component Analysis (KPCA) can be used for face hallucination. Different with other KPCA-based methods, KPCA in this paper handles samples from two subspaces, namely the high- and low- resolution image spaces. As KPCA learns not only linear features but also non-linear features, it is anticipated that more detailed facial features could be synthesized. We propose a new model and give theoretical analysis on when it is applicable. Algorithm is then developed for implementation. Experiments are conducted to compare the proposed method with the existing well-known face hallucination methods in terms of visual quality and mean square error. Our results are better and encouraging. Motivated by Wang and Tang's (2) work and the successful applications of KPCA for other face image preprocessing (10)(12), we develop a new method to learn both linear and non-linear intrinsic properties of low-resolution image and high-resolution image using KPCA. Unlike PCA, KPCA is nonlinear and can model structures in given data set better, so that more detailed features could be synthesized. However, we would analyze in next section that the KPCA-based method can not be developed in a straightforward way just by simple combination of kernel and the PCA-based method. As will be analyzed later, the main difficulty is that it is generally difficult to find the relationship between the two kernel feature spaces corresponding to low- and high- resolution data respectively, while the assumption used in PCA-based face hallucination cannot be applied for kernel approach. In order to solve such problem, we propose a new model by assuming that when suitable kernels are used, different resolution images can approximately have the same intrinsic features in their corresponding kernel principal subspaces. Theoretical analysis why this assumption can be applicable will be given. Following this model, the KPCA-based face hallucination method is then developed. In addition, we also propose a new two-step face hallucination approach by integrating the residue compensation to enhance the equality of reconstructed high- resolution images.
Reducing Impact of Inaccurate User Feedback in Face Retrieval

A main problem in face retrieval is the semantic gap between low-level features and high-level semantic concepts. Relevance feedback (RF) may be used to incorporate to reduce the semantic gap. However, in the search for a specific target in a facial image database, a user's assignment of RF instances may be mistaken. This would make the system prediction of the user's target in a wrong way. Addressing this problem, we propose a new query point movement technique for target search by posing the problem of reducing the impact of inaccurate user feedback as an optimization problem. We develop a support vector machine based method to learn a decision boundary to identify ideal irrelevant images. Then we propose a rank function for finding target images, which would assign high scores to the images near the relevant images and punish those close to the decision boundary. Experiments are performed to show the stability and efficiency of the proposed algorithm.
Restoration of a Frontal Illuminated Face Image Based on KPCA

In this paper, we propose a novel illumination-normalization method. By using the combination of the Kernel Principal Component Analysis (KPCA) and Pre-image technology, this method can restore the frontal-illuminated face image from a single non-frontal-illuminated face image. In this method, a frontal-illumination subspace is first learned by KPCA. For each input face image, we project its large-scale features, which are affected by illumination variations, onto this subspace to normalize the illumination. Then the frontal-illuminated face image is reconstructed by combining the small- and the normalized large- scale features. Unlike most existing techniques, the proposed method does not require any shape modeling or lighting estimation. As a holistic reconstruction, KPCA+Pre-image technology incurs less local distortion. Compared to directly applying KPCA+Pre-image technology on the original image, our proposed method can be better at processing an image of a face that is outside the training set. Experiments on CMU-PIE and Extended Yale B face databases show that the proposed method outperforms state-of-the-art algorithms.
A Matrix Splitting Method for Composite Function Minimization

Composite function minimization captures a wide spectrum of applications in both computer vision and machine learning. It includes bound constrained optimization and cardinality regularized optimization as special cases. This paper proposes and analyzes a new Matrix Splitting Method (MSM) for minimizing composite functions. It can be viewed as a generalization of the classical Gauss-Seidel method and the Successive Over-Relaxation method for solving linear systems in the literature. Incorporating a new Gaussian elimination procedure, the matrix splitting method achieves state-of-the-art performance. For convex problems, we establish the global convergence, convergence rate, and iteration complexity of MSM, while for non-convex problems, we prove its global convergence. Finally, we validate the performance of our matrix splitting method on two particular applications: nonnegative matrix factorization and cardinality regularized sparse coding. Extensive experiments show that our method outperforms existing composite function minimization techniques in term of both efficiency and efficacy.
Depth-based person re-identification

Person re-identification aims to match people across non-overlapping camera views. For this purpose, most works exploit appearance cues, assuming that the color of clothes is discriminative in short term. However, when people appear in extreme illumination or change clothes, appearance-based methods tend to fail. Fortunately, depth images provide more invariant body shape and skeleton information regardless of illumination and color, but only a few depth-based methods have been developed so far. In this paper, we propose a covariance-based rotation invariant 3D descriptor called Eigen-depth to describe pedestrian body shape and the property of rotation invariance is proven in theory. It is also insensitive to slight shape change and invariant to color change and background. We combine our descriptor with skeleton-based feature to get a complete representation of human body. The effectiveness is validated on RGBD-ID and BIWIRGBD-ID datasets.
Cross-Scenario Transfer Person Reidentification

Person reidentification (Re-ID) matches images of the same person captured in disjoint camera views and at different times. To obtain a reliable similarity measurement between images, manually annotating a large amount of pairwise cross-camera-view person images is deemed necessary. However, this kind of annotation is both costly and impractical for efficiently deploying a Re-ID system to a completely new scenario, a new setting of nonoverlapping camera views between which person images are to be matched. To solve this problem, we consider utilizing other existing person images captured in other scenarios to help the Re-ID system in a target (new) scenario, provided that a few samples are captured under the new scenario. More specifically, we tackle this problem by jointly learning the similarity measurements for Re-ID in different scenarios in an asymmetric way. To model the joint learning, we consider that the Re-ID models share certain component across tasks. A distinct consideration in our multitask modeling is to extract the discriminant shared component that reduces the cross-task data overlap in the shared latent space during the joint learning, so as to enhance the target inter-class separation in the shared latent space. For this purpose, we propose to maximize the cross-task data discrepancy on the shared component during asymmetric multitask learning (MTL), along with maximizing the local inter-class variation and minimizing local intra-class variation on all tasks. We call our proposed method the constrained asymmetric multitask discriminant component analysis (cAMT-DCA). We show that cAMT-DCA can be solved by a simple eigen decomposition with a closed form, getting rid of any iterative learning used in most conventional MTL analyses. The experimental results show that the proposed transfer model gains a clear improvement against the related nontransfer and general multitask person Re-ID models.
Normalization of Face Illumination Based on Large-and Small-Scale Features

A face image can be represented by a combination of large-and small-scale features. It is well-known that the variations of illumination mainly affect the large-scale features (low-frequency components), and not so much the small-scale features. Therefore, in relevant existing methods only the small-scale features are extracted as illumination-invariant features for face recognition, while the large-scale intrinsic features are always ignored. In this paper, we argue that both large-and small-scale features of a face image are important for face restoration and recognition. Moreover, we suggest that illumination normalization should be performed mainly on the large-scale features of a face image rather than on the original face image. A novel method of normalizing both the Small-and Large-scale (S&L) features of a face image is proposed. In this method, a single face image is first decomposed into large-and small-scale features. After that, illumination normalization is mainly performed on the large-scale features, and only a minor correction is made on the small-scale features. Finally, a normalized face image is generated by combining the processed large-and small-scale features. In addition, an optional visual compensation step is suggested for improving the visual quality of the normalized image. Experiments on CMU-PIE, Extended Yale B, and FRGC 2.0 face databases show that by using the proposed method significantly better recognition performance and visual results can be obtained as compared to related state-of-the-art methods.
Person re-identification by probabilistic relative distance comparison

Matching people across non-overlapping camera views, known as person re-identification, is challenging due to the lack of spatial and temporal constraints and large visual appearance changes caused by variations in view angle, lighting, background clutter and occlusion. To address these challenges, most previous approaches aim to extract visual features that are both distinctive and stable under appearance changes. However, most visual features and their combinations under realistic conditions are neither stable nor distinctive thus should not be used indiscriminately. In this paper, we propose to formulate person re-identification as a distance learning problem, which aims to learn the optimal distance that can maximises matching accuracy regardless the choice of representation. To that end, we introduce a novel Probabilistic Relative Distance Comparison (PRDC) model, which differs from most existing distance learning methods in that, rather than minimising intra-class variation whilst maximising intra-class variation, it aims to maximise the probability of a pair of true match having a smaller distance than that of a wrong match pair. This makes our model more tolerant to appearance changes and less susceptible to model over-fitting. Extensive experiments are carried out to demonstrate that 1) by formulating the person re-identification problem as a distance learning problem, notable improvement on matching accuracy can be obtained against conventional person re-identification techniques, which is particularly significant when the training sample size is small; and 2) our PRDC outperforms not only existing distance learning methods but also alternative learning methods based on boosting and learning to rank.
On Constrained Sparse Matrix Factorization

Various linear subspace methods can be formulated in the notion of matrix factorization in which a cost function is minimized subject to some constraints. Among them, constraints on sparseness have received much attention recently. Some popular constraints such as non-negativity, lasso penalty, and (plain) orthogonality etc have been so far applied to extract sparse features. However, little work has been done to give theoretical and experimental analyses on the differences of the impacts of different constraints within a framework. In this paper, we analyze the problem in a more general framework called Constrained Sparse Matrix Factorization (CSMF). In CSMF, a particular case called CSMF with non-negative components (CSMFnc) is further discussed. Unlike NMF, CSMFnc allows not only additive but also subtractive combinations of non-negative sparse components. It is useful to produce much sparser features than those produced by NMF and meanwhile have better reconstruction ability, achieving a trade-off between sparseness and low MSE value. Moreover, for optimization, an alternating algorithm is developed and a gentle update strategy is further proposed for handling the alternating process. Experimental analyses are performed on the Swimmer data set and CBCLface database. In particular, CSMF can successfully extract all the proper components without any ghost on Swimmer, gaining a significant improvement over the compared well-known algorithms.
Face illumination normalization on large and small scale features

It is well known that the effect of illumination is mainly on the large-scale features (low-frequency components) of a face image. In solving the illumination problem for face recognition, most (if not all) existing methods either only use extracted small-scale features while discard large-scale features, or perform normalization on the whole image. In the latter case, small-scale features may be distorted when the large-scale features are modified. In this paper, we argue that large-scale features of face image are important and contain useful information for face recognition as well as visual quality of normalized image. Moreover, this paper suggests that illumination normalization should mainly perform on large-scale features of face image rather than the whole face image. Along this line, a novel framework for face illumination normalization is proposed. In this framework, a single face image is first decomposed into large- and small- scale feature images using logarithmic total variation (LTV) model. After that, illumination normalization is performed on large-scale feature image while small-scale feature image is smoothed. Finally, a normalized face image is generated by combination of the normalized large-scale feature image and smoothed small-scale feature image. CMU PIE and (Extended) YaleB face databases with different illumination variations are used for evaluation and the experimental results show that the proposed method outperforms existing methods.
Jointly learning heterogeneous features for RGB-D activity recognition

In this paper, we focus on heterogeneous features learning for RGB-D activity recognition. We find that features from different channels (RGB, depth) could share some similar hidden structures, and then propose a joint learning model to simultaneously explore the shared and feature-specific components as an instance of heterogeneous multi-task learning. The proposed model formed in a unified framework is capable of: 1) jointly mining a set of subspaces with the same dimensionality to exploit latent shared features across different feature channels, 2) meanwhile, quantifying the shared and feature-specific components of features in the subspaces, and 3) transferring feature-specific intermediate transforms (i-transforms) for learning fusion of heterogeneous features across datasets. To efficiently train the joint model, a three-step iterative optimization algorithm is proposed, followed by a simple inference model. Extensive experimental results on four activity datasets have demonstrated the efficacy of the proposed method. A new RGB-D activity dataset focusing on human-object interaction is further contributed, which presents more challenges for RGB-D activity benchmarking.
Towards more reliable matching for person re-identification

Person re-identification is an important problem of matching persons across non-overlapping camera views. However, the re-identification is still far from achieving reliable matching. First, many existing approaches are wholebody- based matching, and how body parts could affect and assist the matching is still not clearly known. Second, the learned similarity measurement/metric is equally used for each pair of probe and gallery images, and the bias of the measurement is not considered. In this paper, we address the above two problems in order to conduct a more reliable matching. More specifically, we propose a reliable integrated matching scheme (IMS), which uses body parts to assist matching of the whole body. Moreover, a sparsity-based confidence is also presented for regulating the learned metric to improve the matching reliability. The experiments conducted on three publicly available datasets confirm that the proposed scheme is effective for person re-identification.
Global-Local Temporal Saliency Action Prediction

Action prediction on a partially observed action sequence is a very challenging task. To address this challenge, we first design a global-local distance model, where a global-temporal distance compares subsequences as a whole and local-temporal distance focuses on individual segment. Our distance model introduces temporal saliency for each segment to adapt its contribution. Finally, a global-local temporal action prediction model is formulated in order to jointly learn and fuse these two types of distances. Such a prediction model is capable of recognizing action of: 1) an on-going sequence and 2) a sequence with arbitrarily frames missing between the beginning and end (known as gap-filling). Our proposed model is tested and compared with related action prediction models on BIT, UCF11, and HMDB data sets. The results demonstrated the effectiveness of our proposal. In particular, we showed the benefit of our proposed model on predicting unseen action types and the advantage on addressing the gapfilling problem as compared with recently developed action prediction models.
Distance learning by treating negative samples differently and exploiting impostors with symmetric triplet constraint for person re-identification

Distance learning (DL) is an effective technique for person reidentification (PR-ID). DL based methods learn the distance metric by exploiting the discriminative information contained in samples. In PR-ID, different types of negative samples own different amounts of discriminative information, and impostor samples usually own more than other well separable negative samples (WSN-samples). Therefore, how to make full use of the different discriminative information conveyed by all negative samples in the DL process is a critical issue to be investigated. In this paper, we propose a novel DL approach for PR-ID. Specifically, for each target sample, we divide its negative samples into impostors and WSN-samples. Then we learn the distance metric by utilizing impostors and WSN-samples differently. For impostors, we design a symmetric triplet constraint, which requires the impostor to be far away from both samples of its corresponding positive sample pair simultaneously; for WSN-samples, we require them to keep their favorable separability. Experimental results on three benchmark datasets demonstrate the effectiveness and efficiency of our approach.
Sketch metric learning

The main theme of this paper is to develop a systematic framework to learn a Mahalanobis distance metric based on matrix sketching. Within this framework, we present a novel sketch metric learning algorithm which sequentially sketches the received samples from training dataset and formulates a new kind of constraint for metric learning. This is in contrast to the traditional constraints that are only consisted of data from training dataset. In this paper, one training instance in the constraint is replaced by a pseudo center, which is generated during the sketching stage. Due to this change, our learning algorithm can focus on pushing every received sample to its corresponding similar pseudo center closer and pulling it far away from the dissimilar one. In addition, it can further achieve better performance of some kinds of time-varying process (e.g. on object tracking) than the compared related competitors. We demonstrate how to implement other methods in our algorithm framework and experiment to show that our method outperforms the competitors and relevant baselines on multiple datasets.
Online Hashing

Although hash function learning algorithms have achieved great success in recent years, most existing hash models are off-line, which are not suitable for processing sequential or online data. To address this problem, this paper proposes an online hash model to accommodate data coming in stream for online learning. Specifically, a new loss function is proposed to measure the similarity loss between a pair of data samples in hamming space. Then, a structured hash model is derived and optimized in a passive-aggressive way. Theoretical analysis on the upper bound of the cumulative loss for the proposed online hash model is provided. Furthermore, we extend our online hashing (OH) from a single model to a multimodel OH that trains multiple models so as to retain diverse OH models in order to avoid biased update. The competitive efficiency and effectiveness of the proposed online hash models are verified through extensive experiments on several large-scale data sets as compared with related hashing methods.
Regularized Locality Preserving Learning of Pre-Image Problem in Kernel Principal Component Analysis

In this paper, we address the pre-image problem in kernel principal component analysis (KPCA). The pre-image problem finds a pattern as the pre-image of a feature vector defined in the nonlinear principal component space produced by KPCA. Since the pre-image typically seldom exists in general, an approximate solution is appreciated. By posing a novel perspective, we find the pre-image with regularized locality preserving learning. Our approach achieves a unique solution, avoiding iteration and numerical instability. Significant superiority of the proposed novel algorithm is demonstrated by driving two applications, namely face denoising and occluded face reconstruction, as comparing with some existing well-known methods on pre-image learning
Latent embeddings for collective activity recognition

Rather than simply recognizing the action of a person individually, collective activity recognition aims to find out what a group of people is acting in a collective scene. Previous state-of-the-art methods using hand-crafted potentials in conventional graphical model which can only define a limited range of relations. Thus, the complex structural dependencies among individuals involved in a collective scenario cannot be fully modeled. In this paper, we overcome these limitations by embedding latent variables into feature space and learning the feature mapping functions in a deep learning framework. The embeddings of latent variables build a global relation containing person-group interactions and richer contextual information by jointly modeling broader range of individuals. Besides, we assemble attention mechanism during embedding for achieving more compact representations. We evaluate our method on three collective activity datasets, where we contribute a much larger dataset in this work. The proposed model has achieved clearly better performance as compared to the state-of-the-art methods in our experiments.