nils-herrmann/clean_scirepeval huggingface dataset:
Reliable and fast tracking of faces under varying pose

This paper presents a system that is able to track multiple faces under varying pose (tilted and rotated) reliably in real-time. The system consists of two interactive modules. The first module performs detection of face subject to rotations. The second does online learning based face tracking. A mechanism of switching between the two modules is embedded into the system to automatically decide the best strategy for reliable tracking. The mechanism enables smooth transit between the detection and tracking module when one of them gives no results or unreliable results. Results demonstrate that the system can make reliable real-time tracking of multiple faces in complex background under out-of-plane rotation, up to 90 degree tilting, fast nonlinear motion, partial occlusion, large scale changes, and camera motion
Acquisition of Face Depth Information from Near Infrared Images

This paper proposes a method for acquiring face depth information directly from near infrared (NIR) images, using statistical learning. To perform such learning, ground truth NIR images and range data are captured. A method of alignment between the two image modalities is proposed. By constructing the low dimensional face subspaces of NIR images and depth maps, the raw data are projected into respective subspaces. The mapping between the two subspaces is learned. The experiment substantiates the accuracy of the depth recovered and the economy of time and memory consumed.
Detecting individual in crowd with moving feature's structure consistency

In this paper, we present a method for detecting individuals in crowd by clustering a group of feature points belonging to the same person. In our approach, a feature point is considered to contain three attributes: the motion trajectory in video sequence, the sparse local appearance around point in current frame, and the structure relationship with body center related with local appearance. We exploit these attributes to cluster them appearing on the same individual to achieve detection purpose. The algorithm does not require observing entire human body and could discriminate different individuals under overlap. Our experiments show that this approach advances the performance of detecting individuals in crowds.
Face liveness detection by exploring multiple scenic clues

Liveness detection is an indispensable guarantee for reliable face recognition, which has recently received enormous attention. In this paper we propose three scenic clues, which are non-rigid motion, face-background consistency and imaging banding effect, to conduct accurate and efficient face liveness detection. Non-rigid motion clue indicates the facial motions that a genuine face can exhibit such as blinking, and a low rank matrix decomposition based image alignment approach is designed to extract this non-rigid motion. Face-background consistency clue believes that the motion of face and background has high consistency for fake facial photos while low consistency for genuine faces, and this consistency can serve as an efficient liveness clue which is explored by GMM based motion detection method. Image banding effect reflects the imaging quality defects introduced in the fake face reproduction, which can be detected by wavelet decomposition. By fusing these three clues, we thoroughly explore sufficient clues for liveness detection. The proposed face liveness detection method achieves 100% accuracy on Idiap print-attack database and the best performance on self-collected face anti-spoofing database.
A convolutional neural network combined with aggregate channel feature for face detection

Face detection has been studied intensively over the past several decades and achieved great improvements via convolutional neural network (CNN) which has greatly improved the performance in image classification and object detection. In this paper, similar to the idea of R-CNN , we present a new method that combines the aggregate channel features (ACF)  and CNN for face detection. The proposed method uses ACF to select the possible human face regions and then trains a CNN model to filter out non-face candidates. Then we merge the results of ACF and CNN to get the final detection window(s). Evaluations on two popular face detection benchmark datasets show that our method outperforms the ACF method and has achieved competitive performance against the state-of-the-art algorithms.
Separating Specular and Diffuse Reflection Components in the HSI Color Space

In this paper, we propose to separate diffuse and specular reflection components for color images in the HSI color space. Under white illumination, pixels with the same diffuse chromaticity have the same hue. Meanwhile, specular pixels have lower saturations than the diffuse ones. Based on these properties, separating reflection components can be achieved by adjusting saturations of specular pixels to the values of diffuse-only pixels with the same diffuse chromaticity. We employ a region-growing algorithm to locate adjacent pixels with similar diffuse chromaticities. Then, the separation of reflection components is achieved byfinding the optimal saturation in each connected region. The experimental results demonstrate that the proposed method is more effective to separate reflection components than the state-of-the-art methods.
The HFB Face Database for Heterogeneous Face Biometrics research

A face database, composed of visual (VIS), near infrared (NIR) and three-dimensional (3D) face images, is collected. Called the HFB face database, it is released now to promote research and development of heterogeneous face biometrics (HFB). This release of version 1 contains a total of 992 images from 100 subjects; there are 4 VIS, 4 NIR, and 1 or 2 3D face images per subject. In this paper, we describe the apparatuses, environments and procedure of the data collection and present baseline performances of the standard PCA and LDA methods on the database.
Advanced guided vehicle: distributed system

This paper focuses on the construction of an advanced guided vehicle in the following areas: localization process, motion control process and the integration of the overall system, vision processing, and path planning. The localization process is to determine the position and orientation of the vehicle continuously during the running of the vehicle. The motion control process commands the motor to drive the vehicle to the desired position. Effort was put into improving the estimation and driving for upto expectation accuracy. The integration task aims at corporating the various tasks together for the completion of the project. Vision processing is to process the image captured by the CCD camera to determine the obstacle information in front of the vehicle. The path planning process is to define the path for the vehicle to travel.
Multi-camera Trajectory Mining: Database and Evaluation

In recent years, large-scale video search and mining has been an active research area. Exploring the trajectory of pedestrian of interest in non-overlapping multi-camera network, namely the trajectory mining, is very useful for visual surveillance and criminal investigation. The trajectory mentioned in our work describes the transition of pedestrian among cameras from a macroscopic perspective which is different from the concept in conventional tracking field. In this paper, we collect a database called TMin to promote research and development of trajectory mining. This release of Version 1 contains 1680 images from 30 subjects, all the images are extracted from 6 surveillance videos over two hours, and each subject appears in at least two different cameras. We describe the apparatuses, environments and procedure of the data collection and present baseline performance on the TMin database.
A Random Field Model for Improved Feature Extraction and Tracking

This paper presents a novel method for illumination-invariant and contrast preserving feature extraction, aimed at improving performance of tracking under complex light condition. Features to be extracted are represented as a weight field. An energy function of the field is defined as an approximate variance in robust statistics. A simple non-linear iterative rule is derived to compute the optimal field. The optimal field is shown to be invariant to global illumination switching, and preserving target/background contrast. We incorporate the feature extraction method into a mean-shift tracker and this achieves reliable results on real-world sequences in complex scenes and varying illumination.
Online Visual Tracking Using Temporally Coherent Part Cluster

Recent advances in visual tracking have focused on handling deformations and occlusions using the part-based appearance model. However, it remains a challenge to come up with a reliable target representation using local parts, and hence existing trackers continue to face drifting problems. To deal with this challenge, we propose a robust online model, formulating the tracking task as a problem of identifying Temporally Coherent Part (TCP) clusters. Specifically, we pose the TCP clusters identification task as a dense neighborhoods searching problem using a relational hyper graph in which the relationship among multiple temporal local parts is encoded as the affinity value of a hyper edge connecting them. Such high-order relations ships among multiple local parts across the temporal domain make our tracker more robust towards deformations and occlusions. Extensive experiments on various challenging video sequences demonstrate that our TCP-based method performs better than the state-of-the-art methods.
Learning Stacked Image Descriptor for Face Recognition

Learning-based face descriptors have constantly improved the face recognition performance. Compared with the hand-crafted features, learning-based features are considered to be able to exploit information with better discriminative ability for specific tasks. Motivated by the recent success of deep learning, in this paper, we extend the original shallow face descriptors to deep discriminant face features by introducing a stacked image descriptor (SID). With deep structure, more complex facial information can be extracted and the discriminant and compactness of feature representation can be improved. The SID is learned in a forward optimization way, which is computational efficient compared with deep learning. Extensive experiments on various face databases are conducted to show that SID is able to achieve high face recognition performance with compact face representation, compared with other state-of-the-art descriptors.
Exploring Structural Information and Fusing Multiple Features for Person Re-identification

Recently, methods with learning procedure have been widely used to solve person re-identification (re-id) problem. However, most existing databases for re-id are smallscale, therefore, over-fitting is likely to occur. To further improve the performance, we propose a novel method by fusing multiple local features and exploring their structural information on different levels. The proposed method is called Structural Constraints Enhanced Feature Accumulation (SCEFA). Three local features (i.e., Hierarchical Weighted Histograms (HWH), Gabor Ternary Pattern HSV (GTP-HSV), Maximally Stable Color Regions (MSCR)) are used. Structural information of these features are deeply explored in three levels: pixel, blob, and part. The matching algorithms corresponding to the features are also discussed. Extensive experiments conducted on three datasets: VIPeR, ETHZ and our own challenging dataset MCSSH, show that our approach outperforms stat-of-the-art methods significantly.
2D–3D face matching using CCA

In recent years, 3D face recognition has obtained much attention. Using 2D face image as probe and 3D face data as gallery is an alternative method to deal with computation complexity, expensive equipment and fussy pretreatment in 3D face recognition systems. In this paper we propose a learning based 2D-3D face matching method using the CCA to learn the mapping between 2D face image and 3D face data. This method makes it possible to match the on-site 2D face image with enrolled 3D face data. Our 2D-3D face matching method decreased the computation complexity drastically compared to the conventional 3D-3D face matching while keeping relative high recognition rate. Furthermore, to simplify the mapping between 2D face image and 3D face data, a patch based strategy is proposed to boost the accuracy of matching. And the kernel method is also evaluated to reveal the non-linear relationship. The experiment results show that CCA based method has good performance and patch based method has significant improvement compared to the holistic method.
Color Models and Weighted Covariance Estimation for Person Re-identification

Due to illumination changes, partial occlusions, and object scale differences, person re-identification over disjoint camera views becomes a challenging problem. To address this problem, a variety of image representations have been put forward. In this paper, the illumination invariance and distinctiveness of different color models including the proposed color model are firstly evaluated. Since color distribution is robust to image scales and partial occlusions, color distributions based on different color models are then calculated and fused in the stage of feature extraction. Different color models obtain robustness to different types of illumination and thus fusing them can compensate each other and contribute to better performance. In the stage of feature matching, a weighted KISSME is presented to learn a better distance metric than the original KISSME. Experimental results demonstrate its feasibility and effectiveness. Finally, image pairs are matched based on the learned distance metric. Experiments conducted on two public benchmark datasets (VIPeR and PRID 450S) show that the proposed algorithm outperforms the state-of-the-art methods.
Deep person re-identification with improved embedding and efficient training

Person re-identification task has been greatly boosted by deep convolutional neural networks (CNNs) in recent years. The core of which is to enlarge the inter-class distinction as well as reduce the intra-class variance. However, to achieve this, existing deep models prefer to adopt image pairs or triplets to form verification loss, which is inefficient and unstable since the number of training pairs or triplets grows rapidly as the number of training data grows. Moreover, their performance is limited since they ignore the fact that different dimension of embedding may play different importance. In this paper, we propose to employ identification loss with center loss to train a deep model for person re-identification. The training process is efficient since it does not require image pairs or triplets for training while the inter-class distinction and intra-class variance are well handled. To boost the performance, a new feature reweighting (FRW) layer is designed to explicitly emphasize the importance of each embedding dimension, thus leading to an improved embedding. Experiments 1 on several benchmark datasets have shown the superiority of our method over the state-of-the-art alternatives on both accuracy and speed.
Texture classification using wavelet decomposition with Markov random field models

We present a new scheme to classify different textures. In the past years, wavelet decomposition has been used in texture classification. The usual features of classification are energy and entropy. In this paper, we propose a scheme to use wavelet decomposition with Markov random field models, the parameters of each Markov random field models are used as features in texture classification. Thus we can analyze the textures with Markov random field models on different scales with the wavelet decomposition.
Stepwise Correlation metric based Discriminant Analysis and multi-probe images fusion for face recognition

Face recognition is a great challenge in practice. Subspace learning method is one of the dominant methods and has achieved great success in face recognition area. In subspace learning, many researches have found that correlation similarity (e.g. cosine distance) usually achieves better classification results than L2 distance with nearest neighbor (NN) classifier in Euclidean space. However, in traditional methods, most of them are devoted to optimize the objective function based on L2 distance, which is not coincident with the classification rule. It is reasonable to obtain better results by optimizing the objective function with correlation metric directly. In this paper, following traditional linear discriminant analysis (LDA), we redefine the between and with-in class scatter with correlation metric and propose an efficient Stepwise Correlation metric based Discriminant Analysis (SCDA) method to derive the sub-optimal discriminant subspace to be classified with correlation similarity. Moreover, we propose a novel weighted fusion mechanism to learn the optimal combination of multi-probe images to be classified. Extensive experiments on PIE and extended Yale-B databases validate the effectiveness of SCDA and the learning based weighted image fusion method.
Dependence-Aware Feature Coding for Person Re-Identification

In this letter, we focus on how to boost the performance of person re-identification by exploring the discriminative information among person pairs. A novel dependence-aware feature coding framework is proposed for this task. Specifically, we employ the Hilbert–Schmidt independence criterion as the discriminative term, which is to explore the dependence between different kinds of person pairs, i.e., the same person pairs should be dependence maximized, while the different ones should be dependence minimized. Theoretical discussion and analysis on the convexity of the proposed constraint, as well as the convergence of our algorithm, are provided. Experimental results on two benchmark datasets have demonstrated the advantages of our method over the state-of-the-art alternatives.
Online Multiple Instance Joint Model for Visual Tracking

Although numerous online learning strategies have been proposed to handle the appearance variation in visual tracking, the existing methods just perform well in certain cases since they lack effective appearance learning mechanism. In this paper, a joint model tracker (JMT) is presented, which consists of a generative model based on Multiple Subspaces and a discriminative model based on improved Multiple Instance Boosting (MIBoosting). The generative model utilizes a series of local constructed subspaces to update the Multiple Subspaces model and considers the energy dissipation of dimension reduction in updating step. The discriminative model adopts the Gaussian Mixture Model (GMM) to estimate the posterior probability of the likelihood function. These two parts supervise each other to update in multiple instance way which helps our tracker recover from drift. Extensive experiments on various databases validate the effectiveness of our proposed method over other state-of-the-art trackers.
Coupled Spectral Regression for matching heterogeneous faces

Face recognition algorithms need to deal with variable lighting conditions. Near infrared (NIR) image based face recognition technology has been proposed to effectively overcome this difficulty. However, it requires that the enrolled face images be captured using NIR images whereas many applications require visual (VIS) images for enrollment templates. To take advantage of NIR face images for illumination-invariant face recognition and allow the use of VIS face images for enrollment, we encounter a new face image pattern recognition problem, that is, heterogeneous face matching between NIR versus VIS faces. In this paper, we present a subspace learning framework named Coupled Spectral Regression (CSR) to solve this challenge problem of coupling the two types of face images and matching between them. CSR first models the properties of different types of data separately and then learns two associated projections to project heterogeneous data (e.g. VIS and NIR) respectively into a discriminative common subspace in which classification is finally performed. Compared to other existing methods, CSR is computational efficient, benefiting from the efficiency of spectral regression and has better generalization performance. Experimental results on VIS-NIR face database show that the proposed CSR method significantly outperforms the existing methods.
Learning sparse feature for eyeglasses problem in face recognition

Occlusion of eyeglasses, and strong specular reflections on eyeglasses (especially in near infrared (NIR) images), can deteriorate face recognition performance. In this paper, we present a novel method to overcome these problems. The proposed method applies the sparse representation (SR) technique in a local feature space so as to be more tolerant to mis-alignment and abnormal specular pixel values. The SR face features are further transformed by using discriminant analysis. These lead to a good balance between efficiency and robustness. Extensive experiments on a large NIR face database containing 292 persons with/without eyeglasses show the superiority of the proposed method compared with state-of-the-art methods.
A surveillance video analysis and storage scheme for scalable synopsis browsing

We propose a framework for efficient storing and scalable browsing of surveillance video based on video synopsis. Our framework employs a novel synopsis analysis scheme named Detail-based video synopsis to generate a set of object flags to store and browse surveillance video synopsis. The main contributions of our work are: 1) highlighting important contents of surveillance video; 2) improving the storage efficiency of original video and synopsis video; 3) realizing multi-scale scalable browsing of synopsis video while reserving essential information. The experiments of implementing the framework are shown compared with the previous independent storage method of original video and synopsis video.
Regularized Transfer Boosting for Face Detection Across Spectrum

This letter addresses the problem of face detection in multispectral illuminations. Face detection in visible images has been well addressed based on the large scale training samples. For the recently emerging multispectral face biometrics, however, the face data is scarce and expensive to collect, and it is usually short of face samples to train an accurate face detector. In this letter, we propose to tackle the issue of multispectral face detection by combining existing large scale visible face images and a few multispectral face images. We cast the problem of face detection across spectrum into the transfer learning framework and try to learn the robust multispectral face detector by exploring relevant knowledge from visible data domain. Specifically, a novel Regularized Transfer Boosting algorithm named R-TrBoost is proposed, with features of weighted loss objective and manifold regularization. Experiments are performed with face images of two spectrums, 850 nm and 365 nm, and the results show significant improvement on multispectral face detection using the proposed algorithm.
Face shape recovery from a single image using CCA mapping between tensor spaces

In this paper, we propose a new approach for face shape recovery from a single image. A single near infrared (NIR) image is used as the input, and a mapping from the NIR tensor space to 3D tensor space, learned by using statistical learning, is used for the shape recovery. In the learning phase, the two tensor models are constructed for NIR and 3D images respectively, and a canonical correlation analysis (CCA) based multi-variate mapping from NIR to 3D faces is learned from a given training set of NIR-3D face pairs. In the reconstruction phase, given an NIR face image, the depth map is computed directly using the learned mapping with the help of tensor models. Experimental results are provided to evaluate the accuracy and speed of the method. The work provides a practical solution for reliable and fast shape recovery and modeling of 3D objects.