nils-herrmann/clean_scirepeval huggingface dataset:
Robust Deformable and Occluded Object Tracking With Dynamic Graph

While some efforts have been paid to handle deformation and occlusion in visual tracking, they are still great challenges. In this paper, a dynamic graph-based tracker (DGT) is proposed to address these two challenges in a unified framework. In the dynamic target graph, nodes are the target local parts encoding appearance information, and edges are the interactions between nodes encoding inner geometric structure information. This graph representation provides much more information for tracking in the presence of deformation and occlusion. The target tracking is then formulated as tracking this dynamic undirected graph, which is also a matching problem between the target graph and the candidate graph. The local parts within the candidate graph are separated from the background with Markov random field, and spectral clustering is used to solve the graph matching. The final target state is determined through a weighted voting procedure according to the reliability of part correspondence, and refined with recourse to a foreground/background segmentation. An effective online updating mechanism is proposed to update the model, allowing DGT to robustly adapt to variations of target structure. Experimental results show improved performance over several state-of-the-art trackers, in various challenging scenarios.
Relaxation labeling using Lagrange-Hopfield method

Relaxation labeling (RL) is a class of parallel iterative numerical procedures which use contextual constraints to reduce ambiguities in image analysis. A novel RL algorithm is proposed. RL is posed as a constrained optimization problem and the solution is found by using the Lagrangian multiplier method and a technique used in the graded Hopfield neural network. In terms of the optimized objective value, the algorithm performs almost as well as simulated annealing, as shown by the experimental results. Also, the resulting algorithm is fully distributive and is suitable for analog implementation.
Enhanced Local Gradient Order Features and Discriminant Analysis for Face Recognition

Robust descriptor-based subspace learning with complex data is an active topic in pattern analysis and machine intelligence. A few researches concentrate the optimal design on feature representation and metric learning. However, traditionally used features of single-type, e.g., image gradient orientations (IGOs), are deficient to characterize the complete variations in robust and discriminant subspace learning. Meanwhile, discontinuity in edge alignment and feature match are not been carefully treated in the literature. In this paper, local order constrained IGOs are exploited to generate robust features. As the difference-based filters explicitly consider the local contrasts within neighboring pixel points, the proposed features enhance the local textures and the order-based coding ability, thus discover intrinsic structure of facial images further. The multimodal features are automatically fused in the most discriminant subspace. The utilization of adaptive interaction function suppresses outliers in each dimension for robust similarity measurement and discriminant analysis. The sparsity-driven regression model is modified to adapt the classification issue of the compact feature representation. Extensive experiments are conducted by using some benchmark face data sets, e.g., of controlled and uncontrolled environments, to evaluate our new algorithm.
Aggregate channel features for multi-view face detection

Face detection has drawn much attention in recent decades since the seminal work by Viola and Jones. While many subsequences have improved the work with more powerful learning algorithms, the feature representation used for face detection still can't meet the demand for effectively and efficiently handling faces with large appearance variance in the wild. To solve this bottleneck, we borrow the concept of channel features to the face detection domain, which extends the image channel to diverse types like gradient magnitude and oriented gradient histograms and therefore encodes rich information in a simple form. We adopt a novel variant called aggregate channel features, make a full exploration of feature design, and discover a multi-scale version of features with better performance. To deal with poses of faces in the wild, we propose a multi-view detection approach featuring score re-ranking and detection adjustment. Following the learning pipelines in Viola-Jones framework, the multi-view face detector using aggregate channel features shows competitive performance against state-of-the-art algorithms on AFW and FDDB test-sets, while runs at 42 FPS on VGA images.
Guest Editorial Special Issue on Biometric Spoofing and Countermeasures

While biometrics technology has created new solutions to person authentication and has evolved to play a critical role in personal, national, and global security, the potential for the technology to be fooled or spoofed is now widely acknowledged. For example, fingerprint verification systems can be spoofed with a synthetic material, such as gelatine, inscribed with the fingerprint ridges of an enrolled individual. Iris and face recognition systems are vulnerable to printed photographs or video sequences of an enrolled user’s eye or face. Speaker recognition systems can be spoofed through the use of replayed, synthesized, or converted speech.
Discriminant image filter learning for face recognition with local binary pattern like representation

Local binary pattern (LBP) and its variants are effective descriptors for face recognition. The traditional LBP like features are extracted based on the original pixel or patch values of images. In this paper, we propose to learn the discriminative image filter to improve the discriminant power of the LBP like feature. The basic idea is after the image filtering with the learned filter, the difference of pixel difference vectors (PDVs) between the images from the same person is consistent and the difference between the images from different persons is enlarged. In this way, the LBP like features extracted from the filtered images are considered to be more discriminant than those extracted from the original images. Moreover, a coupled discriminant image filters learning method is proposed to deal with the heterogenous face images matching problem by reducing the feature gap between the heterogeneous images. Experiments on FERET, FRGC and a VIS-NIR heterogeneous face databases validate the effectiveness of our proposed image filter learning method combined with LBP like features.
Large-Scale Isolated Gesture Recognition Using a Refined Fused Model Based on Masked Res-C3D Network and Skeleton LSTM

In this paper, we focus on large-scale isolated gesture recognition for RGB-D videos. We develop a novel ensemble method to explore deep spatio-temporal features using 3D Convolutional Neural Networks (CNNs) with residual architecture (Res-C3D) and build a time-series model with skeleton information based on Long Short Term Memory network (LSTM). First, relative positions and angles of different keypoints are extracted and used to build time-series model in LSTM. Obtaining the skeleton information (keypoints) of body and reserving arm regions with discarding other parts, masked Res-C3D is obtained, which decreases the effect of the background and other variations, as gestures are mainly derived from the arm or hand movements. Moreover, the weights of each voting sub-classifier being of advantage to a certain class in our ensemble model are adaptively obtained by training in place of fixed weights. Our experimental results show that the proposed method has obtained a state-of-the-art performance with accuracy 0.6842 in the IsoGD dataset.
Water Filling: Unsupervised People Counting via Vertical Kinect Sensor

People counting is one of the key components in video surveillance applications, however, due to occlusion, illumination, color and texture variation, the problem is far from being solved. Different from traditional visible camera based systems, we construct a novel system that uses vertical Kinect sensor for people counting, where the depth information is used to remove the affect of the appearance variation. Since the head is always closer to the Kinect sensor than other parts of the body, people counting task equals to find the suitable local minimum regions. According to the particularity of the depth map, we propose a novel unsupervised water filling method that can find these regions with the property of robustness, locality and scale-invariance. Experimental comparisons with mean shift and random forest on two databases validate the superiority of our water filling algorithm in people counting.
Multi-pedestrian detection in crowded scenes: A global view

Recent state-of-the-art algorithms have achieved good performance on normal pedestrian detection tasks. However, pedestrian detection in crowded scenes is still challenging due to the significant appearance variation caused by heavy occlusions and complex spatial interactions. In this paper we propose a unified probabilistic framework to globally describe multiple pedestrians in crowded scenes in terms of appearance and spatial interaction. We utilize a mixture model, where every pedestrian is assumed in a special subclass and described by the sub-model. Scores of pedestrian parts are used to represent appearance and quadratic kernel is used to represent relative spatial interaction. For efficient inference, multi-pedestrian detection is modeled as a MAP problem and we utilize greedy algorithm to get an approximation. For discriminative parameter learning, we formulate it as a learning to rank problem, and propose Latent Rank SVM for learning from weakly labeled data. Experiments on various databases validate the effectiveness of the proposed approach.
A benchmark study of large-scale unconstrained face recognition

Many efforts have been made in recent years to tackle the unconstrained face recognition challenge. For the benchmark of this challenge, the Labeled Faces in theWild (LFW) database has been widely used. However, the standard LFW protocol is very limited, with only 3,000 genuine and 3,000 impostor matches for classification. Today a 97% accuracy can be achieved with this benchmark, remaining a very limited room for algorithm development. However, we argue that this accuracy may be too optimistic because the underlying false accept rate may still be high (e.g. 3%). Furthermore, performance evaluation at low FARs is not statistically sound by the standard protocol due to the limited number of impostor matches. Thereby we develop a new benchmark protocol to fully exploit all the 13,233 LFW face images for large-scale unconstrained face recognition evaluation under both verification and open-set identification scenarios, with a focus at low FARs. Based on the new benchmark, we evaluate 21 face recognition approaches by combining 3 kinds of features and 7 learning algorithms. The benchmark results show that the best algorithm achieves 41.66% verification rates at FAR=0.1%, and 18.07% open-set identification rates at rank 1 and FAR=1%. Accordingly we conclude that the large-scale unconstrained face recognition problem is still largely unresolved, thus further attention and effort is needed in developing effective feature representations and learning algorithms. We thereby release a benchmark tool to advance research in this field.
CRAFT Objects from Images

Object detection is a fundamental problem in image understanding. One popular solution is the R-CNN framework  and its fast versions [14, 27]. They decompose the object detection problem into two cascaded easier tasks: 1) generating object proposals from images, 2) classifying proposals into various object categories. Despite that we are handling with two relatively easier tasks, they are not solved perfectly and there's still room for improvement. In this paper, we push the "divide and conquer" solution even further by dividing each task into two sub-tasks. We call the proposed method "CRAFT" (Cascade Regionproposal-network And FasT-rcnn), which tackles each task with a carefully designed network cascade. We show that the cascade structure helps in both tasks: in proposal generation, it provides more compact and better localized object proposals, in object classification, it reduces false positives (mainly between ambiguous categories) by capturing both inter-and intra-category variances. CRAFT achieves consistent and considerable improvement over the state-of the-art on object detection benchmarks like PASCAL VOC 07/12 and ILSVRC.
Learning Discriminative Features with Class Encoder

Deep neural networks usually benefit from unsupervised pre-training, e.g. auto-encoders. However, the classifier further needs supervised fine-tuning methods for good discrimination. Besides, due to the limits of full-connection, the application of auto-encoders is usually limited to small, well aligned images. In this paper, we incorporate the supervised information to propose a novel formulation, namely class-encoder, whose training objective is to reconstruct a sample from another one of which the labels are identical. Class-encoder aims to minimize the intra-class variations in the feature space, and to learn a good discriminative manifolds on a class scale. We impose the class-encoder as a constraint into the softmax for better supervised training, and extend the reconstruction on feature-level to tackle the parameter size issue and translation issue. The experiments show that the class-encoder helps to improve the performance on benchmarks of classification and face recognition. This could also be a promising direction for fast training of face recognition models.
Person-specific face tracking with online recognition

Person-specific face tracking is a challenging task for the trackers which only focus on the appearance of the target face, because distraction always happens and the identity is difficult to maintain. In this paper, we design a framework combining an off-line detector, an on-line tracker and an online recognizer to complete the tracking of person-specific face. Recognizer is the key component in our framework, because the most confident target face will be selected by the recognizer from the pool of detected and tracked faces. Since there is no much prior information about the identities available and the face poses change frequently in surveillance scenarios, accurate recognition is extremely difficult and an on-line formulation is required. In order to ensure the precision of identity recognition with different poses, we project the extracted features of faces to a latent space with the help of Canonical Correlation Analysis (CCA) technique, and then these projected features are incrementally trained using an on-line SVM (LASVM). Experimental results demonstrate that our person-specific face tracking outperforms several state-of-the-art face trackers.
Pedestrian Attribute Classification in Surveillance: Database and Evaluation

Attributes are helpful to infer high-level semantic knowledge of pedestrians, thus improving the performance of pedestrian tracking, retrieval, re-identification, etc. However, current pedestrian databases are mainly for the pedestrian detection or tracking application, and semantic attribute annotations related to pedestrians are rarely provided. In this paper, we construct an Attributed Pedestrians in Surveillance (APiS) database with various scenes. The APiS 1.0 database includes 3661 images with 11 binary and 2 multi-class attribute annotations. Moreover, we develop an evaluation protocol for researchers to evaluate pedestrian attribute classification algorithms. With the APiS 1.0 database, we present two baseline methods, one for binary attribute classification and the other for multi-class attribute classification. For binary attribute classification, we train AdaBoost classifiers with color and texture features, while for multi-class attribute classification, we adopt a weighted K Nearest Neighbors (KNN) classifier with color features. Finally, we report and discuss the baseline performance on the APiS 1.0 database following the proposed evaluation protocol.
Shared representation learning for heterogenous face recognition

After intensive research, heterogenous face recognition is still a challenging problem. The main difficulties are owing to the complex relationship between heterogenous face image spaces. The heterogeneity is always tightly coupled with other variations, which makes the relationship of heterogenous face images highly nonlinear. Many excellent methods have been proposed to model the nonlinear relationship, but they apt to overfit to the training set, due to limited samples. Inspired by the unsupervised algorithms in deep learning, this paper proposes a novel framework for heterogeneous face recognition. We first extract Gabor features at some localized facial points, and then use Restricted Boltzmann Machines (RBMs) to learn a shared representation locally to remove the heterogeneity around each facial point. Finally, the shared representations of local RBMs are connected together and processed by PCA. Near infrared (NIR) to visible (VIS) face recognition problem and two databases are selected to evaluate the performance of the proposed method. On CASIA HFB database, we obtain comparable results to state-of-the-art methods. On a more difficult database, CASIA NIR-VIS 2.0, we outperform other methods significantly.
Regularized active shape model for shape alignment

Active shape model (ASM) statistically represents a shape by a set of well-defined landmark points and models object variations using principal component analysis (PCA). However, the extracted shape contour modeled by PCA is still unsmooth when the shape has a large variation compared with the mean shape. In this paper, we propose a regularized ASM (R-ASM) model for shape alignment. During training stage, we present a regularized shape subspace on which image smoothness constraint is imposed, such that the learned components to model shape variations should not only minimize reconstruction error but also obey smoothness principle. During searching stage, a coarse-to-fine parameter adjustment strategy is performed under Bayesian inference. It makes a desired shape smoother and more robust to local noise. Lastly, an inner shape is introduced to further regularize search results. Experiments on face alignment demonstrate the efficiency and effectiveness of our proposed approach.
Robust 3D Morphable Model Fitting by Sparse SIFT Flow

3D Morph able Model (3DMM) has been widely used in face analysis for many years. The most challenging part of 3DMM is to find the correspondences between 3D points and 2D pixels. Existing methods only use key points, edges, specular highlights and image pixels to complete the task, which are not accurate or robust. This paper proposes a new algorithm called Sparse SIFT Flow (SSF) to improve the reconstruction accuracy. We mark a set of salient points to control the shape of facial components and use SSF to find their corresponding pixels on the input image. We also incorporate SSF into Multi-Features Framework to construct a robust 3DMM fitting algorithm. Compared with the state-of-the art, our approach significantly improves the fitting results in facial component area.
Gabor volume based local binary pattern for face representation and recognition

This paper presents a novel face representation and recognition approach. The face image is first decomposed by multi-scale and multi-orientation Gabor filters and local binary pattern (LBP) analysis is then applied on the derived Gabor magnitude responses. Different from (W.C. Zhang et al., 2005), the present method not only describes the neighboring relationship in spatial domain, but also exploit those between different scales (frequency) and orientations. Specifically, we first reformulate the Gabor magnitude responses as a 3rd-order volume and then apply LBP analysis on three orthogonal planes of the Gabor volume, named GV-LBP-TOP in short, in a hope to encode sufficient information for face representation. Further, a computationally effective version, E-GV-LBP, is proposed to depict the neighboring changes in spatial, frequency and orientation domains simultaneously. Finally, the weighted histogram intersection metric is utilized to measure the dissimilarity of faces. Experimental results on FERET and FRGC ver 2.0 databases show the significant advantages of the proposed method.
When Face Recognition Meets with Deep Learning: An Evaluation of Convolutional Neural Networks for Face Recognition

Deep learning, in particular Convolutional Neural Network (CNN), has achieved promising results in face recognition recently. However, it remains an open question: why CNNs work well and how to design a 'good' architecture. The existing works tend to focus on reporting CNN architectures that work well for face recognition rather than investigate the reason. In this work, we conduct an extensive evaluation of CNN-based face recognition systems (CNN-FRS) on a common ground to make our work easily reproducible. Specifically, we use public database LFW (Labeled Faces in the Wild) to train CNNs, unlike most existing CNNs trained on private databases. We propose three CNN architectures which are the first reported architectures trained using LFW data. This paper quantitatively compares the architectures of CNNs and evaluates the effect of different implementation choices. We identify several useful properties of CNN-FRS. For instance, the dimensionality of the learned features can be significantly reduced without adverse effect on face recognition accuracy. In addition, a traditional metric learning method exploiting CNN-learned features is evaluated. Experiments show two crucial factors to good CNN-FRS performance are the fusion of multiple CNNs and metric learning. To make our work reproducible, source code and models will be made publicly available.
Convolutional Channel Features

Deep learning methods are powerful tools but often suffer from expensive computation and limited flexibility. An alternative is to combine light-weight models with deep representations. As successful cases exist in several visual problems, a unified framework is absent. In this paper, we revisit two widely used approaches in computer vision, namely filtered channel features and Convolutional Neural Networks (CNN), and absorb merits from both by proposing an integrated method called Convolutional Channel Features (CCF). CCF transfers low-level features from pre-trained CNN models to feed the boosting forest model. With the combination of CNN features and boosting forest, CCF benefits from the richer capacity in feature representation compared with channel features, as well as lower cost in computation and storage compared with end-to-end CNN methods. We show that CCF serves as a good way of tailoring pre-trained CNN models to diverse tasks without fine-tuning the whole network to each task by achieving state-of-the-art performances in pedestrian detection, face detection, edge detection and object proposal generation.
Low-resolution face recognition via Simultaneous Discriminant Analysis

Low resolution (LR) is an important issue when handling real world face recognition problems. The performance of traditional recognition algorithms will drop drastically due to the loss of facial texture information in original high resolution (HR) images. To address this problem, in this paper we propose an effective approach named Simultaneous Discriminant Analysis (SDA). SDA learns two mappings from LR and HR images respectively to a common subspace where discrimination property is maximized. In SDA, (1) the data gap between LR and HR is reduced by mapping into a common space; and (2) the mapping is designed for preserving most discriminative information. After that, the conventional classification method is applied in the common space for final decision. Extensive experiments are conducted on both FERET and Multi-PIE, and the results clearly show the superiority of the proposed SDA over state-of-the-art methods.