nils-herrmann/clean_scirepeval huggingface dataset:
Multicamera Joint Video Synopsis

Due to an increasing demand for video surveillance, there is an explosive growth of surveillance videos, which causes a big challenge in video storage, browsing, and retrieval. The video synopsis technique is thus developed to extract and rearrange the moving objects so as to handle the massive video browsing challenge. However, the traditional video synopsis (TVS) method only considers the processing videos captured by a single camera, ignoring object interactions in multicamera videos. To address this issue, we propose a novel multicamera joint video synopsis (JVS) algorithm for multicamera surveillance videos. First, a key time stamp (KTS) selection method is designed to find an object's appearing, merging, splitting, and disappearing moments in the frame sequence, called tube, of that object. Second, tubes are rearranged by minimizing a global energy function that involves the overall camera views. Compared with the energy function used in TVS, the proposed global energy function considers the chronological orders of tubes not only in the same camera view but also among different camera views. Moreover, the chronological disorder cost term is formulated based on the KTS labels, and improved by considering the visual similarity between two tubes. Finally, the multicamera synopsis videos are separately generated by stitching together the globally rearranged tubes and background images of the same camera view. Extensive experiments show that the proposed JVS method is better than the traditional single-camera video synopsis method in preserving the chronological orders of moving objects among multicamera synopsis videos.
Face Recognition by Exploring Information Jointly in Space, Scale and Orientation

Information jointly contained in image space, scale and orientation domains can provide rich important clues not seen in either individual of these domains. The position, spatial frequency and orientation selectivity properties are believed to have an important role in visual perception. This paper proposes a novel face representation and recognition approach by exploring information jointly in image space, scale and orientation domains. Specifically, the face image is first decomposed into different scale and orientation responses by convolving multiscale and multi-orientation Gabor filters. Second, local binary pattern analysis is used to describe the neighboring relationship not only in image space, but also in different scale and orientation responses. This way, information from different domains is explored to give a good face representation for recognition. Discriminant classification is then performed based upon weighted histogram intersection or conditional mutual information with linear discriminant analysis techniques. Extensive experimental results on FERET, AR, and FRGC ver 2.0 databases show the significant advantages of the proposed method over the existing ones.
Towards Pose Robust Face Recognition

Most existing pose robust methods are too computational complex to meet practical applications and their performance under unconstrained environments are rarely evaluated. In this paper, we propose a novel method for pose robust face recognition towards practical applications, which is fast, pose robust and can work well under unconstrained environments. Firstly, a 3D deformable model is built and a fast 3D model fitting algorithm is proposed to estimate the pose of face image. Secondly, a group of Gabor filters are transformed according to the pose and shape of face image for feature extraction. Finally, PCA is applied on the pose adaptive Gabor features to remove the redundances and Cosine metric is used to evaluate the similarity. The proposed method has three advantages: (1) The pose correction is applied in the filter space rather than image space, which makes our method less affected by the precision of the 3D model, (2) By combining the holistic pose transformation and local Gabor filtering, the final feature is robust to pose and other negative factors in face recognition, (3) The 3D structure and facial symmetry are successfully used to deal with self-occlusion. Extensive experiments on FERET and PIE show the proposed method outperforms state-of-the-art methods significantly, meanwhile, the method works well on LFW.
Online Principal Background Selection for Video Synopsis

Video synopsis provides a means for fast browsing of activities in video. Principal background selection (PBS) is an important step in video synopsis. Existing methods make PBS in an offline way and at a high memory cost. In this paper we propose a novel background selection method, ``online principal background selection'' (OPBS). The OPBS selects n principal backgrounds from N backgrounds in an online fashion with a low memory cost, making it possible to build an efficient online video synopsis system. Another advantage is that, with OPBS, the selected backgrounds are related to not only background changes over time but also video activities. Experimental results demonstrate the advantages of the proposed OPBS.
Real Time People Tracking and Counting in Visual Surveillance

This work presents a real time system to count human beings passed through a doorway. To get the capacity, there were two effective methods. Firstly, using the characteristic of LAB color space, the calculation became simple and people's head was detected easily. Secondly, local features of moving targets were proposed to tracking the human accurately and quickly even the human turned around. With this algorithm, the system counted the passed human accurately without detecting the target in every frame. This system produces good tracking and counting results at a frame rate of 15-20 fps for image size of 320times240, so it can be very useful in some real time visual surveillance
Moving Cast Shadow Removal Based on Local Descriptors

Moving cast shadow removal is an important yet difficult problem in video analysis and applications. This paper presents a novel algorithm for detection of moving cast shadows, that based on a local texture descriptor called Scale Invariant Local Ternary Pattern (SILTP). An assumption is made that the texture properties of cast shadows bears similar patterns to those of the background beneath them. The likelihood of cast shadows is derived using information in both color and texture. An online learning scheme is employed to update the shadow model adaptively. Finally, the posterior probability of cast shadow region is formulated by further incorporating prior contextual constrains using a Markov Random Field (MRF) model. The optimal solution is found using graph cuts. Experimental results tested on various scenes demonstrate the robustness of the algorithm.
Person re-identification by Local Maximal Occurrence representation and metric learning

Person re-identification is an important technique towards automatic search of a person's presence in a surveillance video. Two fundamental problems are critical for person re-identification, feature representation and metric learning. An effective feature representation should be robust to illumination and viewpoint changes, and a discriminant metric should be learned to match various person images. In this paper, we propose an effective feature representation called Local Maximal Occurrence (LOMO), and a subspace and metric learning method called Cross-view Quadratic Discriminant Analysis (XQDA). The LOMO feature analyzes the horizontal occurrence of local features, and maximizes the occurrence to make a stable representation against viewpoint changes. Besides, to handle illumination variations, we apply the Retinex transform and a scale invariant texture operator. To learn a discriminant metric, we propose to learn a discriminant low dimensional subspace by cross-view quadratic discriminant analysis, and simultaneously, a QDA metric is learned on the derived subspace. We also present a practical computation method for XQDA, as well as its regularization. Experiments on four challenging person re-identification databases, VIPeR, QMUL GRID, CUHK Campus, and CUHK03, show that the proposed method improves the state-of-the-art rank-1 identification rates by 2.2%, 4.88%, 28.91%, and 31.55% on the four databases, respectively.
Spatio-temporal LBP Based Moving Object Segmentation in Compressed Domain

With the increasing amount of surveillance data, moving object segmentation in the compressed domain has drawn broad attention from both academy and industry. In this paper, we propose a novel moving object segmentation method towards H.264 compressed surveillance videos. First, the motion vectors (MV) are accumulated and filtered to achieve reliable motion information. Second, considering the spatial and temporal correlations among adjacent blocks, spatio-temporal Local Binary Pattern (LBP) features of MVs are extracted to obtain coarse and initial object regions. Finally, a coarse-to-fine segmentation algorithm of boundary modification is conducted based on the DCT coefficients. The experimental results validate that the proposed method not only can extract fairly accurate objects in compressed video, but also has a relatively low computational complexity.
A face antispoofing database with diverse attacks

Face antispoofing has now attracted intensive attention, aiming to assure the reliability of face biometrics. We notice that currently most of face antispoofing databases focus on data with little variations, which may limit the generalization performance of trained models since potential attacks in real world are probably more complex. In this paper we release a face antispoofing database which covers a diverse range of potential attack variations. Specifically, the database contains 50 genuine subjects, and fake faces are made from the high quality records of the genuine faces. Three imaging qualities are considered, namely the low quality, normal quality and high quality. Three fake face attacks are implemented, which include warped photo attack, cut photo attack and video attack. Therefore each subject contains 12 videos (3 genuine and 9 fake), and the final database contains 600 video clips. Test protocol is provided, which consists of 7 scenarios for a thorough evaluation from all possible aspects. A baseline algorithm is also given for comparison, which explores the high frequency information in the facial region to determine the liveness. We hope such a database can serve as an evaluation platform for future researches in the literature.
Cross-Modality Face Recognition via Heterogeneous Joint Bayesian

In many face recognition applications, the modalities of face images between the gallery and probe sets are different, which is known as heterogeneous face recognition. How to reduce the feature gap between images from different modalities is a critical issue to develop a highly accurate face recognition algorithm. Recently, joint Bayesian (JB) has demonstrated superior performance on general face recognition compared to traditional discriminant analysis methods like subspace learning. However, the original JB treats the two input samples equally and does not take into account the modality difference between them and may be suboptimal to address the heterogeneous face recognition problem. In this work, we extend the original JB by modeling the gallery and probe images using two different Gaussian distributions to propose a heterogeneous joint Bayesian (HJB) formulation for cross-modality face recognition. The proposed HJB explicitly models the modality difference of image pairs and, therefore, is able to better discriminate the same/different face pairs accurately. Extensive experiments conducted in the case of visible–near-infrared and ID photo versus spot face recognition problems show the superiority of the HJB over previous methods.
Solving the bas-relief ambiguity

Bas-relief ambiguity exists from two views in points correspondence. Solving this problem relies on either prior knowledge of the motion or further projection such as three views. It is shown in this paper that the unique solution exists from only one pair of correspondence with the prior knowledge of known rotation axis. The least square approach is derived to compute the rotation angle in the presence of noise. The advantage of such an approach is that no camera calibration is required. Further, we show that the least squares approach is statistically unbiased and produces stable results with noisy data.<<ETX>>
Normalized LDA for semi-supervised learning

Linear Discriminant Analysis (LDA) has been a popular method for feature extracting and face recognition. As a supervised method, it requires manually labeled samples for training, while making labeled samples is a time consuming and exhausting work. A semi-supervised LDA (SDA) has been proposed recently to enable training of LDA with partially labeled samples. In this paper, we first reformulate supervised LDA based on the normalized perspective of LDA. Then we show that such a reformulation is powerful for semi-supervised learning of LDA. We call this approach Normalized LDA, which uses total diversity to normalize intra-class diversity and aims to find projection directions that minimize normalized intra-class diversity. Although the Normalized LDA is identical to LDA in the supervised situation, a semi-supervised approach can be easily incorporated into its framework to make use of unlabeled samples to improve the performance in the learned subspace. Moreover, different with SDA which uses unlabeled samples to preserve neighboring relations, unlabeled samples in the Normalized LDA are used for a more accurate estimation of data space. Experiments of face recognition on the FRGC version 2 database and CMU PIE database demonstrate that the Normalized LDA outperforms SDA.
Relaxation labeling of Markov random fields

Using Markov random field (MRF) theory, a variety of computer vision problems can be modeled in terms of optimization based on the maximum a posteriori (MAP) criterion. The MAP configuration minimizes the energy of a posterior (Gibbs) distribution. When the label set is discrete, the minimization is combinatorial. This paper proposes to use the continuous relaxation labeling (RL) method for the minimization. The RL converts the original NP complete problem into one of polynomial complexity. Annealing may be combined into the RL process to improve the quality (globalness) of RL solutions. Performance comparison among four different RL algorithms is given.
Illumination Invariant Face Recognition Using Near-Infrared Images

Most current face recognition systems are designed for indoor, cooperative-user applications. However, even in thus-constrained applications, most existing systems, academic and commercial, are compromised in accuracy by changes in environmental illumination. In this paper, we present a novel solution for illumination invariant face recognition for indoor, cooperative-user applications. First, we present an active near infrared (NIR) imaging system that is able to produce face images of good condition regardless of visible lights in the environment. Second, we show that the resulting face images encode intrinsic information of the face, subject only to a monotonic transform in the gray tone; based on this, we use local binary pattern (LBP) features to compensate for the monotonic transform, thus deriving an illumination invariant face representation. Then, we present methods for face recognition using NIR images; statistical learning algorithms are used to extract most discriminative features from a large pool of invariant LBP features and construct a highly accurate face matching engine. Finally, we present a system that is able to achieve accurate and fast face recognition in practice, in which a method is provided to deal with specular reflections of active NIR lights on eyeglasses, a critical issue in active NIR image-based face recognition. Extensive, comparative results are provided to evaluate the imaging hardware, the face and eye detection algorithms, and the face recognition algorithms and systems, with respect to various factors, including illumination, eyeglasses, time lapse, and ethnic groups
High-fidelity Pose and Expression Normalization for face recognition in the wild

Pose and expression normalization is a crucial step to recover the canonical view of faces under arbitrary conditions, so as to improve the face recognition performance. An ideal normalization method is desired to be automatic, database independent and high-fidelity, where the face appearance should be preserved with little artifact and information loss. However, most normalization methods fail to satisfy one or more of the goals. In this paper, we propose a High-fidelity Pose and Expression Normalization (HPEN) method with 3D Morphable Model (3DMM) which can automatically generate a natural face image in frontal pose and neutral expression. Specifically, we firstly make a landmark marching assumption to describe the non-correspondence between 2D and 3D landmarks caused by pose variations and propose a pose adaptive 3DMM fitting algorithm. Secondly, we mesh the whole image into a 3D object and eliminate the pose and expression variations using an identity preserving 3D transformation. Finally, we propose an inpainting method based on Possion Editing to fill the invisible region caused by self occlusion. Extensive experiments on Multi-PIE and LFW demonstrate that the proposed method significantly improves face recognition performance and outperforms state-of-the-art methods in both constrained and unconstrained environments.
Nearest Feature Line: A Tangent Approximation

Nearest feature line (NFL) (S.Z. Li and J. Lu, 1999) is an efficient yet simple classification method for pattern recognition. This paper presents a theoretical analysis and interpretation of NFL from the perspective of manifold analysis, and explains the geometric nature of NFL based similarity measures. It is illustrated that NFL, nearest feature plane (NFP) and nearest feature space (NFS) are special cases of tangent approximation. Under the assumption of manifold, we introduce localized NFL (LNFL) and nearest feature spline (NFB) to further enhance classification ability and reduce computational complexity. The LNFL extends NFL's Euclidean distance to a manifold distance. And for NFB, feature lines are constructed along with a manifold's variation which is defined on a tangent bundle. The proposed methods are validated on a synthetic dataset and two standard face recognition databases (FRGC version 2 and FERET). Experimental results illustrate its efficiency and effectiveness.
Object detection by labeling superpixels

Object detection is often conducted by object proposal generation and classification sequentially. This paper handles object detection in a superpixel oriented manner instead of the proposal oriented. Specially, this paper takes object detection as a multi-label superpixel labeling problem by minimizing an energy function. It uses the data cost term to capture the appearance, smooth cost term to encode the spatial context and label cost term to favor compact detection. The data cost is learned through a convolutional neural network and the parameters in the labeling model are learned through a structural SVM. Compared with proposal generation and classification based methods, the proposed superpixel labeling method can naturally detect objects missed by proposal generation step and capture the global image context to infer the overlapping objects. The proposed method shows its advantage in Pascal VOC and ImageNet. Notably, it performs better than the ImageNet ILSVRC2014 winner GoogLeNet (45.0% V.S. 43.9% in mAP) with much shallower and fewer CNNs.
The Fastest Deformable Part Model for Object Detection

This paper solves the speed bottleneck of deformable part model (DPM), while maintaining the accuracy in detection on challenging datasets. Three prohibitive steps in cascade version of DPM are accelerated, including 2D correlation between root filter and feature map, cascade part pruning and HOG feature extraction. For 2D correlation, the root filter is constrained to be low rank, so that 2D correlation can be calculated by more efficient linear combination of 1D correlations. A proximal gradient algorithm is adopted to progressively learn the low rank filter in a discriminative manner. For cascade part pruning, neighborhood aware cascade is proposed to capture the dependence in neighborhood regions for aggressive pruning. Instead of explicit computation of part scores, hypotheses can be pruned by scores of neighborhoods under the first order approximation. For HOG feature extraction, look-up tables are constructed to replace expensive calculations of orientation partition and magnitude with simpler matrix index operations. Extensive experiments show that (a) the proposed method is 4 times faster than the current fastest DPM method with similar accuracy on Pascal VOC, (b) the proposed method achieves state-of-the-art accuracy on pedestrian and face detection task with frame-rate speed.
Part-based Face Recognition Using Near Infrared Images

Recently, the authors developed NIR based face recognition for highly accurate face recognition under illumination variations. In this paper, we present a part-based method for improving its robustness with respect to pose variations. An NIR face is decomposed into parts. A part classifier is built for each part, using the most discriminative LBP histogram features selected by AdaBoost learning. The outputs of part classifiers are fused to give the final score. Experiments show that the present method outperforms the whole face-based method by 4.53%.
Learning mappings for face synthesis from near infrared to visual light images

This paper deals with a new problem in face recognition research, in which the enrollment and query face samples are captured under different lighting conditions. In our case, the enrollment samples are visual light (VIS) images, whereas the query samples are taken under near infrared (NIR) condition. It is very difficult to directly match the face samples captured under these two lighting conditions due to their different visual appearances. In this paper, we propose a novel method for synthesizing VIS images from NIR images based on learning the mappings between images of different spectra (i.e., NIR and VIS). In our approach, we reduce the inter-spectral differences significantly, thus allowing effective matching between faces taken under different imaging conditions. Face recognition experiments clearly show the efficacy of the proposed approach.
Ensemble-based discriminant learning with boosting for face recognition

In this paper, we propose a novel ensemble-based approach to boost performance of traditional Linear Discriminant Analysis (LDA)-based methods used in face recognition. The ensemble-based approach is based on the recently emerged technique known as "boosting". However, it is generally believed that boosting-like learning rules are not suited to a strong and stable learner such as LDA. To break the limitation, a novel weakness analysis theory is developed here. The theory attempts to boost a strong learner by increasing the diversity between the classifiers created by the learner, at the expense of decreasing their margins, so as to achieve a tradeoff suggested by recent boosting studies for a low generalization error. In addition, a novel distribution accounting for the pairwise class discriminant information is introduced for effective interaction between the booster and the LDA-based learner. The integration of all these methodologies proposed here leads to the novel ensemble-based discriminant learning approach, capable of taking advantage of both the boosting and LDA techniques. Promising experimental results obtained on various difficult face recognition scenarios demonstrate the effectiveness of the proposed approach. We believe that this work is especially beneficial in extending the boosting framework to accommodate general (strong/weak) learners.