nils-herrmann/clean_scirepeval huggingface dataset:
Latent Fingerprint Matching Using Descriptor-Based Hough Transform

Identifying suspects based on impressions of fingers lifted from crime scenes (latent prints) is a routine procedure that is extremely important to forensics and law enforcement agencies. Latents are partial fingerprints that are usually smudgy, with small area and containing large distortion. Due to these characteristics, latents have a significantly smaller number of minutiae points compared to full (rolled or plain) fingerprints. The small number of minutiae and the noise characteristic of latents make it extremely difficult to automatically match latents to their mated full prints that are stored in law enforcement databases. Although a number of algorithms for matching full-to-full fingerprints have been published in the literature, they do not perform well on the latent-to-full matching problem. Further, they often rely on features that are not easy to extract from poor quality latents. In this paper, we propose a new fingerprint matching algorithm which is especially designed for matching latents. The proposed algorithm uses a robust alignment algorithm (descriptor-based Hough transform) to align fingerprints and measures similarity between fingerprints by considering both minutiae and orientation field information. To be consistent with the common practice in latent matching (i.e., only minutiae are marked by latent examiners), the orientation field is reconstructed from minutiae. Since the proposed algorithm relies only on manually marked minutiae, it can be easily used in law enforcement applications. Experimental results on two different latent databases (NIST SD27 and WVU latent databases) show that the proposed algorithm outperforms two well optimized commercial fingerprint matchers. Further, a fusion of the proposed algorithm and commercial fingerprint matchers leads to improved matching accuracy.
Fingerprint pose estimation based on faster R-CNN

Fingerprint pose estimation is one of the bottlenecks of indexing in large scale database. The existing methods of pose estimation are based on manually appointed features (e.g. special points, ridges, orientation filed). In this paper, we propose a method based on deep learning to achieve accurate pose estimation. Faster R-CNN is adopted to detect the center point and rough direction, followed by intra-class and inter-class combination to calculate the precise direction. Extensive experiments on NIST-14 show that (1) the predicted poses are close to manual annotations even when the fingerprints are incomplete or noisy, (2) the estimated poses for matching fingerprint pairs are very consistent and (3) by registering fingerprints using the estimated pose, the accuracy of a state-of-the-art fingerprint indexing system is further improved.
Latent fingerprint enhancement via robust orientation field estimation

Latent fingerprints, or simply latents, have been considered as cardinal evidence for identifying and convicting criminals. The amount of information available for identification from latents is often limited due to their poor quality, unclear ridge structure and occlusion with complex background or even other latent prints. We propose a latent fingerprint enhancement algorithm, which expects manually marked region of interest (ROI) and singular points. The core of the proposed algorithm is a robust orientation field estimation algorithm for latents. Short-time Fourier transform is used to obtain multiple orientation elements in each image block. This is followed by a hypothesize-and-test paradigm based on randomized RANSAC, which generates a set of hypothesized orientation fields. Experimental results on NIST SD27 latent fingerprint database show that the matching performance of a commercial matcher is significantly improved by utilizing the enhanced latent fingerprints produced by the proposed algorithm.
Palmprint indexing based on ridge features

In recent years, law enforcement agencies are increasingly using palmprint to identify criminals. For law enforcement palmprint identification systems, efficiency is a very important but challenging problem because of large database size and poor image quality. Existing palmprint identification systems are not sufficiently fast for practical applications. To solve this problem, a novel palmprint indexing algorithm based on ridge features is proposed in this paper. A palmprint is pre-aligned by registering its orientation field with respect to a set of reference orientation fields, which are obtained by clustering training palmprint orientation fields. Indexing is based on comparing ridge orientation fields and ridge density maps, which is much faster than minutiae matching. Proposed algorithm achieved an error rate of 1% at a penetration rate of 2.25% on a palmprint database consisting of 13,416 palmprints. Searching a query palmprint over the whole database takes only 0.22 seconds.
Action recognition in RGB-D egocentric videos

In this paper, we investigate the problem of action recognition in RGB-D egocentric videos. These self-generated and embodied videos provide richer semantic cues than the conventional videos captured from the third-person view for action recognition. Moreover, they contain both appearance information and 3D structure of the scenes from the RGB modality and depth modality respectively. Motivated by these advantages, we first collect a video-based RGB-D egocentric dataset (THU-READ) with diverse types of daily-life actions. Then we evaluate several approaches including hand-crafted features and deep learning methods on THU-READ. To improve the performance, we further develop a tri-stream convolutional network (TCNet) method, which learns to exploit the fuse with both the RGB and depth modalities for action recognition. Experimental results show that our model achieves competitive performance with state-of-the-art methods.
Exploiting Unsupervised and Supervised Constraints for Subspace Clustering

Data in many image and video analysis tasks can be viewed as points drawn from multiple low-dimensional subspaces with each subspace corresponding to one category or class. One basic task for processing such kind of data is to separate the points according to the underlying subspace, referred to as subspace clustering. Extensive studies have been made on this subject, and nearly all of them use unconstrained subspace models, meaning the points can be drawn from everywhere of a subspace, to represent the data. In this paper, we attempt to do subspace clustering based on a constrained subspace assumption that the data is further restricted in the corresponding subspaces, e.g., belonging to a submanifold or satisfying the spatial regularity constraint. This assumption usually describes the real data better, such as differently moving objects in a video scene and face images of different subjects under varying illumination. A unified integer linear programming optimization framework is used to approach subspace clustering, which can be efficiently solved by a branch-and-bound (BB) method. We also show that various kinds of supervised information, such as subspace number, outlier ratio, pairwise constraints, size prior and etc., can be conveniently incorporated into the proposed framework. Experiments on real data show that the proposed method outperforms the state-of-the-art algorithms significantly in clustering accuracy. The effectiveness of the proposed method in exploiting supervised information is also demonstrated.
Latent Palmprint Matching

The evidential value of palmprints in forensics is clear as about 30% of the latents recovered from crime scenes are from palms. While palmprint-based personal authentication systems have been developed, they mostly deal with low resolution (about 100 ppi) palmprints and only perform full-to-full matching. We propose a latent-to-full palmprint matching system that is needed in forensics. Our system deals with palmprints captured at 500 ppi and uses minutiae as features. Latent palmprint matching is a challenging problem because latents lifted at crime scenes are of poor quality, cover small area of palms and have complex background. Other difficulties include the presence of many creases and a large number of minutiae in palmprints. A robust algorithm to estimate ridge direction and frequency in palmprints is developed. This facilitates minutiae extraction even in poor quality palmprints. A fixed-length minutia descriptor, MinutiaCode, is utilized to capture distinctive information around each minutia and an alignment-based matching algorithm is used to match palmprints. Two sets of partial palmprints (150 live-scan partial palmprints and 100 latents) are matched to a background database of 10,200 full palmprints to test the proposed system. Rank-1 recognition rates of 78.7% and 69%, respectively, were achieved for live-scan palmprints and latents.
Latent fingerprint enhancement using Gabor and minutia dictionaries

Latent fingerprints play important roles in law enforcement agencies. Due to its poor quality caused by unclear ridge structure, uneven contrast and overlapping patterns, a latent fingerprint enhancement is necessary for reliable feature extraction. Gabor function is widely used to characterise ridge structure and used in fingerprint enhancement. However, gabor function can not capture the details of minutia that is the end point or bifurcation of ridge. To utilize the prior knowledge of both ridge and minutia, we propose to construct both ridge and minutia dictionaries, and propose a two-step multi-scale patch based sparse representation to enhance the ridge using ridge dictionaries and enhance the minutia with both dictionaries. Experimental results show that two-step SR algorithm outperforms the SR only using gabor dictionary and gabor filter on both minutia extraction accuracy and matching accuracy.
Fingerprint Matching With Rotation-Descriptor Texture Features

A novel texture correlation matching method for fingerprint verification using Fourier-Mellin descriptor and phase-only correlation function is proposed in this paper. Fourier-Mellin descriptor correlation is used to align the template and query fingerprint images and a matching score is obtained. Matching takes about 1 second in Celeron 2.0 GHz processor, and the experimental results show that EER is 3.8%; fusion with minutia matching gets a better result
Detecting fingerprint distortion from a single image

Elastic distortion of friction ridge skin is one of the major challenges in fingerprint matching. Since existing fingerprint matching systems cannot match seriously distorted fingerprints, criminals may purposely distort their fingerprints to evade identification. Existing distortion detection techniques require availability of specialized hardware or fingerprint video, limiting their use in real applications. In this paper we conduct a study on fingerprint distortion and develop an algorithm to detect fingerprint distortion from a single image which is captured using traditional fingerprint sensing techniques. The detector is based on analyzing ridge period and orientation information. Promising results are obtained on a public domain fingerprint database containing distorted fingerprints.
Latent Fingerprint Matching

Latent fingerprint identification is of critical importance to law enforcement agencies in identifying suspects: Latent fingerprints are inadvertent impressions left by fingers on surfaces of objects. While tremendous progress has been made in plain and rolled fingerprint matching, latent fingerprint matching continues to be a difficult problem. Poor quality of ridge impressions, small finger area, and large nonlinear distortion are the main difficulties in latent fingerprint matching compared to plain or rolled fingerprint matching. We propose a system for matching latent fingerprints found at crime scenes to rolled fingerprints enrolled in law enforcement databases. In addition to minutiae, we also use extended features, including singularity, ridge quality map, ridge flow map, ridge wavelength map, and skeleton. We tested our system by matching 258 latents in the NIST SD27 database against a background database of 29,257 rolled fingerprints obtained by combining the NIST SD4, SD14, and SD27 databases. The minutiae-based baseline rank-1 identification rate of 34.9 percent was improved to 74 percent when extended features were used. In order to evaluate the relative importance of each extended feature, these features were incrementally used in the order of their cost in marking by latent experts. The experimental results indicate that singularity, ridge quality map, and ridge flow map are the most effective features in improving the matching accuracy.
On separating overlapped fingerprints

Fingerprint images generally either contain only a single fingerprint or a set of non-overlapped fingerprints (e.g., slap fingerprints). However, there are situations where more than one fingerprint overlap on each other. Such situations are frequently encountered when latent fingerprints are lifted from crime scenes or residue fingerprints are left on fingerprint sensors. Overlapped fingerprints constitute a serious challenge to existing fingerprint recognition techniques, since these techniques are designed under the assumption that fingerprints have been properly segmented. In this paper, a novel algorithm is proposed to separate overlapped fingerprints into component or individual fingerprints. We first use local Fourier transform to estimate an initial overlapped orientation field, which contains at most two candidate orientations at each location. Then relaxation labeling technique is employed to label each candidate orientation as one of two classes. Based on the labeling result, we separate the initial overlapped orientation field into two orientation fields. Finally, the two fingerprints are obtained by enhancing the overlapped fingerprint using Gabor filters tuned to these two component separated orientation fields, respectively. Experimental results indicate that the algorithm leads to a good separation of overlapped fingerprints.
Efficient Rectification of Distorted Fingerprints

Recently, distortion rectification based on a single fingerprint image has been shown to be able to significantly improve the recognition rate of distorted fingerprints. However, the computational complexity of such a method is too high to be useful in practice. In this paper, we propose a novel method for the rectification of distorted fingerprints, whose speed is over 30 times faster than the existing method. This significant speedup is due to a Hough-forest-based two-step fingerprint pose estimation algorithm and a support vector regressor-based fingerprint distortion field estimation algorithm. Experimental results on public domain databases show that our method can achieve as good rectification performance as the existing method but meanwhile is significantly faster.
On matching latent fingerprints

Latent fingerprint identification is of critical importance to law enforcement agencies in forensics application. While tremendous progress has been made in the field of automatic fingerprint matching, latent fingerprint matching continues to be a difficult problem because the challenges involved in latent print matching are quite different from plain or rolled fingerprint matching. Poor quality of friction ridge impressions, small finger area and large non-linear distortion are some of the main difficulties in latent fingerprint matching. We propose a system for matching latent images to rolled fingerprints that takes into account the specific characteristics of the latent matching problem. In addition to minutiae, additional features like orientation field and quality map are also used in our system. Experimental results on the NIST SD27 latent database indicate that the introduction of orientation field and quality map to minutiae-based matching leads to good recognition performance despite the inherently difficult nature of the problem. We achieve the rank-20 accuracy of 93.4% in retrieving 258 latents from a background database of 2,258 rolled fingerprints.
Context-Aware Local Binary Feature Learning for Face Recognition

In this paper, we propose a context-aware local binary feature learning (CA-LBFL) method for face recognition. Unlike existing learning-based local face descriptors such as discriminant face descriptor (DFD) and compact binary face descriptor (CBFD) which learn each feature code individually, our CA-LBFL exploits the contextual information of adjacent bits by constraining the number of shifts from different binary bits, so that more robust information can be exploited for face representation. Given a face image, we first extract pixel difference vectors (PDV) in local patches, and learn a discriminative mapping in an unsupervised manner to project each pixel difference vector into a context-aware binary vector. Then, we perform clustering on the learned binary codes to construct a codebook, and extract a histogram feature for each face image with the learned codebook as the final representation. In order to exploit local information from different scales, we propose a context-aware local binary multi-scale feature learning (CA-LBMFL) method to jointly learn multiple projection matrices for face representation. To make the proposed methods applicable for heterogeneous face recognition, we present a coupled CA-LBFL (C-CA-LBFL) method and a coupled CA-LBMFL (C-CA-LBMFL) method to reduce the modality gap of corresponding heterogeneous faces in the feature level, respectively. Extensive experimental results on four widely used face datasets clearly show that our methods outperform most state-of-the-art face descriptors.
Robust and Efficient Ridge-Based Palmprint Matching

During the past decade, many efforts have been made to use palmprints as a biometric modality. However, most of the existing palmprint recognition systems are based on encoding and matching creases, which are not as reliable as ridges. This affects the use of palmprints in large-scale person identification applications where the biometric modality needs to be distinctive as well as insensitive to changes in age and skin conditions. Recently, several ridge-based palmprint matching algorithms have been proposed to fill the gap. Major contributions of these systems include reliable orientation field estimation in the presence of creases and the use of multiple features in matching, while the matching algorithms adopted in these systems simply follow the matching algorithms for fingerprints. However, palmprints differ from fingerprints in several aspects: 1) Palmprints are much larger and thus contain a large number of minutiae, 2) palms are more deformable than fingertips, and 3) the quality and discrimination power of different regions in palmprints vary significantly. As a result, these matchers are unable to appropriately handle the distortion and noise, despite heavy computational cost. Motivated by the matching strategies of human palmprint experts, we developed a novel palmprint recognition system. The main contributions are as follows: 1) Statistics of major features in palmprints are quantitatively studied, 2) a segment-based matching and fusion algorithm is proposed to deal with the skin distortion and the varying discrimination power of different palmprint regions, and 3) to reduce the computational complexity, an orientation field-based registration algorithm is designed for registering the palmprints into the same coordinate system before matching and a cascade filter is built to reject the nonmated gallery palmprints in early stage. The proposed matcher is tested by matching 840 query palmprints against a gallery set of 13,736 palmprints. Experimental results show that the proposed matcher outperforms the existing matchers a lot both in matching accuracy and speed.
Orientation Field Estimation for Latent Fingerprint Enhancement

Identifying latent fingerprints is of vital importance for law enforcement agencies to apprehend criminals and terrorists. Compared to live-scan and inked fingerprints, the image quality of latent fingerprints is much lower, with complex image background, unclear ridge structure, and even overlapping patterns. A robust orientation field estimation algorithm is indispensable for enhancing and recognizing poor quality latents. However, conventional orientation field estimation algorithms, which can satisfactorily process most live-scan and inked fingerprints, do not provide acceptable results for most latents. We believe that a major limitation of conventional algorithms is that they do not utilize prior knowledge of the ridge structure in fingerprints. Inspired by spelling correction techniques in natural language processing, we propose a novel fingerprint orientation field estimation algorithm based on prior knowledge of fingerprint structure. We represent prior knowledge of fingerprints using a dictionary of reference orientation patches. which is constructed using a set of true orientation fields, and the compatibility constraint between neighboring orientation patches. Orientation field estimation for latents is posed as an energy minimization problem, which is solved by loopy belief propagation. Experimental results on the challenging NIST SD27 latent fingerprint database and an overlapped latent fingerprint database demonstrate the advantages of the proposed orientation field estimation algorithm over conventional algorithms.
Two-Stream Transformer Networks for Video-Based Face Alignment

In this paper, we propose a two-stream transformer networks (TSTN) approach for video-based face alignment. Unlike conventional image-based face alignment approaches which cannot explicitly model the temporal dependency in videos and motivated by the fact that consistent movements of facial landmarks usually occur across consecutive frames, our TSTN aims to capture the complementary information of both the spatial appearance on still frames and the temporal consistency information across frames. To achieve this, we develop a two-stream architecture, which decomposes the video-based face alignment into spatial and temporal streams accordingly. Specifically, the spatial stream aims to transform the facial image to the landmark positions by preserving the holistic facial shape structure. Accordingly, the temporal stream encodes the video input as active appearance codes, where the temporal consistency information across frames is captured to help shape refinements. Experimental results on the benchmarking video-based face alignment datasets show very competitive performance of our method in comparisons to the state-of-the-arts.
Consistent-Aware Deep Learning for Person Re-identification in a Camera Network

In this paper, we propose a consistent-aware deep learning (CADL) framework for person re-identification in a camera network. Unlike most existing person re-identification methods which identify whether two body images are from the same person, our approach aims to obtain the maximal correct matches for the whole camera network. Different from recently proposed camera network based re-identification methods which only consider the consistent information in the matching stage to obtain a global optimal association, we exploit such consistent-aware information under a deep learning framework where both feature representation and image matching are automatically learned with certain consistent constraints. Specifically, we reach the global optimal solution and balance the performance between different cameras by optimizing the similarity and association iteratively. Experimental results show that our method obtains significant performance improvement and outperforms the state-of-the-art methods by large margins.