nils-herrmann/clean_scirepeval huggingface dataset:
Catadioptric HyperSpectral Light Field Imaging

The complete plenoptic function records radiance of rays from every location, at every angle, for every wavelength and at every time. The signal is multi-dimensional and has long relied on multi-modal sensing such as hybrid light field camera arrays. In this paper, we present a single camera hyperspectral light field imaging solution that we call Snapshot Plenoptic Imager (SPI). SPI uses spectral coded catadioptric mirror arrays for simultaneously acquiring the spatial, angular and spectral dimensions. We further apply a learning-based approach to improve the spectral resolution from very few measurements. Specifically, we demonstrate and then employ a new spectral sparsity prior that allows the hyperspectral profiles to be sparsely represented under a pre-trained dictionary. Comprehensive experiments on synthetic and real data show that our technique is effective, reliable, and accurate. In particular, we are able to produce the first wide FoV multi-spectral light field database.
Image sets alignment for Video-Based Face Recognition

Video-based Face Recognition (VFR) can be converted to the matching of two image sets containing face images captured from each video. For this purpose, we propose to bridge the two sets with a reference image set that is well-defined and pre-structured to a number of local models offline. In other words, given two image sets, as long as each of them is aligned to the reference set, they are mutually aligned and well structured. Therefore, the similarity between them can be computed by comparing only the corresponded local models rather than considering all the pairs. To align an image set with the reference set, we further formulate the problem as a quadratic programming. It integrates three constrains to guarantee robust alignment, including appearance matching cost term exploiting principal angles, geometric structure consistency using affine invariant reconstruction weights, smoothness constraint preserving local neighborhood relationship. Extensive experimental evaluations are performed on three databases: Honda, MoBo and YouTube. Compared with competing methods, our approach can consistently achieve better results.
Artist-Drawing Inspired Automatic Sketch Portrait Rendering

This paper proposes an artist-drawing inspired sketch portrait computer-generation technique. We borrow knowledge from artist-drawing procedure in three phases. The first phase is artist-like visual perception, which parses a photo image to structural appearance from global to local. The second phase is artist-like consideration and judgement, which maps previously parsed visual structure and texture to drawable graphics elements. In the third phase, we render the previously mapped drawing elements in a 'global-local-global' procedure like an artist. The rendering results demonstrate the effectiveness in generating an artistic sketch portrait conveniently from a photograph.
A Sample Pre-mapping Method Enhancing Boosting for Object Detection

We propose a novel method to improve the training efficiency and accuracy of boosted classifiers for object detection. The key step of the proposed method is a sample pre-mapping on original space by referring to the selected ‘reference sample’ before feeding into weak classifiers. The reference sample corresponds to an approximation of the optimal separating hyper-plane in an implicit high dimensional space, so that the resulting classifier could achieve the performance similar to kernel method, while spending the computation cost of linear classifier in both training and detection. We employ two different non-linear mappings to verify the proposed method under boosting framework. Experimental results show that the proposed approach achieves performance comparable with the common used methods on public datasets in both pedestrian detection and car detection.
Detection of text on road signs from video

A fast and robust framework for incrementally detecting text on road signs from video is presented in this paper. This new framework makes two main contributions. 1) The framework applies a divide-and-conquer strategy to decompose the original task into two subtasks, that is, the localization of road signs and the detection of text on the signs. The algorithms for the two subtasks are naturally incorporated into a unified framework through a feature-based tracking algorithm. 2) The framework provides a novel way to detect text from video by integrating two-dimensional (2-D) image features in each video frame (e.g., color, edges, texture) with the three-dimensional (3-D) geometric structure information of objects extracted from video sequence (such as the vertical plane property of road signs). The feasibility of the proposed framework has been evaluated using 22 video sequences captured from a moving vehicle. This new framework gives an overall text detection rate of 88.9% and a false hit rate of 9.2%. It can easily be applied to other tasks of text detection from video and potentially be embedded in a driver assistance system.
A novel two-tier Bayesian based method for hair segmentation

In this paper, a novel two-tier Bayesian based method is proposed for hair segmentation. In the first tier, we construct a Bayesian model by integrating hair occurrence prior probabilities (HOPP) with a generic hair color model (GHCM) to obtain some reliable hair seed pixels. These initial seeds are further propagated to their neighborhood pixels by utilizing segmentation results of Mean Shift, to obtain more seeds. In the second tier, all of these selected seeds are used to train a hair-specific Gaussian model, which are combined with HOPP to build the second Bayesian model for pixel classification. Mean Shift results are further utilized to remove holes and spread hair regions. The experimental results illustrate the effectiveness of our approach.
Kinematic Constrained Cascaded Autoencoder for Real-Time Hand Pose Estimation

Hand pose estimation is an attractive problem in computer vision for its key role in gesture controlled humancomputer interaction (HCI) applications. This problem focuses on revealing the hand skeleton structure from visual information. However, it is very challenging for complicated hand configurations. In this paper, a kinematic constrained cascaded autoencoder regression (KCAE) framework is proposed to estimate the hand pose from a single depth image.We introduce a two-stage cascaded structure to regress the palm direction and the whole hand joints successively. In addition, the edge constraints are first introduced to the loss function with an endto- end manner, which maintains the kinematics of hands and makes the prediction more reasonable. With this framework, different features are evaluated, including the handcrafted features and the features learned from CNN. The experiments widely conducted on our collected dataset and the public MSRA hand gesture database demonstrate the effectiveness of KCAE. Overall, The proposed method achieves comparable performance with state-of-the-arts.
Visual context representation using a combination of feature-driven and object-driven mechanisms

Visual context between objects is an important cue for object position perception. How to effectively represent the visual context is a key issue to study. Some past work introduced task-driven methods for object perception, which led a large coding quantity. This paper proposes an approach that incorporates feature-driven mechanism into object-driven context representation for object locating. As an example, the paper discusses how a neuronal network encodes the visual context between feature salient regions and human eye centers with as little coding quantity as possible. A group of experiments on efficiency of visual context coding and object searching are analyzed and discussed, which show that the proposed method decreases the coding quantity and improve the object searching accuracy effectively.
Automatic detection and recognition of signs from natural scenes

In this paper, we present an approach to automatic detection and recognition of signs from natural scenes, and its application to a sign translation task. The proposed approach embeds multiresolution and multiscale edge detection, adaptive searching, color analysis, and affine rectification in a hierarchical framework for sign detection, with different emphases at each phase to handle the text in different sizes, orientations, color distributions and backgrounds. We use affine rectification to recover deformation of the text regions caused by an inappropriate camera view angle. The procedure can significantly improve text detection rate and optical character recognition (OCR) accuracy. Instead of using binary information for OCR, we extract features from an intensity image directly. We propose a local intensity normalization method to effectively handle lighting variations, followed by a Gabor transform to obtain local features, and finally a linear discriminant analysis (LDA) method for feature selection. We have applied the approach in developing a Chinese sign translation system, which can automatically detect and recognize Chinese signs as input from a camera, and translate the recognized text into English.
Combining LBP Difference and Feature Correlation for Texture Description

Effective characterization of texture images requires exploiting multiple visual cues from the image appearance. The local binary pattern (LBP) and its variants achieve great success in texture description. However, because the LBP(-like) feature is an index of discrete patterns rather than a numerical feature, it is difficult to combine the LBP(-like) feature with other discriminative ones by a compact descriptor. To overcome the problem derived from the nonnumerical constraint of the LBP, this paper proposes a numerical variant accordingly, named the LBP difference (LBPD). The LBPD characterizes the extent to which one LBP varies from the average local structure of an image region of interest. It is simple, rotation invariant, and computationally efficient. To achieve enhanced performance, we combine the LBPD with other discriminative cues by a covariance matrix. The proposed descriptor, termed the covariance and LBPD descriptor (COV-LBPD), is able to capture the intrinsic correlation between the LBPD and other features in a compact manner. Experimental results show that the COV-LBPD achieves promising results on publicly available data sets.
Fast sign language recognition benefited from low rank approximation

This paper proposes a framework based on the Hidden Markov Models (HMMs) benefited from the low rank approximation of the original sign videos for two aspects. First, under the observations that most visual information of a sign sequence typically concentrates on limited key frames, we apply an online low rank approximation of sign videos for the first time to select the key frames. Second, rather than fixing the number of hidden states for large vocabulary of variant signs, we further take the advantage of the low rank approximation to independently determine it for each sign to optimise predictions. With the key frame selection and the variant number of hidden states determination, an advanced framework based on HMMs for Sign Language Recognition (SLR) is proposed, which is denoted as Light-HMMs (because of the fewer frames and proper estimated hidden states). With the Kinect sensor, RGB-D data is fully investigated for the feature representation. In each frame, we adopt Skeleton Pair feature to character the motion and extract the Histograms of Oriented Gradients as the feature of the hand posture appearance. The proposed framework achieves an efficient computing and even better correct rate in classification. The widely experiments are conducted on large vocabulary sign datasets with up to 1000 classes of signs and the encouraging results are obtained.
AgeNet: Deeply Learned Regressor and Classifier for Robust Apparent Age Estimation

Apparent age estimation from face image has attracted more and more attentions as it is favorable in some real-world applications. In this work, we propose an end-to-end learning approach for robust apparent age estimation, named by us AgeNet. Specifically, we address the apparent age estimation problem by fusing two kinds of models, i.e., real-value based regression models and Gaussian label distribution based classification models. For both kind of models, large-scale deep convolutional neural network is adopted to learn informative age representations. Another key feature of the proposed AgeNet is that, to avoid the problem of over-fitting on small apparent age training set, we exploit a general-to-specific transfer learning scheme. Technically, the AgeNet is first pre-trained on a large-scale web-collected face dataset with identity label, and then it is fine-tuned on a large-scale real age dataset with noisy age label. Finally, it is fine-tuned on a small training set with apparent age label. The experimental results on the ChaLearn 2015 Apparent Age Competition demonstrate that our AgeNet achieves the state-of-the-art performance in apparent age estimation.
Fast object detection using boosted co-occurrence histograms of oriented gradients

Co-occurrence histograms of oriented gradients (CoHOG) are powerful descriptors in object detection. In this paper, we propose to utilize a very large pool of CoHOG features with variable-location and variable-size blocks to capture salient characteristics of the object structure. We consider a CoHOG feature as a block with a special pattern described by the offset. A boosting algorithm is further introduced to select the appropriate locations and offsets to construct an efficient and accurate cascade classifier. Experimental results on public datasets show that our approach simultaneously achieves high accuracy and fast speed on both pedestrian detection and car detection tasks.
Salient region detection via texture-suppressed background contrast

We propose a novel salient region detection algorithm by texture-suppressed background contrast. We employ a structure extraction algorithm to suppress the small scale textures which are supposed to be not sensitive for human vision system. Then the texture-suppressed image is segmented into homogeneous superpixels. Motivated by the observation that the spatial distribution of the background has a high probability on the boundaries of images, we estimate the background as superpixels near the image boundaries. The saliency of each superpixel is then defined as the summation of its k minimum color distances to the estimated background superpixels. Finally a post-processing process involving spatial and color adjacency is employed to generate a per-pixel saliency map. Experimental results demonstrate that the proposed method outperforms the state-of-the-art approaches.
Advertisement evaluation using visual saliency based on foveated image

This paper proposes a novel approach to advertisement evaluation using automatic salient regions. The salient regions are detected using a predicting model, in which the estimation are obtained by the space variant foveated image. The saliency is defined as the difference between the input image and its estimation. Then an advertisement is determined as attractive if the detected salient regions are overlapped with the interested regions of the advertisement. The experimental results on the advertisements data set are encouraging.
V-LGBP: Volume based local Gabor binary patterns for face representation and recognition

In this paper, we propose volume based local Gabor binary patterns (V-LGBP) for face representation and recognition. In our method, the Gabor feature set of each gray image is regarded as a three dimensional ldquovolumerdquo, where the first two dimensions are spatial domain and the third dimension is the Gabor filter index. Then, the neighborhood order relationship in the ldquovolumerdquo is encoded by local binary patterns (LBP), which converts the Gabor transformed images into multiple index maps. Finally, the spatial histograms of all the V-LGBP index maps are concatenated together to represent the facial appearances. In addition, in order to reflect the uniform appearances of V-LGBP, its uniform patterns are redefined via statistical analysis. Extensive experiments on FERET dataset validate the effectiveness of our approach.
Deep Supervised Hashing for Fast Image Retrieval

In this paper, we present a new hashing method to learn compact binary codes for highly efficient image retrieval on large-scale datasets. While the complex image appearance variations still pose a great challenge to reliable retrieval, in light of the recent progress of Convolutional Neural Networks (CNNs) in learning robust image representation on various vision tasks, this paper proposes a novel Deep Supervised Hashing (DSH) method to learn compact similarity-preserving binary code for the huge body of image data. Specifically, we devise a CNN architecture that takes pairs of images (similar/dissimilar) as training inputs and encourages the output of each image to approximate discrete values (e.g. +1/-1). To this end, a loss function is elaborately designed to maximize the discriminability of the output space by encoding the supervised information from the input image pairs, and simultaneously imposing regularization on the real-valued outputs to approximate the desired discrete values. For image retrieval, new-coming query images can be easily encoded by propagating through the network and then quantizing the network outputs to binary codes representation. Extensive experiments on two large scale datasets CIFAR-10 and NUS-WIDE show the promising performance of our method compared with the state-of-the-arts.
Continuous heart rate measurement from face: A robust rPPG approach with distribution learning

Non-contact heart rate (HR) measurement via remote photoplethysmography (rPPG) has drawn increasing attention. While a number of methods have been reported, most of them did not take into account the continuous HR measurement problem, which is more challenging due to limited observed video frames and the requirement of speed. In this paper, we present a real-time rPPG method for continuous HR measurement from face videos. We use a multi-patch ROI strategy to remove outlier signals. Chrominance feature is then generated from each ROI to reduce the color channel magnitude differences, which is followed by temporal filtering to suppress the artifacts. In addition, considering the temporal relationship of neighboring HR rhythms, we learn a HR distribution based on historical HR measurements, and apply it to the succeeding HR estimations. Experiment results on the public-domain MAHNOB-HCI database and user tests with commodity webcams show the effectiveness of the proposed approach.
A PDA-based face recognition system

In this paper, we present a PDA-based face recognition system as well as some of the associated challenges of developing a PDA-based face recognition system. We describe a prototype system built from an off the shelf PDA, and introduce algorithms for image preprocessing to enhance the quality of the image by sharpening focus, and normalizing both lighting condition and head rotation. We use a unified LDA/PCA algorithm for face recognition. The algorithm maximizes the LDA criterion directly without a separate PCA step, which eliminates the possibility of losing discriminative information due to a separate PCA step. We demonstrate effectiveness of these algorithms and the feasibility of this system by experimental results. This system has many applications including information retrieval and law enforcement.
Towards non-cooperative iris recognition systems

Iris Technology has been successfully applied to person verification and identification. However, all commercial products require user cooperation for iris image capture. This paper examines the new challenges of iris recognition when extended to less cooperative situations. With the current stress on security and surveillance, this has been an important consideration. First, a summary of research findings of the past decade on iris recognition is described Then we identified new challenges that will be encountered when extending these methods to less cooperative situations. The difficulties are great and this paper describes some initial work into this area. One difficulty studied is the loss of iris details captured. We propose a modified Kolmogorov complexity measure based on maximum Shannon entropy of wavelet packet reconstruction to quantify the iris information. Real-time eye-corner tracking, iris segmentation and feature extraction algorithms are implemented. Video images of the iris are captured by an ordinary CCD camera with a zoom lens. Experiments are performed and the performances and analysis of iris code method and correlation method are described. Several useful findings were reached albeit from a small database. The iris codes were found to contain almost all the discriminating information. Our correlation approach coupled with nearest neighbour classification outperforms the conventional thresholding method for iris recognition with degraded images.
Robust Head-Shoulder Detection Using a Two-Stage Cascade Framework

Head-shoulder detection is widely used in many applications, and robust image descriptors are crucial to the detection performance. In this paper, by exploiting the second-order region covariance descriptor as a complement to widely-used histogram-based descriptors, we propose a new two-stage coarse-to-fine cascade framework to make full use of both types of descriptors for robust head-shoulder detection. Specifically, in the first stage, two histogram-based descriptors, i.e., local Histogram of Oriented Gradients (HOG) and histogram of Local Binary Pattern (LBP), are utilized by a Viola-Jones classifier to rapidly reject most non-head-shoulder candidate windows. In contrast, the second stage further boost the performance via multiple kernel learning on Riemannian manifold formed by Region Covariance Matrix (RCM), a second-order statistic descriptor with stronger discriminative power. Experimental results on a public dataset demonstrate that our method improves detection rate significantly with satisfactory detection speed.
Communication tool for the hard of hearings: A large vocabulary sign language recognition system

Deaf person has a large social community around the world. The smooth communication is very difficult for these hard of hearings. Automatic Sign Language Recognition (SLR) can build the bridge between the deaf and the hearings and turn the seamless interaction into reality. This paper presents a visualized communication tool for the hard of hearings, i.e. a large vocabulary sign language recognition system based on the RGB-D data input. A novel Grassmann Covariance Matrix (GCM) representation is used to encode a long-term dynamics of a sign sequence and the discriminative kernel SVM is adopted for the sign classification. For continuous sign language recognition, a probability inference method is used to determine the spotting from the labels of sequential frames. Some basic evaluation and comparison of our recognition algorithms are conducted in our collected datasets. This demo will show the recognition of both isolated sign words and the continuous sign language sentences.
A Fully End-to-End Cascaded CNN for Facial Landmark Detection

Facial landmark detection plays an important role in computer vision. It is a challenging problem due to various poses, exaggerated expressions and partial occlusions. In this work, we propose a Fully End-to-End Cascaded Convolutional Neural Network (FEC-CNN) for more promising facial landmark detection. Specifically, FEC-CNN includes several sub- CNNs, which progressively refine the shape prediction via finer and finer modeling, and the overall network is optimized fully end-to-end. Experiments on three challenging datasets, IBUG, 300W competition and AFLW, demonstrate that the proposed method is robust to large poses, exaggerated expressions and partial occlusions. The proposed FEC-CNN significantly improves the accuracy of landmark prediction.