nils-herrmann/clean_scirepeval huggingface dataset:
Visual pattern image coding

A novel framework for digital image compression called visual pattern image coding, or VPIC, is presented. In VPIC, set of visual-patterns is defined independent of the images to be coded. Each visual pattern is a subimage of limited spatial support that is visually meaningful to a normal human observer. The patterns are used as a basis for efficient image representation; since it is assumed that the images to be coded are natural optical images to be viewed by human observers, visual pattern design is developed using relevant psychophysical and physiological data. Although VPIC bears certain resemblances to block truncation (BTC) and vector quantification (VQ) image coding, there are important differences. First, there is no training phase required: the visual patterns derive from models of perceptual mechanisms; second, the assignment of patterns to image regions is not based on a standard (norm) error criterion; expensive search operations are eliminated.
Windowed locally monotonic regression

Lomotonicity, the largest degree of local monotonicity that a signal has, is proposed as an appropriate measure of smoothness when studying smoothers such as the median filter. Locally monotonic regression (LMR) optimally solves the problem of smoothing a signal to a specified minimal degree of lomotonicity, but it often requires an excess of computational resources. Windowed locally monotonic regression (WLMR) presents an attractive alternative to the problem of smoothing signals under the smoothness criterion of lomotonicity. This criterion is meaningful, for example, when processing images and speech signals. Unlike LMR, WLMR is defined also for infinite-length signals; this may be useful in more theoretical studies of this tool.<<ETX>>
Multichannel Texture Analysis Using Localized Spatial Filters

A computational approach for analyzing visible textures is described. Textures are modeled as irradiance patterns containing a limited range of spatial frequencies, where mutually distinct textures differ significantly in their dominant characterizing frequencies. By encoding images into multiple narrow spatial frequency and orientation channels, the slowly varying channel envelopes (amplitude and phase) are used to segregate textural regions of different spatial frequency, orientation, or phase characteristics. Thus, an interpretation of image texture as a region code, or carrier of region information, is emphasized. The channel filters used, known as the two-dimensional Gabor functions, are useful for these purposes in several senses: they have tunable orientation and radial frequency bandwidths and tunable center frequencies, and they optimally achieve joint resolution in space and in spatial frequency. By comparing the channel amplitude responses, one can detect boundaries between textures. Locating large variations in the channel phase responses allows discontinuities in the texture phase to be detected. Examples are given of both types of texture processing using a variety of real and synthetic textures.
AM-FM image analysis techniques

We present an overview of AM-FM image analysis techniques. Many nonstationary textured images are multi-partite, and may be considered to comprise several locally narrowband components. We introduce a multi-component AM-FM model useful in describing such images, and show that the components may be demodulated on a spatially local basis using multiband filtering followed by a computationally efficient nonlinear algorithm. We outline practical approaches both for estimating the locally dominant image frequencies and for computing a full AM-FM image representation. Finally, we discuss how the estimated frequencies may be used to compute flow lines.
Toward Naturalistic 2D-to-3D Conversion

Natural scene statistics (NSSs) models have been developed that make it possible to impose useful perceptually relevant priors on the luminance, colors, and depth maps of natural scenes. We show that these models can be used to develop 3D content creation algorithms that can convert monocular 2D videos into statistically natural 3D-viewable videos. First, accurate depth information on key frames is obtained via human annotation. Then, both forward and backward motion vectors are estimated and compared to decide the initial depth values, and a compensation process is applied to further improve the depth initialization. Then, the luminance/chrominance and initial depth map are decomposed by a Gabor filter bank. Each subband of depth is modeled to produce a NSS prior term. The statistical color-depth priors are combined with the spatial smoothness constraint in the depth propagation target function as a prior regularizing term. The final depth map associated with each frame of the input 2D video is optimized by minimizing the target function over all subbands. In the end, stereoscopic frames are rendered from the color frames and their associated depth maps. We evaluated the quality of the generated 3D videos using both subjective and objective quality assessment methods. The experimental results obtained on various sequences show that the presented method outperforms several state-of-the-art 2D-to-3D conversion methods.
Exploiting Feature Hierarchies with Convolutional Neural Networks for Cultural Event Recognition

Cultural events are kinds of typical events closely related to history and nationality, which play an important role in cultural heritage through generations. However, automatically recognizing cultural events still remains a great challenge since it depends on understanding of complex image contents such as people, objects, and scene context. Therefore, it is intuitive to associate this task with other high-level vision problems, e.g., object detection, recognition, and scene understanding. In this paper, we address this problem by combining both ideas of object / scene contents mining and strong image representation via CNN into a whole framework. Specifically, for object / scene contents mining, we employ selective search to extract a batch of bottom-up region proposals, which are served as key object / scene candidates in each event image, while for representation via CNN, we investigate two state-of-the-art deep architectures, VGGNet and GoogLeNet, and adapt them to our task by performing domain-specific (i.e., event) fine-tuning on both global image and hierarchical region proposals. These two models can complementarily exploit feature hierarchies spatially, which simultaneously capture the global context and local evidences within the image. In our final submission for ChaLearn LAP Challenge ICCV 2015, nine kinds of features extracted from five different deep models were exploited and followed with two kinds of classifiers for decision level fusion. Our method achieves the best performance of mAP=0.854 among all the participants in the track of cultural event recognition.
Siamese recurrent architecture for visual tracking

Treating visual tracking as a matching problem, siamese architecture has drawn increasing interest recently. In this paper, we propose a novel siamese recurrent architecture that can enhance the similarity matching by leveraging contextual information. Specifically, the multi-directional Recurrent Neural Network (RNN) is employed to memorize the long-range contextual dependencies of object parts and learn the self-structure information of the object. We test the proposed method on a challenging benchmark, and it gain promising results compared with the existing tracking algorithms.
Discriminative Covariance Oriented Representation Learning for Face Recognition with Image Sets

For face recognition with image sets, while most existing works mainly focus on building robust set models with hand-crafted feature, it remains a research gap to learn better image representations which can closely match the subsequent image set modeling and classification. Taking sample covariance matrix as set model in the light of its recent promising success, we present a Discriminative Covariance oriented Representation Learning (DCRL) framework to bridge the above gap. The framework constructs a feature learning network (e.g. a CNN) to project the face images into a target representation space, and the network is trained towards the goal that the set covariance matrix calculated in the target space has maximum discriminative ability. To encode the discriminative ability of set covariance matrices, we elaborately design two different loss functions, which respectively lead to two different representation learning schemes, i.e., the Graph Embedding scheme and the Softmax Regression scheme. Both schemes optimize the whole network containing both image representation mapping and set model classification in a joint learning manner. The proposed method is extensively validated on three challenging and large scale databases for the task of face recognition with image sets, i.e., YouTube Celebrities, YouTube Face DB and Point-and-Shoot Challenge.
A PDA-based sign translator

We propose an effective approach for a PDA-based sign system and present the sign translator. Its main functions include three parts: detection, recognition and translation. Automatic detection and recognition of text in natural scenes is a prerequisite for the automatic sign translator. In order to make the system robust for text detection in various natural scenes, the detection approach efficiently embeds multi-resolution, adaptive search in a hierarchical framework with different emphases at each layer. We also introduce an intensity-based OCR method to recognize characters in various fonts and lighting conditions, where we employ the Gabor transform to obtain local features, and LDA for selection and classification of features. The recognition rate is 92.4% for the testing set obtained from the natural sign. A sign is different from the normal used sentence. It is brief with a lot of abbreviations and place nouns. We only briefly introduce a rule-based place name translation. We have integrated all these functions in a PDA, which can capture sign images, auto segment and recognize the Chinese sign, and translate it into English.
Self-Error-Correcting Convolutional Neural Network for Learning with Noisy Labels

Convolutional Neural Network (CNN) together with large-scale labeled data has achieved the state-of-theart accuracy in various computer vision tasks. In real-world settings, however, the labels of large scale data can be noisy, which shall seriously degenerate the performance of CNN. In this work, we propose a self-error-correcting CNN (SECCNN) to deal with the noisy labels problem, by simultaneously correcting the improbable labels and optimizing the deep model. Specifically, the SEC-CNN provides an opportunity to correct a wrong label by developing a confidence policy to switch between the label of the sample and the max-activated output neuron of the CNN. Based on the assumption that the deep model is more and more accurate during the training, the confidence policy relies more on the given labels at the beginning stages, but tends to believe that the max-activated neuron of the learned network is reliable. SEC-CNN enables CNN learning to be effective even with 80% noisy labels. Extensive experimental results on MNIST, CIFAR-10, ImageNet and CCFD face dataset demonstrate the effectiveness of the proposed method in dealing with noisy labels.
Probabilistic nearest neighbor search for robust classification of face image sets

Classification with image sets is recently a compelling technique for video-based face recognition. Previous methods in this line mostly assume each image set is pure, i.e., containing well-aligned face images of the same subject, which however is hardly satisfied in real-world applications due to incorrect face detection, questionable tracking, or multiple faces in a single image. This paper proposes a Probabilistic Nearest Neighbor (ProNN) search method to enhance the robustness of NN search against impure image sets by leveraging the statistical distribution of the involved image sets. Specifically, we represent image sets by affine hull, a well-recognized set model, to account for the unseen appearances in each image set. We further exploit a constraint that these unseen appearances statistically follow some pre-specified distribution (Gaussian in this work). Finally, in search of a pair of nearest neighbor points (one per hull), at the same time their distance being minimized, the probability of each point belonging to the same class as that of its corresponding hull is maximized. The proposed ProNN method is evaluated on three widely-studied public databases, Honda/UCSD, YouTube Celebrities and Multiple Biometric Grand Challenge (MBGC), under two kinds of experimental settings where image sets are contaminated either with false positive faces or images of other subjects. Extensive experiments demonstrate the superiority of the proposed approach over state-of-the-art methods.
Automatic detection of signs with affine transformation

In this paper, we propose an approach for detecting signs from natural scenes. The approach efficiently embeds multiresolution, adaptive search, and affine rectification algorithms in a hierarchical framework, with different emphases at each layer. We combine in multi-resolution and multi-scale edge detection techniques to effectively detect text in different sizes. By using the cites from text inside the image, we introduce affine rectification transformation to recover deformation of the text region caused by air inappropriate camera view angle. This procedure can significantly improve text detection rate and OCR (Optical Character Recognition) accuracy. Experimental results have demonstrated feasibility of the proposed algorithms. We have applied the proposed approach to a Chinese sign translation system, which can automatically detect Chinese text input from a camera, recognize the text, and translate the recognized text into English or voice stream.
A Multi-Resolution Dynamic Model for Face Aging Simulation

In this paper we present a dynamic model for simulating face aging process. We adopt a high resolution grammatical face model  and augment it with age and hair features. This model represents all face images by a multi-layer and-or graph and integrates three most prominent aspects related to aging changes: global appearance changes in hair style and shape, deformations and aging effects of facial components, and wrinkles appearance at various facial zones. Then face aging is modeled as a dynamic Markov process on this graph representation which is learned from a large dataset. Given an input image, we firstly compute the graph representation, and then sample the graph structures over various age groups according to the learned dynamic model. Finally we generate new face images with the sampled graphs. Our approach has three novel aspects: (1) the aging model is learned from a dataset of 50,000 adult faces at different ages; (2) we explicitly model the uncertainty in face aging andean sample multiple plausible aged faces for an input image; and (3) we conduct a simple human experiment to validate the simulated aging process.
Face relighting for face recognition under generic illumination

The performance of current face recognition systems suffers heavily from variations in lighting. To deal with this problem, this paper presents a novel illumination normalization approach, by relighting faces to a canonical illumination, based on harmonic images. Benefiting from the observation that human faces share similar shape, and the albedo of the face surface is quasi-constant, we first estimate the nine low-frequency components of the lighting from the input face image. Then, the face image is normalized by relighting it to a canonical illumination, based on the illumination ratio image. For face recognition purposes, two kinds of canonical illumination, uniform and frontal point lighting, are considered, between which the former encodes merely texture information, while the latter encodes both texture and shading information. Our experimental results show that the proposed relighting normalization can significantly improve the performance of a face recognition system.
Two Birds, One Stone: Jointly Learning Binary Code for Large-Scale Face Image Retrieval and Attributes Prediction

We address the challenging large-scale content-based face image retrieval problem, intended as searching images based on the presence of specific subject, given one face image of him/her. To this end, one natural demand is a supervised binary code learning method. While the learned codes might be discriminating, people often have a further expectation that whether some semantic message (e.g., visual attributes) can be read from the human-incomprehensible codes. For this purpose, we propose a novel binary code learning framework by jointly encoding identity discriminability and a number of facial attributes into unified binary code. In this way, the learned binary codes can be applied to not only fine-grained face image retrieval, but also facial attributes prediction, which is the very innovation of this work, just like killing two birds with one stone. To evaluate the effectiveness of the proposed method, extensive experiments are conducted on a new purified large-scale web celebrity database, named CFW 60K, with abundant manual identity and attributes annotation, and experimental results exhibit the superiority of our method over state-of-the-art.
Facial Shape Localization Using Probability Gradient Hints

This letter proposes a novel method to localize facial shape represented by a series of facial landmarks. In our method, the problem of facial shape localization is formulated with a Bayesian inference. Specifically, given a face image, the posterior probability of the facial shape is naturally decomposed into two parts: the likelihood function of local textures and the prior constraints of global shape. The former is provided by the landmark detectors, while the latter is evaluated based on the global shape statistics. The global shape is iteratively estimated in the Maximum A Posteriori (MAP) procedure which is derived in a Lucas-Kanade manner over the probability distribution. Intuitively, in each step, the landmarks are driven by the probability gradient and converge towards the positions which maximize the posterior probability. Experiments on two public databases (XM2VTS and BioID) show the effectiveness of the proposed method.
WLD: A Robust Local Image Descriptor

Inspired by Weber's Law, this paper proposes a simple, yet very powerful and robust local descriptor, called the Weber Local Descriptor (WLD). It is based on the fact that human perception of a pattern depends not only on the change of a stimulus (such as sound, lighting) but also on the original intensity of the stimulus. Specifically, WLD consists of two components: differential excitation and orientation. The differential excitation component is a function of the ratio between two terms: One is the relative intensity differences of a current pixel against its neighbors, the other is the intensity of the current pixel. The orientation component is the gradient orientation of the current pixel. For a given image, we use the two components to construct a concatenated WLD histogram. Experimental results on the Brodatz and KTH-TIPS2-a texture databases show that WLD impressively outperforms the other widely used descriptors (e.g., Gabor and SIFT). In addition, experimental results on human face detection also show a promising performance comparable to the best known results on the MIT+CMU frontal face test set, the AR face data set, and the CMU profile test set.
Spatial Pyramid Covariance-Based Compact Video Code for Robust Face Retrieval in TV-Series.

We address the problem of face video retrieval in TV-series, which searches video clips based on the presence of specific character, given one face track of his/her. This is tremendously challenging because on one hand, faces in TV-series are captured in largely uncontrolled conditions with complex appearance variations, and on the other hand, retrieval task typically needs efficient representation with low time and space complexity. To handle this problem, we propose a compact and discriminative representation for the huge body of video data, named compact video code (CVC). Our method first models the face track by its sample (i.e., frame) covariance matrix to capture the video data variations in a statistical manner. To incorporate discriminative information and obtain more compact video signature suitable for retrieval, the high-dimensional covariance representation is further encoded as a much lower dimensional binary vector, which finally yields the proposed CVC. Specifically, each bit of the code, i.e., each dimension of the binary vector, is produced via supervised learning in a max margin framework, which aims to make a balance between the discriminability and stability of the code. Besides, we further extend the descriptive granularity of covariance matrix from traditional pixel-level to more general patch-level, and proceed to propose a novel hierarchical video representation named spatial pyramid covariance along with a fast calculation method. Face retrieval experiments on two challenging TV-series video databases, i.e., the Big Bang Theory and Prison Break, demonstrate the competitiveness of the proposed CVC over the state-of-the-art retrieval methods. In addition, as a general video matching algorithm, CVC is also evaluated in traditional video face recognition task on a standard Internet database, i.e., YouTube Celebrities, showing its quite promising performance by using an extremely compact code with only 128 bits.
Structured Sparse Linear Discriminant Analysis

Linear Discriminant Analysis (LDA) is an efficient image feature extraction technique by supervised dimensionality reduction. In this paper, we extend LDA to Structured Sparse LDA (SSLDA), where the projecting vectors are not only constrained to sparsity but also structured with a pre-specified set of shapes. While the sparse priors deal with small sample size problem, the proposed structure regularization can also encode higher-order information with better interpretability. We also propose a simple and efficient optimization algorithm to solve the proposed optimization problem. Experiments on face images show the benefits of the proposed structured sparse LDA on both classification accuracy and interpretability.
Noisy Face Image Sets Refining Collaborated with Discriminant Feature Space Learning

Large-scale face data together with deep learningtechnology have significantly improved the performance of facerecognition in the wild. Hereinto, the large-scale face data playsa fundamental role, and it is nontrivial to collect a large-scaleface dataset with accurate class labels. No wonder it is quitemoney and effort consuming by collecting manually, howeverit is easy to access large scale face images by using a searchengine with names as keywords. Unfortunately, the retrievedface images from search engine are usually messed up withsome noise images with wrong labels, which forms a greatneed of developing algorithms to refine the retrieved noisyface image set. In this work, we propose a joint frameworkin which multiple noisy face image sets refining collaborateswith the discriminant feature space learning. Specifically, thetwo modules, refining each noisy face image set by conductingone-class classification based on learnt discriminant feature andlearning discriminant feature space based on refined face imagesets, are updated iteratively inducing an effective refinementmodel. To investigate the proposed method, we collect a realworlddataset for the evaluation including 15,515 images of46 subjects with 40% ∼ 63.5% noise images per subject. Theexperimental results demonstrate that state-of-the-art one-classclassification methods can be significantly improved whenbeing embedded in the proposed framework, and the proposedframework exhibits strong robustness even when the mean noiseproportion is up to 50% ∼ 80%.