nils-herrmann/clean_scirepeval huggingface dataset:
Generating Visual Representations for Zero-Shot Classification

This paper addresses the task of learning an image classifier when some categories are defined by semantic descriptions only (e.g. visual attributes) while the others are defined by exemplar images as well. This task is often referred to as the Zero-Shot classification task (ZSC). Most of the previous methods rely on learning a common embedding space allowing to compare visual features of unknown categories with semantic descriptions. This paper argues that these approaches are limited as i) efficient discriminative classifiers can't be used ii) classification tasks with seen and unseen categories (Generalized Zero-Shot Classification or GZSC) can't be addressed efficiently. In contrast, this paper suggests to address ZSC and GZSC by i) learning a conditional generator using seen classes ii) generate artificial training examples for the categories without exemplars. ZSC is then turned into a standard supervised learning problem. Experiments with 4 generative models and 5 datasets experimentally validate the approach, giving state-of-the-art results on both ZSC and GZSC.
Automatic Discovery of Discriminative Parts as a Quadratic Assignment Problem

Part-based image classification consists in representing categories by small sets of discriminative parts upon which a representation of the images is built. This paper addresses the question of how to automatically learn such parts from a set of labeled training images. We propose to cast the training of parts as a quadratic assignment problem in which optimal correspondences between image regions and parts are automatically learned. The paper analyses different assignment strategies and thoroughly evaluates them on two public datasets: Willow actions and MIT 67 scenes.
On the use of deep neural networks for the detection of small vehicles in ortho-images

This paper addresses the question of the detection of small targets (vehicles) in ortho-images. This question differs from the general task of detecting objects in images by several aspects. First, the vehicles to be detected are small, typically smaller than 20×20 pixels. Second, due to the multifarious-ness of the landscapes of the earth, several pixel structures similar to that of a vehicle might emerge (roof tops, shadow patterns, rocks, buildings), whereas within the vehicle class the inter-class variability is limited as they all look alike from afar. Finally, the imbalance between the vehicles and the rest of the picture is enormous in most cases. Specifically, this paper is focused on the detection tasks introduced by the VEDAI dataset . This work supports an extensive study of the problems one might face when applying deep neural networks with low resolution and scarce data and proposes some solutions. One of the contributions of this paper is a network severely outperforming the state-of-the-art while being much simpler to implement and a lot faster than competitive approaches. We also list the limitations of this approach and provide several new ideas to further improve our results.
Learning Visual Similarity Measures for Comparing Never Seen Objects

In this paper we propose and evaluate an algorithm that learns a similarity measure for comparing never seen objects. The measure is learned from pairs of training images labeled "same" or "different". This is far less informative than the commonly used individual image labels (e.g., "car model X"), but it is cheaper to obtain. The proposed algorithm learns the characteristic differences between local descriptors sampled from pairs of "same" and "different" images. These differences are vector quantized by an ensemble of extremely randomized binary trees, and the similarity measure is computed from the quantized differences. The extremely randomized trees are fast to learn, robust due to the redundant information they carry and they have been proved to be very good clusterers. Furthermore, the trees efficiently combine different feature types (SIFT and geometry). We evaluate our innovative similarity measure on four very different datasets and consistently outperform the state-of-the-art competitive approaches.
Pattern localization in historical document images via template matching

Template matching is a classical and essential step in many pattern recognition, object detection or video tracking systems. This paper aims at integrating and evaluating different template matching methods in the context of pattern spotting in historical document images — i.e. the search for occurrences of a given visual pattern in document images. Given a query image, our pattern spotting system first computes the similarity score between the query signature and the signatures of a few regions provided by a region proposal algorithm. The top ranked regions are then selected for further processing. Template Matching is then applied in the neighborhood of the selected regions to precisely locate and rank the candidate windows that maximize the matching score. This paper builds upon popular template matching approaches and provides a unified testing framework for historical document image pattern spotting. The experimentation offers an insight on how to choose the most promising techniques for historical document images. This paper also proposes an improvement over these standard template matching approaches to significantly increase the overall performance.
PCCA: A new approach for distance learning from sparse pairwise constraints

This paper introduces Pairwise Constrained Component Analysis (PCCA), a new algorithm for learning distance metrics from sparse pairwise similarity/dissimilarity constraints in high dimensional input space, problem for which most existing distance metric learning approaches are not adapted. PCCA learns a projection into a low-dimensional space where the distance between pairs of data points respects the desired constraints, exhibiting good generalization properties in presence of high dimensional data. The paper also shows how to efficiently kernelize the approach. PCCA is experimentally validated on two challenging vision tasks, face verification and person re-identification, for which we obtain state-of-the-art results.
Recognition of 3D textured objects by mixing view-based and model-based representations

A strategy combining advantages of view-based and model-based object recognition approaches has been developed. Textured 3D models are used to produce local appearances of object key-points. Correspondences between 3D points and local appearances are also established during this learning stage. The recognition consists first in matching these local appearances and those extracted from the image. A robust 3D pose estimation is then carried out, discarding spurious correspondences. This algorithm is computationally very efficient. It can analyse images including several objects and can also handle partial object occultations as well as important changes of brightness and contrast.
Unifying discriminative visual codebook generation with classifier training for object category recognition

The idea of representing images using a bag of visual words is currently popular in object category recognition. Since this representation is typically constructed using unsupervised clustering, the resulting visual words may not capture the desired information. Recent work has explored the construction of discriminative visual codebooks that explicitly consider object category information. However, since the codebook generation process is still disconnected from that of classifier training, the set of resulting visual words, while individually discriminative, may not be those best suited for the classifier. This paper proposes a novel optimization framework that unifies codebook generation with classifier training. In our approach, each image feature is encoded by a sequence of ldquovisual bitsrdquo optimized for each category. An image, which can contain objects from multiple categories, is represented using aggregates of visual bits for each category. Classifiers associated with different categories determine how well a given image corresponds to each category. Based on the performance of these classifiers on the training data, we augment the visual words by generating additional bits. The classifiers are then updated to incorporate the new representation. These two phases are repeated until the desired performance is achieved. Experiments compare our approach to standard clustering-based methods and with state-of-the-art discriminative visual codebook generation. The significant improvements over previous techniques clearly demonstrate the value of unifying representation and classification into a single optimization framework.
Visual word disambiguation by semantic contexts

This paper presents a novel schema to address the polysemy of visual words in the widely used bag-of-words model. As a visual word may have multiple meanings, we show it is possible to use semantic contexts to disambiguate these meanings and therefore improve the performance of bag-of-words model. On one hand, for an image, multiple context-specific bag-of-words histograms are constructed, each of which corresponds to a semantic context. Then these histograms are merged by selecting only the most discriminative context for each visual word, resulting in a compact image representation. On the other hand, an image is represented by the occurrence probabilities of semantic contexts. Finally, when classifying an image, two image representations are combined at decision level to utilize the complementary information embedded in them. Experiments on three challenging image databases (PASCAL VOC 2007, Scene-15 and MSRCv2) show that our method significantly outperforms state-of-the-art classification methods.
Hyperplane Approximation for Template Matching

Hager and Belhumeur (1998) proposed a general framework for object tracking in video images. It consists of low-order parametric models for the image motion of a target region. These models are used to predict movement and to track the target. The difference in intensity between the pixels belonging to the current region and the pixels of the selected target (learned during an offline stage) allows a straightforward prediction of the region position in the current image. The main aim of the article is to propose an important improvement within this framework, making the convergence faster with the same amount of online computation.
Learned color constancy from local correspondences

The ability of humans for color constancy, i.e. the ability to correct for color deviation caused by a different illumination, is far beyond computer vision performances: nowadays, automatic color constancy is still a difficult problem. This article proposes a new step forward towards solving this color constancy problem. Basically, it consists in learning how illumination can affect some reference objects. During a learning stage, images are taken under various illuminations, allowing for automatic building of a model explaining color changes. The model can explain complex non-linear color transformations with only a few parameters. Therefore, the observation of color variations in a few reference regions (e.g. known object) is enough to estimate the global color changes.
A Supervised Clustering Algorithm for the Initialization of RBF Neural Network Classifiers

In this paper, we propose a new supervised clustering algorithm, coined as the homogeneous clustering (HC), to find the number and initial locations of the hidden units in radial basis function (RBF) neural network classifiers. In contrast to the traditional clustering algorithms introduced for this goal, the proposed algorithm is a supervised procedure where the number and initial locations of the hidden units are determined based on split of the clusters having overlaps among the classes. The basic idea of the proposed approach is to create class specific homogenous clusters where the corresponding samples are closer to their mean than the means of rival clusters belonging to other class categories. We tested the proposed clustering algorithm along with the RBF network classifier on the Graz02 object database and the ORL face database. The experimental results show that the RBF network classifier performs better when it is initialized with the proposed HC algorithm than an unsupervised k-means algorithm. Moreover, our recognition results exceed the best published results on the Graz02 database and they are comparable to the best results on the ORL face database indicating that the proposed clustering algorithm initializes the hidden unit parameters successfully.
Margin-based discriminant dimensionality reduction for visual recognition

Nearest neighbour classifiers and related kernel methods often perform poorly in high dimensional problems because it is infeasible to include enough training samples to cover the class regions densely. In such cases, test samples often fall into gaps between training samples where the nearest neighbours are too distant to be good indicators of class membership. One solution is to project the data onto a discriminative lower dimensional subspace. We propose a gap-resistant nonparametric method for finding such subspaces: first the gaps are filled by building a convex model of the region spanned by each class - we test the affine and convex hulls and the bounding disk of the class training samples - then a set of highly discriminative directions is found by building and decomposing a scatter matrix of weighted displacement vectors from training examples to nearby rival class regions. The weights are chosen to focus attention on narrow margin cases while still allowing more diversity and hence more discriminability than the 1D linear Support Vector Machine (SVM) projection. Experimental results on several face and object recognition datasets show that the method finds effective projections, allowing simple classifiers such as nearest neighbours to work well in the low dimensional reduced space.
A simple and efficient template matching algorithm

We propose a general framework for object tracking in video images. It consists of low-order parametric models for the image motion of a target region. These models are used to predict the movement and to track the target. The difference of intensity between the pixels belonging to the current region and the pixels of the selected target (learnt during an off-line stage) allows a straightforward prediction of the region position in the current image. The proposed algorithm allows to track in real time (less than 10 ms) any planar textured target under homographic motions. This algorithm is very simple (a few lines of code) and very efficient (less than 10 ms on a 150 MHz hardware).
Creating efficient codebooks for visual recognition

Visual codebook based quantization of robust appearance descriptors extracted from local image patches is an effective means of capturing image statistics for texture analysis and scene classification. Codebooks are usually constructed by using a method such as k-means to cluster the descriptor vectors of patches sampled either densely ('textons') or sparsely ('bags of features' based on key-points or salience measures) from a set of training images. This works well for texture analysis in homogeneous images, but the images that arise in natural object recognition tasks have far less uniform statistics. We show that for dense sampling, k-means over-adapts to this, clustering centres almost exclusively around the densest few regions in descriptor space and thus failing to code other informative regions. This gives suboptimal codes that are no better than using randomly selected centres. We describe a scalable acceptance-radius based clusterer that generates better codebooks and study its performance on several image classification tasks. We also show that dense representations outperform equivalent keypoint based ones on these tasks and that SVM or mutual information based feature selection starting from a dense codebook further improves the performance.
Towards autonomous object reconstruction for visual search by the humanoid robot HRP-2

This paper deals with the problem of object reconstruction for visual search by a humanoid robot. Three problems necessary to achieve the behavior autonomously are considered: full-body motion generation according to a camera pose, general object representation for visual recognition and pose estimation, and far-away visual detection of an object. First we deal with the problem of generating full body motion for a HRP-2 humanoid robot to achieve camera pose given by a Next Best View algorithm. We use an optimization based approach including self-collision avoidance. This is made possible by a body to body distance function having a continuous gradient. The second problem has received a lot of attention for several decades, and we present a solution based on 3D vision together with SIFTs descriptor, making use of the information available from the robot. It is shown in this paper that one of the major limitation of this model is the perception distance. Thus a new approach based on a generative object model is presented to cope with more difficult situations. It relies on a local representation which allows handling occlusion as well as large scale and pose variations.
Discovering and Aligning Discriminative Mid-level Features for Image Classification

This paper proposes a new algorithm for image recognition, which consists of (i) modeling categories as a set of distinctive parts that are discovered automatically, (ii) aligning them across images while learning their visual model, and, finally (iii) encode images as sets of part descriptors. The so-obtained parts are free of any appearance constraint and are optimized to allow the distinction between the categories to be recognized. The algorithm starts by extracting a set of random regions from the images of different classes, and, using a soft assign-like matching algorithm, simultaneously learns the model of each part and assigns image regions to the model's parts. Once the model of the category is trained, it can be used to classify new images by first finding image's regions similar to learned parts and encoding them by the fisher-on-parts encoding, which is another contribution of this paper. The proposed framework is experimentally validated on two publicly available datasets, on which state-of-the-art performance is obtained.
Local Subspace Classifiers: Linear and Nonlinear Approaches

The K-local hyperplane distance nearest neighbor (HKNN) algorithm is a local classification method which builds nonlinear decision surfaces directly in the original sample space by using local linear manifolds. Although the HKNN method has been successfully applied in several classification tasks, it is not possible to employ distance metrics other than the Euclidean distances in this scheme, which can be considered as a major limitation of the method. In this paper we formulate the HKNN method in terms of subspaces. Advantages of the subspace formulation of the method are two-fold. First, it enables us to propose a variant of the HKNN algorithm, the local discriminative common vector (LDCV) method, which is more suitable for classification tasks where classes have similar intra-class variations. Second, the HKNN method along with the proposed method can be extended to the nonlinear case based on subspace concepts. As a result of the nonlinearization process, one may utilize a wide variety of distance functions in those local classifiers. We tested the proposed methods on several classification tasks. Experimental results show that the proposed methods yield comparable or better results than the support vector machine (SVM) classifier and its local counterpart SVM-KNN.
Combining efficient object localization and image classification

In this paper we present a combined approach for object localization and classification. Our contribution is twofold. (a) A contextual combination of localization and classification which shows that classification can improve detection and vice versa. (b) An efficient two stage sliding window object localization method that combines the efficiency of a linear classifier with the robustness of a sophisticated non-linear one. Experimental results evaluate the parameters of our two stage sliding window approach and show that our combined object localization and classification methods outperform the state-of-the-art on the PASCAL VOC 2007 and 2008 datasets.
Modeling spatial layout with fisher vectors for image categorization

We introduce an extension of bag-of-words image representations to encode spatial layout. Using the Fisher kernel framework we derive a representation that encodes the spatial mean and the variance of image regions associated with visual words. We extend this representation by using a Gaussian mixture model to encode spatial layout, and show that this model is related to a soft-assign version of the spatial pyramid representation. We also combine our representation of spatial layout with the use of Fisher kernels to encode the appearance of local features. Through an extensive experimental evaluation, we show that our representation yields state-of-the-art image categorization results, while being more compact than spatial pyramid representations. In particular, using Fisher kernels to encode both appearance and spatial layout results in an image representation that is computationally efficient, compact, and yields excellent performance while using linear classifiers.
Unsupervised deep hashing with stacked convolutional autoencoders

Learning-based image hashing consists in turning high-dimensional image features into compact binary codes, while preserving their semantic similarity (i.e., if two images are close in terms of content, their codes should be close as well). In this context, many existing hashing techniques rely on supervision for preserving these semantic properties. In this paper, we aim at learning such binary codes by exploiting the underlying structure of unlabeled data, using deep learning. The proposed deep network is based on a stacked convolutional autoencoder which hierarchically maps input images into a low-dimensional space. A binary relaxation constraint applied to the middle layer of the network — the one containing the code — makes the codes sparse and binary. To demonstrate the competitiveness of the proposed architecture, we evaluate the so produced hash codes on image retrieval and image classification tasks on the MNIST dataset, and compare its performance with state-of-the-art approaches.
Vehicle Categorization: Parts for Speed and Accuracy

In this paper we propose a framework for categorization of different types of vehicles. The difficulty comes from the high inter-class similarity and the high intra-class variability. We address this problem using a part-based recognition system. We particularly focus on the trade-off between the number of parts included in the vehicle models and the recognition rate, i.e the trade-off between fast computation and high accuracy. We propose a high-level data transformation algorithm and a feature selection scheme adapted to hierarchical SVM classifiers to improve the performance of part-based vehicle models. We have tested the proposed framework on real data acquired by infrared surveillance cameras, and on visible images too. On the infrared dataset, with the same speedup factor of 100, our accuracy is 12% better than the standard one-versus-one SVM.