nils-herrmann/clean_scirepeval huggingface dataset:
Probabilistic coarse-to-fine object recognition

A coarse-to-fine probabilistic model for detecting objects in images is presented. Objects are composed of constellations of features, and features from a same object share the common reference frame of the image in which they are detected. Features appearance and pose are modeled by probabilistic distributions, the parameters of which are shared across features in order to allow training from few examples. In order to avoid an expensive combinatorial search, we propose a coarseto-fine strategy, inspired by the work by Lowe [30, 32] and by Geman and collaborators [1, 19]. Well-established, simple and inexpensive steps are used as successive refinement blocks that discard incorrect matching hypotheses in a cascade. The candidate hypotheses output by our algorithm are evaluated by a generative probabilistic model that takes into account each stage of the matching process. We apply our ideas to the problem of individual object recognition and test our method on several data-sets. We compare with Lowe’s algorithm and demonstrate significantly better performance.
Combining generative models and Fisher kernels for object recognition

Learning models for detecting and classifying object categories is a challenging problem in machine vision. While discriminative approaches to learning and classification have, in principle, superior performance, generative approaches provide many useful features, one of which is the ability to naturally establish explicit correspondence between model components and scene features - this, in turn, allows for the handling of missing data and unsupervised learning in clutter. We explore a hybrid generative/discriminative approach using 'Fisher kernels' by Jaakkola and Haussler (1999) which retains most of the desirable properties of generative methods, while increasing the classification performance through a discriminative setting. Furthermore, we demonstrate how this kernel framework can be used to combine different types of features and models into a single classifier. Our experiments, conducted on a number of popular benchmarks, show strong performance improvements over the corresponding generative approach and are competitive with the best results reported in the literature.
Sampling Methods for Unsupervised Learning

We present an algorithm to overcome the local maxima problem in estimating the parameters of mixture models. It combines existing approaches from both EM and a robust fitting algorithm, RANSAC, to give a data-driven stochastic learning scheme. Minimal subsets of data points, sufficient to constrain the parameters of the model, are drawn from proposal densities to discover new regions of high likelihood. The proposal densities are learnt using EM and bias the sampling toward promising solutions. The algorithm is computationally efficient, as well as effective at escaping from local maxima. We compare it with alternative methods, including EM and RANSAC, on both challenging synthetic data and the computer vision problem of alpha-matting.
Common-Frame Model for Object Recognition

A generative probabilistic model for objects in images is presented. An object consists of a constellation of features. Feature appearance and pose are modeled probabilistically. Scene images are generated by drawing a set of objects from a given database, with random clutter sprinkled on the remaining image surface. Occlusion is allowed. 
 
We study the case where features from the same object share a common reference frame. Moreover, parameters for shape and appearance densities are shared across features. This is to be contrasted with previous work on probabilistic 'constellation' models where features depend on each other, and each feature and model have different pose and appearance statistics [1, 2]. These two differences allow us to build models containing hundreds of features, as well as to train each model from a single example. Our model may also be thought of as a probabilistic revisitation of Lowe's model [3, 4]. 
 
We propose an efficient entropy-minimization inference algorithm that constructs the best interpretation of a scene as a collection of objects and clutter. We test our ideas with experiments on two image databases. We compare with Lowe's algorithm and demonstrate better performance, in particular in presence of large amounts of background clutter.
Recovering Local Shape of a Mirror Surface from Reflection of a Regular Grid

We present a new technique to recover the shape of an unknown smooth specular surface from a single image. A calibrated camera faces a specular surface reflecting a calibrated scene (for instance a checkerboard or grid pattern). The mapping from the scene pattern to its reflected distorted image in the camera changes the local geometrical structure of the scene pattern. We show that if measurements of both local orientation and scale of the distorted scene in the image plane are available, this mapping can be inverted. Specifically, we prove that surface position and shape up to third order can be derived as a function of such local measurements when two orientations are available at the same point (e.g. a corner). Our results generalize previous work [1, 2] where the mirror surface geometry was recovered only up to first order from at least three intersecting lines. We validate our theoretical results with both numerical simulations and experiments with real surfaces.
Semi-supervised learning and recognition of object classes

We present a method for learning object classes without supervision: learning is scale and translation-invariant, does not require alignment nor correspondence between the training images, and is robust to clutter and occlusion. Class models are probabilistic constellations of parts, and their parameters are estimated by maximizing the likelihood of the training data. The appearance of the parts, as well as their mutual position, relative scale and probability of detection are explicitly described in the model. We investigate two models differing in their representation of the part’s configuration. Parts may be regions, representing object appearance patches, or curves representing the local object shape. Recognition takes place in two stages. First, a feature-finder identifies promising locations for the models’ part. Second, the category model is used to compare the likelihood that the observed features are generated by the model, or is generated by background clutter. The flexible nature of the model is demonstrated by excellent results over six diverse object classes including geometrically constrained classes (e.g. faces, cars) and flexible objects (such as animals). We also describe an application of a learnt model to object based retrieval of frames from the situation comedy ‘Fawlty Towers’.
Self-Tuning Spectral Clustering

We study a number of open issues in spectral clustering: (i) Selecting the appropriate scale of analysis, (ii) Handling multi-scale data, (iii) Clustering with irregular background clutter, and, (iv) Finding automatically the number of groups. We first propose that a 'local' scale should be used to compute the affinity between each pair of points. This local scaling leads to better clustering especially when the data includes multiple scales and when the clusters are placed within a cluttered background. We further suggest exploiting the structure of the eigenvectors to infer automatically the number of groups. This leads to a new algorithm in which the final randomly initialized k-means stage is eliminated.
Is bottom-up attention useful for object recognition?

A key problem in learning multiple objects from unlabeled images is that it is a priori impossible to tell which part of the image corresponds to each individual object, and which part is irrelevant clutter which is not associated to the objects. We investigate empirically to what extent pure bottom-up attention can extract useful information about the location, size and shape of objects from images and demonstrate how this information can be utilized to enable unsupervised learning of objects from unlabeled images. Our experiments demonstrate that the proposed approach to using bottom-up attention is indeed useful for a variety of applications.
What do reflections tell us about the shape of a mirror?

Three-dimensional shape may be perceived from static images. Contours, shading, texture gradients, perspective and occlusion are well-studied cues to this percept. When looking at a picture of a specular object, such as a silver vase, one additional cue is potentially available: a deformed picture of the reflected environment is seen in the surface of the object and the amount and type of deformation depend on its shape. Can specular reflections be used as a visual cue for shape perception? Our experiments show that our subjects are very poor at judging the shape of mirror surfaces in absence of other visual cues. However, for a considerable subset of the stimuli, subjects are highly consistent in their (most often wrong) perception. This observation leads us to the hypothesis that our subjects rather than 'computing' a percept from each image based on geometrical considerations, may be associating a shape to each pattern in a stereotypical way, akin to pattern-matching. This behavior is reasonable since, as suggested by our ideal observer analysis, the information available from specular reflections is ambiguous when the surrounding world is (partially) unknown.
3D Photography Using Shadows in Dual-Space Geometry

A simple and inexpensive approach for extracting the three-dimensional shape of objects is presented. It is based on ‘weak structured lighting’. It requires very little hardware besides the camera: a light source (a desk-lamp or the sun), a stick and a checkerboard. The object, illuminated by the light source, is placed on a stage composed of a ground plane and a back plane; the camera faces the object. The user moves the stick in front of the light source, casting a moving shadow on the scene. The 3D shape of the object is extracted from the spatial and temporal location of the observed shadow. Experimental results are presented on five different scenes (indoor with a desk lamp and outdoor with the sun) demonstrating that the error in reconstructing the surface is less than 0.5% of the size of the object. A mathematical formalism is proposed that simplifies the notation and keep the algebra compact. A real-time implementation of the system is also presented.
Recursive 3-D Visual Motion Estimation Using Subspace Constraints

The 3-D motion of a camera within a static environment produces a sequence of time-varying images that can be used for reconstructing the relative motion between the scene and the viewer. The problem of reconstructing rigid motion from a sequence of perspective images may be characterized as the estimation of the state of a nonlinear dynamical system, which is defined by the rigidity constraint and the perspective measurement map. The time-derivative of the measured output of such a system, which is called the “2-D motion field” and is approximated by the “optical flow”, is bilinear in the motion parameters, and may be used to specify a subspace constraint on the direction of heading independent of rotation and depth, and a pseudo-measurement for the rotational velocity as a function of the estimated heading. The subspace constraint may be viewed as an implicit dynamical model with parameters on a differentiable manifold, and the visual motion estimation problem may be cast in a system-theoretic framework as the identification of such an implicit model. We use techniques which pertain to nonlinear estimation and identification theory to recursively estimate 3-D rigid motion from a sequence of images independent of the structure of the scene. Such independence from scene-structure allows us to deal with a variable number of visible feature-points and occlusions in a principled way. The further decoupling of the direction of heading from the rotational velocity generates a filter with a state that belongs to a two-dimensional and highly constrained state-space. As a result, the filter exhibits robustness properties which are highlighted in a series of experiments on real and noisy synthetic image sequences. While the position of feature-points is not part of the state of the model, the innovation process of the filter describes how each feature is compatible with a rigid motion interpretation, which allows us to test for outliers and makes the filter robust with respect to errors in the feature tracking/optical flow, reflections, T-junctions. Once motion has been estimated, the 3-D structure of the scene follows easily. By releasing the constraint that the visible points lie in front of the viewer, one may explain some psychophysical effects on the nonrigid percept of rigidly moving objects.
Recognition by Probabilistic Hypothesis Construction

We present a probabilistic framework for recognizing objects in images of cluttered scenes. Hundreds of objects may be considered and searched in parallel. Each object is learned from a single training image and modeled by the visual appearance of a set of features, and their position with respect to a common reference frame. The recognition process computes identity and position of objects in the scene by finding the best interpretation of the scene in terms of learned objects. Features detected in an input image are either paired with database features, or marked as clutters. Each hypothesis is scored using a generative model of the image which is defined using the learned objects and a model for clutter. While the space of possible hypotheses is enormously large, one may find the best hypothesis efficiently – we explore some heuristics to do so. Our algorithm compares favorably with state-of-the-art recognition systems.
Systeme et procede de selection attentionnelle

L'invention concerne un systeme et un procede de selection attentionnelle. Plus particulierement, l'invention concerne un systeme et un procede de selection et d'isolement automatises de regions saillantes susceptibles de contenir des objets, fondes sur une attention visuelle ascendante, pour permettre un apprentissage non supervise en une fois d'objets multiples dans des images encombrees.
Learning to Recognize Volcanoes on Venus

Dramatic improvements in sensor and image acquisition technology have created a demand for automated tools that can aid in the analysis of large image databases. We describe the development of JARtool, a trainable software system that learns to recognize volcanoes in a large data set of Venusian imagery. A machine learning approach is used because it is much easier for geologists to identify examples of volcanoes in the imagery than it is to specify domain knowledge as a set of pixel-level constraints. This approach can also provide portability to other domains without the need for explicit reprogramming; the user simply supplies the system with a new set of training examples. We show how the development of such a system requires a completely different set of skills than are required for applying machine learning to “toy world” domains. This paper discusses important aspects of the application process not commonly encountered in the “toy world,” including obtaining labeled training data, the difficulties of working with pixel data, and the automatic extraction of higher-level features.
51-Preajttentive texture discrimination with early vision

We preEent a model of human preattentive texture perception. This model consists of three stages: (1) convolution of the image with a bank of even-symmetric linear filters followed by half-wave rectification to give a set of responses modeling outputs of VI simple cells, (2) inhibition, localized in space, within and among the neuralresponse profiles that results in the suppression of weak responses when there are strong responses at the same or nearby Jocations, and (3) texture-boundary detection by using wide odd-symmetric mechanisms. Our model can predict the salience of texture boundaries in any arbitrary gray-scale image. A computer implementation of this model has been tested on many of the classic stimuli from psychophysical literature. Quantitative predictions of the degIee of discriminability of different texture pairs match well with experimental measurements of discriminability in human observers. leiI
Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories

Current computational approaches to learning visual object categories require thousands of training images, are slow, cannot learn in an incremental manner and cannot incorporate prior information into the learning process. In addition, no algorithm presented in the literature has been tested on more than a handful of object categories. We present an method for learning object categories from just a few training images. It is quick and it uses prior information in a principled way. We test it on a dataset composed of images of objects belonging to 101 widely varied categories. Our proposed method is based on making use of prior information, assembled from (unrelated) object categories which were previously learnt. A generative probabilistic model is used, which represents the shape and appearance of a constellation of features belonging to the object. The parameters of the model are learnt incrementally in a Bayesian manner. Our incremental algorithm is compared experimentally to an earlier batch Bayesian algorithm, as well as to one based on maximum-likelihood. The incremental and batch versions have comparable classification performance on small training sets, but incremental learning is significantly faster, making real-time learning feasible. Both Bayesian methods outperform maximum likelihood on small training sets.
On the usefulness of attention for object recognition

Today’s object recognition systems have become very good at learning and recognizing isolated objects or objects in images with little clutter. However, unsupervised learning and recognition in highly cluttered scenes or in scenes with multiple objects are still problematic. Faced with the same issue, the brain employs selective visual attention to select relevant parts of the image and to serialize the perception of individual objects. In this paper we demonstrate the use of a computational model of bottom-up visual attention for object recognition in machine vision. By comparing the performance of David Lowe’s recognition algorithm with and without attention, we quantify the usefulness of attention for learning and recognizing multiple objects from complex scenes, and for learning and recognizing objects in scenes with large amounts of clutter.
CMOS imager with embedded analog early image processor

We present a high quality computational CMOS imager with integrated early image processing general-purpose filter. The early processor identifies in real-time areas of interest within the field of view, and reduces data volume and data-bandwidth to the external processor. Unlike other computational imagers, this approach does not compromise signal integrity, and unlike high-speed imagers, it does not overwhelm the external processor. On focal plane integration of image pre-processing allows an efficient implementation of a variety of computationally intensive applications such as autonomous navigation, object avoidance or intercept, and recognition. A proof of concept imager has been built and high-speed operation of the imager-processor has been confirmed.
The Full Images for Natural Knowledge Caltech Office DB

We describe the image collection and labeling procedures of an office scene database. The office database is aimed at providing training and benchmarking tools for multiclass object detection in natural scenes. The described database is unique in the fact that high quality office images were collected following a protocol that (in theory) does not introduce a human bias in camera positioning and environment manipulation. 225 360deg office images all labeled with 14 objects are available. The objects are: LightSwitch, window, doorFrame, bookshelf, Monitor, whiteboard, Keyboard, chairBack, Television, roundClock, Laptop, cabinet, PictureFrame and openShelf. The use of the office database is conditioned by a contribution to the labeling effort.
Unsupervised Learning of Human Motion

An unsupervised learning algorithm that can obtain a probabilistic model of an object composed of a collection of parts (a moving human body in our examples) automatically from unlabeled training data is presented. The training data include both useful "foreground" features as well as features that arise from irrelevant background clutter - the correspondence between parts and detected features is unknown. The joint probability density function of the parts is represented by a mixture of decomposable triangulated graphs which allow for fast detection. To learn the model structure as well as model parameters, an EM-like algorithm is developed where the labeling of the data (part assignments) is treated as hidden variables. The unsupervised learning technique is not limited to decomposable triangulated graphs. The efficiency and effectiveness of our algorithm is demonstrated by applying it to generate models of human motion automatically from unlabeled image sequences, and testing the learned models on a variety of sequences.
An Improved Scheme for Detection and Labelling in Johansson Displays

Consider a number of moving points, where each point is attached to a joint of the human body and projected onto an image plane. Johannson showed that humans can effortlessly detect and recognize the presence of other humans from such displays. This is true even when some of the body points are missing (e.g. because of occlusion) and unrelated clutter points are added to the display. We are interested in replicating this ability in a machine. To this end, we present a labelling and detection scheme in a probabilistic framework. Our method is based on representing the joint probability density of positions and velocities of body points with a graphical model, and using Loopy Belief Propagation to calculate a likely interpretation of the scene. Furthermore, we introduce a global variable representing the body's centroid. Experiments on one motion-captured sequence suggest that our scheme improves on the accuracy of a previous approach based on triangulated graphical models, especially when very few parts are visible. The improvement is due both to the more general graph structure we use and, more significantly, to the introduction of the centroid variable.
Mutual Boosting for Contextual Inference

Mutual Boosting is a method aimed at incorporating contextual information to augment object detection. When multiple detectors of objects and parts are trained in parallel using AdaBoost , object detectors might use the remaining intermediate detectors to enrich the weak learner set. This method generalizes the efficient features suggested by Viola and Jones  thus enabling information inference between parts and objects in a compositional hierarchy. In our experiments eye-, nose-, mouth- and face detectors are trained using the Mutual Boosting framework. Results show that the method outperforms applications overlooking contextual information. We suggest that achieving contextual integration is a step toward human-like detection capabilities.