nils-herrmann/clean_scirepeval huggingface dataset:
Object class recognition by unsupervised scale-invariant learning

We present a method to learn and recognize object class models from unlabeled and unsegmented cluttered scenes in a scale invariant manner. Objects are modeled as flexible constellations of parts. A probabilistic representation is used for all aspects of the object: shape, appearance, occlusion and relative scale. An entropy-based feature detector is used to select regions and their scale within the image. In learning the parameters of the scale-invariant object model are estimated. This is done using expectation-maximization in a maximum-likelihood setting. In recognition, this model is used in a Bayesian manner to classify images. The flexible nature of the model is demonstrated by excellent results over a range of datasets including geometrically constrained classes (e.g. faces, cars) and flexible objects (such as animals).
A Bayesian approach to unsupervised one-shot learning of object categories

A Bayesian network formulation for relational shape matching is presented. The main advantage of the relational shape matching approach is the obviation of the nonrigid spatial mappings used by recent nonrigid matching approaches. The basic variables that need to be estimated in the relational shape matching objective function are the global rotation and scale and the local displacements and correspondences. The new Bethe free energy approach is used to estimate the pairwise correspondences between links of the template graphs and the data. The resulting framework is useful in both registration and recognition contexts. Results are shown on hand-drawn templates and on 2D transverse T1-weighted MR images.
Mutual Boost ing for Contextual Inference

Mutual Boosting is a method aimed at incorporating contextual information to augment object detection. When multiple detectors of objects and parts are trained in parallel using AdaBoost , object detectors might use the remaining intermediate detectors to enrich the weak learner set. This method generalizes the efficient features suggested by Viola and Jones  thus enabling information inference between parts and objects in a compositional hierarchy. Eye-, nose-, mouthand face-detectors are trained using the Mutual Boosting framework. Results show that the method outperforms applications overlooking contextual information. Mutual referencing of object detectors appears in surprisingly early stages of the training process, thus indicating that contextual information has an essential contribution to object detection processes. We suggest that achieving contextual integration is a step toward human-like detection capabilities.
A Bayesian approach to unsupervised one-shot learning of object categories

Learning visual models of object categories notoriously requires thousands of training examples; this is due to the diversity and richness of object appearance which requires models containing hundreds of parameters. We present a method for learning object categories from just a few images (1 /spl sim/ 5). It is based on incorporating "generic" knowledge which may be obtained from previously learnt models of unrelated categories. We operate in a variational Bayesian framework: object categories are represented by probabilistic models, and "prior" knowledge is represented as a probability density function on the parameters of these models. The "posterior" model for an object category is obtained by updating the prior in the light of one or more observations. Our ideas are demonstrated on four diverse categories (human faces, airplanes, motorcycles, spotted cats). Initially three categories are learnt from hundreds of training examples, and a "prior" is estimated from these. Then the model of the fourth category is learnt from 1 to 5 training examples, and is used for detecting new exemplars a set of test images.
Classification of human actions into dynamics based primitives with application to drawing tasks

We develop the study of primitives of human motion, which we refer to as movemes. The idea is to understand human motion by decomposing it into a sequence of elementary building blocks that belong to a known alphabet of dynamical systems. How can we construct an alphabet of movemes from human data? In this paper we address this issue by introducing the notion of well-posednes. Using examples from human drawing data, we show that the well-posedness notion can be applied in practice so to establish if sets of actions, viewed as signals in time, can define movemes.
A probabilistic approach to human motion detection and labeling

Human motion analysis is a very important task for computer vision with many potential applications. There are several problems in human motion analysis: detection, tracking, and activity interpretation. Detection is the most fundamental problem of the three, but remains untackled due to its inherent difficulty. This thesis develops a solution to the problem. It is based on a learned probabilistic model of the joint positions and velocities of the body parts, where detection and labeling are performed by hypothesis testing on the maximum a posterior estimate of the pose and motion of the body. To achieve efficiency in learning and testing, a graphical model is used to approximate the conditional independence of human motion. This model is also shown to provide a natural way to deal with clutter and occlusion. One key factor in the proposed method is the probabilistic model of human motion. In this thesis, an unsupervised learning algorithm that can obtain the probabilistic model automatically from unlabeled training data is presented. The training data include useful foreground features as well as features that arise from irrelevant background clutter. The correspondence between parts and detected features is also unknown in the training data. To learn the best model structure as well as model parameters, a variant of the EM algorithm is developed where the labeling of the data (part assignments) is treated as hidden variables. We explore two classes of graphical models: trees and decomposable triangulated graphs and find that the later are superior for our application. To better model human motion, we also consider the case when the model consists of mixtures of decomposable triangulated graphs. The efficiency and effectiveness of the algorithm have been demonstrated by applying it to generate models of human motion automatically from unlabeled image sequences, and testing the learned models on a variety of sequences. We find detection rates of over 95% on pairs of frames. This is very promising for building a real-life system, for example, a pedestrian detector.
Segmentation of human motion into dynamics based primitives with application to drawing tasks

Using tools from dynamical systems and systems identification, we develop a framework for the study of primitives for human motion, which we refer to as movemes. The objective is understanding human motion by decomposing it into a sequence of elementary building blocks that belong to a known alphabet of dynamical systems. We develop a segmentation and classification algorithm in order to reduce a complex activity into the sequence of movemes that have generated it. We test our ideas on data sampled from five human subjects who were drawing figures using a computer mouse. Our experiments show that we are able to distinguish between movemes and recognize them even when they take place in activities containing an unspecified number of movemes.
Visual Identification by Signature Tracking

We propose a new camera-based biometric: visual signature identification. We discuss the importance of the parameterization of the signatures in order to achieve good classification results, independently of variations in the position of the camera with respect to the writing surface. We show that affine arc-length parameterization performs better than conventional time and Euclidean arc-length ones. We find that the system verification performance is better than 4 percent error on skilled forgeries and 1 percent error on random forgeries, and that its recognition performance is better than 1 percent error rate, comparable to the best camera-based biometrics.
The Emergence of Visual Categories - A Computational Perspective

When we are born we do not know about sailing boats, frogs, cell-phones and wheelbarrows. By the time we reach school age we can easily recognize these categories of objects and many more using our visual system; by some estimates, we learn around 10 new categories per day with minimal supervision during the first few years of our lives. How can this happen? I will outline a computational approach to the problem of representing the visual properties of object categories, and of learning such models without supervision from cluttered images. Both static images of objects and dynamic displays such as the ones generated by human activity are handled by the theory. Its properties will be exemplified with experiments on a variety of categories.
A Digital Antennal Lobe for Pattern Equalization: Analysis and Design

Re-mapping patterns in order to equalize their distribution may greatly simplify both the structure and the training of classifiers. Here, the properties of one such map obtained by running a few steps of discrete-time dynamical system are explored. The system is called 'Digital Antennal Lobe' (DAL) because it is inspired by recent studies of the antennal lobe, a structure in the olfactory system of the grasshopper. The pattern-spreading properties of the DAL as well as its average behavior as a function of its (few) design parameters are analyzed by extending previous results of Van Vreeswijk and Sompolinsky. Furthermore, a technique for adapting the parameters of the initial design in order to obtain opportune noise-rejection behavior is suggested- Our results are demonstrated with a number of simulations.
Local Analysis for 3D Reconstruction of Specular Surfaces - Part II

We analyze the problem of recovering the shape of a mirror surface. We generalize the results of , where the special case of planar and spherical mirror surfaces was considered, extending that analysis to any smooth surface. A calibrated scene composed of lines passing through a point is assumed. The lines are reflected by the mirror surface onto the image plane of a calibrated camera, where the intersection and orientation of such reflections are measured. The relationship between the local geometry of the surface around the point of reflection and the measurements is analyzed. We give necessary and sufficient conditions, as well as a practical algorithm, for recovering first order local information (positions and normals) when three intersecting lines are visible. A small number of 'ghost solutions' may arise. Second order surface geometry may also be obtained up to one unknown parameter. Experimental results with real mirror surfaces are presented.
Second order local analysis for 3D reconstruction of specular surfaces

We analyze the problem of recovering the shape of a mirror surface. A calibrated scene composed of lines passing through a point is assumed. The lines are reflected by the mirror surface onto the image plane of a calibrated camera, where the intersection, orientation and curvature of such reflections are measured. The relationship between the local geometry of the surface around the point of reflection and the measurements is analyzed. We extend the analysis in Savarese and Perona.
Implementation of a shadow carving system for shape capture

We present a new technique for estimating the 3D shape of an object that combines previous ideas from shape from silhouettes and shape from shadows. We begin with a set-up for robustly extracting object silhouettes by casting a shadow of the object with a point light-source onto a translucent panel. A camera on the opposite side of the panel records an image which is readily processed to obtain the object boundary. We use a space carving technique to extract an initial estimate of the object shape. In a second phase, we record a series of images of the object lit by point light sources. We compare the areas of self shadowing in these images to those expected if our estimated shape from the space carving were correct. The shape of the object is refined by a shadow carving step that adjusts the current shape to resolve contradictions between the captured images and the current shape estimate. The result of the space carving and shadow carving is an estimate of shape that can be further refined by methods that work well in local regions, such as photometric stereo. We have implemented our approach in a simple table top system and present the results of scanning a small object with deep concavities.
Rapid natural scene categorization in the near absence of attention

What can we see when we do not pay attention? It is well known that we can be “blind” even to major aspects of natural scenes when we attend elsewhere. The only tasks that do not need attention appear to be carried out in the early stages of the visual system. Contrary to this common belief, we report that subjects can rapidly detect animals or vehicles in briefly presented novel natural scenes while simultaneously performing another attentionally demanding task. By comparison, they are unable to discriminate large T's from L's, or bisected two-color disks from their mirror images under the same conditions. We conclude that some visual tasks associated with “high-level” cortical areas may proceed in the near absence of attention.
Visual Input for Pen-Based Computers

The design and implementation of a camera-based, human-computer interface for acquisition of handwriting is presented. The camera focuses on a standard sheet of paper and images a common pen; the trajectory of the tip of the pen is tracked and the contact with the paper is detected. The recovered trajectory is shown to have sufficient spatio-temporal resolution and accuracy to enable handwritten character recognition. More than 100 subjects have used the system and have provided a large and heterogeneous set of examples showing that the system is both convenient and accurate.
Primitives for Human Motion: a Dynamical Approach

Using tools from dynamical systems theory and systems identification theory we develop the study of primitives for human motion which we refer to as movemes. We 
introduce basic definitions of dynamical independence 
of linear time-invariant dynamical systems (LTI) and 
segmentability of signals and we develop classification and segmentation algorithms for two dimensional motions. We test our ideas on data sampled from four human subjects who were engaged in a simple real-life activity including two movemes. Our experiments show that we are able to distinguish between the two movemes and recognize 
them even when they take place in an activity containing more than one moveme.
Human action recognition by sequence of movelet codewords

An algorithm for the recognition of human actions in image sequences is presented. The algorithm consists of 3 stages: background subtraction, body pose classification, and action recognition. A pose is represented in space-time, called 'movelet'. A movelet is a collection of the shape, motion and occlusion of image patches corresponding to the main parts of the body. The (infinite) set of all possible movelets is quantized into codewords obtained by the vector quantization. For every pair of frames each codeword is assigned a probability. Recognition is performed by simultaneously estimating the most likely sequence of codewords and the action that took place in a sequence. This is done using hidden Markov models. Experiments on the recognition of 3 periodic human actions, each under 3 different viewpoints, and 8 non-periodic human actions, are presented. training and testing are performed on different subjects with encouraging results. The influence of the number of codewords on the algorithm performance is studied.
Efficient methods for object recognition using the constellation model

We present efficient methods for object recognition, using the constellation model. This model represents objects as constellations of rigid features, with the variability between them being represented by a joint probability density function. Using simple, exhaustive methods, the computational requirements for this approach quickly become prohibitive. By using A* search methods, accompanied by some heuristics, large performance improvements are achievable in recognition, making real-time recognition on large images possible, using complex object models.
Learning probabilistic structure for human motion detection

Decomposable triangulated graphs have been shown to be efficient and effective for modeling the probabilistic spatio-temporal structure of brief stretches of human motion. In previous work such model structure was handcrafted by expert human observers and labeled data were needed for parameter learning. We present a method to build automatically the structure of the decomposable triangulated graph from unlabeled data. It is based on maximum-likelihood. Taking the labeling of the data as hidden variables, a variant of the EM algorithm can be applied. A greedy algorithm is developed to search for the optimal structure of the decomposable model based on the (conditional) differential entropy of variables. Our algorithm is demonstrated by learning models of human motion completely automatically from unlabeled real image sequences with clutter and occlusion. Experiments on both motion captured data and grayscale image sequences show that the resulting models perform better than the hand-constructed models.
Local analysis for 3D reconstruction of specular surfaces

We explore the geometry linking the shape of a curved mirror surface to the distortions it produces on a scene it reflects. Our analysis is local and differential. We assume a simple calibrated scene composed of lines passing through a point. We demonstrate that local information about the geometry of the surface may be recovered up to the second order from either the orientation and curvature of the images of two intersecting lines, or from the orientation of the images of three or more intersecting lines. An explicit solution for calculating shape and position of spherical mirror surfaces is given.
Automatic Extraction of Facial Action Codes

Two systems for capturing and analyzing automatically facial expressions are evaluated and compared. The systems, developed by interdisciplinary research teams at CMU/Pitt and UCSD/Salk, are the first attempt at automatically analyzing spontaneous facial expressions with unconstrained head orientation. The training and test sequences where acquired by Ekman and Frank while the subject was engaged in deception in a realistic scenario. Facial analysis is based on the extraction of action units as defined by Ekman in his classic Facial Action Coding System. While full automation has not yet been achieved the systems correctly classified around 90% of instances of 3 facial movements, or Action Units. The objective of a fully automated system for measuring human expression from facial and body motion and, possibly, speech cues, is a realistic goal for the next 10 years. Applications in intelligence and security, human-machine interfaces, and psychology will be enabled by this technology. The next step to be taken in order to achieve this goal is the acquisition of a 10x larger and better (more sensors and higher resolution) dataset. This should be followed by the creation of a community of interdisciplinary teams working towards this goal.
Unsupervised Learning of Human Motion Models

This paper presents an unsupervised learning algorithm that can derive the probabilistic dependence structure of parts of an object (a moving human body in our examples) automatically from unlabeled data. The distinguished part of this work is that it is based on unlabeled data, i.e., the training features include both useful foreground parts and background clutter and the correspondence between the parts and detected features are unknown. We use decomposable triangulated graphs to depict the probabilistic independence of parts, but the unsupervised technique is not limited to this type of graph. In the new approach, labeling of the data (part assignments) is taken as hidden variables and the EM algorithm is applied. A greedy algorithm is developed to select parts and to search for the optimal structure based on the differential entropy of these variables. The success of our algorithm is demonstrated by applying it to generate models of human motion automatically from unlabeled real image sequences.
Monocular perception of biological motion in

Computer perception of biological motion is key to developing convenient and powerful humancomputer interfaces. Algorithms have been developed for tracking the body; however, initialization is done by hand. We propose a method for detecting a moving human body and for labeling its parts automatically in scenes that include extraneous motions and occlusion. We assume a Johansson display, i.e. that a number of moving features, some representing the unoccluded body joints and some belonging to the background are supplied in the scene. Our method is based on maximizing the joint probability density function (PDF) of the position and velocity of the body parts. The PDF is estimated from training data. Dynamic programming is used for calculating eÆciently the best global labeling on an approximation of the PDF. Detection is performed by hypothesis testing on the best labeling found. The computational cost is on the order of N where N is the number of features detected. We explore the performance of our method with experiments carried on a variety of periodic and non-periodic body motions viewed monocularly for a total of approximately 30,000 frames. The algorithm is demonstrated to be accurate and eÆcient.
Shadow Carving

The shape of an object may be estimated by observing the shadows on its surface. We present a method that is robust with respect to a conservative classification of shadow regions. Assuming that a conservative estimate of the object shape is available, we analyze images of the object illuminated with known point light sources taken from known camera locations. We adjust our surface estimate using the shadow regions to produce a refinement that is still a conservative estimate. A proof of correctness is provided. No assumptions about the object topology are made, although any tangent plane discontinuities over the object's surface are supposed to be detectable. An implementation and some experimental results are presented.
Shadow carving

The shape of an object may be estimated by observing the shadows on its surface. We present a method that is robust with respect to a conservative classification of shadow regions. Assuming that a conservative estimate of the object shape is available, we analyze images of the object illuminated with known point light sources taken from known camera locations. We adjust our surface estimate using the shadow regions to produce a refinement that is still a conservative estimate. A proof of correctness is provided. No assumptions about the object topology are made, although any tangent plane discontinuities over the object's surface are supposed to be detectable. An implementation and some experimental results are presented.