nils-herrmann/clean_scirepeval huggingface dataset:
Monocular Perception of Biological Motion in Johansson Displays

Computer perception of biological motion is key to developing convenient and powerful human?computer interfaces. Algorithms have been developed for tracking the body; however, initialization is done by hand. We propose a method for detecting a moving human body and for labeling its parts automatically in scenes that include extraneous motions and occlusion. We assume a Johansson display, i.e., that a number of moving features, some representing the unoccluded body joints and some belonging to the background, are supplied in the scene. Our method is based on maximizing the joint probability density function (PDF) of the position and velocity of the body parts. The PDF is estimated from training data. Dynamic programming is used for calculating efficiently the best global labeling on an approximation of the PDF. Detection is performed by hypothesis testing on the best labeling found. The computational cost is on the order of N4 where N is the number of features detected. We explore the performance of our method with experiments carried on a variety of periodic and nonperiodic body motions viewed monocularly for a total of approximately 30,000 frames. The algorithm is demonstrated to be accurate and efficient.
A New Geometrical Interpretation of Trilinear Constraints

We give a new geometrical interpretation of the 
well-known algebraic trilinear constraints used in motion 
analysis from three views observation. We show 
that those algebraic equations correspond to depth errors 
appropriately weighted by a function of the relative 
reliability of the corresponding measurements. 
Therefore directly minimizing the algebraic trilinear 
equations, in the least squares sense, is a strategy that 
works well for estimating motion. In addition, we 
propose a new scheme for recovering the scale factor 
for motion estimation that is very insensitive to input 
noise. All our theoretical statements are supported by 
experimental results.
Unsupervised learning of models for object recognition

A method is presented to learn object class models from unlabeled and unsegmented cluttered scenes for the purpose of visual object recognition. The variability across a class of objects is modeled in a principled way, treating objects as flexible constellations of rigid parts (features). Variability is represented by a joint probability density function (pdf) on the shape of the constellation and the output of part detectors. Corresponding “constellation models” can be learned in a completely unsupervised fashion. In a first stage, the learning method automatically identifies distinctive parts in the training set by applying a clustering algorithm to patterns selected by an interest operator. It then learns the statistical shape model using expectation maximization. Mixtures of constellation models can be defined and applied to “discover” object categories in an unsupervised manner. The method achieves very good classification results on human faces, cars, leaves, handwritten letters, and cartoon characters.
Bit-width optimization for configurable DSP's by multi-interval analysis

A new algorithm for bounding the bit-widths of the data registers of an acyclic data-flow graph is presented. The method, based on the propagation of two's complement fixed-point numerical ranges, can be applied to both linear and nonlinear time invariant flow graphs and is well suited to be implemented in Field Programmable Gate Arrays (FPGA's). Numerical values are represented by unions of intervals, allowing automatic monitoring of the growth of the number of bits needed to represent the integer and the fractional part of intermediate variables. Central to this method is the definition of a new interval arithmetic on fixed point multi-intervals. An application of the proposed algorithm to the problem of detecting two dimensional visual features in video images is presented.
Unsupervised Learning of Models for Recognition

We present a method to learn object class models from unlabeled and unsegmented cluttered scenes for the purpose of visual object recognition. We focus on a particular type of model where objects are represented as flexible constellations of rigid parts (features). The variability within a class is represented by a joint probability density function (pdf) on the shape of the constellation and the output of part detectors. In a first stage, the method automatically identifies distinctive parts in the training set by applying a clustering algorithm to patterns selected by an interest operator. It then learns the statistical shape model using expectation maximization. The method achieves very good classification results on human faces and rear views of cars.
A system for real-time 2-D feature detection based on field programmable gate arrays

We have designed and implemented a system for real-time detection of 2-D image features on a reconfigurable computer based on Field Programmable Gate Arrays (FPGA's). We envision this device as the front-end of a system able to track image features in real-time vision applications like autonomous vehicle navigation and "structure from motion". The algorithm employed to select good features is inspired by the method of Tomasi and Kanade. Compared to the original method, the algorithm that we have devised does not require any floating point or transcendental operations, and maps efficiently into a highly pipelined architecture, well suited to be implemented in FPGA technology. We have implemented the algorithm on a low-cost reconfigurable computer equipped with video decoder and encoder interfaces. Reliable operation has been observed on an image stream generated by a standard NTSC commercial video camera at 30~Hz on different scenes and under different light conditions. This result compares favorably to the implementation of the same algorithm on a system based on the TI C80 DSP, able to process at most 5 frames/second.
Towards automatic discovery of object categories

We propose a method to learn heterogeneous models of object classes for visual recognition. The training images contain a preponderance of clutter and learning is unsupervised. Our models represent objects as probabilistic constellations of rigid parts (features). The variability within a class is represented by a join probability density function on the shape of the constellation and the appearance of the parts. Our method automatically identifies distinctive features in the training set. The set of model parameters is then learned using expectation maximization. When trained on different, unlabeled and unsegmented views of a class of objects, each component of the mixture model can adapt to represent a subset of the views. Similarly, different component models can also "specialize" on sub-classes of an object class. Experiments on images of human heads, leaves from different species of trees, and motor-cars demonstrate that the method works well over a wide variety of objects.
Viewpoint-invariant learning and detection of human heads

We present a method to learn models of human heads for the purpose of detection from different viewing angles. We focus on a model where objects are represented as constellations of rigid features (parts). Variability is represented by a joint probability density function (PDF) on the shape of the constellation. In the first stage, the method automatically identifies distinctive features in the training set using an interest operator followed by vector quantization. The set of model parameters, including the shape PDF, is then learned using expectation maximization. Experiments show good generalization performance to novel viewpoints and unseen faces. Performance is above 90% correct with less than 1 s computation time per image.
Special Issue on the Second International Conference on Scale Space Theory in Computer Vision: Guest Editors' Comments

The Second International Conference on Scale Space Theory in Computer Vision was held in Corfu, Greece, September 26–27, 1999. The conference followed the International Conference on Computer Vision , September 20–25, 1999. The first Scale Space Conference, held in Utrecht, the Netherlands, in July 1997, was born from a series of workshops sponsored by the NSF—SPRIT grant for collaboration on Geometry-Driven Diffusion in Computer Vision. Following the success of those meetings, it was decided to hold a biennial conference on the topic. Scale Space ’99 was a success, with 66 papers submitted (up from 44 at Scale Space ’97) and about 100 participants. This special issue presents a few of those papers. The selection was primarily done by the anonymous reviewers, who were explicitly asked if the paper under consideration was of journal quality or not. Unfortunately, due to space limitations, many excellent papers could not be published in this special issue. In addition, reviewers many times considered a paper excellent, but not yet ready for journal publication (a judgment most of the time in agreement with the authors). Those papers could not be included in this special issue either. We thank the reviewers who helped us produce this special issue. In addition, we thank Professor John Tsotsos, General Chair of ICCV ’99, for all his help and support with Scale Space ’99. To conclude, we thank Ole Fogh Olsen, Joachim Weickert, and Peter Johansen from the Scale Space ’99 Program Board and the whole Scale Space ’99 Program Committee for helping with the reviewing process. We hope you will enjoy this special issue, and see you at Scale Space ’01!
An intelligent vision-only operator interface for dexterous robots

The development of a vision-only intelligent interface for dexterous robots is described, capable of tracking operator motions: evaluating trajectory feasibility, and providing visual feedback to the operator. This interface analyses the operator's arm motion for safety, and then converts it into trajectory commands for a mechanical arm. Potentially dangerous situations, such as singularity and self collision, are displayed to the operator as graphical icons superimposed over the robot video images. The paper summarizes the main features of the current implementation, presents the approach developed for singularity identification and describes the performance of the demonstration prototype.
Monocuolar Perception of Biological Motion - Clutter and Partial Occlusion

The problem of detecting and labeling a moving human body viewed monocularly in a cluttered scene is considered. The task is to decide whether or not one or more people are in the scene (detection), to count them, and to label their visible body parts (labeling). 
 
It is assumed that a motion-tracking front end is supplied: a number of moving features, some belonging to the body and some to the background are tracked for two frames and their position and velocity is supplied (Johansson display). It is not guaranteed that all the body parts are visible, nor that the only motion present is the one of the body. 
 
The algorithm is based on our previous work ; we learn a probabilistic model of the position and motion of body features, and calculate maximum-likelihood labels efficiently using dynamic programming on a triangulated approximation of the probabilistic model. We extend those results by allowing an arbitrary number of body parts to be undetected (e.g. because of occlusion) and by allowing an arbitrary number of noise features to be present. We train and test on walking and dancing sequences for a total of approximately 104 frames. The algorithm is demonstrated to be accurate and efficient.
Towards detection of human motion

Detecting humans in images is a useful application of computer vision. Loose and textured clothing, occlusion and scene clutter make it a difficult problem because bottom-up segmentation and grouping do not always work. We address the problem of detecting humans from their motion pattern in monocular image sequences; extraneous motions and occlusion may be present. We assume that we may not rely on segmentation or grouping and that the vision front-end is limited to observing the motion of key points and textured patches in between pairs of frames. We do not assume that we are able to track features for more than two frames. Our method is based on learning an approximate probabilistic model of the joint position and velocity of different body features. Detection is performed by hypothesis testing on the maximum a posteriori estimate of the pose and motion of the body. Our experiments on a dozen of walking sequences indicate that our algorithm is accurate and efficient.
A computational model for motion detection and direction discrimination in humans

Seeing biological motion is very important for both humans and computers. Psychophysics experiments show that the ability of our visual system for biological motion detection and direction discrimination is different from that for simple translation. The existing quantitative models of motion perception cannot explain these findings. We propose a computational model, which uses learning and statistical inference based on the joint probability density function (PDF) of the position and motion of the body, on stimuli similar to (Neri et al., 1998). Our results are consistent with the psychophysics indicating that our model is consistent with human motion perception, accounting for both biological motion and pure translation.
Monocular perception of biological motion-detection and labeling

Computer perception of biological motion is key to developing convenient and powerful human-computer interfaces. Successful body tracking algorithms have been developed; however initialization is done by hand. We propose a method for detecting a moving human body and for labeling its parts automatically. It is based on maximizing the joint probability density function (PDF) of the position and velocity of the body parts. The PDF is estimated from training data. Dynamic programming is used for calculating efficiently the best global labeling on an approximation of the PDF. The computational cost is on the order of N/sup 4/ where N is the number of features detected. We explore the performance of our method with experiments carried on a variety of periodic and non-periodic body motions viewed monocularly for a total of approximately 30,000 frames. Point-markers were strapped to the joints of the subject for facilitating image analysis. We find an average of 2.3% labeling error; the experiments also suggest a high degree of viewpoint-invariance.
Continuous dynamic time warping for translation-invariant curve alignment with applications to signature verification

The problem of establishing correspondence and measuring the similarity of a pair of planar curves arises in many applications in computer vision and pattern recognition. This paper presents a new method for comparing planar curves and for performing matching at sub-sampling resolution. The analysis of the algorithm as well as its structural properties are described. The performance of the new technique applied to the problem of signature verification is shown and compared with the performance of the well-known Dynamic Time Warping algorithm.
Orientation Diiusions

100 Orientation di usions P. Perona Abstract|Di usions are useful for image processing and computer vision because they provide a convenient way of smoothing noisy data, of analyzing images at multiple scales, and of enhancing discontinuities. A number of di usions of image brightness have been de ned and studied so far; they may be applied to scalar and vector-valued quantities that are naturally associated with intervals of either the real line, or other at manifolds. Some quantities of interest in computer vision, and other areas of engineering that deal with images, are de ned on curved manifolds; typical examples are orientation and hue that are de ned on the circle. Generalizing brightness di usions to orientation is not straightforward, especially in the case where a discrete implementation is sought. An example of what may go wrong is presented. A method is proposed to de ne di usions of orientationlike quantities. First a de nition in the continuum is discussed, then a discrete orientation di usion is proposed. The behavior of such di usions is explored both analytically and experimentally. It is shown how such orientation di usions contain a nonlinearity that is reminiscent of edgeprocess and anisotropic di usion. A number of open questions are proposed at the end. Keywords| Orientation analysis, texture analysis, di usions, scale-space
What do planar shadows tell about scene geometry?

A method for reconstructing 3D scene geometry from a set of projected shadows is presented. It is composed of two stages. First, the scene geometry is retrieved up to three scalar unknowns using only the information contained in the observed shadow edges on the image plane. Then, the three remaining unknowns are computed making use of the known depths at three points. This technique improves upon previous results in that it does not require the presence of a reference plane in the background. A mathematical analysis is presented using dual-space geometry, a formalism that provides adequate tools to carry out all the derivations in a compact and intuitive manner. A linear algorithm based on singular value decomposition (SVD) is presented leading to a closed form solution for reconstruction.
Visual methods for three-dimensional modeling

Most animals use vision as a primary sensor to interact with their environment. Navigation or manipulation of objects are among the tasks that can be better achieved while understanding the three-dimensional structure of the scene. 
In this thesis, we present a variety of computational techniques for estimating 3D shape from 2D images, based on both passive and active technologies. 
The first proposed method is purely passive. In this technique, a single camera is moved in an unconstrained manner around the scene to model as it acquires a sequence of images. The reconstruction process consists then of retrieving the trajectory of the camera, as well as the 3D structure of the scene using only the information contained in the images. 
The second method is based on active lighting technology. In the philosophy of standard 3D scanning methods, a projector is used to project light patterns in the scene. The shape of the scene is then inferred from the way the patterns deform on the objects. The main novelty of our scheme compared to traditional methods is in the nature of the patterns, and the type of image processing associated to them. Instead of using standard binary patterns made out of black and white stripes, our scheme uses a sequence of grayscale patterns with a sinusoidal profile in brightness intensity. This choice allows us to establish correspondence (between camera image, and projector image) in a dense fashion, leading to depth computation at (almost) every pixel in the image. 
The last reconstruction method that we propose in this thesis is an alternative 3D scanning scheme that does not require any other device besides a camera. The main idea is to substitute the projector by a standard light source (such as a desk lamp), and use a pencil (or any other object with a straight edge) to cast planar shadows in the scene. The 3D geometry of the scene is then inferred from the way the shadow naturally deforms on the objects in the scene. Since this technology is largely inspired from structured lighting techniques, we call it ‘weakly structured lighting.’
A novel system architecture for real-time low-level vision

A novel system architecture that exploits the spatial locality in memory access that is found in most low-level vision algorithms is presented. A real-time feature selection system is used to exemplify the underlying ideas, and an implementation based on commercially available field programmable gate arrays (FPGAs) and synchronous SRAM memory devices is proposed. The peak memory access rate of a system based on this architecture is estimated at 2.88 G-Bytes/s, which represents a four to five times improvement with respect to existing reconfigurable computers.
Analyse d'ecriture par camera

L'invention concerne un systeme de traitement de l'ecriture qui consiste a utiliser une camera (100) ordinaire comme dispositif d'entree d'image. On utilise les images d'une seule camera (100) pour determiner une fonction de probabilite fournissant une indication sur la probabilite selon laquelle le stylo (104) touche le papier (102). La fonction fait appel a des indices comme l'encre appliquee sur la page et/ou les ombres presentes. Selon une autre variante, on tient compte d'indications relatives au lever et au baisser de stylo pour adapter dynamiquement les indications en termes de deformation temporelle aux fins d'adaptation a un gabarit.
A Probabilistic Approach to Object Recognition Using Local Photometry and Global Geometry

Many object classes, including human faces, can be modeled as a set of characteristic parts arranged in a variable spatial configuration. We introduce a simplified model of a deformable object class and derive the optimal detector for this model. However, the optimal detector is not realizable except under special circumstances (independent part positions). A cousin of the optimal detector is developed which uses “soft” part detectors with a probabilistic description of the spatial arrangement of the parts. Spatial arrangements are modeled probabilistically using shape statistics to achieve invariance to translation, rotation, and scaling. Improved recognition performance over methods based on “hard” part detectors is demonstrated for the problem of face detection in cluttered scenes.
Orientation diffusions

Diffusions are useful for image processing and computer vision because they provide a convenient way of smoothing noisy data, analyzing images at multiple scales, and enhancing discontinuities. A number of diffusions of image brightness have been defined and studied so far; they may be applied to scalar and vector-valued quantities that are naturally associated with intervals of either the real line, or other flat manifolds. Some quantities of interest in computer vision, and other areas of engineering that deal with images, are defined on curved manifolds;typical examples are orientation and hue that are defined on the circle. Generalizing brightness diffusions to orientation is not straightforward, especially in the case where a discrete implementation is sought. An example of what may go wrong is presented.A method is proposed to define diffusions of orientation-like quantities. First a definition in the continuum is discussed, then a discrete orientation diffusion is proposed. The behavior of such diffusions is explored both analytically and experimentally. It is shown how such orientation diffusions contain a nonlinearity that is reminiscent of edge-process and anisotropic diffusion. A number of open questions are proposed at the end.
Scene segmentation from 3D motion

We propose an EM approach combined with the modified separation matrix scheme to perform 3D motion segmentation of the image sequence which contains multiple moving objects. We observe that, given the detected features and their 2D optical flow, in most cases the objects or their flow are separated very well from each other in space. The separation matrix method modified by using normalized cuts achieves expected grouping results for these cases. However, when the objects are overlapped spatially but undergoing independent motions, such as Ullman's co-axial transparent cylinder demonstration, there will be no proper affinity to perform segmentation by that way. Pure underlying 3D motion becomes the only cite to segment the scene. We exploit the EM algorithm to deal with such difficult cases. The scheme is tested on vast number of synthetic image sequences. Results with real image sequence are also given.