nils-herrmann/clean_scirepeval huggingface dataset:
Real-time 2-D feature detection on a reconfigurable computer

We have designed and implemented a system for real-time detection of 2-D features on a reconfigurable computer based on Field Programmable Gate Arrays (FPGA's). We envision this device as the front-end of a system able to track image features in real-time control applications like autonomous vehicle navigation. The algorithm employed to select good features is inspired by Tomasi and Kanade's method. Compared to the original method, the algorithm that we have devised does not require any floating point or transcendental operations, and can be implemented either in hardware or in software. Moreover, it maps efficiently into a highly pipelined architecture, well suited to implementation in FPGA technology. We have implemented the algorithm on a low-cost reconfigurable computer and have observed reliable operation on an image stream generated by a standard NTSC video camera at 30 Hz.
Using hierarchical shape models to spot keywords in cursive handwriting data

Different instances of a handwritten word consist of the same basic features (humps, cusps, crossings, etc.) arranged in a deformable spatial pattern. Thus, keywords in cursive text can be detected by looking for the appropriate features in the "correct" spatial configuration. A keyword can be modeled hierarchically as a set of word fragments, each of which consists of lower-level features. To allow flexibility, the spatial configuration of keypoints within a fragment is modeled using a Dryden-Mardia (DM) probability density over the shape of the configuration. In a writer-dependent test on a transcription of the Declaration of Independence (/spl sim/1300 words, /spl sim/7500 characters), the method detected all eleven instances of the keyword "government" with only four false positives.
Multirate separable implementation of steerable filter banks

This paper describes efficient schemes for the computation of a large number of differently scaled/oriented filtered versions of an image. Previous work has generalized the well-known steerable/scalable ("deformable") filter bank structure by imposing X-Y separability on the basis filters. These systems, designed by an iterative projections technique, were shown to achieve substantial reduction of the computational cost. To reduce the memory requirement, we adopt a multirate implementation. The resulting structure, however, is not shift-invariant-it gives rise to "aliasing". We introduce a design criterion for multirate deformable structures that jointly controls the approximation error and the shift-variance.
Reducing "Structure From Motion": A General Framework for Dynamic Vision Part 2: Implementation and Experimental Assessment

For pt.1 see ibid., p.933-42 (1998). A number of methods have been proposed in the literature for estimating scene-structure and ego-motion from a sequence of images using dynamical models. Despite the fact that all methods may be derived from a "natural" dynamical model within a unified framework, from an engineering perspective there are a number of trade-offs that lead to different strategies depending upon the applications and the goals one is targeting. We want to characterize and compare the properties of each model such that the engineer may choose the one best suited to the specific application. We analyze the properties of filters derived from each dynamical model under a variety of experimental conditions, assess the accuracy of the estimates, their robustness to measurement noise, sensitivity to initial conditions and visual angle, effects of the bas-relief ambiguity and occlusions, dependence upon the number of image measurements and their sampling rate.
Correction to: "Reducing 'Structure From Motion': A General Framework for Dynamic Vision Part 2: Implementation and Experimental Assessment"

A number of methods have been proposed in the literature for estimating scene-structure and ego-motion from a sequence of images using dynamical models. Despite the fact that all methods may be derived from a “natural” dynamical model within a unified framework, from an engineering perspective there are a number of trade-offs that lead to different strategies depending upon the applications and the goals one is targeting. We want to characterize and compare the properties of each model such that the engineer may choose the one best suited to the specific application. We analyze the properties of filters derived from each dynamical model under a variety of experimental conditions, assess the accuracy of the estimates, their robustness to measurement noise, sensitivity to initial conditions and visual angle, effects of the bas-relief ambiguity and occlusions, dependence upon the number of image measurements and their sampling rate.
Where is the sun?

When we interpret a shaded picture as a three-dimensional (3D) scene, our visual system often needs to guess the position of the light source in order to resolve a convex-concave ambiguity. For more than a century, psychologists have known that the visual system assumes that light comes from above and have argued that this assumption is ecologically justified because our everyday light source (the sun) is overhead. Our experiments reveal that people's preferred lighting direction is not directly overhead, but rather shifted to the left, and this preference is reflected in art spanning two millennia. Furthermore, we find a strong correlation between people's handedness and their preferred lighting. We suggest that what counts is not so much where the sun is, but where you like the sun to be.
3D photography on your desk

A simple and inexpensive approach for extracting the three-dimensional shape of objects is presented. It is based on 'weak structured lighting'; it differs from other conventional structured lighting approaches in that it requires very little hardware besides the camera: a desk-lamp, a pencil and a checker-board. The camera faces the object, which is illuminated by the desk-lamp. The user moves a pencil in front of the light source casting a moving shadow on the object. The 3D shape of the object is extracted from the spatial and temporal location of the observed shadow. Experimental results are presented on three different scenes demonstrating that the error in reconstructing the surface is less than 1%.
Probabilistic affine invariants for recognition

Under a weak perspective camera model, the image plane coordinates in different views of a planar object are related by an affine transformation. Because of this property, researchers have attempted to use affine invariants for recognition. However, there are two problems with this approach: (1) objects or object classes with inherent variability cannot be adequately treated using invariants; and (2) in practice the calculated affine invariants can be quite sensitive to errors in the image plane measurements. In this paper we use probability distributions to address both of these difficulties. Under the assumption that the feature positions of a planar object can be modeled using a jointly Gaussian density, we have derived the joint density over the corresponding set of affine coordinates. Even when the assumptions of a planar object and a weak perspective camera model do not strictly hold, the results are useful because deviations from the ideal can be treated as deformability in the underlying object model.
Efficient deformable filter banks

This article describes efficient schemes for the computation of a large number of differently scaled/oriented filtered versions of an image. We generalize the well-known steerable/scalable ("deformable") filter bank structure by imposing X-Y separability on the basis filters. The resulting systems, designed by an iterative projections technique, achieve substantial reduction of the computational cost. To reduce the memory requirement, we adopt a multirate implementation. Due to the inner sampling rate alteration, the resulting structure is not shift invariant. We introduce a design criterion for multirate deformable structures that jointly controls the approximation error and the shift variance.
Reducing "Structure From Motion": A General Framework for Dynamic Vision Part 1: Modeling

The literature on recursive estimation of structure and motion from monocular image sequences comprises a large number of apparently unrelated models and estimation techniques. We propose a framework that allows us to derive and compare all models by following the idea of dynamical system reduction. The "natural" dynamic model, derived from the rigidity constraint and the projection model, is first reduced by explicitly decoupling structure (depth) from motion. Then, implicit decoupling techniques are explored, which consist of imposing that some function of the unknown parameters is held constant. By appropriately choosing such a function, not only can we account for models seen so far in the literature, but we can also derive novel ones.
Real Time Motion Detection System and Scene Segmentation

We address two issues in this report. One is the real time implementation of feature tracking and motion estimation. As a fundamental problem in vision eld, feature tracking needs to be implemented in real time so that researchers can do further analysis on motion such as building real time visual navigation system etc.. Our implementation of the algorithm on the C4x board with parallel processors and its performance are described. Another part of the paper represents a 3D motion segmentation scheme. We propose an EM approach combined with the modiied separation matrix scheme to perform 3D motion segmentation of the image sequence that contains multiple moving objects. We observe that, given the detected features and their 2D optical ow, in most cases the objects or their ow are separated very well from each other in space. The separation matrix method modiied by using normalized cuts achieves expected grouping results for these cases. However, when the objects are overlapped spatially but undergoing independent motions, such as Ullman's co-axial transparent cylinder demonstration, there will be no proper aanity to perform segmentation by that way. Pure underlying 3D motion becomes the only cue to segment the scene. We exploit the EM algorithm to deal with such diicult cases. The scheme is tested on vast number of synthetic image sequences. Results with real image sequence are also given.
Probabilistic Aane Invariants for Recognition

Under a weak perspective camera model, the image plane coordinates in diierent views of a planar object are related by an aane transformation. Because of this property, researchers have attempted to use aane invariants for recognition. However, there are two problems with this approach: (1) objects or object classes with inherent variability cannot be adequately treated using invariants; and (2) in practice the calculated aane invariants can be quite sensitive to errors in the image plane measurements. In this paper we use probability distributions to address both of these diiculties. Under the assumption that the feature positions of a planar object can be modeled using a jointly Gaussian density, we have derived the joint density over the corresponding set of aane coordinates. Even when the assumptions of a planar object and a weak perspective camera model do not strictly hold, the results are useful because deviations from the ideal can be treated as deformability in the underlying object model.
Reach out and touch space (motion learning)

We propose a method for learning models of human motion from a coarsely sampled set of examples. The models we synthesize may be used to generate plausible motions from a high level description consisting of start and stop positions, style, mood, age, etc. In the field of computer vision, such models can be useful for human body motion tracking/estimation and gesture recognition. The models can also be used to generate arbitrary realistic human motion, and may be of help in trying to understand the mechanisms behind the perception of biological motion by the human visual system. Experimental results of the learning technique applied to reaching and drawing motions are presented.
3D photography using shadows

A simple and inexpensive approach for extracting the three-dimensional shape of objects is presented. It is based on `weak structured lighting;' and requires little hardware besides the camera: a light source (a desk-lamp or the sun), a stick and a checkerboard. The object, illuminated by the light source, is placed on a stage composed of a ground plane and a back plane; the camera faces the object. The user moves the stick in front of the light source, casting a moving shadow on the scene. The 3D shape of the object is extracted from the spatial and temporal location of the observed shadow. Experimental results are presented on three different scenes (indoor with a desk lamp and outdoor with the sun) demonstrating that the error in reconstructing the surface is less than 0.5% of the size of the object.
3D photography using shadows

A simple and inexpensive approach for extracting the three-dimensional shape of objects is presented. It is based on 'weak structured lighting;' it differs from previous structured lighting approaches in that it requires little hardware besides the camera: a light source (a desk-lamp or the sun), a stick and a checkerboard. The object, illuminated by the light source, is placed on a stage composed of a ground plane and a back plane; the camera faces the object. The user moves the stick in front of the light source, casting a moving shadow on the scene. The 3D shape of the object is extracted from the spatial and temporal location of the observed shadow. Experimental results are presented on three different scenes (indoor with a desk lamp and outdoor with the sun) demonstrating that the error in reconstructing the surface is less than 0.5% of the size of the object.
Motion from Fixation

We study the problem of estimating rigid motion from a sequence of monocular perspective images obtained by navigating around an object while fixating a particular feature-point. The motivation comes from the mechanics of the human eye, which either pursues smoothly some fixation point in the scene, or `saccades? between different fixation points. In particular, we are interested in understanding whether fixation helps the process of estimating motion in the sense that it makes it more robust, better conditioned or simpler to solve.We cast the problem in the framework of `Epipolar geometry?, and propose a filter based upon an implicit dynamical model for recursively estimating motion under the fixation constraint. This allows us to compare directly the quality of the estimates of motion obtained by imposing the fixation constraint against the estimates obtained assuming a general rigid motion simply by changing the geometry of the parameter space, while maintaining the same structure of the recursive estimator. We also present a closed-form static solution from two views, a recursive estimator of the relative attitude between the viewer and the scene and assess how the estimates degrade in the presence of disturbances in the tracking procedure. All recursive filters are suitable for real-time implementation.
Implementation of Deformable Filter Banks

This paper describes e cient schemes for the computation of a large number of di erentely scaled/oriented ltered versions of an image. We generalize the well-known steerable/scalable ("deformable") lter bank structure by imposing X-Y separability on the basis lters. This systems, designed by an iterative projections technique, achieve substantial reduction of the computational cost. To reduce the memory requirement, we adopt a multirate implementation. The resulting structure, however, is not shift-invariant it gives raise to "aliasing". We introduce a design criterion for multirate deformable structures that jointly controls the approximation error and the shift-variance.
Feature Detection and Tracking

1 Feature points on an image The technique for generic keypoint detection may be extended for performing traking of features in images sequences. The algorithm is based on the SSD test  which allows to deene precisely the concept of \reliable point feature". The technique is best suited for small motions, but it can be extended to larger displacements by being applied in a coarse-tone manner. Features are often deened as characteristic points on an image which are easily recognizable and representative of the structure of the scene being viewed. In order to develop a technique to estract features automatically we need a more operative deenition. First of all it is not easy to identify single points (pixels), so we associate to each point of the image (x; y) a small neighborhood (window) W(x; y). We say that the window W is a reliable window if the integral of the squared gradient of the image is above some threshold : if I(x; y; t) is the brightness of the image at the point(x; y) at time t, and rI(x 0 ; y 0 ; t 0) is the spatial gradient calculated at a point, then the window being reliable can be expressed as min Z W(x 0 ;y 0) rIrI T dxdy ! > ; (1) where min denotes the smallest singular value. In practical implementations the partial derivatives will be approximated by rst diierence operators and the integral by a nite sum. The expression in parenthesis is referred as Sum of Squared Diierence (SSD). Now we say that the point (x 0 ; y 0) is a feature point if W(x 0 ; y 0) is a reliable window. After this deenition it is easy to set an algorithm to extract features: set a threshold for all points in the image compute the SSD IF the minimum singular value of the SSD is bigger than the threshold THEN mark the point as a feature point.
Orientation diffusions

Diffusions provide a convenient way of smoothing noisy brightness images, of analyzing images at multiple scales, and of enhancing discontinuities. Some quantities of interest in computer vision are defined on curved manifolds; typical examples are orientation and hue that are defined on the circle. Generalizing diffusions to orientation is not straightforward, especially in the case where a discrete implementation is sought. An example of what may go wrong is presented. A method is proposed to define diffusions of orientation-like quantities. First a definition in the continuum is discussed, then a discrete orientation diffusion is proposed. The behavior of such diffusions is explored both analytically and experimentally. It is shown how such orientation diffusions contain a nonlinearity that is reminiscent of edge-process and anisotropic diffusion. A number of open questions are proposed at the end.
Shading and Stereo in Early Perception of Shape and Reflectance

Recent experiments involving shaded 2-D stimuli have shown that early-vision mechanisms are capable of interpreting 3-D shape from shading. In particular, target discrimination tasks suggest that a target ‘pops out’ when background distractors, but not the target, can be interpreted as convex and lit from above or top-left. Since the problem of extracting 3-D shape from shading is intrinsically ill-defined, early vision may need to make these twin assumptions of convexity and top-left lighting in order to constrain the problem. Would these assumptions be recognized as unnecessary and consequently discarded when 3-D shape could be unambiguously defined by some other cue, like stereo disparity? A 2AFC stimulus onset asynchrony paradigm with masking was used in target discrimination experiments. The performance of five naive subjects on tasks where only shading cues were present was compared with that on tasks involving shading as well as stereo cues that define shape unambiguously. The results show that although stereo disparity information is incorporated by early-vision 3-D mechanisms, it is not used to overturn the default assumptions of lighting and shape. Stereo information is interpreted within the framework of top-left lighting, and a consistent preference for convexity is seen over concavity.
Reducing "Structure From Motion": a General Framework for Dynamic Vision - Part 2: Experimental Evaluation

A number of methods have been proposed in the literature for estimating scene-structure and ego-motion from a sequence of images using dynamical models. Although all methods may be derived from a "natural" dynamical model within a unified framework, from an engineering perspective there are a number of trade-offs that lead to different strategies depending upon the specific applications and the goals one is targeting. 
 
Which one is the winning strategy? In this paper we analyze the properties of the dynamical models that originate from each strategy under a variety of experimental conditions. For each model we assess the accuracy of the estimates, their robustness to measurement noise, sensitivity to initial conditions and visual angle, effects of the bas-relief ambiguity and occlusions, dependence upon the number of image measurements and their sampling rate.
Recognition of planar object classes

We present a new framework for recognizing planar object classes, which is based on local feature detectors and a probabilistic model of the spatial arrangement of the features. The allowed object deformations are represented through shape statistics, which are learned from examples. Instances of an object in an image are detected by finding the appropriate features in the correct spatial configuration. The algorithm is robust with respect to partial occlusion, detector false alarms, and missed features. A 94% success rate was achieved for the problem of locating quasi-frontal views of faces in cluttered scenes.
Reducing "structure from motion"

The literature on recursive estimation of structure and motion from monocular image sequences comprises a large number of different models and estimation techniques. We propose a framework that allows us to derive and compare all models by following the idea of dynamical system reduction. The "natural" dynamic model, derived by the rigidity constraint and the perspective projection, is first reduced by explicitly decoupling structure (depth) from motion. Then implicit decoupling techniques are explored, which consist of imposing that some function of the unknown parameters is held constant. By appropriately choosing such a function, not only can we account for all models seen so far in the literature, but we can also derive novel ones. Casting all the different models in a common framework allows us to compare their geometric properties on common experimental grounds.
Visual input for pen-based computers

Handwriting may be captured using a video camera, rather than the customary pressure-sensitive tablet. This paper presents a simple system based on correlation and recursive prediction methods that can track the tip of the pen in real time with sufficient spatio-temporal resolution and accuracy to enable handwritten character recognition. The system is tested on a large and heterogeneous set of examples and its performance is compared to that of three human operators and a commercial high-resolution pressure-sensitive tablet.
Motion estimation via dynamic vision

Estimating the three-dimensional motion of an object from a sequence of projections is of paramount importance in a variety of applications in control and robotics, such as autonomous navigation, manipulation, servo, tracking, docking, planning, and surveillance. Although "visual motion estimation" is an old problem (the first formulations date back to the beginning of the century), only recently have tools from nonlinear systems estimation theory hinted at acceptable solutions. In this paper the authors formulate the visual motion estimation problem in terms of identification of nonlinear implicit systems with parameters on a topological manifold and propose a dynamic solution either in the local coordinates or in the embedding space of the parameter manifold. Such a formulation has structural advantages over previous recursive schemes, since the estimation of motion is decoupled from the estimation of the structure of the object being viewed, and therefore it is possible to handle occlusions in a principled way.