nils-herrmann/clean_scirepeval huggingface dataset:
Twenty Questions, Focus of Attention, and A*: A Theoretical Comparison of Optimization Strategies

Many vision problems involve the detection of the boundary of an object, like a hand, or the tracking of a one-dimensional structure, such as a road in an aerial photograph. These problems can be formulated in terms of Bayesian probability theory and hence expressed as optimization problems on trees or graphs. The twenty questions, or minimum entropy, algorithm has recently been developed by Geman and Jedynak (1994) as a highly effective, and intuitive, tree search algorithm for road tracking. In this paper we analyse this algorithm to understand how it compares to existing algorithms used for vision, and related, optimization problems. First we show that it is a special case of the focus of attention planning strategy used on causal graphs, or Bayes nets, [18]. We then show its relations to standard methods, already successfully applied to vision optimization problems, such as dynamic programming, decision trees, the A* algorithm used in artificial intelligence [22] and the, closely related, Dijkstra algorithm of computer science . These comparisons show that twenty questions is often equivalent to an algorithm, which we call A+, which tries to explore the most probable paths first. We show that A+ is a greedy, and suboptimal, variant of A*. This suggests that A+ and twenty questions will be faster than A* and Dijkstra for certain problems but they may occasionally converge to the wrong answer. However, the fact that A+ and twenty questions maintain a probabilistic estimate of how well they are doing may give warning of faulty convergence and also allow intelligent pruning to speed up the search.
Surface shape from warping

We propose a variant of shape from shading which we call shape from image warping. The idea is that the three-dimensional shape of an object is estimated by determining how much the image of the object is warped with respect to the image of a known prototype shape. We demonstrate that, for a class of reflectance functions, there is a direct relationship between these image warps and geometric warps of the underlying three-dimensional shapes. Therefore detecting the image warp relative to a prototype of known shape allows us to reconstruct the shape of the imaged object. We derive properties of these shape warps and illustrate the results by recovering the shapes effaces. This relationship between image and shape warps helps us understand the relationship between image based models of object recognition and approaches based on three-dimensional object models.
The bas-relief ambiguity

Since antiquity, artisans have created flattened forms, often called "bas-reliefs,"-which give an exaggerated perception of depth when viewed from a particular vantage point. This paper presents an explanation of this phenomena, showing that the ambiguity in determining the relief of an object is not confined to bas-relief sculpture but is implicit in the determination of the structure of any object. Formally, if the object's true surface is denoted by z/sub true/=f(x, y), then we define the "generalized bas-relief transformation" as z=/spl lambda/f(x, y)+/spl mu/x+/spl nu/y, with a corresponding transformation of the albedo. For each image of a Lambertian surface f(x, y) produced by a point light source at infinity, there exists an identical image of a bas-relief produced by a transformed light source. This equality holds for both shaded and shadowed regions. Thus, the set of possible images (illumination cone) is invariant over generalized bas-relief transformations. When /spl mu/=/spl nu/=0 (e.g. a classical bas-relief sculpture), we show that the set of possible motion fields are also identical. Thus, neither small unknown motions nor changes of illumination can resolve the bas-relief ambiguity. Implications of this ambiguity on structure recovery and shape representation are discussed.
mpetition: Unifying Snakes, egion Growin and Bayes/MDL for Image Segmentation

We present a novel statistical and variational approach to image segmentation based on a new algorithm named region compefifion. This algorithm is derived by minimizing a generalized BayesiMDL criterion using the variational principle. The algorithm is guaranteed to converge to a local minimum and combines aspects of snakesiballoons and region growing. Indeed the classic snakesiballoons and region growing algorithms can be directly derived from our approach. We provide theoretical analysis of region competition including accuracy of boundary location, criteria for initial conditions, and the relationship to edge detection using filters. It is straightforward to generalize the algorithm to multiband segmentation and we demonstrate it on gray level images, color images and texture images. The novel color model allows us to eliminate intensity gradients and shadows, thereby obtaining segmentation based on the albedos of objects. It also helps detect highlight regions. Index Terms-Image segmentation, region growing, snakes, minimum description length, Bayes statistics, uncertainty principle, color model.
Bayesian decision theory and psychophysics

We argue that Bayesian decision theory provides a good theoretical framework for visual perception Such a theory involves a likelihood function specifying how the scene generates the image s a prior assumption about the scene and a decision rule to determine the scene interpretation This is illustrated by describing Bayesian theories for individual visual cues and showing that perceptual biases found in psychophysical experiments can be interpreted as biases towards prior assumptions made by the visual system We then describe the implications of this framework for the integration of di erent cues We argue that the dependence of cues on prior assumptions means that care must be taken to model these dependencies during integration This suggests that a number of proposed schemes for cue integration which only allow weak interaction between cues are not adequate and instead stronger coupling is often required These theories require the choice of decision rules and we argue that this choice is important since these rules help capture the task dependent nature of vision This is illustrated by analysing the generic viewpoint assumption Finally we suggest that the visual system uses a set of competing prior assumptions rather than the single generic priors or natural constraints commonly used in computational theories of vision
Learning Object Representation form Lighting Variations

Realistic representation of objects requires models which can synthesize the image of an object under all possible viewing conditions. We propose to learn these models from examples. Methods for learning surface geometry and albedo from one or more images under fixed posed and varying lighting conditions are described. Singular value decomposition (SVD) is used to determine shape, albedo, and lighting conditions up to an unknown 3×3 matrix, which is sufficient for recognition. The use of class-specific knowledge and the integrability constraint to determine this matrix is explored. We show that when the integrability constraint is applied to objects with varying albedo it leads to an ambiguity in depth estimation similar to the bas relief ambiguity. The integrability constraint, however, is useful for resolving ambiguities which arise in current photometric theories.
Learning Object Representations from LightingVariationsR

Realistic representation of objects requires models which can synthesize the image of an object under all possible viewing conditions. We propose to learn these models from examples. Methods for learning surface geometry and albedo from one or more images under fixed posed and varying lighting conditions are described. Singular value decomposition (SVD) is used to determine shape, albedo, and lighting conditions up to an unknown 3×3 matrix, which is sufficient for recognition. The use of class-specific knowledge and the integrability constraint to determine this matrix is explored. We show that when the integrability constraint is applied to objects with varying albedo it leads to an ambiguity in depth estimation similar to the bas relief ambiguity. The integrability constraint, however, is useful for resolving ambiguities which arise in current photometric theories. Object Recognition Workshop.
Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation

We present a novel statistical and variational approach to image segmentation based on a new algorithm, named region competition. This algorithm is derived by minimizing a generalized Bayes/minimum description length (MDL) criterion using the variational principle. The algorithm is guaranteed to converge to a local minimum and combines aspects of snakes/balloons and region growing. The classic snakes/balloons and region growing algorithms can be directly derived from our approach. We provide theoretical analysis of region competition including accuracy of boundary location, criteria for initial conditions, and the relationship to edge detection using filters. It is straightforward to generalize the algorithm to multiband segmentation and we demonstrate it on gray level images, color images and texture images. The novel color model allows us to eliminate intensity gradients and shadows, thereby obtaining segmentation based on the albedos of objects. It also helps detect highlight regions.
Learning object representations from greyscale images

This thesis develops algorithms for learning the surface geometry and albedo of an object from image sets taken under varying lighting conditions and from different viewpoints. In principle, recovery of this information allows all sources of image variation--pose, lighting, viewpoint, albedo, geometric deformations, articulations--to be modeled independently and explicitly. The algorithms described rely on the assumption that the lambertian reflectance model is adequate for these surfaces. To support this claim, the dimensionality of the space of appearance changes due to lighting variations is empirically investigated. It is concluded that a low-dimensional lighting model is sufficient even for surfaces with sharp specularities or cast shadows, as these can be treated as outliers. It is then shown that surface normals can be recovered up to an undetermined $3\times 3$ transformation by performing singular value decomposition on the image set. Several methods are explored for determining this matrix, including use of a prototype model, and use of the integrability constraint. This allows surface geometry, albedo, and light-source direction to be determined. Finally, an algorithm is developed for recovery of shape and albedo from a set of images taken from different viewpoints. This algorithm uses both shading information and geometric disparity to determine shape. Key use is made of a prototype shape model to give an initial estimate of the correspondence of points in the several images. This algorithm is demonstrated on synthetic images.
A Theoretical Framework for Visual Motion

We propose a theoretical framework for visual motion. Models in this framework are constructed using two design principles: First, their mechanisms should be biologically plausible and consistent with experiments. Second, they should be well-engineered to perform well on motion data from realistic scenes. The framework is first described at a conceptual level where we show that it can, in principle, account for a range of psychophysical, anatomical, and physiological phenomena. Then we describe some specific models (parts of the framework which we have developed in detail) and show that they agree in detail with some of these phenomena. The framework suggests a number of experimental and computational questions, which must be answered as a complete theory of motion is developed.
Region competition: unifying snakes, region growing, energy/Bayes/MDL for multi-band image segmentation

We present a novel statistical and variational approach to image segmentation based on a new algorithm named region competition. This algorithm is derived by minimizing a generalized Bayes/MDL (Minimum Description Length) criterion using the variational principle. We show that existing techniques in early vision such as, snake/balloon models, region growing, and Bayes/MDL are addressing different aspects of the same problem and they can be unified within a common statistical framework which combines their advantages. We analyze how to optimize the precision of the resulting boundary location by studying the statistical properties of the region competition algorithm and discuss what are good initial conditions for the algorithm. Our method is generalized to color and texture segmentation and is demonstrated on grey level images, color images and texture images.<<ETX>>
Re ion COm et it ion: Unifying Snakes,Region Growing, inergy/Bayes P MDL for Multi-band Image Segmentation

their boundaries and require good initial estimates to yield We present a novel statistical and variational approach to image segmentation baaed on a new algorithm named region competition. This algorithm is derived by minimizing a generalized Bayes/MDL(Minimum Description Length) criterion using the variational principle. We show that existing techniques in early vision such as, snake/balloon models, region growing, and Bayes/MDL are addressing different aspects of the same problem and they can be unified within a common statistical framework which combines their advantages. We analyze how to optimize the precision of the resulting boundary location by studying the statistical properties of the region competition algorithm and discuss what are good initial conditions for the algorithm. Our method is generalized to color and texture segmentation and is demonstrated on grey level images, color images and texture images.
Recovering object surfaces from viewed changes in surface texture patterns

Explores the reconstruction of object surfaces from viewed changes in surface texture patterns. Our approach differs from those in the past in that instead of simply producing local estimates of the surface orientation, our algorithm recovers complete surfaces. Past approaches only found the surface orientation locally and, therefore, did not take advantage of the surface integrability constraint. Our algorithm does not assume that the surface texture pattern is isotropic, and it does not assume that the viewed surface is at some point fronto-parallel. Furthermore, our algorithm has mechanisms for handling texture boundaries and, consequently, does not produce erratic results in the regions abutting these boundaries. Results on real images are presented demonstrating the potential of our approach.<<ETX>>
Region Competition and its Analysis: A Unified Theory for Image Segmentation

We present a novel statistical and variational approach to image segmentation based on a new algorithm named region competition. This algorithm is derived by minimizing a generalized Bayes/MDL criterion using the variational principle. The algorithm is guaranteed to converge to a local minimum and combines aspects of snakes/balloons and region growing. Indeed the classic snakes/balloons and region growing algorithms can be directly derived from our approach. We provide theoretical analysis of region competition including accuracy of boundary location, criteria for initial conditions, and the relationship to edge detection using lters. It is straightforward to generalize the algorithm to multi-band segmentation and we demonstrate it on grey level images, color images and texture images. The novel color model allows us to eliminate intensity gradients and shadows, thereby obtaining segmentation based on the albedos of objects, and it also helps detect highlight regions.
FORMS: a flexible object recognition and modelling system

We briefly describe a generic statistical framework for representing the shapes of animate objects using principal component analysis and stochastic shape grammars. Such a representation scheme gives a formalism for solving the inverse problem-object recognition. Then we show: how these representations can be extracted from 2D silhouettes by a novel method for skeleton extraction and shape segmentation; how a similarity metric can be defined on this shape space; and how we can perform recognition in a bottom up/top down loop. The system is demonstrated to be stable in the presence of noise, the absence of parts, the presence of additional parts, and considerable variations in articulation and viewpoint. Successful categorization is demonstrated on a dataset of seventeen categories of animate objects.<<ETX>>
Robust principal component analysis by self-organizing rules based on statistical physics approach

This paper applies statistical physics to the problem of robust principal component analysis (PCA). The commonly used PCA learning rules are first related to energy functions. These functions are generalized by adding a binary decision field with a given prior distribution so that outliers in the data are dealt with explicitly in order to make PCA robust. Each of the generalized energy functions is then used to define a Gibbs distribution from which a marginal distribution is obtained by summing over the binary decision field. The marginal distribution defines an effective energy function, from which self-organizing rules have been developed for robust PCA. Under the presence of outliers, both the standard PCA methods and the existing self-organizing PCA rules studied in the literature of neural networks perform quite poorly. By contrast, the robust rules proposed here resist outliers well and perform excellently for fulfilling various PCA-like tasks such as obtaining the first principal component vector, the first k principal component vectors, and directly finding the subspace spanned by the first k vector principal component vectors without solving for each vector individually. Comparative experiments have been made, and the results show that the authors' robust rules improve the performances of the existing PCA algorithms significantly when outliers are present.
NEURAL IMPLEMENTATION OF BAYESIAN VISION THEORIES BY UNSUPERVISED LEARNING

Biological organisms have to estimate properties of the world from visual signals. Many authors, see  and references therein, have suggested that Bayesian estimation theory gives a natural framework for visual perception. Bayesian models exist for such visual abilities as depth perception, object recognition, image segmentation and self-organization. Indeed all of cognition can, in principle, be formulated in Bayesian terms1.
Bayesian Self-Organization Driven by Prior Probability Distributions

Recent work by Becker and Hinton (1992) shows a promising mechanism, based on maximizing mutual information assuming spatial coherence, by which a system can self-organize to learn visual abilities such as binocular stereo. We introduce a more general criterion, based on Bayesian probability theory, and thereby demonstrate a connection to Bayesian theories of visual perception and to other organization principles for early vision (Atick and Redlich 1990). Methods for implementation using variants of stochastic learning are described.
5/spl plusmn/2 eigenimages suffice: an empirical investigation of low-dimensional lighting models

Recently, P.W. Hallinan (1994) proposed a low dimensional lighting
model for describing the variations in face images due to altering the
lighting conditions. It was found that five eigenimages were sufficient
to model these variations. This report shows that this model can be
extended to other objects, in particular to those with diffuse
specularities and shadows. We find that sharp specularities and shadows
cannot be well represented by a low dimensional model. However, both
effects can be adequately described as residuals to such a model. We can
deal with occluders in a similar way. We conclude that low dimensional
models, using 5±2 eigenimages, can be usefully extended to
represent arbitrary lighting for many different objects. We discuss
applications of these results to object recognition
Discovering objects and their location in images with Latent Dirichlet Allocation

We seek to discover object categories and their locations in a set of unlabelled images. We achieve this using probabilistic models developed in the text understanding community to discover interesting topics in a corpus of text documents. We hope that the application of these models to a set of images will discover visual topics corresponding to object categories. We show how to form the visual analogue of text documents by vector quantizing SIFT-like regions at points found through two types of interest point detectors. We primarily focus on the Latent Dirichlet Allocation (LDA) model and compare with the very similar probabilistic Latent Semantic Analysis (pLSA) model. We describe the LDA model and the Gibbs sampling solution for Bayesian inference. We fit the LDA model to image sets for which we know ground-truth labels and investigate the visual topics that are discovered. In addition, for images with primarily a single object category depicted, we use the inferred LDA parameters to assign these images to the discovered visual topics and evaluate the goodness of these assignments through confusion matrices. Furthermore, we evaluate assignments to topics on unseen images and compare with an existing supervised method. Finally, we formulate a method for localizing objects in an image and evaluate the resulting segmentations. Since the LDA model assumes that words are exchangable in a document, we show how to improve the segmentations by introducing a new vocabulary called ‘doublets’, which encode spatially local co-occurring regions.
Recovering intrinsic images from a single image

Interpreting real-world images requires the ability distinguish the different characteristics of the scene that lead to its final appearance. Two of the most important of these characteristics are the shading and reflectance of each point in the scene. We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image. Using both color information and a classifier trained to recognize gray-scale patterns, given the lighting direction, each image derivative is classified as being caused by shading or a change in the surface's reflectance. The classifiers gather local evidence about the surface's form and color, which is then propagated using the generalized belief propagation algorithm. The propagation step disambiguates areas of the image where the correct classification is not clear from local evidence. We use real-world images to demonstrate results and show how each component of the system affects the results.
Motion magnification

We present motion magnification, a technique that acts like a microscope for visual motion. It can amplify subtle motions in a video sequence, allowing for visualization of deformations that would otherwise be invisible. To achieve motion magnification, we need to accurately measure visual motions, and group the pixels to be modified. After an initial image registration step, we measure motion by a robust analysis of feature point trajectories, and segment pixels based on similarity of position, color, and motion. A novel measure of motion similarity groups even very small motions according to correlation over time, which often relates to physical cause. An outlier mask marks observations not explained by our layered motion model, and those pixels are simply reproduced on the output from the original registered observations.The motion of any selected layer may be magnified by a user-specified amount; texture synthesis fills-in unseen "holes" revealed by the amplified motions. The resulting motion-magnified images can reveal or emphasize small motions in the original sequence, as we demonstrate with deformations in load-bearing structures, subtle motions or balancing corrections of people, and "rigid" structures bending under hand pressure.