nils-herrmann/clean_scirepeval huggingface dataset:
The Joy of Sampling

A standard method for handling Bayesian models is to use Markov chain Monte Carlo methods to draw samples from the posterior. We demonstrate this method on two core problems in computer vision—structure from motion and colour constancy. These examples illustrate a samplers producing useful representations for very large problems. We demonstrate that the sampled representations are trustworthy, using consistency checks in the experimental design. The sampling solution to structure from motion is strictly better than the factorisation approach, because: it reports uncertainty on structure and position measurements in a direct way; it can identify tracking errors; and its estimates of covariance in marginal point position are reliable. Our colour constancy solution is strictly better than competing approaches, because: it reports uncertainty on surface colour and illuminant measurements in a direct way; it incorporates all available constraints on surface reflectance and on illumination in a direct way; and it integrates a spatial model of reflectance and illumination distribution with a rendering model in a natural way. One advantage of a sampled representation is that it can be resampled to take into account other information. We demonstrate the effect of knowing that, in our colour constancy example, a surface viewed in two different images is in fact the same object. We conclude with a general discussion of the strengths and weaknesses of the sampling paradigm as a tool for computer vision.
Automatic Detection of Human Nudes

This paper demonstrates an automatic system for telling whether there are human nudes present in an image. The system marks skin-like pixels using combined color and texture properties. These skin regions are then fed to a specialized grouper, which attempts to group a human figure using geometric constraints on human structure. If the grouper finds a sufficiently complex structure, the system decides a human is present. The approach is shown to be effective for a wide range of shades and colors of skin and human configurations. This approach offers an alternate view of object recognition, where an object model is an organized collection of grouping hints obtained from a combination of constraints on color and texture and constraints on geometric properties such as the structure of individual parts and the relationships between parts. The system demonstrates excellent performance on a test set of 565 uncontrolled images of human nudes, mostly obtained from the internet, and 4289 assorted control images, drawn from a wide variety of sources.
Motion synthesis

We investigate the problem of synthesizing natural looking motions for virtual characters. Animating virtual characters is an important problem for many applications ranging from training simulations to computer games and movie production. In this thesis, we present motion synthesis methods that generate character motions by cutting and pasting motion capture data. Selecting a collection of clips from a motion database, that yields an acceptable motion is a combinatorial problem. Human motion is dynamically constrained by muscle strength and physics. This means we can not change our motion arbitrarily quickly. We take advantage of this observation and present efficient motion synthesis algorithms for generating motions that satisfy a variety of constraints automatically. These constraints include geometric constraints that prescribe the spatial structure of the motion, such as “Go from point A to point B in a specific amount of time” as well as annotation constraints that prescribe the types of motions to be performed, such as “Walk, pickup an object and then run”. Human looking motions that satisfy the constraints are synthesized interactively. Complex motions can be synthesized by imposing a combination of multiple constraints. 
We also present a semi-supervised motion classification method which is useful for annotating large sets of motion. This method has been successfully used to learn instances of particular types of motion for our synthesis algorithms and is also useful for action recognition.
Artificial Intelligence 3 D object recognition using invariance

The systems and concepts described in this paper document the evolution of the geometric invariance approach to object recognition over the last five years. Invariance overcomes one of the fundamental difficulties in recognising objects from images: that the appearance of an object depends on viewpoint. This problem is entirely avoided if the geometric description is unaffected by the imaging transformation. Such invariant descriptions can be measured from images without any prior knowledge of the position, orientation and calibration of the camera. These invariant measurements can be used to index a library of object models for recognition and provide a principled basis for the other stages of the recognition process such as feature grouping and hypothesis verification. Object models can be acquired directly from images, allowing efficient construction of model libraries without manual intervention. A significant part of the paper is a summary of recent results on the construction of invariants for 3D objects from a single perspective view. A proposed recognition architecture is described which enables the integration of multiple general object classes and provides a means for enforcing global scene consistency. Various criticisms of the invariant approach are articulated and addressed.
Motion synthesis from annotations

This paper describes a framework that allows a user to synthesize human motion while retaining control of its qualitative properties. The user paints a timeline with annotations --- like walk, run or jump --- from a vocabulary which is freely chosen by the user. The system then assembles frames from a motion database so that the final motion performs the specified actions at specified times. The motion can also be forced to pass through particular configurations at particular times, and to go to a particular position and orientation. Annotations can be painted positively (for example, must run), negatively (for example, may not run backwards) or as a don't-care. The system uses a novel search method, based around dynamic programming at several scales, to obtain a solution efficiently so that authoring is interactive. Our results demonstrate that the method can generate smooth, natural-looking motion.The annotation vocabulary can be chosen to fit the application, and allows specification of composite motions (run and jump simultaneously, for example). The process requires a collection of motion data that has been annotated with the chosen vocabulary. This paper also describes an effective tool, based around repeated use of support vector machines, that allows a user to annotate a large collection of motions quickly and easily so that they may be used with the synthesis algorithm.
Finding and tracking people from the bottom up

We describe a tracker that can track moving people in long sequences without manual initialization. Moving people are modeled with the assumption that, while configuration can vary quite substantially from frame to frame, appearance does not. This leads to an algorithm that firstly builds a model of the appearance of the body of each individual by clustering candidate body segments, and then uses this model to find all individuals in each frame. Unusually, the tracker does not rely on a model of human dynamics to identify possible instances of people; such models are unreliable, because human motion is fast and large accelerations are common. We show our tracking algorithm can be interpreted as a loopy inference procedure on an underlying Bayes net. Experiments on video of real scenes demonstrate that this tracker can (a) count distinct individuals; (b) identify and track them; (c) recover when it loses track, for example, if individuals are occluded or briefly leave the view; (d) identify the configuration of the body largely correctly; and (e) is not dependent on particular models of human motion.
Words and Pictures in the News

We discuss the properties of a collection of news photos and captions, collected from the Associated Press and Reuters. Captions have a vocabulary dominated by proper names. We have implemented various text clustering algorithms to organize these items by topic, as well as an iconic matcher that identifies articles that share a picture. We have found that the special structure of captions allows us to extract some names of people actually portrayed in the image quite reliably, using a simple syntactic analysis. We have been able to build a directory of face images of individuals from this collection.
Automatic Annotation of Everyday Movements

This paper describes a system that can annotate a video sequence with: a description of the appearance of each actor; when the actor is in view; and a representation of the actor's activity while in view. The system does not require a fixed background, and is automatic. The system works by (1) tracking people in 2D and then, using an annotated motion capture dataset, (2) synthesizing an annotated 3D motion sequence matching the 2D tracks. The 3D motion capture data is manually annotated off-line using a class structure that describes everyday motions and allows motion annotations to be composed — one may jump while running, for example. Descriptions computed from video of real motions show that the method is accurate.
Using temporal coherence to build models of animals

We describe a system that can build appearance models of animals automatically from a video sequence of the relevant animal with no explicit supervisory information. The video sequence need not have any form of special background. Animals are modeled as a 2D kinematic chain of rectangular segments, where the number of segments and the topology of the chain are unknown. The system detects possible segments, clusters segments whose appearance is coherent over time, and then builds a spatial model of such segment clusters. The resulting representation of the spatial configuration of the animal in each frame can be seen either as a track - in which case the system described should be viewed as a generalized tracker, that is capable of modeling objects while tracking them - or as the source of an appearance model which can be used to build detectors for the particular animal. This is because knowing a video sequence is temporally coherent - i.e. that a particular animal is present through the sequence - is a strong supervisory signal. The method is shown to be successful as a tracker on video sequences of real scenes showing three different animals. For the same reason it is successful as a tracker, the method results in detectors that can be used to find each animal fairly reliably within the Corel collection of images.
Interactive motion generation from examples

There are many applications that demand large quantities of natural looking motion. It is difficult to synthesize motion that looks natural, particularly when it is people who must move. In this paper, we present a framework that generates human motions by cutting and pasting motion capture data. Selecting a collection of clips that yields an acceptable motion is a combinatorial problem that we manage as a randomized search of a hierarchy of graphs. This approach can generate motion sequences that satisfy a variety of constraints automatically. The motions are smooth and human-looking. They are generated in real time so that we can author complex motions interactively. The algorithm generates multiple motions that satisfy a given set of constraints, allowing a variety of choices for the animator. It can easily synthesize multiple motions that interact with each other using constraints. This framework allows the extensive re-use of motion capture data for new purposes.
Computer Vision: A Modern Approach

From the Publisher: 
The accessible presentation of this book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods. A CD-ROM with every copy of the text contains source code for programming practice, color images, and illustrative movies. Comprehensive and up-to-date, this book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.
Shape from Texture without Boundaries

We describe a shape from texture method that constructs a maximum a posteriori estimate of surface coefficients using only the deformation of individual texture elements. Our method does not need to use either the boundary of the observed surface or any assumption about the overall distribution of elements. The method assumes that texture elements are of a limited number of types of fixed shape. We show that, with this assumption and assuming generic view and texture, each texture element yields the surface gradient unique up to a two-fold ambiguity. Furthermore, texture elements that are not from one of the types can be identified and ignored. An EM-like procedure yields a surface reconstruction from the data. The method is defined for othographic views -- an extension to perspective views appears to be complex, but possible. Examples of reconstructions for synthetic images of surfaces are provided, and compared with ground truth. We also provide examples of reconstructions for images of real scenes. We show that our method for recovering local texture imaging transformations can be used to retexture objects in images of real scenes.
Proxy Simulations for Efficient Dynamics

Proxy simulations reduce the cost of simulation in large virtual worlds, such as those used in training simulations or computer games. A proxy takes the place of an accurate simulation for objects that are out of view, while the accurate model continues to manage visible objects. A proxy must ensure that objects enter the view at reasonable times throughout the simulation and in states that reect their time spent out of view. The quality of a proxy simulation is measured by how well it maintains reasonable behaviors, where the denition of reasonable depends on the environment and its application. We present two examples of proxy simulations based on discrete event models: one for city trafc simulation and another for multi-agent path planning and motion. For these examples, we demonstrate dynamics computation speedups of over two orders of magnitude as the environments grow in size and complexity.
Mixtures of trees for object recognition

Efficient detection of objects in images is complicated by variations of object appearance due to intra-class object differences, articulation, lighting, occlusions, and aspect variations. To reduce the search required for detection, we employ the bottom-up approach where we find candidate image features and associate some of them with parts of the object model. We represent objects as collections of local features, and would like to allow any of them to be absent, with only a small subset sufficient for detection;furthermore, our model should allow efficient correspondence search. We propose a model, Mixture of Trees, that achieves these goals. With a mixture of trees, we can model the individual appearances of the features, relationships among them, and the aspect, and handle occlusions. Independences captured in the model make efficient inference possible. In our earlier work, we have shown that mixtures of trees can be used to model objects with a natural tree structure, in the context of human tracking. Now we show that a natural tree structure is not required, and use a mixture of trees for both frontal and view-invariant face detection. We also show that by modeling faces as collections of features we can establish an intrinsic coordinate frame for a face, and estimate the out-of-plane rotation of a face.
Human Tracking with Mixtures of Trees

Tree-structur ed probabilistic modelsadmit simple, fast inference. However, they are not well suitedto phenomena such asocclusion,where multiplecomponentsof an object maydisappearsimultaneously . Mixturesof treesappearto addressthis problem, at the cost of representinga large mixture. We demonstr ate an efficient and compactrepresentationof this mixture, which admitssimplelearningand inferencealgorithms. We usethis methodto build an automatedtracker for Muybridge sequencesof a variety of human activities. Tracking is difficult, becausethe temporal dependencies rule out simple inferencemethods. We showhow to use our modelfor efficient inference, usinga methodthat employsalternatespatial and temporal inference. Theresult is a tracker that (a) usesa veryloosemotionmodel,andso cantrack manydifferentactivitiesat a variable framerate and(b) is entirely automatic.
Efficient multi-agent path planning

Animating goal-driven agents in an environment with obstacles is a time consuming process, particularly when the number of agents is large. In this paper, we introduce an efficient algorithm that creates path plans for objects that move between user defined goal points and avoids collisions. In addition, the system allows "culling" of some of the computation for invisible agents: agents are accurately simulated only if they are visible to the user while the invisible objects are approximated probabilistically. The approximations ensure that the agent's behaviors match those that would occur had they been fully simulated, and result in significant speedups over running the accurate simulation for all agents.
Noise in bilinear problems

Despite the wide application of bilinear problems to problems both in computer vision and in other fields, their behaviour under the effects of noise is still poorly understood. In this paper, we show analytically that marginal distributions on the solution components of a bilinear problem can be bimodal, even with Gaussian measurement error. We demonstrate and compare three different methods of estimating the covariance of a solution. We show that the Hessian at the mode substantially underestimates covariance. Many problems in computer vision can be posed as bilinear problems: i.e. one must find a solution to a set of equations of the form.
Benchmarks for storage and retrieval in multimedia databases

There is a substantial body of research on computer methods for managing collections of images and videos. There is little evidence that this research has had important impact on an any community yet. I use an invitation to speak on a topic on which I am not expert to air some opinions about evaluating image retrieval research. In my opinion, there is little to be gained in measuring current solutions with reference collections, because these solutions differ so widely from user needs that the exercise becomes empty. The user studies literature is not well enough read by the image retrieval community. As a result, we tend to study somewhat artificial problems. A study of the user needs literature suggests that we will need to solve deep problems to produce useful solutions to image retrieval problems, but that there may be a need for a number of technologies that can be built in practice. I believe we should concentrate on these issues, rather than on measuring the performance of current systems.
Human tracking with mixtures of trees

Tree-structured probabilistic models admit simple, fast inference. However they are not well suited to phenonena such as occlusion, where multiple components of an object may disappear simultaneously. We address this problem with mixtures of trees, and demonstrate an efficient and compact representation of this mixture, which admits simple learning and inference algorithms. We use this method to build an automated tracker for Muybridge sequences of a variety of human activities. Tracking is difficult, because the temporal dependencies rule out simple inference methods. We show how to use our model for efficient inference, using a method that employs alternate spatial and temporal inference. The result is a cracker that (a) uses a very loose motion model, and so can track many different activities at a variable frame rate and (b) is entirely, automatic.
Shape from texture and integrability

We describe a shape from texture method that constructs a maximum a posteriori estimate of surface coefficients using both the deformation of individual texture elements-as in local methods-and the overall distribution of elements-as in global methods. The method described applies to a much larger family of textures than any previous method, local or global. We demonstrate an analogy with shape from shading, and use this to produce a numerical method. Examples of reconstructions for synthetic images of surfaces are provided, and compared with ground truth. The method is defined for orthographic views, but can be generalised to perspective views simply.
Towards interactive ray tracing in two- and three-dimensions

One of the main goals of computer graphics is to generate photorealistic images at high frame rates. This thesis presents techniques for generating ray traced images in a 2D environment at interactive rates, and closely approximating ray traced images in a 3D environment at near interactive rates. In the two dimensional case, our technique is based on the visibility complex data structure . The visibility complex is a data structure in a dual space whose faces correspond to contiguous sets of rays in the primary space that share the same forward and backward views. Sweeping a viewing ray around a viewpoint corresponds to walking along a trajectory on the visibility complex. Producing a ray traced image is equivalent to walking along and maintaining a set of trajectories. Generating ray traced images with the visibility complex is very efficient since the coherence among the rays is effectively captured. We have developed a novel algorithm for the construction of the visibility complex which is based on the randomized incremental paradigm. The history of the incremental construction yields an efficient ray query data structure, which is required for spawning new reflected and refracted rays. The performance of our rendering algorithm is analyzed and the rendering speed of our algorithm is experimentally compared with the rendering speed of a conventional ray tracer. 
For the three dimensional case, we have developed a technique called extended environment mapping which is based on the environment mapping algorithm but produces much more accurate images. An extended environment map records the depth information of the scene along with the radiance information stored as an environment map. This technique is based on the observation that capturing the radiance along the wrong direction from the right point produces much more accurate rendering than capturing the radiance along the right direction from the wrong point, which is how an ordinary environment map generates images. The accuracy of our technique is demonstrated by presenting difference images computed between ray traced images and the images rendered using extended environment mapping. The extended environment map produces images up to an order of magnitude faster than a conventional ray tracer.