nils-herrmann/clean_scirepeval huggingface dataset:
Object Detection and Contextual Priming

There is general consensus that context can be a rich source of information about an object’s identity, location and scale. In fact, the structure of many real-world scenes is governed by strong configurational rules akin to those that apply to a single object. There is increasing evidence of an early use of contextual information in human perception. However, object-centered approaches dominate the research in computer vision. Under favorable conditions, the multiplicity of cues (color, shape, texture) in the retinal image produced by an object seems to provide enough information to unambiguously determine the object category. Under such high quality viewing conditions, the object recognition mechanisms could rely exclusively on intrinsic object features ignoring the background. However, in situations with poor viewing quality (caused, for instance, by large distances) context appears to play a major role in enhancing the reliability of recognition. In the absence of enough local evidence about an object’s identity, the scene structure and prior knowledge of world regularities provide additional information for recognizing and localizing an object. Even when objects can be identified via intrinsic information, context can simplify the object discrimination by cutting down on the number of object categories, scales and positions that need to be considered.
Example-Based Super-Resolution

We call methods for achieving high-resolution enlargements of pixel-based images super-resolution algorithms. Many applications in graphics or image processing could benefit from such resolution independence, including image-based rendering (IBR), texture mapping, enlarging consumer photographs, and converting NTSC video content to high-definition television. We built on another training-based super-resolution algorithm and developed a faster and simpler algorithm for one-pass super-resolution. Our algorithm requires only a nearest-neighbor search in the training set for a vector derived from each patch of local image data. This one-pass super-resolution algorithm is a step toward achieving resolution independence in image-based representations. We don't expect perfect resolution independence-even the polygon representation doesn't have that-but increasing the resolution independence of pixel-based representations is an important task for IBR.
Teaching applied computing without programming: a case-based introductory course for general education

We introduce general-education students to key ideas in applied computing through case studies from computer graphics, computer animation, image processing, computer vision, information retrieval, and artificial intelligence. Each case study consists of two lectures: one an intuitive exposition of relevant computer-science concepts, and the other a hands-on introduction to a working system that embodies these concepts. Students use these systems to perform design and problem-solving tasks, thereby reinforcing the abstract concepts presented. Computer programming is neither required nor taught. The course has been offered for two years at the Harvard University Extension School, and has achieved high ratings in student surveys.
Image quilting for texture synthesis and transfer

We present a simple image-based method of generating novel visual appearance in which a new image is synthesized by stitching together small patches of existing images. We call this process image quilting. First, we use quilting as a fast and very simple texture synthesis algorithm which produces surprisingly good results for a wide range of textures. Second, we extend the algorithm to perform texture transfer — rendering an object with a texture taken from a different object. More generally, we demonstrate how an image can be re-rendered in the style of a different image. The method works directly on the images and does not require 3D information.
On the fixed points of the max-product algorithm

Graphical models, such as Bayesian networks and Markov random fields represent statistical dependencies of variables by a graph. The max-product b̈elief propagation̈ algorithm is a localmessage passing algorithm on this graph that is known to converge to a unique fixed point when the graph is a tree. Furthermore, when the graph is a tree, the assignment based on the fixed-point is guaranteed to yields the most probable a posteriori (MAP) values of the unobserved variables given the observed ones. Here we prove a result on the fixed points of max-product on a graph with arbitrary toplogy and with arbitrary probability distributions (discrete or continuous valued nodes). We show that the assignment based on the fixed-point is a n̈eighborhood maximum̈ of the posterior probability: the posterior probability of the max-product assignment is guaranteed to be greater than all other assignments in a particular large region around that assignment. The region includes all assignments that differ from the max-product assignment in any subset of nodes that form no more than a single loop in the graph. In some graphs this neighborhood is exponentially large. We illustrate the analysis with examples. This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved.
Artificial retina chips as on-chip image processors and gesture-oriented interfaces

Players of a video game may sometimes find the use of conventional interfaces inappropriate. In such cases, we think that interfaces realized with a vision-based gesture recognition system may find favor. The artificial retina (AR) chip is a versatile image sensor whose use ranges from normal image acquisition to on-chip image processing, including on-chip image convolution. In this paper, we describe a gesture- input video game system, with the AR module including the AR chip, and motion-based gesture recognition algorithms. We showed that the algorithms can be accelerated by projection data, the direct output from the AR chip. To show its performance, we have applied our system to two commercially available video games.
Computer vision for television and games

Computer vision can be used as a computer interface for applications such as games or television displays. The requirements imposed by the applications of high speed and low cost can be severe. Fortunately, the real-time application often makes the computer vision task easier, by providing constraints which restrict the interpretations possible the visual input, and by inserting a human into the control loop. I'll describe applications with vision interfaces: computer games and television sets. I'll describe the basic vision technologies that make the applications possible: head and hand tracking and gesture recognition. We'll see that it fits together as a system: fast and simple vision algorithms taking advantage of constraints imposed by the real-time applications. Some of these applications can be made faster or cheaper using specialized hardware for detection and processing, such as the "artificial retina" chip.
Markov networks for low-level vision

We seek a learning-based algorithm that applies to various low-level vision problems. For each problem, we want to nd the scene interpretation that best explains image data. For example, we may want to infer the projected velocities (scene) which best explain two consecutive image frames (image). From synthetic data, we model the relationship between local image and scene regions, and between a scene region and neighboring scene regions. Three probabilities are learned, which characterize the low-level vision algorithm: the local prior, the local likelihood, and the the conditional probabilities of scene neighbors. Given a new image, we propagate likelihood functions in a Markov network to infer the underlying scene. We use a factorization approximation, ignoring the e ect of loops. This yields an e cient method to infer low-level scene interpretations, which we always nd to be stable. We illustrate the method with di erent representations, and show it working for three applications: an explanatory example, motion analysis and estimating high resolution images from low-resolution ones. This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonpro t educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Information Technology Center America; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Information Technology Center America. All rights reserved.
Bayesian Reconstruction of 3D Human Motion from Single-Camera Video

The three-dimensional motion of humans is underdetermined when the observation is limited to a single camera, due to the inherent 3D ambiguity of 2D video. We present a system that reconstructs the 3D motion of human subjects from single-camera video, relying on prior knowledge about human motion, learned from training data, to resolve those ambiguities. After initialization in 2D, the tracking and 3D reconstruction is automatic; we show results for several video sequences. The results show the power of treating 3D body tracking as an inference problem.
An Inexpensive, All Solid-state Video and Data Recorder for Accident Reconstruction

Anaccuraterecordoftheeventsprecedingatracaccidentisusefultotrucking companies,carrentalagencies,commercialcarriers,andinsuranceproviders,as well as to passenger car drivers. A video and data recording is very helpful for accurateaccidentreconstruction. Wehavemadeaprototypeofa\personaleye witness": a small video camera with solid state memory and a short recording time. Thecamerarecordscontinuouslyuntilitsensesanaccident, thenitsaves a video record of the events surrounding the accident in time. Since video is available from both before and after the collision, accident reconstruction becomeseasier. Thecamerausesonlysolidstatememorywithnomovingparts, can be manufactured inexpensively, and can be packaged in a crash-resistant and tamper-resistant module. Automobile data can beadded assidebanddata or overlaid text to the video data storage path.
Loopy Belief Propagation Gives Exact Posterior Means for Gaussian

Graphical models, such as Bayesian networks and Markov Random Fields represent joint distributions over a set of variables by means of a graph. When the graph is singly connected, local "belief propagation" rules of the sort proposed by Pearl (1988) are guaranteed to converge to the correct posterior probabilities. Recently, a number of researchers have empirically demonstrated good performance "loopy belief propagation" -- using these same rules on graphs with loops. Perhaps the most dramatic instance is the near Shannon-limit performance of "Turbo Codes" whose decoding algorithm is equivalent to loopy belief propagation. Except for the case of graphs with a single loop, there has been very little theoretical understanding of the performance of loopy propagation. Here we prove that when the nodes in the graph describe jointly Gaussian random variables, if belief propagation converges then it will give the correct posterior means for all graph topologies, not just networks with a single loop. This justifies using belief propagation in a broader class of networks, and helps clarify the empirical performance results.
Learning low-level vision

We show a learning-based method for low-level vision problems-estimating scenes from images. We generate a synthetic world of scenes and their corresponding rendered images. We model that world with a Markov network, learning the network parameters from the examples. Bayesian belief propagation allows us to efficiently find a local maximum of the posterior probability for the scene, given the image. We call this approach VISTA- We apply VISTA to the "super-resolution" problem (estimating high frequency details from a low-resolution image), showing good results. For the motion estimation problem, we show figure/ground discrimination, solution of the aperture problem, and filling-in arising from application of the same probabilistic machinery.
A Factorization Approach to Grouping

The foreground group in a scene may be ‘discovered’ and computed as a factorized approximation to the pairwise affinity of the elements in the scene. A pointwise approximation of the pairwise affinity information may in fact be interpreted as a ‘saliency’ index, and the foreground of the scene may be obtained by thresholding it. An algorithm called ‘affinity factorization’ is thus obtained which may be used for grouping.
William T. Freeman

Image intensity variations can result from several di erent object surface e ects, including shading from 3-dimensional relief of the object, or paint on the surface itself. An essential problem in vision, which people solve naturally, is to attribute the proper physical cause, e.g. surface relief or paint, to an observed image. We addressed this problem with an approach combining psychophysical and Bayesian computational methods. We assessed human performance on a set of test images, and found that people made fairly consistent judgements of surface properties. Our computational model assigned simple prior probabilities to di erent relief or paint explanations for an image, and solved for the most probable interpretation in a Bayesian framework. The ratings of the test images by our algorithm compared surprisingly well with the mean ratings of our subjects.
Bayesian Estimation of 3-D Human Motion

We address the problem of reconstructing the 3-dimensional motions of a human gure from a monocular image sequence. We take a statistical approach, and use a set of motion capture examples to build a gaussian probability model for short human motion sequences. We rst study this model in a simpli ed rendering domain. This yields analytic results for the optimal 3-d estimate given a 2-d temporal sequence, as well as for which motion modes are di cult to estimate. The results from the simpli ed rendering conditions show that if we can overlay a stick gure on an image of a moving human, we can estimate his or her 3-d motion well. We built an interactive tracking system to process real video sequences, and can achieve good 3-d reconstructions of the human gure motion. This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonpro t educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Information Technology Center America; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Information Technology Center America. All rights reserved.
Interfaces for humans (panel): natural interaction, tangible data, and beyond

When users talk about computers, they usually describe the interfaces, because, for most users, the interface is the system. As Bill Buxton says, “The most powerful force in shaping people’s mental model of the nature of the beast is that which they see, feel, and hear.” It seemed to take forever for toggle-switch panels to evolve into today’s WIMPs, and both are still visual/motor-based controls; in fact, switch panels were probably more haptically satisfying! “Keyboards only work for people who know the Roman alphabet. In 20 years, people will laugh at us for calling that technology,” says Mark Lucente.
Learning to Estimate Scenes from Images

We seek the scene interpretation that best explains image data. For example, we may want to infer the projected velocities (scene) which best explain two consecutive image frames (image). From synthetic data, we model the relationship between image and scene patches, and between a scene patch and neighboring scene patches. Given a new image, we propagate likelihoods in a Markov network (ignoring the effect of loops) to infer the underlying scene. This yields an efficient method to form low-level scene interpretations. We demonstrate the technique for motion analysis and estimating high resolution images from low-resolution ones.
Design galleries: a general approach to setting parameters for computer graphics and animation

Image rendering maps scene parameters to output pixel values; animation maps motion-control parameters to trajectory values. Because these mapping functions are usually multidimensional, nonlinear, and discontinuous, finding input parameters that yield desirable output values is often a painful process of manual tweaking. Interactive evolution and inverse design are two general methodologies for computer-assisted parameter setting in which the computer plays a prominent role. In this paper we present another such methodology. Design GalleryTM (DG) interfaces present the user with the broadest selection, automatically generated and organized, of perceptually different graphics or animations that can be produced by varying a given input-parameter vector. The principal technical challenges posed by the DG approach are dispersion, finding a set of input-parameter vectors that optimally disperses the resulting output-value vectors, and arrangement, organizing the resulting graphics for easy and intuitive browsing by the user. We describe the use of DG interfaces for several parameter-setting problems: light selection and placement for image rendering, both standard and image-based; opacity and color transfer-function specification for volume rendering; and motion control for particle-system and articulated-figure animation. CR Categories: I.2.6 [Artificial Intelligence]: Problem Solving, Control Methods and Search—heuristic methods; I.3.6 [Computer Graphics]: Methodology and Techniques—interaction techniques; I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism.
Bayesian color constancy.

The problem of color constancy may be solved if we can recover the physical properties of illuminants and surfaces from photosensor responses. We consider this problem within the framework of Bayesian decision theory. First, we model the relation among illuminants, surfaces, and photosensor responses. Second, we construct prior distributions that describe the probability that particular illuminants and surfaces exist in the world. Given a set of photosensor responses, we can then use Bayes's rule to compute the posterior distribution for the illuminants and the surfaces in the scene. There are two widely used methods for obtaining a single best estimate from a posterior distribution. These are maximum a posteriori (MAP) and minimum mean-square-error (MMSE) estimation. We argue that neither is appropriate for perception problems. We describe a new estimator, which we call the maximum local mass (MLM) estimate, that integrates local probability density. The new method uses an optimality criterion that is appropriate for perception tasks: It finds the most probable approximately correct answer. For the case of low observation noise, we provide an efficient approximation. We develop the MLM estimator for the color-constancy problem in which flat matte surfaces are uniformly illuminated. In simulations we show that the MLM method performs better than the MAP estimator and better than a number of standard color-constancy algorithms. We note conditions under which even the optimal estimator produces poor estimates: when the spectral properties of the surfaces in the scene are biased.
A gesture controlled human interface using an artificial retina chip

To realize advanced multi-media systems for use in computers, advanced home audio-visual equipment, games machines, public information systems, and so on, it is necessary to develop vision-based input methods, which do not require keyboards or joysticks. We have developed a flexible, high-speed and compact artificial retina (AR) chip module composed of an AR chip and a micro-processor for human interface systems. The AR chip is a novel type of CCD image sensor with versatile and high-speed on-chip processing capabilities. The features are suitable for human friendly input systems employing image information. We have also developed a unique gesture recognition algorithm, which is tailored for the AR chip. The gesture recognition system permits applications with gesture controlled human interfaces.
The generic viewpoint assumption in a Bayesian framework

The g̈eneric vieẅassumption states that an observer is not in a special position relative to the scene. It is commonly used to disqualify scene interpretations that assume special viewpoints, following a binary decision that the viewpoint was either generic or accidental. In this chapter, we show how to use the generic view assumption to quantify the likelihood of a view. This quantitative approach can be applied to estimate scene parameters. This approach applies to many vision problems. We show shape from shading examples where we rank shapes or reflectance functions in cases which are otherwise ambiguous. The rankings agree with the perceived values. Bayesian Perspectives on Perception, David Knill and Whitman Richards, editors,  This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved.
Computer vision for computer games

The appeal of computer games may be enhanced by vision-based user inputs. The high speed and low cost requirements for near-term, mass-market game applications make system design challenging. The response time of the vision interface should be less than a video frame time and the interface should cost less than $50 U.S. We meet these constraints with algorithms tailored to particular hardware. We have developed a special detector, called the artificial retina chip, which allows for fast, on-chip image processing. We describe two algorithms, based on image moments and orientation histograms, which exploit the capabilities of the chip to provide interactive response to the player's hand or body positions at 10 msec frame time and at low-cost. We show several possible game interactions.