waashk/dblp huggingface dataset:
a model of figure-ground segregation from kinetic occlusion. the paper presents a model of image segmentation that can distinguish foreground from background purely on the basis of motion information. the main processing steps involved are: detection of motion boundaries, and analysis of figure ground relationship. the proposed model utilizes the observation that in kinetic occlusion, motion boundaries typically display mixture motion information, and foreground surfaces tend to move with motion boundaries. through distributed probabilistic modeling, these constraints can be embedded into computations with efficient network representations. the resulting networks use spatiotemporal gabor filters as front ends, and are suitable for parallel distributed processing. we demonstrate the application of the model in the decomposition of moving images into surfaces according to depth.
motion from the frontier of curved surfaces. the frontier of a curved surface is the envelope of contour generators showing the boundary, at least locally, of the visible region swept out under viewer motion. in general, the outlines of curved surfaces (apparent contours) from different viewpoints are generated by different contour generators on the surface and hence do not provide a constraint on viewer motion. we show that frontier points, however, have projections which correspond to a real point on the surface and can be used to constrain viewer motion by the epipolar constraint. we show how to recover viewer motion from frontier points for both continuous and discrete motion, calibrated and uncalibrated cameras. we present preliminary results of an iterative scheme to recover the epipolar line structure from real image sequences using only the outlines of curved surfaces. a statistical evaluation as also performed to estimate the stability of the solution.
surface geometry from cusps of apparent contours. it is known that the deformations of the apparent contours of a surface under perspective projection and viewer motion enable the recovery of the geometry of the surface, for example by utilising the epipolar parametrization. these methods break down with apparent contours that are singular i.e. with cusps. in this paper, we study this situation in detail and show how, nevertheless, the surface geometry (including the gauss curvature and mean curvature of the surface) can be recovered by following the cusps. indeed the formulae are much simpler in this case, and require lower spatio-temporal derivatives than in the general case of nonsingular apparent contours. we give a simulated example, and also show that following cusps does not by itself provide us with information on ego-motion.
improving laser triangulation sensors using polarization. we report a novel application of polarization based vision addressing the robustness of laser triangulation range sensors. such sensors are based on the accurate detection of a pattern of laser light projected onto a scene, usually a point or line. typical problems arise with highly specularly reflective surfaces, which can generate visible reflections of the light in various parts of the image. this can confuse the detection algorithms and lead to wrong range measurements. this paper demonstrates experimentally the feasibility of polarization based vision for disambiguating multiple specular inter reflections of the laser light. we concentrate on metal components as they have high interest for inspection in manufacturing, and show positive results with situations of various complexities.
background estimation as a labeling problem. we present a new background estimation algorithm that constructs the background of an image sequence with moving objects by copying areas from input frames. the background estimation problem is formulated as an optimal labeling problem in which the label at an output pixel is the frame number from which to copy the background color. the costs of assigning labels encourage seamless copying from regions that are stationary over a period of time in such a way that implied motion boundaries occur at intensity edges. this is accomplished without explicitly tracking the moving objects or computing optical flow. experiments demonstrate that our algorithm is effective in difficult areas where the background is visible for only a small fraction of time, and on inputs with both moving objects that are not always in motion and moving objects with textureless areas.
auxiliary variables for deformable models. we present a mathematical formulation for curve and surface reconstruction algorithms by introduction of auxiliary variables. for deformable models and templates, two step iterative algorithms have been often used where, at each iteration, the model is first locally deformed according to the potential data attraction and then globally smoothed. we show how these approaches can be interpreted as the introduction of auxiliary variables and the minimization of a two variables energy. this permits us to transform an implicit data constraint defined by a non convex potential into an explicit convex reconstruction problem. we show some mathematical properties and results on this new auxiliary problem, in particular when the potential is a function of the distance to the closest feature point. we then illustrate our approach for some deformable models and templates and image restoration.
the earth mover's distance under transformation sets. the earth mover's distance (emd) is a distance measure between distributions with applications in image retrieval and matching. we consider the problem of computing a transformation of one distribution which minimizes its emd to another.the applications discussed here include estimation of the size at which a color pattern occurs in an image, lighting-invariant object recognition, and point feature matching in stereo image pairs. we present a monotonically convergent iteration which can be applied to a large class of emd under transformation problems, although the iteration may converge to only a locally optimal transformation. we also provide algorithms that are guaranteed to compute a globally optimal transformation for a few specific problems, including some emd under translation problems.
tracking meteorological structures through curve matching using geodesic paths. this paper is concerned with the problem of tracking clouds structures like vortices in meteorological images. for this purpose we characterize the deformation between two successive occurrences, by matching their two boundary curves. our approach is based on the computation of the set of paths connecting the two curves to be matched. it minimizes a cost function which measures the local similarity of the two curves. these matching paths are obtained as geodesic curves on this cost surface. moreover our method allows to consider complex curves of arbitrary topologysince these curves are represented through an implicit function rather than through a parameterization. experimental results are given to illustrate the properties of themethod in processing synthetic and then meteorologic remotely-sensed data.
site model acquisition and extension from aerial images. a system has been developed to acquire, extend and refine 3d geometric site models from aerial imagery. this system hypothesizes potential building roofs in an image, automatically locates supporting geometric evidence in other images, and determines the precise shape and position of the new buildings via multiimage triangulation. model-to-image registration techniques are applied to align new, incoming images against the site model. model extension and refinement procedures are then performed to add previously unseen buildings and to improve the geometric accuracy of the existing 3d building models.
on-line selection of discriminative tracking features. this paper presents a method for evaluating multiple feature spaces while tracking, and for adjusting the set of features used to improve tracking performance. our hypothesisis that the features that best discriminate between object and background are also best for tracking the object. we develop an on-line feature selection mechanism based on the two-class variance ratio measure, applied to log likelihood distributions computed with respect to a given feature from samples of object and background pixels. this feature selection mechanism is embedded in a tracking system that adaptively selects the top-ranked discriminative features for tracking. examples are presented to illustrate how the method adapts to changing appearances of both tracked object and scene background.
mean shift analysis and applications. a non-parametric estimator of density gradient, the mean shift, is employed in the joint, spatial-range (value) domain of gray level and color images for discontinuity preserving filtering and image segmentation. properties of the mean shift are reviewed and its convergence on lattices is proven.the proposed filtering method associates with each pixel in the image the closest local mode in the density distribution of the joint domain. segmentation into a piecewise constant structure requires only one more step, fusion of the regions associated with nearby modes.the proposed technique has two parameters controlling the resolution in the spatial and range domains. since convergence is guaranteed, the technique does not require the intervention of the user to stop the filtering at the desired image quality. several examples, for gray and color images, show the versatility of the method and compare favorably with results described in the literature for the same images.
real-time obstacle avoidance using central flow divergence and peripheral flow. the lure of using motion vision as a fundamental element in the perception of space drives this effort to use flow features as the sole cues for robot mobility. real-time estimates of image flow and flow divergence provide the robot's sense of space. the robot steers down a conceptual corridor comparing left and right peripheral flows. large central flow divergence warns the robot of impending collisions at "dead ends." when this occurs, the robot turns around and resumes wandering. behavior is generated by directly using flow-based information in the 2d image sequence; no 3d reconstruction is attempted. active mechanical gate stabilization simplifies the visual interpretation problems by reducing camera rotation. by combining corridor following and dead-end deflection, the robot has wandered around the lab at 30 cm/s for as long as 20 minutes without collision. the ability to support this behavior in real-time with current equipment promises expanded capabilities as computational power increases in the future.
a multi-body factorization method for motion analysis. the structure from motion problem has been extensively studied in the field of computer vision. yet, the bulk of the existing work assumes that the scene contains only a single moving object. the more realistic case where an unknown number of objects move in the scene has received little attention, especially for its theoretical treatment. we present a new method for separating and recovering the motion and shape of multiple independently moving objects in a sequence of images. the method does not require prior knowledge of the number of objects, nor is dependent on any grouping of features into an object at the image level. for this purpose, we introduce a mathematical construct of object shapes, called the shape interaction matrix, which is invariant to both the object motions and the selection of coordinate systems. this invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects. once the structure is computed, it allows for segmenting features into objects by the process of transforming it into a canonical form, as well as recovering the shape and motion of each object.
manhattan world: compass direction from a single image by bayesian inference. when designing computer vision systems for the blind and visually impaired it is important to determine the orientation of the user relative to the scene. we observe that most indoor and outdoor (city) scenes are designed on a manhattan three-dimensional grid. this manhattan grid structure puts strong constraints on the intensity gradients in the image. we demonstrate an algorithm for detecting the orientation of the user in such scenes based on bayesian inference using statistics which we have learnt in this domain. our algorithm requires a single input image and does not involve pre-processing stages such as edge detection and hough grouping. we demonstrate strong experimental results on a range of indoor and outdoor images. we also show that estimating the grid structure makes it significantly easier to detect target objects which are not aligned with the grid.
variational space-time motion segmentation. we propose a variational method for segmenting imagesequences into spatio-temporal domains of homogeneousmotion. to this end, we formulate the problem of motionestimation in the framework of bayesian inference, using aprior which favors domain boundaries of minimal surfacearea. we derive a cost functional which depends on a surfacein space-time separating a set of motion regions, aswell as a set of vectors modeling the motion in each region.we propose a multiphase level set formulation of thisfunctional, in which the surface and the motion regions arerepresented implicitly by a vector-valued level set function.joint minimization of the proposed functional results in aneigenvalue problem for the motion model of each region andin a gradient descent evolution for the separating interface.numerical results on real-world sequences demonstratethat minimization of a single cost functional generates asegmentation of space-time into multiple motion regions.
single view metrology. we describe how 3d affine measurements may be computed from a single perspective view of a scene given only minimal geometric information determined from the image. this minimal information is typically the vanishing line of a reference plane, and a vanishing point for a direction not parallel to the plane. it is shown that affine scene structure may then be determined from the image, without knowledge of the camera's internal calibration (e.g. focal length), nor of the explicit relation between camera and world (pose).in particular, we show how to (i) compute the distance between planes parallel to the reference plane (up to a common scale factor)&semi; (ii) compute area and length ratios on any plane parallel to the reference plane&semi; (iii) determine the camera's location. simple geometric derivations are given for these results. we also develop an algebraic representation which unifies the three types of measurement and, amongst other advantages, permits a first order error propagation analysis to be performed, associating an uncertainty with each measurement.we demonstrate the technique for a variety of applications, including height measurements in forensic images and 3d graphical modelling from single images.
gaze manipulation for one-to-one teleconferencing. a new algorithm is proposed for novel view generation in one-to-one teleconferencing applications. given the video streams acquired by two cameras placed on either side of a computer monitor, the proposed algorithm synthesises images from a virtual camera in arbitrary position (typically located within the monitor) to facilitate eye contact. our technique is based on an improved, dynamic-programming, stereo algorithm for efficient novel-view generation. the two main contributions of this paper are: i) a new type of three-plane graph for dense-stereo dynamic-programming, that encourages correct occlusion labeling; ii) a compact geometric derivation for novel-view synthesis by direct projection of the minimum-cost surface. furthermore, this paper presents a novel algorithm for the temporal maintenance of a background model to enhance the rendering of occlusions and reduce temporal artefacts (flicker); and a cost aggregation algorithm that acts directly on our three-dimensional matching cost space. examples are given that demonstrate the robustness of the new algorithm to spatial and temporal artefacts for long stereo video streams. these include demonstrations of synthesis of cyclopean views of extended conversational sequences. we further demonstrate synthesis from a freely translating virtual camera.
learning-based hand sign recognition using shoslif-m. we present a self-organizing framework called the shoslif-m for learning and recognizing spatiotemporal events (or patterns) from intensity image sequences. the proposed framework consists of a multiclass, multivariate discriminant analysis to automatically select the most discriminating features (mdf), a space partition tree to achieve a logarithmic retrieval time complexity for a database of n items, and a general interpolation scheme to do view inference and generalization in the mdf space based on a small number of training samples. the system is tested to recognize 28 different hand signs. the experimental results show that the learned system can achieve a 96% recognition rate for test sequences that have not been used in the training phase.
better optical triangulation through spacetime analysis. the standard methods for extracting range data from optical triangulation scanners are accurate only for planar objects of uniform reflectance illuminated by an incoherent source. using these methods, curved surfaces, discontinuous surfaces, and surfaces of varying reflectance cause systematic distortions of the range data. coherent light sources such as lasers introduce speckle artifacts that further degrade the data. we present a new ranging method based on analyzing the time evolution of the structured light reflections. using our spacetime analysis, we can correct for each of these artifacts, thereby attaining significantly higher accuracy using existing technology. we present results that demonstrate the validity of our method using a commercial laser stripe triangulation scanner.
a stochastic filter for fluid motion tracking. in this paper we present a method for the tracking of fluid flows velocity fields. the technique we propose is formalized within sequential bayesian filter framework. the filter we propose here combines an itô diffusion process coming from a stochastic formulation of the vorticity-velocity form of navier-stokes equation and discrete measurements extracted from an image sequence. the resulting tracker provides robust and consistent estimations of instantaneous motion fields along the whole image sequence. in order to handle a state space of reasonable dimension for the stochastic filtering problem, we represent the motion field as a combination of adapted basis functions. the used basis functions ensue from a mollification of biot-savart integral and a discretization of the vorticity and divergence maps of the fluid vector field. the efficiency of the method is demonstrated on a long real world sequence showing a vortex launch at tip of airplane wing.
correlation model for 3d texture. while an exact definition of texture is somewhat elusive, texture can be qualitatively described as a distribution of color, albedo or local normal on a surface. in the literature, the word texture is often used to describe a color or albedo variation on a smooth surface. we refer to such texture as 2d texture.in real world scenes, texture is often due to surface height variations and can be termed 3d texture. because of local foreshortening and masking, oblique views of 3d texture are not simple transformations of the frontal view. consequently, texture representations such as the correlation function or power spectrum are also affected by local foreshortening and masking.this work presents a correlation model for a particular class of 3d textures. the model characterizes the spatial relationship among neighboring pixels in an image of 3d texture and the change of this spatial relationship with viewing direction.
object recognition in high clutter images using line features. we present an object recognition algorithm that uses model and image line features to locate complex objects in high clutter environments. finding correspondences between model and image features is the main challenge in most object recognition systems. in our approach, corresponding line features are determined by a three-stage process. the first stage generates a large number of approximate pose hypotheses from correspondences of one or two lines in the model and image. next, the pose hypotheses from the previous stage are quickly ranked by comparing local image neighborhoods to the corresponding local model neighborhoods. fast nearest neighbor and range search algorithms are used to implement a distance measure that is unaffected by clutter and partial occlusion. the ranking of pose hypotheses is invariant to changes in image scale, orientation, and partially invariant to affine distortion. finally, a robust pose estimation algorithmis applied for refinement and verification, starting from the few best approximate poses produced by the previous stages. experiments on real images demonstrate robust recognition of partially occluded objects in very high clutter environments.
error-tolerant visual planning of planar grasp. this paper describes an effcient method to calculate, from an image of an object,configurations of a two-fingered robot gripp er that form a "cage" to contain that object. closing the fingers on the object from these configurations is guaranteed to reach a given desired grasp. this builds on the visual grasping theory of blake, taylor and cox [1], which describes how to find optimal grasps. it extends the results of rimon and blake [10], which show how to construct such cages, in two ways. first, a more effcient algorithm for computing the cage is described. second, a further development deals with occlusion by solving thecaging problem within a restricted image window. the new methods greatly reduce the complexity of the visual caging problem, making it feasible in a real time computer vision system.
calibrating pan-tilt cameras in wide-area surveillance networks. pan-tilt cameras are often used as components of wide-area surveillance systems. it is necessary to calibrate these cameras in relation to one another in order to obtain a consistent representation of the entire space. existing methods for calibrating pan-tilt cameras have assumed an idealized model of camera mechanics. in addition, most methods have been calibrated using only asmall range of camera motion. this paper presents a method for calibrating pan-tilt cameras that introduces a more complete model of camera motion. pan and tilt rotations are modeled as occurring around arbitrary axes in space. in addition, the wide area surveillance system itself is used to build a large virtual calibration object, resulting in better calibration than would be possible with a single small calibration target. finally, the proposed enhancements are validated experimentally, with comparisons showing the improvement provided over more traditional methods.
recognizing human action efforts: an adaptive three-mode pca framework. we present a computational framework capable of labelingthe effort of an action corresponding to the perceivedlevel of exertion by the performer (low - high). the approachinitially factorizes examples (at different efforts) ofan action into its three-mode principal components to reducethe dimensionality. then a learning phase is introducedto compute expressive-feature weights to adjust themodel's estimation of effort to conform to given perceptuallabels for the examples. experiments are demonstrated recognizingthe efforts of a person carrying bags of differentweight and for multiple people walking at different paces.