waashk/dblp huggingface dataset:
bilateral filtering for gray and color images. bilateral filtering smooths images while preserving edges, by means of a nonlinear combination of nearby image values. the method is noniterative, local, and simple. it combines gray levels or colors based on both their geometric closeness and their photometric similarity, and prefers near values to distant values in both domain and range. in contrast with filters that operate on the three bands of a color image separately, a bilateral filter can enforce the perceptual metric underlying the cie-lab color space, and smooth colors and preserve edges in a way that is tuned to human perception. also, in contrast with standard filtering, bilateral filtering produces no phantom colors along edges in color images, and reduces phantom colors where they appear in the original image.
3d tracking = classification + interpolation. hand gestures are examples of fast and complex motions.computers fail to track these in fast video, but sleight ofhand fools humans as well: what happens too quickly wejust cannot see. we show a 3d tracker for these types ofmotions that relies on the recognition of familiar configurationsin 2d images (classification), and fills the gapsin-between (interpolation). we illustrate this idea with experimentson hand motions similar to finger spelling. thepenalty for a recognition failure is often small: if two configurationsare confused, they are often similar to eachother, and the illusion works well enough, for instance, todrive a graphics animation of the moving hand. we contributeadvances in both feature design and classifier training:our image features are invariant to image scale, translation,and rotation, and we propose a classification methodthat combines vqpca with discrimination trees.
an integrated bayesian approach to layer extraction from image sequences. this paper describes a bayesian approach for modeling 3d scenes as a collection of approximately planar layers that are arbitrarily positioned and oriented in the scene. in contrast to much of the previous work on layer-based motion modeling, which computes layered descriptions of 2d image motion, our work leads to a 3d description of the scene. there are two contributions within the paper. the first is to formulate the prior assumptions about the layers and scene within a bayesian decision making framework which is used to automatically determine the number of layers and the assignment of individual pixels to layers. the second is algorithmic. in order to achieve the optimization, a bayesian version of ransac is developed with which to initialize the segmentation. then, a generalized expectation maximization method is used to find the map solution.
robust computation and parametrization of multiple view relations. a new method is presented for robustly estimating multiple view relations from image point correspondences. there are three new contributions, the first is a general purpose method of parametrizing these relations using point correspondences. the second contribution is the formulation of a common maximum likelihood estimate (mle) for each of the multiple view relations. the parametrization facilitates a constrained optimization to obtain the mle. the third contribution is a new robust algorithm, mlesac, for obtaining the point correspondences.the method is general and its use is illustrated for the estimation of fundamental matrices, image to image homographics and quadratic transformations.results are given for both synthetic and real images. it is demonstrated that the method gives results equal or superior to previous approaches.
robust detection of degenerate configurations for the fundamental matrix. new methods are reported for the detection of multiple solutions (degeneracy) when estimating the fundamental matrix, with specific emphasis on robustness in the presence of data contamination (outliers). the fundamental matrix can be used as a first step in the recovery of structure from motion. if the set of correspondences is degenerate then this structure cannot be accurately recovered and many solutions will explain the data equally well. it is essential that we are alerted to such eventualities. however, current feature matchers are very prone to mismatching, giving a high rate of contamination within the data. such contamination can make a degenerate data set appear non degenerate, thus the need for robust methods becomes apparent. the paper presents such methods with a particular emphasis on providing a method that will work on real imagery and with an automated (non perfect) feature detector and matcher. it is demonstrated that proper modelling of degeneracy in the presence of outliers enables the detection of outliers which would otherwise be missed. results using real image sequences are presented. all processing, point matching, degeneracy detection and outlier detection is automatic.
context-based vision system for place and object recognition. while navigating in an environment, a vision system has to be able to recognize where it is and what the main objects in the scene are. in this paper we present a context-based vision system for place and object recognition. the goal is to identify familiar locations (e.g., office 610, conference room 941, main street), to categorize new environments (office, corridor, street) and to use that information to provide contextual priors for object recognition (e.g., tables are more likely in an office than a street). we present a low-dimensional global image representation that provides relevant information for place recognition and categorization, and show how such contextual information introduces strong priors that simplify object recognition. we have trained the system to recognize over 60 locations (indoors and outdoors) and to suggest the presence and locations of more than 20 different object types. the algorithm has been integrated into a mobile system that provides real-time feedback to the user.
semantic organization of scenes using discriminant structural templates. in this paper, we present a procedure for organizing real world scenes along semantic axes. the approach is based on the output energies of linear discriminant filters that take into account, or not, spatial information.we introduce three semantic axes along which pictures are ordered. the main semantic axis computes the degree of naturalness of a scene. then, urban pictures are evaluated according to their degree of verticalness and natural scenes, according to their degree of openness. we observe the emergence of typical scene categories such as beach, mountain, skyscrapers, city center, etc., along the axes.
matching constraints and the joint image. the paper studies the geometry of multi image perspective projection and the matching constraints that this induces on image measurements. the combined image projections define a 3d joint image subspace of the space of combined homogeneous image coordinates. this is a complete projective replica of the 3d world in image coordinates. its location encodes the imaging geometry and is captured by the 4 index joint image grassmannian tensor. projective reconstruction in the joint image is a canonical process requiring only a simple rescaling of image coordinates. reconstruction in world coordinates amounts to a choice of basis in the joint image. the matching constraints are multilinear tensorial equations in image coordinates that tell whether tokens in different images could be the projections of a single world token. for 2d images of 3d points there are exactly three basic types: the epipolar constraint, a. shashua's (1995) trilinear one, and a new quadrilinear 4 image one. for images of lines, r. hartley's (1994) trilinear constraint is the only type. the coefficients of the matching constraints are tensors built directly from the joint image grassmannian. their complex algebraic interdependency is captured by quadratic structural simplicity constraints on the grassmannian.
variational frameworks for dt-mri estimation, regularization and visualization. we address three crucial issues encountered in dt-mri (diffusion tensor magnetic resonance imaging) : diffusion tensor estimation, regularization and fiber bundle visualization. we first review related algorithms existing in the literature and propose then alternative variational formalisms that lead to new and improved schemes, thanks to the preservation of important tensor constraints (positivity, symmetry). we illustrate how our complete dt-mri processing pipeline can be successfully used to construct and draw fiber bundles in the white matter of the brain, from a set of noisy raw mri images.
image parsing: unifying segmentation, detection, and recognition. we propose a general framework for parsing images into regions and objects. in this framework, the detection and recognition of objects proceed simultaneously with image segmentation in a competitive and cooperative manner. we illustrate our approach on natural images of complex city scenes where the objects of primary interest are faces and text. this method makes use of bottom-up proposals combined with top-down generative models using the data driven markov chain monte carlo (ddmcmc) algorithm which is guaranteed to converge to the optimal estimate asymptotically. more precisely, we define generative models for faces, text, and generic regions- e.g. shading, texture, and clutter. these models are activated by bottom-up proposals. the proposals for faces and text are learnt using a probabilistic version of adaboost. the ddmcmc combines reversible jump and diffusion dynamics to enable the generative models to explain the input images in a competitive and cooperative manner. our experiments illustrate the advantages and importance of combining bottom-up and top-down models and of performing segmentation and object detection/recognition simultaneously.
simultaneous multiple 3d motion estimation via mode finding on lie groups. we propose a new method to estimate multiple rigid motions from noisy 3d point correspondences in the presence of outliers. the method does not require prior specification of number of motion groups and estimates all the motion parameters simultaneously. we start with generating samples from the rigid motion distribution. the motion parameters are then estimated via mode finding operations on the sampled distribution. since rigid motions do not lie on a vector space, classical statistical methods can not be used for mode finding. we develop a mean shift algorithm which estimates modes of the sampled distribution using the lie group structure of the rigid motions. we also show that proposed mean shift algorithm is general and can be applied to any distribution having a matrix lie group structure. experimental results on synthetic and real image data demonstrate the superior performance of the algorithm.
state space construction for behavior acquisition in multi agent environments with vision and action. this paper proposes a method which estimates the relationships between learner's behaviors and other agents' ones in the environment through interactions (observation and action) using the method of system identification. in order to identify the model of each agent, akaike's information criterion is applied to the results of canonical variate analysis for the relationship between the observed data in terms of action and future observation. next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior. the proposed method is applied to a soccer playing situation, where a rolling ball and other moving agents are well modeled and the learner's behaviors are successfully acquired by the method. computer simulations and real experiments are shown and a discussion is given.
plane-based calibration algorithm for multi-camera systems via factorization of homography matrices. a new calibration algorithm for multi-camera systemsusing a planar reference pattern is proposed. the algorithmis an extension of sturm-maybank-zhang style plane-basedcalibration technique for use with multiple cameras. rigiddisplacements between the cameras are recovered as well asthe intrinsic parameters only by capturing with the camerasa model plane with known reference points placed at threeor more locations. thus the algorithm yields a simple calibrationmeans for stereo vision systems with an arbitrarynumber of cameras while maintaining the handiness andflexibility of the original method. the algorithm is based onfactorization of homography matrices between the modeland image planes into the camera and plane parameters.to compensate for the indetermination of scaling factors,each homography matrix is rescaled by a double eigenvalueof a planar homology defined by two views and two modelplanes. the obtained parameters are finally refined by anon-linear maximum likelihood estimation (mle) process.the validity of the proposed technique was verified throughsimulation and experiments with real data.
towards an active visual observer. we present a binocular active vision system that can attend to and fixate a moving target. our system has an open and expandable design and it forms the first steps of a long term effort towards developing an active observer using vision to interact with the environment, in particular capable of figure-ground segmentation. we also present partial real-time implementations of this system and show their performance in real-world situations together with motor control. in pursuit we particularly focus on occlusions of other targets, both stationary and moving, and integrate three cues, ego-motion, target motion and target disparity, to obtain an overall robust behavior. an active vision system must be open, expandable, and operate with whatever data are available momentarily. it must also be equipped with means and methods to direct and change its attention. this system is therefore equipped with motion detection for changing attention and pursuit for maintaining attention, both of which run concurrently.
priors for people tracking from small training sets. we advocate the use of scaled gaussian process latent variable models (sgplvm) to learn prior models of 3d human pose for 3d people tracking. the sgplvm simultaneously optimizes a low-dimensional embedding of the high-dimensional pose data and a density function that both gives higher probability to points close to training data and provides a nonlinear probabilistic mapping from the low-dimensional latent space to the full-dimensional pose space. the sgplvm is a natural choice when only small amounts of training data are available. we demonstrate our approach with two distinct motions, golfing and walking. we show that the sgplvm sufficiently constrains the problem such that tracking can be accomplished with straighforward deterministic optimization.
grouping based on projective geometry constraints and uncertainty. the process of grouping and subsequently recognising objects in cluttered images is one laden with difficulties, however, results can be greatly enhanced if the inherent uncertainty of image-features is taken into account. this paper shows that starting with the individual edgel's uncertainty it is possible to calculate covariance-information for all derived quantities. this information can be used to choose between competing algorithms, selecting the one that produces the more reliable results, but also as an aid during the recognition process. the consequent application of error-propagation leads to a new formulation for the calculation of the cross-ratio, which is both robust and efficient in dealing with measured lines, and does notrequire knowledge about the vanishing point. extensive monte-carlo simulations as well as the application to images of cluttered street-scenes demonstrate the robustness and suitability of the approach.
flux maximizing geometric flows. several geometric active contour models have been proposed for segmentation in computer vision and image analysis. the essential idea is to evolve a curve (in 2d) or a surface (in 3d) under constraints from image forces so that it clings to features of interest in an intensity image. recent variations on this theme take into account properties of enclosed regions and allow for multiple curves or surfaces to be simultaneously represented. however, it is still unclear how to apply these techniques to images of narrow elongated structures, such as blood vessels, where intensity contrast may be low and reliable region statistics cannot be computed. to address this problem, we derive the gradient flows which maximize the rate of increase of flux of an appropriate vector field through a curve (in 2d) or a surface (in 3d). the key idea is to exploit the direction of the vector field along with its magnitude. the calculations lead to a simple and elegant interpretation which is essentially parameter free and has the same form in both dimensions. we illustrate its advantages with several level-set-based segmentations of 2d and 3d angiography images of blood vessels.
kalmansac: robust filtering by consensus. we propose an algorithm to perform causal inference of the state of a dynamical model when the measurements are corrupted by outliers. while the optimal (maximum-likelihood) solution has doubly exponential complexity due to the combinatorial explosion of possible choices of inliers, we exploit the structure of the problem to design a sampling-based algorithm that has constant complexity. we derive our algorithm from the equations of the optimal filter, which makes our approximation explicit. our work is motivated by real-time tracking and the estimation of structure from motion (sfm). we test our algorithm for on-line outlier rejection both for tracking and for sfm. we show that our approach can tolerate a large proportion of outliers, whereas previous causal robust statistical inference methods failed with less than half as many. our work can be thought of as the extension of random sample consensus algorithms to dynamic data, or as the implementation of pseudo-bayesian filtering algorithms in a sampling framework.
features for recognition: viewpoint invariance for non-planar scenes. most current local feature detectors/descriptors implicitly assume that the scene is (locally) planar, an assumption that is violated at surface discontinuities. we show that this restriction is, at least in theory, unnecessary, as one can construct local features that are viewpoint-invariant for generic non-planar scenes. however, we show that any such feature necessarily sacrifices shape information, in the sense of being non shape-discriminative. finally, we show that if viewpoint is factored out as part of the matching process, rather than explicitly in the representation, then shape is discriminative indeed. we illustrate our theoretical results empirically by showing that, even for simple scenes, current affine descriptors fail where even a naive 3-d viewpoint invariant succeeds in matching.
three-dimensional scene flow. just as optical flow is the two-dimensional motion of points in an image, scene flow is the three-dimensional motion of points in the world. the fundamental difficulty with optical flow is that only the normal flow can be computed directly from the image measurements, without some form of smoothing or regularization. in this paper, we begin by showing that the same fundamental limitation applies to scene flow; however, many cameras are used to image the scene. there are then two choices when computing scene flow: 1) perform the regularization in the images or 2) perform the regularization on the surface of the object in the scene. in this paper, we choose to compute scene flow using regularization in the images. we describe three algorithms, the first two for computing scene flow from optical flows and the third for constraining scene structure from the inconsistencies in multiple optical flows.
snake pedals: geometric models with physics-based control. in this paper, we introduce a novel geometric shape modeling scheme which allows for representation of global and local shape characteristics of an object. geometric models are traditionally well suited for representing global shapes but not the local details. however, in this paper we propose a powerful geometric shape modeling scheme which allows for the representation of global shap es with local detail and permits model shaping as well as topological changes via physics-based control.the proposed modeling scheme consists of representing shapes by pedal curves and surfaces ¿ pedal curves/surfaces are the loci of the foot of perpendiculars to the tangents of a fixed curve/surface from a fixed point called the pedal point. by varying the location of the pedal point, one can synthesize a large class of shapes which exhibit both local and glob al deformations. we introduce physics-based control for shaping these geometric models by letting the pedal point vary and use a dynamic spline to represent the position of this varying pedal point. the model dubbed as a "snake pedal" allows for interactive manipulation via forces applied to the snake. we demonstrate the applicability of this modeling scheme via examples of shape synthesis and shape estimation from real image data.
maintaining multi-modality through mixture tracking. in recent years particle filters have become a tremendouslypopular tool to perform tracking for non-linearand/or non-gaussian models. this is due to their simplicity,generality and success over a wide range of challengingapplications. particle filters, and monte carlo methodsin general, are however poor at consistently maintainingthe multi-modality of the target distributions that may arisedue to ambiguity or the presence of multiple objects. toaddress this shortcoming this paper proposes to model thetarget distribution as a non-parametric mixture model, andpresents the general tracking recursion in this case. it isshown how a monte carlo implementation of the generalrecursion leads to a mixture of particle filters that interactonly in the computation of the mixture weights, thus leadingto an efficient numerical algorithm, where all the resultspertaining to standard particle filters apply. the ability ofthe new method to maintain posterior multi-modality is illustratedon a synthetic example and a real world trackingproblem involving the tracking of football players in a videosequence.
finding the epipole from uncalibrated optical flow. this paper presents a novel method for determining the location of the instantaneous epipole in a sequence of images acquired by an uncalibrated camera and containing a single, rigid motion (e.g., the camera moves in a static environment). the method uses the full perspective camera model and requires the estimation of the optical flow at a minimum of six image locations. the key observation is that the optical flow equations can be written in terms of the epipole in a strikingly simple form if the translation and rotational flow components are not separated as done usually. the epipole location can then be obtained as the minimum of a least-square residual function asdsociated to the computed optical flow. we report and discuss initial experiments on both synthetic and real data and illustrate possible developments of this method towards the use of uncalibrated optical flow for 3-d motion and structure reconstruction.