waashk/dblp huggingface dataset:
seeing behind occlusions. the location of objects in images is difficult owing to the view variance of geometric features but can be determined by developing view-insensitive descriptions of the intensities local to image points. view-insensitive descriptions are achieved in this work by describing points in terms of the responses of steerable filters at multiple scales. owing to the use of multiple scales, the vector for each point is, for all practical purposes, unique, and thus can be easily matched to other instances of the point in other images. we show that this method can be extended to handle the case where the area near a point of interest is partially occluded. the method uses a description of the occluder in the form of a template that can be obtained easily via active vision systems using a method such as disparity filtering.
generalised epipolar constraints. in this paper we will discuss structure and motion problems for curved surfaces. these will be studied using the silhouettes or apparent contours in the images. the problem of determining camera motion from the apparent contours of curved three-dimensional surfaces, is studied. it will be shown how special points, called epipolar tangency points or frontier points, can be used to solve this problem. a generalised epipolar constraint is introduced, which applies to points, curves, as well as to apparent contours of surfaces. the theory is developed for both continuous and discrete motion, known and unknown orientation, calibrated and uncalibrated, perspective, weak perspective and orthographic cameras. results of an iterative scheme to recover the epipolar line structure from real image sequences using only the outlines of curved surfaces, is presented. a statistical evaluation is performed to estimate the stability of the solution. it is also shown how the motion of the camera from a sequence of images can be obtained from the relative motion between image pairs.
triangulation for points on lines. triangulation consists in finding a 3d point reprojecting the best as possible onto corresponding image points. it is classical to minimize the reprojection error, which, in the pinhole camera model case, is nonlinear in the 3d point coordinates. we study the triangulation of points lying on a 3d line, which is a typical problem for structure-from-motion in man-made environments. we show that the reprojection error can be minimized by finding the real roots of a polynomial in a single variable, which degree depends on the number of images. we use a set of transformations in 3d and in the images to make the degree of this polynomial as low as possible, and derive a practical reconstruction algorithm. experimental comparisons with an algebraic approximation algorithm and minimization of the reprojection error using gauss-newton are reported for simulated and real data. our algorithm finds the optimal solution with high accuracy in all cases, showing that the polynomial equation is very stable. it only computes the roots corresponding to feasible points, and can thus deal with a very large number of views - triangulation from hundreds of views is performed in a few seconds. reconstruction accuracy is shown to be greatly improved compared to standard triangulation methods that do not take the line constraint into account.
symmetrical dense optical flow estimation with occlusions detection. traditional techniques of dense optical flow estimation do not generally yield symmetrical solutions: the results will differ if they are applied between images i 1 and i 2 or between images i 2 and i 1. in this work, we present a method to recover a dense optical flow field map from two images, while explicitely taking into account the symmetry across the images as well as possible occlusions in the flow field. the idea is to consider both displacements vectors from i 1 to i 2 and i 2 to i 1 and to minimise an energy functional that explicitely encodes all those properties. this variational problem is then solved using the gradient flow defined by the euler---lagrange equations associated to the energy. to prove the importance of the concepts of symmetry and occlusions for optical flow computation, we have extended a classical approach to handle those. experiments clearly show the added value of these properties to improve the accuracy of the computed flows.figures appear in color in the online version of this paper.
eigenfaces vs. fisherfaces: recognition using class specific linear projection. we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression. taking a pattern classification approach, we consider each pixel in an image as a coordinate in a high-dimensional space. we take advantage of the observation that the images of a particular face, under varying illumination but fixed pose, lie in a 3d linear subspace of the high dimensional image space¿if the face is a lambertian surface without shadowing. however, since faces are not truly lambertian surfaces and do indeed produce self-shadowing, images will deviate from this linear subspace. rather than explicitly modeling this deviation, we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation. our projection method is based on fisher's linear discriminant and produces well separated classes in a low-dimensional subspace, even under severe variation in lighting and facial expressions. the eigenface technique, another method based on linearly projecting the image space to a low dimensional subspace, has similar computational requirements. yet, extensive experimental results demonstrate that the proposed "fisherface" method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases.
linear pose estimation from points or lines. estimation of camera pose from an image of n points or lines with known correspondence is a thoroughly studied problem in computer vision. most solutions are iterative and depend on nonlinear optimization of some geometric constraint, either on the world coordinates or on the projections to the image plane. for real-time applications, we are interested in linear or closed-form solutions free of initialization. we present a general framework which allows for a novel set of linear solutions to the pose estimation problem for both n points and n lines. we then analyze the sensitivity of our solutions to image noise and show that the sensitivity analysis can be used as a conservative predictor of error for our algorithms. we present a number of simulations which compare our results to two other recent linear algorithms, as well as to iterative approaches. we conclude with tests on real imagery in an augmented reality setup.
markov random field models in computer vision. :markov random field (mrf) theory provides a basis for modeling contextual constraints in visual processing and interpretation. it enables us to develop optimal vision algorithms systematically when used with optimization principles. this book presents a comprehensive study on the use of mrfs for solving computer vision problems. the book covers the following parts essential to the subject: introduction to fundamental theories, formulations of mrf vision models, mrf parameter estimation, and optimization algorithms. various vision models are presented in a unified framework, including image restoration and reconstruction, edge and region segmentation, texture, stereo and motion, object matching and recognition, and pose estimation. this book is an excellent reference for researchers working in computer vision, image processing, statistical pattern recognition, and applications of mrfs. it is also suitable as a text for advanced courses in these areas. the acm portal is published by the association for computing machinery. copyright © 2010 acm, inc. terms of usage privacy policy code of ethics contact us useful downloads: adobe acrobat quicktime windows media player real player
robust affine structure matching for 3d object recognition. we consider model-based object localization based on local geometric feature matching between the model and the image data. the method is based on geometric constraint analysis, working in transformation space. we present a formal method which guarantees finding all feasible matchings in polynomial time. from there we develop more computationally feasible algorithms based on conservative approximations of the formal method. additionally, our formalism relates object localization, affine model indexing, and structure from multiple views to one another.
a probabilistic theory of occupancy and emptiness. this paper addresses the inference of 3d shape from a set of n noisy photos. we derive a probabilistic framework to specify what one can infer about 3d shape for arbitrarily-shaped, lambertian scenes and arbitrary viewpoint configurations. based on formal definitions of visibility, occupancy, emptiness, and photo-consistency, the theoretical development yields a formulation of the photo hull distribution, the tightest probabilistic bound on the scene''s true shape that can be inferred from the photos. we show how to (1) express this distribution in terms of image measurements, (2) represent it compactly by assigning an occupancy probability to each point in space, and (3) design a stochastic reconstruction algorithm that draws fair samples (i.e., 3d photo hulls) from it. we also show experimental results on two complex scenes.
eigentracking: robust matching and tracking of articulated objects using a view-based representation. this paper describes an approach for tracking rigid and articulated objects using a view-based representation. the approach builds on and extends work on eigenspace representations, robust estimation techniques, and parameterized optical flow estimation. first, we note that the least-squares image reconstruction of standard eigenspace techniques has a number of problems and we reformulate the reconstruction problem as one of robust estimation. second we define a &ldquo;subspace constancy assumption&rdquo; that allows us to exploit techniques for parameterized optical flow estimation to simultaneously solve for the view of an object and the affine transformation between the eigenspace and the image. to account for large affine transformations between the eigenspace and the image we define a multi-scale eigenspace representation and a coarse-to-fine matching strategy. finally, we use these techniques to track objects over long image sequences in which the objects simultaneously undergo both affine image motions and changes of view. in particular we use this &ldquo;eigentracking&rdquo; technique to track and recognize the gestures of a moving hand.
directions of motion fields are hardly ever ambiguous. if instead of the full motion field, we consider only the direction of the motion field due to a rigid motion, what can we say about the three-dimensional motion information contained in it? this paper provides a geometric analysis of this question based solely on the constraint that the depth of the surfaces in view is positive. the motivation behind this analysis is to provide a theoretical foundation for image constraints employing only the sign of flow in various directions and justify their utilization for addressing 3d dynamic vision problems.it is shown that, considering as the imaging surface the whole sphere, independently of the scene in view, two different rigid motions cannot give rise to the same directional motion field. if we restrict the image to half of a sphere (or an infinitely large image plane) two different rigid motions with instantaneous translational and rotational velocities(t<math>_1 &omega;_1) and(t<math>_2 ,&omega;_2) cannot give rise to the same directional motion field unless the plane through t_1 and t_2 is perpendicular to the plane through &omega;_1 and &omega;_2 (i.e., (t_1 &times; t_2) &middot; (&omega;_1 &times; &omega;_2) &equals; 0. in addition, in order to give practical significance to these uniqueness results for the case of a limited field of view, we also characterize the locations on the image where the motion vectors due to the different motions must have different directions.if (&omega;_1 &times; &omega;_2) &middot; (t_1 &times; t_2) &equals; 0 and certain additional constraints are met, then the two rigid motions could produce motion fields with the same direction. for this to happen the depth of each corresponding surface has to be within a certain range, defined by a second and a third order surface. similar more restrictive constraints are obtained for the case of multiple motions. consequently, directions of motion fields are hardly ever ambiguous. a byproduct of the analysis is that full motion fields are never ambiguous with a half sphere as the imaging surface.
active appearance models. we describe a new method of matching statistical models of appearance to images. a set of model parameters control modes of shape and gray-level variation learned from a training set. we construct an efficient iterative matching algorithm by learning the relationship between perturbations in the model parameters and the induced image errors.
analytical results on error sensitivity of motion estimation from two views. fundamental instabilities have been observed in the performance of the majority of the algorithms for three dimensional motion estimation from two views. many geometric and intuitive interpretations have been offered to explain the error sensitivity of the estimated parameters. in this contribution, the importance of the form of the error norm to be minimized with respect to the motion parameters is addressed. the error norms used by the existing algorithms are described in a unifying notation, and a geometric interpretation of them is given. then, for the continuous case of pure translational motion it is proved that the minimization of the objective function leading to an eigenvector solution suffers from a crucial instability. the analyticity of the results allows the examination of error sensitivity in terms of the translation direction, the viewing angle and the distance of the moving object from the camera. a norm possessing a reasonable geometric interpretation in the image plane is proposed to eliminate the effects of the instability mentioned above. due to the high nonlinearity of this norm it has not been possible to prove explicitly its stabilizing role. it is shown by analytical means that a simplification of this norm - leading to a closed form solution - has undesirable properties.
softposit: simultaneous pose and correspondence determination. the problem of pose estimation arises in many areas of computer vision, including object recognition, object tracking, site inspection and updating, and autonomous navigation when scene models are available. we present a new algorithm, called softposit, for determining the pose of a 3d object from a single 2d image when correspondences between object points and image points are not known. the algorithm combines the iterative softassign algorithm (gold and rangarajan, 1996; gold et al., 1998) for computing correspondences and the iterative posit algorithm (dementhon and davis, 1995) for computing object pose under a full-perspective camera model. our algorithm, unlike most previous algorithms for pose determination, does not have to hypothesize small sets of matches and then verify the remaining image points. instead, all possible matches are treated identically throughout the search for an optimal pose. the performance of the algorithm is extensively evaluated in monte carlo simulations on synthetic data under a variety of levels of clutter, occlusion, and image noise. these tests show that the algorithm performs well in a variety of difficult scenarios, and empirical evidence suggests that the algorithm has an asymptotic run-time complexity that is better than previous methods by a factor of the number of image points. the algorithm is being applied to a number of practical autonomous vehicle navigation problems including the registration of 3d architectural models of a city to images, and the docking of small robots onto larger robots.
tracking line segments. the development and implementation of a line segment-based token tracker is described. given a sequence of time-varying images, the goal is to track line segments corresponding to the edges extracted from the image being analysed. two representations for the line segments are presented and discussed. an uncertainty analysis on the parameters of each representation is performed, and the appropriate representation for the tracking problem is then derived. a tracking approach is presented that combines prediction and matching steps. the prediction step is a kalman filtering-based approach that is used to provide reasonable estimates of the region where the matching process has to seek for a possible match between tokens. correspondence in the search area is done through the use of a similarity function based on mahalanobis distance between carefully chosen attributes of the line segments. it is worthwhile to note that tokens as points of interest (corners, triple points) can also be considered without affecting deeply the algorithm. the efficiency of the proposed approach is illustrated in several experiments that have been carried out considering noisy synthetic data and real scenes obtained from the inria mobile robot.
region matching with missing parts. we present a variational approach to the problem of registering planar shapes despite missing parts. registration is achieved through the evolution of a partial differential equation that simultaneously estimates the shape of the missing region, the underlying 'complete shape' and the collection of group elements (euclidean, affine) corresponding to the registration. our technique can be used both for shapes, for instance represented as characteristic functions (binary images) and for grayscale images where it can be interpreted as region 'inpainting.' the novelty of the approach lies on the fact that, rather than estimating the missing region in each image independently, we pose the problem as a joint registration with respect to an underlying 'complete shape' from which the complete version of the original data is obtained via a group action. we simultaneously estimate the complete shape and the group action in an alternating minimization scheme.
local scale control for edge detection and blur estimation. the standard approach to edge detection is based on a model of edges as large step changes in intensity. this approach fails to reliably detect and localize edges in natural images where blur scale and contrast can vary over a broad range. the main problem is that the appropriate spatial scale for local estimation depends upon the local structure of the edge, and thus varies unpredictably over the image. here we show that knowledge of sensor properties and operator norms can be exploited to define a unique, locally computable minimum reliable scale for local estimation at each point in the image. this method for local scale control is applied to the problem of detecting and localizing edges in images with shallow depth of field and shadows. we show that edges spanning a broad range of blur scales and contrasts can be recovered accurately by a single system with no input parameters other than the second moment of the sensor noise. a natural dividend of this approach is a measure of the thickness of contours which can be used to estimate focal and penumbral blur. local scale control is shown to be important for the estimation of blur in complex images, where the potential for interference between nearby edges of very different blur scale requires that estimates be made at the minimum reliable scale.
self-calibration of a 1d projective camera and its application to the self-calibration of a 2d projective camera. we introduce the concept of self-calibration of a 1d projective camera from point correspondences, and describe a method for uniquely determining the two internal parameters of a 1d camera, based on the trifocal tensor of three 1d images. the method requires the estimation of the trifocal tensor which can be achieved linearly with no approximation unlike the trifocal tensor of 2d images and solving for the roots of a cubic polynomial in one variable. interestingly enough, we prove that a 2d camera undergoing planar motion reduces to a 1d camera. from this observation, we deduce a new method for self-calibrating a 2d camera using planar motions. both the self-calibration method for a 1d camera and its applications for 2d camera calibration are demonstrated on real image sequences.
carved visual hulls for image-based modeling. this article presents a novel method for acquiring high-quality solid models of complex 3d shapes from multiple calibrated photographs. after the purely geometric constraints associated with the silhouettes found in each image have been used to construct a coarse surface approximation in the form of a visual hull, photoconsistency constraints are enforced in three consecutive steps: (1) the rims where the surface grazes the visual hull are first identified through dynamic programming; (2) with the rims now fixed, the visual hull is carved using graph cuts to globally optimize the photoconsistency of the surface and recover its main features; (3) an iterative (local) refinement step is finally used to recover fine surface details. the proposed approach has been implemented, and experiments with seven real data sets are presented, along with qualitative and quantitative comparisons with several state-of-the-art image-based-modeling algorithms.
parallel and deterministic algorithms from mrfs: surface reconstruction and integration. in recent years many researchers have investigated the use of markov random fields (mrfs) for computer vision. the computational complexity of the implementation has been a drawback of mrfs. in this paper we derive deterministic approximations to mrfs models. all the theoretical results are obtained in the framework of the mean field theory from statistical mechanics. because we use mrfs models the mean field equations lead to parallel and iterative algorithms. one of the considered models for image reconstruction is shown to give in a natural way the graduate non-convexity algorithm proposed by blake and zisserman.
epipolar fields on surfaces. the view lines associated with a family of profile curves of the projection of a surface onto the retina of a moving camera defines a multi-valued vector field on the surface. the integral curves of this field are called epipolar curves and together with a parametrization of the profiles provide a parametrization of regions of the surface. in addition, one has the epipolar constraints which define curves in the images. these image curves are related to the epipolar curves on the surface but not by a simple projection. we present an investigation of epipolar curves on the object surface, in the spatio-temporal surface and the traces in the images. we address the question of when there is an epipolar parametrization. we have obtained detailed results which depend on a classification \cite{davydov} of vector fields on surfaces with boundary. these results give a systematic way of detecting the gaps left by reconstruction of a surface from profiles. they also suggest methods for filling in these gaps. this work was supported by nato grant crg 910221. in addition, the second author would like to acknowledge the support of darpa and tacom under contract daae07-91-c-r035 and nsf under grants iri-920892 and iri-9116297.