waashk/dblp huggingface dataset:
efficient learning of relational object class models. we present an efficient method for learning part-based object class models. the models include location and scale relations between parts, as well as part appearance. models are learnt from raw object and background images, represented as an unordered set of features extracted using an interest point detector. the object class is generatively modeled using a simple bayesian network with a central hidden node containing location and scale information, and nodes describing object parts. the model¿s parameters, however, are optimized to reduce a loss function which reflects training error, as in discriminative methods. specifically, the optimization is done using a boosting-like technique with complexity linear in the number of parts and the number of features per image. this efficiency allows our method to learn relational models with many parts and features, and leads to improved results when compared with other methods. extensive experimental results are described, using some common bench-mark datasets and three sets of newly collected data, showing the relative advantage of our method.
nonlinear manifold learning for visual speech recognition. a technique for representing and learning smooth nonlinear manifolds is presented and applied to several lip reading tasks. given a set of points drawn from a smooth manifold in an abstract feature space, the technique is capable of determining the structure of the surface and of finding the closest manifold point to a given query point. we use this technique to learn the "space of lips" in a visual speech recognition task. the learned manifold is used for tracking and extracting the lips, for interpolating between frames in an image sequence and for providing features for recognition. we describe a system based on hidden markov models and this learned lip manifold that significantly improves the performance of acoustic speech recognizers in degraded environments. we also present preliminary results on a purely visual lip reader.
graph partition by swendsen-wang cuts. vision tasks, such as segmentation, grouping, recognition, can be formulated as graph partition problems. the recent literature witnessed two popular graph cut algorithms: the ncut using spectral graph analysis and the minimum-cut using the maximum flow algorithm. this paper presents a third major approach by generalizing the swendsen-wang method- a well celebrated algorithm in statistical mechanics. our algorithm simulates ergodic, reversible markov chain jumps in the space of graph partitions to sample a posterior probability. at each step, the algorithm splits, merges, or re-groups a sizable subgraph, and achieves fast mixing at low temperature enabling a fast annealing procedure. experiments show it converges in 2-30seconds in a pc for image segmentation. this is 400 times faster than the single-site update gibbs sampler, and 20-40 times faster than the ddmcmc algorithm. the algorithm can optimize over the number of models and works for general forms of posterior probabilities, so it is more general than the existing graph cut approaches.
integrated spatial and frequency domain 2d motion segmentation and estimation. a video containing multiple objects in rotational and translational motion is analyzed through a combination of spatial and frequency domain representations. it is argued that the combined analysis can take advantage of the strengths of both representations. initial estimates of constant, as well as time-varying, translation and rotation velocities are obtained from frequency analysis. improved motion estimates and motion segmentation for the case of translation are achieved by integrating spatial and fourier domain information. for combined rotational and translational motions, the frequency representation is used for motion estimation, but only spatial information can be used to separate and extract the independently moving objects. the proposed algorithms are tested on synthetic and real videos.
incorporating visual knowledge representation in stereo reconstruction. in this paper, we present a two-layer generative model that incorporates generic middle-level visual knowledge for dense stereo reconstruction. the visual knowledge is represented by a dictionary of surface primitives including various categories of boundary discontinuities and junctions in parametric form. given a stereo pair, we first compute a primal sketch representation which decomposes the image into a structural part for object boundaries and high intensity contrast represented by a 2d sketch graph, and a structureless part represented by markov random field on pixels. then we label the sketch graph and compute the 3d sketch (like a wire-frame) by fitting the primitive dictionary to the sketch graph. the surfaces between the 3d sketches are filled in by computing the depth of the mrf model on the structureless part. these two levels interact closely since the mrf is used to propagate information between the primitives, and at the same time, the primitives act as boundary conditions for the mrf. the two processes maximize a bayesian posterior probability jointly. we propose anmcmc algorithm that simultaneously infers the 3d primitive types and parameters and estimates the depth of the scene. our experiments show that this representation can infer the depth map with sharp boundaries and junctions for textureless images, curve objects and free-form shapes.
self-calibration from image derivatives. this study investigates the problem of estimating the calibration parameters from image motion fields induced by a rigidly moving camera with unknown calibration parameters, where the image formation is modeled with a linear pinhole-camera model. the equations obtained show the flow to be clearly separated into a component due to the rotation and the calibration parameters. a set of parameters encoding the latter component are linearly related to the flow, and from these parameters the calibration can be determined. however, as for discrete motion, in the general case it is not possible, to decouple image measurements from two frames only into their translational and rotational component.geometrically, the ambiguity takes the form of a part of the rotational component being parallel to the translational component, and thus the scene can be reconstructed only up to a projective transformation. in general, for a full calibration at least four successiveimage frames are necessary with the 3d-rotation changing between the measurements.the geometric analysis gives rise to a direct self-calibration method that avoids computation of optical flow or point correspondences and uses only normal flow measurements. in this technique the direction of translation is estimated employing in a novel way smoothness constraints. then the calibration parameters are estimated from the rotational components of several flow fields using levenberg-marquardt parameter estimation, iterative in the calibration parameters only. the technique proposed does not require calibration objects in the scene or special camera motions and it also avoids the computation of exact correspondence. this makes it suitable for the calibration of active vision systems which have to acquire knowledge about their intrinsic parameters while they perform other tasks, or as a tool for analyzing image sequences in large video databases.
regular polygon detection. this paper describes a new robust regular polygon detector. the regular polygon transform is posed as a mixture of regular polygons in a five dimensional space. given the edge structure of an image, we derive the a posteriori probability for a mixture of regular polygons, and thus the probability density function for the appearance of a mixture of regular polygons. likely regular polygons can be isolated quickly by discretising and collapsing the search space into three dimensions. the remaining dimensions may be efficiently recovered subsequently using maximum likelihood at the locations of the most likely polygons in the subspace. this leads to an efficient algorithm. also the a posteriori formulation facilitates inclusion of additional a priori information leading to real-time application to road sign detection. the use of gradient information also reduces noise compared to existing approaches such as the generalised hough transform. results are presented for images with noise to show stability. the detector is also applied to two separate applications: real-time road sign detection for on-line driver assistance; and feature detection, recovering stable features in rectilinear environments.
paracatadioptric camera calibration using lines. paracatadioptric sensors combine a parabolic shaped mirrorand a camera inducing an orthographic projection.such a configuration provides a wide field of view whilekeeping a single effective viewpoint. previous work in centralcatadioptric sensors proved that a line projects into aconic curve and that three line images are enough to calibratethe system. however the estimation of the coniccurves where lines are mapped is hard to accomplish. ingeneral only a small arc of the conic is visible in the imageand conventional conic fitting techniques are unable to correctlyestimate the curve. the present work shows that a setof conic curves corresponds to paracatadioptric line imagesif, and only if, certain properties are verified. these propertiesare used to constraint the search space and correctlyestimate the curves. the accurate estimation of a minimumof three line images allows the complete calibration of theparacatadioptric camera. if the camera is skewless and theaspect ratio is known then the conic fitting problem is solvednaturally by an eigensystem. for the general situation theconic curves are estimated using non-linear optimization.
fundamental matrix for cameras with radial distortion. when deploying a heterogeneous camera network or when we use cheap zoom cameras like in cell-phones, it is not practical, if not impossible to off-line calibrate the radial distortion of each camera using reference objects. it is rather desirable to have an automatic procedure without strong assumptions about the scene. in this paper, we present a new algorithm for estimating the epipolar geometry of two views where the two views can be radially distorted with different distortion factors. it is the first algorithm in the literature solving the case of different distortion in the left and right view linearly and without assuming the existence of lines in the scene. points in the projective plane are lifted to a quadric in three-dimensional projective space. a radial distortion of the projective plane results to a matrix transformation in the space of lifted coordinates. the new epipolar constraint depends linearly on a 4x4 radial fundamental matrix which has 9 degrees of freedom. a complete algorithm is presented and tested on real imagery.
recognising panoramas. the problem considered in this paper is the fully asutomaticconstruction of panoramas.fundamentally, thisproblem requires recognition, as we need to know whichparts of the panorama join up.previous approaches haveused human input or restrictions on the image sequencefor the matching step.in this work we use object recognitiontechniques based on invariant local features to selectmatchings images, and a probabilistic model for verification.because of this our method is insensitive to the ordering, orientation, scale and illumination of the images.it is also insensitive to 'noise' images which are not partof the panorama at all, that is, it recognises panoramas.this suggests a useful application for photographers: thesystem takes as input the images on an entire flash card orfilm, recognises images that form part of a panorama, andstitches them with no user input whatsoever.
towards gauge invariant bundle adjustment: a solution based on gauge dependent damping. bundle ajustment is used to obtain accurate visual reconstructionsby minimizing the reprojection error. the coordinateframe ambiguity, or more generality the gauge freedoms,has been dealt with in different manners. it has oftenbeen reported that standard bundle adjustment algorithmswere not gauge invariant: two iterations within differentgauges can lead to geometrically very different results. surprisingly,most algorithms do not exploit gauge freedoms toimprove performances. we consider this issue. we analyzetheoretically the impact of the gauge on standard algorithms.we show that a sufficiently general damping matrixin levenberg-marquardt iteration can be used to implicitlyreproduce a gauge transformation. we show that if thedamping matrix is chosen such that the decrease in the reprojectionerror is maximized, then the iteration is gauge invariant.experimental results on simulated and real data showthat our gauge invariant bundle adjustment algorithm outperformsexisting ones in terms of stability.
optimum fiducials under weak perspective projection. we investigate how a given fixed number of points should be located in space so that the pose of a camera viewing them from unknown locations can be estimated with the greatest accuracy. we show that optimum solutions are obtained when the points form concentric complete regular polyhedra. for the case of optimal configurations we provide a worst-case error analysis and use it to analyze the effects of weak perspective approximation to true perspective viewing. comprehensive computer simulations validate the theoretical results.
multiple-view structure and motion from line correspondences. we address the problem of camera motion and structure reconstruction from line correspondences across multiple views, from initialization to final bundle adjustment. one of the main difficulties when dealing with line features is their algebraic representation. first, we consider the triangulation problem. based on pl&uuml;cker coordinates to represent the lines, we propose a maximum likelihood algorithm, relying on linearising the pl&uuml;cker constraint, and on a pl&uuml;cker correction procedure to compute the closest pl&uuml;cker coordinates to a given 6-vector. second, we consider the bundle adjustment problem. previous overparameterizations of 3d lines induce gauge freedoms and/or internal consistency constraints. we propose the orthonormal representation, which allows handy non-linear optimization of 3d lines using the minimum 4 parameters, within an unconstrained non-linear optimizer. we compare our algorithms to existing ones on simulated and real data.
towards ultimate motion estimation: combining highest accuracy with real-time performance. although variational methods are among the most accurate techniques for estimating the optical flow, they have not yet entered the field of real-time vision. main reason is the great popularity of standard numerical schemes that are easy to implement, however, at the expense of being too slow for real-time performance. in our paper we address this problem in two ways: (i) we present an improved version of the highly accurate technique of brox et al. [9]. thereby we show that a separate robustification of the constancy assumptions is very useful, in particular if the 1-norm is used as penalizer. as a result, a method is obtained that yields the lowest angular errors in the literature. (ii) we develop an efficient numerical scheme for the proposed approach that allows real-time performance for sequences of size 160 × 120. to this end, we combine two hierarchical strategies: a coarse-to-fine warping strategy as implementation of a fixed point iteration for a non-convex optimisation problem and a nonlinear full multigrid method ¿ a so called full approximation scheme (fas) ¿ for solving the highly nonlinear equation systems at each warping level. in the experimental section the advantage of the proposed approach becomes obvious: outperforming standard numerical schemes by two orders of magnitude frame rates of six high quality flow fields per second are obtained on a 3.06 ghz pentium4 pc.
model selection and surface merging in reconstruction algorithms. the problem of model selection is relevant to many areas of computer vision. model selection criteria have been used in the vision literature and many more have been proposed in statistics, but the relative strengths of these criteria have not been analyzed in vision. more importantly, suitable extensions to these criteria must be made to solve problems unique to computer vision. using the problem of surface reconstruction as our context, we analyze existing criteria using simulations and sensor data, introduce new criteria from statistics, develop novel criteria capable of handling unknown error distributions and outliers, and extendmodel selection criteria to apply to the surface merging problem. the new surface merging rules improve upon previous results, and work well even at small step heights (h = 3¿) and crease discontinuities. our results show that a bayesian criteria and its bootstrapped variant perform the best, although for time-sensitive applications, a variant of the akaike criterion may be a better choice. unfortunately, none of the criteria work reliably for small region sizes, implying that model selection and surface merging should be avoided unless the region size is sufficiently large.
separability of pose and expression in facial tracing and animation. we explore the application of facial tracking to automated re-animation. to this end, it is necessary to recover both head-pose and facial expression from the facial movement of a performer. however, both effects are coupled. this is a serious problem, which previous studies haven't fully considered. the solution to this interaction problem proposed here is to solve explicitly, at each timestep, for pose and expression variables. in principle this is a nonlinear inverse problem. however, appropriate parameterisation of pose in terms of affine transformations with parallax, and of expression in terms of key-frames, reduces the problem to a bilinear one. this can then be solved directly by singular value decomposition. thus actor-driven animation has b een implemented in real-time, at video field-rate, using two indy desktop workstations.
color recognition in outdoor images. the color associated with an object in machine vision images is not constant; under varying illuminating and viewing conditions (such as in outdoor images), the perceived color of an object can vary significantly, thus making color-based recognition difficult. existing methods in color-based recognition have been applied mostly to indoor and/or constrained imagery, but not to realistic outdoor data.this work analyzes the variation of object color in outdoor images with respect to existing models of day-light illumination and surface reflectance. two approaches for color recognition are then proposed: the first develops context-based models of daylight illumination and hybrid surface reflectance, and predicts the color of objects based on scene context. the secondmethod shows that object color can be nonparametrically "learned" through classification methods such as neural networks and multivariate decision trees. the methods have been successfully tested in domains such as road/highway scenes, off-road navigation and military target detection.
region tracking through image sequences. the paper describes an approach to the tracking of complex shapes through image sequences, that combines deformable region models and deformable contours. a deformable region model is presented: its optimisation is based on texture correlation and is constrained by the use of a motion model, such as rigid, affine or homographic. the use of texture information (versus edge information) noticeably improves the tracking performances of deformable models in the presence of texture. then the region contour is refined using an edge based deformable model in order to better deal with specularities, non planar objects and occlusions. the method is illustrated and validated by experimental results on real images.
electronically directed "focal" stereo. a key to developing computationally efficient stereo vision is the incorporation of intelligent control. stereo is most effective when it is able to "focus" its analysis on regions and details of a scene that are important to the task at hand, while avoiding less important regions and unnecessary detail. the paper describes two methods for electronically "focusing" stereo measurement through simple image pre-processing. the first allows measurement sensitivity to be adjusted. the second allows the shape of the 3d region in which measurements are gathered to be matched to the shape of surfaces in the scene.
lambertian reflectance and linear subspaces. we prove that the set of all reflectance functions (the mapping from surface normals to intensities) produced by lambertian objects under distant, isotropic lighting lies close to a 9d linear subspace. this implies that, in general, the set of images of a convex lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace, explaining prior empirical results. we also provide a simple analytic characterization of this linear space. we obtain these results by representing lighting using spherical harmonics and describing the effects of lambertian materials as the analog of a convolution. these results allow us to construct algorithms for object recognition based on linear methods as well as algorithms that use convex optimization to enforce non-negative lighting functions. finally, we show a simple way to enforce non-negative lighting when the images of an object lie near a 4d linear space.
automatic tracking of human motion in indoor scenes across multiple synchronized video streams. this pap er presents a comprehensive framework for tracking moving humans in an indoor environment from sequences of synchronized monocular grayscale images captured from multiple fixed cameras. the proposed framework consists of three main modules: single view tracking (svt), multiple view transition tracking (mvtt), and automatic camera switching (acs). bayesian classification schemes based on motion analysis of human features are used to track (spatially and temporally) a subject image of interest between consecutive frames. the automatic camera switching module predicts the position of the subject along a spatial-temporal domain, and then selects the camera which provides the best view and requires the least switching to continue tracking. limited degrees of occlusion are tolerated within the system. tracking is based up on the images of upper human bodies captured from various viewing angles, and non-human moving objects are excluded using principal component analysis (pca). experimental results are presented to evaluate the performance of the tracking system.