waashk/dblp huggingface dataset:
universal mosaicing using pipe projection. video mosaicing is commonly used to increase the visual field of view by pasting together many video frames. existing mosaicing methods are effective only in very limited cases where the image motion is almost a uniform translation or the camera performs a pure pan. forward camera motion or camera zoom are very problematic for traditional mosaicing.a mosaicing methodology to allow image mosaicing in the most general cases is presented, where frames in the video sequence are transformed such that the optical flow becomes parallel. this transformation is an oblique projection of the image into a "viewing pipe" whose central axis is the trajectory of the camera.the "pipe projection" enables to define high-quality mosaicing even for the most challenging cases of forward motion and of zoom. in addition, view interpolation, generating dense intermediate views, is used to overcome parallax effects.
view-based object matching. we introduce a novel view-based object representation, called the saliency map graph (smg), which captures the salient regions of an object view at multiple scales using a wavelet transform. this compact representation is highly invariant to translation, rotation (image and depth), and scaling, and offers the locality of representation required for occluded object recognition. to compare two saliency map graphs, we introduce two graph similarity algorithms. the first computes the topological similarity between two smg's, providing a coarse-level matching of two graphs. the second computes the geometrical similarity betweentwo smg's, providing a fine-level matching of two graphs. we test and compare these two algorithms on a large database of model object views.
a maximum-flow formulation of the n-camera stereo correspondence problem. this paper describes a new algorithm for solving the n-camera stereo correspondence problem by transforming it into a maximum-flow problem. once solved, the minimum-cut associated to the maximum-flow yields a disparity surface for the whole image at once. this global approach to stereo analysis provides a more accurate and coherent depth map than the traditional line-by-line stereo. moreover, the optimality of the depth surface is guaranteed and can be shown to be a generalization of the dynamic programming approach that is widely used in standard stereo. results show improved depth estimation as well as better handling of depth discontinuities. while the worst case running time is o(n2d2 log(nd)), the observed average running time is o(n1.2d1.3) for an image size of n pixels and depth resolution d.
contour-based learning for object detection. we present a novel categorical object detection scheme that uses only local contour-based features. a two-stage, partially supervised learning architecture is proposed: a rudimentary detector is learned from a very small set of segmented images and applied to a larger training set of un-segmented images; the second stage bootstraps these detections to learn an improved classifier while explicitly training against clutter. the detectors are learned with a boosting algorithm which creates a location-sensitive classifier using a discriminative set of features from a randomly chosen dictionary of contour fragments. we present results that are very competitive with other state-of-the-art object detection schemes and show robustness to object articulations, clutter, and occlusion. our major contributions are the application of boosted local contour-based features for object detection in a partially supervised learning framework, and an efficient new boosting procedure for simultaneously selecting features and estimating per-feature parameters.
an integral approach to free-formed object modeling. presents a new approach to free-formed object modeling from multiple range images. in most conventional approaches, successive views are registered sequentially. in contrast to the sequential approaches, we propose an integral approach which reconstructs statistically optimal object models by simultaneously aggregating all data from multiple views into a weighted least-squares (wls) formulation. the integral approach has two components. first, a global resampling algorithm constructs partial representations of the object from individual views so that correspondences can be established among different views. the global resampling algorithm is based on the spherical attribute image (sai) previously introduced in the context of object representation and recognition. second, a weighted least-squares algorithm integrates resampled partial representations of multiple views, using the technique of principal component analysis with missing data (pcamd). experiments using real range images show that our approach is robust against noise and mismatches, and generates accurate object models.
texture-based image retrieval without segmentation. image segmentation is not only hard and unnecessary for texture-based image retrieval, but can even be harmful. images of either individual or multiple textures are best described by distributions of spatial frequency descriptors, rather than single descriptor vectors over pre-segmented regions.a retrieval method based on the earth movers distance with an appropriate ground distance is shown to handle both complete and partial multi-textured queries. as an illustration, different images of the same type of animal are easily retrieved together. at the same time, animals with subtly different coats, like cheetahs and leopards, are properly distinguished.
a metric for distributions with applications to image databases. we introduce a new distance between two distributions that we call the earth mover's distance (emd), which reflects the minimal amount of work that must be performed to transform one distribution into the other by moving "distribution mass" around. this is a special case of the transportation problem from linear optimization, for which efficient algorithms are available. the emd also allows for partial matching. when used to compare distributions thathave the same overall mass, the emd is a true metric, and has easy-to-compute lower bounds. in this paper we focus on applications to image databases, especially color and texture. we use the emd to exhibit the structure of color-distribution and texture spaces by means of multi-dimensional scaling displays. we also propose a novel approach to the problem of navigating through a collection of color images, which leads to a new paradigm for image database search.
construction and refinement of panoramic mosaics with global and local alignment. this paper presents techniques for constructing full view panoramic mosaics from sequences of images. our representation associates a rotation matrix (and optionally a focal length) with each input image, rather than explicitly projecting all of the images onto a common surface (e.g., a cylinder). in order to reduce accumulated registration errors, we apply global alignment (block adjustment) to the whole sequence of images, which results in an optimal image mosaic (in the least-squares sense). to compensate for small amounts of motion parallax introduced by translations of the camera and other unmodeled distortions, we develop a local alignment (deghosting) technique which warps each image based on the results of pairwise local image registrations. by combining both global and local alignment, we significantly improve the quality of our image mosaics, thereby enabling the creation of full view panoramic mosaics with hand-held cameras.
locating objects using the hausdorff distance. the hausdorff distance is a measure defined between two point sets representing a model and an image. in the past, it has been used to search images for instances of a model that has been translated or translated and scaled by finding transformations that bring a large number of model features close to image features, and vice versa. the hausdorff distance is reliable even when the image contains multiple objects, noise, spurious features, and occlusions. we apply it to the task of locating an affine transformation of a model in an image; this corresponds to determining the pose of a planar object that has undergone weak perspective projection. we develop a rasterised approach to the search and a number of techniques that allow us to quickly locate all transformations of the model that satisfy two quality criteria; we can also quickly locate only the best transformation. we discuss an implementation of this approach, and present some examples of its use.
rigid and articulated motion seen with an uncalibrated stereo rig. this paper establishes a link between uncalibrated stereo vision and the motion of rigid and articulated bodies. the variation in the projective reconstruction of a dynamic scene over time allows an uncalibrated stereo rig to be used as a faithful motion capturing device.we introduce an original theoretical framework - projective kinematics - which allows rigid and articulated motion to be represented within the transformation group of projective space. corresponding projective velocities are defined in the tangent space. most importantly, these projective motions inherit the lie-group structure of the displacement group.these theoretical results lead immediately to non-metric formulations of visual serving, tracking, motion capturing and motion synthesis systems, that no longer require the metric geometry of a stereo camera or of the articulated body to be known. we report on such a non-metric formulation of a visual serving system and present simulated experimental results.
fast texture-based tracking and delineation using texture entropy. we propose a fast texture-segmentation approach to the problem of 2-d and 3-d model-based contour tracking, which is suitable for real-time or interactive applications. our approach relies on detecting texture boundaries in the direction normal to the contour boundaries and on using a hidden markov model to link these boundary points in the other direction. the probabilities that appear in this computation closely relate to texture entropy and kullback-leibler divergence, a property we use to compute and update dynamic texture models. we demonstrate results both in the context of interactive 2-d delineation and fast 3-d tracking.
multiperspective projection and collineation. we present theories of multiperspective projection and collineation. given an arbitrary multiperspective imaging system that captures smoothly varying set of rays, we show how to map the rays onto a 2d ray manifold embedded in a 4d linear vector space. the characteristics of this imaging system, such as its projection, collineation, and image distortions can be analyzed by studying the 2-d tangent planes of this ray manifold. these tangent planes correspond to the recently proposed general linear camera (glc) model. in this paper, we study the imaging process of the glcs. we show the glc imaging process can be broken down into two separate stages: the mapping of 3d geometry to rays and the sampling of those rays over an image plane. we derive a closed-form solution to projecting 3d points in a scene to rays in a glc. a glc image is created by sampling these rays over an image plane. we develop a notion of glc collineation analogous to pinhole cameras. glc collineation describes the transformation between the images of a single glc due to changes in sampling and image plane selection. we show that general glc collineations can be characterized by a quartic (4th order) rational function. glc projection and collineation provides a basis for developing new computer vision algorithms suitable for analyzing a wider range of imaging systems than current methods, based on simple pinhole projection models, permit.
a new paradigm for recognizing 3-d object shapes from range data. most of the work on 3-d object recognition from rangedata has used an alignment-verification approach in whicha specific 3-d object is matched to an exact instance of thesame object in a scene. this approach has been successfullyused in industrial machine vision, but it is not capable ofdealing with the complexities of recognizing classes of similarobjects. this paper undertakes this task by proposingand testing a component-based methodology encompassingthree main ingredients: 1) a new way of learning and extractingshape-class components from surface shape information;2) a new shape representation called a symbolicsurface signature that summarizes the geometric relationshipsamong components; and 3) an abstract representationof shape classes formed by a hierarchy of classifiersthat learn object-class parts and their spatial relationshipsfrom examples.
the hamilton-jacobi skeleton. the eikonal equation and variants of it are of significant interest for problems in computer vision and image processing. it is the basis for continuous versions of mathematical morphology, stereo, shape-from-shading and for recent dynamic theories of shape. its numerical simulation can be delicate, owing to the formation of singularities in the evolving front, and is typically based on level set methods. however, there are more classical approaches rooted in hamiltonian physics, which have received little consideration in computer vision.in this paper we first introduce a new algorithm for simulating the eikonal equation, which offers a number of computational and conceptual advantages over the earlier methods when it comes to shock tracking. next, we introduce a very efficient algorithm for shock detection, where the key idea is to measure the net outward flux of a vector field per unit volume, and to detect locations where a conservation of energy principle is violated. we illustrate the approach with several numerical examples including skeletons of complex 2d and 3d shapes.
fast intensity-based 2d-3d image registration of clinical data using light fields. registration of a preoperative ct (3d) image to one or more x-ray projection (2d) images, a special case of the pose estimation problem, has been attempted in a variety of ways with varying degrees of success. recently, there has been a great deal of interest in intensity-based methods. one of the drawbacks to such methods is the need to create digitally reconstructed radiographs (drrs) at each step of the optimization process. drrs are typically generated by ray casting, an operation that requires o(n3) time, where we assume that n is approximately the size (in voxels) of one side of the drr as well as one side of the ct volume. we address this issue by extending light field rendering techniques from the computer graphics community to generate drrs instead of conventional rendered images. using light fields allows most of the computation to be performed in a preprocessing step; after this precomputation, very accurate drrs can be generated in o(n2) time. another important issue for 2d-3d registration algorithms is validation. previously reported 2d-3d registration algorithms were validated using synthetic data or phantoms but not clinical data. we present an intensity-based 2d-3d registration system that generates drrs using light fields; we validate its performance using clinical data with a known gold standard transformation.
shock graphs and shape matching. we have been developing a theory for the generic representation of 2-d shape, where structural descriptions are derived from the shocks "singularities" of a curve evolution process, acting on bounding contours. we now apply the theory to the problem of shape matching. the shocks are organized into a directed, acyclic shock graph, and complexity is managed by attending to the most significant "central" shape components first. the space of all such graphs is highly structured and can be characterized by the rules of a shock graph grammar. the grammar permits a reduction of a shock graph to a unique rooted shock tree. we introduce a novel tree matching algorithm which finds the best set of corresponding nodes between two shock trees in polynomial time. using a diverse database of shapes, we demonstrate our system's performance under articulation, occlusion, and changes in viewpoint.
corner detection in textured color images. corner models in the literature have lagged behind edge models with respect to color and shading. we use both a region model, based on distributions of pixel colors, and an edge model, which removes false positives, to perform corner detection on color images whose regions contain texture. we show results on a variety of natural images at different scales that highlight the problems that occur when boundaries between regions have curvature.
hyperbolic "smoothing" of shapes. we have been developing a theory of generic 2-d shape based on a reaction-diffusion model from mathematical physics. the description of a shape is derived from the singularities of a curve evolution process driven by the reaction (hyperbolic) term. the diffusion (parabolic) term is related to smoothing and shape simplification. however, the unification of the two is problematic, because the slightest amount of diffusion dominates and prevents the formation of generic first-order shocks. the technical issue is whether it is possible to smooth a shape, in any sense, without destroying the shocks. we now report a constructive solution to this problem, by embedding the smoothing term in a global metric against which a purely hyperbolic evolution is performed from the initial curve. this is a new flow for shape, that extends the advantages of the original one. specific metrics are developed, which lead to a natural hierarchy of shape features, analogous to the simplification one might perceive when viewing an object from increasing distances. we illustrate our new flow with a variety of examples.
visual speech recognition with loosely synchronized feature streams. we present an approach to detecting and recognizing spoken isolated phrases based solely on visual input. we adopt an architecture that first employs discriminative detection of visual speech and articulatory features, and then performs recognition using a model that accounts for the loose synchronization of the feature streams. discriminative classifiers detect the subclass of lip appearance corresponding to the presence of speech, and further decompose it into features corresponding to the physical components of articulatory production. these components often evolve in a semi-independent fashion, and conventional viseme-based approaches to recognition fail to capture the resulting co-articulation effects. we present a novel dynamic bayesian network with a multi-stream structure and observations consisting of articulatory feature classifier scores, which can model varying degrees of co-articulation in a principled way. we evaluate our visual-only recognition system on a command utterance task. we show comparative results on lip detection and speech/nonspeech classification, as well as recognition performance against several baseline systems.
egomotion estimation using log-polar images. we address the problem of egomotion estimation of a monocular observer moving with arbitrary translation and rotation in an unknown environment, using log-polar images.the method we propose is uniquely based on the spatio-temporal image derivatives, or the normal flow. thus, we avoid computing the complete optical flow field, which is an ill-posed problem due to the aperture problem. we use a search paradigm based on geometric properties of the normal flow field, and consider a family of search subspaces to estimate the egomotion parameters. these algorithms are particularly well-suited for the log-polar image geometry, as we use a selection of special normal flow vectors with simple representation in logpolar coordinates. this approach highlights the close coupling between algorithmic aspects and the sensor geometry (retina physiology), often found in nature.finally, we present and discuss a set of experiments, for various kinds of camera motions, which show encouraging results.
visual routines for autonomous driving. the paper describes visual routines based on models of color and shape, as well as crucial issues involving the sche duling of such routines. the visual routines are developed in a unique platform. the view from a car driving in a simulated world is fed into a datacube pipeline video processor. the use of this simulation provides a flexible environment from which to set crucial image processing parameters of the individual routines. in addition to the simulations are also tested in similar images generated by driving in the real world, to assure the generalizability of the simulation.
learning and evaluating visual features for pose estimation. we present a method for learning a set of visual landmarks which are useful for pose estimation. the landmark learning mechanism is designed to be applicable to a wide range of environments, and generalized for different approaches to computing a pose estimate. initially, each landmark is detected as a local extreme of a measure of distinctiveness and represented by a principal components encoding which is exploited for matching. attributes of the observed landmarks can be parameterized using a generic parameterization method and then evaluated in terms of their utility for pose estimation. we present experimental evidence that demonstrates the utility of the method.
dense shape reconstruction of a moving object under arbitrary, unknown lighting. we present a method for shape reconstruction from severalimages of a moving object. the reconstruction is dense(up to image resolution). the method assumes that themotion is known, e.g., by tracking a small number of featurepoints on the object. the object is assumed lambertian(completely matte), light sources should not be veryclose to the object but otherwise arbitrary, and no knowledgeof lighting conditions is required. an object changesits appearance significantly when it changes its orientationrelative to light sources, causing violation of the commonbrightness constancy assumption. while a lot of effort isdevoted to deal with this violation, we demonstrate howto exploit it to recover 3d structure from 2d images. wepropose a new correspondence measure that enables pointmatching across views of a moving object. the method hasbeen tested both on computer simulated examples and on areal object.
coupled lighting direction and shape estimation from single images. this paper presents a new method for the simultaneous estimation of lighting direction and shape from shading. the method estimates the shape and the lighting direction using a two step iterative process. we assume an initial (possibly incorrect) estimate of the lighting position. a stiff deformable model is then fitted to the image, assuming this lighting position. next, a least-squares estimate of the lighting position is derived from the model using the levenberg-marquart method.the two steps - model fitting and lighting-position estimation - are iterated. once the light direction has converged to a stable solution the deformable model stiffness is lowered and the model fits accurately given the lighting model. in addition, we show how the method can be used with either orthographic or perspective projection assumptions. in a variety of experiments on real and synthetic data, the method is robust to errors both to the initial light position and shape estimates.
a two-stage robust statistical method for temporal registration from features of various type. a model registration system capable of tracking an object, the model of which is known, in an image sequence is presented. it integrates tracking, pose determination and updating of the visible features. the which handles various features (points, lines and free-form curves) in a very robust way and is able to give a correct estimate of the pose even when tracking errorsoccur. the reliability of the system is shown on an augmented reality project.