waashk/dblp huggingface dataset:
true multi-image alignment and its application to mosaicing and lens distortion correction. multiple images of a scene are related through 2d/3d view transformations and linear and non-linear camera transformations. in the traditional techniques to compute these transformations, especially the ones relying on direct intensity gradients, one image and its coordinate system have been assumed to be ideal and distortion free. in this paper, we present an algorithm for true multi-image alignment that does not rely on the measurements of a reference image being distortion free. the algorithm is developed to specifically align and mosaic images using parametric transformations in the presence of lens distortion. when lens distortion is present, none of the images can be assumed to be ideal. in our formulation, all the images are modeled as intensity measurements represented in their respective coordinate systems, each of which is related to an ideal coordinate system through an interior camera transformation and an exterior view transformation. the goal of the accompanying algorithm is to compute an image in the ideal coordinate system while solving for the transformations that relate the ideal system with each of the data images. key advantages of the technique presented in this paper are: (i) no reliance on one distortion free image, (ii) ability to register images and compute coordinate transformations even when the multiple images are of an extended scene with no overlap between the first and last frame of the sequence, and (iii) ability to handle linear and non-linear transformations within the same framework. results of applying the algorithm are presented for the correction of lens distortion, and creation of video mosaics.
stereo vision for view synthesis. we propose a new method for view synthesis from real images using stereo vision. the method does not explicitly model scene geometry, and enables fast and exact generation of synthetic views. we also re-evaluate the requirements on stereo algorithms for the application of view synthesis and discuss ways of dealing with partially occluded regions of unknown depth and with completely occluded regions of unknown texture. our experiments demonstrate that it is possible to efficiently synthesize realistic new views even from inaccurate and incomplete depth information.
stereo matching with non-linear diffusion. one of the central problems in stereo matching (and other image registration tasks) is the selection of optimal window sizes for comparing image regions. this paper addresses this problem with some novel algorithms based on iteratively diffusing support at different disparity hypotheses, and locally controlling the amount of diffusion based on the current quality of the disparity estimate. it also develops a novel bayesian estimation technique which significantly outperforms techniques based on area-based matching (ssd) and regular diffusion. we provide experimental results on both synthetic and real stereo image pairs.
combining greyvalue invariants with local constraints for object recognition. this paper addresses the problem of recognizing objects in large image databases. the method is based on local characteristics which are invariant to similarity transformations in the image. these characteristics are computed at automatically detected keypoints using the greyvalue signal. the method therefore works on images such as paintings for which geometry based recognition fails.due to the locality of the method, images can be recognized being given part of an image and in the presence of occlusions. applying a voting algorithm and semi-local constraints makes the method robust to noise, scene clutter and small perspective deformations. experiments show an efficient recognition for different types of images. the approach has been validated on an image database containing 1020 images, some of them being very similar by structure, texture or shape.
automatic line matching across views. the paper presents a new method for matching individual line segments between images. the method uses both grey-level information and the multiple view geometric relations between the images. for image pairs epipolar geometry facilitates the computation of a cross-correlation based matching score for putative line correspondences. for image triplets cross-correlation matching scores are used in conjunction with line transfer based on the trifocal geometry. algorithms are developed for both short and long range motion. in the case of long range motion the algorithm involves evaluating a one parameter family of plane induced homographies. the algorithms are robust to deficiencies in the line segment extraction and partial occlusion. experimental results are given for image pairs and triplets, for varying motions between views, and for different scene types. the three view algorithm eliminates all mismatches.
structure from multiple 2d affine correspondences without camera calibration. image motion induced by camera or object motion can be approximated locally by an affine coordinate transformation. we extract 3d information directly from the affine parameters, without camera calibration. the derivation relies on the following assumptions: the object is rigid, locally planar, and its local 3d motion is translation. these assumptions enable complete recovery of 3d structure, whereas it is impossible to compute the direction (and magnitude) of the motion. still, it is possible to distinguish between objects moving differently. explicit expressions for the structure and the motion indicators are given in terms of the 6 affine parameters, computed for each image patch. results of experiments on data with known ground truth are described.
coregistration of range and optical images using coplanarity and orientation constraints. a least-squares method simultaneously solves for the model-to-sensor-suite pose and sensor-to-sensor registration. the development is for a sensor-suite containing separate range and optical sensors. to address outliers and, more generally, match finding, a statistical method (median filtering) and a search method (local search) are developed. sensitivity to gaussian noise and the choice of initial pose estimates is investigated on synthetic data. both of the matching methods are demonstrated on real data.
photorealistic scene reconstruction by voxel coloring. a novel scene reconstruction technique is presented, different from previous approaches in its ability to cope with large changes in visibility and its modeling of intrinsic scene color and texture information. the method avoids image correspondence problems by working in a discretized scene space whose voxels are traversed in a fixed visibility ordering. this strategy takes full account of occlusions and allows the input cameras to be far apart and widely distributed about the environment. the algorithm identifies a special set of invariant voxels which together form a spatial and photometric reconstruction of the scene, fully consistent with the input images. the approach is evaluated with images from both inward-facing and outward-facing cameras.
a common framework for curve evolution, segmentation and anisotropic diffusio. in recent years, curve evolution has developed into an important tool in computer vision and has been applied to a wide variety of problems such as smoothing of shapes, shape analysis and shape recovery. the underlying principle is the evolution of a simple closed curve whose points move in the direction of the normal with prescribed velocity. a fundamental limitation of the method as it stands is that it cannot deal with important image features such as triple points. the method also requires a choice of an "edge-strength" function, defined over the image domain, indicating the likelihood of an object boundary being present at any point in the image domain. this implies a separate preprocessing step, in essence precomputing approximate boundaries in the presence of noise. one also has to choose the initial curve. it is shown here that the different versions of curve evolution used in computer vision together with the preprocessing step can be integrated in the form of a new segmentation functional which overcomes these limitations and extends curve evolution models. moreover, the numerical solutions obtained retain sharp discontinuities or "shocks", thus providing sharp demarcation of object boundaries.
a bayesian segmentation framework for textured visual images. this paper presents a new framework for segmentation of textured visual imagery. the proposed method consists of a bayesian formulation for labeling similar regions. similarity is defined via texture features obtained by gabor wavelets. multivariate gaussian distributions are employed to model the feature class-conditional densities, while the markov process is used to characterize the distributions of the region labeling due to each feature. a coarse nearest neighbor clustering is performed over the feature space to estimate the initial labelings. an iterative solution to the maximum a posteriori(map) estimation is developed, where the parameters of the prior distribution of region labels are estimated using the expectation-maximization(em) algorithm. finally, for man-made object segmentation, a region-growing procedure is used to analyze the classified texture regions by incorporating measures of local shape characteristics to obtain smooth boundaries and region homogeneity. results of the developed algorithm on real scene images are presented.
completion energies and scale. the detection of smooth curves in images and their completion over gaps are two important problems in perceptual grouping. in this study we examine the notion of completion energy of curve elements, showing and exploiting its intrinsic dependence on length and width scales. we introduce a fast method for computing the most likely completion between two elements, by developing novel analytic approximations and a fast numerical procedure for computing the curve of least energy. we then use our newly developed energies to find the most likely completions in images through a generalized summation of induction fields. this is done through multiscale procedures, i.e., separate processing at different scales with some interscale interactions. such procedures allow the summation of all induction fields to be done in a total of only o(n logn) operations, where n is the number of pixels in the image. more important, such procedures yield a more realistic dependence of the induction field on the length and width scales: the field of a long element is very different from the sum of the fields of its composing short segments.
subpixel image registration by estimating the polyphase decomposition of cross power spectrum. a method of registering images at subpixel accuracy has been proposed, which does not resort to interpolation. the method is based on the phase correlation method and is remarkably robust to correlated noise and uniform variations of luminance. we have shown that the cross power spectrum of two images, containing subpixel shifts, is a polyphase decomposition of a dirac delta function. by estimating the sum of polyphase components one can then determine subpixel shifts along each axis.
normalized cuts and image segmentation. we propose a novel approach for solving the perceptual grouping problem in vision. rather than focusing on local features and their consistencies in the image data, our approach aims at extracting the global impression of an image. we treat image segmentation as a graph partitioning problem and propose a novel global criterion, the normalized cut, for segmenting the graph. the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups. we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion. we have applied this approach to segmenting static images, as well as motion sequences, and found the results to be very encouraging.
ridge's corner detection and correspondence. traditionally, corners are found along step edges. in this paper we present an alternative approach-corners along ridges/troughs and local minima points. these features seem to be more reliable for tracking. a new approach for sub-pixel localization of these corners is suggested, using a local approximation of the image surface.
a shock grammar for recognition. we confront the theoretical and practical difficulties of computing a representation for two-dimensional shape, based on shocks or singularities that arise as the shape's boundary is deformed. first, we develop subpixel local detectors for finding and classifying shocks. second, to show that shock patterns are not arbitrary but obey the rules of a grammar, and in addition satisfy specific topological and geometric constraints. shock hypotheses that violate the grammar or are topologically or geometrically invalid are pruned to enforce global consistency. survivors are organized into a hierarchical graph of shock groups computed in the reaction-diffusion space, where diffusion plays a role of regularization to determine the significance of each shock group. the shock groups can be functionally related to the object's parts, protrusions and bends, and the representation is suited to recognition: several examples illustrate its stability with rotations, scale changes, occlusion and movement of parts, even at very low resolutions.
area and length minimizing flows for shape segmentation. several active contour models have been proposed to unify the curve evolution framework with classical energy minimization techniques for segmentation, such snakes. the essential idea is to evolve a curve (in 20) or a surface (in 30) under constraints from image forces so that it clings to features of interest in an intensity image. recently the evolution equation has. been derived from first principles as the gradient flow that minimizes a modified length functional, tailored fo features such as edges. however, because the jlow may slow to converge in practice, a constant (hyperbolic) term is added to keep the curve/surface moving in the desired direction. in this paper, we provide a justification for this term based on the gradient flow derived from a weighted area functional, with image dependent weighting factor. when combined with the earlier modified length gradient flow we obtain a pde which offers number of advantages, as illustrated by several examples of shape segmentation on medical images. in many cases the weighted area flow may be used on its own, with significant computational savings.
further constraints on visual articulated motions. this paper derives what we term the euclidean hinge constraint for projective reconstruction of objects displaying articulated motion. a euclidean hinge is defined here to be an articulation axis with the proviso that any plane perpendicular to the articulation axis in link 1 has a coincident plane which is perpendicular to the articulation ads in link 2 and that these planes remain coincident under articulated motion. this constraint permits the independent projective reconstructions of two adjacent articulated links to be placed in a common frame. the constraint may be expressed mathematically by considering what we define as circular parallax. additionally the existence of a euclidean hinge permits the permits the projective frame to be brought nearer to a euclidean frame for the reconstructed object. a brief reprise of the method of articulation axis estimation is given together with a more extensive series of experimental results.
using a spectral reflectance model for the illumination-invariant recognition of local image structure. we represent local spatial structure in a color image using feature matrices that are computed from an image region. feature matrices contain significantly more information about local image structure than previous representations. although feature matrices are useful for surface recognition, this representation depends on the spectral properties of the scene illumination. using a finite dimensional linear model for surface spectral reflectance with the same number of parameters as the number of color bands, we show that illumination changes correspond to linear transformations of the feature matrices and that surface rotations correspond to circular shifts of the matrices. from these relationships we derive an algorithm for illumination and geometry invariant recognition of local surface structure. we demonstrate the algorithm with a series of experiments on images of real objects.
object recognition using invariant profiles. we derive a sensitivity analysis for moment invariants of multidimensional distributions. these invariants have many uses in computational systems and have recently been used for illumination-invariant recognition in color images. in this context, the sensitivity analysis predicts the response of moment invariants to partial occlusion. using the results of the sensitivity analysis, we develop a novel surface representation called the invariant profile which captures color distribution and spatial information while remaining invariant to the spectral content of the scene illumination. unlike previous representations, the recognition of invariant profiles does not require illumination correction. we demonstrate the sensitivity analysis and the use of invariant profiles for recognition with a set of experiments on color images.
video skimming and characterization through the combination of image and language understanding techniques. digital video is rapidly becoming important for education, entertainment, and a host of multimedia applications. with the size of the video collections growing to thousands of hours, technology is needed to effectively browse seg ments in a short time without losing the content of the video. we propose a method to extract the significant audio and video information and create a "skim" video which represents a very short synopsis of the original. the goal of this work is to show the utility of integrating lan guage and image understanding techniques for video skimming by extraction of significant information, such as specific objects, audio keywords and relevant video struc ture. the resulting skim video is much shorter, where com paction is as high as 20:1, and yet retains the essential content of the original segment.
motion from fixation. we study the problem of estimating rigid motion from a sequence of monocular perspective images obtained by navigating around an object while fixating a particular feature point. we cast the problem in the framework of "epipolar geometry", and propose a filter based upon implicit dynamical model for recursively estimating motion under the fixation constraint. this allows us to compare the quality of the estimates directly against the ones obtained assuming a general rigid motion simply by changing the geometry of the parameter space, while maintaining the same structure of the recursive estimator. we also present a closed-form static solution from two views, and a recursive estimator of the relative pose between the viewer and the scene.
reducing "structure from motion". the literature on recursive estimation of structure and motion from monocular image sequences comprises a large number of different models and estimation techniques. we propose a framework that allows us to derive and compare all models by following the idea of dynamical system reduction. the "natural" dynamic model, derived by the rigidity constraint and the perspective projection, is first reduced by explicitly decoupling structure (depth) from motion. then implicit decoupling techniques are explored, which consist of imposing that some function of the unknown parameters is held constant. by appropriately choosing such a function, not only can we account for all models seen so far in the literature, but we can also derive novel ones. casting all the different models in a common framework allows us to compare their geometric properties on common experimental grounds.
removing the bias from line detection. the extraction of curvilinear structures is an important low-level operation in computer vision. most existing operators use a simple model for the line that is to be extracted, i.e., they do not take into account the surroundings of a line. therefore, they will estimate a wrong line position whenever a line with different lateral contrast is extracted. in contrast, the algorithm proposed in this paper uses an explicit model for lines and their surroundings. by analyzing the scale-space behavior of a model line profile, it is shown how the bias that is induced by asymmetrical lines can be removed. thus, the algorithm is able to extract an unbiased line position and width, both with sub-pixel accuracy.
lens distortion calibration using point correspondences. this paper describes a new method for lens distortion calibration using only point correspondences in multiple views, without the need to know either the 3d location of the points or the camera locations. the standard lens distortion model is a model of the deviations of a real camera from the ideal pinhole or projective camera model.given multiple views of a set of corresponding points taken by ideal pinhole cameras there exist epipolar and trilinear constraints among pairs and triplets of these views. in practice, due to noise in the feature detection and due to lens distortion these constraints do not hold exactly and we get some error. the calibration is a search for the lens distortion parameters that minimize this error. using simulation and experimental results with real images we explore the properties of this method. we describe the use of this method with the standard lens distortion model, radial and decentering, but it could also be used with any other parametric distortion models. finally we demonstrate that lens distortion calibration improves the accuracy of 3d reconstruction.
model-based brightness constraints: on direct estimation of structure and motion. we describe a new direct method for estimating structure and motion from image intensities of multiple views. we extend the direct methods of horn and weldon [18] to three views. adding the third view enables us to solve for motion and compute a dense depth map of the scene, directly from image spatio-temporal derivatives in a linear manner without first having to find point correspondences or compute optical flow. we describe the advantages and limitations of this method which are then verified with experiments using real images
direct estimation of motion and extended scene structure from a moving stereo rig. we describe a new method for motion estimation and 3d reconstruction from stereo image sequences obtained by a stereo rig moving through a rigid world. we show that given two stereo pairs one can compute the motion of the stereo rig directly from the image derivatives (spatial and temporal). correspondences are not required. one can then use the images from both pairs combined to compute a dense depth map. the motion estimates between stereo pairs enable us to combine depth maps from all the pairs in the sequence to form an extended scene reconstruction and we show results from a real image sequence. the motion computation is a linear least squares computation using all the pixels in the image. areas with little or no contrast are implicitly weighted less so one does not have to explicitly apply a confidence measure.
towards accurate recovery of shape from shading under diffuse lighting. a new surface radiance model for diffuse lighting is presented which incorporates shadows, interreflections, and surface orientation. an algorithm is presented that uses this model to compute shape-from-shading under diffuse lighting. the algorithm is tested on both synthetic and real images, and is found to perform more accurately than the only previous algorithm for this problem.