waashk/dblp huggingface dataset:
efficiently registering video into panoramic mosaics. we present an automatic and efficient method to register and stitch thousands of video frames into a large panoramic mosaic. our method preserves the robustness and accuracy of image stitchers that match all pairs of images while utilizing the ordering information provided by video. we reduce the cost of searching for matches between video frames by adaptively identifying key frames based on the amount of image-to-image overlap. key frames are matched to all other key frames, but intermediate video frames are only matched to temporally neighboring key frames and intermediate frames. image orientations can be estimated from this sparse set of matches in time quadratic to cubic in the number of key frames but only linear in the number of intermediate frames. additionally, the matches between pairs of images are compressed by replacing measurements within small windows in the image with a single representative measurement. we show that this approach substantially reduces the time required to estimate the image orientations with minimal loss of accuracy. finally, we demonstrate both the efficiency and quality of our results by registering several long video sequences.
subpixel-precise extraction of watersheds. an approach to extract watersheds and watercourses, as well as their corresponding valleys and hills, from images with subpixel precision is proposed. the critical points of the terrain are essential as the starting points for the construction of these separatrices. they are extracted efficiently with subpixel precision using an approach based on derivatives of gaussian filters. the separatrices are extracted by integrating their defining differential equation.finally, the hills and valleys are constructed by an efficient graph search algorithm. examples show the quality of the results that can be achieved with the proposed approach.
accurate internal camera calibration using rotation, with analysis of sources of error. describes a simple and accurate method for internal camera calibration based on tracking image features through a sequence of images while the camera undergoes pure rotation. a special calibration object is not required and the method can therefore be used both for laboratory calibration and for self calibration in autonomous robots. experimental results with real images show that focal length and aspect ratio can be found to within 0.15 percent, and lens distortion error can be reduced to a fraction of a pixel. the location of the principal point and the location of the center of radial distortion can each be found to within a few pixels. we perform a simple analysis to show to what extent the various technical details affect the accuracy of the results. we show that having pure rotation is important if the features are derived from objects close to the camera. in the basic method accurate angle measurement is important. the need to accurately measure the angles can be eliminated by rotating the camera through a complete circle while taking an overlapping sequence of images and using the constraint that the sum of the angles must equal 960 degrees.
filtering using a tree-based estimator. within this paper a new framework for bayesian tracking ispresented, which approximates the posterior distribution atmultiple resolutions. we propose a tree-based representationof the distribution, where the leaves define a partition ofthe state space with piecewise constant density. the advantageof this representation is that regions with low probabilitymass can be rapidly discarded in a hierarchical search,and the distribution can be approximated to arbitrary precision.we demonstrate the effectiveness of the technique byusing it for tracking 3d articulated and non-rigid motionin front of cluttered background. more specifically, we areinterested in estimating the joint angles, position and orientationof a 3d hand model in order to drive an avatar.
robot aerobics: four easy steps to a more flexible calibration. presents a method for calibrating intrinsic and extrinsic camera parameters. this algorithm can easily be modified by other users to suit their particular calibration needs, without requiring a high-precision calibration target or complicated linear algebra. the algorithm uses controlled motions and a single light source to simulate calibration targets in convenient 3d locations. these convenient calibration targets enable us to simplify the calibration algorithm and gather dense data for lens distortion. dense data makes the distortion correction more accurate than traditional low-order polynomial fits, and allows us to calibrate wide-angle lenses (
expected performance of robust estimators near discontinuities. in extracting a polynomial surface patch near an intensity or range discontinuity, a robust estimator must tolerate not only the truly random bad data ("random outliers"), but also the coherently structured points ("pseudo outliers") that belong to a different surface. to characterize the performance of least median of squares, m estimators, hough transforms, ransac, and minpran on data containing both random and pseudo outliers, we develop two analytical measures, "pseudo outlier bias" and "pseudo outlier breakdown". using these measures, we find that each robust estimator has surprisingly poor performance, even under the best possible circumstances, implying that present estimators should be used with care and new estimators should be developed.
how hard is 3-view triangulation really? we present a solution for optimal triangulation in three views. the solution is guaranteed to find the optimal solution because it computes all the stationary points of the (maximum likelihood) objective function. internally, the solution is found by computing roots of multi-variate polynomial equations, directly solving the conditions for stationarity. the solver makes use of standard methods from computational commutative algebra to convert the root-finding problem into a 47 × 47 non-symmetric eigen-problem. although there are in general 47 roots, counting both real and complex ones, the number of real roots is usually much smaller. we also show experimentally that the number of stationary points that are local minima and lie in front of each camera is small but does depend on the scene geometry.
dense matching of multiple wide-baseline views. this paper describes a pde-based method for densedepth extraction from multiple wide-baseline images. emphasislies on the usage of only a small amount of images.the integration of these multiple wide-baseline views isguided by the relative confidence that the system has in thematching to different views. this weighting is fine-grainedin that it is determined for every pixel at every iteration.reliable information spreads fast at the expense of less reliabledata, both in terms of spatial communications withina view and in terms of information exchange between theviews. changes in intensity between images can be handledin a similar fine grained fashion.
segmentation and range sensing using a moving-aperture lens. the use of a novel motorized lens to perform segmentation of image sequences is presented in this paper. the lens has the effect of introducing small, repeating movements of the camera center so that objects appear to translate in the image by an amount that depends on the distance from the plane of focus. for a stationary scene, optical flow magnitudes are therefore directly related to three-dimensional object distance from the observer. we describe a segmentation procedure that exploits these controlled observer movements and present experimental results that demonstrate the successful extraction of objects at different depths. potential applications of our approach include image compositing, teleconferencing, and range estimation.
learning hierarchical models of scenes, objects, and parts. we describe a hierarchical probabilistic model for the detection and recognition of objects in cluttered, natural scenes. the model is based on a set of parts which describe the expected appearance and position, in an object centered coordinate frame, of features detected by a low-level interest operator. each object category then has its own distribution over these parts, which are shared between objects. we learn the parameters of this model via a gibbs sampler which uses the graphical model¿s structure to analytically average over many parameters. applied to a database of images of isolated objects, the sharing of parts among objects improves detection accuracy when few training examples are available. we also extend this hierarchical framework to scenes containing multiple objects.
object localization by bayesian correlation. maximization of cross-correlation is a commonly used principle for intensity-based object localization that gives a single estimate of location. however, to facilitate sequential inference (eg over time or scale) and to allow the representation of ambiguity, it is desirable to represent an entire probability distribution for object location. although the cross-correlation itself (or some function of it) has sometimes been treated as a probability distribution, this is not generally justifiable.bayesian correlation achieves a consistent probabilistic treatment by combining several developments. the first is the interpretation of correlation matching functions in probabilistic terms, as observation likelihoods. second, probability distributions of filter-bank responses are learned from training examples. inescapably, response-learning also demands statistical modeling of background intensities, and there are links here with image coding and independent component analysis. lastly, multi-scale processing is achieved, in a bayesian context, by means of a new algorithm, layered sampling, for which asymptotic properties are derived.
automatic model construction, pose estimation, and object recognition from photographs using triangular splines. this paper proposes a method for automatically constructing triangular g1 spline models of complex three-dimensional objects from a few registered photographs. these models are used for pose estimation from monocular silhouette data and they form the basis for a simple recognition strategy. the proposed approach is demonstrated by several experiments.
recognition of 3d free-form objects using segment-based stereo vision. we propose a new method to recognize 3d free-form objects from their apparent contours. it is the extension of our established method to recognize objects with fixed edges. object models are compared with 3d boundaries which are extracted by segment-based stereo vision. based on the local shapes of the boundaries, candidate transformations are generated. the candidates are verified and adjusted based on the whole shapes of the boundaries. themodels are built from all-around range data of the objects. experimental results show the effectiveness of the method.
video input driven animation (vida). there are many challenges associated with the integration of synthetic and real imagery. one particularly difficult problem is the automatic extraction of salient parameters of natural phenomena in real video footage for subsequent application to synthetic objects. can we ensure that the hair and clothing of a synthetic actor placed in a meadow of swaying grass will move consistently with the wind that moved that grass? the video footage can be seen as a controller for the motion of synthetic features, a concept we call video input driven animation (vida). we propose a schema that analyzes an input video sequence, extracts parameters from the motion of objects in the video, and uses this information to drive the motion of synthetic objects. to validate the principles of vida, we approximate the inverse problem to harmonic oscillation, which we use to extract parameters of wind and of regular water waves. we observe the effect of wind on a tree in a video, estimate wind speed parameters from its motion, and then use this to make synthetic objects move. we also extract water elevation parameters from the observed motion of boats and apply the resulting water waves to synthetic boats.
geometric and photometric restoration of distorted documents. we present a system to restore the 2d content printed on distorted documents. our system works by acquiring a 3d scan of the document¿s surface together with a high-resolution image. using the 3d surface information and the 2d image, we can ameliorate unwanted surface distortion and effects from non-uniform illumination. our system can process arbitrary geometric distortions, not requiring any pre-assumed parametric models for the document¿s geometry. the illumination correction uses the 3d shape to distinguish content edges from illumination edges to recover the 2d content¿s reflectance image while making no assumptions about light sources and their positions. results are shown for real objects, demonstrating a complete framework capable of restoring geometric and photometric artifacts on distorted documents.
bi-directional tracking using trajectory segment analysis. in this paper, we present a novel approach to keyframe-based tracking, called bi-directional tracking. given two object templates in the beginning and ending keyframes, the bi-directional tracker outputs the map (maximum a posterior) solution of the whole state sequence of the target object in the bayesian framework. first, a number of 3d trajectory segments of the object are extracted from the input video, using a novel trajectory segment analysis. second, these disconnected trajectory segments due to occlusion are linked by a number of inferred occlusion segments. last, the map solution is obtained by trajectory optimization in a coarse-to-fine manner. experimental results show the robustness of our approach with respect to sudden motion, ambiguity, and short and long periods of occlusion.
more-than-topology-preserving flows for active contours and polygons. active contour and active polygon models have been used widely for image segmentation. in some applications, the topology of the object(s) to be detected from an image is known a priori, despite an unknown complex geometry, and it is important that the active contour or polygon maintain the desired topology. in this work, we construct a novel geometric flow that can be added to image based evolutions of active contours and polygons so that the topology of the initial contour or polygon is preserved. indeed, the proposed geometric flow ensures more than just correct topology; it ensures that the active contour or polygon is, in some sense, kept far away from a topology change. smoothness properties similar to curvature flow are also guaranteed by the proposed geometric flow. the proposed topology preserving geometric flow is the gradient flow arising from an energy that is based on electrostatic principles. the evolution of a single point on the contour depends on all other points of the contour, which is different from traditional curve evolutions in computer vision literature.
prediction error as a quality metric for motion and stereo. this paper presents a new methodology for evaluating the quality of motion estimation and stereo correspondence algorithms. motivated by applications such as novel view generation and motion-compensated compression, we suggest that the ability to predict new views or frames is a natural metric for evaluating such algorithms.our new metric has several advantages over comparing algorithm outputs to true motions or depths. first of all, it does not require the knowledge of ground truth data, which may be difficult or laborious to obtain. second, it more closely matches the ultimate requirements of the application, which are typically tolerant of errors in uniform color regions, but very sensitive to isolated pixel errors or disocclusion errors.in the paper, we develop a number of error metrics based on this paradigm, including forward and inverse prediction errors, residual motion error, and local motion-compensated prediction error. we show results on a number of widely used motion and stereo sequences, many of which do not have associated ground truth data.
stereo matching with transparency and matting. this paper formulates and solves a new variant of the stereo correspondence problem: simultaneously recovering the disparities, true colors, and opacities of visible surface elements. this problem arises in newer applications of stereo reconstruction, such as view interpolation and the layering of real imagery with synthetic graphics for special effects and virtual studio applications. while this problem is intrinsically more difficult than traditional stereo correspondence, where only the disparities are being recovered, it provides a principled way of dealing with commonly occuring problems such as occlusions and the handling of mixed (foreground/background) pixels near depth discontinuities. it also provides a novel means for separating foreground and background objects (matting) without the use of a special blue screen. we formulate the problem as the recovery of colors and opacities in a generalized 3-d (x, y, d) disparity space, and solve the problem using a combination of initial evidence aggregation followed by iterative energy minimization.
motion estimation with quadtree splines. this paper presents a motion estimation algorithm based on a new multiresolution representation, the quadtree spline. this representation describes the motion field as a collection of smoothly connected patches of varying size, where the patch size is automatically adapted to the complexity of the underlying motion. the topology of the patches is determined by a quadtree data structure, and both split and merge techniques are developed for estimating this spatial subdivision. the quadtree spline is implemented using another novel representation, the adaptive hierarchical basis spline, and combines the advantages of adaptively-sized correlation windows with the speedups obtained with hierarchical basis preconditioners. results are presented on some standard motion sequences.
a geometric criterion for shape-based non-rigid correspondence. a geometric criterion is developed for establishing shape based non rigid correspondence between plane curves. unlike previous efforts, the criterion does not use rigid invariants of shape. instead, shapes are compared non rigidly from the vantage point of the correspondence. geometric invariants are proposed for curves whose shapes can be exactly matched by a non rigid correspondence. the invariants are based on angular deviations of convex and concave segments of the curves. examples of correspondences between curves obtained from medical images are provided.
separating reflection components of textured surfaces using a single image. the presence of highlights, which in dielectric inhomogeneousobjects are linear combination of specular and diffusereflection components, is inevitable. a number of methodshave been developed to separate these reflection components.to our knowledge, all methods that use a singleinput image require explicit color segmentation to deal withmulticolored surfaces. unfortunately, for complex texturedimages, current color segmentation algorithms are stillproblematic to segment correctly. consequently, a methodwithout explicit color segmentation becomes indispensable,and this paper presents such a method. the method is basedsolely on colors, particularly chromaticity, without requiringany geometrical parameter information. one of the basicideas is to compare the intensity logarithmic differentiationof specular-free images and input images iteratively.the specular-free image is a pseudo-code of diffuse componentsthat can be generated by shifting a pixel's intensityand chromaticity nonlinearly while retaining its hue. allprocesses in the method are done locally, involving a maximumof only two pixels. the experimental results on naturalimages show that the proposed method is accurate and robustunder known scene illumination chromaticity. unlikethe existing methods that use a single image, our methodis effective for textured objects with complex multicoloredscenes.
highlight removal by illumination-constrained inpainting. we present a single-image highlight removal method that incorporates illumination-based constraints into image inpainting. unlike occluded image regions filled by traditional inpainting, highlight pixels contain some useful information for guiding the inpainting process. constraints provided by observed pixel colors, highlight color analysis and illumination color uniformity are employed in our method to improve estimation of the underlying diffuse color. the inclusion of these illumination constraints allows for better recovery of shading and textures by inpainting. experimental results are given to demonstrate the performance of our method.
common pattern discovery using earth mover's distance and local flow maximization. in this paper, we present a novel segmentation-insensitive approach for mining common patterns from 2 images. we develop an algorithm using the earth movers distance (emd) framework, unary and adaptive neighborhood color similarity. we then propose a novel local flow maximization approach to provide the best estimation of location and scale of the common pattern. this is achieved by performing an iterative optimization in search of the most stable flows¿ centroid. common pattern discovery isdifficult owing to the huge search space and problem domain. we intend to solve this problem by reducing the search space through identifying the location and a reduced spatial space for common pattern discovery. experimental results justify the effectiveness and the potential of the approach.
integrated surface, curve and junction inference from sparse 3-d data sets. we are interested in descriptions of 3-d data sets, as obtained from stereo or a 3-d digitizer. we therefore consider as input a sparse set of points, possibly associated with orientation information. in this paper, we address the problem of inferring integrated high-level descriptionssuch as surfaces, curves, and junctions from a sparse point set. while the method described in [5], [6] provides excellent results for smooth structures, it only detects discontinuities, but does not localize them. for precise localization, we propose a non-iterative cooperative algorithm in which surfaces, curves, and junctions work together: initial estimates are computed based on [5], [6], where each point in the given sparse and possibly noisy point set is convolved with a predefined vector mask to produce dense saliency maps. these maps serve as input to our novel maximal surface and curve marching algorithms for initial surface and curve extraction. refinement of initial estimates is achieved by hybrid voting using excitatory and inhibitory fields for inferring reliable and natural extension so that surface/curve and curve/junction discontinuities are preserved. results on several synthetic as well as real data sets are presented.
direction diffusion. in a number of disciplines, directional data provides a fundamental source of information. a novel framework for isotropic and anisotropic diffusion of directions is presented in this paper. the framework can be applied both to regularize directional data and to obtain multi-scale representations of it. the basic idea is to apply and extend results from the theory of harmonic maps in liquid crystals.this theory deals with the regularization of vectorial data, while satisfying the unit norm constraint of directional data. we show the corresponding variational and partial differential equations formulations for isotropic diffusion, obtained from an l2 norm, and edge preserving diffusion, obtained from an l1 norm.in contrast with previous approaches, the framework is valid for directions in any dimensions, supports non-smooth data, and gives both isotropic and anisotropic formulations. we present a number of theoretical results, open questions, and examples for gradient vectors, optical flow, and color images.