waashk/dblp huggingface dataset:
outlier correction in image sequences for the affine camera. it is widely known that, for the affine camera model, both shape and motion can be factorized directly from the so-called image measurement matrix constructed from image point coordinates. the ability to extract both shape and motion from this matrix by a single svd operation makes this shape-from-motion approach attractive; however, it cannot deal with missing feature points and, in the presence of outliers, a direct svd to the matrix would yield highly unreliable shape and motion components. in this paper, we present an outlier correction scheme that iteratively updates the elements of the image measurement matrix. the magnitude and sign of the update to each element is dependent upon the residual robustly estimated in each iteration. the result is that outliers are corrected and retained, giving improved reconstruction and smaller reprojection errors. our iterative outlier correction scheme has been applied to both synthesized and real video sequences. the results obtained are remarkably good.
reconstructing the geometry of flowing water. we present a recording scheme, image formation model and reconstruction method that enables image-based modeling of flowing bodies of water from multi-video input data. the recorded water is dyed with a fluorescent chemical to measure the thickness of a column of water, which leads to an image formation model based on integrated emissivities along a viewing ray. this model allows for a photo-consistency based error measure for a weighted minimal surface, which is recovered using a pde obtained from the euler-lagrangian formulation of the problem. the resulting equation is solved using the level set method.
ensuring color consistency across multiple cameras. most multi-camera vision applications assume a single common color response for all cameras. however different cameras ¿ even of the same type ¿ can exhibit radically different color responses, and the differences can cause significant errors in scene interpretation. to address this problem we have developed a robust system aimed at inter-camera color consistency. our method consists of two phases: an iterative closed-loop calibration phase that searches for the per-camera hardware register settings that best balance linearity and dynamic range, followed by a refinement phase that computes the per-camera parametric values for an additional software-based color mapping.
closed-world tracking. a new approach to tracking weakly modeled objects in a semantically rich domain is presented. we define a closed-world as a space-time region of an image sequence in which the complete taxonomy of objects is known, and in which each pixel should be explained as belonging to one of those objects. given contextual object information, context-specific features can be dynamically selected as the basis for tracking. a context-specific feature is one that has been chosen based upon the context to maximize the chance of successful tracking between frames. our work is motivated by the goal of video annotation-the semi-automatic generation of symbolic descriptions of action taking place in a contextually-rich dynamic scene. we describe how contextual knowledge in the "football domain" can be applied to closed-world football player tracking and present the details of our implementation. we include tracking results based on hundreds of images that demonstrate the wide range of tracking situations the algorithm successfully handles as well as a few examples of where the algorithm fails.
finding people by sampling. we show how to use a sampling method to find sparsely clad people in static images. people are modeled as an assembly of nine cylindrical segments. segments are found using an em algorithm, and then assembled into hypotheses incrementally, using a learned likelihood model. each assembly step passes on a set of samples of its likelihood to the next; this yields effective pruning of the space of hypotheses. the collection of available nine-segment hypotheses is then represented by a set of equivalence classes, which yield an efficient pruning process. the posterior for the number of people is obtained from the class representatives. people are counted quite accurately in images of real scenes using an map estimate. we show the method allows top-down as well as bottom up reasoning. while the method can be overwhelmed by very large numbers of segments, we show that this problem can be avoided by quite simple pruning steps.
robust multi-sensor image alignment. this paper presents a method for alignment of images acquired by sensors of different modalities (e.g., eo and ir). the paper has two main contributions: (i) it identifies an appropriate image representation for multi-sensor alignment, i.e., a representation which emphasizes the common information between the two multi-sensor images, suppresses the non-common information, and is adequate for coarse-to-fine processing. (ii) it presents a new alignment technique, which applies global estimation to any choice of a local similarity measure. in particular, it is shown that when this registration technique is applied to the chosen image representation with a local-normalized-correlation similarity measure, it providesa new multi-sensor alignment algorithm which is robust to outliers, and applies to a wide variety of globally complex brightness transformations between the two images.our proposed image representation does not rely on sparse image features (e.g., edge, contour, or point features). it is continuous and does not eliminate the detailed variations within local image regions. our method naturally extends to coarse-to-fine processing, and applies even in situations when the multi-sensor signals are globally characterized by low statistical correlation.
mosaic based representations of video sequences and their applications. recently, there has been a growing interest in the use of mosaic images to represent the information contained in video sequences. the paper systematically investigates how to go beyond thinking of the mosaic simply as a visualization device, but rather as a basis for efficient representation of video sequences. we describe two different types of mosaics called the static and the dynamic mosaic that are suitable for different needs and scenarios. we discuss a series of extensions to these basic mosaics to provide representations at multiple spatial and temporal resolutions and to handle 3d scene information. we describe techniques for the basic elements of the mosaic construction process, namely alignment, integration, and residual analysis. we describe several applications of mosaic representations including video compression, enhancement, enhanced visualization, and other applications in video indexing, search, and manipulation.
a mixed-state condensation tracker with automatic model-switching. there is considerable interest in the computer vision community in representing and modelling motion. motion models are use d as predictors to increase the robustness and accuracy of visual trackers, and as classifiers for gesture recognition. this paper presents a significant development of random sampling methods to allow automatic switching between multiple motion models as a natural extension of the tracking process. the bayesian mixed-state framework is described in its generality, and the example of a bouncing ball is used to demonstrate that a mixed-state model can significantly improve tracking performance in heavy clutter. the relevance of the approach to the problem of gesture recognition is then investigated using a tracker which is able to follow the natural drawing action of a hand holding a pen, and switches state according to the hand's motion.
finding tree structures by grouping symmetries. the representation of objects in images as tree structures is of great interest to vision, as they can represent articulated objects such as people as well as other structured objects like arteries in human bodies, roads, circuit board patterns, etc. tree structures are often related to the symmetry axis representation of shapes, which captures their local symmetries. algorithms have been introduced to detect (i) open contours in images in quadratic time (ii) closed contours in images in cubic time, and (iii) tree structures from contours in quadratic time. the algorithms are based on dynamic programming and single source shortest path algorithms. however, in this paper, we show that the problem of finding tree structures in images in a principled manner is a much harder problem. we argue that the optimization problem of finding tree structures in images is essentially equivalent to a variant of the steiner tree problem, which is np-hard. nevertheless, an approximate polynomial-time algorithm for this problem exists: we apply a fast implementation of the goemans-williamson approximate algorithm to the problem of finding a tree representation after an image is transformed by a local symmetry mapping. examples of extracting tree structures from images illustrate the idea and applicability of the approximate method.
stochastic refinement of the visual hull to satisfy photometric and silhouette consistency constraints. an iterative method for reconstructing a 3d polygonalmesh and color texture map from multiple views of an objectis presented. in each iteration, the method first estimates atexture map given the current shape estimate. the texturemap and its associated residual error image are obtainedvia maximum a posteriori estimation and reprojection of themultiple views into texture space. next, the surface shape isadjusted to minimize residual error in texture space. thesurface is deformed towards a photometrically-consistentsolution via a series of 1d epipolar searches at randomlyselected surface points. the texture space formulation hasimproved computational complexity over standard image-basederror aproaches, and allows computation of the reprojectionerror and uncertainty for any point on the surface.moreover, shape adjustments can be constrained suchthat the recovered model's silhouette matches those of theinput images. experiments with real world imagery demonstratethe validity of the approach.
robust contour tracking in echocardiographic sequences. in this paper we present an evaluation of a robust visual image tracker on echocardiographic image sequences. we show how the tracking framework can be customised to define an appropriate shape-space that describes heart shape deformations that can be learnt from a training data set. we also investigate an energy-based temporal boundary enhancement method to improve image feature measurement. preliminary results are presented demonstrating tracking on real normal heart motion data sequences and synthesised and real abnormal heart motion data sequences. we conclude by discussing some of our current research efforts.
condensing image databases when retrieval is based on non-metric distances. one of the key problems in appearance-based-vision is understanding how to use a set of labeled images to classify new images. classification systems that can model human performance, or that use robust image matching methods, often make use of similarity judgments that are non-metric, but when the triangle inequality is not obeyed, most existing pattern recognition techniques are not applicable. we note that exemplar-based (or nearest-neighbor) methods can be applied naturally when using a wide class of non-metric similarity functions. the key issue, however, is to find methods for choosing good representatives of a class that accurately characterize it. we show that existing condensing techniques for finding class representatives are ill-suited to deal with non-metric dataspaces.we then focus on developing techniques for solving this problem, emphasizing two points: first, we show that the distance between two images is not a good measure of how well one image can represent another in non-metric spaces. instead, we use the vector correlation between the distances from each image to other previously seen images. second, we show that in non-metric spaces, boundary points are less significant for capturing the structure of a class than they are in euclidean spaces. we suggest that atypical points may be more important in describing classes. we demonstrate the importance of these ideas to learning thatgeneralizes from experience by improving performance using both synthetic and real images.
saliency maps and attention selection in scale and spatial coordinates: an information theoretic approach. information measures with respect to spatial locations and scales of objects in an image are important to image processing and interpretation. it allows us to focus attention on relevant data, saving effort and reducing false positives. in particular, the information content of a man-made scene is typically confined to a small set of scales. we devise a scale space based measure of image information. kullback contrasts between successive resolution lengths gives the differential information gain. experiments show that this measure gives a clear indication of characteristic lengths in a variety of real world images and is superior to power spectrum based measurements. decomposing the expected information gain into spatial coordinates gives us a saliency map for use by an attention selector. we combine the scale and spatial decompositions into a single information measure, giving both the spatial extent and scale range of interest. the information measure has an efficient implementation, and thus can be used routinely in early vision processing.
multimodal human computer interaction: a survey. in this paper, we review the major approaches to multimodal human-computer interaction, giving an overview of the field from a computer vision perspective. in particular, we focus on body, gesture, gaze, and affective interaction (facial expression recognition and emotion in audio). we discuss user and task modeling, and multimodal fusion, highlighting challenges, open issues, and emerging applications for multimodal human-computer interaction (mmhci) research.
tracking across multiple cameras with disjoint views. conventional tracking approaches assume proximity inspace, time and appearance of objects in successive observations.however, observations of objects are often widelyseparated in time and space when viewed from multiplenon-overlapping cameras. to address this problem, wepresent a novel approach for establishing object correspondenceacross non-overlapping cameras. our multi-cameratracking algorithm exploits the redundance in paths thatpeople and cars tend to follow, e.g. roads, walk-ways orcorridors, by using motion trends and appearance of objects,to establish correspondence. our system does notrequire any inter-camera calibration, instead the systemlearns the camera topology and path probabilities of objectsusing parzen windows, during a training phase. oncethe training is complete, correspondences are assigned usingthe maximum a posteriori (map) estimation framework.the learned parameters are updated with changing trajectorypatterns. experiments with real world videos are reported,which validate the proposed approach.
images as bags of pixels. we propose modeling images and related visual objects as bags of pixels or sets of vectors. for instance, gray scale images are modeled as a collection or bag of (x, y, i) pixel vectors. this representation implies a permutational invariance over the bag of pixels which is naturally handled by endowing each image with a permutation matrix. each matrix permits the image to span a manifold of multiple configurations, capturing the vector set's invariance to orderings or permutation transformations. permutation configurations are optimized while jointly modeling many images via maximum likelihood. the solution is a uniquely solvable convex program which computes correspondence simultaneously for all images (as opposed to traditional pairwise correspondence solutions). maximum likelihood performs a nonlinear dimensionality reduction, choosing permutations that compact the permuted image vectors into a volumetrically minimal subspace. this is highly suitable for principal components analysis which, when applied to the permutationally invariant bag of pixels representation, outperforms pca on appearance-based vectorization by orders of magnitude.furthermore, the bag of pixels subspace benefits from automatic correspondence estimation, giving rise to meaningful linear variations such as morphings, translations, and jointly spatio-textural image transformations. results are shown for several datasets.
shape gradients for histogram segmentation using active contours. we consider the problem of image segmentation using active contours through the minimization of an energy criterion involving both region and boundary functionals. these functionals are derived through a shape derivative approach instead of classical calculus of variation. the equations can be elegantly derived without converting the region integrals into boundary integrals. from the derivative, we deduce the evolution equation of an active contour that makes it evolve towards a minimum of the criterion. we focus more particularly on statistical features globally attached to the region and especially to the probability density functions of image features such as the color histogram of a region. a theoretical framework is set for the minimization of the distance between two histograms for matching or tracking purposes. an application of this framework to the segmentation of color histograms in video sequences is then proposed. we briefly describe our numerical scheme and show some experimental results.
qualitative probabilities for image interpretation. two basic problems in image interpretation are: a) determining which interpretations are the most plausible amongst many possibilities; and b) controlling the search for plausible interpretations. we address these issues using a bayesian approach, with the plausibility ordering and search pruning based on the posterior probabilities of interpretations. however, due to the need for detailed quantitative prior probabilities and the need to evaluate complex integrals over various conditional distributions, a full bayesian approach is currently impractical except in tightly constrained domains.to circumvent these difficulties we introduce the notion of qualitative probabilistic analysis. in particular, given spatial and contrast resolution parameters, we consider only the asymptotic order of the posterior probability for any interpretation as these resolutions are made finer. we introduce this approach for a simple card-world domain, and present computational results for blocks-world images.
globally optimal regions and boundaries. we propose a new form of energy functional for the segmentation of regions in images, and an efficient method for finding its global optima. the energy can have contributions from both the region and its boundary, thus combining the best features of region- and boundary-based approaches to segmentation. by transforming the region energy into a boundary energy, we can treat both contributions on an equal footing, and solve the global optimization problem as a minimum mean weight cycle problem on a directed graph. the simple, polynomial-time algorithm requires no initialization and is highly parallelizable
multi-modal tensor face for simultaneous super-resolution and recognition. face images of non-frontal views under poor illumination with low resolution reduce dramatically face recognition accuracy. this is evident most compellingly by the very low recognition rate of all existing face recognition systems when applied to live cctv camera input. in this paper, we present a bayesian framework to perform multi-modal (such as variations in viewpoint and illumination) face image super-resolution for recognition in tensor space. given a single modal low-resolution face image, we benefit from the multiple factor interactions of training tensor, and super-resolve its high-resolution reconstructions across different modalities for face recognition. instead of performing pixel-domain super-resolution and recognition independently as two separate sequential processes, we integrate the tasks of super-resolution and recognition by directly computing a maximum likelihood identity parameter vector in high-resolution tensor space for recognition. we show results from multi-modal super-resolution and face recognition experiments across different imaging modalities, using low-resolution images as testing inputs and demonstrate improved recognition rates over standard tensorface and eigenface representations.
image registration with global and local luminance alignment. inspired by tensor voting, we present luminance voting, a novel approach for image registration with global and local luminance alignment. the key to our modeless approach is the direct estimation of replacement function, by reducing the complex estimation problem to the robust 2d tensor voting in the corresponding voting spaces. no model for replacement function is assumed. luminance data are first encoded into 2d ball tensors. subject to the monotonic constraint only, we vote for an optimal replacement function by propagating the smoothness constraint using a dense tensor field. our method effectively infers missing curve segments and rejects image outliers without assuming any simplifying or complex curve model. the voted replacement functions are used in our iterative registration algorithm for computing the best warping matrix. unlike previous approaches, our robust method corrects exposure disparity even if the two overlapping images are initially misaligned. luminance voting is effective in correcting exposure difference, eliminating vignettes, and thus improving image registration. we present results on a variety of images.
eliminating structure and intensity misalignment in image stitching. the aim of this paper is to achieve seamless image stitching for eliminating obvious visual artifact caused by severe intensity discrepancy, image distortion and structure mis-alignment, given that the input images are globally registered. our approach is based on structure deformation and propagation while maintaining the overall appearance affinity of the result to the input images. this new approach is proven to be effective in solving the above problems, and has found applications in mosaic deghosting, image blending and intensity correction. our new method consists of the following main processes. first, salient features or structures are robustly detected and aligned along the optimal partitioning boundary between the input images. from these features, we derive sparse deformation vectors to uniformly encode the underlying structure and intensity misalignment. these sparse deformation cues will then be propagated robustly and smoothly into the interior of the target image by solving the associated laplace equations in the image gradient domain. we present convincing results to show that our method can handle significant structure and intensity misalignment in image stitching.
a robust algorithm for point set registration using mixture of gaussians. this paper proposes a novel and robust approach to the point set registration problem in the presence of large amounts of noise and outliers. each of the point sets is represented by a mixture of gaussians and the point set registration is treated as a problem of aligning the two mixtures. we derive a closed-form expression for the l¿ distance between two gaussian mixtures, which in turn leads to a computationally efficient registration algorithm. this new algorithm has an intuitive interpretation, is simple to implement and exhibits inherent statistical robustness. experimental results indicate that our algorithm achieves very good performance in terms of both robustness and accuracy.