waashk/dblp huggingface dataset:
integrating the effects of motion, illumination and structure in video sequences. most work in computer vision has concentrated on studying the individual effects of motion and illumination on a 3d object. in this paper, we present a theory for combining the effects of motion, illumination, 3d structure, albedo, and camera parameters in a sequence of images obtained by a perspective camera. we show that the set of all lambertian reflectance functions of a moving object, illuminated by arbitrarily distant light sources, lies "close" to a bilinear subspace consisting of nine illumination variables and six motion variables. this result implies that, given an arbitrary video sequence, it is possible to recover the 3d structure, motion and illumination conditions simultaneously using the bilinear subspace formulation. the derivation is based on the intuitive notion that, given an illumination direction, the images of a moving surface cannot change suddenly over a short time period. we experimentally compare the images obtained using our theory with ground truth data and show that the difference is small and acceptable. we also provide experimental results on real data by synthesizing video sequences of a 3d face with various combinations of motion and illumination directions.
edit distance from graph spectra. this paper is concerned with computing graph edit distance. one of the criticisms that can be leveled at existing methods for computing graph edit distance is that it lacks the formality and rigour of the computation of string edit distance. hence, our aim is to convert graphs to string sequences so that standard string edit distance techniques can be used. to do this we use graph spectral seriation method to convert the adjacency matrix into a string or sequence order. we pose the problem of graph-matching as maximum aposteriori probability alignment of the seriation sequences for pairs of graphs. this treatment leads to an expression for the edit costs. we compute the edit distance by finding the sequence of string edit operations which minimise the cost of the path traversing the edit lattice. the edit costs are defined in terms of the a posteriori probability of visiting a site on the lattice. we demonstrate the method with results on a data-set of delaunay graphs.
phase field models and higher-order active contours. the representation and modelling of regions is an important topic in computer vision. in this paper, we represent a region via a level set of a ¿phase field¿ function. the function is not constrained, e.g. to be a distance function; nevertheless, phase field energies equivalent to classical active contour energies can be defined. they represent an advantageous alternative to other methods: a linear representation space; ease of implementation (a pde with no reinitialization); neutral initialization; greater topological freedom. we extend the basic phase field model with terms that reproduce ¿higher-order active contour¿ energies, a powerful way of including prior geometric knowledge in the active contour framework via nonlocal interactions between contour points. in addition to the above advantages, the phase field greatly simplifies the analysis and implementation of the higher-order terms. we define a phase field model that favours regions composed of thin arms meeting at junctions, combine this with image terms, and apply the model to the extraction of line networks from remote sensing images.
automatic registration of 3-d ultrasound images. one of the most promising applications of 3-d ultrasound lies in the visualisation and volume estimation of internal 3-d structures. unfortunately, artifacts and speckle make automatic analysis of the data difficult. in this paper we investigate the use of 3-d spatial compounding to improve data quality, and find that accurate registration is the key. a correlation-based registration technique is applied to 3-d ultrasound data acquired from in-vivo examinations of a human gall bladder. we find that the registration technique performs well, and visualisation and segmentation of the compounded data are clearly improved.
trilinearity of three perspective views and its associated tensor. it has been established that certain trilinear forms of three perspective views give rise to a tensor of 27 intrinsic coefficients. we show in this paper that a permutation of the the trilinear coefficients produces three homography matrices (projective transformations of planes) of three distinct intrinsic planes, respectively. this, in turn, yields the result that 3d invariants are recovered directly-simply by appropriate arrangement of the tensor's coefficients. on a secondary level, we show new relations between fundamental matrix, epipoles, euclidean structure and the trilinear tensor. on the practical side, the new results extend the existing envelope of methods of 3d recovery from 2d views-for example, new linear methods that cut through the epipolar geometry, and new methods for computing epipolar geometry using redundancy available across many views.
object tracking across multiple independently moving aerial cameras. a camera mounted on an aerial vehicle provides an excellent means for monitoring large areas of a scene. utilizing several such cameras on different aerial vehicles allows further flexibility, in terms of increased visual scope and in the pursuit of multiple targets. in this paper, we address the problem of tracking objects across multiple moving airborne cameras. since the cameras are moving and often widely separated, direct appearance-based or proximity-based constraints cannot be used. instead, we exploit geometric constraints on the relationship between the motion of each object across cameras, to test multiple correspondence hypotheses, without assuming any prior calibration information. we propose a statistically and geometrically meaningful means of evaluating a hypothesized correspondence between two observations in different cameras. second, since multiple cameras exist, ensuring coherency in correspondence, i.e. transitive closure is maintained between more than two cameras, is an essential requirement. to ensure such coherency we pose the problem of object tracking across cameras as a k-dimensional matching and use an approximation to find the maximum likelihood assignment of correspondence. third, we show that as a result of tracking objects across the cameras, a concurrent visualization of multiple aerial video streams is possible. results are shown on a number of real and controlled scenarios with multiple objects observed by multiple cameras, validating our qualitative models.
efficient, robust and accurate fitting of a 3d morphable model. 3d morphable models, as a means to generate images of a class of objects and to analyze them, have become increasingly popular. the problematic part of this frameworkis the registration of the model to an image, a.k.a. the fitting. the characteristic features of a fitting algorithm are its efficiency, robustness, accuracy and automation. many accurate algorithms based on gradient descent techniques exist which are unfortunately short on the other features. recently, an efficient algorithm called inverse compositional image alignment (icia) algorithm, able to fit 2d images, was introduced. in this paper, we extent this algorithm to fit 3d morphable models using a novel mathematical notation which facilitates the formulation of the fitting problem. this formulation enables us to avoid a simplification so far used in the icia, being as efficient and leading to improved fitting precision. additionally, the algorithm is robust without sacrificing its efficiency and accuracy, thereby conforming to three of the four characteristics of a good fitting algorithm.
exploring the space of a human action. one of the fundamental challenges of recognizing actions is accounting for the variability that arises when arbitrary cameras capture humans performing actions. in this paper, we explicitly identify three important sources of variability: (1) viewpoint, (2) execution rate, and (3) anthropometry of actors, and propose a model of human actions that allows us to investigate all three. our hypothesis is that the variability associated with the execution of an action can be closely approximated by a linear combination of action bases in joint spatio-temporal space. we demonstrate that such a model bounds the rank of a matrix of image measurements and that this bound can be used to achieve recognition of actions based only on imaged data. a test employing principal angles between subspaces that is robust to statistical fluctuations in measurement data is presented to find the membership of an instance of an action. the algorithm is applied to recognize several actions, and promising results have been obtained.
unsupervised image translation. an interesting and potentially useful vision/graphics task is to render an input image in an enhanced form or also in an unusual style; for example with increased sharpness or with some artistic qualities. in previous work [10, 5],researchers showed that by estimating the mapping from an input image to a registered (aligned) image of the same scene in a different style or resolution, the mapping could be used to render a new input image in that style or resolution. frequently a registered pair is not available, but instead the user may have only a source image of an unrelated scene that contains the desired style. in this case, the task of inferring the output image is much more difficult since the algorithm must both infer correspondences between features in the input image and the source image, and infer the unknown mapping between the images. we describe a bayesian technique for inferring the most likely output image. the prior on the output image p ( x )is a patch-based markov random field obtained from the source image. the likelihood of the input p ( y|x )is a bayesian network that can represent different rendering styles. we describe a computationally efficient, probabilistic inference and learning algorithm for inferring the most likely output image and learning the rendering style. we also show that current techniques for image restoration or reconstruction proposed in the vision literature (e.g., image super-resolution or de-noising) and image-based non-photorealistic rendering could be seen as special cases of our model. we demonstrate our technique using several tasks, including rendering a photograph in the artistic style of an unrelated scene, de-noising, and texture transfer.
fast global kernel density mode seeking with application to localisation and tracking. we address the problem of seeking the global mode of a density function using the mean shift algorithm. mean shift, like other gradient ascent optimisation methods, is susceptible to local maxima, and hence often fails to find the desired global maximum. in this work, we propose a multi-bandwidth mean shift procedure that alleviates this problem, which we term annealed mean shift, as it shares similarities with the annealed importance sampling procedure. the bandwidth of the algorithm plays the same role as the temperature in annealing. we observe that the over-smoothed density function with a sufficiently large bandwidth is uni-modal. using a continuation principle, the influence of the global peak in the density function is introduced gradually. in this way the global maximum is more reliably located. generally, the price of this annealing-like procedure is that more iterations are required. since it is imperative that the computation complexity is minimal in real-time applications such as visual tracking. we propose an accelerated version of the mean shift algorithm. compared with the conventional mean shift algorithm, the accelerated mean shift can significantly decrease the number of iterations required for convergence. the proposed algorithm is applied to the problems of visual tracking and object localisation. we empirically show on various data sets that the proposed algorithm can reliably find the true object location when the starting position of mean shift is far away from the global maximum, in contrast with the conventional mean shift algorithm that will usually get trapped in a spurious local maximum.
learning and inferring image segmentations using the gbp typical cut algorithm. significant progress in image segmentation has beenmade by viewing the problem in the framework of graphpartitioning. in particular, spectral clustering methods suchas "normalized cuts" (ncuts) can efficiently calculate goodsegmentations using eigenvector calculations. however,spectral methods when applied to images with local connectivityoften oversegment homogenous regions. more importantly,they lack a straightforward probabilistic interpretationwhich makes it difficult to automatically set parametersusing training data.in this paper we revisit the typical cut criterion proposedin [1, 5]. we show that computing the typical cut isequivalent to performing inference in an undirected graphicalmodel. this equivalence allows us to use the powerfulmachinery of graphical models for learning and inferringimage segmentations. for inferring segmentations weshow that the generalized belief propagation (gbp) algorithmcan give excellent results with a runtime that is usuallyfaster than the ncut eigensolver. for learning segmentationswe derive a maximum likelihood learning algorithmto learn affinity matrices from labelled datasets. we illustrateboth learning and inference on challenging real andsynthetic images.
thresholding for change detection. image differencing is used for many applications involving change detection. although it is usually followed by a thresholding operation to isolate regions of change there are few methods available in the literature specific to (and appropriate for) change detection. we describe four different methods for selecting thresholds that work on very different principles. either the noise or the signal is modelled, and the model covers either the spatial or intensity distribution characteristics. the methods are: 1/ a normal model is used for the noise intensity distribution, 2/ signal intensities are tested by making local intensity distribution comparisons in the two image frames (i.e. the difference map is not used), 3/ the spatial properties of the noise are modelled by a poisson distribution, and 4/ the spatial properties of the signal are modelled as a stable number of regions (or stable euler number).
motion segmentation and tracking using normalized cuts. we propose a motion segmentation algorithm that aims to break a scene into its most prominent moving groups. a weighted graph is constructed on the image sequence by connecting pixels that are in the spatiotemporal neighborhood of each other. at each pixel, we define motion profile vectors which capture the probability distrobution of the image velocity. the distance between motion profiles is used to assign a weight on the graph edges. using normalized cuts we find the most salient partitions of the spaciotemporal graph formed by the image sequence. for segmenting long image sequences, we have developed a recursive update procedure that incorporates knowledge of segmentation in previous frames for efficiently finding the group correspondence in the new frame.
fusing points and lines for high performance tracking. this paper addresses the problem of real-time 3d model-based tracking by combining point-based and edge-based tracking systems. we present a careful analysis of the properties of these two sensor systems and show that this leads to some non-trivial design choices that collectively yield extremely high performance. in particular, we present a method for integrating the two systems and robustly combining the pose estimates they produce. further we show how on-line learning can be used to improve the performance of feature tracking. finally, to aid real-time performance, we introduce the fast feature detector which can perform full-frame feature detection at 400hz. the combination of these techniques results in a system which is capable of tracking average prediction errors of 200 pixels. this level of robustness allows us to track very rapid motions, such as 50.camera shake at 6hz.
a model-based integrated approach to track myocardial deformation using displacement and velocity constraints. accurate estimation of heart wall dense field motion and deformation could help to better understand the physiological processes associated with ischemic heart diseases, and to provide significant improvement in patient treatment. we present a new method of estimating left ventricular deformation which integrates instantaneous velocity information obtained within the mid-wall region with shape information found on the boundaries of the left ventricle. velocity information is obtained from phase contrast magnetic resonance images, and boundary information is obtained from shape-based motion tracking of the endo- and cardial boundaries. the integration takes place within a continuum biomechanical heart model which is embedded in a finite element framework. we also employ a feedback mechanism to improve tracking accuracy. the integration of the two disparate but complementary sources overcomes some of the limitations of previous work in the field which concentrates on motion estimation from a single image-derived source.
on the spatial statistics of optical flow. we develop a method for learning the spatial statistics of optical flow fields from a novel training database. training flow fields are constructed using range images of natural scenes and 3d camera motions recovered from hand-held and car-mounted video sequences. a detailed analysis of optical flow statistics in natural scenes is presented and machine learning methods are developed to learn a markov random field model of optical flow. the prior probability of a flow field is formulated as a field-of-experts model that captures the higher order spatial statistics in overlapping patches and is trained using contrastive divergence. this new optical flow prior is compared with previous robust priors and is incorporated into a recent, accurate algorithm for dense optical flow computation. experiments with natural and synthetic sequences illustrate how the learned optical flow prior quantitatively improves flow accuracy and how it captures the rich spatial structure found in natural scene motion.
learning non-generative grammatical models for document analysis. we present a general approach for the hierarchical segmentation and labeling of document layout structures. this approach models document layout as a grammar and performs a global search for the optimal parse based on a grammatical cost function. our contribution is to utilize machine learning to discriminatively select features and set all parameters in the parsing process. therefore, and unlike many other approaches for layout analysis, ours can easily adapt itself to a variety of document analysis problems. one need only specify the page grammar and provide a set of correctly labeled pages. we apply this technique to two document image analysis tasks: page layout structure extraction and mathematical expression interpretation. experiments demonstrate that the learned grammars can be used to extract the document structure in 57 files from the uwiii document image database. we also show that the same framework can be used to automatically interpret printed mathematical expressions so as to recreate the original latex.
linear multi-view reconstruction of points, lines, planes and cameras using a reference plane. this paper presents a new linear method for reconstructingsimultaneously 3d features (points, lines and planes)and cameras from many perspective views by solving a singlelinear system. it assumes that a real or virtual referenceplane is visible in all views. we call it the direct referenceplane (drp) method. it is well known that the projection relationshipbetween uncalibrated cameras and 3d featuresis non-linear in the absence of a reference plane. with aknown reference plane, points and cameras have a linearrelationship, as shown in [16]. the main contribution ofthis paper is that lines and cameras, as well as, planes andcameras also have a linear relationship. consequently, all3d features and all cameras can be reconstructed simultaneouslyfrom a single linear system, which handles missingimage measurements naturally. a further contribution is anextensive experimental comparison, using real data, of differentreference plane and non-reference plane reconstructionmethods. for difficult reference plane scenarios, withpoint or line features, the drp method is superior to allcompared methods. finally, an extensive list of referenceplane scenarios is presented, which shows the wide applicabilityof the drp method.
probabilistic 3d object recognition. a probabilistic 3d object recognition algorithm is presented. in order to guide the recognition process the probability that match hypotheses between image features and model features are correct is computed. a model is developed which uses the probabilistic peaking effect of measured angles and ratios of lengths by tracing iso angle and iso ratio curves on the viewing sphere. the model also accounts for various types of uncertainty in the input such as incomplete and inexact edge detection. for each match hypothesis the pose of the object and the pose uncertainty which is due to the uncertainty in vertex position are recovered. this is used to find sets of hypotheses which reinforce each other by matching features of the same object with compatible uncertainty subsets. a probabalistic expression is used to rank these hypothesis sets. the hypothesis sets with the highest rank are output. the algorithm has been fully implemented, and tested on real images.
a comparison of projective reconstruction methods for pairs of views. recently, different approaches for uncalibrated stereo have been suggested which permit projective reconstruction from multiple views. these use weak calibration which is represented by the epipolar geometry, and so no knowledge of the intrinsic or extrinsic camera parameters is required. we consider projective reconstructions from pairs of views, and compare a number of the available methods. consequently we conclude which methods are most likely to be of use in applications that are dependent on 3d uncalibrated reconstructions.
non-negative lighting and specular object recognition. recognition of specular objects is particularly difficult because their appearance is much more sensitive to lighting changes than that of lambertian objects. we consider an approach in which we use a 3d model to deduce the lighting that best matches the model to the image. in this case, an important constraint is that incident lighting should be non-negative everywhere. in this paper, we propose a new method to enforce this constraint and explore its usefulness in specular object recognition, using the spherical harmonic representation of lighting. the method follows from a novel extension of szego¿s eigenvalue distribution theorem to spherical harmonics, and uses semidefinite programming to perform a constrained optimization. the new method is faster as well as more accurate than previous methods. experiments on both synthetic and real data indicate that the constraint can improve recognition of specular objects by better separating the correct and incorrect models.