on the estimation of optical flow relations between different approaches and some new results abstract a common basis is suggested for the optical flow estimation approaches of nagel 1983 haralick and lee 1983 and tretiak and pastor 1984 based on a discussion of these approaches an exact solution for the system of partial differential equations proposed by horn and schunck 1981 is given at gray value corners and extrema the insight gained by this solution results in a modification of the oriented smoothness approach of nagel 1983 which thereby becomes considerably simpler in addition the optical flow estimation approach of hildreth 1983 1984 can be shown to represent a kind of special case of this modified oriented smoothness approach in a more direct manner than discussed in nagel 1984

computing optical flow via variational techniques defined as the apparent motion in a sequence of images the optical flow is very important in the computer vision community where its accurate estimation is necessary for many applications it is o

adaptive grid refinement procedures for efficient optical flow computation two approaches are described that improve the efficiency of optical flow computation without incurring loss of accuracy the first approach segments images into regions of moving objects the method is based on a previously defined galerkin finite element method on a triangular mesh combined with a multiresolution segmentation approach for object flow computation images are automatically segmented into subdomains of moving objects by an algorithm that employs a hierarchy of mesh coarseness for the flow computation and these subdomains are reconstructed over a finer mesh on which to recompute flow more accurately the second approach uses an adaptive mesh in which the resolution increases where motion is found to occur optical flow is computed over a reasonably coarse mesh and this is used to construct an optimal adaptive mesh in a way that is different from the gradient methods reported in the literature the finite element mesh facilitates a reduction in computational effort by enabling processing to focus on particular objects of interest in a scene i e those areas where motion is detected the proposed methods were tested on real and synthetic image sequences and promising results are reported

frequency domain analysis and synthesis of image pyramid generating kernels construction of image pyramids is described as a two di mensional decimation process frequently employed generating kernels are compared to the optimal kernel that assures minimal information loss after the resolution reduction i e the one corresponding to an ideal low pass filter physically realizable optimal generating kernels are presented the amount of computation required for generation of the image pyramid can be reduced significantly by employing half band filters as components of the optimal kernel image pyramids generated by the optimal kernel show a better command of details than the ones generated by a simple 4 4 averaging or a computationally equivalent kernel

computation of discontinuous optical flow by domain decomposition and shape optimization a well known method for the reconstruction of motion fields from noisy image data is to determine flow fields by the minimization of a quadratic functional the first approach of this class has been proposed by horn and schunck 1981 a drawback of such approaches is that an explicit representation of the discontinuities of the motion field is lacking and that in general the resulting flow fields approximate the motion fields only badly at the corresponding locations in the image plane in this article we discuss the possibility to improve the results by hypothesizing the qualitative structure of the motion field in terms of certain parameters we decompose the image plane into disjoint sets restrict the domain of definition of the functionals to these sets and use the hypotheses to deform and to move the boundaries of the sets within the image plane we discuss the range of applicability of this new technique and illustrate the algorithm by numerical examples this article is a revised and extended version of schnorr 1990

generalised nonlocal image smoothing we propose a discrete variational approach for image smoothing consisting of nonlocal data and smoothness constraints that penalise general dissimilarity measures defined on image patches one of such dissimilarity measures is the weighted l 2 distance between patches in such a case we derive an iterative neighbourhood filter that induces a new similarity measure in the photometric domain it can be regarded as an extended patch similarity measure that evaluates not only the patch similarity of two chosen pixels but also the similarity of their corresponding neighbours this leads to a more robust smoothing process since the pixels selected for averaging are more coherent with the local image structure by slightly modifying the way the similarities are computed we obtain two related filters the nl means filter of buades et al siam multiscale model simul 4 2 490 530 2005b and the nds filter of mrazek et al geometric properties for incomplete data computational imaging and vision vol 31 pp 335 352 springer dordrecht 2006 in fact the proposed approach can be considered as a generalisation of the latter filter to the space of patches we also provide novel insights into relations of the nds filter with diffusion regularisation methods as well as with some recently proposed graph regularisation techniques we evaluate our method for the task of denoising greyscale and colour images degraded with gaussian and salt and pepper noise demonstrating that it compares very well to other more sophisticated approaches

determining optical flow for irregular domains by minimizing quadratic functionals of a certain class snyder 1989 has recently classified all smoothness terms which involve first order derivatives of the flowfield u x t and of the image grey value function g x t the physically plausible smoothness terms belonging to this class are known from the work of horn and schunck 1981 and nagel 1987 in this paper we discuss the possibilities of approximating the solutions to the minimization problems of horn schunk 1981 and nagel 1987 in particular it is shown that these solutions exist are unique and depend continuously on the input data these properties make it possible while taking into consideration arbitrary models of the grey value function to approximate efficiently the weak solutions of the associated boundary value problems in irregularly shaped domains with a sufficiently smooth boundary using finite elements experiments with image sequences from synthetic as well as outdoor scenes show how the orientation dependency of the smoothness term in nagel s approach influences the results

analysis and improvement of particle swarm optimization algorithm the effects of inertia weight on particle swarm optimization pso performance are analyzed a novel method of selecting inertia weight in pso is developed which can tune the expectations of inertia weights adaptively when the inertia weights are randomly selected and lead to effectively balance between the local and global search ability results of the two benchmark functions indicate that the pso algorithm based on the strategy of random inertia weight riw has been significantly improved on both optimization speed and computational accuracy

scene segmentation from visual motion using global optimization this paper presents results from computer experiments with an algorithm to perform scene disposition and motion segmentation from visual motion or optic flow the maximum a posteriori map criterion is used to formulate what the best segmentation or interpretation of the scene should be where the scene is assumed to be made up of some fixed number of moving planar surface patches the bayesian approach requires first specification of prior expectations for the optic flow field which here is modeled as spatial and temporal markov random fields and secondly a way of measuring how well the segmentation predicts the measured flow field the markov random fields incorporate the physical constraints that objects and their images are probably spatially continuous and that their images are likely to move quite smoothly across the image plane to compute the flow predicted by the segmentation a recent method for reconstructing the motion and orientation of planar surface facets is used the search for the globally optimal segmentation is performed using simulated annealing

a hierarchical optical flow estimation algorithm based on the interlevel motion smoothness constraint abstract an optical flow estimation algorithm is proposed based on the multiresolution representation of the image i e the image pyramid since the image pyramid consists of several low pass filtered versions of the image sequence the motion vector at each pyramid level is also the low pass filtered versions of the motion vector at the finest level thus the interlevel motion smoothness constraint is introduced in the motion estimation by defining the operators to establish an appropriate relationship between two adjacent pyramid levels the employment of the image pyramid allows us to estimate the large flow vectors and to adopt easily the multigrid technique to improve the convergence speed of the proposed algorithm the simulation results reveal that the proposed algorithm provides more accurate motion estimation compared to the methods using the smoothness constraints such as horn and schunck s algorithm in addition the theoretical analysis of the convergence behavior of the proposed algorithm is also provided in this paper

direct analytical methods for solving poisson equations in computer vision problems direct analytical methods are discussed for solving poisson equations of the general form delta u f on a rectangular domain some embedding techniques that may be useful when boundary conditions obtained from stereo and occluding boundary are defined on arbitrary contours are described the suggested algorithms are computationally efficient owing to the use of fast orthogonal transforms applications to shape from shading lightness and optical flow problems are also discussed a proof for the existence and convergence of the flow estimates is given experiments using synthetic images indicate that results comparable to those using multigrid can be obtained in a very small number of iterations

motion estimation with quadtree splines this paper presents a motion estimation algorithm based on a new multiresolution representation the quadtree spline this representation describes the motion field as a collection of smoothly connected patches of varying size where the patch size is automatically adapted to the complexity of the underlying motion the topology of the patches is determined by a quadtree data structure and both split and merge techniques are developed for estimating this spatial subdivision the quadtree spline is implemented using another novel representation the adaptive hierarchical basis spline and combines the advantages of adaptively sized correlation windows with the speedups obtained with hierarchical basis preconditioners results are presented on some standard motion sequences

self adaptive regularization often an image g x y is regularized and even restored by minimizing the mumford shah functional properties of the regularized image u x y depends critically on the numerical value of the two parameters spl alpha and spl beta controlling smoothness and fidelity when spl alpha and spl beta are constant over the image small details are lost when an extensive filtering is used in order to remove noise in this paper it is shown how the two parameters spl alpha and spl beta can be made self adaptive in fact spl alpha and spl beta are not constant but automatically adapt to the local scale and contrast of features in the image in this way edges at all scales are detected and boundaries are well localized and preserved in order to preserve trihedral junctions spl alpha and spl beta become locally small and the regularized image u x y maintains sharp and well defined trihedral junctions images regularized by the proposed procedure are well suited for further processing such as image segmentation and object recognition

optical flow a curve evolution approach a novel approach for the computation of optical flow based on an l sup 1 type minimization is presented it is shown that the approach has inherent advantages since it does not smooth the flow velocity across the edges and hence preserves edge information a numerical approach based on computation of evolving curves is proposed for computing the optical flow field computations are carried out on a number of real image sequences in order to illustrate the theory as well as the numerical approach

a theoretical framework for convex regularizers in pde based computation of image motion many differential methods for the recovery of the optic flow field from an image sequence can be expressed in terms of a variational problem where the optic flow minimizes some energy typically these energy functionals consist of two terms a data term which requires e g that a brightness constancy assumption holds and a regularizer that encourages global or piecewise smoothness of the flow field in this paper we present a systematic classification of rotation invariant convex regularizers by exploring their connection to diffusion filters for multichannel images this taxonomy provides a unifying framework for data driven and flow driven isotropic and anisotropic as well as spatial and spatio temporal regularizers while some of these techniques are classic methods from the literature others are derived here for the first time we prove that all these methods are well posed they posses a unique solution that depends in a continuous way on the initial data an interesting structural relation between isotropic and anisotropic flow driven regularizers is identified and a design criterion is proposed for constructing anisotropic flow driven regularizers in a simple and direct way from isotropic ones its use is illustrated by several examples

symmetrical dense optical flow estimation with occlusions detection traditional techniques of dense optical flow estimation do not generally yield symmetrical solutions the results will differ if they are applied between images i 1 and i 2 or between images i 2 and i 1 in this work we present a method to recover a dense optical flow field map from two images while explicitely taking into account the symmetry across the images as well as possible occlusions in the flow field the idea is to consider both displacements vectors from i 1 to i 2 and i 2 to i 1 and to minimise an energy functional that explicitely encodes all those properties this variational problem is then solved using the gradient flow defined by the euler lagrange equations associated to the energy to prove the importance of the concepts of symmetry and occlusions for optical flow computation we have extended a classical approach to handle those experiments clearly show the added value of these properties to improve the accuracy of the computed flows figures appear in color in the online version of this paper

reliable estimation of dense optical flow fields with large displacements in this paper we show that a classic optical flow technique by nagel and enkelmann 1986 ieee trans pattern anal mach intell vol 8 pp 565 593 can be regarded as an early anisotropic diffusion method with a diffusion tensor we introduce three improvements into the model formulation that i avoid inconsistencies caused by centering the brightness term and the smoothness term in different images ii use a linear scale space focusing strategy from coarse to fine scales for avoiding convergence to physically irrelevant local minima and iii create an energy functional that is invariant under linear brightness changes applying a gradient descent method to the resulting energy functional leads to a system of diffusion reaction equations we prove that this system has a unique solution under realistic assumptions on the initial data and we present an efficient linear implicit numerical scheme in detail our method creates flow fields with 100 density over the entire image domain it is robust under a large range of parameter variations and it can recover displacement fields that are far beyond the typical one pixel limits which are characteristic for many differential methods for determining optical flow we show that it performs better than the optical flow methods with 100 density that are evaluated by barron et al 1994 int j comput vision vol 12 pp 43 47 our software is available from the internet

an analysis of physics based optical flow a variational solution of the physics based optical flow equation is studied for extraction of high resolution velocity fields from flow visualization images the solution to preserve discontinuities in a fluid velocity field is mathematically justified the uniqueness of the solution and the convergence of a successive approximation sequence are proven a numerical algorithm is developed and examined through numerical experiments

image analysis using multigrid relaxation methods image analysis problems posed mathematically as variational principles or as partial differential equations are amenable to numerical solution by relaxation algorithms that are local iterative and often parallel although they are well suited structurally for implementation on massively parallel locally interconnected computational architectures such distributed algorithms are seriously handi capped by an inherent inefficiency at propagating constraints between widely separated processing elements hence they converge extremely slowly when confronted by the large representations of early vision application of multigrid methods can overcome this drawback as we showed in previous work on 3 d surface reconstruction in this paper we develop multiresolution iterative algorithms for computing lightness shape from shading and optical flow and we examine the efficiency of these algorithms using synthetic image inputs the multigrid methodology that we describe is broadly applicable in early vision notably it is an appealing strategy to use in conjunction with regularization analysis for the efficient solution of a wide range of ill posed image analysis problems

combining total variation regularization with window based time delay estimation in ultrasound elastography a major challenge of free hand palpation ultrasound elastography use is estimating the displacement of rf samples between pre and post compressed rf data the problem of displacement estimation is ill posed since the displacement of one sample by itself cannot be uniquely calculated to resolve this problem two categories of methods have emerged the first category assumes that the displacement of samples within a small window surrounding the reference sample is constant the second class imposes smoothness regularization and optimizes an energy function herein we propose a novel method that combines both approaches and as such is more robust to noise the second contribution of this work is the introduction of the l1 norm as the regularization term in our cost function which is often referred to as the total variation tv regularization compared to previous work that used the l2 norm regularization optimization of the new cost function is more challenging however the advantages of using the l1 norm are twofold first it leads to substantial improvement in the sharpness of displacement estimates second to optimize the cost function with the l1 norm regularization we use an iterative method that further increases the robustness we name our proposed method total variation regularization and window based time delay estimation overwind and show that it is robust to signal decorrelation and generates sharp displacement and strain maps for simulated experimental phantom and in vivo data in particular overwind improves strain contrast to noise ratio cnr by 27 26 144 05 and 49 90 on average in simulation phantom and in vivo data respectively compared to our recent global ultrasound elastography glue method

using geometric distance fits for 3 d object modeling and recognition addresses the problems of automatically constructing algebraic surface models from sets of 2d and 3d images and using these models in pose computation motion and deformation estimation and object recognition we propose using a combination of constrained optimization and nonlinear least squares estimation techniques to minimize the mean squared geometric distance between a set of points or rays and a parameterized surface in modeling tasks the unknown parameters are the surface coefficients while in pose and deformation estimation tasks they represent the transformation which maps the observer s coordinate system onto the modeled surface s own coordinate system we have applied this approach to a variety of real range computerized tomography and video images

dense estimation and object based segmentation of the optical flow with robust techniques we address the issue of recovering and segmenting the apparent velocity field in sequences of images as for motion estimation we minimize an objective function involving two robust terms the first one cautiously captures the optical flow constraint while the second a priori term incorporates a discontinuity preserving smoothness constraint to cope with the nonconvex minimization problem thus defined we design an efficient deterministic multigrid procedure it converges fast toward estimates of good quality while revealing the large discontinuity structures of flow fields we then propose an extension of the model by attaching to it a flexible object based segmentation device based on deformable closed curves different families of curve equipped with different kinds of prior can be easily supported experimental results on synthetic and natural sequences are presented including an analysis of sensitivity to parameter tuning

optical flow with an intensity weighted smoothing matching algorithms which use image structure such as edges and junctions cannot perform well on unstructured images for such images global gradient based methods may be more appropriate some of the assumptions involved in applying discretized conservation equations such as the optical flow equation are discussed and an intensity weighted method of estimating dense displacement fields is presented which attempts to avoid some of the problems of the standard methods the only derivative required is the intensity gradient and the method includes procedures for automatic parameter evaluation the algorithm is expected to perform better than conventional algorithms on images without strong texture as is demonstrated on the test set of images

on the mathematical foundations of smoothness constraints for the determination of optical flow and for surface reconstruction gradient based approaches to the computation of optical flow often use a minimization technique incorporating a smoothness constraint on the optical flow field the author derives the most general form of such a smoothness constraint that is quadratic in first derivatives of the grey level image intensity function based on three simple assumptions about the smoothness constraint 1 it must be expressed in a form that is independent of the choice of cartesian coordinate system in the image 2 it must be positive definite and 3 it must not couple different component of the optical flow it is shown that there are essentially only four such constraints any smoothness constraint satisfying 1 2 or 3 must be a linear combination of these four possibly multiplied by certain quantities invariant under a change in the cartesian coordinate system beginning with the three assumptions mentioned above the author mathematically demonstrates that all best known smoothness constraints appearing in the literature are special cases of this general form and in particular that the weight matrix introduced by h h nagel is essentially modulo invariant quantities the only physically plausible such constraint

on functionals with greyvalue controlled smoothness terms for determining optical flow the modification by h h nagel 1987 of the approach developed by b k p horn and b g schunck 1981 for determining optical flow is generalized to the case where local motion information is given by more than one constraint equation applying this scheme to three constraint equations reported in the literature as a special case a generalization of nagel s approach is obtained an existence and uniqueness result of solutions under very general conditions that in turn ensures the applicability of standard techniques to compute an approximate solution is presented

an investigation of smoothness constraints for the estimation of displacement vector fields from image sequences a mapping between one frame from an image sequence and the preceding or following frame can be represented as a displacement vector field in most situations the mere gray value variations do not provide sufficient information in order to estimate such a displacement vector field supplementary constraints are necessary for example the postulate that a displacement vector field varies smoothly as a function of the image position taken as a general requirement this creates difficulties at gray value transitions which correspond to occluding contours nagel therefore introduced the oriented smoothness requirement which restricts variations of the displacement vector field only in directions with small or no variation of gray values this contribution reports results of an investigation about how such an oriented smoothness constraint may be formulated and evaluated

estimation of optical flow based on higher order spatiotemporal derivatives in interlaced and non interlaced image sequences abstract this contribution 1 investigates local differential techniques for estimating optical flow and its derivatives based on the brightness change constraint by using the tensor calculus representation we build the taylor expansion of the gray value derivatives as well as of the optical flow in a spatiotemporal neighborhood such a formulation provides a unifying framework for all existing local differential approaches and allows to derive new systems of equations for the estimation of the optical flow and of its derivatives we also tested various optical flow estimation approaches on real image sequences recorded by a calibrated camera which was fixed on the arm of a robot by moving the arm of the robot along a precisely defined trajectory we can determine the true displacement rate of scene surface elements projected into the image plane and compare it quantitatively with the results of different optical flow estimators since the optical flow estimators are based on gray value derivatives of up to fourth order we were forced to develop modified gaussian derivative filters to obtain acceptable estimates for the derivatives further we show quantitatively that these filters contribute to a much more robust optical flow estimation in addition successive lines of tv cameras have an offset in time due to the interlace technique we demonstrate the adaptation of filter kernels for estimating higher order spatiotemporal derivatives in interlaced image sequences

particle swarm optimization for determining fuzzy measures from data fuzzy measures and fuzzy integrals have been successfully used in many real applications how to determine fuzzy measures is the most difficult problem in these applications though there have existed some methodologies for solving this problem such as genetic algorithms gradient descent algorithms and neural networks it is hard to say which one is more appropriate and more feasible each method has its advantages and limitations therefore it is necessary to develop new methods or techniques to learn distinct fuzzy measures in this paper we make the first attempt to design a special particle swarm algorithm to determine a type of general fuzzy measures from data and demonstrate that the algorithm is effective and efficient furthermore we extend this algorithm to identify and revise other types of fuzzy measures to test our algorithms we compare them with the basic particle swarm algorithms gradient descent algorithms and genetic algorithms in literatures in addition for verifying whether our algorithms are robust in noisy situations a number of numerical experiments are conducted theoretical analysis and experimental results show that for determining fuzzy measures the particle swarm optimization is feasible and has a better performance than the existing genetic algorithms and gradient descent algorithms

a computational framework and an algorithm for the measurement of visual the robust measurement of visual motion from digitized image sequences has been an important but difficult problem in computer vision this paper describes a hierarchical computational framework for the determination of dense displacement fields from a pair of images and an algorithm consist ent with that framework our framework is based on the separation of the image intensity information as well as the process of measuring motion according to scale the large scale intensity information is first used to obtain rough estimates of image motion which are then refined by using intensity information at smaller scales the estimates are in the form of displacement or velocity vectors for pixels and are accompanied by a direction dependent confidence measure a smoothness constraint is employed to propagate the measurements with high confidence to their neigboring areas where the confidences are low at all levels the computations are pixel parallel uniform across the image and based on information from a small neighborhood of a pixel for our algorithm the local displacement vectors are determind by mini mizing the sum of squared differences ssd of intensities the confidence measures are derived from the shape of the ssd surface and the smoothness constraint is cast in the form of energy minimization results of applying our algorithm to pairs of real images are included in addition to our own

autonomous construction of three dimensional models from range data a substantial body of literature exists in the field of model based vision and object recognition a model based vision system assumes that some kind of model exists for each object the system is to recognize unfortunately in the past each new recognition scheme has frequently required its own specialized models this in turn has required that a time consuming and often manual model construction process be completed first development of an automated method for constructing reasonably detailed general purpose models would both simplify the model construction process and provide models that could be used by many different recognition schemes this dissertation presents a method for autonomously constructing a complete general purpose surface description model of an object the models are constructed based on a large number of noisy range images from an essentially arbitrary set of viewpoints around the object the resulting models consist of a set of parametric surface patches results of testing the model building algorithm on six synthetic test objects are also presented in addition an object recognition scheme is presented which uses models constructed by the model builder the object recognition results clearly demonstrate that the models constructed by the system are of sufficient accuracy and detail for use in model based vision tasks

lucas kanade meets horn schunck combining local and global optic flow methods differential methods belong to the most widely used techniques for optic flow computation in image sequences they can be classified into local methods such as the lucas kanade technique or bigun s structure tensor method and into global methods such as the horn schunck approach and its extensions often local methods are more robust under noise while global techniques yield dense flow fields the goal of this paper is to contribute to a better understanding and the design of novel differential methods in four ways i we juxtapose the role of smoothing regularisation processes that are required in local and global differential methods for optic flow computation ii this discussion motivates us to describe and evaluate a novel method that combines important advantages of local and global approaches it yields dense flow fields that are robust against noise iii spatiotemporal and nonlinear extensions as well as multiresolution frameworks are presented for this hybrid method iv we propose a simple confidence measure for optic flow methods that minimise energy functionals it allows to sparsify a dense flow field gradually depending on the reliability required for the resulting flow comparisons with experiments from the literature demonstrate the favourable performance of the proposed methods and the confidence measure

robust optical flow computation based on least median of squares regression an optical flow estimation technique is presented which is based on the least median of squares lmeds robust regression algorithm enabling more accurate flow estimates to be computed in the vicinity of motion discontinuities the flow is computed in a blockwise fashion using an affine model through the use of overlapping blocks coupled with a block shifting strategy redundancy is introduced into the computation of the flow this eliminates blocking effects common in most other techniques based on blockwise processing and also allows flow to be accurately computed in regions containing three distinct motions a multiresolution version of the technique is also presented again based on lmeds regression which enables image sequences containing large motions to be effectively handled an extensive set of quantitative comparisons with a wide range of previously published methods are carried out using synthetic realistic computer generated images of natural scenes with known flow and natural images both angular and absolute flow errors are calculated for those sequences with known optical flow displaced frame difference error used extensively in video compression is used for those natural scenes with unknown flow in all of the sequences tested a comparison with those methods that result in a dense flow field greater than 80 spatial coverage show that the lmeds technique produces the least error irrespective of the error measure used

a modified particle swarm optimizer with dynamic adaptation this paper proposes a modified particle swarm optimization algorithm with dynamic adaptation in this algorithm a modified velocity updating formula of the particle is used where the randomness in the course of updating particle velocity is relatively decreased and the inertia weight of each particle is different moreover this algorithm introduces two parameter describing the evolving state of the algorithm the evolution speed factor and aggregation degree factor by analyzing the influence of two parameters on the pso search ability a new strategy is presented that the inertia weight dynamically changes based on the run and evolution state in the strategy the inertia weight is given by a function of evolution speed factor and aggregation degree factor and the value of inertia weight is dynamically adjusted according to the evolution speed and aggregation degree the feature of the proposed algorithm is analyzed and several testing functions are performed in simulation study experimental results show that the proposed algorithm remarkably improves the ability of pso to jump out of the local optima and significantly enhance the convergence precision

computation and analysis of image motion a synopsis of current problems and methods the goal of this paper is to offer a structured synopsis of the problems in image motion computation and analysis and of the methods proposed exposing the underlying models and supporting assumptions a sufficient number of pointers to the literature will be given concentrating mostly on recent contributions emphasis will be on the detection measurement and segmentation of image motion tracking and deformable motion issues will be also addressed finally a number of related questions which could require more investigations will be presented

domain decomposition for variational optical flow computation we present an approach to parallel variational optical flow computation by using an arbitrary partition of the image plane and iteratively solving related local variational problems associated with each subdomain the approach is particularly suited for implementations on pc clusters because interprocess communication is minimized by restricting the exchange of data to a lower dimensional interface our mathematical formulation supports various generalizations to linear nonlinear convex variational approaches three dimensional image sequences spatiotemporal regularization and unstructured geometries and triangulations results concerning the effects of interface preconditioning as well as runtime and communication volume measurements on a pc cluster are presented our approach provides a major step toward real time two dimensional image processing using off the shelf pc hardware and facilitates the efficient application of variational approaches to large scale image processing problems

computation of component image velocity from local phase information we present a technique for the computation of 2d component velocity from image sequences initially the image sequence is represented by a family of spatiotemporal velocity tuned linear filters component velocity computed from spatiotemporal responses of identically tuned filters is expressed in terms of the local first order behavior of surfaces of constant phase justification for this definition is discussed from the perspectives of both 2d image translation and deviations from translation that are typical in perspective projections of 3d scenes the resulting technique is predominantly linear efficient and suitable for parallel processing moreover it is local in space time robust with respect to noise and permits multiple estimates within a single neighborhood promising quantiative results are reported from experiments with realistic image sequences including cases with sizeable perspective deformation

multimodal estimation of discontinuous optical flow using markov random fields the estimation of dense velocity fields from image sequences is basically an ill posed problem primarily because the data only partially constrain the solution it is rendered especially difficult by the presence of motion boundaries and occlusion regions which are not taken into account by standard regularization approaches in this paper the authors present a multimodal approach to the problem of motion estimation in which the computation of visual motion is based on several complementary constraints it is shown that multiple constraints can provide more accurate flow estimation in a wide range of circumstances the theoretical framework relies on bayesian estimation associated with global statistical models namely markov random fields the constraints introduced here aim to address the following issues optical flow estimation while preserving motion boundaries processing of occlusion regions fusion between gradient and feature based motion constraint equations deterministic relaxation algorithms are used to merge information and to provide a solution to the maximum a posteriori estimation of the unknown dense motion field the algorithm is well suited to a multiresolution implementation which brings an appreciable speed up as well as a significant improvement of estimation when large displacements are present in the scene experiments on synthetic and real world image sequences are reported

multiscale modeling and estimation of motion fields for video coding we present a systematic approach to forward motion compensated predictive video coding the first step is the definition of a flexible model that compactly represents motion fields the inhomogeneity and spatial coherence properties of motion fields are captured using linear multiscale models one possible design is based on linear finite elements and yields a multiscale extension of the triangle motion compensation tmc method the second step is the choice of a computational technique that identifies the coefficients of the linear model we study a modified optical flow technique and minimize a cost function closely related to horn and schunck s 1981 criterion the cost function balances accuracy and complexity of the motion compensated predictor and is viewed as a measure of goodness of the motion field it determines not only the coefficients of the model but also the quantization method we formulate the estimation and quantization problems jointly as a discrete optimization problem and solve it using a fast multiscale relaxation algorithm a hierarchical extension of the algorithm allows proper handling of large displacements simulations on a variety of video sequences have produced improvements over tmc and over the half pel accuracy full search block matching algorithm in excess of 0 5 db in average the results are visually superior as well in particular the reconstructed video is entirely free of blocking artifacts

structural indexing efficient 3 d object recognition the authors present an approach for the recognition of multiple 3 d object models from three 3 d scene data the approach uses two different types of primitives for matching small surface patches where differential properties can be reliably computed and lines corresponding to depth or orientation discontinuities these are represented by splashes and 3 d curves respectively it is shown how both of these primitives can be encoded by a set of super segments consisting of connected linear segments these super segments are entered into a table and provide the essential mechanism for fast retrieval and matching the issues of robustness and stability of the features are addressed in detail the acquisition of the 3 d models is performed automatically by computing splashes in highly structured areas of the objects and by using boundary and surface edges for the generation of 3 d curves the authors present results with the current system 3 d object recognition based on super segments and discuss further extensions

optical flow and advection on 2 riemannian manifolds a common framework dynamic pattern analysis and motion extraction can be efficiently addressed using optical flow techniques this paper presents a generalization of these questions to nonflat surfaces where optical flow is tackled through the problem of evolution processes on non euclidean domains the classical equations of optical flow in the euclidean case are transposed to the theoretical framework of differential geometry we adopt this formulation for the regularized optical flow problem prove its mathematical well posedness and combine it with the advection equation the optical flow and advection problems are dual a motion field may be retrieved from some scalar evolution using optical flow conversely a scalar field may be deduced from a velocity field using advection these principles are illustrated with qualitative and quantitative evaluations from numerical simulations bridging both approaches the proof of concept is further demonstrated with preliminary results from time resolved functional brain imaging data where organized propagations of cortical activation patterns are evidenced using our approach

inferring segmented dense motion layers using 5d tensor voting we present a novel local spatiotemporal approach to produce motion segmentation and dense temporal trajectories from an image sequence a common representation of image sequences is a 3d spatiotemporal volume x y t and its corresponding mathematical formalism is the fiber bundle however directly enforcing the spatiotemporal smoothness constraint is difficult in the fiber bundle representation thus we convert the representation into a new 5d space x y t vx vy with an additional velocity domain where each moving object produces a separate 3d smooth layer the smoothness constraint is now enforced by extracting 3d layers using the tensor voting framework in a single step that solves both correspondence and segmentation simultaneously motion segmentation is achieved by identifying those layers and the dense temporal trajectories are obtained by converting the layers back into the fiber bundle representation we proceed to address three applications tracking mosaic and 3d reconstruction that are hard to solve from the video stream directly because of the segmentation and dense matching steps but become straightforward with our framework the approach does not make restrictive assumptions about the observed scene or camera motion and is therefore generally applicable we present results on a number of data sets

variational methods for multimodal image matching matching images of different modalities can be achieved by the maximization of suitable statistical similarity measures within a given class of geometric transformations handling complex nonrigid deformations in this context turns out to be particularly difficult and has attracted much attention in the last few years the thrust of this paper is that many of the existing methods for nonrigid monomodal registration that use simple criteria for comparing the intensities e g ssd can be extended to the multimodal case where more complex intensity similarity measures are necessary to this end we perform a formal computation of the variational gradient of a hierarchy of statistical similarity measures and use the results to generalize a recently proposed and very effective optical flow algorithm l alvarez j weickert and j sanchez 2000 technical report and ijcv 39 1 41 56 to the case of multimodal image registration our method readily extends to the case of locally computed similarity measures thus providing the flexibility to cope with spatial non stationarities in the way the intensities in the two images are related the well posedness of the resulting equations is proved in a complementary work o d faugeras and g hermosillo 2001 technical report 4235 inria using well established techniques in functional analysis we briefly describe our numerical implementation of these equations and show results on real and synthetic data