analysing images of curved surfaces a reflectance map makes the relationship between image intensity and surface orientation explicit trade offs between image intensity and surface orientation emerge which cannot be resolved locally in a single view existing methods for determining surface orientation from a single view embody assumptions about surface curvature the hessian matrix is introduced to represent surface curvature properties of surface curvature are expressed as properties of the hessian matrix for several classes of surface image analysis simplifies this result has already been established for planar surfaces forming trihedral corners similar simplification is demonstrated for developable surfaces and for the subclass of surfaces known as generalized cones these studies help to delineate shape information that can be determined from geometric measurements at object boundaries and shape information that can be determined from intensity measurements over sections of smoothly curved surface a novel technique called photometric stereo is discussed the idea of stereo is to obtain multiple images in order to determine the underlying scene precisely in photometric stereo the viewing direction is constant multiple images are obtained by varying the incident illumination it is shown that this provides sufficient information to determine surface orientation at each image point

finite element methods for active contour models and balloons for 2 d and 3 d images the use of energy minimizing curves known as snakes to extract features of interest in images has been introduced by kass witkin and terzopoulos 1987 a balloon model was introduced by cohen 1991 as a way to generalize and solve some of the problems encountered with the original method a 3 d generalization of the balloon model as a 3 d deformable surface which evolves in 3 d images is presented it is deformed under the action of internal and external forces attracting the surface toward detected edgels by means of an attraction potential we also show properties of energy minimizing surfaces concerning their relationship with 3 d edge points to solve the minimization problem for a surface two simplified approaches are shown first defining a 3 d surface as a series of 2 d planar curves then after comparing finite element method and finite difference method in the 2 d problem we solve the 3 d model using the finite element method yielding greater stability and faster convergence this model is applied for segmenting magnetic resonance images

elastically adaptive deformable models we present a technique for the automatic adaptation of a deformable model s elastic parameters within a kalman filter framework for shape estimation applications the novelty of the technique is that the model s elastic parameters are not constant but spatio temporally varying the variation of the elastic parameters depends on the distance of the model from the data and the rate of change of this distance each pass of the algorithm uses physics based modeling techniques to iteratively adjust both the geometric and the elastic degrees of freedom of the model in response to forces that are computed from the discrepancy between the model and the data by augmenting the state equations of an extended kalman filter to incorporate these additional variables we are able to significantly improve the quality of the shape estimation therefore the model s elastic parameters are always initialized to the same value and they are subsequently modified depending on the data and the noise distribution we present results demonstrating the effectiveness of our method for both two dimensional and three dimensional data

physically based adaptive preconditioning for early vision several problems in early vision have been formulated in the past in a regularization framework these problems when discretized lead to large sparse linear systems in this paper we present a novel physically based adaptive preconditioning technique which can be used in conjunction with a conjugate gradient algorithm to dramatically improve the speed of convergence for solving the aforementioned linear systems a preconditioner based on the membrane spline or the thin plate spline or a convex combination of the two is termed a physically based preconditioner for obvious reasons the adaptation of the preconditioner to an early vision problem is achieved via the explicit use of the spectral characteristics of the regularization filter in conjunction with the data this spectral function is used to modulate the frequency characteristics of a chosen wavelet basis and these modulated values are then used in the construction of our preconditioner we present the preconditioner construction for three different early vision problems namely the surface reconstruction the shape from shading and the optical flow computation problems performance of the preconditioning scheme is demonstrated via experiments on synthetic and real data sets

pattern recognition by graph matching using the potts mft neural networks this paper is concerned with programming of the potts mean field theory neural networks for pattern recognition by homomorphic mapping of the attributed relational graphs arg in order to generate the homomorphic mapping from the scene relational graph to the model graph we make use of the recently introduced suganthan technical report nanyang technical university 1994 compatibility functions in relation to the hopfield network an efficient pose clustering algorithm is used to separate and localize different occurrences of any particular object model in the scene the pose clustering algorithm also eliminates spurious hypotheses generated by the network and resolves ambiguities in the final interpretation the performance of the proposed approach to pattern recognition by homomorphism is demonstrated using a number of line patterns silhouette images and circle patterns

geometric multigrid for high order regularizations of early vision problems abstract the surface estimation problem is used as a model to demonstrate a framework for solving early vision problems by high order regularization with natural boundary conditions because the application of algebraic multigrid is usually constrained by an m matrix condition which does not hold for discretizations of high order problems a geometric multigrid framework is developed for the efficient solution of the associated optimality systems it is shown that the convergence criteria of hackbusch w hackbusch iterative solution of large sparse systems of equations springer 1993 are met and in particular the general elliptic regularity required is proved further the galerkin formalism is used together with a multicolored ordering of unknowns to permit vectorization of a symmetric gauss seidel relaxation in image processing systems the implementation is analyzed computationally and inaccuracies are corrected by lumping and by proper floating point representations direct one dimensional calculations are used to estimate the effect of regularization order regularization strength relaxation and data support on the multigrid reduction factor a finite difference formulation is ruled out in favor of a finite element formulation a representative problem from magnetic resonance coil sensitivity estimation is solved using increasingly higher orders of regularization and the results are compared in terms of accuracy and multigrid convergence

fast surface interpolation using hierarchical basis functions an alternative to multigrid relaxation that is much easier to implement and more generally applicable is presented conjugate gradient descent is used in conjunction with a hierarchical multiresolution set of basis functions the resultant algorithm uses a pyramid to smooth the residual vector before the direction is computed simulation results showing the speed of convergence and its dependence on the choice of interpolator the number of smoothing levels and other factors are presented the relationship of this approach to other multiresolution relaxation and representation schemes is also discussed

multigrid tomographic inversion with variable resolution data and image spaces a multigrid inversion approach that uses variable resolutions of both the data space and the image space is proposed since the computational complexity of inverse problems typically increases with a larger number of unknown image pixels and a larger number of measurements the proposed algorithm further reduces the computation relative to conventional multigrid approaches which change only the image space resolution at coarse scales the advantage is particularly important for data rich applications where data resolutions may differ for different scales applications of the approach to bayesian reconstruction algorithms in transmission and emission tomography with a generalized gaussian markov random field image prior are presented both with a poisson noise model and with a quadratic data term simulation results indicate that the proposed multigrid approach results in significant improvement in convergence speed compared to the fixed grid iterative coordinate descent method and a multigrid method with fixed data resolution

comments on the convergence of mean field procedures for mrf s the author comments that zhang see ibid vol 5 no 12 p 1662 65 1996 attempts to establish the convergence of a mean field iteration for an ising markov random field mrf for large values of the hyperparameter value spl beta is not correct for the function t sub spl delta u defined the convergence of the mean field iteration remains an open question for large spl beta

dynamic 3d models with local and global deformations deformable superquadrics the authors present a physically based approach to fitting complex three dimensional shapes using a novel class of dynamic models that can deform both locally and globally they formulate the deformable superquadrics which incorporate the global shape parameters of a conventional superellipsoid with the local degrees of freedom of a spline the model s six global deformational degrees of freedom capture gross shape features from visual data and provide salient part descriptors for efficient indexing into a database of stored models the local deformation parameters reconstruct the details of complex shapes that the global abstraction misses the equations of motion which govern the behavior of deformable superquadrics make them responsive to externally applied forces the authors fit models to visual data by transforming the data into forces and simulating the equations of motion through time to adjust the translational rotational and deformational degrees of freedom of the models model fitting experiments involving 2d monocular image data and 3d range data are presented

viewpoint invariant recovery of visual surfaces from sparse data an algorithm for the reconstruction of visual surfaces from sparse data is proposed an important aspect of this algorithm is that the surface estimated from the sparse data is approximately invariant with respect to rigid transformation of the surface in 3d space the algorithm is based on casting the problem as an ill posed inverse problem that must be stabilized using a priori information related to the image and constraint formation to form a surface estimate that is approximately invariant with respect to viewpoint the stabilizing information is based on invariant surface characteristics with appropriate approximations this results in a convex functional to minimize which is then solved using finite element analysis the relationship of this algorithm to several previously proposed reconstruction algorithms is discussed and several examples that demonstrate its effectiveness in reconstructing viewpoint invariant surface estimates are given

reconstruction of nonuniformly sampled images in spline spaces this paper presents a novel approach to the reconstruction of images from nonuniformly spaced samples this problem is often encountered in digital image processing applications nonrecursive video coding with motion compensation spatiotemporal interpolation of video sequences and generation of new views in multicamera systems are three possible applications we propose a new reconstruction algorithm based on a spline model for images we use regularization since this is an ill posed inverse problem we minimize a cost function composed of two terms one related to the approximation error and the other related to the smoothness of the modeling function all the processing is carried out in the space of spline coefficients this space is discrete although the problem itself is of a continuous nature the coefficients of regularization and approximation filters are computed exactly by using the explicit expressions of b spline functions in the time domain the regularization is carried out locally while the computation of the regularization factor accounts for the structure of the nonuniform sampling grid the linear system of equations obtained is solved iteratively our results show a very good performance in motion compensated interpolation applications

the computation of visible surface representations a computational theory of visible surface representations is developed the visible surface reconstruction process that computes these quantitative representations unifies formal solutions to the key problems of 1 integrating multiscale constraints on surface depth and orientation from multiple visual sources 2 interpolating dense piecewise smooth surfaces from these constraints 3 detecting surface depth and orientation discontinuities to apply boundary conditions on interpolation and 4 structuring large scale distributed surface representations to achieve computational efficiency visible surface reconstruction is an inverse problem a well posed variational formulation results from the use of a controlled continuity surface model discontinuity detection amounts to the identification of this generic model s distributed parameters from the data finite element shape primitives yield a local discretization of the variational principle the result is an efficient algorithm for visible surface reconstruction

using dynamic programming for solving variational problems in vision dynamic programming is discussed as an approach to solving variational problems in vision dynamic programming ensures global optimality of the solution is numerically stable and allows for hard constraints to be enforced on the behavior of the solution within a natural and straightforward structure as a specific example of the approach s efficacy applying dynamic programming to the energy minimizing active contours is described the optimization problem is set up as a discrete multistage decision process and is solved by a time delayed discrete dynamic programming algorithm a parallel procedure for decreasing computational costs is discussed

computation of discontinuous optical flow by domain decomposition and shape optimization a well known method for the reconstruction of motion fields from noisy image data is to determine flow fields by the minimization of a quadratic functional the first approach of this class has been proposed by horn and schunck 1981 a drawback of such approaches is that an explicit representation of the discontinuities of the motion field is lacking and that in general the resulting flow fields approximate the motion fields only badly at the corresponding locations in the image plane in this article we discuss the possibility to improve the results by hypothesizing the qualitative structure of the motion field in terms of certain parameters we decompose the image plane into disjoint sets restrict the domain of definition of the functionals to these sets and use the hypotheses to deform and to move the boundaries of the sets within the image plane we discuss the range of applicability of this new technique and illustrate the algorithm by numerical examples this article is a revised and extended version of schnorr 1990

sherman morrison woodbury formula based algorithms for the surface smoothing problem abstract surface smoothing applied to range elevation data acquired using a variety of sources has been a very active area of research in computational vision over the past decade generalized splines have emerged as the single most popular approximation tool to this end in this paper we present a new and fast algorithm for solving the surface smoothing problem using a membrane a thin plate or a thin plate membrane spline for data containing discontinuities our approach involves imbedding the surface smoothing problem specified on an irregular domain in the sense of discontinuities and boundaries in a rectangular region using the capacitance matrix method based on the sherman morrison woodbury formula of matrix analysis this formula is used in converting the problem of solving the original linear system resulting from a finite element discretization of the variational formulation of the surface smoothing problem to solving a lyapunov matrix equation or a cascade of two lyapunov matrix equations the reduced problem can then be solved very efficiently using the adi method and the bi conjugate gradient technique our solution requires the generation of a dense capacitance matrix for which we propose a practical and efficient method we demonstrate the efficiency of our algorithm via experiments on sparse data surface smoothing making performance comparisons with the conjugate gradient and hierarchical basis preconditioned conjugate gradient algorithms

surface from gradients without discrete integrability enforcement a gaussian kernel approach representative surface reconstruction algorithms taking a gradient field as input enforce the integrability constraint in a discrete manner while enforcing integrability allows the subsequent integration to produce surface heights existing algorithms have one or more of the following disadvantages they can only handle dense per pixel gradient fields smooth out sharp features in a partially integrable field or produce severe surface distortion in the results in this paper we present a method which does not enforce discrete integrability and reconstructs a 3d continuous surface from a gradient or a height field or a combination of both which can be dense or sparse the key to our approach is the use of kernel basis functions which transfer the continuous surface reconstruction problem into high dimensional space where a closed form solution exists by using the gaussian kernel we can derive a straightforward implementation which is able to produce results better than traditional techniques in general an important advantage of our kernel based method is that the method does not suffer discretization and finite approximation both of which lead to surface distortion which is typical of fourier or wavelet bases widely adopted by previous representative approaches we perform comparisons with classical and recent methods on benchmark as well as challenging data sets to demonstrate that our method produces accurate surface reconstruction that preserves salient and sharp features the source code and executable of the system are available for downloading

a geometric model for active contours in image processing we propose a new model for active contours based on a geometric partial differential equation our model is intrinsec stable satisfies the maximum principle and permits a rigorous mathematical analysis it enables us to extract smooth shapes we cannot retrieve angles and it can be adapted to find several contours simultaneously moreover as a consequence of the stability we can design robust algorithms which can be engineed with no parameters in applications numerical experiments are presented

a fast scalable algorithm for discontinuous optical flow estimation multiple moving objects partially occluded objects or even a single object moving against the background gives rise to discontinuities in the optical flow field in corresponding image sequences while uniform global regularization based moderately fast techniques cannot provide accurate estimates of the discontinuous flow field statistical optimization based accurate techniques suffer from excessive solution time a weighted anisotropic smoothness based numerically robust algorithm is proposed that can generate discontinuous optical flow field with high speed and linear computational complexity weighted sum of the first order spatial derivatives of the flow field is used for regularization less regularization is performed where strong gradient information is available the flow field at any point is interpolated more from those at neighboring points along the weaker intensity gradient component such intensity gradient weighted regularization leads to euler lagrange equations with strong anisotropies coupled with discontinuities in their coefficients a robust multilevel iterative technique that recursively generates coarse level problems based on intensity gradient weighted smoothing weights is employed to estimate discontinuous optical flow field experimental results are presented to demonstrate the efficacy of the proposed technique

a general framework for nonlinear multigrid inversion a variety of new imaging modalities such as optical diffusion tomography require the inversion of a forward problem that is modeled by the solution to a three dimensional 3d partial differential equation for these applications image reconstruction is particularly difficult because the forward problem is both nonlinear and computationally expensive to evaluate we propose a general framework for nonlinear multigrid inversion that is applicable to a wide variety of inverse problems the multigrid inversion algorithm results from the application of recursive multigrid techniques to the solution of optimization problems arising from inverse problems the method works by dynamically adjusting the cost functionals at different scales so that they are consistent with and ultimately reduce the finest scale cost functional in this way the multigrid inversion algorithm efficiently computes the solution to the desired fine scale inversion problem importantly the new algorithm can greatly reduce computation because both the forward and inverse problems are more coarsely discretized at lower resolutions an application of our method to bayesian optical diffusion tomography with a generalized gaussian markov random field image prior model shows the potential for very large computational savings numerical data also indicates robust convergence with a range of initialization conditions for this nonconvex optimization problem

cooperative robust estimation using layers of support we present an approach to the problem of representing images that contain multiple objects or surfaces rather than using an edge based approach to represent the segmentation of a scene we propose a multilayer estimation framework which uses support maps to represent the segmentation of the image into homogeneous chunks this support based approach can represent objects that are split into disjoint regions or have surfaces that are transparently interleaved our framework is based on an extension of robust estimation methods that provide a theoretical basis for support based estimation we use a selection criteria derived from the minimum description length principle to decide how many support maps to use in describing an image our method has been applied to a number of different domains including the decomposition of range images into constituent objects the segmentation of image sequences into homogeneous higher order motion fields and the separation of tracked motion features into distinct rigid body motions

ranging through gabor logons a consistent hierarchical approach in this work the correspondence problem in stereo vision is handled by matching two sets of dense feature vectors inspired by biological evidence these feature vectors are generated by a correlation between a bank of gabor sensors and the intensity image the sensors consist of two dimensional gabor filters at various scales spatial frequencies and orientations which bear close resemblance to the receptive field profiles of simple v1 cells in visual cortex a hierarchical stochastic relaxation method is then used to obtain the dense stereo disparities unlike traditional hierarchical methods for stereo feature based hierarchical processing yields consistent disparities to avoid false matchings due to static occlusion a dual matching based on the imaging geometry is used

a two stage algorithm for discontinuity preserving surface reconstruction a two stage algorithm for visual surface reconstruction from scattered data while preserving discontinuities is presented the first stage consists of a robust local approximation algorithm the moving least median of squares mlms of error to clean the data and create a grid from the original scattered data points this process is discontinuity preserving the second stage introduces a weighted bicubic spline wbs as a surface descriptor the wbs has a factor in the regularizing term that adapts the behavior of the spline across discontinuities the weighted bicubic approximating spline can approximate data with step discontinuities with no discernible distortion in the approximating surface the combination of robust surface fitting and wbss removes outliers and reduces gaussian noise either stage by itself would not effectively remove both kinds of noise experimental results with the two stage algorithm are presented

discontinuity preserving and viewpoint invariant reconstruction of visible surfaces using a first order regularization this paper describes the application of a first order regularization technique to the reconstruction of visible surfaces our approach is a computationally efficient first order method that simultaneously achieves approximate invariance and preservation of discontinuities it is also robust with respect to the smoothing parameter spl lambda the robustness property to spl lambda allows a free choice of spl lambda without struggling to determine an optimal spl lambda that provides the best reconstruction a new approximately invariant first order stabilizing function for surface reconstruction is obtained by employing a first order taylor expansion of a nonconvex invariant stabilizing function that is expanded at the estimated value of the squared gradient instead of at zero the data compatibility measure is the squared perpendicular distance between the reconstructed surface and the constraint surface this combination of stabilizing function and data compatibility measure is necessary to achieve invariance with respect to rotations and translations sharp preservation of discontinuities is achieved by a weighted sum of adjacent pixels the results indicate that the proposed methods perform well on sparse noisy range data in addition the volume between two surfaces normalized by the surface area interpreted as average distance between two surfaces is proposed as an invariant measure for the comparison of reconstruction results

activity representation with motion hierarchies complex activities e g pole vaulting are composed of a variable number of sub events connected by complex spatio temporal relations whereas simple actions can be represented as sequences of short temporal parts in this paper we learn hierarchical representations of activity videos in an unsupervised manner these hierarchies of mid level motion components are data driven decompositions specific to each video we introduce a spectral divisive clustering algorithm to efficiently extract a hierarchy over a large number of tracklets i e local trajectories we use this structure to represent a video as an unordered binary tree we model this tree using nested histograms of local motion features we provide an efficient positive definite kernel that computes the structural and visual similarity of two hierarchical decompositions by relying on models of their parent child relations we present experimental results on four recent challenging benchmarks the high five dataset patron perez et al high five recognising human interactions in tv shows 2010 the olympics sports dataset niebles et al modeling temporal structure of decomposable motion segments for activity classification 2010 the hollywood 2 dataset marszalek et al actions in context 2009 and the hmdb dataset kuehne et al hmdb a large video database for human motion recognition 2011 we show that per video hierarchies provide additional information for activity recognition our approach improves over unstructured activity models baselines using other motion decomposition algorithms and the state of the art

a probabilistic approach to the coupled reconstruction and restoration of underwater acoustic images describes a probabilistic technique for the coupled reconstruction and restoration of underwater acoustic images the technique is founded on the physics of the image formation process beamforming a method widely applied in acoustic imaging is used to build a range image from backscattered echoes associated point by point with another type of information representing the reliability or confidence of such an image unfortunately this kind of images is plagued by problems due to the nature of the signal and to the related sensing system in the proposed algorithm the range and confidence images are modeled as markov random fields whose associated probability distributions are specified by a single energy function this function has been designed to fully embed the physics of the acoustic image formation process by modeling a priori knowledge of the acoustic system the considered scene and the noise affecting measures and also by integrating reliability information to allow the coupled and simultaneous reconstruction and restoration of both images optimal in the maximum a posteriori probability sense estimates of the reconstructed range image map and the restored confidence image are obtained by minimizing the energy function using simulated annealing experimental results show the improvement of the processed images over those obtained by other methods performing separate reconstruction and restoration processes that disregard reliability information

on edge detection edge detection is the process that attempts to characterize the intensity changes in the image in terms of the physical processes that have originated them a critical intermediate goal of edge detection is the detection and characterization of significant intensity changes this paper discusses this part of the edge detection problem to characterize the types of intensity changes derivatives of different types and possibly different scales are needed thus we consider this part of edge detection as a problem in numerical differentiation we show that numerical differentiation of images is an ill posed problem in the sense of hadamard differentiation needs to be regularized by a regularizing filtering operation before differentiation this shows that this part of edge detection consists of two steps a filtering step and a differentiation step following this perspective the paper discusses in detail the following theoretical aspects of edge detection 1 the properties of different types of filters with minimal uncertainty with a bandpass spectrum and with limited support are derived minimal uncertainty filters optimize a tradeoff between computational efficiency and regularizing properties 2 relationships among several 2 d differential operators are established in particular we characterize the relation between the laplacian and the second directional derivative along the gradient zero crossings of the laplacian are not the only features computed in early vision 3 geometrical and topological properties of the zero crossings of differential operators are studied in terms of transversality and morse theory

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

image analysis using multigrid relaxation methods image analysis problems posed mathematically as variational principles or as partial differential equations are amenable to numerical solution by relaxation algorithms that are local iterative and often parallel although they are well suited structurally for implementation on massively parallel locally interconnected computational architectures such distributed algorithms are seriously handi capped by an inherent inefficiency at propagating constraints between widely separated processing elements hence they converge extremely slowly when confronted by the large representations of early vision application of multigrid methods can overcome this drawback as we showed in previous work on 3 d surface reconstruction in this paper we develop multiresolution iterative algorithms for computing lightness shape from shading and optical flow and we examine the efficiency of these algorithms using synthetic image inputs the multigrid methodology that we describe is broadly applicable in early vision notably it is an appealing strategy to use in conjunction with regularization analysis for the efficient solution of a wide range of ill posed image analysis problems

surface interpolation networks orthogonal wavelets can be used as models for receptive fields in the human visual system they may also be used to solve spatial interpolation problems formulated either as regularization or 2 d kalman filtering the solutions take the form of simple feedback networks and only a few iterations are required for convergence

regularization of inverse visual problems involving discontinuities inverse problems such as the reconstruction problems that arise in early vision tend to be mathematically ill posed through regularization they may be reformulated as well posed variational principles whose solutions are computable standard regularization theory employs quadratic stabilizing functionals that impose global smoothness constraints on possible solutions discontinuities present serious difficulties to standard regularization however since their reconstruction requires a precise spatial control over the smoothing properties of stabilizers this paper proposes a general class of controlled continuity stabilizers which provide the necessary control over smoothness these nonquadratic stabilizing functionals comprise multiple generalized spline kernels combined with noncontinuous continuity control functions in the context of computational vision they may be thought of as controlled continuity constraints these generic constraints are applicable to visual reconstruction problems that involve both continuous regions and discontinuities for which global smoothness constraints fail

on three dimensional surface reconstruction methods a survey is presented of some of the surface reconstruction methods that can be found in the literature the focus is on a small recent and important subset of the published reconstruction techniques the techniques are classified based on the surface representation used implicit versus explicit functions a study is made of the important aspects of the surface reconstruction techniques one aspect is the viewpoint invariance of the methods this is an important property if object recognition is the ultimate objective the robustness of the various methods is examined it is determined whether the parameter estimates are biased and the sensitivity to obscuration is addressed the latter two aspects are particularly important for fitting functions in the implicit form a detailed description is given of a parametric reconstruction method for three dimensional object surfaces that involves numeric grid generation techniques and variational principle formulations this technique is invariant to rigid motion in dimensional space

photometric stereo via expectation maximization this paper presents a robust and automatic approach to photometric stereo where the two main components namely surface normals and visible surfaces are respectively optimized by expectation maximization em a dense set of input images is conveniently captured using a digital video camera while a handheld spotlight is being moved around the target object and a small mirror sphere in our approach the inherently complex optimization problem is simplified into a two step optimization where em is employed in each step 1 using the dense input the weight or importance of each observation is alternately optimized with the normal and albedo at each pixel and 2 using the optimized normals and employing the markov random fields mrfs surface integrabilities and discontinuities are alternately optimized in visible surface reconstruction our mathematical derivation gives simple updating rules for the em algorithms leading to a stable practical and parameter free implementation that is very robust even in the presence of complex geometry shadows highlight and transparency we present high quality results on normal and visible surface reconstruction where fine geometric details are automatically recovered by our method

localized measurement of emergent image frequencies by gabor wavelets the authors derive implement and demonstrate a computational approach for the measurement of emergent image frequencies measuring emergent signal frequencies requires spectral measurements accurate in both frequency and time or space conflicting requirements that are shown to be balanced by a generalized uncertainty relationship such spectral measurements can be obtained from the responses of multiple wavelet like channel filters that sample the signal spectrum and that yield a locus of possible solutions for each locally emergent frequency it is shown analytically that this locus of solutions is maximally localized in both space and frequency if the channel filters used are gabor wavelets a constrained solution is obtained by imposing a stabilizing term that develops naturally from the assumptions on the signal the measurement of frequencies is then cast as an ill posed extremum problem regularized by the stabilizing term leading to an iterative constraint propagation algorithm the technique is demonstrated by application to a variety of 2 d textured images

robust parameter estimation in computer vision estimation techniques in computer vision applications must estimate accurate model parameters despite small scale noise in the data occasional large scale measurement errors outliers and measurements from multiple populations in the same data set increasingly robust estimation techniques some borrowed from the statistics literature and others described in the computer vision literature have been used in solving these parameter estimation problems ideally these techniques should effectively ignore the outliers and measurements from other populations treating them as outliers when estimating the parameters of a single population two frequently used techniques are least median of squares lms p j rousseeuw j amer statist assoc 79 1984 pp 871 880 and m estimators robust statistics the approach based on influence functions f r hampel et al john wiley 1986 robust statistics p j huber john wiley 1981 lms handles large fractions of outliers up to the theoretical limit of 50 for estimators invariant to affine changes to the data but has low statistical efficiency m estimators have higher statistical efficiency but tolerate much lower percentages of outliers unless properly initialized while robust estimators have been used in a variety of computer vision applications three are considered here in analysis of range images images containing depth or x y z measurements at each pixel instead of intensity measurements robust estimators have been used successfully to estimate surface model parameters in small image regions in stereo and motion analysis they have been used to estimate parameters of what is called the fundamental matrix which characterizes the relative imaging geometry of two cameras imaging the same scene recently robust estimators have been applied to estimating a quadratic image to image transformation model necessary to create a composite mosaic image from a series of images of the human retina in each case a straightforward application of standard robust estimators is insufficient and carefully developed extensions are used to solve the problem

invariant surface segmentation through energy minimization with discontinuities the computational problems in segmenting range data into surface patches based on the invariant surface properties i e mean curvature h and gaussian curvature k are investigated the goal is to obtain reliable hk surface maps two commonly encountered problems are firstly the noise effect in computing derivative estimates and secondly the smoothing across discontinuities here the segmentation is formulated as finding minimization solutions of energy functionals involving discontinuities a two stage approach to the goal is presented stage 1 from a range image to curvature images and stage 2 from the curvature images to the hk maps in both stages solutions are found through minimizing energy functionals that measure the degree of bias of a solution from two constraints the closeness of the solution to the data and the smoothness of the solution controlled by predetermined discontinuities propagation across discontinuities is prevented during minimization which preserves the original surface shapes experimental results are given for a variety of test images

interpolation using wavelet bases efficient solutions to regularization problems can be obtained using orthogonal wavelet bases for preconditioning good approximate solutions can be obtained in only two or three iterations with each iteration requiring only o n operations and o n storage locations two and three dimensional examples are shown using both synthetic and real range data

left ventricular motion reconstruction based on elastic vector splines in medical imaging it is common to reconstruct dense motion estimates from sparse measurements of that motion using some form of elastic spline thin plate spline snakes and other deformable models etc usually the elastic spline uses only bending energy second order smoothness constraint or stretching energy first order smoothness constraint or a combination of the two these elastic splines belong to a family of elastic vector splines called the laplacian splines this spline family is derived from an energy minimization functional which is composed of multiple order smoothness constraints these splines can be explicitly tuned to vary the smoothness of the solution according to the deformation in the modeled material tissue in this context it is natural to question which members of the family will reconstruct the motion more accurately the authors compare different members of this spline family to assess how well these splines reconstruct human cardiac motion they find that the commonly used splines containing first order and or second order smoothness terms only are not the most accurate for modeling human cardiac motion

depth and image recovery using a mrf model this paper deals with the problem of depth recovery and image restoration from sparse and noisy image data the image is modeled as a markov random field and a new energy function is developed to effectively detect discontinuities in highly sparse and noisy images the model provides an alternative to the use of a line process interpolation over missing data sites is first done using local characteristics to obtain initial estimates and then simulated annealing is used to compute the maximum a posteriori map estimate a threshold on energy reduction per iteration is used to speed up simulated annealing by avoiding computation that contributes little to the energy minimization moreover a minor modification of the posterior energy function gives improved results for random as well as structured sparsing problems results of simulations carried out on real range and intensity images along with details of the simulations are presented

robust b spline image modeling with application to image processing we present a new approach to two dimensional 2 d robust spline image smoothing based on the m estimator algorithm unlike in other m estimator based image processing algorithms the new algorithm takes into consideration the spatial relations between picture elements the contribution of the sample to the model depends not only on the current residual of that sample but also on the neighboring residuals a smoothing parameter is estimated separately for each processing window and it adapts to the local structure of the image the proposed algorithm is applied to image filtering the resulting filter preserves details and suppresses additive gaussian and impulsive noise efficiently

fast surface interpolation using multiresolution wavelet transform discrete formulation of the surface interpolation problem usually leads to a large sparse linear equation system due to the poor convergence condition of the equation system the convergence rate of solving this problem with iterative method is very slow to improve this condition a multiresolution basis transfer scheme based on the wavelet transform is proposed by applying the wavelet transform the original interpolation basis is transformed into two sets of bases with larger supports while the admissible solution space remains unchanged with this basis transfer a new set of nodal variables results and an equivalent equation system with better convergence condition can be solved the basis transfer can be easily implemented by using an qmf matrix pair associated with the chosen interpolation basis the consequence of the basis transfer scheme can be regarded as a preconditioner to the subsequent iterative computation method the effect of the transfer is that the interpolated surface is decomposed into its low frequency and high frequency portions in the frequency domain it has been indicated that the convergence rate of the interpolated surface is dominated by the low frequency portion with this frequency domain decomposition the low frequency portion of the interpolated surface can be emphasized as compared with other acceleration methods this basis transfer scheme provides a more systematical approach for fast surface interpolation the easy implementation and high flexibility of the proposed algorithm also make it applicable to various regularization problems

orientation matching minimization for image denoising and inpainting in this paper we propose an orientation matching functional minimization for image denoising and image inpainting following the two step tv stokes algorithm rahman et al in scale space and variational methods in computer vision pp 473 482 springer heidelberg 2007 tai et al in image processing based on partial differential equations pp 3 22 springer heidelberg 2006 bertalmio et al in proc conf comp vision pattern rec pp 355 362 2001 a regularized tangential vector field with zero divergence condition is first obtained then a novel approach to reconstruct the image is proposed instead of finding an image that fits the regularized normal direction from the first step we propose to minimize an orientation matching cost measuring the alignment between the image gradient and the regularized normal direction this functional yields a new nonlinear partial differential equation pde for reconstructing denoised and inpainted images the equation has an adaptive diffusivity depending on the orientation of the regularized normal vector field providing reconstructed images which have sharp edges and smooth regions the additive operator splitting aos scheme is used for discretizing euler lagrange equations we present the results of various numerical experiments that illustrate the improvements obtained with the new functional

high order differentiation filters that work reliable derivatives of digital images have always been hard to obtain especially but not only at high orders we analyze the sources of errors in traditional filters such as derivatives of the gaussian that are used for differentiation we then study a class of filters which is much more suitable for our purpose namely filters that preserve polynomials up to a given order we show that the errors in differentiation can be corrected using these filters we derive a condition for the validity domain of these filters involving some characteristics of the filter and of the shape our experiments show a very good performance for smooth functions

edge detection and surface reconstruction using refined regularization an edge detection and surface reconstruction algorithm in which the smoothness is controlled spatially over the image space is presented the values of parameters in the model are adaptively determined by an iterative refinement process hence the image dependent parameters such as the optimum value of the regularization parameter or the filter size are eliminated the algorithm starts with an oversmoothed regularized solution and iteratively refines the surface around discontinuities by using the structure exhibited in the error signal the spatial control of smoothness is shown to resolve the conflict between detection and localization criteria of edge detection by smoothing the noise in continuous regions while preserving discontinuities the performance of the algorithm is quantitatively and qualitatively evaluated on real and synthetic images and it is compared with those of marr hildreth and canny edge detectors

spl lambda spl tau space representation of images and generalized edge detector an image and surface representation based on regularization theory is introduced in this paper this representation is based on a hybrid model derived from the physical membrane and plate models the representation called the spl lambda spl tau representation has two dimensions one dimension represents smoothness or scale while the other represents the continuity of the image or surface it contains images surfaces sampled both in scale space and the weighted sobolev space of continuous functions thus this new representation can be viewed as an extension of the well known scale space representation we have experimentally shown that the proposed hybrid model results in improved results compared to the two extreme constituent models i e the membrane and the plate models based on this hybrid model a generalized edge detector ged which encompasses most of the well known edge detectors under a common framework is developed the existing edge detectors can be obtained from the generalized edge detector by simply specifying the values of two parameters one of which controls the shape of the filter spl tau and the other controls the scale of the filter spl lambda by sweeping the values of these two parameters continuously one can generate an edge representation in the spl lambda spl tau space which is very useful for developing a goal directed edge detection scheme for a specific task the proposed representation and the edge detector have been evaluated qualitatively and quantitatively on several different types of image data such as intensity range and stereo images

design and recovery of 2 d and 3 d shapes using rational gaussian curves and surfaces a new representation for parametric curves and surfaces is introduced here it is in rational form and uses rational gaussian bases this representation allows design of 2 d and 3 d shapes and makes recovery of shapes from noisy image data possible the standard deviations of gaussians in a curve or surface control the smoothness of a recovered shape the control points of a surface in this representation are not required to form a regular grid and a scattered set of control points is sufficient to reconstruct a surface examples of shape design shape recovery and image segmentation using the proposed representation are given

shadow graphs and 3d texture reconstruction we present methods for recovering surface height fields such as geometric details of 3d textures by incorporating shadow constraints we introduce shadow graphs which give a new graph based representation for shadow constraints it can be shown that the shadow graph alone is sufficient to solve the shape from shadow problem from a dense set of images shadow graphs provide a simpler and more systematic approach to represent and integrate shadow constraints from multiple images to recover height fields from a sparse set of images we propose a method for integrated shadow and shading constraints previous shape from shadow algorithms do not consider shading constraints while shape from shading usually assumes there is no shadow our method is based on collecting a set of images from a fixed viewpoint as a known light source changes its position it first builds a shadow graph from shadow constraints from which an upper bound for each pixel can be derived if the height values of a small number of pixels are initialized correctly finally a constrained optimization procedure is designed to make the results from shape from shading consistent with the height bounds derived from the shadow constraints our technique is demonstrated on both synthetic and real imagery

the local structure of image discontinuities in one dimension the detailed structure of intensities in the local neighborhood of an edge can often indicate the nature of the physical event givinig rise to that edge we argue that the limit as we approach arbitrarily close to either side of an edge of such image parameters as type of texture texture gradient color appropriate directional derivatives of intensity etc is a key aspect of this structure however the general problem of capturing this local structure is surprisingly complex thus we restrict ourselves in this paper to a relatively simple domain one dimensional cuts through idealized images modeled by piecewise smooth c1 functions corrupted by gaussian noise within this domain we define local structure to be the limit of the uncorrupted intensity and of its derivatives as we approach arbitrarily close to either side of a discontinuity we develop a technique that captures this local structure while simultaneously locating the discontinuities and demonstrate that these tasks are in fact inseparable the technique is an extension using estimation theory of the classical definition of discontinuity it handles in a consistent fashion both jump discontinuities in the function and jump discontinuities in its first derivative so called step edges are a special case of the former and roof edges of the latter it also integrates again in a consistent fashion information derived from a number of different neighborhood sizes

generalized capacitance matrix theorems and algorithm for solving linear systems the capacitance matrix method has been widely used as an efficient numerical tool for solving the boundary value problems on irregular regions initially this method was based on the sherman morrison woodbury formula an expression for the inverse of the matrix a uvt with bf a in re n times n and bf u v in re n times p extensions of this method reported in literature have made restrictive assumptions on the matrices a and a uvt in this paper we present several theorems which are generalizations of the capacitance matrix theorem in cite buzbee and are suited for very general matrices a and a uvt a generalized capacitance matrix algorithm is developed from these theorems and holds the promise of being applicable to more general problems in addition it gives ample freedom to choose the matrix a for developing very efficient numerical algorithms we demonstrate the usefulness of our generalized capacitance matrix algorithm by applying it to efficiently solve large sparse linear systems in several computer vision problems namely the surface reconstruction and interpolation and the shape from orientation problem

reconstruction and segmentation of underwater acoustic images combining confidence information in mrf models abstract this paper describes a technique for the integration of confidence information using markov random fields models to improve the segmentation and reconstruction of three dimensional acoustical images a range image in which each point is associated with a reliability measure namely its confidence is directly provided by acoustic image formation process in this paper range and confidence images are modelled as markov random fields over which several energy formulations are devised to exploit both types of data leading to an accurate reconstruction and segmentation of such images results show the better performances of the proposed approach as compared with classical methods disregarding reliability information

multiresolution discontinuity preserving surface reconstruction abstract this paper proposes a single multiresolution framework for both surface fitting and discontinuities identification in the surface reconstruction problem in particular the mallat zhong wavelet transform modulus maxima wtmm representation is employed for representing the surface which allows the discontinuity detection and surface fitting to be carried out simultaneously in a cooperative manner under the same multiresolution framework on the one hand the multiresolution feature analysis inherent in the wtmm representation helps characterize the strength of the different significant features and thus by updating the wtmm surface in a number of energy minimization processes the discontinuities can be detected and tracked across various resolution levels on the other hand the detected discontinuities in turn control spatially the suitable degree of smoothness at various image positions when the surface is fitted to the input data in the energy minimizing processes in this way the discontinuity detection and the surface fitting processes mutually assist each other experimental results show that a piecewise smooth surface can be reconstructed without discontinuity being over smoothed

scale invariant image recognition based on higher order autocorrelation features we propose a framework and a complete implementation of a translation and scale invariant image recognition system for natural indoor scenes the system employs higher order autocorrelation features of scale space data which permit linear classification an optimal linear classification method is presented which is able to cope with a large number of classes represented by many as well as very few samples in the course of the analysis of our system we examine which numerical methods for feature transformation and classification show sufficient stability to fulfill these demands the implementation has been extensively tested we present the results of our own application and several classification benchmarks

approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations summary structured additive regression models are perhaps the most commonly used class of models in statistical applications it includes among others generalized linear models generalized additive models smoothing spline models state space models semiparametric regression spatial and spatiotemporal models log gaussian cox processes and geostatistical and geoadditive models we consider approximate bayesian inference in a popular subset of structured additive regression models latent gaussian models where the latent field is gaussian controlled by a few hyperparameters and with non gaussian response variables the posterior marginals are not available in closed form owing to the non gaussian response variables for such models markov chain monte carlo methods can be implemented but they are not without problems in terms of both convergence and computational time in some practical applications the extent of these problems is such that markov chain monte carlo sampling is simply not an appropriate tool for routine analysis we show that by using an integrated nested laplace approximation and its simplified version we can directly compute very accurate approximations to the posterior marginals the main benefit of these approximations is computational where markov chain monte carlo algorithms need hours or days to run our approximations provide more precise estimates in seconds or minutes another advantage with our approach is its generality which makes it possible to perform bayesian analysis in an automatic streamlined way and to compute model comparison criteria and various predictive measures so that models can be compared and the model under study can be challenged

partial smoothing splines for noisy boundaries with corners investigates the estimation of 2 d boundary functions from sampled data sets where both noise and corners are present the approach is based on the partial smoothing spline in which the estimated boundary function consists of an ordinary smoothing spline and a parametric function that describes the discontinuities i e corners of the boundary prior knowledge about the boundary such as the number of corners their locations noise levels and the amount of smoothness is not required for the boundary estimate the smoothing parameter and the corner locations of the spline which are parts of the estimate are determined by the generalized cross validation method whereby statistical properties are gathered from the input sampled data rather than specified a priori this approach enables the smoothing of a noisy boundary while retaining an accurate description of the boundary corners extensive experiments were conducted to verify its ability to smooth noise while retaining a good representation of boundary corners and do not rely on any prior information

reconstructing surfaces by volumetric regularization using radial basis functions we present a new method of surface reconstruction that generates smooth and seamless models from sparse noisy nonuniform and low resolution range data data acquisition techniques from computer vision such as stereo range images and space carving produce 3d point sets that are imprecise and nonuniform when compared to laser or optical range scanners traditional reconstruction algorithms designed for dense and precise data do not produce smooth reconstructions when applied to vision based data sets our method constructs a 3d implicit surface formulated as a sum of weighted radial basis functions we achieve three primary advantages over existing algorithms 1 the implicit functions we construct estimate the surface well in regions where there is little data 2 the reconstructed surface is insensitive to noise in data acquisition because we can allow the surface to approximate rather than exactly interpolate the data and 3 the reconstructed surface is locally detailed yet globally smooth because we use radial basis functions that achieve multiple orders of smoothness

recovery of the 3 d shape of the left ventricle from echocardiographic images a computational method is reported which allows the fully automated recovery of the three dimensional shape of the cardiac left ventricle from a reduced set of apical echo views two typically ill posed problems have been faced 1 the detection of the left ventricle contours in each view and 2 the integration of the detected contour points which form a sparse and partially inconsistent data set into a single surface representation the authors solution to these problems is based on a careful integration of standard computer vision algorithms with neural networks boundary detection comprises three steps edge detection edge grouping and edge classification the first and second steps which are typical early vision tasks not involving specific domain knowledge have been performed through fast well established algorithms of computer vision the higher level task of left ventricle edge discrimination which involves the exploitation of specific knowledge about the left ventricle silhouette has been performed by feedforward neural networks following the most recent results in the field of computer vision the first step in solving the problem of recovering the ventricle surface has been the adoption of a physically inspired model of it basically the authors have modeled the left ventricle surface as a closed thin elastic surface and the data as a set of radial springs acting on it the recovery process is equivalent to the settling of the surface plus springs system into a stable configuration of minimum potential energy the finite element discretization of this model leads directly to an analog neural network implementation the efficiency of such an implementation has been remarkably enhanced through a learning algorithm which embeds specific knowledge about the shape of the left ventricle in the network experiments using clinical echographic sequences are described four apical views each with a different rotation of the probe have been acquired during a heartbeat from a set of seven normal subjects these images have been utilized to set the various processing modules and test their capabilities

iterative image restoration combining total variation minimization and a second order functional a noise removal technique using partial differential equations pdes is proposed here it combines the total variational tv filter with a fourth order pde filter the combined technique is able to preserve edges and at the same time avoid the staircase effect in smooth regions a weighting function is used in an iterative way to combine the solutions of the tv filter and the fourth order filter numerical experiments confirm that the new method is able to use less restrictive time step than the fourth order filter numerical examples using images with objects consisting of edge flat and intermediate regions illustrate advantages of the proposed model

interpreting line drawings as three dimensional surfaces understanding how line drawings convey tri dimensionality is of fundamental importance in explaining surface perception when photometry is either uninformative or too complex to model analytically we put forward here a computational model for interpreting line drawings as three dimensional surfaces based on constraints on local surface orientation along extremal and discontinuity boundaries specific techniques are described for two key processes recovering the three dimensional conformation of a space curve e g a surface boundary from its two dimensional projection in an image and interpolating smooth surfaces from orientation constraints along extremal boundaries the relevance of the model to a general theory of low level vision is discussed

shape from shading with a linear triangular element surface model the authors propose to combine a triangular element surface model with a linearized reflectance map to formulate the shape from shading problem the main idea is to approximate a smooth surface by the union of triangular surface patches called triangular elements and express the approximating surface as a linear combination of a set of nodal basis functions since the surface normal of a triangular element is uniquely determined by the heights of its three vertices or nodes image brightness can be directly related to nodal heights using the linearized reflectance map the surface height can then be determined by minimizing a quadratic cost functional corresponding to the squares of brightness errors and solved effectively with the multigrid computational technique the proposed method does not require any integrability constraint or artificial assumptions on boundary conditions simulation results for synthetic and real images are presented to illustrate the performance and efficiency of the method

materials with prescribed constitutive parameters an inverse homogenization problem abstract this paper deals with the construction of materials with arbitrary prescribed positive semi definite constitutive tensors the construction problem can be called an inverse problem of finding a material with given homogenized coefficients the inverse problem is formulated as a topology optimization problem i e finding the interior topology of a base cell such that cost is minimized and the constraints are defined by the prescribed constitutive parameters numerical values of the constitutive parameters of a given material are found using a numerical homogenization method expressed in terms of element mutual energies numerical results show that arbitrary materials including materials with poisson s ratio 1 0 and other extreme materials can be obtained by modelling the base cell as a truss structure furthermore a wide spectrum of materials can be constructed from base cells modelled as continuous discs of varying thickness only the two dimensional case is considered in this paper but formulation and numerical procedures can easily be extended to the three dimensional case

constructing intrinsic parameters with active models for invariant surface reconstruction a technique for constructing a canonical surface parameterization in terms of lines of curvature is presented two methods of computing the canonical invariant representation are also presented in the first method a static instance of the controlled continuity spline is used for the stabilizer ways to modify it to reflect a change of parameters to the lines of curvature are described in the second method the dynamic instance of the controlled continuity spline called the deformable model is used a force field defined in terms of the principal vectors is synthesized and applied to the parameter curves of the deformable model to coerce them along the lines of curvature in essence any transformation of parameters requires a modification of the stabilizer in the first method whereas in the second method it is tantamount to synthesizing a new force field experimental results with real and synthetic range data are included