fourier lucas kanade algorithm in this paper we propose a framework for both gradient descent image and object alignment in the fourier domain our method centers upon the classical lucas x0026 kanade lk algorithm where we represent the source and template model in the complex 2d fourier domain rather than in the spatial 2d domain we refer to our approach as the fourier lk flk algorithm the flk formulation is advantageous when one preprocesses the source image and template model with a bank of filters e g oriented edges gabor etc as 1 it can handle substantial illumination variations 2 the inefficient preprocessing filter bank step can be subsumed within the flk algorithm as a sparse diagonal weighting matrix 3 unlike traditional lk the computational cost is invariant to the number of filters and as a result is far more efficient and 4 this approach can be extended to the inverse compositional ic form of the lk algorithm where nearly all steps including fourier transform and filter bank preprocessing can be precomputed leading to an extremely efficient and robust approach to gradient descent image matching further these computational savings translate to nonrigid object alignment tasks that are considered extensions of the lk algorithm such as those found in active appearance models aams

discriminative face alignment this paper proposes a discriminative framework for efficiently aligning images although conventional active appearance models aams based approaches have achieved some success they suffer from the generalization problem i e how to align any image with a generic model we treat the iterative image alignment problem as a process of maximizing the score of a trained two class classifier that is able to distinguish correct alignment positive class from incorrect alignment negative class during the modeling stage given a set of images with ground truth landmarks we train a conventional point distribution model pdm and a boosting based classifier which acts as an appearance model when tested on an image with the initial landmark locations the proposed algorithm iteratively updates the shape parameters of the pdm via the gradient ascent method such that the classification score of the warped image is maximized we use the term boosted appearance models bams to refer to the learned shape and appearance models as well as our specific alignment method the proposed framework is applied to the face alignment problem using extensive experimentation we show that compared to the aam based approach this framework greatly improves the robustness accuracy and efficiency of face alignment by a large margin especially for unseen data

application of the karhunen loeve procedure for the characterization of human faces the use of natural symmetries mirror images in a well defined family of patterns human faces is discussed within the framework of the karhunen loeve expansion this results in an extension of the data and imposes even and odd symmetry on the eigenfunctions of the covariance matrix without increasing the complexity of the calculation the resulting approximation of faces projected from outside of the data set onto this optimal basis is improved on average

manifold based constraints for operations in face space in this paper we constrain faces to points on a manifold within the parameter space of a linear statistical model the manifold is the subspace of faces which have maximally likely distinctiveness and different points correspond to unique identities we provide a detailed empirical validation for the chosen manifold we show how the log and exponential maps for a hyperspherical manifold can be used to replace linear operations such as warping and averaging with operations on this manifold finally we use the manifold to develop a new method for fitting a statistical face shape model to data which is both robust avoids overfitting and overcomes model dominance is not susceptible to local minima close to the mean face we provide experimental results for fitting a dense 3d morphable face model to data using two different objective functions one underconstrained and one with many local minima our method outperforms generic nonlinear optimisers based on the bfgs quasi newton method and the levenberg marquardt algorithm when fitting using the basel face model highlightswe decompose statistical face models into identity and distinctiveness subspaces the identity subspace forms a hyperspherical manifold that we validate empirically the manifold provides non linear alternatives to warping and averaging we use the manifold to constrain optimisation based model fitting this outperforms two existing algorithms on over and under constrained problems

feature driven direct non rigid image registration the direct registration problem for images of a deforming surface has been well studied parametric flexible warps based for instance on the free form deformation or a radial basis function such as the thin plate spline are often estimated using additive gauss newton like algorithms the recently proposed compositional framework has been shown to be more efficient but cannot be directly applied to such non groupwise warps our main contribution in this paper is the feature driven framework it makes possible the use of compositional algorithms for most parametric warps such as those above mentioned two algorithms are proposed to demonstrate the relevance of our feature driven framework the feature driven inverse compositional and the feature driven learning based algorithms as another contribution a detailed derivation of the feature driven warp parameterization is given for the thin plate spline and the free form deformation we experimentally show that these two types of warps have a similar representational power experimental results show that our feature driven registration algorithms are more efficient in terms of computational cost without loss of accuracy compared to existing methods

nonrigid registration of 2 d and 3 d dynamic cell nuclei images for improved classification of subcellular particle motion the observed motion of subcellular particles in fluorescence microscopy image sequences of live cells is generally a superposition of the motion and deformation of the cell and the motion of the particles decoupling the two types of movements to enable accurate classification of the particle motion requires the application of registration algorithms we have developed an intensity based approach for nonrigid registration of multichannel microscopy image sequences of cell nuclei first based on 3 d synthetic images we demonstrate that cell nucleus deformations change the observed motion types of particles and that our approach allows to recover the original motion second we have successfully applied our approach to register 2 d and 3 d real microscopy image sequences a quantitative experimental comparison with previous approaches for nonrigid registration of cell microscopy has also been performed

a shape reconstructability measure of object part importance with applications to object detection and localization we propose a computational model which computes the importance of 2 d object shape parts and we apply it to detect and localize objects with and without occlusions the importance of a shape part a localized contour fragment is considered from the perspective of its contribution to the perception and recognition of the global shape of the object accordingly the part importance measure is defined based on the ability to estimate recall the global shapes of objects from the local part namely the part s shape reconstructability more precisely the shape reconstructability of a part is determined by two factors part variation and part uniqueness i part variation measures the precision of the global shape reconstruction i e the consistency of the reconstructed global shape with the true object shape and ii part uniqueness quantifies the ambiguity of matching the part to the object i e taking into account that the part could be matched to the object at several different locations taking both these factors into consideration an information theoretic formulation is proposed to measure part importance by the conditional entropy of the reconstruction of the object shape from the part experimental results demonstrate the benefit with the proposed part importance in object detection including the improvement of detection rate localization accuracy and detection efficiency by comparing with other state of the art object detectors in a challenging but common scenario object detection with occlusions we show a considerable improvement using the proposed importance measure with the detection rate increased over 10 10 on a subset of the challenging pascal dataset the interpolated average precision as used in the pascal voc challenge is improved by 4 8 moreover we perform a psychological experiment which provides evidence suggesting that humans use a similar measure for part importance when perceiving and recognizing shapes

sign pattern matrices that admit p0 matrices abstract for sign patterns corresponding to directed or undirected cycles we identify conditions under which the patterns admit or require p 0 matrices

computing accurate correspondences across groups of images groupwise image registration algorithms seek to establish dense correspondences between sets of images typically they involve iteratively improving the registration between each image and an evolving mean a variety of methods have been proposed which differ in their choice of objective function representation of deformation field and optimization methods given the complexity of the task the final accuracy is significantly affected by the choices made for each component here we present a groupwise registration algorithm which can take advantage of the statistics of both the image intensities and the range of shapes across the group to achieve accurate matching by testing on large sets of images in both 2d and 3d we explore the effects of using different image representations and different statistical shape constraints we demonstrate that careful choice of such representations can lead to significant improvements in overall performance

tensor based aam with continuous variation estimation application to variation robust face recognition the active appearance model aam is a well known model that can represent a non rigid object effectively however the fitting result is often unsatisfactory when an input image deviates from the training images due to its fixed shape and appearance model to obtain more robust aam fitting we propose a tensor based aam that can handle a variety of subjects poses expressions and illuminations in the tensor algebra framework which consists of an image tensor and a model tensor the image tensor estimates image variations such as pose expression and illumination of the input image using two different variation estimation techniques discrete and continuous variation estimation the model tensor generates variation specific aam basis vectors from the estimated image variations which leads to more accurate fitting results to validate the usefulness of the tensor based aam we performed variation robust face recognition using the tensor based aam fitting results to do we propose indirect aam feature transformation experimental results show that tensor based aam with continuous variation estimation outperforms that with discrete variation estimation and conventional aam in terms of the average fitting error and the face recognition rate

fast active appearance model search using canonical correlation analysis a fast aam search algorithm based on canonical correlation analysis cca aam is introduced it efficiently models the dependency between texture residuals and model parameters during search experiments show that cca aams while requiring similar implementation effort consistently outperform standard search with regard to convergence speed by a factor of four

multi view aam fitting and construction active appearance models aams are generative parametric models that have been successfully used in the past to model deformable objects such as human faces the original aams formulation was 2d but they have recently been extended to include a 3d shape model a variety of single view algorithms exist for fitting and constructing 3d aams but one area that has not been studied is multi view algorithms in this paper we present multi view algorithms for both fitting and constructing 3d aams fitting an aam to an image consists of minimizing the error between the input image and the closest model instance i e solving a nonlinear optimization problem in the first part of the paper we describe an algorithm for fitting a single aam to multiple images captured simultaneously by cameras with arbitrary locations rotations and response functions this algorithm uses the scaled orthographic imaging model used by previous authors and in the process of fitting computes or calibrates the scaled orthographic camera matrices in the second part of the paper we describe an extension of this algorithm to calibrate weak perspective or full perspective camera models for each of the cameras in essence we use the human face as a non rigid calibration grid we demonstrate that the performance of this algorithm is roughly comparable to a standard algorithm using a calibration grid in the third part of the paper we show how camera calibration improves the performance of aam fitting a variety of non rigid structure from motion algorithms both single view and multi view have been proposed that can be used to construct the corresponding 3d non rigid shape models of a 2d aam in the final part of the paper we show that constructing a 3d face model using non rigid structure from motion suffers from the bas relief ambiguity and may result in a scaled stretched compressed model we outline a robust non rigid motion stereo algorithm for calibrated multi view 3d aam construction and show how using calibrated multi view motion stereo can eliminate the bas relief ambiguity and yield face models with higher 3d fidelity

face alignment by explicit shape regression we present a very efficient highly accurate explicit shape regression approach for face alignment unlike previous regression based approaches we directly learn a vectorial regression function to infer the whole facial shape a set of facial landmarks from the image and explicitly minimize the alignment errors over the training data the inherent shape constraint is naturally encoded into the regressor in a cascaded learning framework and applied from coarse to fine during the test without using a fixed parametric shape model as in most previous methods to make the regression more effective and efficient we design a two level boosted regression shape indexed features and a correlation based feature selection method this combination enables us to learn accurate models from large training data in a short time 20 min for 2 000 training images and run regression extremely fast in test 15 ms for a 87 landmarks shape experiments on challenging data show that our approach significantly outperforms the state of the art in terms of both accuracy and efficiency

a lattice based mrf model for dynamic near regular texture tracking a near regular texture nrt is a geometric and photometric deformation from its regular origin a congruent wallpaper pattern formed by 2d translations of a single tile a dynamic nrt is an nrt under motion although nrts are pervasive in man made and natural environments effective computational algorithms for nrts are few this paper addresses specific computational challenges in modeling and tracking dynamic nrts including ambiguous correspondences occlusions and drastic illumination and appearance variations we propose a lattice based markov random field mrf model for dynamic nrts in a 3d spatiotemporal space our model consists of a global lattice structure that characterizes the topological constraint among multiple textons and an image observation model that handles local geometry and appearance variations based on the proposed mrf model we develop a tracking algorithm that utilizes belief propagation and particle filtering to effectively handle the special challenges of the dynamic nrt tracking without any assumption on the motion types or lighting conditions we provide quantitative evaluations of the proposed method against existing tracking algorithms and demonstrate its applications in video editing

a functional regression approach to facial landmark tracking linear regression is a fundamental building block in many face detection and tracking algorithms typically used to predict shape displacements from image features through a linear mapping this paper presents a functional regression solution to the least squares problem which we coin continuous regression resulting in the first real time incremental face tracker contrary to prior work in functional regression in which b splines or fourier series were used we propose to approximate the input space by its first order taylor expansion yielding a closed form solution for the continuous domain of displacements we then extend the continuous least squares problem to correlated variables and demonstrate the generalisation of our approach we incorporate continuous regression into the cascaded regression framework and show its computational benefits for both training and testing we then present a fast approach for incremental learning within cascaded continuous regression coined iccr and show that its complexity allows real time face tracking being 20 times faster than the state of the art to the best of our knowledge this is the first incremental face tracker that is shown to operate in real time we show that iccr achieves state of the art performance on the 300 vw dataset the most recent large scale benchmark for face tracking

sign patterns of inverse nonnegative matrices abstract we determine here the 0 sign patterns which occur among the inverses of nonsingular entrywise nonnegative matrices these results complete a sequence of work which began with the characterization of all possible sign patterns which occur among inverses of positive matrices and the characterization in this general case is more involved to state than the earlier ones

rotation invariant kernels and their application to shape analysis shape analysis requires invariance under translation scale and rotation translation and scale invariance can be realized by normalizing shape vectors with respect to their mean and norm this maps the shape feature vectors onto the surface of a hypersphere after normalization the shape vectors can be made rotational invariant by modeling the resulting data using complex scalar rotation invariant distributions defined on the complex hypersphere e g using the complex bingham distribution however the use of these distributions is hampered by the difficulty in estimating their parameters and the nonlinear nature of their formulation in the present paper we show how a set of kernel functions that we refer to as rotation invariant kernels can be used to convert the original nonlinear problem into a linear one as their name implies these kernels are defined to provide the much needed rotation invariance property allowing one to bypass the difficulty of working with complex spherical distributions the resulting approach provides an easy fast mechanism for 2d 3d shape analysis extensive validation using a variety of shape modeling and classification problems demonstrates the accuracy of this proposed approach

dynamic affine invariant shape appearance handshape features and classification in sign language videos we propose the novel approach of dynamic affine invariant shape appearance model aff sam and employ it for handshape classification and sign recognition in sign language sl videos aff sam offers a compact and descriptive representation of hand configurations as well as regularized model fitting assisting hand tracking and extracting handshape features we construct sa images representing the hand s shape and appearance without landmark points we model the variation of the images by linear combinations of eigenimages followed by affine transformations accounting for 3d hand pose changes and improving model s compactness we also incorporate static and dynamic handshape priors offering robustness in occlusions which occur often in signing the approach includes an affine signer adaptation component at the visual level without requiring training from scratch a new singer specific model we rather employ a short development data set to adapt the models for a new signer experiments on the boston university 400 continuous sl corpus demonstrate improvements on handshape classification when compared to other feature extraction approaches supplementary evaluations of sign recognition experiments are conducted on a multi signer 100 sign data set from the greek sign language lemmas corpus these explore the fusion with movement cues as well as signer adaptation of aff sam to multiple signers providing promising results

statistical approach to shape from shading reconstruction of three dimensional face surfaces from single two dimensional images the human visual system is proficient in perceiving three dimensional shape from the shading patterns in a two dimensional image how it does this is not well understood and continues to be a question of fundamental and practical interest in this paper we present a new quantitative approach to shape from shading that may provide some answers we suggest that the brain through evolution or prior experience has discovered that objects can be classified into lower dimensional object classes as to their shape extraction of shape from shading is then equivalent to the much simpler problem of parameter estimation in a low dimensional space we carry out this proposal for an important class of three dimensional 3d objects human heads from an ensemble of several hundred laser scanned 3d heads we use principal component analysis to derive a low dimensional parameterization of head shape space an algorithm for solving shape from shading using this representation is presented it works well even on real images where it is able to recover the 3d surface for a given person maintaining facial detail and identity from a single 2d image of his face this algorithm has applications in face recognition and animation

robust object detection with interleaved categorization and segmentation this paper presents a novel method for detecting and localizing objects of a visual category in cluttered real world scenes our approach considers object categorization and figure ground segmentation as two interleaved processes that closely collaborate towards a common goal as shown in our work the tight coupling between those two processes allows them to benefit from each other and improve the combined performance the core part of our approach is a highly flexible learned representation for object shape that can combine the information observed on different training examples in a probabilistic extension of the generalized hough transform the resulting approach can detect categorical objects in novel images and automatically infer a probabilistic segmentation from the recognition result this segmentation is then in turn used to again improve recognition by allowing the system to focus its efforts on object pixels and to discard misleading influences from the background moreover the information from where in the image a hypothesis draws its support is employed in an mdl based hypothesis verification stage to resolve ambiguities between overlapping hypotheses and factor out the effects of partial occlusion an extensive evaluation on several large data sets shows that the proposed system is applicable to a range of different object categories including both rigid and articulated objects in addition its flexible representation allows it to achieve competitive object detection performance already from training sets that are between one and two orders of magnitude smaller than those used in comparable systems

matching 2 5d face scans to 3d models the performance of face recognition systems that use two dimensional images depends on factors such as lighting and subject s pose we are developing a face recognition system that utilizes three dimensional shape information to make the system more robust to arbitrary pose and lighting for each subject a 3d face model is constructed by integrating several 2 5d face scans which are captured from different views 2 5d is a simplified 3d x y z surface representation that contains at most one depth value z direction for every point in the x y plane two different modalities provided by the facial scan namely shape and texture are utilized and integrated for face matching the recognition engine consists of two components surface matching and appearance based matching the surface matching component is based on a modified iterative closest point icp algorithm the candidate list from the gallery used for appearance matching is dynamically generated based on the output of the surface matching component which reduces the complexity of the appearance based matching stage three dimensional models in the gallery are used to synthesize new appearance samples with pose and illumination variations and the synthesized face images are used in discriminant subspace analysis the weighted sum rule is applied to combine the scores given by the two matching components experimental results are given for matching a database of 200 3d face models with 598 2 5d independent test scans acquired under different pose and some lighting and expression changes these results show the feasibility of the proposed matching scheme

discriminative deep face shape model for facial point detection facial point detection is an active area in computer vision due to its relevance to many applications it is a nontrivial task since facial shapes vary significantly with facial expressions poses or occlusion in this paper we address this problem by proposing a discriminative deep face shape model that is constructed based on an augmented factorized three way restricted boltzmann machines model specifically the discriminative deep model combines the top down information from the embedded face shape patterns and the bottom up measurements from local point detectors in a unified framework in addition along with the model effective algorithms are proposed to perform model learning and to infer the true facial point locations from their measurements based on the discriminative deep face shape model 68 facial points are detected on facial images in both controlled and in the wild conditions experiments on benchmark data sets show the effectiveness of the proposed facial point detection algorithm against state of the art methods

morphable surface models we describe a novel automatic technique for finding a dense correspondence between a pair of n dimensional surfaces with arbitrary topologies this method employs a different formulation than previous correspondence algorithms such as optical flow and includes images as a special case we use this correspondence algorithm to build morphable surface models an extension of morphable models from examples we present a method for matching the model to new surfaces and demonstrate their use for analysis synthesis and clustering

learning aam fitting through simulation the active appearance model aam is a powerful method for modeling and segmenting deformable visual objects the utility of the aam stems from two fronts its compact representation as a linear object class and its rapid fitting procedure which utilizes fixed linear updates although the original fitting procedure works well for objects with restricted variability when initialization is close to the optimum its efficacy deteriorates in more general settings with regards to both accuracy and capture range in this paper we propose a novel fitting procedure where training is coupled with and directly addresses aam fitting in its deployment this is achieved by simulating the conditions of real fitting problems and learning the best set of fixed linear mappings such that performance over these simulations is optimized the power of the approach does not stem from an update model with larger capacity but from addressing the whole fitting procedure simultaneously to motivate the approach it is compared with a number of existing aam fitting procedures on two publicly available face databases it is shown that this method exhibits convergence rates capture range and convergence accuracy that are significantly better than other linear methods and comparable to a nonlinear method whilst affording superior computational efficiency

global propagation of affine invariant features for robust matching local invariant features have been successfully used in image matching to cope with viewpoint change partial occlusion and clutters however when these factors become too strong there will be a lot of mismatches due to the limited repeatability and discriminative power of features in this paper we present an efficient approach to remove the false matches and propagate the correct ones for the affine invariant features which represent the state of the art local invariance first a pair wise affine consistency measure is proposed to evaluate the consensus of the matches of affine invariant regions the measure takes into account both the keypoint location and the region shape size and orientation based on this measure a geometric filter is then presented which can efficiently remove the outliers from the initial matches and is robust to severe clutters and non rigid deformation to increase the correct matches we propose a global match refinement and propagation method that simultaneously finds a optimal group of local affine transforms to relate the features in two images the global method is capable of producing a quasi dense set of matches even for the weakly textured surfaces that suffer strong rigid transformation or non rigid deformation the strong capability of the proposed method in dealing with significant viewpoint change non rigid deformation and low texture objects is demonstrated in experiments of image matching object recognition and image based rendering

direct estimation of nonrigid registrations with image based self occlusion reasoning the registration problem for images of a deforming surface has been well studied external occlusions are usually well handled in 2d image based registration self occlusions are more challenging consequently the surface is usually assumed to be only slightly self occluding this paper is about image based nonrigid registration with self occlusion reasoning a specific framework explicitly modeling self occlusions is proposed it is combined with an intensity based ldquodirectrdquo data term for registration self occlusions are detected as shrinkage areas in the 2d warp experimental results on several challenging data sets show that our approach successfully registers images with self occlusions while effectively detecting the self occluded regions

a fast 2d shape recovery approach by fusing features and appearance in this paper we present a fusion approach to solve the nonrigid shape recovery problem which takes advantage of both the appearance information and the local features we have two major contributions first we propose a novel progressive finite newton optimization scheme for the feature based nonrigid surface detection problem which is reduced to only solving a set of linear equations the key is to formulate the nonrigid surface detection as an unconstrained quadratic optimization problem that has a closed form solution for a given set of observations second we propose a deformable lucas kanade algorithm that triangulates the template image into small patches and constrains the deformation through the second order derivatives of the mesh vertices we formulate it into a sparse regularized least squares problem which is able to reduce the computational cost and the memory requirement the inverse compositional algorithm is applied to efficiently solve the optimization problem we have conducted extensive experiments for performance evaluation on various environments whose promising results show that the proposed algorithm is both efficient and effective

unsupervised image matching based on manifold alignment this paper challenges the issue of automatic matching between two image sets with similar intrinsic structures and different appearances especially when there is no prior correspondence an unsupervised manifold alignment framework is proposed to establish correspondence between data sets by a mapping function in the mutual embedding space we introduce a local similarity metric based on parameterized distance curves to represent the connection of one point with the rest of the manifold a small set of valid feature pairs can be found without manual interactions by matching the distance curve of one manifold with the curve cluster of the other manifold to avoid potential confusions in image matching we propose an extended affine transformation to solve the nonrigid alignment in the embedding space the comparatively tight alignments and the structure preservation can be obtained simultaneously the point pairs with the minimum distance after alignment are viewed as the matchings we apply manifold alignment to image set matching problems the correspondence between image sets of different poses illuminations and identities can be established effectively by our approach

graph based construction of textured large field of view mosaics for bladder cancer diagnosis large field of view panoramic images greatly facilitate bladder cancer diagnosis and follow up such 2d mosaics can be obtained by registering the images of a video sequence acquired during cystoscopic examinations the scientific challenge in the registration process lies in the strong inter and intra patient texture variability of the images from which primitives cannot be robustly extracted state of the art registration methods are not at the same time robust and accurate especially for image pairs with a small amount of overlap less than 90 or strong perspective transformations moreover no previous contribution to cystoscopy mosaicing presents panoramic images created from multiple overlapping sequences e g zigzags or loop trajectories we show how such overlapping sections can be automatically detected and present a novel registration algorithm that robustly superimposes non consecutive image pairs which are related by stronger perspective transformations and share less overlap than consecutive images less than 50 globally coherent panoramic images are constructed using a non linear optimization and a novel contrast enhancing stitching method results on both phantom and patient data are obtained using constant algorithm parameters which demonstrate the robustness of the proposed method while the methods presented in this contribution are specifically designed for cystoscopy mosaicing they can also be applied to more general mosaicing problems we demonstrate this on a traditional stitching application where a set of pictures of a building are stitched into a seamless globally coherent panoramic image

pose and expression independent facial landmark localization using dense surf and the hausdorff distance we present an approach to automatic localization of facial feature points which deals with pose expression and identity variations combining 3d shape models with local image patch classification the latter is performed by means of densely extracted surf like features which we call du surf while the former is based on a multiclass version of the hausdorff distance to address local classification errors and nonvisible points the final system is able to localize facial points in real world scenarios dealing with out of plane head rotations expression changes and different lighting conditions extensive experimentation with the proposed method has been carried out showing the superiority of our approach with respect to other state of the art systems finally du surf features have been compared with other modern features and we experimentally demonstrate their competitive classification accuracy and computational efficiency

morphable models for the analysis and synthesis of complex motion patterns the linear combination of prototypical views provides a powerful approach for the recognition and the synthesis of images of stationary three dimensional objects in this article we present initial results that demonstrate that similar ideas can be developed for the recognition and synthesis of complex motion patterns we present a technique that permits to represent complex motion or action patterns by linear combinations of a small number of prototypical image sequences we demonstrate the applicability of this new approach for the synthesis and analysis of biological motion using simulated and real video data from different locomotion patterns our results show that complex motion patterns are embedded in pattern spaces with a defined topological structure which can be uncovered with our methods the underlying pattern space seems to have locally but not globally the properties of a linear vector space we show how the knowledge about the topology of the pattern space can be exploited during pattern recognition our method may provide a new interesting approach for the analysis and synthesis of video sequences and complex movements

multidimensional morphable models a framework for representing and matching object classes we describe a flexible model for representing images of objects of a certain class known a priori such as faces and introduce a new algorithm for matching it to a novel image and thereby perform image analysis the flexible model known as a multidimensional morphable model is learned from example images of objects of a class in this paper we introduce an effective stochastic gradient descent algorithm that automatically matches a model to a novel image several experiments demonstrate the robustness and the broad range of applicability of morphable models our approach can provide novel solutions to several vision tasks including the computation of image correspondence object verification and image compression

reconstruction of partially damaged face images based on a morphable face model this paper proposes a method for reconstructing partially damaged faces based on a morphable face model faces are modeled by linear combinations of prototypes of shape and texture with the shape and texture information from an undamaged region only we can estimate optimal coefficients for linear combinations of prototypes of shape and texture by simple projection for least square minimization lsm our experimental results show that reconstructed faces are very natural and plausible like real photos

characterizations of sign patterns of inverse positive matrices abstract a result of johnson leighton and robinson characterizing sign patterns of real matrices with nonzero entries whose inverses are entrywise positive is generalized the restriction to matrices with nonzero entries is removed and additional five equivalent conditions are established one of them using a graph theoretical approach expedites a simple criterion for recognition of such sign patterns

a fast approach to deformable surface 3d tracking deformable surface 3d tracking is a severely under constrained problem and great efforts have been made to solve it a recent state of the art approach solves this problem by formulating it as a second order cone programming socp problem however one drawback of this approach is that it is time consuming in this paper we propose an effective method for 3d deformable surface tracking first we formulate the deformable surface tracking problem as a linear programming lp problem then we solve the lp problem with an algorithm which converges superlinearly rather than bisection algorithm whose convergence speed is linear our experimental studies on synthetic and real data have demonstrated the proposed method can not only reliably recover 3d structures of surfaces but also run faster than the state of the art method

a physics based analysis of image appearance models linear and multilinear models pca 3dmm aam asm and multilinear tensors of object shape appearance have been very popular in computer vision in this paper we analyze the applicability of these heuristic models from the fundamental physical laws of object motion and image formation we prove that under suitable conditions the image appearance space can be closely approximated to be multilinear with the illumination and texture subspaces being trilinearly combined with the direct sum of the motion and deformation subspaces this result provides a physics based understanding of many of the successes and limitations of the linear and multilinear approaches existing in the computer vision literature and also identifies some of the conditions under which they are valid it provides an analytical representation of the image space in terms of different physical factors that affect the image formation process numerical analysis of the accuracy of the physics based models is performed and tracking results on real data are presented

consistent image registration presents a new method for image registration based on jointly estimating the forward and reverse transformations between two images while constraining these transforms to be inverses of one another this approach produces a consistent set of transformations that have less pairwise registration error i e better correspondence than traditional methods that estimate the forward and reverse transformations independently the transformations are estimated iteratively and are restricted to preserve topology by constraining them to obey the laws of continuum mechanics the transformations are parameterized by a fourier series to diagonalize the covariance structure imposed by the continuum mechanics constraints and to provide a computationally efficient numerical implementation results using a linear elastic material constraint are presented using both magnetic resonance and x ray computed tomography image data the results show that the joint estimation of a consistent set of forward and reverse transformations constrained by linear elasticity give better registration results than using either constraint alone or none at all

metric learning for image alignment image alignment has been a long standing problem in computer vision parameterized appearance models pams such as the lucas kanade method eigentracking and active appearance models are commonly used to align images with respect to a template or to a previously learned model while pams have numerous advantages relative to alternate approaches they have at least two drawbacks first they are especially prone to local minima in the registration process second often few if any of the local minima of the cost function correspond to acceptable solutions to overcome these problems this paper proposes a method to learn a metric for pams that explicitly optimizes that local minima occur at and only at the places corresponding to the correct fitting parameters to the best of our knowledge this is the first paper to address the problem of learning a metric to explicitly model local properties of the pams error surface synthetic and real examples show improvement in alignment performance in comparison with traditional approaches in addition we show how the proposed criteria for a good metric can be used to select good features to track

statistical shape analysis clustering learning and testing using a differential geometric treatment of planar shapes we present tools for 1 hierarchical clustering of imaged objects according to the shapes of their boundaries 2 learning of probability models for clusters of shapes and 3 testing of newly observed shapes under competing probability models clustering at any level of hierarchy is performed using a minimum variance type criterion and a markov process statistical means of clusters provide shapes to be clustered at the next higher level thus building a hierarchy of shapes using finite dimensional approximations of spaces tangent to the shape space at sample means we implicitly impose probability models on the shape space and results are illustrated via random sampling and classification hypothesis testing together hierarchical clustering and hypothesis testing provide an efficient framework for shape retrieval examples are presented using shapes and images from eth surrey and amcom databases

sign pattern matrices that admit m n p or inverse m matrices in this paper we identify the sign pattern matrices that occur among the n matrices the p matrices and the m matrices we also address to the class of inverse m matrices and the related admissibility of sign pattern matrices problem

anisotropic motion estimation on edge preserving riesz wavelets for robust video mosaicing image mosaicing is a technique widely used for extending the field of view of industrial medical outdoor or indoor scenes however image registration can be very challenging e g due to large texture variability illumination changes image blur and camera perspective changes in this paper a total variational optical flow approach is investigated to estimate dense point correspondences between image pairs an edge preserving riesz wavelet scale space combined with a novel tv regularizer is proposed for preserving motion discontinuities along the edges of weak textures and for handling strong in plane rotations present in image sequences an anisotropic weighted median filtering is implemented for minimizing outliers quantitative evaluation of the method on the middlebury image database and simulated sequences with known ground truth demonstrates high accuracy of the proposed method in comparison with other state of the art methods including a robust graph cut method and a patch matching approach qualitative results on video sequences of difficult real scenes demonstrate the robustness of the proposed method highlightsmosaicing of images with strong texture and illumination variability fast robust and accurate optical flow computation tv l1 method on a second order riesz wavelet basis for preserving texture discontinuities novel anisotropic tv regularizer accurate results for both the middlebury data set and various complicated scenes

global 2d digital image correlation for motion estimation in a finite element framework a variational formulation and a regularized pyramidal multi grid implementation in this study the inverse problem of reconstructing the in plane 2d displacements of a monitored surface through a sequence of two dimensional digital images severely ill posed in hadamard s sense is deeply investigated a novel variational formulation is presented for the continuum 2d digital image correlation problem and critical issues such as semi coercivity and solution multiplicity are discussed by functional analysis tools in the framework of a galerkin finite element discretization of the displacement field a robust implementation for 2d digital image correlation is outlined aiming to attenuate the spurious oscillations which corrupt the deformation scenario especially when very fine meshes are adopted recourse is made to a hierarchical family of grids linked by suitable restriction and prolongation operators and defined over an image pyramid multi grid cycles are performed ascending and descending along the pyramid with only one newton iteration per level irrespective of the tolerance satisfaction as if the problem were linear at each level the conventional least square matching functional is herein enriched by a tychonoff regularization provision preserving the solution against an unstable response the algorithm is assessed on the basis of both synthetic and truly experimental image pairs copyright 2013 john wiley sons ltd

transformation invariant subspace clustering subspace clustering has achieved great success in many computer vision applications however most subspace clustering algorithms require well aligned data samples which is often not straightforward to achieve this paper proposes a transformation invariant subspace clustering framework by jointly aligning data samples and learning subspace representation by alignment the transformed data samples become highly correlated and a better affinity matrix can be obtained the joint problem can be reduced to a sequence of least squares regression problems which can be efficiently solved we verify the effectiveness of the proposed method with extensive experiments on unaligned real data demonstrating its higher clustering accuracy than the state of the art subspace clustering and transformation invariant clustering algorithms highlightsa transformation invariant subspace clustering algorithm is proposed alignment and subspace clustering are mutually dependent state of the art clustering results on the unconstrained face dataset

synergy between object recognition and image segmentation using the expectation maximization algorithm in this work we formulate the interaction between image segmentation and object recognition in the framework of the expectation maximization em algorithm we consider segmentation as the assignment of image observations to object hypotheses and phrase it as the e step while the m step amounts to fitting the object models to the observations these two tasks are performed iteratively thereby simultaneously segmenting an image and reconstructing it in terms of objects we model objects using active appearance models aams as they capture both shape and appearance variation during the e step the fidelity of the aam predictions to the image is used to decide about assigning observations to the object for this we propose two top down segmentation algorithms the first starts with an oversegmentation of the image and then softly assigns image segments to objects as in the common setting of em the second uses curve evolution to minimize a criterion derived from the variational interpretation of em and introduces aams as shape priors for the m step we derive aam fitting equations that accommodate segmentation information thereby allowing for the automated treatment of occlusions apart from top down segmentation results we provide systematic experiments on object detection that validate the merits of our joint segmentation and recognition approach

active appearance models revisited active appearance models aams and the closely related concepts of morphable models and active blobs are generative models of a certain visual phenomenon although linear in both shape and appearance overall aams are nonlinear parametric models in terms of the pixel intensities fitting an aam to an image consists of minimising the error between the input image and the closest model instances i e solving a nonlinear optimisation problem we propose an efficient fitting algorithm for aams based on the inverse compositional image alignment algorithm we show that the effects of appearance variation during fitting can be precomputed projected out using this algorithm and how it can be extended to include a global shape normalising warp typically a 2d similarity transformation we evaluate our algorithm to determine which of its novel aspects improve aam fitting performance

rationalizing efficient compositional image alignment we study the issue of computational efficiency for gauss newton gn non linear least squares optimization in the context of image alignment we introduce the constant jacobian gauss newton cjgn optimization a gn scheme with constant jacobian and hessian matrices and the equivalence and independence conditions as the necessary requirements that any function of residuals must satisfy to be optimized with this efficient approach we prove that the inverse compositional ic image alignment algorithm is an instance of a cjgn scheme and formally derive the compositional and extended brightness constancy assumptions as the necessary requirements that must be satisfied by any image alignment problem so it can be solved with an efficient compositional scheme moreover in contradiction with previous results we also prove that the forward and inverse compositional algorithms are not equivalent they are equivalent however when the extended brightness constancy assumption is satisfied to analyze the impact of the satisfaction of these requirements we introduce a new image alignment evaluation framework and the concepts of short and wide baseline jacobian in wide baseline jacobian problems the optimization will diverge if the requirements are not satisfied however with a good initialization a short baseline jacobian problem may converge even if the requirements are not satisfied

active appearance models we describe a new method of matching statistical models of appearance to images a set of model parameters control modes of shape and gray level variation learned from a training set we construct an efficient iterative matching algorithm by learning the relationship between perturbations in the model parameters and the induced image errors

higher order complementary filters a higher order generalization of magnitude and power complementary filters is proposed the proposed scheme is shown to have superior frequency characteristics compared to the ordinary complementary filters applications of these generalized complementary filters include subband coding for audio and video and sharpening of magnitude characteristics of digital filters this new design procedure can be used to generate ordinary multichannel magnitude and power complementary filters with sharper passband responses a new method is outlined to obtain the class of filters that satisfies the higher order complementary condition for an arbitrary number of channels

parametric image alignment using enhanced correlation coefficient maximization in this work we propose the use of a modified version of the correlation coefficient as a performance criterion for the image alignment problem the proposed modification has the desirable characteristic of being invariant with respect to photometric distortions since the resulting similarity measure is a nonlinear function of the warp parameters we develop two iterative schemes for its maximization one based on the forward additive approach and the second on the inverse compositional method as it is customary in iterative optimization in each iteration the nonlinear objective function is approximated by an alternative expression for which the corresponding optimization is simple in our case we propose an efficient approximation that leads to a closed form solution per iteration which is of low computational complexity the latter property being particularly strong in our inverse version the proposed schemes are tested against the forward additive lucas kanade and the simultaneous inverse compositional algorithm through simulations under noisy conditions and photometric distortions our forward version achieves more accurate alignments and exhibits faster convergence whereas our inverse version has similar performance as the simultaneous inverse compositional algorithm but at a lower computational complexity

dynamic shape and appearance models we propose a model of the joint variation of shape and appearance of portions of an image sequence the model is conditionally linear and can be thought of as an extension of active appearance models to exploit the temporal correlation of adjacent image frames inference of the model parameters can be performed efficiently using established numerical optimization techniques borrowed from finite element analysis and system identification techniques

using bounded diameter minimum spanning trees to build dense active appearance models we present a method for producing dense active appearance models aams suitable for video realistic synthesis to this end we estimate a joint alignment of all training images using a set of pairwise registrations and ensure that these pairwise registrations are only calculated between similar images this is achieved by defining a graph on the image set whose edge weights correspond to registration errors and computing a bounded diameter minimum spanning tree dense optical flow is used to compute pairwise registration and a flow refinement method to align small scale texture is introduced further given the registration of training images vertices are added to the aam to minimise the error between the observed flow fields and the flow fields interpolated between the aam mesh points we demonstrate a significant improvement in model compactness

design of nonuniform near allpass complementary fir filters via a semi infinite programming technique in this correspondence we consider the problem of designing a set of nonuniform near allpass complementary finite impulse response fir filters this problem can be formulated as a quadratic semi infinite programming problem where the objective is to minimize the sum of the ripple energy for the individual filters subject to the passband and stopband specifications as well as to the allpass complementary specification the dual parameterization method is used for solving the linear quadratic semi infinite programming problem

bayesian constrained local models revisited this paper presents a novel bayesian formulation for aligning faces in unseen images our approach revisits the constrained local models clm formulation where an ensemble of local feature detectors are constrained to lie within the subspace spanned by a point distribution model pdm fitting such a model to an image typically involves two main steps a local search using a detector obtaining response maps for each landmark likelihood term and a global optimization that finds the pdm parameters that jointly maximize all the detections at once the so called global optimization can be posed as a bayesian inference problem where the posterior distribution of the shape and pose parameters can be inferred in a maximum a posteriori map sense this work introduces an extended bayesian global optimization strategy that includes two novel additions 1 to perform second order updates of the pdm parameters accounting for their covariance and 2 to model the underlying dynamics of the shape variations encoded in the prior term by using recursive bayesian estimation extensive evaluations were performed against state of the art methods on several standard datasets imm bioid xm2vts lfw and fgnet talking face results show that the proposed approach significantly increases the fitting performance

feature based lucas kanade and active appearance models lucas kanade and active appearance models are among the most commonly used methods for image alignment and facial fitting respectively they both utilize nonlinear gradient descent which is usually applied on intensity values in this paper we propose the employment of highly descriptive densely sampled image features for both problems we show that the strategy of warping the multichannel dense feature image at each iteration is more beneficial than extracting features after warping the intensity image at each iteration motivated by this observation we demonstrate robust and accurate alignment and fitting performance using a variety of powerful feature descriptors especially with the employment of histograms of oriented gradient and scale invariant feature transform features our method significantly outperforms the current state of the art results on in the wild databases

sequential non rigid structure from motion using physical priors we propose a new approach to simultaneously recover camera pose and 3d shape of non rigid and potentially extensible surfaces from a monocular image sequence for this purpose we make use of the extended kalman filter based simultaneous localization and mapping ekf slam formulation a bayesian optimization framework traditionally used in mobile robotics for estimating camera pose and reconstructing rigid scenarios in order to extend the problem to a deformable domain we represent the object s surface mechanics by means of navier s equations which are solved using a finite element method fem with these main ingredients we can further model the material s stretching allowing us to go a step further than most of current techniques typically constrained to surfaces undergoing isometric deformations we extensively validate our approach in both real and synthetic experiments and demonstrate its advantages with respect to competing methods more specifically we show that besides simultaneously retrieving camera pose and non rigid shape our approach is adequate for both isometric and extensible surfaces does not require neither batch processing all the frames nor tracking points over the whole sequence and runs at several frames per second

robust segment based object tracking using generalized hyperplane approximation tracking based on gradient descent algorithm using image gradient is one of the popular object tracking method however it easily fails to track when illumination changes although several illumination invariant features have been proposed applying the invariant feature to the gradient descent method is not easy because the invariant feature is represented as a non linear function of image pixel values and its jacobian cannot be calculated in a closed form to make it possible we introduce the generalized hyperplane approximation technique and apply it to histogram of oriented gradient hog feature one of the well known illumination invariant feature in addition we achieve partial occlusion invariance using image segments the hyperplanes are calculated from training segment images obtained by perturbing the motion parameter around the target region then it is used to map the difference in non linear feature of image onto the increment of alignment parameters this process is mathematically same to the gradient descent method the information from each segment is integrated by a simple weighted linear combination with confidence weights of segments compared to the previous tracking algorithms our method shows very fast and stable tracking results in experiments on several practical image sequences

generalizing inverse compositional and esm image alignment inverse compositional ic image alignment baker and matthews in int j comput vis 56 3 221 255 2004 uses the symmetry between the roles of the fixed and moving images for faster processing however it requires implementation of compositional optimizer update steps the ic approach can be viewed as an efficient way of computing the similarity measure derivative relative to the fixed image warp parameters since the mapping between the fixed and moving warp parameters is continuous and differentiable this derivative can be converted into the moving warp space using the chain rule this avoids the need for compositional update steps our generalization also allows the efficient second order method esm malis in proceedings of the 2004 ieee international conference on robotics and automation icra04 pp 1843 1848 2004 benhimane and malis in ieee rsj international conference on intelligent robots and systems 2004 malis and benhimane in robot auton syst 52 1 39 52 2005 to be applied to general parameterizations of the transformation experiments using multiple similarity measures and optimizers show that our generalized ic method equals or exceeds the performance of the original ic approach the generalized esm approach is more reliable than the classic approach as it increases the capture radius of the optimization

a class of magnitude complementary loudspeaker crossovers necessary and sufficient conditions are developed for a pair of transfer functions to have magnitude responses which sum to a constant if in addition the transfer function pair is constrained to be all pass complementary the transfer function pair so obtained is well suited for loudspeaker crossover applications such crossover systems are characterized by a pair of transfer functions which exhibit the same phase angle at all frequencies this property has twofold implications i in active crossover biamplified audio systems whereby the low frequencies and high frequencies are reproduced using separate power amplifiers in phase crossover transfer functions require less power in subsequent amplifier stages than other designs to achieve a given acoustic sound pressure level and ii the summed acoustic magnitude response is least sensitive to noncoincidental mounting of the low and high frequency transducers when the crossover transfer functions exhibit the same phase angle at all frequencies the class of transfer functions realizable is quite wide and includes squared versions of butterworth chebyshev and elliptic transfer functions of all orders

the laplacian pyramid as a compact image code we describe a technique for image encoding in which local operators of many scales but identical shape serve as the basis functions the representation differs from established techniques in that the code elements are localized in spatial frequency as well as in space pixel to pixel correlations are first removed by subtracting a lowpass filtered copy of the image from the image itself the result is a net data compression since the difference or error image has low variance and entropy and the low pass filtered image may represented at reduced sample density further data compression is achieved by quantizing the difference image these steps are then repeated to compress the low pass image iteration of the process at appropriately expanded scales generates a pyramid data structure the encoding process is equivalent to sampling the image with laplacian operators of many scales thus the code tends to enhance salient image features a further advantage of the present code is that it is well suited for many image analysis tasks as well as for image compression fast algorithms are described for coding and decoding

a unified gradient based approach for combining asm into aam active appearance model aam framework is a very useful method that can fit the shape and appearance model to the input image for various image analysis and synthesis problems however since the goal of the aam fitting algorithm is to minimize the residual error between the model appearance and the input image it often fails to accurately converge to the landmark points of the input image to alleviate this weakness we have combined active shape models asm into aams in which asms try to find correct landmark points using the local profile model since the original objective function of the asm search is not appropriate for combining these methods we derive a gradient based iterative method by modifying the objective function of the asm search then we propose a new fitting method that combines the objective functions of both asm and aam into a single objective function in a gradient based optimization framework experimental results show that the proposed fitting method reduces the average fitting error when compared with existing fitting methods such as asm aam and texture constrained asm tc asm and improves the performance of facial expression recognition significantly

monocular 3d tracking of deformable surfaces using sequential second order cone programming reconstructing structures of deformable objects from monocular image sequences is important for applications like visual servoing and augmented reality in this paper we propose a method to recover 3d shapes of deformable surfaces using sequential second order cone programming socp the key of our approach is to represent the surface as a triangulated mesh and introduce two sets of constraints one for model to image keypoint correspondences which are socp constraints another for retaining the original lengths of the mesh edges which are non convex constraints in the process of tracking the surface structure is iteratively updated by solving sequential socp feasibility problems in which the non convex constraints are replaced by a set of convex constraints over a local convex region the shape constraints used in our approach is more generic than previous methods that enables us to reliably recover surface shapes with smooth sharp and other complex deformations the capability and efficiency of our approach are evaluated quantitatively with synthetic image sequences and qualitatively with real image sequences

recognition by linear combinations of models an approach to visual object recognition in which a 3d object is represented by the linear combination of 2d images of the object is proposed it is shown that for objects with sharp edges as well as with smooth bounding contours the set of possible images of a given object is embedded in a linear space spanned by a small number of views for objects with sharp edges the linear combination representation is exact for objects with smooth boundaries it is an approximation that often holds over a wide range of viewing angles rigid transformations with or without scaling can be distinguished from more general linear transformations of the object by testing certain constraints placed on the coefficients of the linear combinations three alternative methods of determining the transformation that matches a model to a given image are proposed

eigentracking robust matching and tracking of articulated objects using a view based representation this paper describes an approach for tracking rigid and articulated objects using a view based representation the approach builds on and extends work on eigenspace representations robust estimation techniques and parameterized optical flow estimation first we note that the least squares image reconstruction of standard eigenspace techniques has a number of problems and we reformulate the reconstruction problem as one of robust estimation second we define a subspace constancy assumption that allows us to exploit techniques for parameterized optical flow estimation to simultaneously solve for the view of an object and the affine transformation between the eigenspace and the image to account for large affine transformations between the eigenspace and the image we define a multi scale eigenspace representation and a coarse to fine matching strategy finally we use these techniques to track objects over long image sequences in which the objects simultaneously undergo both affine image motions and changes of view in particular we use this eigentracking technique to track and recognize the gestures of a moving hand

surface deformation models for nonrigid 3d shape recovery three dimensional detection and shape recovery of a nonrigid surface from video sequences require deformation models to effectively take advantage of potentially noisy image data here we introduce an approach to creating such models for deformable 3d surfaces we exploit the fact that the shape of an inextensible triangulated mesh can be parameterized in terms of a small subset of the angles between its facets we use this set of angles to create a representative set of potential shapes which we feed to a simple dimensionality reduction technique to produce low dimensional 3d deformation models we show that these models can be used to accurately model a wide range of deforming 3d surfaces from video sequences acquired under realistic conditions

natural facial expression recognition using differential aam and manifold learning this paper proposes a novel natural facial expression recognition method that recognizes a sequence of dynamic facial expression images using the differential active appearance model aam and manifold learning as follows first the differential aam features dafs are computed by the difference of the aam parameters between an input face image and a reference neutral expression face image second manifold learning embeds the dafs on the smooth and continuous feature space third the input facial expression is recognized through two steps 1 computing the distances between the input image sequence and gallery image sequences using directed hausdorff distance dhd and 2 selecting the expression by a majority voting of k nearest neighbors k nn sequences in the gallery the dafs are robust and efficient for the facial expression analysis due to the elimination of the inter person camera and illumination variations since the dafs treat the neutral expression image as the reference image the neutral expression image must be found effectively this is done via the differential facial expression probability density model dfepdm using the kernel density approximation of the positively directional dafs changing from neutral to angry happy surprised and negatively directional dafs changing from angry happy surprised to neutral then a face image is considered to be the neutral expression if it has the maximum dfepdm in the input sequences experimental results show that 1 the dafs improve the facial expression recognition performance over conventional aam features by 20 and 2 the sequence based k nn classifier provides a 95 facial expression recognition performance on the facial expression database fed06

face relighting from a single image under arbitrary unknown lighting conditions in this paper we present a new method to modify the appearance of a face image by manipulating the illumination condition when the face geometry and albedo information is unknown this problem is particularly difficult when there is only a single image of the subject available recent research demonstrates that the set of images of a convex lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low dimensional linear subspace using a spherical harmonic representation moreover morphable models are statistical ensembles of facial properties such as shape and texture in this paper we integrate spherical harmonics into the morphable model framework by proposing a 3d spherical harmonic basis morphable model shbmm the proposed method can represent a face under arbitrary unknown lighting and pose simply by three low dimensional vectors i e shape parameters spherical harmonic basis parameters and illumination coefficients which are called the shbmm parameters however when the image was taken under an extreme lighting condition the approximation error can be large thus making it difficult to recover albedo information in order to address this problem we propose a subregion based framework that uses a markov random field to model the statistical distribution and spatial coherence of face texture which makes our approach not only robust to extreme lighting conditions but also insensitive to partial occlusions the performance of our framework is demonstrated through various experimental results including the improved rates for face recognition under extreme lighting conditions

projective structure from uncalibrated images structure from motion and recognition address the problem of reconstructing 3 d space in a projective framework from two or more views and the problem of artificially generating novel views of the scene from two given views reprojection the author describes an invariance relation that provides a new description of structure which the author calls projective depth that is captured by a single equation relating image point correspondences across two or more views and the homographics of two arbitrary virtual planes the framework is based on knowledge of correspondence of features across views is linear and extremely simple and the computations of structure readily extend to overdetermination using multiple views experimental results demonstrate a high degree of accuracy in both tasks reconstruction and reprojection

lucas kanade 20 years on a unifying framework since the lucas kanade algorithm was proposed in 1981 image alignment has become one of the most widely used techniques in computer vision applications range from optical flow and tracking to layered motion mosaic construction and face coding numerous algorithms have been proposed and a wide variety of extensions have been made to the original formulation we present an overview of image alignment describing most of the algorithms and their extensions in a consistent framework we concentrate on the inverse compositional algorithm an efficient algorithm that we recently proposed we examine which of the extensions to lucas kanade can be used with the inverse compositional algorithm without any significant loss of efficiency and which cannot in this paper part 1 in a series of papers we cover the quantity approximated the warp update rule and the gradient descent approximation in future papers we will cover the choice of the error function how to allow linear appearance variation and how to impose priors on the parameters

graph laplace for occluded face completion and recognition this paper proposes a spectral graph based algorithm for face image repairing which can improve the recognition performance on occluded faces the face completion algorithm proposed in this paper includes three main procedures 1 sparse representation for partially occluded face classification 2 image based data mining and 3 graph laplace gl for face image completion the novel part of the proposed framework is gl as named from graphical models and the laplace equation and can achieve a high quality repairing of damaged or occluded faces the relationship between the gl and the traditional poisson equation is proven we apply our face repairing algorithm to produce completed faces and use face recognition to evaluate the performance of the algorithm experimental results verify the effectiveness of the gl method for occluded face completion

theory and practice of hierarchical data driven descent for optimal deformation estimation real world surfaces such as clothing water and human body deform in complex ways estimating deformation parameters accurately and reliably is hard due to its high dimensional and non convex nature optimization based approaches require good initialization while regression based approaches need a large amount of training data recently to achieve globally optimal estimation data driven descent tian and narasimhan in int j comput vis 98 279 302 2012 applies nearest neighbor estimators trained on a particular distribution of training samples to obtain a globally optimal and dense deformation field between a template and a distorted image in this work we develop a hierarchical structure that first applies nearest neighbor estimators on the entire image iteratively to obtain a rough estimation and then applies estimators with local image support to refine the estimation compared to its non hierarchical version our approach has the theoretical guarantees with significantly fewer training samples is faster by several orders provides a better metric deciding whether a given image requires more or fewer samples and can handle more complex scenes that include a mixture of global motion and local deformation we demonstrate in both simulation and real experiments that the proposed algorithm successfully tracks a broad range of non rigid scenes including water clothing and medical images and compares favorably against several other deformation estimation and tracking approaches that do not provide optimality guarantees

shape appearance correlated active appearance model among the challenges faced by current active shape or appearance models facial feature localization in the wild with occlusion in a novel face image i e in a generic environment is regarded as one of the most difficult computer vision tasks in this paper we propose an active appearance model aam to tackle the problem of generic environment firstly a fast face model initialization scheme is proposed based on the idea that the local appearance of feature points can be accurately approximated with locality constraints nearest neighbors which have similar poses and textures to a test face are retrieved from a training set for constructing the initial face model to further improve the fitting of the initial model to the test face an orthogonal cca occa is employed to increase the correlation between shape features and appearance features represented by principal component analysis pca with these two contributions we propose a novel aam namely the shape appearance correlated aam sac aam and the optimization is solved by using the recently proposed fast simultaneous inverse compositional fast sic algorithm experiment results demonstrate a 5 10 improvement on controlled and semi controlled datasets and with around 10 improvement on wild face datasets in terms of fitting accuracy compared to other state of the art aam models a fast face model initialization scheme is proposed for aams orthogonal cca is employed to increase the correlation between shape and appearance a shape appearance correlated aam is proposed to tackle the generic problem of localizing feature points on unseen faces

constrained active appearance models for segmentation of triplane echocardiograms this paper presents multiview and multiframe active appearance models aams for left ventricular segmentation in triplane echocardiograms we describe a general way of integrating local edge detector based segmentation algorithms into the aam framework the feasibility of this approach is evaluated by comparing an aam constrained by a dynamic programming dp based snake with an unconstrained aam and an aam constrained by manually defined landmarks a leave one out validation scheme was used for training and testing of the methods evaluation was done in 36 patients suffering from various heart diseases using manually determined volumes and ejection fractions ef as reference the segmentation was initialized by manual selection of the mitral annulus and apex in three imaging planes the differences in volume between manual segmentation and the best automatic method dp constrained aam were 3 1 plusmn 20 ml meanplusmnsd at end diastole and 0 61 plusmn 13 ml at end systole the difference in ef was 1 3 plusmn 6 3 comparable to the interobserver variability we show that 1 constraining the model to manually defined landmarks improves volume and ef estimates compared to unconstrained aams 2 further improvement is achieved using a dp constrained aam and 3 segmentation in triplane echocardiograms gives higher accuracy than single plane data

convex optimization for nonrigid stereo reconstruction we present a method for recovering 3 d nonrigid structure from an image pair taken with a stereo rig more specifically we dedicate to recover shapes of nearly inextensible deformable surfaces in our approach we represent the surface as a 3 d triangulated mesh and formulate the reconstruction problem as an optimization problem consisting of data terms and shape terms the data terms are model to image keypoint correspondences which can be formulated as second order cone programming socp constraints using l norm the shape terms are designed to retaining original lengths of mesh edges which are typically nonconvex constraints we will show that this optimization problem can be turned into a sequence of socp feasibility problems in which the nonconvex constraints are approximated as a set of convex constraints thanks to the efficient socp solver the reconstruction problem can then be solved reliably and efficiently as opposed to previous methods ours neither involves smoothness constraints nor need an initial estimation which enables us to recover shapes of surfaces with smooth sharp and other complex deformations from a single image pair the robustness and accuracy of our approach are evaluated quantitatively on synthetic data and qualitatively on real data

linear local models for monocular reconstruction of deformable surfaces recovering the 3d shape of a nonrigid surface from a single viewpoint is known to be both ambiguous and challenging resolving the ambiguities typically requires prior knowledge about the most likely deformations that the surface may undergo it often takes the form of a global deformation model that can be learned from training data while effective this approach suffers from the fact that a new model must be learned for each new surface which means acquiring new training data and may be impractical in this paper we replace the global models by linear local models for surface patches which can be assembled to represent arbitrary surface shapes as long as they are made of the same material not only do they eliminate the need to retrain the model for different surface shapes they also let us formulate 3d shape reconstruction from correspondences as either an algebraic problem that can be solved in closed form or a convex optimization problem whose solution can be found using standard numerical packages we present quantitative results on synthetic data as well as qualitative results on real images

monocular 3 d tracking of inextensible deformable surfaces under l_2 norm we present a method for recovering the 3 d shape of an inextensible deformable surface from a monocular image sequence state of the art methods on this problem utilize l norm of reprojection residual vectors and formulate the tracking problem as a second order cone programming socp problem instead of using l which is sensitive to outliers we use l 2 norm of reprojection errors generally using l 2 leads a nonconvex optimization problem which is difficult to minimize instead of solving the nonconvex problem directly we design an iterative l 2 norm approximation process to approximate the nonconvex objective function in which only a linear system needs to be solved at each iteration furthermore we introduce a shape regularization term into this iterative process in order to keep the inextensibility of the recovered mesh compared with previous methods ours performs more robust to image noises outliers and large interframe motions with high computational efficiency the robustness and accuracy of our approach are evaluated quantitatively on synthetic data and qualitatively on real data

globally optimal estimation of nonrigid image distortion image alignment in the presence of non rigid distortions is a challenging task typically this involves estimating the parameters of a dense deformation field that warps a distorted image back to its undistorted template generative approaches based on parameter optimization such as lucas kanade can get trapped within local minima on the other hand discriminative approaches like nearest neighbor require a large number of training samples that grows exponentially with respect to the dimension of the parameter space and polynomially with the desired accuracy 1 in this work we develop a novel data driven iterative algorithm that combines the best of both generative and discriminative approaches for this we introduce the notion of a pull back operation that enables us to predict the parameters of the test image using training samples that are not in its neighborhood not close in the parameter space we prove that our algorithm converges to the global optimum using a significantly lower number of training samples that grows only logarithmically with the desired accuracy we analyze the behavior of our algorithm extensively using synthetic data and demonstrate successful results on experiments with complex deformations due to water and clothing