optimization algorithms on subspaces revisiting missing data problem in low rank matrix low rank matrix approximation has applications in many fields such as 3d reconstruction from an image sequence and 2d filter design in this paper one issue with low rank matrix approximation is re investigated the missing data problem much effort was devoted to this problem and the wiberg algorithm or the damped newton algorithm were recommended in previous studies however the wiberg or damped newton algorithms do not suit for large especially long matrices because one needs to solve a large linear system in every iteration in this paper we revitalize the usage of the levenberg marquardt algorithm for solving the missing data problem by utilizing the property that low rank approximation is a minimization problem on subspaces in two proposed implementations of the levenberg marquardt algorithm one only needs to solve a much smaller linear system in every iteration especially for long matrices simulations and experiments on real data show the superiority of the proposed algorithms though the proposed algorithms achieve a high success rate in estimating the optimal solution by random initialization as illustrated by real examples it still remains an open issue how to properly do the initialization in a severe situation that is a large amount of data is missing and with high level noise

l_ 1 norm low rank matrix decomposition by neural networks and mollifiers the l 1 norm cost function of the low rank approximation of the matrix with missing entries is not smooth and also cannot be transformed into a standard linear or quadratic programming problem and thus the optimization of this cost function is still not well solved to tackle this problem first a mollifier is used to smooth the cost function high closeness of the smoothed function to the original one can be obtained by tuning the parameters contained in the mollifier next a recurrent neural network is proposed to optimize the mollified function which will converge to a local minimum in addition to boost the speed of the system the mollifying process is implemented by a filtering procedure the influence of two mollifier parameters is theoretically analyzed and experimentally confirmed showing that one of the parameters is critical to computational efficiency and accuracy while the other not a large number of experiments on synthetic data show that the proposed method is competitive to the state of the art methods in particular the experiments on large matrices and a real application in the structure from motion indicate that the memory requirement of the proposed algorithm is mild making it suitable for real applications that often involve large scale matrix decomposition

toward dynamic image mosaic generation with robustness to parallax mosaicing is largely dependent on the quality of registration among the constituent input images parallax and object motion present challenges to image registration leading to artifacts in the result to reduce the impact of these artifacts traditional image mosaicing approaches often impose planar scene constraints or rely on purely rotational camera motion or dense sampling however these requirements are often impractical or fail to address the needs of all applications instead taking advantage of depth cues and a smooth transition criterion we achieve significantly improved mosaicing results for static scenes coping effectively with nontrivial parallax in the input we extend this approach to the synthesis of dynamic video mosaics incorporating foreground background segmentation and a consistent motion perception criterion although further additions are required to cope with unconstrained object motion our algorithm can synthesize a perceptually convincing output conveying the same appearance of motion as seen in the input sequences

on projection matrices cal p k rightarrow cal p 2 k 3 ldots 6 and their applications in computer vision projection matrices from projective spaces cal p 3 to cal p 2 have long been used in multiple view geometry to model the perspective projection created by the pin hole camera in this work we introduce higher dimensional mappings cal p k rightarrow cal p 2 k 3 4 5 6 for the representation of various applications in which the world we view is no longer rigid we also describe the multi view constraints from these new projection matrices where k 3 and methods for extracting the non rigid structure and motion for each application

a paraperspective factorization method for shape and motion recovery the factorization method first developed by tomasi and kanade 1992 recovers both the shape of an object and its motion from a sequence of images using many images and tracking many feature points to obtain highly redundant feature position information the method robustly processes the feature trajectory information using singular value decomposition svd taking advantage of the linear algebraic properties of orthographic projection however an orthographic formulation limits the range of motions the method can accommodate paraperspective projection first introduced by ohta et al 1981 is a projection model that closely approximates perspective projection by modeling several effects not modeled under orthographic projection while retaining linear algebraic properties our paraperspective factorization method can be applied to a much wider range of motion scenarios including image sequences containing motion toward the camera and aerial image sequences of terrain taken from a low altitude airplane

a multibody factorization method for independently moving objects the structure from motion problem has been extensively studied in the field of computer vision yet the bulk of the existing work assumes that the scene contains only a single moving object the more realistic case where an unknown number of objects move in the scene has received little attention especially for its theoretical treatment in this paper we present a new method for separating and recovering the motion and shape of multiple independently moving objects in a sequence of images the method does not require prior knowledge of the number of objects nor is dependent on any grouping of features into an object at the image level for this purpose we introduce a mathematical construct of object shapes called the shape interaction matrix which is invariant to both the object motions and the selection of coordinate systems this invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects once the matrix is computed it allows for segmenting features into objects by the process of transforming it into a canonical form as well as recovering the shape and motion of each object the theory works under a broad set of projection models scaled orthography paraperspective and affine but they must be linear so it excludes projective cameras

3d trajectory reconstruction under perspective projection we present an algorithm to reconstruct the 3d trajectory of a moving point from its correspondence in a collection of temporally non coincidental 2d perspective images given the time of capture that produced each image and the relative camera poses at each time instant triangulation based solutions do not apply as multiple views of the point may not exist at each time instant we represent a 3d trajectory using a linear combination of compact trajectory basis vectors such as the discrete cosine transform basis that have been shown to approximate object independence we note that such basis vectors are also coordinate independent which allows us to directly use camera poses estimated from stationary areas in the scene in contrast to nonrigid structure from motion techniques where cameras are simultaneously estimated this reduces the reconstruction optimization to a linear least squares problem allowing us to robustly handle missing data that often occur due to motion blur texture deformation and self occlusion we present an algorithm to determine the number of trajectory basis vectors individually for each trajectory via a cross validation scheme and refine the solution by minimizing the geometric error the relationship between point and camera motion can cause degeneracies to occur we geometrically analyze the problem by studying the relationship of the camera motion point motion and trajectory basis vectors we define the reconstructability of a 3d trajectory under projection and show that the estimate approaches the ground truth when reconstructability approaches infinity this analysis enables us to precisely characterize cases when accurate reconstruction is achievable we present qualitative results for the reconstruction of several real world scenes from a series of 2d projections where high reconstructability can be guaranteed and report quantitative results on motion capture sequences

spatio temporal union of subspaces for multi body non rigid structure from motion abstract non rigid structure from motion nrsfm has so far been mostly studied for recovering 3d structure of a single non rigid deforming object to handle the real world challenging multiple deforming objects scenarios existing methods either pre segment different objects in the scene or treat multiple non rigid objects as a whole to obtain the 3d non rigid reconstruction however these methods fail to exploit the inherent structure in the problem as the solution of segmentation and the solution of reconstruction could not benefit each other in this paper we propose a unified framework to jointly segment and reconstruct multiple non rigid objects to compactly represent complex multi body non rigid scenes we propose to exploit the structure of the scenes along both temporal direction and spatial direction thus achieving a spatio temporal representation specifically we represent the 3d non rigid deformations as lying in a union of subspaces along the temporal direction and represent the 3d trajectories as lying in the union of subspaces along the spatial direction this spatio temporal representation not only provides competitive 3d reconstruction but also outputs robust segmentation of multiple non rigid objects the resultant optimization problem is solved efficiently using the alternating direction method of multipliers admm extensive experimental results on both synthetic and real multi body nrsfm datasets demonstrate the superior performance of our proposed framework compared with the state of the art methods

subspace procrustes analysis procrustes analysis pa has been a popular technique to align and build 2 d statistical models of shapes given a set of 2 d shapes pa is applied to remove rigid transformations later a non rigid 2 d model is computed by modeling the residual e g pca although pa has been widely used it has several limitations for modeling 2 d shapes occluded landmarks and missing data can result in local minima solutions and there is no guarantee that the 2 d shapes provide a uniform sampling of the 3 d space of rotations for the object to address previous issues this paper proposes subspace pa spa given several instances of a 3 d object spa computes the mean and a 2 d subspace that can model rigid and non rigid deformations of the 3 d object we propose a discrete dspa and continuous cspa formulation for spa assuming that 3 d samples of an object are provided dspa extends the traditional pa and produces unbiased 2 d models by uniformly sampling different views of the 3 d object cspa provides a continuous approach to uniformly sample the space of 3 d rotations being more efficient in space and time we illustrate the benefits of spa in two different applications first spa is used to learn 2 d face and body models from 3 d datasets experiments on the facewarehouse and cmu motion capture mocap datasets show the benefits of our 2 d models against the state of the art pa approaches and conventional 3 d models second spa learns an unbiased 2 d model from cmu mocap dataset and it is used to estimate the human pose on the leeds sports dataset

self calibration of an affine camera from multiple views a key limitation of all existing algorithms for shape and motion from image sequences under orthographic weak perspective and para perspective projection is that they require the calibration parameters of the camera we present in this paper a new approach that allows the shape and motion to be computed from image sequences without having to know the calibration parameters this approach is derived with the affine camera model introduced by mundy and zisserman 1992 which is a more general class of projections including orthographic weak perspective and para perspective projection models the concept of self calibration introduced by maybank and faugeras 1992 for the perspective camera and by hartley 1994 for the rotating camera is then applied for the affine camera this paper introduces the 3 intrinsic parameters that the affine camera can have at most the intrinsic parameters of the affine camera are closely related to the usual intrinsic parameters of the pin hole perspective camera but are different in the general case based on the invariance of the intrinsic parameters methods of self calibration of the affine camera are proposed it is shown that with at least four views an affine camera may be self calibrated up to a scaling factor leading to euclidean similarity shape reconstruction up to a global scaling factor another consequence of the introduction of intrinsic and extrinsic parameters of the affine camera is that all existing algorithms using calibrated affine cameras can be assembled into the same framework and some of them can be easily extented to a batch solution experimental results are presented and compared with other methods using calibrated affine cameras

quasi perspective projection model theory and application to structure and motion factorization from uncalibrated image sequences this paper addresses the problem of factorization based 3d reconstruction from uncalibrated image sequences previous studies on structure and motion factorization are either based on simplified affine assumption or general perspective projection the affine approximation is widely adopted due to its simplicity whereas the extension to perspective model suffers from recovering projective depths to fill the gap between simplicity of affine and accuracy of perspective model we propose a quasi perspective projection model for structure and motion recovery of rigid and nonrigid objects based on factorization framework the novelty and contribution of this paper are as follows firstly under the assumption that the camera is far away from the object with small lateral rotations we prove that the imaging process can be modeled by quasi perspective projection which is more accurate than affine model from both geometrical error analysis and experimental studies secondly we apply the model to establish a framework of rigid and nonrigid factorization under quasi perspective assumption finally we propose an extended cholesky decomposition to recover the rotation part of the euclidean upgrading matrix we also prove that the last column of the upgrading matrix corresponds to a global scale and translation of the camera thus may be set freely the proposed method is validated and evaluated extensively on synthetic and real image sequences and improved results over existing schemes are observed

procrustean normal distribution for non rigid structure from motion a well defined deformation model can be vital for non rigid structure from motion nrsfm most existing methods restrict the deformation space by assuming a fixed rank or smooth deformation which are not exactly true in the real world and they require the degree of deformation to be predetermined which is impractical meanwhile the errors in rotation estimation can have severe effects on the performance i e these errors can make a rigid motion be misinterpreted as a deformation in this paper we propose an alternative to resolve these issues motivated by an observation that non rigid deformations excluding rigid changes can be concisely represented in a linear subspace without imposing any strong constraints such as smoothness or low rank this observation is embedded in our new prior distribution the procrustean normal distribution pnd which is a shape distribution exclusively for non rigid deformations because of this unique characteristic of the pnd rigid and non rigid changes can be strictly separated which leads to better performance the proposed algorithm em pnd fits a pnd to given 2d observations to solve nrsfm without any user determined parameters the experimental results show that em pnd gives the state of the art performance for the benchmark data sets confirming the adequacy of the new deformation model

complex non rigid 3d shape recovery using a procrustean normal distribution mixture model recovering the 3d shape of a non rigid object is a challenging problem existing methods make the low rank assumption and do not scale well with the increased degree of freedom found in complex non rigid deformations or shape variations moreover in general the degree of freedom of deformation is assumed to be known in advance which limits the applicability of non rigid structure from motion algorithms in a practical situation in this paper we propose a method for handling complex shape variations based on the assumption that complex shape variations can be represented probabilistically by a mixture of primitive shape variations the proposed model is a generative probabilistic model called a procrustean normal distribution mixture model which can model complex shape variations without rank constraints experimental results show that the proposed method significantly outperforms existing methods

nonrigid structure from motion estimating shape and motion with hierarchical priors this paper describes methods for recovering time varying shape and motion of nonrigid 3d objects from uncalibrated 2d point tracks for example given a video recording of a talking person we would like to estimate the 3d shape of the face at each instant and learn a model of facial deformation time varying shape is modeled as a rigid transformation combined with a nonrigid deformation reconstruction is ill posed if arbitrary deformations are allowed and thus additional assumptions about deformations are required we first suggest restricting shapes to lie within a low dimensional subspace and describe estimation algorithms however this restriction alone is insufficient to constrain reconstruction to address these problems we propose a reconstruction method using a probabilistic principal components analysis ppca shape model and an estimation algorithm that simultaneously estimates 3d shape and motion for each instant learns the ppca model parameters and robustly fills in missing data points we then extend the model to represent temporal dynamics in object shape allowing the algorithm to robustly handle severe cases of missing data

sparse representation for 3d shape estimation a convex relaxation approach we investigate the problem of estimating the 3d shape of an object defined by a set of 3d landmarks given their 2d correspondences in a single image a successful approach to alleviating the reconstruction ambiguity is the 3d deformable shape model and a sparse representation is often used to capture complex shape variability but the model inference is still challenging due to the nonconvexity in the joint optimization of shape and viewpoint in contrast to prior work that relies on an alternating scheme whose solution depends on initialization we propose a convex approach to addressing this challenge and develop an efficient algorithm to solve the proposed convex program we further propose a robust model to handle gross errors in the 2d correspondences we demonstrate the exact recovery property of the proposed method the advantage compared to several nonconvex baselines and the applicability to recover 3d human poses and car models from single images

kruppa s equations derived from the fundamental matrix the purpose of this paper is to give a specific form for kruppa s equations in terms of the fundamental matrix kruppa s equations can be written explicitly in terms of the singular value decomposition svd of the fundamental matrix

a simple prior free method for non rigid structure from motion factorization this paper proposes a simple prior free method for solving the non rigid structure from motion nrsfm factorization problem other than using the fundamental low order linear combination model assumption our method does not assume any extra prior knowledge either about the non rigid structure or about the camera motions yet it works effectively and reliably producing optimal results and not suffering from the inherent basis ambiguity issue which plagued most conventional nrsfm factorization methods our method is very simple to implement which involves solving a very small sdp semi definite programming of fixed size and a nuclear norm minimization problem we also present theoretical analysis on the uniqueness and the relaxation gap of our solutions extensive experiments on both synthetic and real motion capture data assuming following the low order linear combination model are conducted which demonstrate that our method indeed outperforms most of the existing non rigid factorization methods this work offers not only new theoretical insight but also a practical everyday solution to nrsfm

depth estimation of face images using the nonlinear least squares model in this paper we propose an efficient algorithm to reconstruct the 3d structure of a human face from one or more of its 2d images with different poses in our algorithm the nonlinear least squares model is first employed to estimate the depth values of facial feature points and the pose of the 2d face image concerned by means of the similarity transform furthermore different optimization schemes are presented with regard to the accuracy levels and the training time required our algorithm also embeds the symmetrical property of the human face into the optimization procedure in order to alleviate the sensitivities arising from changes in pose in addition the regularization term based on linear correlation is added in the objective function to improve the estimation accuracy of the 3d structure further a model integration method is proposed to improve the depth estimation accuracy when multiple nonfrontal view face images are available experimental results on the 2d and 3d databases demonstrate the feasibility and efficiency of the proposed methods

adaptive non rigid registration and structure from motion from image trajectories this paper addresses the problem of registering a known 3d model to a set of 2d deforming image trajectories the proposed approach can adapt to a scenario where the 3d model to register is not an exact description of the measured image data this results in finding a 2d 3d registration given the complexity of having both 2d deforming data and a coarse description of the image observations the method acts in two distinct phases first an affine step computes a factorization for both the 2d image data and the 3d model using a joint subspace decomposition this initial solution is then upgraded by finding the best projection to the image plane complying with the metric constraints given by a scaled orthographic camera both steps are computed efficiently in closed form with the additional feature of being robust to degenerate motions which may possibly affect the 2d image data i e lack of relevant rigid motion a further extension of the approach allows to compute the full 3d deformations of the shape given the first initial rigid registration this step results in solving a non rigid structure from motion nrsfm problem using the 3d known shape as a prior experimental results show the robustness of the method in registration tasks such as pose estimation and 3d reconstruction when degenerate image motion is present

multiframe motion segmentation with missing data using powerfactorization and gpca we consider the problem of segmenting multiple rigid body motions from point correspondences in multiple affine views we cast this problem as a subspace clustering problem in which point trajectories associated with each motion live in a linear subspace of dimension two three or four our algorithm involves projecting all point trajectories onto a 5 dimensional subspace using the svd the powerfactorization method or ransac and fitting multiple linear subspaces representing different rigid body motions to the points in 5 using gpca unlike previous work our approach does not restrict the motion subspaces to be four dimensional and independent instead it deals gracefully with all the spectrum of possible affine motions from two dimensional and partially dependent to four dimensional and fully independent our algorithm can handle the case of missing data meaning that point tracks do not have to be visible in all images by using the powerfactorization method to project the data in addition our method can handle outlying trajectories by using ransac to perform the projection we compare our approach to other methods on a database of 167 motion sequences with full motions independent motions degenerate motions partially dependent motions missing data outliers etc on motion sequences with complete data our method achieves a misclassification error of less that 5 for two motions and 29 for three motions

stratified generalized procrustes analysis generalized procrustes analysis computes the best set of transformations that relate matched shape data in shape analysis the transformations are usually chosen as similarities while in general statistical data analysis other types of transformation groups such as the affine group may be used generalized procrustes analysis has a nonlinear and nonconvex formulation the classical approach alternates the computation of a so called reference shape and the computation of transformations relating this reference shape to each shape datum in turn we propose the stratified approach to generalized procrustes analysis it first uses the affine transformation group to analyze the data and then upgrades the solution to the sought after group whether euclidean or similarity we derive a convex formulation for each of these two steps and efficient practical algorithms that gracefully handle missing data incomplete shapes extensive experimental results show that our approaches perform well on simulated and real data in particular our closed form solution gives very accurate results for generalized procrustes analysis of euclidean data

combining local physical and global statistical models for sequential deformable shape from motion in this paper we simultaneously estimate camera pose and non rigid 3d shape from a monocular video using a sequential solution that combines local and global representations we model the object as an ensemble of particles each ruled by the linear equation of the newton s second law of motion this dynamic model is incorporated into a bundle adjustment framework in combination with simple regularization components that ensure temporal and spatial consistency the resulting approach allows to sequentially estimate shape and camera poses while progressively learning a global low rank model of the shape that is fed back into the optimization scheme introducing thus global constraints the overall combination of local physical and global statistical constraints yields a solution that is both efficient and robust to several artifacts such as noisy and missing data or sudden camera motions without requiring any training data at all validation is done in a variety of real application domains including articulated and non rigid motion both for continuous and discontinuous shapes our on line methodology yields significantly more accurate reconstructions than competing sequential approaches being even comparable to the more computationally demanding batch methods

a robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry abstract this paper proposes a robust approach to image matching by exploiting the only available geometric constraint namely the epipolar constraint the images are uncalibrated namely the motion between them and the camera parameters are not known thus the images can be taken by different cameras or a single camera at different time instants if we make an exhaustive search for the epipolar geometry the complexity is prohibitively high the idea underlying our approach is to use classical techniques correlation and relaxation methods in our particular implementation to find an initial set of matches and then use a robust technique the least median of squares lmeds to discard false matches in this set the epipolar geometry can then be accurately estimated using a meaningful image criterion more matches are eventually found as in stereo matching by using the recovered epipolar geometry a large number of experiments have been carried out and very good results have been obtained regarding the relaxation technique we define a new measure of matching support which allows a higher tolerance to deformation with respect to rigid transformations in the image plane and a smaller contribution for distant matches than for nearby ones a new strategy for updating matches is developed which only selects those matches having both high matching support and low matching ambiguity the update strategy is different from the classical winner take all which is easily stuck at a local minimum and also from loser take nothing which is usually very slow the proposed algorithm has been widely tested and works remarkably well in a scene with many repetitive patterns

the quasi perspective model geometric properties and 3d reconstruction the paper investigates geometric properties of quasi perspective projection model in one and two view geometry the main results are as follows i quasi perspective projection matrix has nine degrees of freedom dof and the parallelism along x and y directions in world system are preserved in images ii quasi fundamental matrix can be simplified to a special form with only six dofs the fundamental matrix is invariant to any non singular projective transformation iii plane induced homography under quasi perspective model can be simplified to a special form defined by six dofs the quasi homography may be recovered from two pairs of corresponding points with known fundamental matrix iv any two reconstructions in quasi perspective space are defined up to a non singular quasi perspective transformation the results are validated experimentally on both synthetic and real images

monocular template based reconstruction of inextensible surfaces we present a monocular 3d reconstruction algorithm for inextensible deformable surfaces it uses point correspondences between a single image of the deformed surface taken by a camera with known intrinsic parameters and a template the main assumption we make is that the surface shape as seen in the template is known since the surface is inextensible its deformations are isometric to the template we exploit the distance preservation constraints to recover the 3d surface shape as seen in the image though the distance preservation constraints have already been investigated in the literature we propose a new way to handle them spatial smoothness priors are easily incorporated as well as temporal smoothness priors in the case of reconstruction from a video the reconstruction can be used for 3d augmented reality purposes thanks to a fast implementation we report results on synthetic and real data some of them are compared to stereo based 3d reconstructions to demonstrate the efficiency of our method

edge union of networks on the same vertex set random network generators such as erd s r nyi watts strogatz and barab si albert models are used as models to study real world networks let g1 v e1 and g2 v e2 be two such networks on the same vertex set v this paper studies the degree distribution and clustering coefficient of the resultant networks g v e1 e2

convolutional sparse coding for trajectory reconstruction trajectory basis non rigid structure from motion nrsfm refers to the process of reconstructing the 3d trajectory of each point of a non rigid object from just their 2d projected trajectories reconstruction relies on two factors i the condition of the composed camera trajectory basis matrix and ii whether the trajectory basis has enough degrees of freedom to model the 3d point trajectory these two factors are inherently conflicting employing a trajectory basis with small capacity has the positive characteristic of reducing the likelihood of an ill conditioned system when composed with the camera during reconstruction however this has the negative characteristic of increasing the likelihood that the basis will not be able to fully model the object s true 3d point trajectories in this paper we draw upon a well known result centering around the reduced isometry property rip condition for sparse signal reconstruction rip allow us to relax the requirement that the full trajectory basis composed with the camera matrix must be well conditioned further we propose a strategy for learning an over complete basis using convolutional sparse coding from naturally occurring point trajectory corpora to increase the likelihood that the rip condition holds for a broad class of point trajectories and camera motions finally we propose an ell 1 inspired objective for trajectory reconstruction that is able to adaptively select the smallest sub matrix from an over complete trajectory basis that balances i and ii we present more practical 3d reconstruction results compared to current state of the art in trajectory basis nrsfm

self calibration of a moving camera from pointcorrespondences and fundamental matrices we address the problem of estimating three dimensional motion and structure from motion with an uncalibrated moving camera we show that point correspondences between three images and the fundamental matrices computed from these point correspondences are sufficient to recover the internal orientation of the camera its calibration the motion parameters and to compute coherent perspective projection matrices which enable us to reconstruct 3 d structure up to a similarity in contrast with other methods no calibration object with a known 3 d shape is needed and no limitations are put upon the unknown motions to be performed or the parameters to be recovered as long as they define a projective camera the theory of the method which is based on the constraint that the observed points are part of a static scene thus allowing us to link the intrinsic parameters and the fundamental matrix via the absolute conic is first detailed several algorithms are then presented and their performances compared by means of extensive simulations and illustrated by several experiments with real images

euclidean shape and motion from multiple perspective views by affine iterations we describe a method for solving the euclidean reconstruction problem with a perspective camera model by incrementally performing euclidean reconstruction with either a weak or a paraperspective camera model with respect to other methods that compute shape and motion from a sequence of images with a calibrated camera this method converges in a few iterations is computationally efficient and solves for the sign reversal ambiguity we give a detailed account of the method analyze its convergence and test it with both synthetic and real data

deformotion deforming motion shape average and the joint registration and approximation of structures in images what does it mean for a deforming object to be moving how can we separate the overall motion a finite dimensional group action from the more general deformation a diffeomorphism in this paper we propose a definition of motion for a deforming object and introduce a notion of shape average as the entity that separates the motion from the deformation our definition allows us to derive novel and efficient algorithms to register non identical shapes using region based methods and to simultaneously approximate and align structures in greyscale images we also extend the notion of shape average to that of a moving average in order to track moving and deforming objects through time the algorithms we propose extend prior work on landmark based matching to smooth curves and involve the numerical integration of partial differential equations which we address within the framework of level set methods

stochastic exploration of ambiguities for nonrigid shape recovery recovering the 3d shape of deformable surfaces from single images is known to be a highly ambiguous problem because many different shapes may have very similar projections this is commonly addressed by restricting the set of possible shapes to linear combinations of deformation modes and by imposing additional geometric constraints unfortunately because image measurements are noisy such constraints do not always guarantee that the correct shape will be recovered to overcome this limitation we introduce a stochastic sampling approach to efficiently explore the set of solutions of an objective function based on point correspondences this allows us to propose a small set of ambiguous candidate 3d shapes and then use additional image information to choose the best one as a proof of concept we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult nonlinear minimization problem the advantages of our method are demonstrated on a variety of problems including both real and synthetic data

computing smooth time trajectories for camera and deformable shape in structure from motion with occlusion we address the classical computer vision problems of rigid and nonrigid structure from motion sfm with occlusion we assume that the columns of the input observation matrix w describe smooth 2d point trajectories over time we then derive a family of efficient methods that estimate the column space of w using compact parameterizations in the discrete cosine transform dct domain our methods tolerate high percentages of missing data and incorporate new models for the smooth time trajectories of 2d points affine and weak perspective cameras and 3d deformable shape we solve a rigid sfm problem by estimating the smooth time trajectory of a single camera moving around the structure of interest by considering a weak perspective camera model from the outset we directly compute euclidean 3d shape reconstructions without requiring postprocessing steps such as euclidean upgrade and bundle adjustment our results on real sfm data sets with high percentages of missing data compared positively to those in the literature in nonrigid sfm we propose a novel 3d shape trajectory approach that solves for the deformable structure as the smooth time trajectory of a single point in a linear shape space a key result shows that compared to state of the art algorithms our nonrigid sfm method can better model complex articulated deformation with higher frequency dct components while still maintaining the low rank factorization constraint finally we also offer an approach for nonrigid sfm when w is presented with missing data

comparison of communities detection algorithms for multiplex abstract multiplex is a set of graphs on the same vertex set i e g v e 1 g v e m it is a type of generalized graph to model the multiple relationships in a system with parallel edges between vertices an important application in network science is to capture community structures in multiplex as a way to modularize the system this paper is a literature review and comparative analysis on the existing communities detection algorithms for multiplex the conclusion is that many of the algorithms deviate in the concept of multi relational communities and the wrong choice of algorithm can deviate one from his intended concept

a closed form solution to non rigid shape and motion recovery recovery of three dimensional 3d shape and motion of non static scenes from a monocular video sequence is important for applications like robot navigation and human computer interaction if every point in the scene randomly moves it is impossible to recover the non rigid shapes in practice many non rigid objects e g the human face under various expressions deform with certain structures their shapes can be regarded as a weighted combination of certain shape bases shape and motion recovery under such situations has attracted much interest previous work on this problem bregler c hertzmann a and biermann h 2000 in proc int conf computer vision and pattern recognition brand m 2001 in proc int conf computer vision and pattern recognition torresani l yang d alexander g and bregler c 2001 in proc int conf computer vision and pattern recognition utilized only orthonormality constraints on the camera rotations rotation constraints this paper proves that using only the rotation constraints results in ambiguous and invalid solutions the ambiguity arises from the fact that the shape bases are not unique an arbitrary linear transformation of the bases produces another set of eligible bases to eliminate the ambiguity we propose a set of novel constraints basis constraints which uniquely determine the shape bases we prove that under the weak perspective projection model enforcing both the basis and the rotation constraints leads to a closed form solution to the problem of non rigid shape and motion recovery the accuracy and robustness of our closed form solution is evaluated quantitatively on synthetic data and qualitatively on real video sequences

optimal metric projections for deformable and articulated structure from motion this paper describes novel algorithms for recovering the 3d shape and motion of deformable and articulated objects purely from uncalibrated 2d image measurements using a factorisation approach most approaches to deformable and articulated structure from motion require to upgrade an initial affine solution to euclidean space by imposing metric constraints on the motion matrix while in the case of rigid structure the metric upgrade step is simple since the constraints can be formulated as linear deformability in the shape introduces non linearities in this paper we propose an alternating bilinear approach to solve for non rigid 3d shape and motion associated with a globally optimal projection step of the motion matrices onto the manifold of metric constraints our novel optimal projection step combines into a single optimisation the computation of the orthographic projection matrix and the configuration weights that give the closest motion matrix that satisfies the correct block structure with the additional constraint that the projection matrix is guaranteed to have orthonormal rows i e its transpose lies on the stiefel manifold this constraint turns out to be non convex the key contribution of this work is to introduce an efficient convex relaxation for the non convex projection step efficient in the sense that for both the cases of deformable and articulated motion the proposed relaxations turned out to be exact i e tight in all our numerical experiments the convex relaxations are semi definite sdp or second order cone socp programs which can be readily tackled by popular solvers an important advantage of these new algorithms is their ability to handle missing data which becomes crucial when dealing with real video sequences with self occlusions we show successful results of our algorithms on synthetic and real sequences of both deformable and articulated data we also show comparative results with state of the art algorithms which reveal that our new methods outperform existing ones

anomalous percolation properties of growing networks we describe the anomalous phase transition of the emergence of the giant connected component in scale free networks growing under mechanism of preferential linking we obtain exact results for the size of the giant connected component and the distribution of vertices among connected components we show that all the derivatives of the giant connected component size s over the rate b of the emergence of new edges are zero at the percolation threshold b c and s ensuremath propto mathrm exp ensuremath d ensuremath gamma b ensuremath b c ensuremath 1 2 where the coefficient d is a function of the degree distribution exponent ensuremath gamma in the entire phase without the giant component these networks are in a critical state the probability mathcal p k that a vertex belongs to a connected component of a size k is of a power law form at the phase transition point mathcal p k ensuremath sim 1 k mathrm ln k 2 in the phase with the giant component mathcal p k has an exponential cutoff at k c ensuremath propto 1 s in the simplest particular case we present exact results for growing exponential networks

iterative extensions of the sturm triggs algorithm convergence and nonconvergence we give the first complete theoretical convergence analysis for the iterative extensions of the sturm triggs algorithm we show that the simplest extension siesta converges to nonsense results another proposed extension has similar problems and experiments with balanced iterations show that they can fail to converge or become unstable we present ciesta an algorithm that avoids these problems it is identical to siesta except for one simple extra computation under weak assumptions we prove that ciesta iteratively decreases an error and approaches fixed points with one more assumption we prove it converges uniquely our results imply that ciesta gives a reliable way of initializing other algorithms such as bundle adjustment a descent method such as gauss newton can be used to minimize the ciesta error combining quadratic convergence with the advantage of minimizing in the projective depths experiments show that ciesta performs better than other iterations

sequential non rigid structure from motion using physical priors we propose a new approach to simultaneously recover camera pose and 3d shape of non rigid and potentially extensible surfaces from a monocular image sequence for this purpose we make use of the extended kalman filter based simultaneous localization and mapping ekf slam formulation a bayesian optimization framework traditionally used in mobile robotics for estimating camera pose and reconstructing rigid scenarios in order to extend the problem to a deformable domain we represent the object s surface mechanics by means of navier s equations which are solved using a finite element method fem with these main ingredients we can further model the material s stretching allowing us to go a step further than most of current techniques typically constrained to surfaces undergoing isometric deformations we extensively validate our approach in both real and synthetic experiments and demonstrate its advantages with respect to competing methods more specifically we show that besides simultaneously retrieving camera pose and non rigid shape our approach is adequate for both isometric and extensible surfaces does not require neither batch processing all the frames nor tracking points over the whole sequence and runs at several frames per second

procrustean regression a flexible alignment based framework for nonrigid structure estimation non rigid structure from motion nrsfm is a fundamental problem of computer vision recently it has been shown that incorporating shape alignment in nrsfm can improve the performance significantly compared with the other algorithms which do not consider shape alignment however realizing this idea was at a cost of a heavy complicated process which limits its usefulness and possible extensions in this paper we propose a novel regression framework for nrsfm of which the variables 3d shapes are regularized based on their aligned shapes we show that this can be casted into an unconstrained problem or a problem with simple bound constraints which can be efficiently solved by existing solvers this framework can be easily integrated with numerous existing models and assumptions such as orthographic or perspective camera models occlusion low rank assumption smooth deformations and so on which makes it more practical for various real situations the experimental results show that the proposed method gives competitive result to the state of the art methods for orthographic projection with much less time complexity and memory requirement and outperforms the existing methods for perspective projection

on dimension formulas for jacobi forms in this paper we discuss the dimension formula for jacobi forms via the selberg trace formula an explicit dimension formula for the space of jacobi cusp forms of degree 2 is given as an application

self calibration and metric reconstruction inspite of varying and unknown intrinsic camera parameters in this paper the theoretical and practical feasibility of self calibration in the presence of varying intrinsic camera parameters is under investigation the paper s main contribution is to propose a self calibration method which efficiently deals with all kinds of constraints on the intrinsic camera parameters within this framework a practical method is proposed which can retrieve metric reconstruction from image sequences obtained with uncalibrated zooming focusing cameras the feasibility of the approach is illustrated on real and synthetic examples besides this a theoretical proof is given which shows that the absence of skew in the image plane is sufficient to allow for self calibration a counting argument is developed which depending on the set of constraints gives the minimum sequence length for self calibration and a method to detect critical motion sequences is proposed

a survey of the s lemma in this survey we review the many faces of the s lemma a result about the correctness of the s procedure the basic idea of this widely used method came from control theory but it has important consequences in quadratic and semidefinite optimization convex geometry and linear algebra as well these were all active research areas but as there was little interaction between researchers in these different areas their results remained mainly isolated here we give a unified analysis of the theory by providing three different proofs for the s lemma and revealing hidden connections with various areas of mathematics we prove some new duality results and present applications from control theory error estimation and computational geometry

shape and motion from image streams under orthography a factorization method inferring scene geometry and camera motion from a stream of images is possible in principle but is an ill conditioned problem when the objects are distant with respect to their size we have developed a factorization method that can overcome this difficulty by recovering shape and motion under orthography without computing depth as an intermediate step an image stream can be represented by the 2fxp measurement matrix of the image coordinates of p points tracked through f frames we show that under orthographic projection this matrix is of rank 3 based on this observation the factorization method uses the singular value decomposition technique to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively two of the three translation components are computed in a preprocessing stage the method can also handle and obtain a full solution from a partially filled in measurement matrix that may result from occlusions or tracking failures the method gives accurate results and does not introduce smoothing in either shape or motion we demonstrate this with a series of experiments on laboratory and outdoor image streams with and without occlusions

bilinear modeling via augmented lagrange multipliers balm this paper presents a unified approach to solve different bilinear factorization problems in computer vision in the presence of missing data in the measurements the problem is formulated as a constrained optimization where one of the factors must lie on a specific manifold to achieve this we introduce an equivalent reformulation of the bilinear factorization problem that decouples the core bilinear aspect from the manifold specificity we then tackle the resulting constrained optimization problem via augmented lagrange multipliers the strength and the novelty of our approach is that this framework can seamlessly handle different computer vision problems the algorithm is such that only a projector onto the manifold constraint is needed we present experiments and results for some popular factorization problems in computer vision such as rigid non rigid and articulated structure from motion photometric stereo and 2d 3d non rigid registration

confluent hypergeometric functions on tube domains our problem specialized to the case of the siegel upper half space h of degree m concerns the fourier expansion of a series s z fl det z a det a a l here z is a variable on h l is a lattice in the space v of all real symmetric matrices of size m and fl ec it can be shown that this is convergent if re f l m and also that if re m 2 and r e f l m 2 it has a fourier expansion of the form

trajectory space a dual representation for nonrigid structure from motion existing approaches to nonrigid structure from motion assume that the instantaneous 3d shape of a deforming object is a linear combination of basis shapes these bases are object dependent and therefore have to be estimated anew for each video sequence in contrast we propose a dual approach to describe the evolving 3d structure in trajectory space by a linear combination of basis trajectories we describe the dual relationship between the two approaches showing that they both have equal power for representing 3d structure we further show that the temporal smoothness in 3d trajectories alone can be used for recovering nonrigid structure from a moving camera the principal advantage of expressing deforming 3d structure in trajectory space is that we can define an object independent basis this results in a significant reduction in unknowns and corresponding stability in estimation we propose the use of the discrete cosine transform dct as the object independent basis and empirically demonstrate that it approaches principal component analysis pca for natural motions we report the performance of the proposed method quantitatively using motion capture data and qualitatively on several video sequences exhibiting nonrigid motions including piecewise rigid motion partially nonrigid motion such as a facial expressions and highly nonrigid motion such as a person walking or dancing

a theory of self calibration of a moving camera there is a close connection between the calibration of a single camera and the epipolar transformation obtained when the camera undergoes a displacement the epipolar transformation imposes two algebraic constraints on the camera calibration if two epipolar transformations arising from different camera displacements are available then the compatible camera calibrations are parameterized by an algebraic curve of genus four the curve can be represented either by a space curve of degree seven contained in the intersection of two cubic surfaces or by a curve of degree six in the dual of the image plane the curve in the dual plane has one singular point of order three and three singular points of order two if three epipolar transformations are available then two curves of degree six can be obtained in the dual plane such that one of the real intersections of the two yields the correct camera calibration the two curves have a common singular point of order three experimental results are given to demonstrate the feasibility of camera calibration based on the epipolar transformation the real intersections of the two dual curves are found by locating the zeros of a function defined on the interval 0 2 the intersection yielding the correct camera calibration is picked out by referring back to the three epipolar transformations

probabilistic principal component analysis principal component analysis pca is a ubiquitous technique for data analysis and processing but one which is not based upon a probability model in this paper we demonstrate how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model closely related to factor analysis we consider the properties of the associated likelihood function giving an em algorithm for estimating the principal subspace iteratively and discuss the advantages conveyed by the definition of a probability density function for pca

on minimizing a quadratic function on stiefel manifold abstract in this paper we propose a novel approach to a particular quadratic programming problem when the optimization is performed over the set o 3 2 of 3 2 stiefel matrices we rewrite the original nonconvex problem as a semi definite programming problem by computing a convex hull tight convex relaxation of a certain set of matrices we give an efficient quick algorithm for the minimization of a quadratic function over stiefel manifold we report some numerical experiments to illustrate the tightness of the convex approximation obtained by the two aforementioned methods standard and ours our result is of immediate interest in computer vision including structure from motion sfm problems and 2d 3d registration

multifractal network generator we introduce a new approach to constructing networks with realistic features our method in spite of its conceptual simplicity it has only two parameters is capable of generating a wide variety of network types with prescribed statistical properties e g with degree or clustering coefficient distributions of various very different forms in turn these graphs can be used to test hypotheses or as models of actual data the method is based on a mapping between suitably chosen singular measures defined on the unit square and sparse infinite networks such a mapping has the great potential of allowing for graph theoretical results for a variety of network topologies the main idea of our approach is to go to the infinite limit of the singular measure and the size of the corresponding graph simultaneously a very unique feature of this construction is that with the increasing system size the generated graphs become topologically more structured we present analytic expressions derived from the parameters of the to be iterated initial generating measure for such major characteristics of graphs as their degree clustering coefficient and assortativity coefficient distributions the optimal parameters of the generating measure are determined from a simple simulated annealing process thus the present work provides a tool for researchers from a variety of fields such as biology computer science biology or complex systems enabling them to create a versatile model of their network data

analysis of complex contagions in random multiplex networks we study the diffusion of influence in random multiplex networks where links can be of r different types and for a given content e g rumor product or political view each link type is associated with a content dependent parameter ci in 0 that measures the relative bias type i links have in spreading this content in this setting we propose a linear threshold model of contagion where nodes switch state if their perceived proportion of active neighbors exceeds a threshold namely a node connected to mi active neighbors and ki mi inactive neighbors via type i links will turn active if cimi ciki exceeds its threshold under this model we obtain the condition probability and expected size of global spreading events our results extend the existing work on complex contagions in several directions by i providing solutions for coupled random networks whose vertices are neither identical nor disjoint ii highlighting the effect of content on the dynamics of complex contagions and iii showing that content dependent propagation over a multiplex network leads to a subtle relation between the giant vulnerable component of the graph and the global cascade condition that is not seen in the existing models in the literature

convex optimization for nonrigid stereo reconstruction we present a method for recovering 3 d nonrigid structure from an image pair taken with a stereo rig more specifically we dedicate to recover shapes of nearly inextensible deformable surfaces in our approach we represent the surface as a 3 d triangulated mesh and formulate the reconstruction problem as an optimization problem consisting of data terms and shape terms the data terms are model to image keypoint correspondences which can be formulated as second order cone programming socp constraints using l norm the shape terms are designed to retaining original lengths of mesh edges which are typically nonconvex constraints we will show that this optimization problem can be turned into a sequence of socp feasibility problems in which the nonconvex constraints are approximated as a set of convex constraints thanks to the efficient socp solver the reconstruction problem can then be solved reliably and efficiently as opposed to previous methods ours neither involves smoothness constraints nor need an initial estimation which enables us to recover shapes of surfaces with smooth sharp and other complex deformations from a single image pair the robustness and accuracy of our approach are evaluated quantitatively on synthetic data and qualitatively on real data

monocular 3 d tracking of inextensible deformable surfaces under l_2 norm we present a method for recovering the 3 d shape of an inextensible deformable surface from a monocular image sequence state of the art methods on this problem utilize l norm of reprojection residual vectors and formulate the tracking problem as a second order cone programming socp problem instead of using l which is sensitive to outliers we use l 2 norm of reprojection errors generally using l 2 leads a nonconvex optimization problem which is difficult to minimize instead of solving the nonconvex problem directly we design an iterative l 2 norm approximation process to approximate the nonconvex objective function in which only a linear system needs to be solved at each iteration furthermore we introduce a shape regularization term into this iterative process in order to keep the inextensibility of the recovered mesh compared with previous methods ours performs more robust to image noises outliers and large interframe motions with high computational efficiency the robustness and accuracy of our approach are evaluated quantitatively on synthetic data and qualitatively on real data