inductive robust principal component analysis in this paper we address the error correction problem that is to uncover the low dimensional subspace structure from high dimensional observations which are possibly corrupted by errors when the errors are of gaussian distribution principal component analysis pca can find the optimal in terms of least square error low rank approximation to high dimensional data however the canonical pca method is known to be extremely fragile to the presence of gross corruptions recently wright established a so called robust principal component analysis rpca method which can well handle the grossly corrupted data however rpca is a transductive method and does not handle well the new samples which are not involved in the training procedure given a new datum rpca essentially needs to recalculate over all the data resulting in high computational cost so rpca is inappropriate for the applications that require fast online computation to overcome this limitation in this paper we propose an inductive robust principal component analysis irpca method given a set of training data unlike rpca that targets on recovering the original data matrix irpca aims at learning the underlying projection matrix which can be used to efficiently remove the possible corruptions in any datum the learning is done by solving a nuclear norm regularized minimization problem which is convex and can be solved in polynomial time extensive experiments on a benchmark human face dataset and two video surveillance datasets show that irpca cannot only be robust to gross corruptions but also handle the new data well and in an efficient way

a proximal parallel splitting method for minimizing sum of convex functions with linear constraints in this paper we propose a proximal parallel decomposition algorithm for solving the optimization problems where the objective function is the sum of m separable functions i e they have no crossed variables and the constraint set is the intersection of cartesian products of some simple sets and a linear manifold the m subproblems are solved simultaneously per iterations which are sum of the decomposed subproblems of the augmented lagrange function and a quadratic term hence our algorithm is named as the proximal parallel splitting method we prove the global convergence of the proposed algorithm under some mild conditions that the underlying functions are convex and the solution set is nonempty to make the subproblems easier some linearized versions of the proposed algorithm are also presented together with their global convergence analysis finally some preliminary numerical results are reported to support the efficiency of the new algorithms

two modified augmented lagrange multiplier algorithms for toeplitz matrix compressive recovery abstract in this paper we propose two modified augmented lagrange multiplier algorithms by mean value for toeplitz matrix compressive recovery in the algorithms the mean value modification makes the iteration matrices keep the toeplitz structure which contributes to reduce the svd time and cpu time numerical experiments show that the proposed algorithms achieve better precision than the augmented lagrange multiplier method especially when the matrix e is less sparse convergence analysis of the proposed algorithms is also given in detail

alternating direction method with gaussian back substitution for separable convex programming we consider the linearly constrained separable convex minimization problem whose objective function is separable into m individual convex functions with nonoverlapping variables a douglas rachford alternating direction method of multipliers adm has been well studied in the literature for the special case of m 2 but the convergence of extending adm to the general case of m ge 3 is still open in this paper we show that the straightforward extension of adm is valid for the general case of m ge 3 if it is combined with a gaussian back substitution procedure the resulting adm with gaussian back substitution is a novel approach towards the extension of adm from m 2 to m ge 3 and its algorithmic framework is new in the literature for the adm with gaussian back substitution we prove its convergence via the analytic framework of contractive type methods and we show its numerical efficiency by some application problems

blessing of dimensionality recovering mixture data via dictionary pursuit this paper studies the problem of recovering the authentic samples that lie on a union of multiple subspaces from their corrupted observations due to the high dimensional and massive nature of today s data driven community it is arguable that the target matrix i e authentic sample matrix to recover is often low rank in this case the recently established robust principal component analysis rpca method already provides us a convenient way to solve the problem of recovering mixture data however in general rpca is not good enough because the incoherent condition assumed by rpca is not so consistent with the mixture structure of multiple subspaces namely when the subspace number grows the row coherence of data keeps heightening and accordingly rpca degrades to overcome the challenges arising from mixture data we suggest to consider lrr in this paper we elucidate that lrr can well handle mixture data as long as its dictionary is configured appropriately more precisely we mathematically prove that lrr can weaken the dependence on the row coherence provided that the dictionary is well conditioned and has a rank of not too high in particular if the dictionary itself is sufficiently low rank then the dependence on the row coherence can be completely removed these provide some elementary principles for dictionary learning and naturally lead to a practical algorithm for recovering mixture data our experiments on randomly generated matrices and real motion sequences show promising results

maximum likelihood nonparametric estimation of smooth spectra from irregularly sampled data this paper introduces a maximum likelihood method for the nonparametric estimation of smooth spectra from irregularly sampled observations which is abbreviated as limes likelihood based method for estimation of spectra as a byproduct limes also provides an estimate of the data covariance matrix that may be of interest in its own right spectral estimation from irregularly sampled data is a rather difficult problem and there are only a handful of methods in the literature that can be used for such a task of these already existing methods we consider the daniell method dam for comparison with limes computationally limes is more complex than dam on the other hand dam is much less accurate than limes in the irregularly sampled data case and for spectra with a relatively large bandwidth in a nutshell limes should be the method of choice in the unevenly sampled data applications that require high statistical performance and can tolerate an increased computational burden

optimal data scaling for principal component pursuit a lyapunov approach to convergence in principle component pursuit pcp the essential idea is to replace the original non convex optimization problem of the matrix rank and the count of non zero entries by a convex optimization problem of the nuclear and ell 1 norms in the pcp literature it is rigorously proved that the validity of this idea depends on the coherence of the uncontaminated data specifically the lower the coherence is the equivalence of the convex optimization problem to the original non convex one will hold by a larger probability although the coherence index is fixed for a given data set it is possible to adjust this index by introducing different scalings to the data the target of this work is thus to find the optimal scaling of the data such that the coherence index is minimized based on the analysis of the pcp problem structure a non convex optimization problem with implicit dependence on the scaling parameters is firstly formulated to solve this problem a coordinate descent algorithm is proposed under mild conditions on the structure of the data matrix the convergence of the algorithm to a global optimal point is rigorously proved by treating the algorithm as a discrete time dynamic system and utilizing a lyapunov type approach monte carlo simulation experiments are performed to verify the effectiveness of the developed results

robust estimation of dispersion matrices and principal components abstract this paper uses monte carlo methods to compare the performances of several robust procedures for estimating a correlation matrix and its principal components the estimators are formed either from separate bivariate analyses or by simultaneous manipulation of all variables by using techniques such as multivariate trimming and m estimation the m estimators stand up exceptionally well they and the multivariate trimming procedure are especially effective at estimating the principal components including a near singularity however the m estimators can break down relatively easily when the dimensionality is large and the outliers are asymmetric with missing data the element wise approach becomes more attractive

dense error correction via ell 1 minimization this paper studies the problem of recovering a sparse signal x n from highly corrupted linear measurements y ax e m where e is an unknown error vector whose nonzero entries may be unbounded motivated by an observation from face recognition in computer vision this paper proves that for highly correlated and possibly overcomplete dictionaries a any sufficiently sparse signal x can be recovered by solving an l1 minimization problem min x 1 e 1 subject to y ax e more precisely if the fraction of the support of the error e is bounded away from one and the support of a is a very small fraction of the dimension m then as m becomes large the above l1 minimization succeeds for all signals x and almost all sign and support patterns of e this result suggests that accurate recovery of sparse signals is possible and computationally feasible even with nearly 100 of the observations corrupted the proof relies on a careful characterization of the faces of a convex polytope spanned together by the standard crosspolytope and a set of independent identically distributed i i d gaussian vectors with nonzero mean and small variance dubbed the cross and bouquet cab model simulations and experiments corroborate the findings and suggest extensions to the result

wideband spectrum sensing from compressed measurements using spectral prior information wideband spectrum sensing wss encompasses a collection of techniques intended to estimate or to decide over the occupancy parameters of a wide frequency band however broad bands require expensive acquisition systems thus motivating the use of compressive schemes in this context previous works in compressive wss have already realized that great compression rates can be achieved if only second order statistics are of interest in spectrum sensing in this paper we go a step further by exploiting spectral prior information that is typically available in practice in order to reduce the sampling rate even more the signal model assumes that the acquisition is done by means of an analog to information converter a2i the input signal is the linear combination of a number of signals whose second order statistics are known and the goal is to estimate decide over the coefficients of this combination the problem is thus a particular instance of the well known structured covariance estimation problem unfortunately the algorithms used in this area are extremely complex for inexpensive spectrum sensors so that alternative techniques need to be devised exploiting the fact that the basis matrices are toeplitz we use the asymptotic theory of circulant matrices to propose a dimensionality reduction technique that simplifies existing structured covariance estimation algorithms achieving a similar performance at a much lower computational cost

recovering low rank matrices from corrupted observations via the linear conjugate gradient algorithm the matrix nuclear norm minimization problem has received much attention in recent years largely because its highly related to the matrix rank minimization problem arising from controller design signal processing and model reduction the alternating direction method is a very popular way to solve this problem due to its simplicity low storage practical computation efficiency and nice convergence properties in this paper we propose an alternating direction method where one variable is determined explicitly and the other variable is computed by a linear conjugate gradient algorithm at each iteration the method involves a singular value thresholding and its convergence result is guaranteed in this literature extensive experiments illustrate that the proposed algorithm compares favorable with the state of the art algorithms fpca and iadm bb which were specifically designed in recent years

new tools for residual analysis techniques are presented for stepwise selection of observations in order of discrepancy from a linear model and for assigning a p value to each selected observation a general class of criteria for discrepancy is defined and leading members of the class are discussed two ways to define the sequence of p values for a general selection criterion are discussed and their relative advantages are compared the techniques are illustrated on several data sets using only the computationally simpler of the two approaches to p values

outlier robust pca the high dimensional case principal component analysis plays a central role in statistics engineering and science because of the prevalence of corrupted data in real world applications much research has focused on developing robust algorithms perhaps surprisingly these algorithms are unequipped indeed unable to deal with outliers in the high dimensional setting where the number of observations is of the same magnitude as the number of variables of each observation and the dataset contains some arbitrarily corrupted observations we propose a high dimensional robust principal component analysis algorithm that is efficient robust to contaminated points and easily kernelizable in particular our algorithm achieves maximal robustness it has a breakdown point of 50 the best possible while all existing algorithms have a breakdown point of zero moreover our algorithm recovers the optimal solution exactly in the case where the number of corrupted points grows sublinearly in the dimension

bayesian robust tensor factorization for incomplete multiway data we propose a generative model for robust tensor factorization in the presence of both missing data and outliers the objective is to explicitly infer the underlying low candecomp parafac cp rank tensor capturing the global information and a sparse tensor capturing the local information also considered as outliers thus providing the robust predictive distribution over missing entries the low cp rank tensor is modeled by multilinear interactions between multiple latent factors on which the column sparsity is enforced by a hierarchical prior while the sparse tensor is modeled by a hierarchical view of student t distribution that associates an individual hyperparameter with each element independently for model learning we develop an efficient variational inference under a fully bayesian treatment which can effectively prevent the overfitting problem and scales linearly with data size in contrast to existing related works our method can perform model selection automatically and implicitly without the need of tuning parameters more specifically it can discover the groundtruth of cp rank and automatically adapt the sparsity inducing priors to various types of outliers in addition the tradeoff between the low rank approximation and the sparse representation can be optimized in the sense of maximum model evidence the extensive experiments and comparisons with many state of the art algorithms on both synthetic and real world data sets demonstrate the superiorities of our method from several perspectives

accelerated dynamic mri exploiting sparsity and low rank structure k t slr we introduce a novel algorithm to reconstruct dynamic magnetic resonance imaging mri data from under sampled k t space data in contrast to classical model based cine mri schemes that rely on the sparsity or banded structure in fourier space we use the compact representation of the data in the karhunen louve transform klt domain to exploit the correlations in the dataset the use of the data dependent kl transform makes our approach ideally suited to a range of dynamic imaging problems even when the motion is not periodic in comparison to current klt based methods that rely on a two step approach to first estimate the basis functions and then use it for reconstruction we pose the problem as a spectrally regularized matrix recovery problem by simultaneously determining the temporal basis functions and its spatial weights from the entire measured data the proposed scheme is capable of providing high quality reconstructions at a range of accelerations in addition to using the compact representation in the klt domain we also exploit the sparsity of the data to further improve the recovery rate validations using numerical phantoms and in vivo cardiac perfusion mri data demonstrate the significant improvement in performance offered by the proposed scheme over existing methods

a simple prior free method for non rigid structure from motion factorization this paper proposes a simple prior free method for solving the non rigid structure from motion nrsfm factorization problem other than using the fundamental low order linear combination model assumption our method does not assume any extra prior knowledge either about the non rigid structure or about the camera motions yet it works effectively and reliably producing optimal results and not suffering from the inherent basis ambiguity issue which plagued most conventional nrsfm factorization methods our method is very simple to implement which involves solving a very small sdp semi definite programming of fixed size and a nuclear norm minimization problem we also present theoretical analysis on the uniqueness and the relaxation gap of our solutions extensive experiments on both synthetic and real motion capture data assuming following the low order linear combination model are conducted which demonstrate that our method indeed outperforms most of the existing non rigid factorization methods this work offers not only new theoretical insight but also a practical everyday solution to nrsfm

robust subspace discovery via relaxed rank minimization this letter examines the problem of robust subspace discovery from input data samples instances in the presence of overwhelming outliers and corruptions a typical example is the case where we are given a set of images each image contains for example a face at an unknown location of an unknown size our goal is to identify or detect the face in the image and simultaneously learn its model we employ a simple generative subspace model and propose a new formulation to simultaneously infer the label information and learn the model using low rank optimization solving this problem enables us to simultaneously identify the ownership of instances to the subspace and learn the corresponding subspace model we give an efficient and effective algorithm based on the alternating direction method of multipliers and provide extensive simulations and experiments to verify the effectiveness of our method the proposed scheme can also be used to tackle many related high dimensional combinatorial selection problems

computing a nearest symmetric positive semidefinite matrix the nearest symmetric positive semidefinite matrix in the frobenius norm to an arbitrary real matrix a is shown to be b h 2 where h is the symmetric polar factor of b a at 2 in the 2 norm a nearest symmetric positive semidefinite matrix and its distance 2 a from a are given by a computationally challenging formula due to halmos we show how the bisection method can be applied to this formula to compute upper and lower bounds for 2 a differing by no more than a given amount a key ingredient is a stable and efficient test for positive definiteness based on an attempted choleski decomposition for accurate computation of 2 a we formulate the problem as one of zero finding and apply a hybrid newton bisection algorithm some numerical difficulties are discussed and illustrated by example

general subspace learning with corrupted training data via graph embedding we address the following subspace learning problem supposing we are given a set of labeled corrupted training data points how to learn the underlying subspace which contains three components an intrinsic subspace that captures certain desired properties of a data set a penalty subspace that fits the undesired properties of the data and an error container that models the gross corruptions possibly existing in the data given a set of data points these three components can be learned by solving a nuclear norm regularized optimization problem which is convex and can be efficiently solved in polynomial time using the method as a tool we propose a new discriminant analysis i e supervised subspace learning algorithm called corruptions tolerant discriminant analysis ctda in which the intrinsic subspace is used to capture the features with high within class similarity the penalty subspace takes the role of modeling the undesired features with high between class similarity and the error container takes charge of fitting the possible corruptions in the data we show that ctda can well handle the gross corruptions possibly existing in the training data whereas previous linear discriminant analysis algorithms arguably fail in such a setting extensive experiments conducted on two benchmark human face data sets and one object recognition data set show that ctda outperforms the related algorithms

convergence rate analysis for the alternating direction method of multipliers with a substitution procedure for separable convex programming recently in he et al he bs tao m yuan xm 2012 alternating direction method with gaussian back substitution for separable convex programming siam j optim 22 2 313 340 we have showed the first possibility of combining the douglas rachford alternating direction method of multipliers admm with a gaussian back substitution procedure for solving a convex minimization model with a general separable structure this paper is a further study on this theme we first derive a general algorithmic framework to combine admm with either a forward or backward substitution procedure then we show that convergence of this framework can be easily proved from the contraction perspective and its local linear convergence rate is provable if certain error bound condition is assumed without such an error bound assumption we can estimate its worst case convergence rate measured by the iteration complexity

a customized douglas rachford splitting algorithm for separable convex minimization with linear constraints we consider applying the douglas rachford splitting method drsm to the convex minimization problem with linear constraints and a separable objective function the dual application of drsm has been well studied in the literature resulting in the well known alternating direction method of multipliers admm in this paper we show that the primal application of drsm in combination with an appropriate decomposition can yield an efficient structure exploiting algorithm for the model under consideration whose subproblems could be easier than those of admm both the exact and inexact versions of this customized drsm are studied and their numerical efficiency is demonstrated by some preliminary numerical results

projection pursuit approach to robust dispersion matrices and principal components primary theory and monte carlo abstract this article proposes and discusses a type of new robust estimators for covariance correlation matrices and principal components via projection pursuit techniques the most attractive advantage of the new procedures is that they are of both rotational equivariance and high breakdown point besides they are qualitatively robust and consistent at elliptic underlying distributions the monte carlo study shows that the best of the new estimators compare favorably with other robust methods they provide as good a performance as m estimators and somewhat better empirical breakdown properties

robust pca via outlier pursuit singular value decomposition svd and principal component analysis pca is one of the most widely used techniques for dimensionality reduction successful and efficiently computable it is nevertheless plagued by a well known well documented sensitivity to outliers recent work has considered the setting where each point has a few arbitrarily corrupted components yet in applications of svd or pca such as robust collaborative filtering or bioinformatics malicious agents defective genes or simply corrupted or contaminated experiments may effectively yield entire points that are completely corrupted we present an efficient convex optimization based algorithm that we call outlier pursuit which under some mild assumptions on the uncorrupted points satisfied e g by the standard generative assumption in pca problems recovers the exact optimal low dimensional subspace and identifies the corrupted points such identification of corrupted points that do not conform to the low dimensional approximation is of paramount interest in bioinformatics financial applications and beyond our techniques involve matrix decomposition using nuclear norm minimization however our results setup and approach necessarily differ considerably from the existing line of work in matrix completion and matrix decomposition since we develop an approach to recover the correct column space of the uncorrupted matrix rather than the exact matrix itself in any problem where one seeks to recover a structure rather than the exact initial matrices techniques developed thus far relying on certificates of optimality will fail we present an important extension of these methods which allows the treatment of such problems

monte carlo sure a black box optimization of regularization parameters for general denoising algorithms we consider the problem of optimizing the parameters of a given denoising algorithm for restoration of a signal corrupted by white gaussian noise to achieve this we propose to minimize stein s unbiased risk estimate sure which provides a means of assessing the true mean squared error mse purely from the measured data without need for any knowledge about the noise free signal specifically we present a novel monte carlo technique which enables the user to calculate sure for an arbitrary denoising algorithm characterized by some specific parameter setting our method is a black box approach which solely uses the response of the denoising operator to additional input noise and does not ask for any information about its functional form this therefore permits the use of sure for optimization of a wide variety of denoising algorithms we justify our claims by presenting experimental results for sure based optimization of a series of popular image denoising algorithms such as total variation denoising wavelet soft thresholding and wiener filtering smoothing splines in the process we also compare the performance of these methods we demonstrate numerically that sure computed using the new approach accurately predicts the true mse for all the considered algorithms we also show that sure uncovers the optimal values of the parameters in all cases

the power of convex relaxation near optimal matrix completion this paper is concerned with the problem of recovering an unknown matrix from a small fraction of its entries this is known as the matrix completion problem and comes up in a great number of applications including the famous netflix prize and other similar questions in collaborative filtering in general accurate recovery of a matrix from a small number of entries is impossible but the knowledge that the unknown matrix has low rank radically changes this premise making the search for solutions meaningful this paper presents optimality results quantifying the minimum number of entries needed to recover a matrix of rank r exactly by any method whatsoever the information theoretic limit more importantly the paper shows that under certain incoherence assumptions on the singular vectors of the matrix recovery is possible by solving a convenient convex program as soon as the number of entries is on the order of the information theoretic limit up to logarithmic factors this convex program simply finds among all matrices consistent with the observed entries that with minimum nuclear norm as an example we show that on the order of nr log n samples are needed to recover a random n x n matrix of rank r by any method and to be sure nuclear norm minimization succeeds as soon as the number of entries is of the form nr polylog n

spectral analysis of nonuniformly sampled data a new approach versus the periodogram we begin by revisiting the periodogram to explain why arguably the plain least squares periodogram lsp is preferable to the ldquoclassicalrdquo fourier periodogram from a data fitting viewpoint as well as to the frequently used form of lsp due to lomb and scargle from a computational standpoint then we go on to introduce a new enhanced method for spectral analysis of nonuniformly sampled data sequences the new method can be interpreted as an iteratively weighted lsp that makes use of a data dependent weighting matrix built from the most recent spectral estimate because this method is derived for the case of real valued data which is typically more complicated to deal with in spectral analysis than the complex valued data case it is iterative and it makes use of an adaptive i e data dependent weighting we refer to it as the real valued iterative adaptive approach riaa lsp and riaa are nonparametric methods that can be used for the spectral analysis of general data sequences with both continuous and discrete spectra however they are most suitable for data sequences with discrete spectra i e sinusoidal data which is the case we emphasize in this paper

s estimators for functional principal component analysis principal component analysis is a widely used technique that provides an optimal lower dimensional approximation to multivariate or functional datasets these approximations can be very useful in identifying potential outliers among high dimensional or functional observations in this article we propose a new class of estimators for principal components based on robust scale estimators for a fixed dimension q we robustly estimate the q dimensional linear space that provides the best prediction for the data in the sense of minimizing the sum of robust scale estimators of the coordinates of the residuals we also study an extension to the infinite dimensional case our method is consistent for elliptical random vectors and is fisher consistent for elliptically distributed random elements on arbitrary hilbert spaces numerical experiments show that our proposal is highly competitive when compared with other methods we illustrate our approach on a real dataset where the robust estimator discovers atypi

computationally efficient truncated nuclear norm minimization for high dynamic range imaging matrix completion is a rank minimization problem to recover a low rank data matrix from a small subset of its entries since the matrix rank is nonconvex and discrete many existing approaches approximate the matrix rank as the nuclear norm however the truncated nuclear norm is known to be a better approximation to the matrix rank than the nuclear norm exploiting a priori target rank information about the problem in rank minimization in this paper we propose a computationally efficient truncated nuclear norm minimization algorithm for matrix completion which we call tnnm alm we reformulate the original optimization problem by introducing slack variables and considering noise in the observation the central contribution of this paper is to solve it efficiently via the augmented lagrange multiplier alm method where the optimization variables are updated by closed form solutions we apply the proposed tnnm alm algorithm to ghost free high dynamic range imaging by exploiting the low rank structure of irradiance maps from low dynamic range images experimental results on both synthetic and real visual data show that the proposed algorithm achieves significantly lower reconstruction errors and superior robustness against noise than the conventional approaches while providing substantial improvement in speed thereby applicable to a wide range of imaging applications

alternating projection proximal methods for convex programming and variational inequalities we consider a mixed problem composed in part of finding a zero of a maximal monotone operator and in part of solving a monotone variational inequality problem we propose a solution method for this problem that alternates between a proximal step for the maximal monotone operator part and a projection type step for the monotone variational inequality part and analyze its convergence and rate of convergence this method extends a decomposition method of chen and teboulle math programming 64 1994 pp 81 101 for convex programming and yields as a by product new decomposition methods

robust bilinear factorization with missing and grossly corrupted observations recovering low rank and sparse matrices from incomplete or corrupted observations is an important problem in statistics machine learning computer vision as well as signal and image processing in theory this problem can be solved by the natural convex joint mixed relaxations i e l 1 norm and trace norm under certain conditions however all current provable algorithms suffer from superlinear per iteration cost which severely limits their applicability to large scale problems in this paper we propose a scalable provable and structured robust bilinear factorization rbf method to recover low rank and sparse matrices from missing and grossly corrupted data i e robust matrix completion rmc or incomplete and grossly corrupted measurements i e compressive principal component pursuit cpcp specifically we first present two small scale matrix trace norm regularized bilinear factorization models for rmc and cpcp problems in which repetitively calculating svd of a large scale matrix is replaced by updating two much smaller factor matrices then we apply the alternating direction method of multipliers admm to efficiently solve the rmc problems finally we provide the convergence analysis of our algorithm and extend it to address general cpcp problems experimental results verified both the efficiency and effectiveness of our method compared with the state of the art methods

revisiting l 2 1 norm robustness with vector outlier regularization in many real world applications data usually contain outliers one popular approach is to use the l 2 1 norm function as a robust loss error function however the robustness of the l 2 1 norm function is not well understood so far in this brief we propose a new vector outlier regularization vor framework to understand and analyze the robustness of the l 2 1 norm function our vor function defines a data point to be the outlier if it is outside a threshold with respect to a theoretical prediction and regularizes it i e pull it back to the threshold line thus in the vor function how far an outlier lies away from its theoretical predicted value does not affect the final regularization and analysis results one important aspect of the vor function is that it has an equivalent continuous formulation based on which we can prove that the l 2 1 norm function is the limiting case of the proposed vor function based on this theoretical result we thus provide a new and intuitive explanation for the robustness property of the l 2 1 norm function as an example we use the vor function to matrix factorization and propose a vor principal component analysis pca vorpca we show some benefits of vorpca on data reconstruction and clustering tasks

low rank phase retrieval we develop two iterative algorithms for solving the low rank phase retrieval lrpr problem lrpr refers to recovering a low rank matrix boldsymbol x from magnitude only phaseless measurements of random linear projections of its columns both methods consist of a spectral initialization step followed by an iterative algorithm to maximize the observed data likelihood we obtain sample complexity bounds for our proposed initialization approach to provide a good approximation of the true boldsymbol x when the rank is low enough these bounds are significantly lower than what existing single vector phase retrieval algorithms need via extensive experiments we show that the same is also true for the proposed complete algorithms

intrinsic grassmann averages for online linear robust and nonlinear subspace learning principal component analysis pca and kernel principal component analysis kpca are fundamental methods in machine learning for dimensionality reduction the former is a technique for finding this approximation in finite dimensions and the latter is often in an infinite dimensional reproducing kernel hilbert space rkhs in this paper we present a geometric framework for computing the principal linear subspaces in both situations as well as for the robust pca case that amounts to computing the intrinsic average on the space of all subspaces the grassmann manifold points on this manifold are defined as the subspaces spanned by k tuples of observations the intrinsic grassmann average of these subspaces are shown to coincide with the principal components of the observations when they are drawn from a gaussian distribution we show similar results in the rkhs case and provide an efficient algorithm for computing the projection onto the this average subspace the result is a method akin to kpca which is substantially faster further we present a novel online version of the kpca using our geometric framework competitive performance of all our algorithms are demonstrated on a variety of real and synthetic data sets

a fast majorize minimize algorithm for the recovery of sparse and low rank matrices we introduce a novel algorithm to recover sparse and low rank matrices from noisy and undersampled measurements we pose the reconstruction as an optimization problem where we minimize a linear combination of data consistency error nonconvex spectral penalty and nonconvex sparsity penalty we majorize the nondifferentiable spectral and sparsity penalties in the criterion by quadratic expressions to realize an iterative three step alternating minimization scheme since each of these steps can be evaluated either analytically or using fast schemes we obtain a computationally efficient algorithm we demonstrate the utility of the algorithm in the context of dynamic magnetic resonance imaging mri reconstruction from sub nyquist sampled measurements the results show a significant improvement in signal to noise ratio and image quality compared with classical dynamic imaging algorithms we expect the proposed scheme to be useful in a range of applications including video restoration and multidimensional mri

a proximal point algorithm for log determinant optimization with group lasso regularization we consider the covariance selection problem where variables are clustered into groups and the inverse covariance matrix is expected to have a blockwise sparse structure this problem is realized via penalizing the maximum likelihood estimation of the inverse covariance ma trix by group lasso regularization we propose to solve the resulting log determinant optimization problem with the classical proximal point algorithm ppa at each iteration as it is difficult to update the primal variables directly we first solve the dual subproblem by an inexact semismooth newton cg method and then update the primal variables by explicit formulas based on the com puted dual variables we also propose to accelerate the ppa by an inexact generalized newton s method when the iterate is close to the solution theoretically we prove that at the optimal solution the nonsingularity of the generalized hessian matrices of the dual subproblem is equivalent to the constraint nondegeneracy condition for the primal problem global and local convergence results are also presented for the proposed ppa moreover based on the augmented lagrangian function of the dual problem we derive an alternating direction method adm which is easily implementable and is demonstrated to be efficient for random problems numerical results including comparisons with the adm on both synthetic and real data are presented to demonstrate that the proposed newton cg based ppa is stable and efficient and in particular outperforms the adm when high accuracy is required

a modified alternating direction method for convex quadratically constrained quadratic semidefinite programs we propose a modified alternating direction method for solving convex quadratically constrained quadratic semidefinite optimization problems the method is a first order method therefore requires much less computational effort per iteration than the second order approaches such as the interior point methods or the smoothing newton methods in fact only a single inexact metric projection onto the positive semidefinite cone is required at each iteration we prove global convergence and provide numerical evidence to show the effectiveness of this method

completing low rank matrices with corrupted samples from few coefficients in general basis subspace recovery from the corrupted and missing data is crucial for various applications in signal processing and information theory to complete missing values and detect column corruptions the existing robust matrix completion mc methods mostly concentrate on recovering a low rank matrix from a few corrupted coefficients with respect to standard basis which however does not apply to more general basis e g fourier basis in this paper we prove that the range space of an m times n matrix with rank r can be exactly recovered from a few coefficients with respect to general basis though r and the number of corrupted samples are both as high as o min m n log 3 m n our model covers the previous ones as special cases and robust mc can recover the intrinsic matrix with a higher rank moreover we suggest a universal choice of the regularization parameter which is lambda 1 sqrt log n by our ell 2 1 filtering algorithm which has theoretical guarantees we can further reduce the computational cost of our model as an application we also find that the solutions to extended robust low rank representation and to our extended robust mc are mutually expressible so both our theory and algorithm can be applied to the subspace clustering problem with missing values under certain conditions the experiments verify our theories

optshrink an algorithm for improved low rank signal matrix denoising by optimal data driven singular value shrinkage the truncated singular value decomposition of the measurement matrix is the optimal solution to the representation problem of how to best approximate a noisy measurement matrix using a low rank matrix here we consider the unobservable denoising problem of how to best approximate a low rank signal matrix buried in noise by optimal re weighting of the singular vectors of the measurement matrix we exploit recent results from random matrix theory to exactly characterize the large matrix limit of the optimal weighting coefficients and show that they can be computed directly from data for a large class of noise models that includes the independent identically distributed gaussian noise case our analysis brings into sharp focus the shrinkage and thresholding form of the optimal weights the nonconvex nature of the associated shrinkage function on the singular values and explains why matrix regularization via singular value thresholding with convex penalty functions such as the nuclear norm will always be suboptimal we validate our theoretical predictions with numerical simulations develop an implementable algorithm optshrink that realizes the predicted performance gains and show how our methods can be used to improve estimation in the setting where the measured matrix has missing entries

coupled variational image decomposition and restoration model for blurred cartoon plus texture images with missing pixels in this paper we develop a decomposition model to restore blurred images with missing pixels our assumption is that the underlying image is the superposition of cartoon and texture components we use the total variation norm and its dual norm to regularize the cartoon and texture respectively we recommend an efficient numerical algorithm based on the splitting versions of augmented lagrangian method to solve the problem theoretically the existence of a minimizer to the energy function and the convergence of the algorithm are guaranteed in contrast to recently developed methods for deblurring images the proposed algorithm not only gives the restored image but also gives a decomposition of cartoon and texture parts these two parts can be further used in segmentation and inpainting problems numerical comparisons between this algorithm and some state of the art methods are also reported

relations among some low rank subspace recovery models recovering intrinsic low dimensional subspaces from data distributed on them is a key preprocessing step to many applications in recent years a lot of work has modeled subspace recovery as low rank minimization problems we find that some representative models such as robust principal component analysis r pca robust low rank representation r lrr and robust latent low rank representation r latlrr are actually deeply connected more specifically we discover that once a solution to one of the models is obtained we can obtain the solutions to other models in closed form formulations since r pca is the simplest our discovery makes it the center of low rank subspace recovery models our work has two important implications first r pca has a solid theoretical foundation under certain conditions we could find globally optimal solutions to these low rank models at an overwhelming probability although these models are nonconvex second we can obtain significantly faster algorithms for these models by solving r pca first the computation cost can be further cut by applying low complexity randomized algorithms for example our novel filtering algorithm to r pca although for the moment the formal proof of our filtering algorithm is not yet available experiments verify the advantages of our algorithm over other state of the art methods based on the alternating direction method

analysis and design for delta operator systems with actuator saturation this paper studies the problem of estimating the domain of attraction for delta operator systems under a pre designed saturated linear feedback ellipsoids are used as the estimates of domain of attraction for delta operator systems a condition is derived in terms of an auxiliary feedback matrix for determining if a given ellipsoid is contractive invariant moreover some necessary and sufficient conditions are given for single input delta operator systems the results are also further extended to determine the invariant sets for delta operator systems with persistent disturbances linear matrix inequalities based on the methods are developed for constructing feedback laws that achieve disturbance rejection with guaranteed stability requirements some numerical examples are given to illustrate the effectiveness of the developed techniques

a new inexact alternating directions method for monotone variational inequalities the alternating directions method adm is an effective method for solving a class of variational inequalities vi when the proximal and penalty parameters in sub vi problems are properly selected in this paper we propose a new adm method which needs to solve two strongly monotone sub vi problems in each iteration approximately and allows the parameters to vary from iteration to iteration the convergence of the proposed adm method is proved under quite mild assumptions and flexible parameter conditions

simultaneous video stabilization and moving object detection in turbulence turbulence mitigation refers to the stabilization of videos with nonuniform deformations due to the influence of optical turbulence typical approaches for turbulence mitigation follow averaging or dewarping techniques although these methods can reduce the turbulence they distort the independently moving objects which can often be of great interest in this paper we address the novel problem of simultaneous turbulence mitigation and moving object detection we propose a novel three term low rank matrix decomposition approach in which we decompose the turbulence sequence into three components the background the turbulence and the object we simplify this extremely difficult problem into a minimization of nuclear norm frobenius norm and 21 norm our method is based on two observations first the turbulence causes dense and gaussian noise and therefore can be captured by frobenius norm while the moving objects are sparse and thus can be captured by 21 norm second since the object s motion is linear and intrinsically different from the gaussian like turbulence a gaussian based turbulence model can be employed to enforce an additional constraint on the search space of the minimization we demonstrate the robustness of our approach on challenging sequences which are significantly distorted with atmospheric turbulence and include extremely tiny moving objects

principal component analysis based on robust estimators of the covariance or correlation matrix influence functions and efficiencies a robust principal component analysis can be easily performed by computing the eigenvalues and eigenvectors of a robust estimator of the covariance or correlation matrix in this paper we derive the influence functions and the corresponding asymptotic variances for these robust estimators of eigenvalues and eigenvectors the behaviour of several of these estimators is investigated by a simulation study it turns out that the theoretical results and simulations favour the use of s estimators since they combine a high efficiency with appealing robustness properties 2000 biometrika trust

convergent prediction correction based admm for multi block separable convex programming abstract the direct extension of the classic alternating direction method with multipliers admme to the multi block separable convex optimization problem is not necessarily convergent though it often performs very well in practice in order to preserve the numerical advantages of admme and obtain convergence many modified admm were proposed by correcting the output of admme or employing proximal terms to solve inexactly the subproblems in admme in this paper we present an efficient prediction correction based admm pcb admm to solve the multi block separable convex minimization model the prediction step takes a special block coordinate descent bcd cycle to update the variable blocks then the correction step corrects the output slightly by computing a convex combination of two points from the prediction step and previous iteration the convergence property is obtained by using the variational inequality the numerical experiments illustrate effectiveness of the proposed pcb admm to solve the quadratic semidefinite programming and image decomposition

robust matrix decomposition with sparse corruptions suppose a given observation matrix can be decomposed as the sum of a low rank matrix and a sparse matrix and the goal is to recover these individual components from the observed sum such additive decompositions have applications in a variety of numerical problems including system identification latent variable graphical modeling and principal components analysis we study conditions under which recovering such a decomposition is possible via a combination of l1 norm and trace norm minimization we are specifically interested in the question of how many sparse corruptions are allowed so that convex programming can still achieve accurate recovery and we obtain stronger recovery guarantees than previous studies moreover we do not assume that the spatial pattern of corruptions is random which stands in contrast to related analyses under such assumptions via matrix completion

robust principal component analysis by self organizing rules based on statistical physics approach this paper applies statistical physics to the problem of robust principal component analysis pca the commonly used pca learning rules are first related to energy functions these functions are generalized by adding a binary decision field with a given prior distribution so that outliers in the data are dealt with explicitly in order to make pca robust each of the generalized energy functions is then used to define a gibbs distribution from which a marginal distribution is obtained by summing over the binary decision field the marginal distribution defines an effective energy function from which self organizing rules have been developed for robust pca under the presence of outliers both the standard pca methods and the existing self organizing pca rules studied in the literature of neural networks perform quite poorly by contrast the robust rules proposed here resist outliers well and perform excellently for fulfilling various pca like tasks such as obtaining the first principal component vector the first k principal component vectors and directly finding the subspace spanned by the first k vector principal component vectors without solving for each vector individually comparative experiments have been made and the results show that the authors robust rules improve the performances of the existing pca algorithms significantly when outliers are present

randomized robust subspace recovery and outlier detection for high dimensional data matrices this paper explores and analyzes two randomized designs for robust principal component analysis employing low dimensional data sketching in one design a data sketch is constructed using random column sampling followed by low dimensional embedding while in the other sketching is based on random column and row sampling both designs are shown to bring about substantial savings in complexity and memory requirements for robust subspace learning over conventional approaches that use the full scale data a characterization of the sample and computational complexity of both designs is derived in the context of two distinct outlier models namely sparse and independent outlier models the proposed randomized approach can provably recover the correct subspace with computational and sample complexity which depend only weakly on the size of the data only through the coherence parameters the results of the mathematical analysis are confirmed through numerical simulations using both synthetic and real data

solving semidefinite quadratic linear programs using sdpt3 this paper discusses computational experiments with linear optimization problems involving semidefinite quadratic and linear cone constraints sqlps many test problems of this type are solved using a new release of sdpt3 a matlab implementation of infeasible primal dual path following algorithms the software developed by the authors uses mehrotra type predictor corrector variants of interior point methods and two types of search directions the hkm and nt directions a discussion of implementation details is provided and computational results on problems from the sdplib and dimacs challenge collections are reported

an algorithm twisted from generalized admm for multi block separable convex minimization models the alternating direction method with multipliers admm has been one of most powerful and successful methods for solving a two block linearly constrained convex minimization model whose objective function is the sum of two functions without coupled variables it is known that the numerical efficiency is inherited for a large number of applications but the convergence is not guaranteed if the admm is directly extended to a multiple block convex minimization model whose objective function has more than two functions this viewpoint was in fact the motivation for developing efficient algorithms that cannot only preserve the numerical advantages of the direct extension of admm but also guarantee convergence one way is to correct the output of the direct extension of admm slightly via a simple correction step and the other is to employ a simple proximal to solve inexactly each subproblem in the direct extension of admm in this paper in order to solve the multi block separable convex minimization model efficiently we present a method which is a combination of the above two ways that is we first solve each subproblem with a simple proximal then we correct the output via a simple correction step theoretically we derive global convergence results for this method and establish a worst case o 1 k iteration complexity numerically the efficiency of this method can be showed by testing the problem of recovering low rank and sparse components of matrices from incomplete and noisy observation

numerical sensitivity of linear matrix inequalities using shift and delta operators the numerical sensitivity of linear matrix inequalities lmis arising from discrete time control with short sampling periods is analyzed using shift and delta operators the delta operator avoids cancellation problems for short sampling periods and it includes a system scaling proportional to the inverse of the sampling period the numerical sensitivity of both these mechanisms is investigated analytically and verified by numerical examples the conclusion is that the scaling procedure is somewhat surprisingly much more essential for shorter sampling periods than avoiding the cancellation problem

robust pca with partial subspace knowledge in recent work robust principal components analysis pca has been posed as a problem of recovering a low rank matrix bf l and a sparse matrix bf s from their sum bf m bf l bf s and a provably exact convex optimization solution called pcp has been proposed this work studies the following problem suppose that we have partial knowledge about the column space of the low rank matrix bf l can we use this information to improve the pcp solution i e allow recovery under weaker assumptions we propose here a simple but useful modification of the pcp idea called modified pcp that allows us to use this knowledge we derive its correctness result which shows that when the available subspace knowledge is accurate modified pcp indeed requires significantly weaker incoherence assumptions than pcp extensive simulations are also used to illustrate this comparisons with pcp and other existing work are shown for a stylized real application as well finally we explain how this problem naturally occurs in many applications involving time series data i e in what is called the online or recursive robust pca problem a corollary for this case is also given

recursive robust pca or recursive sparse recovery in large but structured noise this paper studies the recursive robust principal components analysis problem if the outlier is the signal of interest this problem can be interpreted as one of recursively recovering a time sequence of sparse vectors st in the presence of large but structured noise lt the structure that we assume on lt is that lt is dense and lies in a low dimensional subspace that is either fixed or changes slowly enough a key application where this problem occurs is in video surveillance where the goal is to separate a slowly changing background lt from moving foreground objects st on the fly to solve the above problem in recent work we introduced a novel solution called recursive projected cs reprocs in this paper we develop a simple modification of the original reprocs idea and analyze it this modification assumes knowledge of a subspace change model on the lt s under mild assumptions and a denseness assumption on the unestimated part of the subspace of lt at various times we show that with high probability the proposed approach can exactly recover the support set of st at all times and the reconstruction errors of both st and lt are upper bounded by a time invariant and small value in simulation experiments we observe that the last assumption holds as long as there is some support change of st every few frames

motion adaptive patch based low rank approach for compressed sensing cardiac cine mri one of the technical challenges in cine magnetic resonance imaging mri is to reduce the acquisition time to enable the high spatio temporal resolution imaging of a cardiac volume within a short scan time recently compressed sensing approaches have been investigated extensively for highly accelerated cine mri by exploiting transform domain sparsity using linear transforms such as wavelets and fourier however in cardiac cine imaging the cardiac volume changes significantly between frames and there often exist abrupt pixel value changes along time in order to effectively sparsify such temporal variations it is necessary to exploit temporal redundancy along motion trajectories this paper introduces a novel patch based reconstruction method to exploit geometric similarities in the spatio temporal domain in particular we use a low rank constraint for similar patches along motion based on the observation that rank structures are relatively less sensitive to global intensity changes but make it easier to capture moving edges a nash equilibrium formulation with relaxation is employed to guarantee convergence experimental results show that the proposed algorithm clearly reconstructs important anatomical structures in cardiac cine image and provides improved image quality compared to existing state of the art methods such as k t focuss k t slr and master

bilinear generalized approximate message passing part i derivation in this paper we extend the generalized approximate message passing g amp approach originally proposed for high dimensional generalized linear regression in the context of compressive sensing to the generalized bilinear case which enables its application to matrix completion robust pca dictionary learning and related matrix factorization problems here in part i of a two part paper we derive our bilinear g amp big amp algorithm as an approximation of the sum product belief propagation algorithm in the high dimensional limit where central limit theorem arguments and taylor series approximations apply and under the assumption of statistically independent matrix entries with known priors in addition we propose an adaptive damping mechanism that aids convergence under finite problem sizes an expectation maximization em based method to automatically tune the parameters of the assumed priors and two rank selection strategies in part ii of the paper we will discuss the specializations of em big amp to the problems of matrix completion robust pca and dictionary learning and we will present the results of an extensive empirical study comparing em big amp to state of the art algorithms on each problem

multi stage convex relaxation method for low rank and sparse matrix separation problem the low rank and sparse matrix separation problem recovering a low rank matrix and a sparse matrix from a small number of linear observations has arisen in a broad set of applications the desire for efficiency motivated the research on it however most of them focus on the nuclear norm plus 1 norm minimization problem which is a single convex relaxation of the low rank and sparse matrix separation problem this paper is devoted to investigate a multi stage convex relaxation method to recover the low rank and sparse components via solving a sequence of convex relaxations while the first stage of the method includes the nuclear norm plus 1 norm minimization problem numerical studies indicate that the multi stage convex relaxation outperforms the one stage convex relaxation especially when observations are incomplete and noisy we have also verified the efficiency of the proposed method by applying it to the static background extraction from surveillance video

on the rotational invariant l1 norm pca abstract principal component analysis pca is a powerful tool for dimensionality reduction unfortunately it is sensitive to outliers so that various robust pca variants were proposed in the literature one of the most frequently applied methods for high dimensional data reduction is the rotational invariant l 1 norm pca of ding and coworkers so far no convergence proof for this algorithm was available the main topic of this paper is to fill this gap we reinterpret this robust approach as a conditional gradient algorithm and show moreover that it coincides with a gradient descent algorithm on grassmann manifolds based on the latter point of view we prove global convergence of the whole series of iterates to a critical point using the kurdyka ojasiewicz property of the objective function where we have to pay special attention to so called anchor points where the function is not differentiable

on the linear convergence of the alternating direction method of multipliers we analyze the convergence rate of the alternating direction method of multipliers admm for minimizing the sum of two or more nonsmooth convex separable functions subject to linear constraints previous analysis of the admm typically assumes that the objective function is the sum of only two convex functions defined on two separable blocks of variables even though the algorithm works well in numerical experiments for three or more blocks moreover there has been no rate of convergence analysis for the admm without strong convexity in the objective function in this paper we establish the global r linear convergence of the admm for minimizing the sum of any number of convex separable functions assuming that a certain error bound condition holds true and the dual stepsize is sufficiently small such an error bound condition is satisfied for example when the feasible set is a compact polyhedron and the objective function consists of a smooth strictly convex function composed with a linear mapping and a nonsmooth ell 1 l1 regularizer this result implies the linear convergence of the admm for contemporary applications such as lasso without assuming strong convexity of the objective function

deformation corrected compressed sensing dc cs a novel framework for accelerated dynamic mri we propose a novel deformation corrected compressed sensing dc cs framework to recover contrast enhanced dynamic magnetic resonance images from undersampled measurements we introduce a formulation that is capable of handling a wide class of sparsity compactness priors on the deformation corrected dynamic signal in this work we consider example compactness priors such as sparsity in temporal fourier domain sparsity in temporal finite difference domain and nuclear norm penalty to exploit low rank structure using variable splitting we decouple the complex optimization problem to simpler and well understood sub problems the resulting algorithm alternates between simple steps of shrinkage based denoising deformable registration and a quadratic optimization step additionally we employ efficient continuation strategies to reduce the risk of convergence to local minima the decoupling enabled by the proposed scheme enables us to apply this scheme to contrast enhanced mri applications through experiments on numerical phantom and in vivo myocardial perfusion mri datasets we observe superior image quality of the proposed dc cs scheme in comparison to the classical k t focuss with motion estimation correction scheme and demonstrate reduced motion artifacts over classical compressed sensing schemes that utilize the compact priors on the original deformation uncorrected signal

a variable penalty alternating directions method for convex optimization we study a generalized version of the method of alternating directions as applied to the minimization of the sum of two convex functions subject to linear constraints the method consists of solving consecutively in each iteration two optimization problems which contain in the objective function both lagrangian and proximal terms the minimizers determine the new proximal terms and a simple update of the lagrangian terms follows we prove a convergence theorem which extends existing results by relaxing the assumption of uniqueness of minimizers another novelty is that we allow penalty matrices and these may vary per iteration this can be beneficial in applications since it allows additional tuning of the method to the problem and can lead to faster convergence relative to fixed penalties as an application we derive a decomposition scheme for block angular optimization and present computational results on a class of dual block angular problems

matrix completion with column manipulation near optimal sample robustness rank tradeoffs this paper considers the problem of matrix completion when some number of the columns are completely and arbitrarily corrupted potentially by a malicious adversary it is well known that standard algorithms for matrix completion can return arbitrarily poor results if even a single column is corrupted one direct application comes from robust collaborative filtering here some number of users are so called manipulators who try to skew the predictions of the algorithm by calibrating their inputs to the system in this paper we develop an efficient algorithm for this problem based on a combination of a trimming procedure and a convex program that minimizes the nuclear norm and the ell 1 2 norm our theoretical results show that given a vanishing fraction of observed entries it is nevertheless possible to complete the underlying matrix even when the number of corrupted columns grows significantly our results hold without any assumptions on the locations or values of the observed entries of the manipulated columns moreover we show by an information theoretic argument that our guarantees are nearly optimal in terms of the fraction of sampled entries on the authentic columns the fraction of corrupted columns and the rank of the underlying matrix our results therefore sharply characterize the tradeoffs between sample robustness and rank in matrix completion

spectral estimation of irregularly sampled multidimensional processes by generalized prolate spheroidal sequences a nonparametric spectral estimation method is presented for bandlimited random processes that have been sampled at arbitrary points in one or more dimensions the method makes simultaneous use of several weight sequences that depend on the set of sampling point the signal band and the frequency band being analyzed these sequences are solutions to a generalized matrix eigenvalue problem and are termed generalized prolate spheroidal sequences being extensions of the familiar discrete prolate spheroidal sequences statistics of the estimator are derived and the tradeoff among bias variance and resolution is quantified the method avoids several problems typically associated with irregularly sampled data and multidimensional processes a related method is suggested that has nearly as good performance while requiring significantly fewer computations

characterization of the subdifferential of some matrix norms abstract a characterization is given of the subdifferential of matrix norms from two classes orthogonally invariant norms and operator or subordinate norms specific results are derived for some special cases

a deterministic analysis for lrr the recently proposed low rank representation lrr method has been empirically shown to be useful in various tasks such as motion segmentation image segmentation saliency detection and face recognition while potentially powerful lrr depends heavily on the configuration of its key parameter lambda in realistic environments where the prior knowledge about data is lacking however it is still unknown how to choose lambda in a suitable way even more there is a lack of rigorous analysis about the success conditions of the method and thus the significance of lrr is a little bit vague in this paper we therefore establish a theoretical analysis for lrr striving for figuring out under which conditions lrr can be successful and deriving a moderately good estimate to the key parameter lambda as well simulations on synthetic data points and experiments on real motion sequences verify our claims

a novel local surface feature for 3d object recognition under clutter and occlusion this paper presents a highly distinctive local surface feature called the trisi feature for recognizing 3d objects in the presence of clutter and occlusion for a feature point we first construct a unique and repeatable local reference frame lrf using the implicit geometrical information of neighboring triangular faces we then generate three signatures from the three orthogonal coordinate axes of the lrf these signatures are concatenated and then compressed into a trisi feature finally we propose an effective 3d object recognition algorithm based on hierarchical feature matching we tested our trisi feature on two popular datasets rigorous experimental results show that the trisi feature was highly descriptive and outperformed existing algorithms under all levels of gaussian noise laplacian noise shot noise varying mesh resolutions occlusion and clutter moreover we tested our trisi based 3d object recognition algorithm on four standard datasets the experimental results show that our algorithm achieved the best overall recognition results on these datasets

projective multiview structure and motion from element wise factorization the sturm triggs type iteration is a classic approach for solving the projective structure from motion sfm factorization problem which iteratively solves the projective depths scene structure and camera motions in an alternated fashion like many other iterative algorithms the sturm triggs iteration suffers from common drawbacks such as requiring a good initialization the iteration may not converge or may only converge to a local minimum and so on in this paper we formulate the projective sfm problem as a novel and original element wise factorization i e hadamard factorization problem as opposed to the conventional matrix factorization thanks to this formulation we are able to solve the projective depths structure and camera motions simultaneously by convex optimization to address the scalability issue we adopt a continuation based algorithm our method is a global method in the sense that it is guaranteed to obtain a globally optimal solution up to relaxation gap another advantage is that our method can handle challenging real world situations such as missing data and outliers quite easily and all in a natural and unified manner extensive experiments on both synthetic and real images show comparable results compared with the state of the art methods

adaptive rate reconstruction of time varying signals with application in compressive foreground extraction we propose and analyze an online algorithm for reconstructing a sequence of signals from a limited number of linear measurements the signals are assumed sparse with unknown support and evolve over time according to a generic nonlinear dynamical model our algorithm based on recent theoretical results for ell 1 ell 1 minimization is recursive and computes the number of measurements to be taken at each time on the fly as an example we apply the algorithm to online compressive video foreground extraction a problem stated as follows given a set of measurements of a sequence of images with a static background simultaneously reconstruct each image while separating its foreground from the background the performance of our method is illustrated on sequences of real images we observe that it allows a dramatic reduction in the number of measurements or reconstruction error with respect to state of the art compressive background subtraction schemes

on the solution of large scale sdp problems by the modified barrier method using iterative solvers the limiting factors of second order methods for large scale semidefinite optimization are the storage and factorization of the newton matrix for a particular algorithm based on the modified barrier method we propose to use iterative solvers instead of the routinely used direct factorization techniques the preconditioned conjugate gradient method proves to be a viable alternative for problems with a large number of variables and modest size of the constrained matrix we further propose to avoid explicit calculation of the newton matrix either by an implicit scheme in the matrix vector product or using a finite difference formula this leads to huge savings in memory requirements and for certain problems to further speed up of the algorithm

low rank matrix completion in the presence of high coherence prevalent matrix completion methods capture only the low rank property which gives merely a constraint that the data points lie on some low dimensional subspace but generally ignore the extra structures beyond low rank that specify in more detail how the data points lie on the subspace whenever the data points are not uniformly distributed on the low dimensional subspace the row coherence of the target matrix to recover could be considerably high and accordingly prevalent methods might fail even if the target matrix is fairly low rank to relieve this challenge we suggest to consider a model termed low rank factor decomposition lrfd which imposes an additional restriction that the data points must be represented as linear compressive combinations of the bases in a given dictionary we show that lrfd can effectively mitigate the challenges of high row coherence provided that its dictionary is configured properly namely it is mathematically proven that if the dictionary is well conditioned and low rank then lrfd can weaken the dependence on the row coherence in particular if the dictionary itself is low rank then the dependence on the row coherence can be entirely removed subsequently we devise two practical algorithms to obtain proper dictionaries in unsupervised environments one uses the existing matrix completion methods to construct the dictionary in lrfd and the other tries to learn a proper dictionary from the data given experiments on randomly generated matrices and motion datasets show superior performance of our proposed algorithms

iterative and active graph clustering using trace norm minimization without cluster size constraints this paper investigates graph clustering under the planted partition model in the presence of small clusters traditional results dictate that for an algorithm to provably correctly recover the underlying clusters all clusters must be sufficiently large in particular the cluster sizes need to be n where n is the number of nodes of the graph we show that this is not really a restriction by a refined analysis of a convex optimization based recovery approach we prove that small clusters under certain mild assumptions do not hinder recovery of large ones based on this result we further devise an iterative algorithm to provably recover almost all clusters via a peeling strategy we recover large clusters first leading to a reduced problem and repeat this procedure these results are extended to the partial observation setting in which only a chosen part of the graph is observed the peeling strategy gives rise to an active learning algorithm in which edges adjacent to smaller clusters are queried more often after large clusters are learned and removed we expect that the idea of iterative peeling that is sequentially identifying a subset of the clusters and reducing the problem to a smaller one is useful more broadly beyond the specific implementations based on convex optimization used in this paper

set membership identification of switched linear systems with known number of subsystems this paper addresses the problem of robust identification of a class of discrete time linear hybrid systems switched linear models in a set membership framework given a finite collection of input output data from a noisy process the objective is twofold i establish whether this data was generated by a system that switches amongst an a priori known number of subsystems and ii in that case identify a suitable set of linear models along with a switching sequence that can explain the available experimental information our main result shows that these problems are equivalent to minimizing the rank of a matrix whose entries are affine in the optimization variables subject to a convex constraint imposing that these variables are the moments of an unknown borel measure with finite support the use of well known tight convex relaxations of rank allows for further reducing the problem to a semidefinite optimization that can be efficiently solved in the second part of the paper we extend these results to handle sensor failures that result in corrupted input output measurements assuming that these failures are infrequent we show that the problem can be recast into an optimization form where the objective is to simultaneously minimize the rank of a matrix and the number of nonzero rows of a second one in both cases appealing to well known convex relaxations of rank and sparsity leads to overall semidefinite optimization problems that can be efficiently solved these results are illustrated with multiple examples showing substantially improved identification performance in the presence of noise and sensor faults

recovering low rank and sparse components of matrices from incomplete and noisy observations many problems can be characterized by the task of recovering the low rank and sparse components of a given matrix recently it was discovered that this nondeterministic polynomial time hard np hard task can be well accomplished both theoretically and numerically via heuristically solving a convex relaxation problem where the widely acknowledged nuclear norm and l 1 norm are utilized to induce low rank and sparsity this paper studies the recovery task in the general settings that only a fraction of entries of the matrix can be observed and the observation is corrupted by both impulsive and gaussian noise we show that the resulting model falls into the applicable scope of the classical augmented lagrangian method moreover the separable structure of the new model enables us to solve the involved subproblems more efficiently by splitting the augmented lagrangian function hence some splitting numerical algorithms are developed for solving the new recovery model some preliminary numerical experiments verify that these augmented lagrangian based splitting algorithms are easily implementable and surprisingly efficient for tackling the new recovery model

laplacian regularized low rank representation and its applications low rank representation lrr has recently attracted a great deal of attention due to its pleasing efficacy in exploring low dimensional subspace structures embedded in data for a given set of observed data corrupted with sparse errors lrr aims at learning a lowest rank representation of all data jointly lrr has broad applications in pattern recognition computer vision and signal processing in the real world data often reside on low dimensional manifolds embedded in a high dimensional ambient space however the lrr method does not take into account the non linear geometric structures within data thus the locality and similarity information among data may be missing in the learning process to improve lrr in this regard we propose a general laplacian regularized low rank representation framework for data representation where a hypergraph laplacian regularizer can be readily introduced into i e a non negative sparse hyper laplacian regularized lrr model nshlrr by taking advantage of the graph regularizer our proposed method not only can represent the global low dimensional structures but also capture the intrinsic non linear geometric information in data the extensive experimental results on image clustering semi supervised image classification and dimensionality reduction tasks demonstrate the effectiveness of the proposed method

robust matrix completion this paper considers the problem of estimation of a low rank matrix when most of its entries are not observed and some of the observed entries are corrupted the observations are noisy realizations of a sum of a low rank matrix which we wish to estimate and a second matrix having a complementary sparse structure such as elementwise sparsity or columnwise sparsity we analyze a class of estimators obtained as solutions of a constrained convex optimization problem combining the nuclear norm penalty and a convex relaxation penalty for the sparse constraint our assumptions allow for simultaneous presence of random and deterministic patterns in the sampling scheme we establish rates of convergence for the low rank component from partial and corrupted observations in the presence of noise and we show that these rates are minimax optimal up to logarithmic factors

low rank matrix recovery from errors and erasures this paper considers the recovery of a low rank matrix from an observed version that simultaneously contains both 1 erasures most entries are not observed and 2 errors values at a constant fraction of unknown locations are arbitrarily corrupted we provide a new unified performance guarantee on when minimizing nuclear norm plus l1 norm succeeds in exact recovery our result allows for the simultaneous presence of random and deterministic components in both the error and erasure patterns by specializing this one single result in different ways we recover up to poly log factors as corollaries all the existing results in exact matrix completion and exact sparse and low rank matrix decomposition our unified result also provides the first guarantees for 1 recovery when we observe a vanishing fraction of entries of a corrupted matrix and 2 deterministic matrix completion

minimax lower bounds for noisy matrix completion under sparse factor models this paper examines fundamental error characteristics for a general class of matrix completion problems where the matrix of interest is a product of two a priori unknown matrices one of which is sparse and the observations are noisy our main contributions come in the form of minimax lower bounds for the expected per element squared error for this problem under several common noise models specifically we analyze scenarios where the corruptions are characterized by additive gaussian noise or additive heavier tailed laplace noise poisson distributed observations and highly quantized e g one bit observations as instances of our general result our results establish that the error bounds derived in soni et al 2016 for complexity regularized maximum likelihood estimators achieve up to multiplicative constants and logarithmic factors the minimax error rates in each of these noise scenarios provided that the nominal number of observations is large enough and the sparse factor has on an average at least one non zero per column

finite sample approximation results for principal component analysis a matrix perturbation approach principal component analysis pca is a standard tool for dimensional reduction of a set of n observations samples each with p variables in this paper using a matrix perturbation approach we study the nonasymptotic relation between the eigenvalues and eigenvectors of pca computed on a finite sample of size n and those of the limiting population pca as n oo as in machine learning we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample pca and population pca under a spiked covariance model in addition we also consider the relation between finite sample pca and the asymptotic results in the joint limit p n with p n c we present a matrix perturbation view of the phase transition phenomenon and a simple linear algebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit moreover our analysis also applies for finite p n where we show that although there is no sharp phase transition as in the infinite case either as a function of noise level or as a function of sample size n the eigenvector of sample pca may exhibit a sharp loss of tracking suddenly losing its relation to the true eigenvector of the population pca matrix this occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise whose eigenvector points in a random direction

a proximal based decomposition method for convex minimization problems this paper presents a decomposition method for solving convex minimization problems at each iteration the algorithm computes two proximal steps in the dual variables and one proximal step in the primal variables we derive this algorithm from rockafellar s proximal method of multipliers which involves an augmented lagrangian with an additional quadratic proximal term the algorithm preserves the good features of the proximal method of multipliers with the additional advantage that it leads to a decoupling of the constraints and is thus suitable for parallel implementation we allow for computing approximately the proximal minimization steps and we prove that under mild assumptions on the problem s data the method is globally convergent and at a linear rate the method is compared with alternating direction type methods and applied to the particular case of minimizing a convex function over a finite intersection of closed convex sets

a weak to strong convergence principle for feje monotone methods in hilbert spaces we consider a wide class of iterative methods arising in numerical mathematics and optimization that are known to converge only weakly exploiting an idea originally proposed by haugazeau we present a simple modification of these methods that makes them strongly convergent without additional assumptions several applications are discussed