modified principal component analysis an integration of multiple similarity subspace models we modify the conventional principal component analysis pca and propose a novel subspace learning framework modified pca mpca using multiple similarity measurements mpca computes three similarity matrices exploiting the similarity measurements 1 mutual information 2 angle information and 3 gaussian kernel similarity we employ the eigenvectors of similarity matrices to produce new subspaces referred to as similarity subspaces a new integrated similarity subspace is then generated using a novel feature selection approach this approach needs to construct a kind of vector set termed weak machine cell wmc which contains an appropriate number of the eigenvectors spanning the similarity subspaces combining the wrapper method and the forward selection scheme mpca selects a wmc at a time that has a powerful discriminative capability to classify samples mpca is very suitable for the application scenarios in which the number of the training samples is less than the data dimensionality mpca outperforms the other state of the art pca based methods in terms of both classification accuracy and clustering result in addition mpca can be applied to face image reconstruction mpca can use other types of similarity measurements extensive experiments on many popular real world data sets such as face databases show that mpca achieves desirable classification results as well as has a powerful capability to represent data

principal component analysis based on l1 norm maximization a method of principal component analysis pca based on a new l1 norm optimization technique is proposed unlike conventional pca which is based on l2 norm the proposed method is robust to outliers because it utilizes l1 norm which is less sensitive to outliers it is invariant to rotations as well the proposed l1 norm optimization technique is intuitive simple and easy to implement it is also proven to find a locally maximal solution the proposed method is applied to several datasets and the performances are compared with those of other conventional methods

eigenfaces vs fisherfaces recognition using class specific linear projection we develop a face recognition algorithm which is insensitive to large variation in lighting direction and facial expression taking a pattern classification approach we consider each pixel in an image as a coordinate in a high dimensional space we take advantage of the observation that the images of a particular face under varying illumination but fixed pose lie in a 3d linear subspace of the high dimensional image space if the face is a lambertian surface without shadowing however since faces are not truly lambertian surfaces and do indeed produce self shadowing images will deviate from this linear subspace rather than explicitly modeling this deviation we linearly project the image into a subspace in a manner which discounts those regions of the face with large deviation our projection method is based on fisher s linear discriminant and produces well separated classes in a low dimensional subspace even under severe variation in lighting and facial expressions the eigenface technique another method based on linearly projecting the image space to a low dimensional subspace has similar computational requirements yet extensive experimental results demonstrate that the proposed fisherface method has error rates that are lower than those of the eigenface technique for tests on the harvard and yale face databases

image outlier detection and feature extraction via l1 norm based 2d probabilistic pca this paper introduces an l1 norm based probabilistic principal component analysis model on 2d data l1 2dppca based on the assumption of the laplacian noise model the laplacian or l1 density function can be expressed as a superposition of an infinite number of gaussian distributions under this expression a bayesian inference can be established based on the variational expectation maximization approach all the key parameters in the probabilistic model can be learned by the proposed variational algorithm it has experimentally been demonstrated that the newly introduced hidden variables in the superposition can serve as an effective indicator for data outliers experiments on some publicly available databases show that the performance of l1 2dppca has largely been improved after identifying and removing sample outliers resulting in more accurate image reconstruction than the existing pca based methods the performance of feature extraction of the proposed method generally outperforms other existing algorithms in terms of reconstruction errors and classification accuracy

inductive robust principal component analysis in this paper we address the error correction problem that is to uncover the low dimensional subspace structure from high dimensional observations which are possibly corrupted by errors when the errors are of gaussian distribution principal component analysis pca can find the optimal in terms of least square error low rank approximation to high dimensional data however the canonical pca method is known to be extremely fragile to the presence of gross corruptions recently wright established a so called robust principal component analysis rpca method which can well handle the grossly corrupted data however rpca is a transductive method and does not handle well the new samples which are not involved in the training procedure given a new datum rpca essentially needs to recalculate over all the data resulting in high computational cost so rpca is inappropriate for the applications that require fast online computation to overcome this limitation in this paper we propose an inductive robust principal component analysis irpca method given a set of training data unlike rpca that targets on recovering the original data matrix irpca aims at learning the underlying projection matrix which can be used to efficiently remove the possible corruptions in any datum the learning is done by solving a nuclear norm regularized minimization problem which is convex and can be solved in polynomial time extensive experiments on a benchmark human face dataset and two video surveillance datasets show that irpca cannot only be robust to gross corruptions but also handle the new data well and in an efficient way

robust factorization factorization algorithms for recovering structure and motion from an image stream have many advantages but they usually require a set of well tracked features such a set is in generally not available in practical applications there is thus a need for making factorization algorithms deal effectively with errors in the tracked features we propose a new and computationally efficient algorithm for applying an arbitrary error function in the factorization scheme this algorithm enables the use of robust statistical techniques and arbitrary noise models for the individual features these techniques and models enable the factorization scheme to deal effectively with mismatched features missing features and noise on the individual features the proposed approach further includes a new method for euclidean reconstruction that significantly improves convergence of the factorization algorithms the proposed algorithm has been implemented as a modification of the christy horaud factorization scheme which yields a perspective reconstruction based on this implementation a considerable increase in error tolerance is demonstrated on real and synthetic data the proposed scheme can however be applied to most other factorization algorithms

subspace learning from image gradient orientations we introduce the notion of subspace learning from image gradient orientations for appearance based object recognition as image data are typically noisy and noise is substantially different from gaussian traditional subspace learning from pixel intensities very often fails to estimate reliably the low dimensional subspace of a given data population we show that replacing pixel intensities with gradient orientations and the l2 norm with a cosine based distance measure offers to some extend a remedy to this problem within this framework which we coin image gradient orientations igo subspace learning we first formulate and study the properties of principal component analysis of image gradient orientations igo pca we then show its connection to previously proposed robust pca techniques both theoretically and experimentally finally we derive a number of other popular subspace learning techniques namely linear discriminant analysis lda locally linear embedding lle and laplacian eigenmaps le experimental results show that our algorithms significantly outperform popular methods such as gabor features and local binary patterns and achieve state of the art performance for difficult problems such as illumination and occlusion robust face recognition in addition to this the proposed igo methods require the eigendecomposition of simple covariance matrices and are as computationally efficient as their corresponding l2 norm intensity based counterparts matlab code for the methods presented in this paper can be found at http ibug doc ic ac uk resources

principal components and orthogonal regression based on robust scales both principal components analysis pca and orthogonal regression deal with finding a p dimensional linear manifold minimizing a scale of the orthogonal distances of the m dimensional data points to the manifold the main conceptual difference is that in pca p is estimated from the data to attain a small proportion of unexplained variability whereas in orthogonal regression p equals m 1 the two main approaches to robust pca are using the eigenvectors of a robust covariance matrix and searching for the projections that maximize or minimize a robust univariate dispersion measure this article is more akin to second approach but rather than finding the components one by one we directly undertake the problem of finding for a given p a p dimensional linear manifold minimizing a robust scale of the orthogonal distances of the data points to the manifold the scale may be either a smooth m scale or a trimmed scale an iterative algorithm is developed that is shown to converge to a local minimum a

a majorize minimize strategy for subspace optimization applied to image restoration this paper proposes accelerated subspace optimization methods in the context of image restoration subspace optimization methods belong to the class of iterative descent algorithms for unconstrained optimization at each iteration of such methods a stepsize vector allowing the best combination of several search directions is computed through a multidimensional search it is usually obtained by an inner iterative second order method ruled by a stopping criterion that guarantees the convergence of the outer algorithm as an alternative we propose an original multidimensional search strategy based on the majorize minimize principle it leads to a closed form stepsize formula that ensures the convergence of the subspace algorithm whatever the number of inner iterations the practical efficiency of the proposed scheme is illustrated in the context of edge preserving image restoration

linear discriminant analysis based on l1 norm maximization linear discriminant analysis lda is a well known dimensionality reduction technique which is widely used for many purposes however conventional lda is sensitive to outliers because its objective function is based on the distance criterion using l2 norm this paper proposes a simple but effective robust lda version based on l1 norm maximization which learns a set of local optimal projection vectors by maximizing the ratio of the l1 norm based between class dispersion and the l1 norm based within class dispersion the proposed method is theoretically proved to be feasible and robust to outliers while overcoming the singular problem of the within class scatter matrix for conventional lda experiments on artificial datasets standard classification datasets and three popular image databases demonstrate the efficacy of the proposed method

dispersion operators and resistant second order functional data analysis inferences related to the second order properties of functional data as expressed by covariance structure can become unreliable when the data are non gaussian or contain unusual observations in the functional setting it is often difficult to identify atypical observations as their distinguishing characteristics can be manifold but subtle in this paper we introduce the notion of a dispersion operator investigate its use in probing the second order structure of functional data and develop a test for comparing the second order characteristics of two functional samples that is resistant to atypical observations and departures from normality the proposed test is a regularized m test based on a spectrally truncated version of the hilbert schmidt norm of a score operator defined via the dispersion operator we derive the asymptotic distribution of the test statistic investigate the behaviour of the test in a simulation study and illustrate the method on a structural biology dataset

the role of critical exponents in blow up theorems the sequel abstract in 27 fujita showed that for positive solutions the initial value problem in r n for u t u u p with p 1 exhibited the following behavior if p p c 1 2 n then the initial value problem does not have any nontrivial non negative solution existing on r n 0 a global solution whereas if p p c there exist global small data positive solutions as well as solutions which are non global we call such a result a blow up theorem of fujita type in 50 levine discussed the various theorems of this type that had appeared in the literature prior to 1990 in this paper we revisit the literature since 1990

rotated orthogonal transform rot for motion compensation residual coding discrete cosine transform dct is the orthogonal transform that is most commonly used in image and video compression the motion compensation residual mc residual is also compressed with the dct in most video codecs however the mc residual has different characteristics from a nature image in this paper we develop a new orthogonal transform rotated orthogonal transform rot that can perform better on the mc residual than the dct for coding purposes we derive the proposed rot based on orthogonal constrained l1 norm minimization problem for its sparse property using the dct matrix as the starting point a better orthogonal transform matrix is derived in addition by exploring inter frame dependency and local motion activity transmission of substantial side information is avoided the experiment results confirm that with small computation overhead the rot is adaptive to change of local spatial characteristic of mc residual frame and provides higher compression efficiency for the mc residual than dct especially for high and complex motion videos

l1 norm kernel discriminant analysis via bayes error bound optimization for robust feature extraction a novel discriminant analysis criterion is derived in this paper under the theoretical framework of bayes optimality in contrast to the conventional fisher s discriminant criterion the major novelty of the proposed one is the use of l1 norm rather than l2 norm which makes it less sensitive to the outliers with the l1 norm discriminant criterion we propose a new linear discriminant analysis l1 lda method for linear feature extraction problem to solve the l1 lda optimization problem we propose an efficient iterative algorithm in which a novel surrogate convex function is introduced such that the optimization problem in each iteration is to simply solve a convex programming problem and a close form solution is guaranteed to this problem moreover we also generalize the l1 lda method to deal with the nonlinear robust feature extraction problems via the use of kernel trick and hereafter proposed the l1 norm kernel discriminant analysis l1 kda method extensive experiments on simulated and real data sets are conducted to evaluate the effectiveness of the proposed method in comparing with the state of the art methods

probably correct k nearest neighbor search in high dimensions a novel approach for k nearest neighbor k nn searching with euclidean metric is described it is well known that many sophisticated algorithms cannot beat the brute force algorithm when the dimensionality is high in this study a probably correct approach in which the correct set of k nearest neighbors is obtained in high probability is proposed for greatly reducing the searching time we exploit the marginal distribution of the k th nearest neighbors in low dimensions which is estimated from the stored data an empirical percentile approach we analyze the basic nature of the marginal distribution and show the advantage of the implemented algorithm which is a probabilistic variant of the partial distance searching its query time is sublinear in data size n that is o email protected with d o 1 in n and email protected 1 for any fixed dimension m

independent component analysis by lp norm optimization abstract in this paper a couple of new algorithms for independent component analysis ica are proposed in the proposed methods the independent sources are assumed to follow a predefined distribution of the form f s exp s p and a maximum likelihood estimation is used to separate the sources in the first method a gradient ascent method is used for the maximum likelihood estimation while in the second a non iterative algorithm is proposed based on the relaxation of the problem the maximization of the log likelihood of the estimated source xtw given the parameter p and the data x is shown to be equivalent to the minimization of lp norm of the projected data xtw this formulation of ica has a very close relationship with the lp pca where the maximization of the same objective function is solved the proposed algorithm solves an approximation of the lp norm minimization problem for both super p 2 cases and shows superior performance in separating independent sources than the state of the art algorithms for ica computation

robust functional principal components a projection pursuit approach in many situations data are recorded over a period of time and may be regarded as realizations of a stochastic process in this paper robust estimators for the principal components are considered by adapting the projection pursuit approach to the functional data setting our approach combines robust projection pursuit with different smoothing methods consistency of the estimators are shown under mild assumptions the performance of the classical and robust procedures are compared in a simulation study under different contamination schemes 1 introduction analogous to classical principal components analysis pca the projection pursuit approach to robust pca is based on finding projections of the data which have maximal dispersion instead of using the variance as a measure of dispersion a robust scale estimator sn is used for the maximization problem this approach was introduced by li and chen 1985 who proposed estimators based on maximizing or minimizing a robust scale in this way given a sample xi r d 1 i n the first robust principal component vector is defined as

a distribution free m estimator of multivariate scatter the existence and uniqueness of a limiting form of a huber type m estimator of multivariate scatter is established under certain conditions on the observed sample these conditions hold with probability one when sampling randomly from a continuous multivariate distribution the existence of the estimator is proven by showing that it is the limiting point of a specific algorithm hence the proof is constructive for continuous populations the estimator of multivariate scatter is shown to be strongly consistent and asymptotically normal an important property of the estimator is that its asymptotic distribution is distribution free with respect to the class of continuous elliptically distributed populations this distribution free property also holds for the finite sample size distribution when the location parameter is known in addition the estimator is the most robust estimator of the scatter matrix of an elliptical distribution in the sense of minimizing the maximum asymptotic variance

ranking graph embedding for learning to rerank dimensionality reduction is a key step to improving the generalization ability of reranking in image search however existing dimensionality reduction methods are typically designed for classification clustering and visualization rather than for the task of learning to rank without using of ranking information such as relevance degree labels direct utilization of conventional dimensionality reduction methods in ranking tasks generally cannot achieve the best performance in this paper we show that introducing ranking information into dimensionality reduction significantly increases the performance of image search reranking the proposed method transforms graph embedding a general framework of dimensionality reduction into ranking graph embedding range by modeling the global structure and the local relationships in and between different relevance degree sets respectively the proposed method also defines three types of edge weight assignment between two nodes binary reconstruction and global in addition a novel principal components analysis based similarity calculation method is presented in the stage of global graph construction extensive experimental results on the msra mm database demonstrate the effectiveness and superiority of the proposed range method and the image search reranking framework

sliced inverse regression for dimension reduction abstract modern advances in computing power have greatly widened scientists scope in gathering and investigating information from many variables information which might have been ignored in the past yet to effectively scan a large pool of variables is not an easy task although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics in this article we propose a novel data analytic tool sliced inverse regression sir for reducing the dimension of the input variable x without going through any parametric or nonparametric model fitting process this method explores the simplicity of the inverse view of regression that is instead of regressing the univariate output variable y against the multivariate x we regress x against y forward regression and inverse regression are connected by a theorem that motivates this method the theoretical properties of sir are investigated under a model of the form y f 1 x k x e where the k s are the unknown

optimizing the kernel in the empirical feature space in this paper we present a method of kernel optimization by maximizing a measure of class separability in the empirical feature space an euclidean space in which the training data are embedded in such a way that the geometrical structure of the data in the feature space is preserved employing a data dependent kernel we derive an effective kernel optimization algorithm that maximizes the class separability of the data in the empirical feature space it is shown that there exists a close relationship between the class separability measure introduced here and the alignment measure defined recently by cristianini extensive simulations are carried out which show that the optimized kernel is more adaptive to the input data and leads to a substantial sometimes significant improvement in the performance of various data classification algorithms

life span and a new critical exponent for a degenerate parabolic equation abstract in this paper we consider the positive solution of the cauchy problem for the equation u t u p u u q p 1 q 1 and give a secondary critical exponent of the behavior of initial value at infinity for the existence of global and nonglobal solutions of the cauchy problem furthermore the life span of solutions are also studied

robust orthogonal complement principal component analysis abstractrecently the robustification of principal component analysis pca has attracted lots of attention from statisticians engineers and computer scientists in this work we study the type of outliers that are not necessarily apparent in the original observation space but can seriously affect the principal subspace estimation based on a mathematical formulation of such transformed outliers a novel robust orthogonal complement principal component analysis roc pca is proposed the framework combines the popular sparsity enforcing and low rank regularization techniques to deal with row wise outliers as well as element wise outliers a nonasymptotic oracle inequality guarantees the accuracy and high breakdown performance of roc pca in finite samples to tackle the computational challenges an efficient algorithm is developed on the basis of stiefel manifold optimization and iterative thresholding furthermore a batch variant is proposed to significantly reduce the cost in ultra high dimensions th

cones of matrices and set functions and 0 1 optimization it has been recognized recently that to represent a polyhedron as the projection of a higher dimensional but simpler polyhedron is a powerful tool in polyhedral combinatorics a general method is developed to construct higher dimensional polyhedra or in some cases convex sets whose projection approximates the convex hull of 0 1 valued solutions of a system of linear inequalities an important feature of these approximations is that one can optimize any linear objective function over them in polynomial time in the special case of the vertex packing polytope a sequence of systems of inequalities is obtained such that the first system already includes clique odd hole odd antihole wheel and orthogonality constraints in particular for perfect and many other graphs this first system gives the vertex packing polytope for various classes of graphs including t perfect graphs it follows that the stable set polytope is the projection of a polytope with a polynomial number of facets an extension o

l1 norm based kpca for novelty detection novelty detection is a one class classification problem and it builds up the model with only normal samples based on which the novelty is detected though conventional kpca is an effective method of building one class classification models it is prone to being affected by the presence of outliers due to its inherent properties of l2 norm in this paper we propose a new optimization problem l1 norm based kpca which is robust to outliers correspondingly we present the algorithm and the measure of novelty the proposed method is applied to novelty detection and performs well on the simulation data sets

robpca a new approach to robust principal component analysis we introduce a new method for robust principal component analysis pca classical pca is based on the empirical covariance matrix of the data and hence is highly sensitive to outlying observations two robust approaches have been developed to date the first approach is based on the eigenvectors of a robust scatter matrix such as the minimum covariance determinant or an s estimator and is limited to relatively low dimensional data the second approach is based on projection pursuit and can handle high dimensional data here we propose the robpca approach which combines projection pursuit ideas with robust scatter matrix estimation robpca yields more accurate estimates at noncontaminated datasets and more robust estimates at contaminated data robpca can be computed rapidly and is able to detect exact fit situations as a by product robpca produces a diagnostic plot that displays and classifies the outliers we apply the algorithm to several datasets from chemometrics and engineering

towards robust discriminative projections learning via non greedy l2 1 norm minmax linear discriminant analysis lda is one of the most successful supervised dimensionality reduction methods and has been widely used in many real world applications however the l 2 norm is employed as the distance measure in objective of lda which is sensitive to outliers many previous works improve the robustness of lda by using l1 norm distance however the robustness against outliers is limited and the solver of l1 norm are mostly based on greedy search strategy which is time consuming and easy to get stuck in a local optimum in this paper we propose a novel robust lda measured by l 2 1 norm to learn robust discriminative projections since the proposed model needs to minimize and maximize minmax l 2 1 norm terms simultaneously it is challenging to solve as a result we first systematically derive an efficient iterative optimization algorithm to solve a general ratio minimization problem and rigorously prove its convergence more importantly an efficient non greedy iterative re weighted optimization algorithm is developed based on preceding approach for solving proposed l 2 1 norm minmax problem besides an optimal weighted mean mechanism is driven according to designed objective and solver which can be applied to other approach for robustness improvement extensive experimental results on several real world datasets show the effectiveness of proposed method

compressed submanifold multifactor analysis although widely used multilinear pca mpca one of the leading multilinear analysis methods still suffers from four major drawbacks first it is very sensitive to outliers and noise second it is unable to cope with missing values third it is computationally expensive since mpca deals with large multi dimensional datasets finally it is unable to maintain the local geometrical structures due to the averaging process this paper proposes a novel approach named compressed submanifold multifactor analysis csma to solve the four problems mentioned above our approach can deal with the problem of missing values and outliers via svd l1 the random projection method is used to obtain the fast low rank approximation of a given multifactor dataset in addition it is able to preserve the geometry of the original data our csma method can be used efficiently for multiple purposes e g noise and outlier removal estimation of missing values biometric applications we show that csma method can achieve good results and is very efficient in the inpainting problem as compared to 1 2 our method also achieves higher face recognition rates compared to lrtc spma mpca and some other methods i e pca lda and lpp on three challenging face databases i e cmu mpie cmu pie and extended yale b

general subspace learning with corrupted training data via graph embedding we address the following subspace learning problem supposing we are given a set of labeled corrupted training data points how to learn the underlying subspace which contains three components an intrinsic subspace that captures certain desired properties of a data set a penalty subspace that fits the undesired properties of the data and an error container that models the gross corruptions possibly existing in the data given a set of data points these three components can be learned by solving a nuclear norm regularized optimization problem which is convex and can be efficiently solved in polynomial time using the method as a tool we propose a new discriminant analysis i e supervised subspace learning algorithm called corruptions tolerant discriminant analysis ctda in which the intrinsic subspace is used to capture the features with high within class similarity the penalty subspace takes the role of modeling the undesired features with high between class similarity and the error container takes charge of fitting the possible corruptions in the data we show that ctda can well handle the gross corruptions possibly existing in the training data whereas previous linear discriminant analysis algorithms arguably fail in such a setting extensive experiments conducted on two benchmark human face data sets and one object recognition data set show that ctda outperforms the related algorithms

strengthening the lovasz bound for graph coloring the lovasz number is a way to approximate the independence number of a graph but also its chromatic number we express the lovasz bound as the continuous relaxation of a discrete lovasz number which we derive from karger et al s formulation and which is equal to the chromatic number we also give another relaxation a la schrijver mceliece which is better than the lovasz number

the operator psi for the chromatic number of a graph we investigate hierarchies of semidefinite approximations for the chromatic number chi g of a graph g we introduce an operator psi mapping any graph parameter beta g nested between the stability number alpha g and chi overline g to a new graph parameter psi beta g nested between alpha overline g and chi g psi beta g is polynomial time computable if beta g is as an application there is no polynomial time computable graph parameter nested between the fractional chromatic number chi cdot and chi cdot unless text p text np moreover based on the motzkin straus formulation for alpha g we give quadratically constrained quadratic and copositive programming formulations for chi g under some mild assumptions n beta g le psi beta g but while n beta g remains below chi g psi beta g can reach chi g e g for beta cdot alpha cdot we also define new polynomial time computable lower bounds for chi g improving the classic lovasz theta number and its strengthenings obtained by adding nonnegativity and triangle inequalities experimental results on hamming graphs kneser graphs and dimacs benchmark graphs will be given in the follow up paper n gvozdenovic and m laurent siam j optim 19 2008 pp 592 615

the second critical exponent for a semilinear nonlocal parabolic equation abstract this article considers the cauchy problem for a semilinear nonlocal parabolic equation u t u r n k y u q y t d y p 1 q u r 1 in r n 0 t where p q 1 r 0 and p r 1 we study the second critical exponent i e describing the critical smallness of initial data required by global solutions non global solutions via the decay rates of the initial data at spatial infinity differently from other parabolic equations the second critical exponent is related to n in this problem when k l 1 r n

joint low rank and sparse principal feature coding for enhanced robust representation and visual classification recovering low rank and sparse subspaces jointly for enhanced robust representation and classification is discussed technically we first propose a transductive low rank and sparse principal feature coding lspfc formulation that decomposes given data into a component part that encodes low rank sparse principal features and a noise fitting error part to well handle the outside data we then present an inductive lspfc i lspfc i lspfc incorporates embedded low rank and sparse principal features by a projection into one problem for direct minimization so that the projection can effectively map both inside and outside data into the underlying subspaces to learn more powerful and informative features for representation to ensure that the learned features by i lspfc are optimal for classification we further combine the classification error with the feature coding error to form a unified model discriminative lspfc d lspfc to boost performance the model of d lspfc seamlessly integrates feature coding and discriminative classification so the representation and classification powers can be enhanced the proposed approaches are more general and several recent existing low rank or sparse coding algorithms can be embedded into our problems as special cases visual and numerical results demonstrate the effectiveness of our methods for representation and classification

double nuclear norm based matrix decomposition for occluded image recovery and background modeling robust principal component analysis rpca is a new emerging method for exact recovery of corrupted low rank matrices it assumes that the real data matrix has low rank and the error matrix is sparse this paper presents a method called double nuclear norm based matrix decomposition dnmd for dealing with the image data corrupted by continuous occlusion the method uses a unified low rank assumption to characterize the real image data and continuous occlusion specifically we assume all image vectors form a low rank matrix and each occlusion induced error image is a low rank matrix as well compared with rpca the low rank assumption of dnmd is more intuitive for describing occlusion moreover dnmd is solved by alternating direction method of multipliers our algorithm involves only one operator the singular value shrinkage operator dnmd as a transductive method is further extended into inductive dnmd idnmd both dnmd and idnmd use nuclear norm for measuring the continuous occlusion induced error while many previous methods use l 1 l 2 or other m estimators extensive experiments on removing occlusion from face images and background modeling from surveillance videos demonstrate the effectiveness of the proposed methods

robust bilateral lp norm two dimensional linear discriminant analysis abstract traditional linear discriminant analysis lda may suffer from a sensitivity to outliers and the small sample size sss problem while the lp norm measure for 0

asymptotic distributions of principal components based on robust dispersions algebraically principal components can be defined as the eigenvalues and eigenvectors of a covariance or correlation matrix but they are statistically meaningful as successive projections of the multivariate data in the direction of maximal variability an attractive alternative in robust principal component analysis is to replace the classical variability measure i e variance by a robust dispersion measure this projection pursuit approach was first proposed in li chen 1985 as a method of constructing a robust scatter matrix recent unpublished work of c croux and a ruiz gazen provided the influence functions of the resulting principal components the present paper focuses on the asymptotic distributions of robust principal components in particular we obtain the asymptotic normality of the principal components that maximise a robust dispersion measure we also explain the need to use a dispersion functional with a continuous influence function

semidefinite programming relaxations for graph coloring and maximal clique problems the semidefinite programming formulation of the lovasz theta number does not only give one of the best polynomial simultaneous bounds on the chromatic number g or the clique number g of a graph but also leads to heuristics for graph coloring and extracting large cliques this semidefinite programming formulation can be tightened toward either g or g by adding several types of cutting planes we explore several such strengthenings and show that some of them can be computed with the same effort as the theta number we also investigate computational simplifications for graphs with rich automorphism groups

projection pursuit approach to robust dispersion matrices and principal components primary theory and monte carlo abstract this article proposes and discusses a type of new robust estimators for covariance correlation matrices and principal components via projection pursuit techniques the most attractive advantage of the new procedures is that they are of both rotational equivariance and high breakdown point besides they are qualitatively robust and consistent at elliptic underlying distributions the monte carlo study shows that the best of the new estimators compare favorably with other robust methods they provide as good a performance as m estimators and somewhat better empirical breakdown properties

robust low rank approximation of data matrices with elementwise contamination we propose a robust method to approximate an n p data matrix with one of rank q the method is based on yohai s regression mm estimates it is intended to be resistant against the existence of both atypical rows and of scattered atypical cells and also to be able to cope with missing values we propose an algorithm based on alternating m regressions and a starting estimate based on successive rank one fits which involves o npq operations simulations show that our estimate outperforms competing estimates in terms of both efficiency and resistance three high dimensional real data sets are analyzed the running time of our estimate for large data sets is shown to be less than that of its competitors

robust singular value decompositions a new approach to projection pursuit abstract robust location and covariance estimators are developed via general m estimation for covariance matrix eigenvectors and eigenvalues the solution to this gm estimation problem is obtained by transforming it into a series of robust regression problems based on a new algorithm for the singular value decomposition it is shown here that the singular value decomposition can be represented as an iteration of two steps a least squares regression fit of the data matrix followed by a rotation to the regression hyperplanes an algorithm to obtain the solution to this gm estimation problem is presented along with results of a monte carlo study and examples of its application in addition it is shown how the output of this algorithm can be used to numerically search for multivariate outliers which is especially useful in exploratory data analysis with high dimensional data and large sample sizes where standard graphical techniques are difficult to implement because the algorithm computes robust estima

low rank matrix fitting based on subspace perturbation analysis with applications to structure from motion the task of finding a low rank r matrix that best fits an original data matrix of higher rank is a recurring problem in science and engineering the problem becomes especially difficult when the original data matrix has some missing entries and contains an unknown additive noise term in the remaining elements the former problem can be solved by concatenating a set of r column matrices that share a common single r dimensional solution space unfortunately the number of possible submatrices is generally very large and hence the results obtained with one set of r column matrices will generally be different from that captured by a different set ideally we would like to find that solution that is least affected by noise this requires that we determine which of the r column matrices i e which of the original feature points are less influenced by the unknown noise term this paper presents a criterion to successfully carry out such a selection our key result is to formally prove that the more distinct the r vectors of the r column matrices are the less they are swayed by noise this key result is then combined with the use of a noise model to derive an upper bound for the effect that noise and occlusions have on each of the r column matrices it is shown how this criterion can be effectively used to recover the noise free matrix of rank r finally we derive the affine and projective structure from motion sfm algorithms using the proposed criterion extensive validation on synthetic and real data sets shows the superiority of the proposed approach over the state of the art

robust projection indices loosely speaking a robust projection index is one that prefers projections involving true clusters over projections consisting of a cluster and an outlier we introduce a mathematical definition of one dimensional index robustness and describe a numerical experiment to measure it we design five new indices based on measuring divergence from student s t distribution which are intended to be especially robust the experiment shows that they are more robust than several established indices the experiment also reveals more generally that the robustness of moment indices depends on the number of approximation terms providing additional practical guidance for existing projection pursuit implementations we investigate the theoretical properties of one new student t index and hall s index and show that the new index automatically adapts its robustness to the degree of outlier contamination we conclude by outlining the possibilities for extending our experiments to both higher dimensions and other new indices

second critical exponent and life span for pseudo parabolic equation this paper studies the second critical exponent and life span of solutions for the pseudo parabolic equation ut k ut u up in rn 0 t with p 1 k 0 it is proved that the second critical exponent i e the decay order of the initial data required by global solutions in the coexistence region of global and non global solutions is independent of the pseudo parabolic parameter k nevertheless it is revealed that the viscous term k ut relaxes restrictions on the amplitude of the initial data required by the global solutions moreover it is observed that the life span of the non global solutions will be delayed by the third order viscous term finally some numerical examples are given to illustrate all these results

the np completeness of edge coloring we show that it is np complete to determine the chromatic index of an arbitrary graph the problem remains np complete even for cubic graphs

robust pca via outlier pursuit singular value decomposition svd and principal component analysis pca is one of the most widely used techniques for dimensionality reduction successful and efficiently computable it is nevertheless plagued by a well known well documented sensitivity to outliers recent work has considered the setting where each point has a few arbitrarily corrupted components yet in applications of svd or pca such as robust collaborative filtering or bioinformatics malicious agents defective genes or simply corrupted or contaminated experiments may effectively yield entire points that are completely corrupted we present an efficient convex optimization based algorithm that we call outlier pursuit which under some mild assumptions on the uncorrupted points satisfied e g by the standard generative assumption in pca problems recovers the exact optimal low dimensional subspace and identifies the corrupted points such identification of corrupted points that do not conform to the low dimensional approximation is of paramount interest in bioinformatics financial applications and beyond our techniques involve matrix decomposition using nuclear norm minimization however our results setup and approach necessarily differ considerably from the existing line of work in matrix completion and matrix decomposition since we develop an approach to recover the correct column space of the uncorrupted matrix rather than the exact matrix itself in any problem where one seeks to recover a structure rather than the exact initial matrices techniques developed thus far relying on certificates of optimality will fail we present an important extension of these methods which allows the treatment of such problems

approximate graph coloring by semidefinite programming we consider the problem of coloring k colorable graphs with the fewest possible colors we present a randomized polynomial time algorithm that colors a 3 colorable graph on n vertices with min o 1 3 log 1 2 log n o n 1 4 log 1 2 n colors where is the maximum degree of any vertex besides giving the best known approximation ratio in terms of n this marks the first nontrivial approximation result as a function of the maximum degree this result can be generalized to k colorable graphs to obtain a coloring using min o 1 2 k log 1 2 log n o n 1 3 k 1 log 1 2 n colors our results are inspired by the recent work of goemans and williamson who used an algorithm for semidefinite optimization problems which generalize linear programs to obtain improved approximations for the max cut and max 2 sat problems an intriguing outcome of our work is a duality relationship established between the value of the optimum solution to our semidefinite program and the lovasz t function we show lower bounds on the gap between the optimum solution of our semidefinite program and the actual chromatic number by duality this also demonstrates interesting new facts about the t function

principal components analysis based on multivariate mm estimators with fast and robust bootstrap we consider robust principal components analysis pca based on multivariate mm estimators we first study the robustness and efficiency of these estimators particularly in terms of eigenvalues and eigenvectors we then focus on inference procedures based on a fast and robust bootstrap for mm estimators this method is an alternative to the approach based on the asymptotic distribution of the estimators and can also be used to assess the stability of the principal components a formal consistency proof for the bootstrap method is given and its finite sample performance is investigated through simulations we illustrate the use of the robust pca and the bootstrap inference on a real dataset

weighted nuclear norm minimization and its applications to low level vision as a convex relaxation of the rank minimization model the nuclear norm minimization nnm problem has been attracting significant research interest in recent years the standard nnm regularizes each singular value equally composing an easily calculated convex norm however this restricts its capability and flexibility in dealing with many practical problems where the singular values have clear physical meanings and should be treated differently in this paper we study the weighted nuclear norm minimization wnnm problem which adaptively assigns weights on different singular values as the key step of solving general wnnm models the theoretical properties of the weighted nuclear norm proximal wnnp operator are investigated albeit nonconvex we prove that wnnp is equivalent to a standard quadratic programming problem with linear constrains which facilitates solving the original problem with off the shelf convex optimization solvers in particular when the weights are sorted in a non descending order its optimal solution can be easily obtained in closed form with wnnp the solving strategies for multiple extensions of wnnm including robust pca and matrix completion can be readily constructed under the alternating direction method of multipliers paradigm furthermore inspired by the reweighted sparse coding scheme we present an automatic weight setting method which greatly facilitates the practical implementation of wnnm the proposed wnnm methods achieve state of the art performance in typical low level vision tasks including image denoising background subtraction and image inpainting

computing semidefinite programming lower bounds for the fractional chromatic number via block diagonalization recently we investigated in siam j optim 19 2008 pp 572 591 hierarchies of semidefinite approximations for the chromatic number chi g of a graph g in particular we introduced two hierarchies of lower bounds the psi hierarchy converging to the fractional chromatic number and the psi hierarchy converging to the chromatic number of a graph in both hierarchies the first order bounds are related to the lovasz theta number while the second order bounds would already be too costly to compute for large graphs as an alternative relaxations of the second order bounds are proposed in siam j optim 19 2008 pp 572 591 we present here our experimental results with these relaxed bounds for hamming graphs kneser graphs and dimacs benchmark graphs symmetry reduction plays a crucial role as it permits us to compute the bounds by using more compact semidefinite programs in particular for hamming and kneser graphs we use the explicit block diagonalization of the terwilliger algebra given by schrijver ieee trans inform theory 51 2005 pp 2859 2866 our numerical results indicate that the new bounds can be much stronger than the lovasz theta number for some of the dimacs instances we improve the best known lower bounds significantly

robust sparse principal component analysis a method for principal component analysis is proposed that is sparse and robust at the same time the sparsity delivers principal components that have loadings on a small number of variables making them easier to interpret the robustness makes the analysis resistant to outlying observations the principal components correspond to directions that maximize a robust measure of the variance with an additional penalty term to take sparseness into account we propose an algorithm to compute the sparse and robust principal components the algorithm computes the components sequentially and thus it can handle datasets with more variables than observations the method is applied on several real data examples and diagnostic plots for detecting outliers and for selecting the degree of sparsity are provided a simulation experiment studies the effect on statistical efficiency by requiring both robustness and sparsity supplementary materials are available online on the journal web site

chromatic gallai identities operating on lovasz number if g is a triangle free graph then two gallai identities can be written as alpha g overline chi l g v g alpha l g overline chi g where alpha and overline chi denote the stability number and the clique partition number and l g is the line graph of g we show that surprisingly both equalities can be preserved for any graph g by deleting the edges of the line graph corresponding to simplicial pairs of adjacent arcs according to any acyclic orientation of g as a consequence one obtains an operator phi which associates to any graph parameter beta such that alpha g le beta g le overline chi g for all graph g a graph parameter phi beta such that alpha g le phi beta g le overline chi g for all graph g we prove that vartheta g le phi vartheta g and that phi overline chi f g le overline chi f g for all graph g where vartheta is lovasz theta function and overline chi f is the fractional clique partition number moreover overline chi f g le phi vartheta g for triangle free g comparing to the previous strengthenings psi vartheta and vartheta triangle of vartheta numerical experiments show that phi vartheta is a significant better lower bound for overline chi than vartheta

the nearest subclass classifier a compromise between the nearest mean and nearest neighbor classifier we present the nearest subclass classifier nsc which is a classification algorithm that unifies the flexibility of the nearest neighbor classifier with the robustness of the nearest mean classifier the algorithm is based on the maximum variance cluster algorithm and as such it belongs to the class of prototype based classifiers the variance constraint parameter of the cluster algorithm serves to regularize the classifier that is to prevent overfitting with a low variance constraint value the classifier turns into the nearest neighbor classifier and with a high variance parameter it becomes the nearest mean classifier with the respective properties in other words the number of prototypes ranges from the whole training set to only one per class in the experiments we compared the nsc with regard to its performance and data set compression ratio to several other prototype based methods on several data sets the nsc performed similarly to the k nearest neighbor classifier which is a well established classifier in many domains also concerning storage requirements and classification speed the nsc has favorable properties so it gives a good compromise between classification performance and efficiency

rank regularized estimation of approximate factor models abstract it is known that the common factors in a large panel of data can be consistently estimated by the method of principal components and principal components can be constructed by iterative least squares regressions replacing least squares with ridge regressions turns out to have the effect of removing the contribution of factors associated with small singular values from the common component the method has been used in the machine learning literature to recover low rank matrices we study the procedure from the perspective of estimating an approximate factor model under the rank constraint the common component is estimated by the space spanned by factors whose singular values exceed a threshold the desire for minimum rank and parsimony lead to a data dependent penalty for selecting the number of factors the new criterion is more conservative than the existing deterministic penalties and is appropriate when the nominal number of factors is inflated by the presence of weak factors or large measurement noise we provide asymptotic results that can be used to test economic hypotheses

the percentage bend correlation coefficient a well known result is that the usual correlation coefficient is highly nonrobust very slight changes in only one of the marginal distributions can alter by a substantial amount there are a variety of methods for correcting this problem this paper identifies one particular method which is useful in psychometrics and provides a simple test for independence it is not recommended that the new test replace the usual test ofh 0 0 but the new test has important advantages over the usual test in terms of both type i errors and power

robust estimation of dispersion matrices and principal components abstract this paper uses monte carlo methods to compare the performances of several robust procedures for estimating a correlation matrix and its principal components the estimators are formed either from separate bivariate analyses or by simultaneous manipulation of all variables by using techniques such as multivariate trimming and m estimation the m estimators stand up exceptionally well they and the multivariate trimming procedure are especially effective at estimating the principal components including a near singularity however the m estimators can break down relatively easily when the dimensionality is large and the outliers are asymmetric with missing data the element wise approach becomes more attractive

robust estimation in the errors in variables model summary orthogonal regression analogues of m estimates of regression called here orthogonal regression m estimates are defined these estimates are shown to be consistent at elliptical errors in variables models and robust if the corresponding loss function is bounded the orthogonal regression analogues of regression scale estimates called here orthogonal regression s estimates are considered as well in particular they provide a robust estimate for the scale of the orthogonal residuals a crucial quantity in the computation of orthogonal regression m estimates finally we present an algorithm for computing orthogonal regression s and m estimates and the results of a small monte carlo experiment

linear dimensionality reduction via a heteroscedastic extension of lda the chernoff criterion we propose an eigenvector based heteroscedastic linear dimension reduction ldr technique for multiclass data the technique is based on a heteroscedastic two class technique which utilizes the so called chernoff criterion and successfully extends the well known linear discriminant analysis lda the latter which is based on the fisher criterion is incapable of dealing with heteroscedastic data in a proper way for the two class case the between class scatter is generalized so to capture differences in co variances it is shown that the classical notion of between class scatter can be associated with euclidean distances between class means from this viewpoint the between class scatter is generalized by employing the chernoff distance measure leading to our proposed heteroscedastic measure finally using the results from the two class case a multiclass extension of the chernoff criterion is proposed this criterion combines separation information present in the class mean as well as the class covariance matrices extensive experiments and a comparison with similar dimension reduction techniques are presented

s estimators for functional principal component analysis principal component analysis is a widely used technique that provides an optimal lower dimensional approximation to multivariate or functional datasets these approximations can be very useful in identifying potential outliers among high dimensional or functional observations in this article we propose a new class of estimators for principal components based on robust scale estimators for a fixed dimension q we robustly estimate the q dimensional linear space that provides the best prediction for the data in the sense of minimizing the sum of robust scale estimators of the coordinates of the residuals we also study an extension to the infinite dimensional case our method is consistent for elliptical random vectors and is fisher consistent for elliptically distributed random elements on arbitrary hilbert spaces numerical experiments show that our proposal is highly competitive when compared with other methods we illustrate our approach on a real dataset where the robust estimator discovers atypi

on the rotational invariant l1 norm pca abstract principal component analysis pca is a powerful tool for dimensionality reduction unfortunately it is sensitive to outliers so that various robust pca variants were proposed in the literature one of the most frequently applied methods for high dimensional data reduction is the rotational invariant l 1 norm pca of ding and coworkers so far no convergence proof for this algorithm was available the main topic of this paper is to fill this gap we reinterpret this robust approach as a conditional gradient algorithm and show moreover that it coincides with a gradient descent algorithm on grassmann manifolds based on the latter point of view we prove global convergence of the whole series of iterates to a critical point using the kurdyka ojasiewicz property of the objective function where we have to pay special attention to so called anchor points where the function is not differentiable

global existence large time behavior and life span of solutions of a semilinear parabolic cauchy problem we investigate the behavior of the solution u x t of where 1 1 n 2 xi 2 is the laplace operator p 1 is a constant t 0 and is a nonnegative bounded continuous function in r n the main results are for the case when the initial value has polynomial decay near x assuming 1 x 75a with a 0 various questions of global in time existence and nonexistence large time behavior or life span of the solution u x t are answered in terms of simple conditions on a p and the space dimension n

robust elastic net subspace representation recently finding the low dimensional structure of high dimensional data has gained much attention given a set of data points sampled from a single subspace or a union of subspaces the goal is to learn or capture the underlying subspace structure of the data set in this paper we propose elastic net subspace representation a new subspace representation framework using elastic net regularization of singular values due to the strong convexity enforced by elastic net the proposed method is more stable and robust in the presence of heavy corruptions compared with existing lasso type rank minimization approaches for discovering a single low dimensional subspace we propose a computationally efficient low rank factorization algorithm called facten using a property of the nuclear norm and the augmented lagrangian method then clusten is proposed to handle the general case in which the data samples are drawn from a union of multiple subspaces for joint subspace clustering and estimation the proposed algorithms are applied to a number of subspace representation problems to evaluate the robustness and efficiency under various noisy conditions and experimental results show the benefits of the proposed method compared with existing methods

on principal hessian directions for data visualization and dimension reduction another application of stein s lemma abstract modern graphical tools have enhanced our ability to learn many things from data directly with much user friendly graphical software available we are encouraged to plot a lot more often than before the benefits from direct interaction with graphics have been enormous but trailing behind these high tech advances is the issue of appropriate guidance on what to plot there are too many directions to project a high dimensional data set and unguided plotting can be time consuming and fruitless in a recent article li set up a statistical framework for study on this issue based on a notion of effective dimension reduction edr directions they are the directions to project a high dimensional input variable for the purpose of effectively viewing and studying its relationship with an output variable a methodology sliced inverse regression was introduced and shown to be useful in finding edr directions this article introduces another method for finding edr directions it begins with the observat

outlier detection and robust normal curvature estimation in mobile laser scanning 3d point cloud data this paper proposes two robust statistical techniques for outlier detection and robust saliency features such as surface normal and curvature estimation in laser scanning 3d point cloud data one is based on a robust z score and the other uses a mahalanobis type robust distance the methods couple the ideas of point to plane orthogonal distance and local surface point consistency to get maximum consistency with minimum distance mcmd the methods estimate the best fit plane based on most probable outlier free and most consistent points set in a local neighbourhood then the normal and curvature from the best fit plane will be highly robust to noise and outliers experiments are performed to show the performance of the algorithms compared to several existing well known methods from computer vision data mining machine learning and statistics using synthetic and real laser scanning datasets of complex planar and non planar objects results for plane fitting denoising sharp feature preserving and segmentation are significantly improved the algorithms are demonstrated to be significantly faster more accurate and robust quantitatively for a sample size of 50 with 20 outliers the proposed mcmd z is approximately 5 15 and 98 times faster than the existing methods ulsif ransac and rpca respectively the proposed mcmd md method can tolerate 75 clustered outliers whereas rpca and ransac can only tolerate 47 and 64 outliers respectively in terms of outlier detection for the same dataset mcmd z has an accuracy of 99 72 0 4 false positive rate and 0 false negative rate for rpca ransac and ulsif the accuracies are 97 05 47 06 and 94 54 respectively and they have misclassification rates higher than the proposed methods the new methods have potential for local surface reconstruction fitting and other point cloud processing tasks highlightstwo statistical techniques are proposed for outlier detection in point cloud data the proposed methods can fit robust plane in laser scanning data the proposed methods produce robust normal and curvature in point cloud processing they are faster and robust than ransac robust pca and other existing efficient methods they have potential for point cloud denoising segmentation and reconstruction