information theoretic subspace clustering this paper addresses the problem of grouping the data points sampled from a union of multiple subspaces in the presence of outliers information theoretic objective functions are proposed to combine structured low rank representations lrrs to capture the global structure of data and information theoretic measures to handle outliers in theoretical part we point out that group sparsity induced measures ell 2 1 norm ell alpha norm and correntropy can be justified from the viewpoint of half quadratic hq optimization which facilitates both convergence study and algorithmic development in particular a general formulation is accordingly proposed to unify hq based group sparsity methods into a common framework in algorithmic part we develop information theoretic subspace clustering methods via correntropy with the help of parzen window estimation correntropy is used to handle either outliers under any distributions or sample specific errors in data pairwise link constraints are further treated as a prior structure of lrrs based on the hq framework iterative algorithms are developed to solve the nonconvex information theoretic loss functions experimental results on three benchmark databases show that our methods can further improve the robustness of lrr subspace clustering and outperform other state of the art subspace clustering methods

learning graphs with monotone topology properties and multiple connected components recent papers have formulated the problem of learning graphs from data as an inverse covariance estimation problem with graph laplacian constraints while such problems are convex existing methods cannot guarantee that solutions will have specific graph topology properties e g being a tree which are desirable for some applications the problem of learning a graph with topology properties is in general non convex in this paper we propose an approach to solve these problems by decomposing them into two sub problems for which efficient solutions are known specifically a graph topology inference gti step is employed to select a feasible graph topology then a graph weight estimation gwe step is performed by solving a generalized graph laplacian estimation problem where edges are constrained by the topology found in the gti step our main result is a bound on the error of the gwe step as a function of the error in the gti step this error bound indicates that the gti step should be solved using an algorithm that approximates the data similarity matrix by another matrix whose entries have been thresholded to zero to have the desired type of graph topology the gti stage can leverage existing methods which are typically based on minimizing the total weight of removed edges since the gwe stage is an inverse covariance estimation problem with linear constraints it can be solved using existing convex optimization methods we demonstrate that our approach can achieve good results for both synthetic and texture image data

multibody grouping from motion images we want to deduce from a sequence of noisy two dimensional images of a scene of several rigid bodies moving independently in three dimensions the number of bodies and the grouping of given feature points in the images to the bodies prior processing is assumed to have identified features or points common to all frames and the images are assumed to be created by orthographic projection i e perspective effects are minimal we describe a computationally inexpensive algorithm that can determine which points or features belong to which rigid body using the fact that with exact observations in orthographic projection points on a single body lie in a three or less dimensional linear manifold of frame space if there are enough observations and independent motions these manifolds can be viewed as a set linearly independent four or less dimensional subspaces we show that the row echelon canonical form provides direct information on the grouping of points to these subspaces treatment of the noise is the most difficult part of the problem this paper uses a statistical approach to estimate the grouping of points to subspaces in the presence of noise by computing which partition has the maximum likelihood the input data is assumed to be contaminated with independent gaussian noise the algorithm can base its estimates on a user supplied standard deviation of the noise or it can estimate the noise from the data the algorithm can also be used to estimate the probability of a user specified partition so that the hypothesis can be combined with others using bayesian statistics

a multibody factorization method for independently moving objects the structure from motion problem has been extensively studied in the field of computer vision yet the bulk of the existing work assumes that the scene contains only a single moving object the more realistic case where an unknown number of objects move in the scene has received little attention especially for its theoretical treatment in this paper we present a new method for separating and recovering the motion and shape of multiple independently moving objects in a sequence of images the method does not require prior knowledge of the number of objects nor is dependent on any grouping of features into an object at the image level for this purpose we introduce a mathematical construct of object shapes called the shape interaction matrix which is invariant to both the object motions and the selection of coordinate systems this invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects once the matrix is computed it allows for segmenting features into objects by the process of transforming it into a canonical form as well as recovering the shape and motion of each object the theory works under a broad set of projection models scaled orthography paraperspective and affine but they must be linear so it excludes projective cameras

the lagrange multiplier rule for multifunctions in banach spaces we study general constrained multiobjective optimization problems with objectives being closed multifunctions in banach spaces in terms of the coderivatives and normal cones we provide generalized lagrange multiplier rules as necessary optimality conditions of the above problems in an asplund space setting sharper results are presented

spatio temporal union of subspaces for multi body non rigid structure from motion abstract non rigid structure from motion nrsfm has so far been mostly studied for recovering 3d structure of a single non rigid deforming object to handle the real world challenging multiple deforming objects scenarios existing methods either pre segment different objects in the scene or treat multiple non rigid objects as a whole to obtain the 3d non rigid reconstruction however these methods fail to exploit the inherent structure in the problem as the solution of segmentation and the solution of reconstruction could not benefit each other in this paper we propose a unified framework to jointly segment and reconstruct multiple non rigid objects to compactly represent complex multi body non rigid scenes we propose to exploit the structure of the scenes along both temporal direction and spatial direction thus achieving a spatio temporal representation specifically we represent the 3d non rigid deformations as lying in a union of subspaces along the temporal direction and represent the 3d trajectories as lying in the union of subspaces along the spatial direction this spatio temporal representation not only provides competitive 3d reconstruction but also outputs robust segmentation of multiple non rigid objects the resultant optimization problem is solved efficiently using the alternating direction method of multipliers admm extensive experimental results on both synthetic and real multi body nrsfm datasets demonstrate the superior performance of our proposed framework compared with the state of the art methods

linear regression for face recognition in this paper we present a novel approach of face identification by formulating the pattern recognition problem in terms of linear regression using a fundamental concept that patterns from a single object class lie on a linear subspace we develop a linear model representing a probe image as a linear combination of class specific galleries the inverse problem is solved using the least squares method and the decision is ruled in favor of the class with the minimum reconstruction error the proposed linear regression classification lrc algorithm falls in the category of nearest subspace classification the algorithm is extensively evaluated on several standard databases under a number of exemplary evaluation protocols reported in the face recognition literature a comparative study with state of the art algorithms clearly reflects the efficacy of the proposed approach for the problem of contiguous occlusion we propose a modular lrc approach introducing a novel distance based evidence fusion def algorithm the proposed methodology achieves the best results ever reported for the challenging problem of scarf occlusion

an elastic viscoplastic quasistatic contact problem with damage abstract in this paper the contact between an elastic viscoplastic body and a deformable obstacle is studied the effect of the damage due to tension or compression is also considered the variational formulation leads to a coupled system of evolutionary equations for which an existence and uniqueness result is stated then a fully discrete scheme is introduced using the finite element method to approximate the spatial variable and an euler scheme to discretize the time derivatives error estimates are obtained and as a consequence the convergence of the algorithm is deduced finally some numerical simulations are presented to demonstrate the accuracy of the algorithm

a regularized approach for geodesic based semisupervised multimanifold learning geodesic distance as an essential measurement for data dissimilarity has been successfully used in manifold learning however most geodesic distance based manifold learning algorithms have two limitations when applied to classification 1 class information is rarely used in computing the geodesic distances between data points on manifolds and 2 little attention has been paid to building an explicit dimension reduction mapping for extracting the discriminative information hidden in the geodesic distances in this paper we regard geodesic distance as a kind of kernel which maps data from linearly inseparable space to linear separable distance space in doing this a new semisupervised manifold learning algorithm namely regularized geodesic feature learning algorithm is proposed the method consists of three techniques a semisupervised graph construction method replacement of original data points with feature vectors which are built by geodesic distances and a new semisupervised dimension reduction method for feature vectors experiments on the mnist usps handwritten digit data sets mit cbcl face versus nonface data set and an intelligent traffic data set show the effectiveness of the proposed algorithm

discriminative elastic net regularized linear regression in this paper we aim at learning compact and discriminative linear regression models linear regression has been widely used in different problems however most of the existing linear regression methods exploit the conventional zero one matrix as the regression targets which greatly narrows the flexibility of the regression model another major limitation of these methods is that the learned projection matrix fails to precisely project the image features to the target space due to their weak discriminative capability to this end we present an elastic net regularized linear regression enlr framework and develop two robust linear regression models which possess the following special characteristics first our methods exploit two particular strategies to enlarge the margins of different classes by relaxing the strict binary targets into a more feasible variable matrix second a robust elastic net regularization of singular values is introduced to enhance the compactness and effectiveness of the learned projection matrix third the resulting optimization problem of enlr has a closed form solution in each iteration which can be solved efficiently finally rather than directly exploiting the projection matrix for recognition our methods employ the transformed features as the new discriminate representations to make final image classification compared with the traditional linear regression model and some of its variants our method is much more accurate in image classification extensive experiments conducted on publicly available data sets well demonstrate that the proposed framework can outperform the state of the art methods the matlab codes of our methods can be available at http www yongxu org lunwen html

estimation of subspace arrangements with applications in modeling and segmenting mixed data recently many scientific and engineering applications have involved the challenging task of analyzing large amounts of unsorted high dimensional data that have very complicated structures from both geometric and statistical points of view such unsorted data are considered mixed as different parts of the data have significantly different structures which cannot be described by a single model in this paper we propose to use subspace arrangements a union of multiple subspaces for modeling mixed data each subspace in the arrangement is used to model just a homogeneous subset of the data thus multiple subspaces together can capture the heterogeneous structures within the data set in this paper we give a comprehensive introduction to a new approach for the estimation of subspace arrangements this is known as generalized principal component analysis gpca in particular we provide a comprehensive summary of important algebraic properties and statistical facts that are crucial for making the inference of subspace arrangements both efficient and robust even when the given data are corrupted by noise or contaminated with outliers this new method in many ways improves and generalizes extant methods for modeling or clustering mixed data there have been successful applications of this new method to many real world problems in computer vision image processing and system identification in this paper we will examine several of those representative applications this paper is intended to be expository in nature however in order that this may serve as a more complete reference for both theoreticians and practitioners we take the liberty of filling in several gaps between the theory and the practice in the existing literature

self regularized fixed rank representation for subspace segmentation abstract consider a set of data points generated from various linear subspaces subspace segmentation tasks which are important in fields such as computer vision and image processing aim to partition the set of data points so as to recover these subspaces the subspace segmentation method fixed rank representation frr was introduced to remedy the problem of insufficient sampling in classical low rank representation lrr in many subspace segmentation applications frr has achieved much better results than those of lrr in this paper a new frr related algorithm called self regularized fixed rank representation srfrr is proposed in srfrr a laplacian regularizer is constructed using the coefficient matrix obtained by srfrr itself further by proving that the laplacian regularizer can be transformed into a structure constraint on the coefficient matrix we show that another existing method sparse frr sfrr is a special case of srfrr to implement the srfrr method we present two optimization algorithms experiments on both synthetic and real databases show that srfrr outperforms some existing frr and lrr related algorithms

on the approximability of minimizing nonzero variables or unsatisfied relations in linear systems we investigate the computational complexity of two closely related classes of combinatorial optimization problems for linear systems which arise in various fields such as machine learning operations research and pattern recognition in the first class min ulr one wishes given a possibly infeasible system of linear relations to find a solution that violates as few relations as possible while satisfying all the others in the second class min rvls the linear system is supposed to be feasible and one looks for a solution with as few nonzero variables as possible for both min ulr and min rvls the four basic types of relational operators and are considered while min rvls with equations was mentioned to be np hard in garey and johnson 1979 we established in amaldi 1992 amaldi and kann 1995 that min ulr with equalities and inequalities are np hard even when restricted to homogeneous systems with bipolar coefficients the latter problems have been shown hard to approximate in arora et al 1993 in this paper we determine strong bounds on the approximability of various variants of min rvls and min ulr including constrained ones where the variables are restricted to take binary values or where some relations are mandatory while others are optional the various np hard versions turn out to have different approximability properties depending on the type of relations and the additional constraints but none of them can be approximated within any constant factor unless p np particular attention is devoted to two interesting special cases that occur in discriminant analysis and machine learning in particular we disprove a conjecture of van horn and martinez 1992 regarding the existence of a polynomial time algorithm to design linear classifiers or perceptrons that involve a close to minimum number of features

subspace learning by l0 induced sparsity subspace clustering methods partition the data that lie in or close to a union of subspaces in accordance with the subspace structure such methods with sparsity prior such as sparse subspace clustering ssc elhamifar and vidal in ieee trans pattern anal mach intell 35 11 2765 2781 2013 with the sparsity induced by the ell 1 norm are demonstrated to be effective in subspace clustering most of those methods require certain assumptions e g independence or disjointness on the subspaces however these assumptions are not guaranteed to hold in practice and they limit the application of existing sparse subspace clustering methods in this paper we propose ell 0 induced sparse subspace clustering ell 0 ssc in contrast to the required assumptions such as independence or disjointness on subspaces for most existing sparse subspace clustering methods we prove that ell 0 ssc guarantees the subspace sparse representation a key element in subspace clustering for arbitrary distinct underlying subspaces almost surely under the mild i i d assumption on the data generation we also present the no free lunch theorem which shows that obtaining the subspace representation under our general assumptions can not be much computationally cheaper than solving the corresponding ell 0 sparse representation problem of ell 0 ssc a novel approximate algorithm named approximate ell 0 ssc a ell 0 ssc is developed which employs proximal gradient descent to obtain a sub optimal solution to the optimization problem of ell 0 ssc with theoretical guarantee the sub optimal solution is used to build a sparse similarity matrix upon which spectral clustering is performed for the final clustering results extensive experimental results on various data sets demonstrate the superiority of a ell 0 ssc compared to other competing clustering methods furthermore we extend ell 0 ssc to semi supervised learning by performing label propagation on the sparse similarity matrix learnt by a ell 0 ssc and demonstrate the effectiveness of the resultant semi supervised learning method termed ell 0 sparse subspace label propagation ell 0 sslp

relative pareto minimizers for multiobjective problems existence and optimality conditions in this paper we introduce and study enhanced notions of relative pareto minimizers for constrained multiobjective problems that are defined via several kinds of relative interiors of ordering cones and occupy intermediate positions between the classical notions of pareto and weak pareto efficiency minimality using advanced tools of variational analysis and generalized differentiation we establish the existence of relative pareto minimizers for general multiobjective problems under a refined version of the subdifferential palais smale condition for set valued mappings with values in partially ordered spaces and then derive necessary optimality conditions for these minimizers as well as for conventional efficient and weak efficient counterparts that are new in both finite dimensional and infinite dimensional settings our proofs are based on variational and extremal principles of variational analysis in particular on new versions of the ekeland variational principle and the subdifferential variational principle for set valued and single valued mappings in infinite dimensional spaces

motion segmentation in the presence of outlying incomplete or corrupted trajectories in this paper we study the problem of segmenting tracked feature point trajectories of multiple moving objects in an image sequence using the affine camera model this problem can be cast as the problem of segmenting samples drawn from multiple linear subspaces in practice due to limitations of the tracker occlusions and the presence of nonrigid objects in the scene the obtained motion trajectories may contain grossly mistracked features missing entries or corrupted entries in this paper we develop a robust subspace separation scheme that deals with these practical issues in a unified mathematical framework our methods draw strong connections between lossy compression rank minimization and sparse representation we test our methods extensively on the hopkins155 motion segmentation database and other motion sequences with outliers and missing data we compare the performance of our methods to state of the art motion segmentation methods based on expectation maximization and spectral clustering for data without outliers or missing information the results of our methods are on par with the state of the art results and in many cases exceed them in addition our methods give surprisingly good performance in the presence of the three types of pathological trajectories mentioned above all code and results are publicly available at http perception csl uiuc edu coding motion

recovery guarantees for exemplar based clustering for a certain class of distributions we prove that the linear programming relaxation of k medoids clustering a variant of k means clustering where means are replaced by exemplars from within the dataset distinguishes points drawn from nonoverlapping balls with high probability once the number of points drawn and the separation distance between any two balls are sufficiently large our results hold in the nontrivial regime where the separation distance is small enough that points drawn from different balls may be closer to each other than points drawn from the same ball in this case clustering by thresholding pairwise distances between points can fail we also exhibit numerical evidence of high probability recovery in a substantially more permissive regime

adaptive graph construction using data self representativeness for pattern classification graph construction from data constitutes a pre stage in many machine learning and computer vision tasks like semi supervised learning manifold learning and spectral clustering the influence of graph construction procedures on learning tasks and their related applications has only received limited study despite its critical impact on accuracy state of the art graphs are built via sparse coding adopting 1 regularization those graphs exhibit good performance in many computer vision applications however the locality and similarity among instances are not explicitly used in the coding scheme furthermore due to the use of 1 regularization these construction approaches can be computationally expensive in this paper we investigate graph construction using the data self representativeness property by incorporating a variant of locality constrained linear coding llc we introduce and derive four variants for graph construction these variants adopt a two phase llc tpllc compared with the recent 1 graphs our proposed objective function associated with three variants has an analytical solution and thus is more efficient a key element of the proposed methods is the second phase of coding that allows data closeness or locality to be naturally incorporated it performs a coding over some selected relevant samples and reinforces the individual regularization terms by exploiting the coefficients estimated in the first phase comprehensive experimental results using several benchmark datasets show that it can achieve or outperform existing state of the art results furthermore it is shown to be more efficient than the robust 1 graph construction schemes

lambertian reflectance and linear subspaces we prove that the set of all lambertian reflectance functions the mapping from surface normals to intensities obtained with arbitrary distant light sources lies close to a 9d linear subspace this implies that in general the set of images of a convex lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low dimensional linear subspace explaining prior empirical results we also provide a simple analytic characterization of this linear space we obtain these results by representing lighting using spherical harmonics and describing the effects of lambertian materials as the analog of a convolution these results allow us to construct algorithms for object recognition based on linear methods as well as algorithms that use convex optimization to enforce nonnegative lighting functions we also show a simple way to enforce nonnegative lighting when the images of an object lie near a 4d linear space we apply these algorithms to perform face recognition by finding the 3d model that best matches a 2d query image

robust subspace clustering via thresholding the problem of clustering noisy and incompletely observed high dimensional data points into a union of low dimensional subspaces and a set of outliers is considered the number of subspaces their dimensions and their orientations are assumed unknown we propose a simple low complexity subspace clustering algorithm which applies spectral clustering to an adjacency matrix obtained by thresholding the correlations between data points in other words the adjacency matrix is constructed from the nearest neighbors of each data point in spherical distance a statistical performance analysis shows that the algorithm exhibits robustness to additive noise and succeeds even when the subspaces intersect specifically our results reveal an explicit tradeoff between the affinity of the subspaces and the tolerable noise level we furthermore prove that the algorithm succeeds even when the data points are incompletely observed with the number of missing entries allowed to be up to a log factor linear in the ambient dimension we also propose a simple scheme that provably detects outliers and we present numerical results on real and synthetic data

a geometric analysis of subspace clustering with outliers this paper considers the problem of clustering a collection of unlabeled data points assumed to lie near a union of lower dimensional planes as is common in computer vision or unsupervised learning applications we do not know in advance how many subspaces there are nor do we have any information about their dimensions we develop a novel geometric analysis of an algorithm named sparse subspace clustering ssc 11 which signicantly broadens the range of problems where it is provably eective for instance we show that ssc can recover multiple subspaces each of dimension comparable to the ambient dimension we also prove that ssc can correctly cluster data points even when the subspaces of interest intersect further we develop an extension of ssc that succeeds when the data set is corrupted with possibly overwhelmingly many outliers underlying our analysis are clear geometric insights which may bear on other sparse recovery problems a numerical study complements our theoretical analysis and demonstrates the eectiveness of these methods

matrix completion by least square low rank and sparse self representations abstract conventional matrix completion methods are generally based on rank minimization these methods assume that the given matrix is of low rank and the data points are drawn from a single subspace of low dimensionality therefore they are not effective in completing matrices where the data are drawn from multiple subspaces in this paper we establish a novel matrix completion framework that is based on self representation specifically least square low rank and sparse self representations based matrix completion algorithms are provided the underlying idea is that one data point can be efficiently reconstructed by other data points belonging to a common subspace where the missing entries are recovered so as to fit the common subspace the proposed algorithms actually maximize the weighted correlations among the columns of a given matrix we prove that the proposed algorithms are approximations for rank minimization of the incomplete matrix in addition they are able to complete high rank or even full rank matrices when the data are drawn from multiple subspaces comparative studies are conducted on synthetic datasets natural image inpainting tasks temperature prediction task and collaborative filtering tasks the results show that the proposed algorithms often outperform other state of the art algorithms in various tasks

a frictionless contact problem for elastic viscoplastic materials with normal compliance numerical analysis and computational experiments in this paper we consider a frictionless contact problem between an elastic viscoplastic body and an obstacle the process is assumed to be quasistatic and the contact is modeled with normal compliance we present a variational formulation of the problem and prove the existence and uniqueness of the weak solution using strongly monotone operators arguments and banach s fixed point theorem we also study the numerical approach to the problem using spatially semi discrete and fully discrete finite elements schemes with implicit and explicit discretization in time we show the existence of the unique solution for each of the schemes and derive error estimates on the approximate solutions finally we present some numerical results involving examples in one two and three dimensions

numerical analysis of two frictionless elastic piezoelectric contact problems in this work we consider two frictionless contact problems between an elastic piezoelectric body and an obstacle the linear elastic piezoelectric constitutive law is employed to model the piezoelectric material and either the signorini condition if the obstacle is rigid or the normal compliance condition if the obstacle is deformable are used to model the contact the variational formulations are derived in a form of a coupled system for the displacement and electric potential fields an existence and uniqueness result is recalled then a discrete scheme is introduced based on the finite element method to approximate the spatial variable error estimates are derived on the approximate solutions and as a consequence the linear convergence of the algorithm is deduced under suitable regularity conditions finally some two dimensional examples are presented to demonstrate the performance of the algorithm

enhancing low rank subspace clustering by manifold regularization recently low rank representation lrr method has achieved great success in subspace clustering sc which aims to cluster the data points that lie in a union of low dimensional subspace given a set of data points lrr seeks the lowest rank representation among the many possible linear combinations of the bases in a given dictionary or in terms of the data itself however lrr only considers the global euclidean structure while the local manifold structure which is often important for many real applications is ignored in this paper to exploit the local manifold structure of the data a manifold regularization characterized by a laplacian graph has been incorporated into lrr leading to our proposed laplacian regularized lrr laplrr an efficient optimization procedure which is based on alternating direction method of multipliers admm is developed for laplrr experimental results on synthetic and real data sets are presented to demonstrate that the performance of lrr has been enhanced by using the manifold regularization

transformation invariant subspace clustering subspace clustering has achieved great success in many computer vision applications however most subspace clustering algorithms require well aligned data samples which is often not straightforward to achieve this paper proposes a transformation invariant subspace clustering framework by jointly aligning data samples and learning subspace representation by alignment the transformed data samples become highly correlated and a better affinity matrix can be obtained the joint problem can be reduced to a sequence of least squares regression problems which can be efficiently solved we verify the effectiveness of the proposed method with extensive experiments on unaligned real data demonstrating its higher clustering accuracy than the state of the art subspace clustering and transformation invariant clustering algorithms highlightsa transformation invariant subspace clustering algorithm is proposed alignment and subspace clustering are mutually dependent state of the art clustering results on the unconstrained face dataset

convex sparse spectral clustering single view to multi view spectral clustering sc is one of the most widely used methods for data clustering it first finds a low dimensional embedding u of data by computing the eigenvectors of the normalized laplacian matrix and then performs k means on text u top to get the final clustering result in this paper we observe that in the ideal case text u text u top should be block diagonal and thus sparse therefore we propose the sparse sc ssc method that extends the sc with sparse regularization on text u text u top to address the computational issue of the nonconvex ssc model we propose a novel convex relaxation of ssc based on the convex hull of the fixed rank projection matrices then the convex ssc model can be efficiently solved by the alternating direction method of multipliers furthermore we propose the pairwise ssc that extends ssc to boost the clustering performance by using the multi view information of data experimental comparisons with several baselines on real world datasets testify to the efficacy of our proposed methods

block sparse recovery via convex optimization given a dictionary that consists of multiple blocks and a signal that lives in the range space of only a few blocks we study the problem of finding a block sparse representation of the signal i e a representation that uses the minimum number of blocks motivated by signal image processing and computer vision applications such as face recognition we consider the block sparse recovery problem in the case where the number of atoms in each block is arbitrary possibly much larger than the dimension of the underlying subspace to find a block sparse representation of a signal we propose two classes of nonconvex optimization programs which aim to minimize the number of nonzero coefficient blocks and the number of nonzero reconstructed vectors from the blocks respectively since both classes of problems are np hard we propose convex relaxations and derive conditions under which each class of the convex programs is equivalent to the original nonconvex formulation our conditions depend on the notions of mutual and cumulative subspace coherence of a dictionary which are natural generalizations of existing notions of mutual and cumulative coherence we evaluate the performance of the proposed convex programs through simulations as well as real experiments on face recognition we show that treating the face recognition problem as a block sparse recovery problem improves the state of the art results by 10 with only 25 of the training data

multiscale hybrid linear models for lossy image representation in this paper we introduce a simple and efficient representation for natural images we view an image in either the spatial domain or the wavelet domain as a collection of vectors in a high dimensional space we then fit a piece wise linear model i e a union of affine subspaces to the vectors at each downsampling scale we call this a multiscale hybrid linear model for the image the model can be effectively estimated via a new algebraic method known as generalized principal component analysis gpca the hybrid and hierarchical structure of this model allows us to effectively extract and exploit multimodal correlations among the imagery data at different scales it conceptually and computationally remedies limitations of many existing image representation methods that are based on either a fixed linear transformation e g dct wavelets or an adaptive uni modal linear transformation e g pca or a multimodal model that uses only cluster means e g vq we will justify both quantitatively and experimentally why and how such a simple multiscale hybrid model is able to reduce simultaneously the model complexity and computational cost despite a small overhead of the model our careful and extensive experimental results show that this new model gives more compact representations for a wide variety of natural images under a wide range of signal to noise ratios than many existing methods including wavelets we also briefly address how the same hybrid linear modeling paradigm can be extended to be potentially useful for other applications such as image segmentation

robust recovery of subspace structures by low rank representation in this paper we address the subspace clustering problem given a set of data samples vectors approximately drawn from a union of multiple subspaces our goal is to cluster the samples into their respective subspaces and remove possible outliers as well to this end we propose a novel objective function named low rank representation lrr which seeks the lowest rank representation among all the candidates that can represent the data samples as linear combinations of the bases in a given dictionary it is shown that the convex program associated with lrr solves the subspace clustering problem in the following sense when the data is clean we prove that lrr exactly recovers the true subspace structures when the data are contaminated by outliers we prove that under certain conditions lrr can exactly recover the row space of the original data and detect the outlier as well for data corrupted by arbitrary sparse errors lrr can also approximately recover the row space with theoretical guarantees since the subspace membership is provably determined by the row space these further imply that lrr can perform robust subspace clustering and error correction in an efficient and effective way

mixtures of probabilistic principal component analyzers principal component analysis pca is one of the most popular techniques for processing compressing and visualizing data although its effectiveness is limited by its global linearity while nonlinear variants of pca have been proposed an alternative paradigm is to capture data complexity by a combination of local linear pca projections however conventional pca does not correspond to a probability density and so there is no unique way to combine pca models therefore previous attempts to formulate mixture models for pca have been ad hoc to some extent in this article pca is formulated within a maximum likelihood framework based on a specific form of gaussian latent variable model this leads to a well defined mixture model for probabilistic principal component analyzers whose parameters can be determined using an expectationmaximization algorithm we discuss the advantages of this model in the context of clustering density modeling and local dimensionality reduction and we demonstrate its application to image compression and handwritten digit recognition

generalized principal component analysis gpca this paper presents an algebro geometric solution to the problem of segmenting an unknown number of subspaces of unknown and varying dimensions from sample data points we represent the subspaces with a set of homogeneous polynomials whose degree is the number of subspaces and whose derivatives at a data point give normal vectors to the subspace passing through the point when the number of subspaces is known we show that these polynomials can be estimated linearly from data hence subspace segmentation is reduced to classifying one point per subspace we select these points optimally from the data set by minimizing certain distance function thus dealing automatically with moderate noise in the data a basis for the complement of each subspace is then recovered by applying standard pca to the collection of derivatives normal vectors extensions of gpca that deal with data in a high dimensional space and with an unknown number of subspaces are also presented our experiments on low dimensional data show that gpca outperforms existing algebraic algorithms based on polynomial factorization and provides a good initialization to iterative techniques such as k subspaces and expectation maximization we also present applications of gpca to computer vision problems such as face clustering temporal video segmentation and 3d motion segmentation from point correspondences in multiple affine views

blessing of dimensionality recovering mixture data via dictionary pursuit this paper studies the problem of recovering the authentic samples that lie on a union of multiple subspaces from their corrupted observations due to the high dimensional and massive nature of today s data driven community it is arguable that the target matrix i e authentic sample matrix to recover is often low rank in this case the recently established robust principal component analysis rpca method already provides us a convenient way to solve the problem of recovering mixture data however in general rpca is not good enough because the incoherent condition assumed by rpca is not so consistent with the mixture structure of multiple subspaces namely when the subspace number grows the row coherence of data keeps heightening and accordingly rpca degrades to overcome the challenges arising from mixture data we suggest to consider lrr in this paper we elucidate that lrr can well handle mixture data as long as its dictionary is configured appropriately more precisely we mathematically prove that lrr can weaken the dependence on the row coherence provided that the dictionary is well conditioned and has a rank of not too high in particular if the dictionary itself is sufficiently low rank then the dependence on the row coherence can be completely removed these provide some elementary principles for dictionary learning and naturally lead to a practical algorithm for recovering mixture data our experiments on randomly generated matrices and real motion sequences show promising results

simultaneous bayesian sparse approximation with structured sparse models sparse approximation is key to many signal processing image processing and machine learning applications if multiple signals maintain some degree of dependency for example the support sets are statistically related then it will generally be advantageous to jointly estimate the sparse representation vectors from the measurement vectors as opposed to solving for each signal individually in this paper we propose simultaneous sparse bayesian learning sbl for joint sparse approximation with two structured sparse models ssms where one is row sparse with embedded element sparse and the other one is row sparse plus element sparse while sbl has attracted much attention as a means to deal with a single sparse approximation problem it is not obvious how to extend sbl to ssms by capitalizing on a dual space view of existing convex methods for sms we showcase the precision component model and covariance component model for ssms where both models involve a common hyperparameter and an innovation hyperparameter that together control the prior variance for each coefficient the statistical perspective of precision component versus covariance component models unfolds the intrinsic mechanism in ssms and also leads to our development of sbl inspired cost functions for ssms centralized algorithms that include ell 1 and ell 2 reweighting algorithms and consensus based decentralized algorithms are developed for simultaneous sparse approximation with ssms in addition theoretical analysis is conducted to provide valuable insights into the proposed approach which includes global minima analysis of the sbl inspired nonconvex cost functions and convergence analysis of the proposed ell 1 reweighting algorithms for ssms superior performance of the proposed algorithms is demonstrated by numerical experiments

robust kernel low rank representation recently low rank representation lrr has shown promising performance in many real world applications such as face clustering however lrr may not achieve satisfactory results when dealing with the data from nonlinear subspaces since it is originally designed to handle the data from linear subspaces in the input space meanwhile the kernel based methods deal with the nonlinear data by mapping it from the original input space to a new feature space through a kernel induced mapping to effectively cope with the nonlinear data we first propose the kernelized version of lrr in the clean data case we also present a closed form solution for the resultant optimization problem moreover to handle corrupted data we propose the robust kernel lrr rklrr approach and develop an efficient optimization algorithm to solve it based on the alternating direction method in particular we show that both the subproblems in our optimization algorithm can be efficiently and exactly solved and it is guaranteed to obtain a globally optimal solution besides our proposed algorithm can also solve the original lrr problem which is a special case of our rklrr when using the linear kernel in addition based on our new optimization technique the kernelization of some variants of lrr can be similarly achieved comprehensive experiments on synthetic data sets and real world data sets clearly demonstrate the efficiency of our algorithm as well as the effectiveness of rklrr and the kernelization of two variants of lrr

spectral curvature clustering scc this paper presents novel techniques for improving the performance of a multi way spectral clustering framework govindu in proceedings of the 2005 ieee computer society conference on computer vision and pattern recognition cvpr 05 vol 1 pp 1150 1157 2005 chen and lerman 2007 preprint in the supplementary webpage for segmenting affine subspaces specifically it suggests an iterative sampling procedure to improve the uniform sampling strategy an automatic scheme of inferring the tuning parameter from the data a precise initialization procedure for k means as well as a simple strategy for isolating outliers the resulting algorithm spectral curvature clustering scc requires only linear storage and takes linear running time in the size of the data it is supported by theory which both justifies its successful performance and guides our practical choices we compare it with other existing methods on a few artificial instances of affine subspaces application of the algorithm to several real world problems is also discussed

discriminative block diagonal representation learning for image recognition existing block diagonal representation studies mainly focuses on casting block diagonal regularization on training data while only little attention is dedicated to concurrently learning both block diagonal representations of training and test data in this paper we propose a discriminative block diagonal low rank representation bdlrr method for recognition in particular the elaborate bdlrr is formulated as a joint optimization problem of shrinking the unfavorable representation from off block diagonal elements and strengthening the compact block diagonal representation under the semisupervised framework of lrr to this end we first impose penalty constraints on the negative representation to eliminate the correlation between different classes such that the incoherence criterion of the extra class representation is boosted moreover a constructed subspace model is developed to enhance the self expressive power of training samples and further build the representation bridge between the training and test samples such that the coherence of the learned intraclass representation is consistently heightened finally the resulting optimization problem is solved elegantly by employing an alternative optimization strategy and a simple recognition algorithm on the learned representation is utilized for final prediction extensive experimental results demonstrate that the proposed method achieves superb recognition results on four face image data sets three character data sets and the 15 scene multicategories data set it not only shows superior potential on image recognition but also outperforms the state of the art methods

linear time subspace clustering via bipartite graph modeling we present a linear time subspace clustering approach that combines sparse representations and bipartite graph modeling the signals are modeled as drawn from a union of low dimensional subspaces and each signal is represented by a sparse combination of basis elements termed atoms which form the columns of a dictionary matrix the sparse representation coefficients are arranged in a sparse affinity matrix which defines a bipartite graph of two disjoint sets 1 atoms and 2 signals subspace clustering is obtained by applying low complexity spectral bipartite graph clustering that exploits the small number of atoms for complexity reduction the complexity of the proposed approach is linear in the number of signals thus it can rapidly cluster very large data collections performance evaluation of face clustering and temporal video segmentation demonstrates comparable clustering accuracies to state of the art at a significantly lower computational load

hypergraph modelling for geometric model fitting in this paper we propose a novel hypergraph based method called hf to fit and segment multi structural data the proposed hf formulates the geometric model fitting problem as a hypergraph partition problem based on a novel hypergraph model in the hypergraph model vertices represent data points and hyperedges denote model hypotheses the hypergraph with large and data determined degrees of hyperedges can express the complex relationships between model hypotheses and data points in addition we develop a robust hypergraph partition algorithm to detect sub hypergraphs for model fitting hf can effectively and efficiently estimate the number of and the parameters of model instances in multi structural data heavily corrupted with outliers simultaneously experimental results show the advantages of the proposed method over previous methods on both synthetic data and real images highlightswe propose a novel hypergraph based method to fit and segment multi structural data the proposed method includes a hypergraph model with large degrees of hyperedges the proposed method includes a robust hypergraph partition algorithm experimental results show that the proposed method is superior to some state of the art fitting methods

shape and motion from image streams under orthography a factorization method inferring scene geometry and camera motion from a stream of images is possible in principle but is an ill conditioned problem when the objects are distant with respect to their size we have developed a factorization method that can overcome this difficulty by recovering shape and motion under orthography without computing depth as an intermediate step an image stream can be represented by the 2fxp measurement matrix of the image coordinates of p points tracked through f frames we show that under orthographic projection this matrix is of rank 3 based on this observation the factorization method uses the singular value decomposition technique to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively two of the three translation components are computed in a preprocessing stage the method can also handle and obtain a full solution from a partially filled in measurement matrix that may result from occlusions or tracking failures the method gives accurate results and does not introduce smoothing in either shape or motion we demonstrate this with a series of experiments on laboratory and outdoor image streams with and without occlusions

acquiring linear subspaces for face recognition under variable lighting previous work has demonstrated that the image variation of many objects human faces in particular under variable lighting can be effectively modeled by low dimensional linear spaces even when there are multiple light sources and shadowing basis images spanning this space are usually obtained in one of three ways a large set of images of the object under different lighting conditions is acquired and principal component analysis pca is used to estimate a subspace alternatively synthetic images are rendered from a 3d model perhaps reconstructed from images under point sources and again pca is used to estimate a subspace finally images rendered from a 3d model under diffuse lighting based on spherical harmonics are directly used as basis images in this paper we show how to arrange physical lighting so that the acquired images of each object can be directly used as the basis vectors of a low dimensional linear space and that this subspace is close to those acquired by the other methods more specifically there exist configurations of k point light source directions with k typically ranging from 5 to 9 such that by taking k images of an object under these single sources the resulting subspace is an effective representation for recognition under a wide range of lighting conditions since the subspace is generated directly from real images potentially complex and or brittle intermediate steps such as 3d reconstruction can be completely avoided nor is it necessary to acquire large numbers of training images or to physically construct complex diffuse harmonic light fields we validate the use of subspaces constructed in this fashion within the context of face recognition

sparse subspace clustering algorithm theory and applications many real world problems deal with collections of high dimensional data such as images videos text and web documents dna microarray data and more often such high dimensional data lie close to low dimensional structures corresponding to several classes or categories to which the data belong in this paper we propose and study an algorithm called sparse subspace clustering to cluster data points that lie in a union of low dimensional subspaces the key idea is that among the infinitely many possible representations of a data point in terms of other points a sparse representation corresponds to selecting a few points from the same subspace this motivates solving a sparse optimization program whose solution is used in a spectral clustering framework to infer the clustering of the data into subspaces since solving the sparse optimization program is in general np hard we consider a convex relaxation and show that under appropriate conditions on the arrangement of the subspaces and the distribution of the data the proposed minimization program succeeds in recovering the desired sparse representations the proposed algorithm is efficient and can handle data points near the intersections of subspaces another key advantage of the proposed algorithm with respect to the state of the art is that it can deal directly with data nuisances such as noise sparse outlying entries and missing entries by incorporating the model of the data into the sparse optimization program we demonstrate the effectiveness of the proposed algorithm through experiments on synthetic data as well as the two real world problems of motion segmentation and face clustering

joint sparse representation and embedding propagation learning a framework for graph based semisupervised learning in this paper we propose a novel graph based semisupervised learning framework called joint sparse representation and embedding propagation learning jsrepl the idea of jsrepl is to join epl with sparse representation to perform label propagation like most of graph based semisupervised propagation learning algorithms jsrepl also constructs weights graph matrix from given data different from classical approaches which build weights graph matrix and estimate the labels of unlabeled data in sequence jsrepl simultaneously builds weights graph matrix and estimates the labels of unlabeled data we also propose an efficient algorithm to solve the proposed problem the proposed method is applied to the problem of semisupervised image clustering using the orl yale pie and yaleb data sets our experiments demonstrate the effectiveness of our proposed algorithm

algebraic clustering of affine subspaces subspace clustering is an important problem in machine learning with many applications in computer vision and pattern recognition prior work has studied this problem using algebraic iterative statistical low rank and sparse representation techniques while these methods have been applied to both linear and affine subspaces theoretical results have only been established in the case of linear subspaces for example algebraic subspace clustering asc is guaranteed to provide the correct clustering when the data points are in general position and the union of subspaces is transversal in this paper we study in a rigorous fashion the properties of asc in the case of affine subspaces using notions from algebraic geometry we prove that the homogenization trick which embeds points in a union of affine subspaces into points in a union of linear subspaces preserves the general position of the points and the transversality of the union of subspaces in the embedded space thus establishing the correctness of asc for affine subspaces

extrinsic methods for coding and dictionary learning on grassmann manifolds sparsity based representations have recently led to notable results in various visual recognition tasks in a separate line of research riemannian manifolds have been shown useful for dealing with features and models that do not lie in euclidean spaces with the aim of building a bridge between the two realms we address the problem of sparse coding and dictionary learning in grassmann manifolds i e the space of linear subspaces to this end we propose to embed grassmann manifolds into the space of symmetric matrices by an isometric mapping this in turn enables us to extend two sparse coding schemes to grassmann manifolds furthermore we propose an algorithm for learning a grassmann dictionary atom by atom lastly to handle non linearity in data we extend the proposed grassmann sparse coding and dictionary learning algorithms through embedding into higher dimensional hilbert spaces experiments on several classification tasks gender recognition gesture classification scene analysis face recognition action recognition and dynamic texture classification show that the proposed approaches achieve considerable improvements in discrimination accuracy in comparison to state of the art methods such as kernelized affine hull method and graph embedding grassmann discriminant analysis

a grassmann framework for 4d facial shape analysis in this paper we investigate the contribution of dynamic evolution of 3d faces to identity recognition to this end we adopt a subspace representation of the flow of curvature maps computed on 3d facial frames of a sequence after normalizing their pose such representation allows us to embody the shape as well as its temporal evolution within the same subspace representation dictionary learning and sparse coding over the space of fixed dimensional subspaces called grassmann manifold have been used to perform face recognition we have conducted extensive experiments on the bu 4dfe dataset the obtained results of the proposed approach provide promising results highlightsrole of facial shape dynamics in identity recognition effective representation of 3d faces and their dynamics on grassmann manifolds sparse coding and dictionary learning for 4d faces classification

numerical analysis of a frictionless viscoelastic contact problem with normal damped response abstract we consider a mathematical model which describes the frictionless contact between a viscoelastic body and a deformable foundation we model the material s behavior with a nonlinear kelvin voigt constitutive law the process is assumed to be quasistatic and the contact is modeled with a general normal damped response condition we present the variational formulation of the problem including the existence of a unique weak solution to the model we then study the numerical approach to the problem using a fully discrete finite elements scheme with an explicit discretization in time we state the existence of the unique solution for the scheme and derive an error estimate on the approximate solutions finally we present some numerical results involving examples in one and two dimensions

domain class consistency based transfer learning for image classification across domains distribution mismatch between the modeling data and the query data is a known domain adaptation issue in machine learning to this end in this paper we propose a l2 1 norm based discriminative robust kernel transfer learning dktl method for high level recognition tasks the key idea is to realize robust domain transfer by simultaneously integrating domain class consistency dcc metric based discriminative subspace learning kernel learning in reproduced kernel hilbert space and representation learning between source and target domain the dcc metric includes two properties domain consistency used to measure the between domain distribution discrepancy and class consistency used to measure the within domain class separability the essential objective of the proposed transfer learning method is to maximize the dcc metric which is equivalently to minimize the domain class inconsistency dcic such that domain distribution mismatch and class inseparability are well formulated and unified simultaneously the merits of the proposed method include 1 the robust sparse coding selects a few valuable source data with noises outliers removed during knowledge transfer and 2 the proposed dcc metric can pursue more discriminative subspaces of different domains as a result the maximum class separability is also well guaranteed extensive experiments on a number of visual datasets demonstrate the superiority of the proposed method over other state of the art domain adaptation and transfer learning methods

robust elastic net subspace representation recently finding the low dimensional structure of high dimensional data has gained much attention given a set of data points sampled from a single subspace or a union of subspaces the goal is to learn or capture the underlying subspace structure of the data set in this paper we propose elastic net subspace representation a new subspace representation framework using elastic net regularization of singular values due to the strong convexity enforced by elastic net the proposed method is more stable and robust in the presence of heavy corruptions compared with existing lasso type rank minimization approaches for discovering a single low dimensional subspace we propose a computationally efficient low rank factorization algorithm called facten using a property of the nuclear norm and the augmented lagrangian method then clusten is proposed to handle the general case in which the data samples are drawn from a union of multiple subspaces for joint subspace clustering and estimation the proposed algorithms are applied to a number of subspace representation problems to evaluate the robustness and efficiency under various noisy conditions and experimental results show the benefits of the proposed method compared with existing methods

lagrange necessary conditions for pareto minimizers in asplund spaces and applications abstract in this paper new necessary conditions for pareto minimal points to sets and pareto minimizers for constrained multiobjective optimization problems are established without the sequentially normal compactness property and the asymptotical compactness condition imposed on closed and convex ordering cones in bao and mordukhovich 10 and durea and dutta 5 respectively our approach is based on a version of the separation theorem for nonconvex sets and the subdifferentials of vector valued and set valued mappings furthermore applications in mathematical finance and approximation theory are discussed