strong mixed integer programming formulations for trained neural networks we present strong mixed integer programming mip formulations for high dimensional piecewise linear functions that correspond to trained neural networks these formulations can be used for a number of important tasks such as verifying that an image classification network is robust to adversarial inputs or solving decision problems where the objective function is a machine learning model we present a generic framework which may be of independent interest that provides a way to construct sharp or ideal formulations for the maximum of d affine functions over arbitrary polyhedral input domains we apply this result to derive mip formulations for a number of the most popular nonlinear operations e g relu and max pooling that are strictly stronger than other approaches from the literature we corroborate this computationally showing that our formulations are able to offer substantial improvements in solve time on verification tasks for image classification networks

small and strong formulations for unions of convex sets from the cayley embedding there is often a significant trade off between formulation strength and size in mixed integer programming when modeling convex disjunctive constraints e g unions of convex sets adding auxiliary continuous variables can sometimes help resolve this trade off however standard formulations that use such auxiliary continuous variables can have a worse than expected computational effectiveness which is often attributed precisely to these auxiliary continuous variables for this reason there has been considerable interest in constructing strong formulations that do not use continuous auxiliary variables we introduce a technique to construct formulations without these detrimental continuous auxiliary variables to develop this technique we introduce a natural non polyhedral generalization of the cayley embedding of a family of polytopes and show it inherits many geometric properties of the original embedding we then show how the associated formulation technique can be used to construct small and strong formulation for a wide range of disjunctive constraints in particular we show it can recover and generalize all known strong formulations without continuous auxiliary variables

information theoretic subspace clustering this paper addresses the problem of grouping the data points sampled from a union of multiple subspaces in the presence of outliers information theoretic objective functions are proposed to combine structured low rank representations lrrs to capture the global structure of data and information theoretic measures to handle outliers in theoretical part we point out that group sparsity induced measures ell 2 1 norm ell alpha norm and correntropy can be justified from the viewpoint of half quadratic hq optimization which facilitates both convergence study and algorithmic development in particular a general formulation is accordingly proposed to unify hq based group sparsity methods into a common framework in algorithmic part we develop information theoretic subspace clustering methods via correntropy with the help of parzen window estimation correntropy is used to handle either outliers under any distributions or sample specific errors in data pairwise link constraints are further treated as a prior structure of lrrs based on the hq framework iterative algorithms are developed to solve the nonconvex information theoretic loss functions experimental results on three benchmark databases show that our methods can further improve the robustness of lrr subspace clustering and outperform other state of the art subspace clustering methods

learning graphs with monotone topology properties and multiple connected components recent papers have formulated the problem of learning graphs from data as an inverse covariance estimation problem with graph laplacian constraints while such problems are convex existing methods cannot guarantee that solutions will have specific graph topology properties e g being a tree which are desirable for some applications the problem of learning a graph with topology properties is in general non convex in this paper we propose an approach to solve these problems by decomposing them into two sub problems for which efficient solutions are known specifically a graph topology inference gti step is employed to select a feasible graph topology then a graph weight estimation gwe step is performed by solving a generalized graph laplacian estimation problem where edges are constrained by the topology found in the gti step our main result is a bound on the error of the gwe step as a function of the error in the gti step this error bound indicates that the gti step should be solved using an algorithm that approximates the data similarity matrix by another matrix whose entries have been thresholded to zero to have the desired type of graph topology the gti stage can leverage existing methods which are typically based on minimizing the total weight of removed edges since the gwe stage is an inverse covariance estimation problem with linear constraints it can be solved using existing convex optimization methods we demonstrate that our approach can achieve good results for both synthetic and texture image data

numerical dispersion in dg fetd method using brick and tetrahedral edge elements a numerical dispersion analysis nda of the discontinuous galerkin finite element time domain dg fetd method is presented vector basis functions under the nda are with brick and tetrahedral elements this nda shows that there exist both normal and spurious modes in the dg fetd method a dg fetd modeling of transverse electromagnetic tem wave propagation in a parallel waveguide is applied to verify the nda for zeroth order vector bases the effect of wave propagation direction and electrical size of elements on numerical dispersion is investigated it is shown from the nda for higher order interpolatory vector bases that the phase error of normal modes can be significantly reduced by using higher order bases

multibody grouping from motion images we want to deduce from a sequence of noisy two dimensional images of a scene of several rigid bodies moving independently in three dimensions the number of bodies and the grouping of given feature points in the images to the bodies prior processing is assumed to have identified features or points common to all frames and the images are assumed to be created by orthographic projection i e perspective effects are minimal we describe a computationally inexpensive algorithm that can determine which points or features belong to which rigid body using the fact that with exact observations in orthographic projection points on a single body lie in a three or less dimensional linear manifold of frame space if there are enough observations and independent motions these manifolds can be viewed as a set linearly independent four or less dimensional subspaces we show that the row echelon canonical form provides direct information on the grouping of points to these subspaces treatment of the noise is the most difficult part of the problem this paper uses a statistical approach to estimate the grouping of points to subspaces in the presence of noise by computing which partition has the maximum likelihood the input data is assumed to be contaminated with independent gaussian noise the algorithm can base its estimates on a user supplied standard deviation of the noise or it can estimate the noise from the data the algorithm can also be used to estimate the probability of a user specified partition so that the hypothesis can be combined with others using bayesian statistics

embedding formulations and complexity for unions of polyhedra it is well known that selecting a good mixed integer programming mip formulation is crucial for effectively obtaining a solution with state of the art solvers although best practices and guidelines for constructing good formulations abound there is rarely a single systematic construction approach that leads to the best possible formulation here we introduce embedding formulations and complexity as a new mip formulation paradigm for systematically constructing formulations for disjunctive constraints that are optimal with respect to size more specifically they yield the smallest possible ideal formulation i e one whose lp relaxation has integral extreme points among all formulations that only use 0 1 auxiliary variables we use the paradigm to characterize optimal formulations for special ordered sets of type 2 and certain piecewise linear functions of two variables we also show that the resultant formulations can provide a significant computational advantage over all known formulations for pi

simultaneous clustering and model selection algorithm theory and applications while clustering has been well studied in the past decade model selection has drawn much less attention due to the difficulty of the problem in this paper we address both problems in a joint manner by recovering an ideal affinity tensor from an imperfect input by taking into account the relationship of the affinities induced by the cluster structures we are able to significantly improve the affinity input such as repairing those entries corrupted by gross outliers more importantly the recovered ideal affinity tensor also directly indicates the number of clusters and their membership thus solving the model selection and clustering jointly to enforce the requisite global consistency in the affinities demanded by the cluster structure we impose a number of constraints specifically among others the tensor should be low rank and sparse and it should obey what we call the rank 1 sum constraint to solve this highly non smooth and non convex problem we exploit the mathematical structures and express the original problem in an equivalent form amenable for numerical optimization and convergence analysis to scale to large problem sizes we also propose an alternative formulation so that those problems can be efficiently solved via stochastic optimization in an online fashion we evaluate our algorithm with different applications to demonstrate its superiority and show it can adapt to a large variety of settings

a multibody factorization method for independently moving objects the structure from motion problem has been extensively studied in the field of computer vision yet the bulk of the existing work assumes that the scene contains only a single moving object the more realistic case where an unknown number of objects move in the scene has received little attention especially for its theoretical treatment in this paper we present a new method for separating and recovering the motion and shape of multiple independently moving objects in a sequence of images the method does not require prior knowledge of the number of objects nor is dependent on any grouping of features into an object at the image level for this purpose we introduce a mathematical construct of object shapes called the shape interaction matrix which is invariant to both the object motions and the selection of coordinate systems this invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects once the matrix is computed it allows for segmenting features into objects by the process of transforming it into a canonical form as well as recovering the shape and motion of each object the theory works under a broad set of projection models scaled orthography paraperspective and affine but they must be linear so it excludes projective cameras

two view multibody structure from motion we present an algebraic geometric approach to 3 d motion estimation and segmentation of multiple rigid body motions from noise free point correspondences in two perspective views our approach exploits the algebraic and geometric properties of the so called multibody epipolar constraint and its associated multibody fundamental matrix which are natural generalizations of the epipolar constraint and of the fundamental matrix to multiple motions we derive a rank constraint on a polynomial embedding of the correspondences from which one can estimate the number of independent motions as well as linearly solve for the multibody fundamental matrix we then show how to compute the epipolar lines from the first order derivatives of the multibody epipolar constraint and the epipoles by solving a plane clustering problem using generalized pca gpca given the epipoles and epipolar lines the estimation of individual fundamental matrices becomes a linear problem the clustering of the feature points is then automatically obtained from either the epipoles and epipolar lines or from the individual fundamental matrices although our approach is mostly designed for noise free correspondences we also test its performance on synthetic and real data with moderate levels of noise

recursive identification of switched arx systems we consider the problem of recursively identifying the parameters of a deterministic discrete time switched auto regressive exogenous sarx model under the assumption that the number of models the model orders and the mode sequence are unknown the key to our approach is to view the identification of multiple arx models as the identification of a single though more complex lifted dynamical model built by applying a polynomial embedding to the input output data we show that the dynamics of this lifted model do not depend on the value of the discrete state or the switching mechanism and are linear on the so called hybrid model parameters therefore one can identify the parameters of the lifted model using a standard recursive identifier applied to the embedded input output data the estimated hybrid model parameters are then used to build a polynomial whose derivatives at a regressor give an estimate of the parameters of the arx model generating that regressor the estimated arx model parameters are shown to converge exponentially to their true values under a suitable persistence of excitation condition on a projection of the embedded input output data such a condition is a natural generalization of the well known result for arx models although our algorithm is designed for perfect input output data our experiments also evaluate its performance as a function of the level of noise for different choices of the number of models and model orders we also present an application to temporal video segmentation

spatio temporal union of subspaces for multi body non rigid structure from motion abstract non rigid structure from motion nrsfm has so far been mostly studied for recovering 3d structure of a single non rigid deforming object to handle the real world challenging multiple deforming objects scenarios existing methods either pre segment different objects in the scene or treat multiple non rigid objects as a whole to obtain the 3d non rigid reconstruction however these methods fail to exploit the inherent structure in the problem as the solution of segmentation and the solution of reconstruction could not benefit each other in this paper we propose a unified framework to jointly segment and reconstruct multiple non rigid objects to compactly represent complex multi body non rigid scenes we propose to exploit the structure of the scenes along both temporal direction and spatial direction thus achieving a spatio temporal representation specifically we represent the 3d non rigid deformations as lying in a union of subspaces along the temporal direction and represent the 3d trajectories as lying in the union of subspaces along the spatial direction this spatio temporal representation not only provides competitive 3d reconstruction but also outputs robust segmentation of multiple non rigid objects the resultant optimization problem is solved efficiently using the alternating direction method of multipliers admm extensive experimental results on both synthetic and real multi body nrsfm datasets demonstrate the superior performance of our proposed framework compared with the state of the art methods

on deriving a locally corrected nystrom scheme from a quadrature sampled moment method a novel high order method of moment procedure with quadrature point based discretization is presented the scheme is equivalent to a moment method employing smooth basis and testing functions applying a fixed point numerical quadrature approximation for the outer integral mapping the current to the quadrature points then leads to a formulation that is equivalent to that derived via the locally corrected nystrom method the convergence properties of the current density and rcs for smooth and singular geometries in two and three dimensions are also studied

observed baseline convergence rates and superconvergence in the scattering cross section obtained from numerical solutions of the mfie the scattering cross section data obtained from numerical solutions of the magnetic field integral equation mfie using a variety of basis functions and both galerkin and non galerkin testing schemes are compared to study the convergence rates of the results for the basis functions considered apparent superconvergence is observed in the mfie scattering cross section when linear tangential linear normal vector basis functions are used with galerkin testing but not with other basis functions

characterizations of mixed binary convex quadratic representable sets representability results play a fundamental role in optimization since they provide characterizations of the feasible sets that arise from optimization problems in this paper we study the sets that appear in the feasibility version of mixed binary convex quadratic optimization problems we provide a complete characterization of the sets that can be obtained as the projection of such feasible regions in order to obtain this result we first provide a complete characterization of these sets in the special cases where 1 the feasible region is bounded 2 only binary extended variables are present and 3 only continuous variables are present

a regularized approach for geodesic based semisupervised multimanifold learning geodesic distance as an essential measurement for data dissimilarity has been successfully used in manifold learning however most geodesic distance based manifold learning algorithms have two limitations when applied to classification 1 class information is rarely used in computing the geodesic distances between data points on manifolds and 2 little attention has been paid to building an explicit dimension reduction mapping for extracting the discriminative information hidden in the geodesic distances in this paper we regard geodesic distance as a kind of kernel which maps data from linearly inseparable space to linear separable distance space in doing this a new semisupervised manifold learning algorithm namely regularized geodesic feature learning algorithm is proposed the method consists of three techniques a semisupervised graph construction method replacement of original data points with feature vectors which are built by geodesic distances and a new semisupervised dimension reduction method for feature vectors experiments on the mnist usps handwritten digit data sets mit cbcl face versus nonface data set and an intelligent traffic data set show the effectiveness of the proposed algorithm

estimation of subspace arrangements with applications in modeling and segmenting mixed data recently many scientific and engineering applications have involved the challenging task of analyzing large amounts of unsorted high dimensional data that have very complicated structures from both geometric and statistical points of view such unsorted data are considered mixed as different parts of the data have significantly different structures which cannot be described by a single model in this paper we propose to use subspace arrangements a union of multiple subspaces for modeling mixed data each subspace in the arrangement is used to model just a homogeneous subset of the data thus multiple subspaces together can capture the heterogeneous structures within the data set in this paper we give a comprehensive introduction to a new approach for the estimation of subspace arrangements this is known as generalized principal component analysis gpca in particular we provide a comprehensive summary of important algebraic properties and statistical facts that are crucial for making the inference of subspace arrangements both efficient and robust even when the given data are corrupted by noise or contaminated with outliers this new method in many ways improves and generalizes extant methods for modeling or clustering mixed data there have been successful applications of this new method to many real world problems in computer vision image processing and system identification in this paper we will examine several of those representative applications this paper is intended to be expository in nature however in order that this may serve as a more complete reference for both theoreticians and practitioners we take the liberty of filling in several gaps between the theory and the practice in the existing literature

singularity evaluation of the straight wire mixed potential integral equation in the method of moments procedure a rigorous treatment on the computation of the various integrals that arise in the method of moments mom formulation of the straight wire electric field integral equation is provided for triangle basis functions along with delta function pulse or triangle weights particular attention is given to integrals whose integrands are weakly singular a singularity extraction technique is employed that splits the integral under question into two parts one that is numerically integrable and one that is analytically integrable closed form approximations based on taylor series techniques are also provided for the former these approximations are very robust resulting in errors less than 0 1 when 2 1 here is the length of a single mom segment and is the wire radius results are compared with data from the literature to demonstrate the robustness of the presented approach for fat wires

self regularized fixed rank representation for subspace segmentation abstract consider a set of data points generated from various linear subspaces subspace segmentation tasks which are important in fields such as computer vision and image processing aim to partition the set of data points so as to recover these subspaces the subspace segmentation method fixed rank representation frr was introduced to remedy the problem of insufficient sampling in classical low rank representation lrr in many subspace segmentation applications frr has achieved much better results than those of lrr in this paper a new frr related algorithm called self regularized fixed rank representation srfrr is proposed in srfrr a laplacian regularizer is constructed using the coefficient matrix obtained by srfrr itself further by proving that the laplacian regularizer can be transformed into a structure constraint on the coefficient matrix we show that another existing method sparse frr sfrr is a special case of srfrr to implement the srfrr method we present two optimization algorithms experiments on both synthetic and real databases show that srfrr outperforms some existing frr and lrr related algorithms

learning from hidden traits joint factor analysis and latent clustering dimensionality reduction techniques play an essential role in data analytics signal processing and machine learning dimensionality reduction is usually performed in a preprocessing stage that is separate from subsequent data analysis such as clustering or classification finding reduced dimension representations that are well suited for the intended task is more appealing this paper proposes a joint factor analysis and latent clustering framework which aims at learning cluster aware low dimensional representations of matrix and tensor data the proposed approach leverages matrix and tensor factorization models that produce essentially unique latent representations of the data to unravel latent cluster structure which is otherwise obscured because of the freedom to apply an oblique transformation in latent space at the same time latent cluster structure is used as prior information to enhance the performance of factorization specific contributions include several custom built problem formulations corresponding algorithms and discussion of associated convergence properties besides extensive simulations real world datasets such as reuters document data and mnist image data are also employed to showcase the effectiveness of the proposed approaches

on the approximability of minimizing nonzero variables or unsatisfied relations in linear systems we investigate the computational complexity of two closely related classes of combinatorial optimization problems for linear systems which arise in various fields such as machine learning operations research and pattern recognition in the first class min ulr one wishes given a possibly infeasible system of linear relations to find a solution that violates as few relations as possible while satisfying all the others in the second class min rvls the linear system is supposed to be feasible and one looks for a solution with as few nonzero variables as possible for both min ulr and min rvls the four basic types of relational operators and are considered while min rvls with equations was mentioned to be np hard in garey and johnson 1979 we established in amaldi 1992 amaldi and kann 1995 that min ulr with equalities and inequalities are np hard even when restricted to homogeneous systems with bipolar coefficients the latter problems have been shown hard to approximate in arora et al 1993 in this paper we determine strong bounds on the approximability of various variants of min rvls and min ulr including constrained ones where the variables are restricted to take binary values or where some relations are mandatory while others are optional the various np hard versions turn out to have different approximability properties depending on the type of relations and the additional constraints but none of them can be approximated within any constant factor unless p np particular attention is devoted to two interesting special cases that occur in discriminant analysis and machine learning in particular we disprove a conjecture of van horn and martinez 1992 regarding the existence of a polynomial time algorithm to design linear classifiers or perceptrons that involve a close to minimum number of features

subspace learning by l0 induced sparsity subspace clustering methods partition the data that lie in or close to a union of subspaces in accordance with the subspace structure such methods with sparsity prior such as sparse subspace clustering ssc elhamifar and vidal in ieee trans pattern anal mach intell 35 11 2765 2781 2013 with the sparsity induced by the ell 1 norm are demonstrated to be effective in subspace clustering most of those methods require certain assumptions e g independence or disjointness on the subspaces however these assumptions are not guaranteed to hold in practice and they limit the application of existing sparse subspace clustering methods in this paper we propose ell 0 induced sparse subspace clustering ell 0 ssc in contrast to the required assumptions such as independence or disjointness on subspaces for most existing sparse subspace clustering methods we prove that ell 0 ssc guarantees the subspace sparse representation a key element in subspace clustering for arbitrary distinct underlying subspaces almost surely under the mild i i d assumption on the data generation we also present the no free lunch theorem which shows that obtaining the subspace representation under our general assumptions can not be much computationally cheaper than solving the corresponding ell 0 sparse representation problem of ell 0 ssc a novel approximate algorithm named approximate ell 0 ssc a ell 0 ssc is developed which employs proximal gradient descent to obtain a sub optimal solution to the optimization problem of ell 0 ssc with theoretical guarantee the sub optimal solution is used to build a sparse similarity matrix upon which spectral clustering is performed for the final clustering results extensive experimental results on various data sets demonstrate the superiority of a ell 0 ssc compared to other competing clustering methods furthermore we extend ell 0 ssc to semi supervised learning by performing label propagation on the sparse similarity matrix learnt by a ell 0 ssc and demonstrate the effectiveness of the resultant semi supervised learning method termed ell 0 sparse subspace label propagation ell 0 sslp

minimum error entropy based sparse representation for robust subspace clustering this paper addresses the problem of clustering data points that are approximately drawn from multiple subspaces recently a large family of spectral clustering based methods such as sparse subspace clustering ssc and low rank representation lrr have been proposed in this paper we present a general formulation to unify many of them within a common framework based on atomic representation since mean square error mse relies heavily on the gaussianity assumption the previous mse based subspace clustering methods have the limitation of being sensitive to non gaussian noise in this paper we develop a novel subspace clustering method termed meessc by specifying the minimum error entropy mee as the loss function and the sparsity inducing atomic set we show that meessc can well overcome the above limitation the experimental results on both synthetic and real data verify the effectiveness of the proposed method

motion segmentation in the presence of outlying incomplete or corrupted trajectories in this paper we study the problem of segmenting tracked feature point trajectories of multiple moving objects in an image sequence using the affine camera model this problem can be cast as the problem of segmenting samples drawn from multiple linear subspaces in practice due to limitations of the tracker occlusions and the presence of nonrigid objects in the scene the obtained motion trajectories may contain grossly mistracked features missing entries or corrupted entries in this paper we develop a robust subspace separation scheme that deals with these practical issues in a unified mathematical framework our methods draw strong connections between lossy compression rank minimization and sparse representation we test our methods extensively on the hopkins155 motion segmentation database and other motion sequences with outliers and missing data we compare the performance of our methods to state of the art motion segmentation methods based on expectation maximization and spectral clustering for data without outliers or missing information the results of our methods are on par with the state of the art results and in many cases exceed them in addition our methods give surprisingly good performance in the presence of the three types of pathological trajectories mentioned above all code and results are publicly available at http perception csl uiuc edu coding motion

recovery guarantees for exemplar based clustering for a certain class of distributions we prove that the linear programming relaxation of k medoids clustering a variant of k means clustering where means are replaced by exemplars from within the dataset distinguishes points drawn from nonoverlapping balls with high probability once the number of points drawn and the separation distance between any two balls are sufficiently large our results hold in the nontrivial regime where the separation distance is small enough that points drawn from different balls may be closer to each other than points drawn from the same ball in this case clustering by thresholding pairwise distances between points can fail we also exhibit numerical evidence of high probability recovery in a substantially more permissive regime

adaptive graph construction using data self representativeness for pattern classification graph construction from data constitutes a pre stage in many machine learning and computer vision tasks like semi supervised learning manifold learning and spectral clustering the influence of graph construction procedures on learning tasks and their related applications has only received limited study despite its critical impact on accuracy state of the art graphs are built via sparse coding adopting 1 regularization those graphs exhibit good performance in many computer vision applications however the locality and similarity among instances are not explicitly used in the coding scheme furthermore due to the use of 1 regularization these construction approaches can be computationally expensive in this paper we investigate graph construction using the data self representativeness property by incorporating a variant of locality constrained linear coding llc we introduce and derive four variants for graph construction these variants adopt a two phase llc tpllc compared with the recent 1 graphs our proposed objective function associated with three variants has an analytical solution and thus is more efficient a key element of the proposed methods is the second phase of coding that allows data closeness or locality to be naturally incorporated it performs a coding over some selected relevant samples and reinforces the individual regularization terms by exploiting the coefficients estimated in the first phase comprehensive experimental results using several benchmark datasets show that it can achieve or outperform existing state of the art results furthermore it is shown to be more efficient than the robust 1 graph construction schemes

domain class consistency based transfer learning for image classification across domains distribution mismatch between the modeling data and the query data is a known domain adaptation issue in machine learning to this end in this paper we propose a l2 1 norm based discriminative robust kernel transfer learning dktl method for high level recognition tasks the key idea is to realize robust domain transfer by simultaneously integrating domain class consistency dcc metric based discriminative subspace learning kernel learning in reproduced kernel hilbert space and representation learning between source and target domain the dcc metric includes two properties domain consistency used to measure the between domain distribution discrepancy and class consistency used to measure the within domain class separability the essential objective of the proposed transfer learning method is to maximize the dcc metric which is equivalently to minimize the domain class inconsistency dcic such that domain distribution mismatch and class inseparability are well formulated and unified simultaneously the merits of the proposed method include 1 the robust sparse coding selects a few valuable source data with noises outliers removed during knowledge transfer and 2 the proposed dcc metric can pursue more discriminative subspaces of different domains as a result the maximum class separability is also well guaranteed extensive experiments on a number of visual datasets demonstrate the superiority of the proposed method over other state of the art domain adaptation and transfer learning methods

lambertian reflectance and linear subspaces we prove that the set of all lambertian reflectance functions the mapping from surface normals to intensities obtained with arbitrary distant light sources lies close to a 9d linear subspace this implies that in general the set of images of a convex lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low dimensional linear subspace explaining prior empirical results we also provide a simple analytic characterization of this linear space we obtain these results by representing lighting using spherical harmonics and describing the effects of lambertian materials as the analog of a convolution these results allow us to construct algorithms for object recognition based on linear methods as well as algorithms that use convex optimization to enforce nonnegative lighting functions we also show a simple way to enforce nonnegative lighting when the images of an object lie near a 4d linear space we apply these algorithms to perform face recognition by finding the 3d model that best matches a 2d query image

robust subspace clustering via thresholding the problem of clustering noisy and incompletely observed high dimensional data points into a union of low dimensional subspaces and a set of outliers is considered the number of subspaces their dimensions and their orientations are assumed unknown we propose a simple low complexity subspace clustering algorithm which applies spectral clustering to an adjacency matrix obtained by thresholding the correlations between data points in other words the adjacency matrix is constructed from the nearest neighbors of each data point in spherical distance a statistical performance analysis shows that the algorithm exhibits robustness to additive noise and succeeds even when the subspaces intersect specifically our results reveal an explicit tradeoff between the affinity of the subspaces and the tolerable noise level we furthermore prove that the algorithm succeeds even when the data points are incompletely observed with the number of missing entries allowed to be up to a log factor linear in the ambient dimension we also propose a simple scheme that provably detects outliers and we present numerical results on real and synthetic data

coherence pursuit fast simple and robust principal component analysis this paper presents a remarkably simple yet powerful algorithm termed coherence pursuit cop to robust principal component analysis pca as inliers lie in a low dimensional subspace and are mostly correlated an inlier is likely to have strong mutual coherence with a large number of data points by contrast outliers either do not admit low dimensional structures or form small clusters in either case an outlier is unlikely to bear strong resemblance to a large number of data points given that cop sets an outlier apart from an inlier by comparing their coherence with the rest of the data points the mutual coherences are computed by forming the gram matrix of the normalized data points subsequently the sought subspace is recovered from the span of the subset of the data points that exhibit strong coherence with the rest of the data as cop only involves one simple matrix multiplication it is significantly faster than the state of the art robust pca algorithms we derive analytical performance guarantees for cop under different models for the distributions of inliers and outliers in both noise free and noisy settings cop is the first robust pca algorithm that is simultaneously non iterative provably robust to both unstructured and structured outliers and can tolerate a large number of unstructured outliers

a geometric analysis of subspace clustering with outliers this paper considers the problem of clustering a collection of unlabeled data points assumed to lie near a union of lower dimensional planes as is common in computer vision or unsupervised learning applications we do not know in advance how many subspaces there are nor do we have any information about their dimensions we develop a novel geometric analysis of an algorithm named sparse subspace clustering ssc 11 which signicantly broadens the range of problems where it is provably eective for instance we show that ssc can recover multiple subspaces each of dimension comparable to the ambient dimension we also prove that ssc can correctly cluster data points even when the subspaces of interest intersect further we develop an extension of ssc that succeeds when the data set is corrupted with possibly overwhelmingly many outliers underlying our analysis are clear geometric insights which may bear on other sparse recovery problems a numerical study complements our theoretical analysis and demonstrates the eectiveness of these methods

frequency independent scattering from a flat strip with rm te _ z polarized fields the surface current for scattering from a pec strip naturally decomposes into three slowly varying functions modulating rapidly oscillating phase factors we exploit this structure to derive a numerical solution that is error controllable and exhibits a bounded error over the full range of frequencies frequency independence is obtained by expanding the current in terms of slowly varying amplitude functions stretching coordinates in the boundary layer and employing a frequency independent quadrature rule though the total current solution is always well defined unique minimally varying amplitude functions may also be found by using the minimum norm concept

a performance baseline for the convergence of electromagnetic integral equation calculations using pulse functions a parametric study characterizes the residual errors associated with calculating electromagnetic integral equations using pulse function expansions to accomplish this the integral equations are first reduced to straightforwardly computable convergent integrals of one variable pulse function expansions then lead to approximations based on admittance matrices construction of the admittances is an order n operation explicitly quantified contributions by integrand singularities become dominant error contributors if not judiciously treated for both parallel plate and rectangular tez mode waveguides at most thirteen samples per waveguide mode half cycle are sufficient to produce magnitude residual errors that are less than one percent of the calculated quantity

three view multibody structure from motion we propose a geometric approach to 3d motion segmentation from point correspondences in three perspective views we demonstrate that after applying a polynomial embedding to the point correspondences they become related by the so called multibody trilinear constraint and its associated multibody trifocal tensor which are natural generalizations of the trilinear constraint and the trifocal tensor to multiple motions we derive a rank constraint on the embedded correspondences from which one can estimate the number of independent motions as well as linearly solve for the multibody trifocal tensor we then show how to compute the epipolar lines associated with each image point from the common root of a set of univariate polynomials and the epipoles by solving a pair of plane clustering problems using generalized principal component analysis gpca the individual trifocal tensors are then obtained from the second order derivatives of the multibody trilinear constraint given epipolar lines and epipoles or trifocal tensors one can immediately obtain an initial clustering of the correspondences we use this clustering to initialize an iterative algorithm that alternates between the computation of the trifocal tensors and the segmentation of the correspondences we test our algorithm on various synthetic and real scenes and compare it with other algebraic and iterative algorithms

matrix completion by least square low rank and sparse self representations abstract conventional matrix completion methods are generally based on rank minimization these methods assume that the given matrix is of low rank and the data points are drawn from a single subspace of low dimensionality therefore they are not effective in completing matrices where the data are drawn from multiple subspaces in this paper we establish a novel matrix completion framework that is based on self representation specifically least square low rank and sparse self representations based matrix completion algorithms are provided the underlying idea is that one data point can be efficiently reconstructed by other data points belonging to a common subspace where the missing entries are recovered so as to fit the common subspace the proposed algorithms actually maximize the weighted correlations among the columns of a given matrix we prove that the proposed algorithms are approximations for rank minimization of the incomplete matrix in addition they are able to complete high rank or even full rank matrices when the data are drawn from multiple subspaces comparative studies are conducted on synthetic datasets natural image inpainting tasks temperature prediction task and collaborative filtering tasks the results show that the proposed algorithms often outperform other state of the art algorithms in various tasks

enhancing low rank subspace clustering by manifold regularization recently low rank representation lrr method has achieved great success in subspace clustering sc which aims to cluster the data points that lie in a union of low dimensional subspace given a set of data points lrr seeks the lowest rank representation among the many possible linear combinations of the bases in a given dictionary or in terms of the data itself however lrr only considers the global euclidean structure while the local manifold structure which is often important for many real applications is ignored in this paper to exploit the local manifold structure of the data a manifold regularization characterized by a laplacian graph has been incorporated into lrr leading to our proposed laplacian regularized lrr laplrr an efficient optimization procedure which is based on alternating direction method of multipliers admm is developed for laplrr experimental results on synthetic and real data sets are presented to demonstrate that the performance of lrr has been enhanced by using the manifold regularization

exploiting unsupervised and supervised constraints for subspace clustering data in many image and video analysis tasks can be viewed as points drawn from multiple low dimensional subspaces with each subspace corresponding to one category or class one basic task for processing such kind of data is to separate the points according to the underlying subspace referred to as subspace clustering extensive studies have been made on this subject and nearly all of them use unconstrained subspace models meaning the points can be drawn from everywhere of a subspace to represent the data in this paper we attempt to do subspace clustering based on a constrained subspace assumption that the data is further restricted in the corresponding subspaces e g belonging to a submanifold or satisfying the spatial regularity constraint this assumption usually describes the real data better such as differently moving objects in a video scene and face images of different subjects under varying illumination a unified integer linear programming optimization framework is used to approach subspace clustering which can be efficiently solved by a branch and bound bb method we also show that various kinds of supervised information such as subspace number outlier ratio pairwise constraints size prior and etc can be conveniently incorporated into the proposed framework experiments on real data show that the proposed method outperforms the state of the art algorithms significantly in clustering accuracy the effectiveness of the proposed method in exploiting supervised information is also demonstrated

transformation invariant subspace clustering subspace clustering has achieved great success in many computer vision applications however most subspace clustering algorithms require well aligned data samples which is often not straightforward to achieve this paper proposes a transformation invariant subspace clustering framework by jointly aligning data samples and learning subspace representation by alignment the transformed data samples become highly correlated and a better affinity matrix can be obtained the joint problem can be reduced to a sequence of least squares regression problems which can be efficiently solved we verify the effectiveness of the proposed method with extensive experiments on unaligned real data demonstrating its higher clustering accuracy than the state of the art subspace clustering and transformation invariant clustering algorithms highlightsa transformation invariant subspace clustering algorithm is proposed alignment and subspace clustering are mutually dependent state of the art clustering results on the unconstrained face dataset

convex sparse spectral clustering single view to multi view spectral clustering sc is one of the most widely used methods for data clustering it first finds a low dimensional embedding u of data by computing the eigenvectors of the normalized laplacian matrix and then performs k means on text u top to get the final clustering result in this paper we observe that in the ideal case text u text u top should be block diagonal and thus sparse therefore we propose the sparse sc ssc method that extends the sc with sparse regularization on text u text u top to address the computational issue of the nonconvex ssc model we propose a novel convex relaxation of ssc based on the convex hull of the fixed rank projection matrices then the convex ssc model can be efficiently solved by the alternating direction method of multipliers furthermore we propose the pairwise ssc that extends ssc to boost the clustering performance by using the multi view information of data experimental comparisons with several baselines on real world datasets testify to the efficacy of our proposed methods

multiscale hybrid linear models for lossy image representation in this paper we introduce a simple and efficient representation for natural images we view an image in either the spatial domain or the wavelet domain as a collection of vectors in a high dimensional space we then fit a piece wise linear model i e a union of affine subspaces to the vectors at each downsampling scale we call this a multiscale hybrid linear model for the image the model can be effectively estimated via a new algebraic method known as generalized principal component analysis gpca the hybrid and hierarchical structure of this model allows us to effectively extract and exploit multimodal correlations among the imagery data at different scales it conceptually and computationally remedies limitations of many existing image representation methods that are based on either a fixed linear transformation e g dct wavelets or an adaptive uni modal linear transformation e g pca or a multimodal model that uses only cluster means e g vq we will justify both quantitatively and experimentally why and how such a simple multiscale hybrid model is able to reduce simultaneously the model complexity and computational cost despite a small overhead of the model our careful and extensive experimental results show that this new model gives more compact representations for a wide variety of natural images under a wide range of signal to noise ratios than many existing methods including wavelets we also briefly address how the same hybrid linear modeling paradigm can be extended to be potentially useful for other applications such as image segmentation

block sparse recovery via convex optimization given a dictionary that consists of multiple blocks and a signal that lives in the range space of only a few blocks we study the problem of finding a block sparse representation of the signal i e a representation that uses the minimum number of blocks motivated by signal image processing and computer vision applications such as face recognition we consider the block sparse recovery problem in the case where the number of atoms in each block is arbitrary possibly much larger than the dimension of the underlying subspace to find a block sparse representation of a signal we propose two classes of nonconvex optimization programs which aim to minimize the number of nonzero coefficient blocks and the number of nonzero reconstructed vectors from the blocks respectively since both classes of problems are np hard we propose convex relaxations and derive conditions under which each class of the convex programs is equivalent to the original nonconvex formulation our conditions depend on the notions of mutual and cumulative subspace coherence of a dictionary which are natural generalizations of existing notions of mutual and cumulative coherence we evaluate the performance of the proposed convex programs through simulations as well as real experiments on face recognition we show that treating the face recognition problem as a block sparse recovery problem improves the state of the art results by 10 with only 25 of the training data

human motion segmentation via robust kernel sparse subspace clustering studies on human motion have attracted a lot of attentions human motion capture data which much more precisely records human motion than videos do has been widely used in many areas motion segmentation is an indispensable step for many related applications but current segmentation methods for motion capture data do not effectively model some important characteristics of motion capture data such as riemannian manifold structure and containing non gaussian noise in this paper we convert the segmentation of motion capture data into a temporal subspace clustering problem under the framework of sparse subspace clustering we propose to use the geodesic exponential kernel to model the riemannian manifold structure use correntropy to measure the reconstruction error use the triangle constraint to guarantee temporal continuity in each cluster and use multi view reconstruction to extract the relations between different joints therefore exploiting some special characteristics of motion capture data we propose a new segmentation method which is robust to non gaussian noise since correntropy is a localized similarity measure we also develop an efficient optimization algorithm based on block coordinate descent method to solve the proposed model our optimization algorithm has a linear complexity while sparse subspace clustering is originally a quadratic problem extensive experiment results both on simulated noisy data set and real noisy data set demonstrate the advantage of the proposed method

top k feature selection framework using robust 0 1 integer programming feature selection fs which identifies the relevant features in a data set to facilitate subsequent data analysis is a fundamental problem in machine learning and has been widely studied in recent years most fs methods rank the features in order of their scores based on a specific criterion and then select the k top ranked features where k is the number of desired features however these features are usually not the top k features and may present a suboptimal choice to address this issue we propose a novel fs framework in this article to select the exact top k features in the unsupervised semisupervised and supervised scenarios the new framework utilizes the l0 2 norm as the matrix sparsity constraint rather than its relaxations such as the l1 2 norm since the l0 2 norm constrained problem is difficult to solve we transform the discrete l0 2 norm based constraint into an equivalent 0 1 integer constraint and replace the 0 1 integer constraint with two continuous constraints the obtained top k fs framework with two continuous constraints is theoretically equivalent to the l0 2 norm constrained problem and can be optimized by the alternating direction method of multipliers admm unsupervised and semisupervised fs methods are developed based on the proposed framework and extensive experiments on real world data sets are conducted to demonstrate the effectiveness of the proposed fs framework

robust recovery of subspace structures by low rank representation in this paper we address the subspace clustering problem given a set of data samples vectors approximately drawn from a union of multiple subspaces our goal is to cluster the samples into their respective subspaces and remove possible outliers as well to this end we propose a novel objective function named low rank representation lrr which seeks the lowest rank representation among all the candidates that can represent the data samples as linear combinations of the bases in a given dictionary it is shown that the convex program associated with lrr solves the subspace clustering problem in the following sense when the data is clean we prove that lrr exactly recovers the true subspace structures when the data are contaminated by outliers we prove that under certain conditions lrr can exactly recover the row space of the original data and detect the outlier as well for data corrupted by arbitrary sparse errors lrr can also approximately recover the row space with theoretical guarantees since the subspace membership is provably determined by the row space these further imply that lrr can perform robust subspace clustering and error correction in an efficient and effective way

generalized principal component analysis gpca this paper presents an algebro geometric solution to the problem of segmenting an unknown number of subspaces of unknown and varying dimensions from sample data points we represent the subspaces with a set of homogeneous polynomials whose degree is the number of subspaces and whose derivatives at a data point give normal vectors to the subspace passing through the point when the number of subspaces is known we show that these polynomials can be estimated linearly from data hence subspace segmentation is reduced to classifying one point per subspace we select these points optimally from the data set by minimizing certain distance function thus dealing automatically with moderate noise in the data a basis for the complement of each subspace is then recovered by applying standard pca to the collection of derivatives normal vectors extensions of gpca that deal with data in a high dimensional space and with an unknown number of subspaces are also presented our experiments on low dimensional data show that gpca outperforms existing algebraic algorithms based on polynomial factorization and provides a good initialization to iterative techniques such as k subspaces and expectation maximization we also present applications of gpca to computer vision problems such as face clustering temporal video segmentation and 3d motion segmentation from point correspondences in multiple affine views

mixtures of probabilistic principal component analyzers principal component analysis pca is one of the most popular techniques for processing compressing and visualizing data although its effectiveness is limited by its global linearity while nonlinear variants of pca have been proposed an alternative paradigm is to capture data complexity by a combination of local linear pca projections however conventional pca does not correspond to a probability density and so there is no unique way to combine pca models therefore previous attempts to formulate mixture models for pca have been ad hoc to some extent in this article pca is formulated within a maximum likelihood framework based on a specific form of gaussian latent variable model this leads to a well defined mixture model for probabilistic principal component analyzers whose parameters can be determined using an expectationmaximization algorithm we discuss the advantages of this model in the context of clustering density modeling and local dimensionality reduction and we demonstrate its application to image compression and handwritten digit recognition

blessing of dimensionality recovering mixture data via dictionary pursuit this paper studies the problem of recovering the authentic samples that lie on a union of multiple subspaces from their corrupted observations due to the high dimensional and massive nature of today s data driven community it is arguable that the target matrix i e authentic sample matrix to recover is often low rank in this case the recently established robust principal component analysis rpca method already provides us a convenient way to solve the problem of recovering mixture data however in general rpca is not good enough because the incoherent condition assumed by rpca is not so consistent with the mixture structure of multiple subspaces namely when the subspace number grows the row coherence of data keeps heightening and accordingly rpca degrades to overcome the challenges arising from mixture data we suggest to consider lrr in this paper we elucidate that lrr can well handle mixture data as long as its dictionary is configured appropriately more precisely we mathematically prove that lrr can weaken the dependence on the row coherence provided that the dictionary is well conditioned and has a rank of not too high in particular if the dictionary itself is sufficiently low rank then the dependence on the row coherence can be completely removed these provide some elementary principles for dictionary learning and naturally lead to a practical algorithm for recovering mixture data our experiments on randomly generated matrices and real motion sequences show promising results

variational nature of galerkin and non galerkin moment method solutions there is renewed interest in the use of variational methods in conjunction with numerical solutions of electromagnetic radiation and scattering problems the variational aspects of secondary calculations based on a method of moments mom solution are investigated these calculations exhibit a type of second order accuracy regardless of whether or not the operator being discretized is self adjoint and regardless of whether or not testing functions are identical to the basis functions galerkin s method numerical results support these conclusions and suggest that the advantage of galerkin s method in actual calculations is grossly overstated

simultaneous bayesian sparse approximation with structured sparse models sparse approximation is key to many signal processing image processing and machine learning applications if multiple signals maintain some degree of dependency for example the support sets are statistically related then it will generally be advantageous to jointly estimate the sparse representation vectors from the measurement vectors as opposed to solving for each signal individually in this paper we propose simultaneous sparse bayesian learning sbl for joint sparse approximation with two structured sparse models ssms where one is row sparse with embedded element sparse and the other one is row sparse plus element sparse while sbl has attracted much attention as a means to deal with a single sparse approximation problem it is not obvious how to extend sbl to ssms by capitalizing on a dual space view of existing convex methods for sms we showcase the precision component model and covariance component model for ssms where both models involve a common hyperparameter and an innovation hyperparameter that together control the prior variance for each coefficient the statistical perspective of precision component versus covariance component models unfolds the intrinsic mechanism in ssms and also leads to our development of sbl inspired cost functions for ssms centralized algorithms that include ell 1 and ell 2 reweighting algorithms and consensus based decentralized algorithms are developed for simultaneous sparse approximation with ssms in addition theoretical analysis is conducted to provide valuable insights into the proposed approach which includes global minima analysis of the sbl inspired nonconvex cost functions and convergence analysis of the proposed ell 1 reweighting algorithms for ssms superior performance of the proposed algorithms is demonstrated by numerical experiments

robust kernel low rank representation recently low rank representation lrr has shown promising performance in many real world applications such as face clustering however lrr may not achieve satisfactory results when dealing with the data from nonlinear subspaces since it is originally designed to handle the data from linear subspaces in the input space meanwhile the kernel based methods deal with the nonlinear data by mapping it from the original input space to a new feature space through a kernel induced mapping to effectively cope with the nonlinear data we first propose the kernelized version of lrr in the clean data case we also present a closed form solution for the resultant optimization problem moreover to handle corrupted data we propose the robust kernel lrr rklrr approach and develop an efficient optimization algorithm to solve it based on the alternating direction method in particular we show that both the subproblems in our optimization algorithm can be efficiently and exactly solved and it is guaranteed to obtain a globally optimal solution besides our proposed algorithm can also solve the original lrr problem which is a special case of our rklrr when using the linear kernel in addition based on our new optimization technique the kernelization of some variants of lrr can be similarly achieved comprehensive experiments on synthetic data sets and real world data sets clearly demonstrate the efficiency of our algorithm as well as the effectiveness of rklrr and the kernelization of two variants of lrr

balas formulation for the union of polytopes is optimal a celebrated theorem of balas gives a linear mixed integer formulation for the union of two nonempty polytopes whose relaxation gives the convex hull of this union the number of inequalities in balas formulation is linear in the number of inequalities that describe the two polytopes and the number of variables is doubled in this paper we show that this is best possible in every dimension there exist two nonempty polytopes such that if a formulation for the convex hull of their union has a number of inequalities that is polynomial in the number of inequalities that describe the two polytopes then the number of additional variables is at least linear in the dimension of the polytopes we then show that this result essentially carries over if one wants to approximate the convex hull of the union of two polytopes and also in the more restrictive setting of lift and project

spectral curvature clustering scc this paper presents novel techniques for improving the performance of a multi way spectral clustering framework govindu in proceedings of the 2005 ieee computer society conference on computer vision and pattern recognition cvpr 05 vol 1 pp 1150 1157 2005 chen and lerman 2007 preprint in the supplementary webpage for segmenting affine subspaces specifically it suggests an iterative sampling procedure to improve the uniform sampling strategy an automatic scheme of inferring the tuning parameter from the data a precise initialization procedure for k means as well as a simple strategy for isolating outliers the resulting algorithm spectral curvature clustering scc requires only linear storage and takes linear running time in the size of the data it is supported by theory which both justifies its successful performance and guides our practical choices we compare it with other existing methods on a few artificial instances of affine subspaces application of the algorithm to several real world problems is also discussed

an interior penalty method for the generalized method of moments the generalized method of moments gmm is a technique to discretize integral equations that permits integration of different types of basis functions as well as different geometric descriptions using a partition of unity framework while accuracy and efficacy of the method have been demonstrated the integration quadratures required to compute the inner products are often high as they have to respect spatial variation of the integrand to overcome this problem we introduce an interior penalty function method within the gmm framework the penalty formulation yields solutions that are smooth and accurate both in surface currents and fields with significantly lower numerical quadrature orders than would be required for the uncompensated operator to demonstrate and analyze the method we conduct an analytical and numerical investigation of the properties of the penalty method applied to the 2 d text te z electric field integral equation efie operator

higher order frequency domain computational electromagnetics a review of the higher order computational electromagnetics cem for antenna wireless and microwave engineering applications is presented higher order cem techniques use current field basis functions of higher orders defined on large e g on the order of a wavelength in each dimension curvilinear geometrical elements which greatly reduces the number of unknowns for a given problem the paper reviews all major surface volume integral and differential equation electromagnetic formulations within a higher order computational framework focusing on frequency domain solutions with a systematic and unified review of generalized curved parametric quadrilateral triangular hexahedral and tetrahedral elements and various types of higher order hierarchical and interpolatory vector basis functions in both divergence and curl conforming arrangements a large number of actual higher order techniques representing various combinations of formulations elements bases and solution procedures are identified and discussed the examples demonstrate the accuracy efficiency and versatility of higher order techniques and their advantages over low order discretizations the most important one being a much faster higher order convergence of the solution it is demonstrated that both components of the higher order modeling namely higher order geometrical modeling and higher order current field modeling are essential for accurate and efficient cem analysis of general antenna scattering and microwave structures

a sharp bound for the castelnuovo mumford regularity of subspace arrangements abstract we show that the ideal of an arrangement of d linear subspaces of projective space is d regular in the sense of castelnuovo and mumford answering a question of b sturmfels in particular this implies that the ideal of an arrangement of d subspaces is generated in degrees less than or equal to d

hybrid linear modeling via local best fit flats we present a simple and fast geometric method for modeling data by a union of affine subspaces the method begins by forming a collection of local best fit affine subspaces i e subspaces approximating the data in local neighborhoods the correct sizes of the local neighborhoods are determined automatically by the jones 2 numbers we prove under certain geometric conditions that our method finds the optimal local neighborhoods the collection of subspaces is further processed by a greedy selection procedure or a spectral method to generate the final model we discuss applications to tracking based motion segmentation and clustering of faces under different illuminating conditions we give extensive experimental evidence demonstrating the state of the art accuracy and speed of the suggested algorithms on these problems and also on synthetic hybrid linear data as well as the mnist handwritten digits data and we demonstrate how to use our algorithms for fast determination of the number of affine subspaces

linear time subspace clustering via bipartite graph modeling we present a linear time subspace clustering approach that combines sparse representations and bipartite graph modeling the signals are modeled as drawn from a union of low dimensional subspaces and each signal is represented by a sparse combination of basis elements termed atoms which form the columns of a dictionary matrix the sparse representation coefficients are arranged in a sparse affinity matrix which defines a bipartite graph of two disjoint sets 1 atoms and 2 signals subspace clustering is obtained by applying low complexity spectral bipartite graph clustering that exploits the small number of atoms for complexity reduction the complexity of the proposed approach is linear in the number of signals thus it can rapidly cluster very large data collections performance evaluation of face clustering and temporal video segmentation demonstrates comparable clustering accuracies to state of the art at a significantly lower computational load

hypergraph modelling for geometric model fitting in this paper we propose a novel hypergraph based method called hf to fit and segment multi structural data the proposed hf formulates the geometric model fitting problem as a hypergraph partition problem based on a novel hypergraph model in the hypergraph model vertices represent data points and hyperedges denote model hypotheses the hypergraph with large and data determined degrees of hyperedges can express the complex relationships between model hypotheses and data points in addition we develop a robust hypergraph partition algorithm to detect sub hypergraphs for model fitting hf can effectively and efficiently estimate the number of and the parameters of model instances in multi structural data heavily corrupted with outliers simultaneously experimental results show the advantages of the proposed method over previous methods on both synthetic data and real images highlightswe propose a novel hypergraph based method to fit and segment multi structural data the proposed method includes a hypergraph model with large degrees of hyperedges the proposed method includes a robust hypergraph partition algorithm experimental results show that the proposed method is superior to some state of the art fitting methods

shape and motion from image streams under orthography a factorization method inferring scene geometry and camera motion from a stream of images is possible in principle but is an ill conditioned problem when the objects are distant with respect to their size we have developed a factorization method that can overcome this difficulty by recovering shape and motion under orthography without computing depth as an intermediate step an image stream can be represented by the 2fxp measurement matrix of the image coordinates of p points tracked through f frames we show that under orthographic projection this matrix is of rank 3 based on this observation the factorization method uses the singular value decomposition technique to factor the measurement matrix into two matrices which represent object shape and camera rotation respectively two of the three translation components are computed in a preprocessing stage the method can also handle and obtain a full solution from a partially filled in measurement matrix that may result from occlusions or tracking failures the method gives accurate results and does not introduce smoothing in either shape or motion we demonstrate this with a series of experiments on laboratory and outdoor image streams with and without occlusions

ellipsoidal mixed integer representability representability results for mixed integer linear systems play a fundamental role in optimization since they give geometric characterizations of the feasible sets that can be formulated by mixed integer linear programming we consider a natural extension of mixed integer linear systems obtained by adding just one ellipsoidal inequality the set of points that can be described possibly using additional variables by these systems are called ellipsoidal mixed integer representable in this work we give geometric conditions that characterize ellipsoidal mixed integer representable sets

acquiring linear subspaces for face recognition under variable lighting previous work has demonstrated that the image variation of many objects human faces in particular under variable lighting can be effectively modeled by low dimensional linear spaces even when there are multiple light sources and shadowing basis images spanning this space are usually obtained in one of three ways a large set of images of the object under different lighting conditions is acquired and principal component analysis pca is used to estimate a subspace alternatively synthetic images are rendered from a 3d model perhaps reconstructed from images under point sources and again pca is used to estimate a subspace finally images rendered from a 3d model under diffuse lighting based on spherical harmonics are directly used as basis images in this paper we show how to arrange physical lighting so that the acquired images of each object can be directly used as the basis vectors of a low dimensional linear space and that this subspace is close to those acquired by the other methods more specifically there exist configurations of k point light source directions with k typically ranging from 5 to 9 such that by taking k images of an object under these single sources the resulting subspace is an effective representation for recognition under a wide range of lighting conditions since the subspace is generated directly from real images potentially complex and or brittle intermediate steps such as 3d reconstruction can be completely avoided nor is it necessary to acquire large numbers of training images or to physically construct complex diffuse harmonic light fields we validate the use of subspaces constructed in this fashion within the context of face recognition

variational aspects of the reaction in the method of moments the reaction r j sub 1 j sub 2 between two electric surface current densities j sub 1 and j sub 2 is the integral of the dot product of j sub 1 with the electric field produced by j sub 2 jack h richmond see ibid vol 39 no 4 p 473 479 1991 has correctly shown that the reaction is variational a formula is variational if it has a stationary point at the quantity of interest with galerkin s method but is not necessarily variational when the non galerkin element method is applied in the manner where one set of functions is used to expand both j sub 1 and j sub 2 and another set of functions is used to test both the integral equation for j sub 1 and that for j sub 2 in this paper the reaction r j sub 1 j sub 2 is shown to be variational when the non galerkin method is applied in the manner of operation where the set of functions used to expand j sub 2 is the set of functions used to test the integral equation for j sub 1 and the set of functions used to test the integral equation for j sub 2 is the set of functions used to expand j sub 1 hence the manner of operation just described may be more appropriate than richmond s if a variational result is desired a few numerical results are included to compare the performance of both manners of operation

sparse subspace clustering algorithm theory and applications many real world problems deal with collections of high dimensional data such as images videos text and web documents dna microarray data and more often such high dimensional data lie close to low dimensional structures corresponding to several classes or categories to which the data belong in this paper we propose and study an algorithm called sparse subspace clustering to cluster data points that lie in a union of low dimensional subspaces the key idea is that among the infinitely many possible representations of a data point in terms of other points a sparse representation corresponds to selecting a few points from the same subspace this motivates solving a sparse optimization program whose solution is used in a spectral clustering framework to infer the clustering of the data into subspaces since solving the sparse optimization program is in general np hard we consider a convex relaxation and show that under appropriate conditions on the arrangement of the subspaces and the distribution of the data the proposed minimization program succeeds in recovering the desired sparse representations the proposed algorithm is efficient and can handle data points near the intersections of subspaces another key advantage of the proposed algorithm with respect to the state of the art is that it can deal directly with data nuisances such as noise sparse outlying entries and missing entries by incorporating the model of the data into the sparse optimization program we demonstrate the effectiveness of the proposed algorithm through experiments on synthetic data as well as the two real world problems of motion segmentation and face clustering

algebraic clustering of affine subspaces subspace clustering is an important problem in machine learning with many applications in computer vision and pattern recognition prior work has studied this problem using algebraic iterative statistical low rank and sparse representation techniques while these methods have been applied to both linear and affine subspaces theoretical results have only been established in the case of linear subspaces for example algebraic subspace clustering asc is guaranteed to provide the correct clustering when the data points are in general position and the union of subspaces is transversal in this paper we study in a rigorous fashion the properties of asc in the case of affine subspaces using notions from algebraic geometry we prove that the homogenization trick which embeds points in a union of affine subspaces into points in a union of linear subspaces preserves the general position of the points and the transversality of the union of subspaces in the embedded space thus establishing the correctness of asc for affine subspaces

joint sparse representation and embedding propagation learning a framework for graph based semisupervised learning in this paper we propose a novel graph based semisupervised learning framework called joint sparse representation and embedding propagation learning jsrepl the idea of jsrepl is to join epl with sparse representation to perform label propagation like most of graph based semisupervised propagation learning algorithms jsrepl also constructs weights graph matrix from given data different from classical approaches which build weights graph matrix and estimate the labels of unlabeled data in sequence jsrepl simultaneously builds weights graph matrix and estimates the labels of unlabeled data we also propose an efficient algorithm to solve the proposed problem the proposed method is applied to the problem of semisupervised image clustering using the orl yale pie and yaleb data sets our experiments demonstrate the effectiveness of our proposed algorithm

mixed integer linear programming formulation techniques a wide range of problems can be modeled as mixed integer linear programming mip problems using standard formulation techniques however in some cases the resulting mip can be either too weak or

extrinsic methods for coding and dictionary learning on grassmann manifolds sparsity based representations have recently led to notable results in various visual recognition tasks in a separate line of research riemannian manifolds have been shown useful for dealing with features and models that do not lie in euclidean spaces with the aim of building a bridge between the two realms we address the problem of sparse coding and dictionary learning in grassmann manifolds i e the space of linear subspaces to this end we propose to embed grassmann manifolds into the space of symmetric matrices by an isometric mapping this in turn enables us to extend two sparse coding schemes to grassmann manifolds furthermore we propose an algorithm for learning a grassmann dictionary atom by atom lastly to handle non linearity in data we extend the proposed grassmann sparse coding and dictionary learning algorithms through embedding into higher dimensional hilbert spaces experiments on several classification tasks gender recognition gesture classification scene analysis face recognition action recognition and dynamic texture classification show that the proposed approaches achieve considerable improvements in discrimination accuracy in comparison to state of the art methods such as kernelized affine hull method and graph embedding grassmann discriminant analysis

structured sparse representation with union of data driven linear and multilinear subspaces model for compressive video sampling the standard compressive sensing cs theory can be improved for robust recovery with fewer measurements under the assumption that signals lie on a union of subspaces uos however the uos model is restricted to specific types of signal regularities with predetermined topology for subspaces this paper proposes a generalized model which adaptively decomposes signals into a union of data driven subspaces uods for structured sparse representation the proposed uods model leverages subspace clustering to derive the optimal structures and bases for the subspaces conditioned on the sample signals for multidimensional signals with various statistics it supports linear and multilinear subspace learning for compressive sampling as an improvement for generic cs model the basis which represents the sparsity of sample signals is adaptively generated via linear subspace learning method furthermore a generalized model with multilinear subspace learning is considered for cs to avoid vectorization of high dimensional signals in comparison to uos the uods model requires fewer degrees of freedom for a desirable recovery quality experimental results demonstrate that the proposed model for video sampling is promising and applicable

a grassmann framework for 4d facial shape analysis in this paper we investigate the contribution of dynamic evolution of 3d faces to identity recognition to this end we adopt a subspace representation of the flow of curvature maps computed on 3d facial frames of a sequence after normalizing their pose such representation allows us to embody the shape as well as its temporal evolution within the same subspace representation dictionary learning and sparse coding over the space of fixed dimensional subspaces called grassmann manifold have been used to perform face recognition we have conducted extensive experiments on the bu 4dfe dataset the obtained results of the proposed approach provide promising results highlightsrole of facial shape dynamics in identity recognition effective representation of 3d faces and their dynamics on grassmann manifolds sparse coding and dictionary learning for 4d faces classification

restricted isometry property of gaussian random projection for finite set of subspaces dimension reduction plays an essential role when decreasing the complexity of solving large scale problems the well known johnson lindenstrauss jl lemma and restricted isometry property rip admit the use of random projection to reduce the dimension while keeping the euclidean distance which leads to the boom of compressed sensing and the field of sparsity related signal processing recently successful applications of sparse models in computer vision and machine learning have increasingly hinted that the underlying structure of high dimensional data looks more like a union of subspaces in this paper motivated by jl lemma and an emerging field of compressed subspace clustering we study for the first time the rip of gaussian random matrices for the compression of two subspaces based on the generalized projection f norm distance we theoretically prove that with high probability the affinity or distance between two projected subspaces are concentrated around their estimates when the ambient dimension after projection is sufficiently large the affinity and distance between two subspaces almost remain unchanged after random projection numerical experiments verify the theoretical work

evaluation of galerkin integrals involving triangular type wire to surface junctions in the method of moments the objective of this paper is to provide efficient and reliable numerical procedures for the evaluation of the galerkin testing integrals with the use of the triangular type wire to surface junction function the 1 spl rho singular behavior of the testing integrands is removed before the evaluation leading to two dimensional integrations suited for its numerical implementation compact closed form approximations are also given that allows to speed up the computations while maintaining a high level of accuracy the proposed procedure guarantees the reliable application of the galerkin s formulation to the analysis of arbitrary configurations of wires and surfaces using the rao wilton glisson basis functions for surfaces and its counterparts for wires and wire to surface junctions

robust subspace clustering subspace clustering refers to the task of nding a multi subspace representation that best ts a collection of points taken from a high dimensional space this paper introduces an algorithm inspired by sparse subspace clustering ssc 25 to cluster noisy data and develops some novel theory demonstrating its correctness in particular the theory uses ideas from geometric functional analysis to show that the algorithm can accurately recover the underlying subspaces under minimal requirements on their orientation and on the number of samples per subspace synthetic as well as real data experiments complement our theoretical study illustrating our approach and demonstrating its eectiveness

robust elastic net subspace representation recently finding the low dimensional structure of high dimensional data has gained much attention given a set of data points sampled from a single subspace or a union of subspaces the goal is to learn or capture the underlying subspace structure of the data set in this paper we propose elastic net subspace representation a new subspace representation framework using elastic net regularization of singular values due to the strong convexity enforced by elastic net the proposed method is more stable and robust in the presence of heavy corruptions compared with existing lasso type rank minimization approaches for discovering a single low dimensional subspace we propose a computationally efficient low rank factorization algorithm called facten using a property of the nuclear norm and the augmented lagrangian method then clusten is proposed to handle the general case in which the data samples are drawn from a union of multiple subspaces for joint subspace clustering and estimation the proposed algorithms are applied to a number of subspace representation problems to evaluate the robustness and efficiency under various noisy conditions and experimental results show the benefits of the proposed method compared with existing methods

exact algorithms for the equitable traveling salesman problem abstract given a weighted graph g v e the equitable traveling salesman problem etsp asks for two perfect matchings in g such that 1 the two matchings together form a hamiltonian cycle in g and 2 the absolute difference in costs between the two matchings is minimized the problem is shown to be np hard even when the graph g is complete we present two integer programming models to solve the etsp problem and compare the strength of these formulations one model is solved through branch and cut whereas the other model is solved through a branch and price framework a simple local search heuristic is also implemented we conduct computational experiments on different types of instances often derived from the tsplib it turns out that the behavior of the different approaches varies with the type of instances for small and medium sized instances branch and bound and branch and price produce comparable results however for larger instances branch and bound outperforms branch and price