feature adaptive co segmentation by complexity awareness in this paper we propose a novel feature adaptive co segmentation method that can learn adaptive features of different image groups for accurate common objects segmentation we also propose image complexity awareness for adaptive feature learning in the proposed method the original images are first ranked according to the image complexities that are measured by superpixel changing cue and object detection cue then the unsupervised segments of the simple images are used to learn the adaptive features which are achieved using an expectation minimization algorithm combining l 1 regularized least squares optimization with the consideration of the confidence of the simple image segmentation accuracies and the fitness of the learned model the error rate of the final co segmentation is tested by the experiments on different image groups and verified to be lower than the existing state of the art co segmentation methods

convexity shape prior for binary segmentation convexity is a known important cue in human vision we propose shape convexity as a new high order regularization constraint for binary image segmentation in the context of discrete optimization object convexity is represented as a sum of three clique potentials penalizing any 1 0 1 configuration on all straight lines we show that these non submodular potentials can be efficiently optimized using an iterative trust region approach at each iteration the energy is linearly approximated and globally optimized within a small trust region around the current solution while the quadratic number of all three cliques is prohibitively high we design a dynamic programming technique for evaluating and approximating these cliques in linear time we also derive a second order approximation model that is more accurate but computationally intensive we discuss limitations of our local optimization and propose gradual non submodularization scheme that alleviates some limitations our experiments demonstrate general usefulness of the proposed convexity shape prior on synthetic and real image segmentation examples unlike standard second order length regularization our convexity prior does not have shrinking bias and is robust to changes in scale and parameter selection

covariance regularization by thresholding this paper considers regularizing a covariance matrix of p variables estimated from n observations by hard thresholding we show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense the variables are gaussian or sub gaussian and log p n 0 and obtain explicit rates the results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity we discuss an intuitive resampling scheme for threshold selection and prove a general cross validation result that justifies this approach we also compare thresholding to other covariance estimators in simulations and on an example from climate data 1 introduction estimation of covariance matrices is important in a number of areas of statistical analysis including dimension reduction by principal component analysis pca classification by linear or quadratic discriminant analysis lda and qda establishing independence and conditional independence relations in the context of graphical models and setting confidence intervals on linear functions of the means of the components in recent years many application areas where these tools are used have been dealing with very high dimensional datasets and sample sizes can be very small relative to dimension examples include genetic data brain imaging spectroscopic imaging climate data and many others it is well known by now that the empirical covariance matrix for samples of size n from a p variate gaussian distribution np p is not a good estimator of the population covariance if p is large many results in random matrix theory illustrate this from the classical mary law 29 to the more recent work of johnstone and his students on the theory of the largest eigenvalues 12 23 30 and associated eigenvectors 24 however with the exception of a method for estimating the covariance spectrum 11 these probabilistic results do not offer alternatives to the sample covariance matrix alternative estimators for large covariance matrices have therefore attracted a lot of attention recently two broad classes of covariance estimators have emerged those that rely on a natural ordering among variables and assume that variables

on consistency and sparsity for principal components analysis in high dimensions principal components analysis pca is a classic method for the reduction of dimensionality of data in the form of n observations or cases of a vector with p variables contemporary datasets often have p comparable with or even much larger than n our main assertions in such settings are a that some initial reduction in dimensionality is desirable before applying any pca type search for principal modes and b the initial reduction in dimensionality is best achieved by working in a basis in which the signals have a sparse representation we describe a simple asymptotic model in which the estimate of the leading principal component vector via standard pca is consistent if and only if p n n 0 we provide a simple algorithm for selecting a subset of coordinates with largest sample variances and show that if pca is done on the selected subset then consistency is recovered even if p n n

optimal detection of sparse principal components in high dimension we perform a finite sample analysis of the detection levels for sparse principal components of a high dimensional covariance matrix our minimax optimal test is based on a sparse eigenvalue statistic alas computing this test is known to be np complete in general and we describe a computationally efficient alternative test using convex relaxations our relaxation is also proved to detect sparse principal components at near optimal detection levels and performs very well on simulated datasets ams 2000 subject classifications primary 62h25 secondary 62f04 90c22

high dimensional semiparametric scale invariant principal component analysis we propose a new high dimensional semiparametric principal component analysis pca method named copula component analysis coca the semiparametric model assumes that after unspecified marginally monotone transformations the distributions are multivariate gaussian coca improves upon pca and sparse pca in three aspects i it is robust to modeling assumptions ii it is robust to outliers and data contamination iii it is scale invariant and yields more interpretable results we prove that the coca estimators obtain fast estimation rates and are feature selection consistent when the dimension is nearly exponentially large relative to the sample size careful experiments confirm that coca outperforms sparse pca on both synthetic and real world data sets

the probable value of the lovasz schrijver relaxations for maximum independent set lov asz and schrijver siam j optim 1 1991 pp 166 190 devised a lift and project method that produces a sequence of convex relaxations for the problem of finding in a graph an independent set or a clique of maximum size each relaxation in the sequence is tighter than the one before it while the first relaxation is already at least as strong as the lov asz theta function ieee trans inform theory 25 1979 pp 1 7 we show that on a random graph gn 1 2 the value of the rth relaxation in the sequence is roughly rule 0pt 7pt smash sqrt rule 0pt 7pt smash n 2 r almost surely it follows that for those relaxations known to be efficiently computable namely for r o 1 the value of the relaxation is comparable to the theta function furthermore a perfectly tight relaxation is almost surely obtained only at the r theta log n relaxation in the sequence

contour detection and hierarchical image segmentation this paper investigates two fundamental problems in computer vision contour detection and image segmentation we present state of the art algorithms for both of these tasks our contour detector combines multiple local cues into a globalization framework based on spectral clustering our segmentation algorithm consists of generic machinery for transforming the output of any contour detector into a hierarchical region tree in this manner we reduce the problem of image segmentation to that of contour detection extensive experimental evaluation demonstrates that both our contour detection and segmentation methods significantly outperform competing algorithms the automatically generated hierarchical segmentations can be interactively refined by user specified annotations computation at multiple image resolutions provides a means of coupling our system to recognition applications

a direct formulation for sparse pca using semidefinite programming given a covariance matrix we consider the problem of maximizing the variance explained by a particular linear combination of the input variables while constraining the number of nonzero coefficients in this combination this problem arises in the decomposition of a covariance matrix into sparse factors or sparse principal component analysis pca and has wide applications ranging from biology to finance we use a modification of the classical variational representation of the largest eigenvalue of a symmetric matrix where cardinality is constrained and derive a semidefinite programming based relaxation for our problem we also discuss nesterov s smooth minimization technique applied to the semidefinite program arising in the semidefinite relaxation of the sparse pca problem the method has complexity o n 4 sqrt log n epsilon where n is the size of the underlying covariance matrix and epsilon is the desired absolute accuracy on the optimal value of the problem

gaussian determinantal processes a new model for directionality in data determinantal point processes dpps have recently become popular tools for modeling the phenomenon of negative dependence or repulsion in data however our understanding of an analogue of a classical parametric statistical theory is rather limited for this class of models in this work we investigate a parametric family of gaussian dpps with a clearly interpretable effect of parametric modulation on the observed points we show that parameter modulation impacts the observed points by introducing directionality in their repulsion structure and the principal directions correspond to the directions of maximal i e the most long ranged dependency this model readily yields a viable alternative to principal component analysis pca as a dimension reduction tool that favors directions along which the data are most spread out this methodological contribution is complemented by a statistical analysis of a spiked model similar to that employed for covariance matrices as a framework to study pca these theoretical investigations unveil intriguing questions for further examination in random matrix theory stochastic geometry and related topics

normalized cuts and image segmentation we propose a novel approach for solving the perceptual grouping problem in vision rather than focusing on local features and their consistencies in the image data our approach aims at extracting the global impression of an image we treat image segmentation as a graph partitioning problem and propose a novel global criterion the normalized cut for segmenting the graph the normalized cut criterion measures both the total dissimilarity between the different groups as well as the total similarity within the groups we show that an efficient computational technique based on a generalized eigenvalue problem can be used to optimize this criterion we applied this approach to segmenting static images as well as motion sequences and found the results to be very encouraging

on the distribution of the largest eigenvalue in principal components analysis let x 1 denote the square of the largest singular value of an n p matrix x all of whose entries are independent standard gaussian variates equivalently x 1 is the largest principal component variance of the covariance matrix x x or the largest eigenvalue of a p variate wishart distribution on n degrees of freedom with identity covariance consider the limit of large p and n with n p gamma ge 1 when centered by mu p sqrt n 1 sqrt p 2 and scaled by sigma p sqrt n 1 sqrt p 1 sqrt n 1 1 sqrt p 1 3 the distribution of x 1 approaches the tracey widom law of order 1 which is defined in terms of the painleve ii differential equation and can be numerically evaluated and tabulated in software simulations show the approximation to be informative for n and p as small as 5 the limit is derived via a corresponding result for complex wishart matrices using methods from random matrix theory the result suggests that some aspects of large p multivariate distribution theory may be easier to apply in practice than their fixed p counterparts

adaptive estimation of a quadratic functional by model selection we consider the problem of estimating s 2 when s belongs to some separable hilbert space and one observes the gaussian process y t langles t rangle sigmal t for all t epsilon mathbb h where l is some gaussian isonormal process this framework allows us in particular to consider the classical gaussian sequence model for which mathbb h l 2 mathbb n and l t sum lambda geq1 t lambda varepsilon lambda where varepsilon lambda lambda geq1 is a sequence of i i d standard normal variables our approach consists in considering some at most countable families of finite dimensional linear subspaces of mathbb h the models and then using model selection via some conveniently penalized least squares criterion to build new estimators of s 2 we prove a general nonasymptotic risk bound which allows us to show that such penalized estimators are adaptive on a variety of collections of sets for the parameter s depending on the family of models from which they are built in particular in the context of the gaussian sequence model a convenient choice of the family of models allows defining estimators which are adaptive over collections of hyperrectangles ellipsoids l p bodies or besov bodies we take special care to describe the conditions under which the penalized estimator is efficient when the level of noise sigma tends to zero our construction is an alternative to the one by efroimovich and low for hyperrectangles and provides new results otherwise

asymptotic power of sphericity tests for high dimensional data this paper studies the asymptotic power of tests of sphericity against perturbations in a single unknown direction as both the dimensionality of the data and the number of observations go to infinity we establish the convergence under the null hypothesis and the alternative of the log ratio of the joint densities of the sample covariance eigenvalues to a gaussian process indexed by the norm of the perturbation when the perturbation norm is larger than the phase transition threshold studied in baik et al 2005 the limiting process is degenerate and discrimination between the null and the alternative is asymptotically certain when the norm is below the threshold the process is non degenerate so that the joint eigenvalue densities under the null and alternative hypotheses are mutually contiguous using the asymptotic theory of statistical experiments we obtain asymptotic power envelopes and derive the asymptotic power for various sphericity tests in the contiguity region in particular we show that the asymptotic power of the tracy widom type tests is trivial whereas that of the eigenvalue based likelihood ratio test is strictly larger than the size and close to the power envelope

operator norm consistent estimation of large dimensional sparse covariance matrices estimating covariance matrices is a problem of fundamental importance in multivariate statistics in practice it is increasingly frequent to work with data matrices x of dimension n x p where p and n are both large results from random matrix theory show very clearly that in this setting standard estimators like the sample covariance matrix perform in general very poorly in this large n large p setting it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0 and hence this matrix is sparse we develop an estimator to handle this situation the estimator is shown to be consistent in operator norm when for instance we have p n as n oo in other words the largest singular value of the difference between the estimator and the population covariance matrix goes to zero this implies consistency of all the eigenvalues and consistency of eigenspaces associated to isolated eigenvalues we also propose a notion of sparsity for matrices that is compatible with spectral analysis and is independent of the ordering of the variables

large covariance estimation through elliptical factor models we propose a general principal orthogonal complement thresholding poet framework for large scale covariance matrix estimation based on the approximate factor model a set of high level sufficient conditions for the procedure to achieve optimal rates of convergence under different matrix norms is established to better understand how poet works such a framework allows us to recover existing results for sub gaussian data in a more transparent way that only depends on the concentration properties of the sample covariance matrix as a new theoretical contribution for the first time such a framework allows us to exploit conditional sparsity covariance structure for the heavy tailed data in particular for the elliptical distribution we propose a robust estimator based on the marginal and spatial kendall s tau to satisfy these conditions in addition we study conditional graphical model under the same framework the technical tools developed in this paper are of general interest to high dimensional principal component analysis thorough numerical results are also provided to back up the developed theory

high dimensional analysis of semidefinite relaxations for sparse principal components principal component analysis pca is a classical method for dimensionality reduction based on extracting the dominant eigenvectors of the sample covariance matrix however pca is well known to behave poorly in the large p small n setting in which the problem dimension p is comparable to or larger than the sample size n this paper studies pca in this high dimensional regime but under the additional assumption that the maximal eigenvector is sparse say with at most k nonzero components we consider a spiked covariance model in which a base matrix is perturbed by adding a k sparse maximal eigenvector and we analyze two computationally tractable methods for recovering the support set of this maximal eigenvector as follows a a simple diagonal thresholding method which transitions from success to failure as a function of the rescaled sample size dia n p k n k 2 log p k and b a more sophisticated semidefinite programming sdp relaxation which succeeds once the rescaled sample size sdp n p k n klog p k is larger than a critical threshold in addition we prove that no method including the best method which has exponential time complexity can succeed in recovering the support if the order parameter sdp n p k is below a threshold our results thus highlight an interesting trade off between computational and statistical efficiency in high dimensional inference

fixed b subsampling and the block bootstrap improved confidence sets based on p value calibration summary subsampling and block based bootstrap methods have been used in a wide range of inference problems for time series to accommodate the dependence these resampling methods involve a bandwidth parameter such as the subsampling window width and block size in the block based bootstrap in empirical work using different bandwidth parameters could lead to different inference results but traditional first order asymptotic theory does not capture the choice of the bandwidth we propose to adopt the fixed b approach as advocated by kiefer and vogelsang in the heteroscedasticity auto correlation robust testing context to account for the influence of the bandwidth on inference under the fixed b asymptotic framework we derive the asymptotic null distribution of the p values for subsampling and the moving block bootstrap and further propose a calibration of the traditional small b based confidence intervals regions or bands and tests our treatment is fairly general as it includes both finite dimensional parameters and infinite dimensional parameters such as the marginal distribution function simulation results show that the fixed b approach is more accurate than the traditional small b approach in terms of approximating the finite sample distribution and that the calibrated confidence sets tend to have smaller coverage errors than the uncalibrated counterparts

certifying the restricted isometry property is hard this paper is concerned with an important matrix condition in compressed sensing known as the restricted isometry property rip we demonstrate that testing whether a matrix satisfies rip is np hard as a consequence of our result it is impossible to efficiently test for rip provided p np

tensor svd statistical and computational limits in this paper we propose a general framework for tensor singular value decomposition tensor singular value decomposition svd which focuses on the methodology and theory for extracting the hidden low rank structure from high dimensional tensor data comprehensive results are developed on both the statistical and computational limits for tensor svd this problem exhibits three different phases according to the signal to noise ratio snr in particular with strong snr we show that the classical higher order orthogonal iteration achieves the minimax optimal rate of convergence in estimation with weak snr the information theoretical lower bound implies that it is impossible to have consistent estimation in general with moderate snr we show that the non convex maximum likelihood estimation provides optimal solution but with np hard computational cost moreover under the hardness hypothesis of hypergraphic planted clique detection there are no polynomial time algorithms performing consistently in general

phase transitions for detecting latent geometry in random graphs random graphs with latent geometric structure are popular models of social and biological networks with applications ranging from network user profiling to circuit design these graphs are also of purely theoretical interest within computer science probability and statistics a fundamental initial question regarding these models is when are these random graphs affected by their latent geometry and when are they indistinguishable from simpler models without latent structure such as the erd s renyi graph mathcal g n p we address this question for two of the most well studied models of random graphs with latent geometry the random intersection and random geometric graph our results are as follows a the random intersection graph is defined by sampling n random sets s 1 s 2 ldots s n by including each element of a set of size d in each s i independently with probability delta and including the edge i j if s i cap s j ne emptyset we prove that the random intersection graph converges in total variation to an erd s renyi graph if d tilde omega n 3 and does not if d o n 3 for both dense and sparse edge densities p this resolves an open problem in fill et al random struct algorithms 16 2 156 176 2000 rybarczyk random struct algorithms 38 1 2 205 234 2011 and kim et al random struct algorithms 52 4 662 679 2018 the same result was obtained simultaneously and independently by bubeck et al when random intersection graphs lose geometry manuscript 2019 b we strengthen the preceding argument to show that the matrix of random intersection sizes s i cap s j converges in total variation to a symmetric matrix with independent poisson entries this yields the first total variation convergence result for tau random intersection graphs where the edge i j is included if s i cap s j ge tau more precisely our results imply that if p is bounded away from 1 then the tau random intersection graph with edge density p converges to mathcal g n p if d omega tau 3 n 3 c the random geometric graph on mathbb s d 1 is defined by sampling x 1 x 2 dots x n uniformly at random from mathbb s d 1 and including the edge i j if vert x i x j vert 2 le tau a result of bubeck et al random struct algorithms 49 503 532 2016 showed that this model converges to mathcal g n p in total variation where p is chosen so that the models have matching edge densities as long as d omega n 3 it was conjectured in bubeck et al 2016 that this threshold decreases drastically for p small we make the first progress towards this conjecture by showing convergence if d tilde omega left min pn 3 p 2 n 7 2 right our proofs are a hybrid of combinatorial arguments direct couplings and applications of information inequalities previous upper bounds on the total variation distance between random graphs with latent geometry and mathcal g n p have typically not been both combinatorial and information theoretic while this interplay is essential to the sharpness of our bounds

detection of correlations we consider the hypothesis testing problem of deciding whether an observed high dimensional vector has independent normal components or alternatively if it has a small subset of correlated components the correlated components may have a certain combinatorial structure known to the statistician we establish upper and lower bounds for the worst case minimax risk in terms of the size of the correlated subset the level of correlation and the structure of the class of possibly correlated sets we show that some simple tests have near optimal performance in many cases while the generalized likelihood ratio test is suboptimal in some important cases

computational and statistical boundaries for submatrix localization in a large noisy matrix we study in this paper computational and statistical boundaries for submatrix localization given one observation of one or multiple nonoverlapping signal submatrix of magnitude lambda and size k m times k n embedded in a large noise matrix of size m times n the goal is to optimal identify the support of the signal submatrix computationally and statistically two transition thresholds for the signal to noise ratio lambda sigma are established in terms of m n k m and k n the first threshold sf snr c corresponds to the computational boundary we introduce a new linear time spectral algorithm that identifies the submatrix with high probability when the signal strength is above the threshold sf snr c below this threshold it is shown that no polynomial time algorithm can succeed in identifying the submatrix under the hidden clique hypothesis the second threshold sf snr s captures the statistical boundary below which no method can succeed in localization with probability going to one in the minimax sense the exhaustive search method successfully finds the submatrix above this threshold in marked contrast to submatrix detection and sparse pca the results show an interesting phenomenon that sf snr c is always significantly larger than sf snr s under the sub gaussian error model which implies an essential gap between statistical optimality and computational efficiency for submatrix localization

sparse pca optimal rates and adaptive estimation principal component analysis pca is one of the most commonly used statistical procedures with a wide range of applications this paper considers both minimax and adaptive estimation of the principal subspace in the high dimensional setting under mild technical conditions we first establish the optimal rates of convergence for estimating the principal subspace which are sharp with respect to all the parameters thus providing a complete characterization of the difficulty of the estimation problem in term of the convergence rate the lower bound is obtained by calculating the local metric entropy and an application of fano s lemma the rate optimal estimator is constructed using aggregation which however might not be computationally feasible we then introduce an adaptive procedure for estimating the principal subspace which is fully data driven and can be computed efficiently it is shown that the estimator attains the optimal rates of convergence simultaneously over a large collection of the parameter spaces a key idea in our construction is a reduction scheme which reduces the sparse pca problem to a high dimensional multivariate regression problem this method is potentially also useful for other related problems

robust covariance estimation for approximate factor models in this paper we study robust covariance estimation under the approximate factor model with observed factors we propose a novel framework to first estimate the initial joint covariance matrix of the observed data and the factors and then use it to recover the covariance matrix of the observed data we prove that once the initial matrix estimator is good enough to maintain the element wise optimal rate the whole procedure will generate an estimated covariance with desired properties for data with only bounded fourth moment we propose to use adaptive huber loss minimization to give the initial joint covariance estimation this approach is applicable to a much wider class of distributions beyond sub gaussian and elliptical distributions we also present an asymptotic result for adaptive huber s m estimator with a diverging parameter the conclusions are demonstrated by extensive simulations and real data analysis

information theoretic bounds and phase transitions in clustering sparse pca and submatrix localization we study the problem of detecting a structured low rank signal matrix corrupted with additive gaussian noise this includes clustering in a gaussian mixture model sparse pca and submatrix localization each of these problems is conjectured to exhibit a sharp information theoretic threshold below which the signal is too weak for any algorithm to detect we derive upper and lower bounds on these thresholds by applying the first and second moment methods to the likelihood ratio between these planted models and null models where the signal matrix is zero for sparse pca and submatrix localization we determine this threshold exactly in the limit where the number of blocks is large or the signal matrix is very sparse for the clustering problem our bounds differ by a factor of sqrt 2 when the number of clusters is large moreover our upper bounds show that for each of these problems there is a significant regime where reliable detection is information theoretically possible but where known algorithms such as pca fail completely since the spectrum of the observed matrix is uninformative this regime is analogous to the conjectured hard but detectable regime for community detection in sparse graphs

diagnostic checking in multivariate arma models with dependent errors using normalized residual autocorrelations in this paper we derive the asymptotic distribution of normalized residual empirical autocovariances and autocorrelations under weak assumptions on the noise we propose new portmanteau statistics

computational and statistical tradeoffs via convex relaxation modern massive datasets create a fundamental problem at the intersection of the computational and statistical sciences how to provide guarantees on the quality of statistical inference given bounds on computational resources such as time or space our approach to this problem is to define a notion of algorithmic weakening in which a hierarchy of algorithms is ordered by both computational efficiency and statistical efficiency allowing the growing strength of the data at scale to be traded off against the need for sophisticated processing we illustrate this approach in the setting of denoising problems using convex relaxation as the core inferential tool hierarchies of convex relaxations have been widely used in theoretical computer science to yield tractable approximation algorithms to many computationally intractable tasks in the current paper we show how to endow such hierarchies with a statistical characterization and thereby obtain concrete tradeoffs relating algorithmic runtime to amount of data

slic superpixels compared to state of the art superpixel methods computer vision applications have come to rely increasingly on superpixels in recent years but it is not always clear what constitutes a good superpixel algorithm in an effort to understand the benefits and drawbacks of existing methods we empirically compare five state of the art superpixel algorithms for their ability to adhere to image boundaries speed memory efficiency and their impact on segmentation performance we then introduce a new superpixel algorithm simple linear iterative clustering slic which adapts a k means clustering approach to efficiently generate superpixels despite its simplicity slic adheres to boundaries as well as or better than previous methods at the same time it is faster and more memory efficient improves segmentation performance and is straightforward to extend to supervoxel generation

a probabilistic associative model for segmenting weakly supervised images weakly supervised image segmentation is an important yet challenging task in image processing and pattern recognition fields it is defined as in the training stage semantic labels are only at the image level without regard to their specific object scene location within the image given a test image the goal is to predict the semantics of every pixel superpixel in this paper we propose a new weakly supervised image segmentation model focusing on learning the semantic associations between superpixel sets graphlets in this paper in particular we first extract graphlets from each image where a graphlet is a small sized graph measures the potential of multiple spatially neighboring superpixels i e the probability of these superpixels sharing a common semantic label such as the sky or the sea to compare different sized graphlets and to incorporate image level labels a manifold embedding algorithm is designed to transform all graphlets into equal length feature vectors finally we present a hierarchical bayesian network to capture the semantic associations between postembedding graphlets based on which the semantics of each superpixel is inferred accordingly experimental results demonstrate that 1 our approach performs competitively compared with the state of the art approaches on three public data sets and 2 considerable performance enhancement is achieved when using our approach on segmentation based photo cropping and image categorization

spherical cap packing asymptotics and rank extreme detection we study the spherical cap packing problem with a probabilistic approach such probabilistic considerations result in an asymptotic sharp universal uniform bound on the maximal inner product between any set of unit vectors and a stochastically independent uniformly distributed unit vector when the set of unit vectors are themselves independently uniformly distributed we further develop the extreme value distribution limit of the maximal inner product which characterizes its uncertainty around the bound as applications of the above mentioned asymptotic results we derive 1 an asymptotic sharp universal uniform bound on the maximal spurious correlation as well as its uniform convergence in distribution when the explanatory variables are independently gaussian distributed and 2 an asymptotic sharp universal bound on the maximum norm of a low rank elliptically distributed vector as well as related limiting distributions with these results we develop a fast detection method for a low rank structure in high dimensional gaussian data without using the spectrum information

interactive cosegmentation using global and local energy optimization we propose a novel interactive cosegmentation method using global and local energy optimization the global energy includes two terms 1 the global scribbled energy and 2 the interimage energy the first one utilizes the user scribbles to build the gaussian mixture model and improve the cosegmentation performance the second one is a global constraint which attempts to match the histograms of common objects to minimize the local energy we apply the spline regression to learn the smoothness in a local neighborhood this energy optimization can be converted into a constrained quadratic programming problem to reduce the computational complexity we propose an iterative optimization algorithm to decompose this optimization problem into several subproblems the experimental results show that our method outperforms the state of the art unsupervised cosegmentation and interactive cosegmentation methods on the icoseg and msrc benchmark data sets

detection of an anomalous cluster in a network we consider the problem of detecting whether or not in a given sensor network there is a cluster of sensors which exhibit an unusual behavior formally suppose we are given a set of nodes and attach a random variable to each node we observe a realization of this process and want to decide between the following two hypotheses under the null the variables are i i d standard normal under the alternative there is a cluster of variables that are i i d normal with positive mean and unit variance while the rest are i i d standard normal we also address surveillance settings where each sensor in the network collects information over time the resulting model is similar now with a time series attached to each node we again observe the process over time and want to decide between the null where all the variables are i i d standard normal and the alternative where there is an emerging cluster of i i d normal variables with positive mean and unit variance the growth models used to represent the emerging cluster are quite general and in particular include cellular automata used in modelling epidemics in both settings we consider classes of clusters that are quite general for which we obtain a lower bound on their respective minimax detection rate and show that some form of scan statistic by far the most popular method in practice achieves that same rate within a logarithmic factor our results are not limited to the normal location model but generalize to any one parameter exponential family when the anoma lous clusters are large enough

minimax bounds for sparse pca with noisy high dimensional data we study the problem of estimating the leading eigenvectors of a high dimensional population covariance matrix based on independent gaussian observations we establish a lower bound on the minimax risk of estimators under the l2 loss in the joint limit as dimension and sample size increase to infinity under various models of sparsity for the population eigenvectors the lower bound on the risk points to the existence of different regimes of sparsity of the eigenvectors we also propose a new method for estimating the eigenvectors by a two stage coordinate selection scheme

optimal rates of convergence for covariance matrix estimation covariance matrix plays a central role in multivariate statistical analysis significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices however a minimax theory has yet been developed in this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and frobenius norm it is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation the minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties a key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound the technical analysis requires new ideas that are quite different from those used in the more conventional function sequence estimation problems

shape matching and object recognition using shape contexts we present a novel approach to measuring similarity between shapes and exploit it for object recognition in our framework the measurement of similarity is preceded by 1 solving for correspondences between points on the two shapes 2 using the correspondences to estimate an aligning transform in order to solve the correspondence problem we attach a descriptor the shape context to each point the shape context at a reference point captures the distribution of the remaining points relative to it thus offering a globally discriminative characterization corresponding points on two similar shapes will have similar shape contexts enabling us to solve for correspondences as an optimal assignment problem given the point correspondences we estimate the transformation that best aligns the two shapes regularized thin plate splines provide a flexible class of transformation maps for this purpose the dissimilarity between the two shapes is computed as a sum of matching errors between corresponding points together with a term measuring the magnitude of the aligning transform we treat recognition in a nearest neighbor classification framework as the problem of finding the stored prototype shape that is maximally similar to that in the image results are presented for silhouettes trademarks handwritten digits and the coil data set

how hard is it to approximate the best nash equilibrium the quest for a polynomial time approximation scheme ptas for nash equilibrium in a two player game which emerged as a major open question in algorithmic game theory seeks to circumvent the ppad completeness of finding an exact nash equilibrium by finding an approximate equilibrium the closely related problem of finding an equilibrium maximizing a certain objective such as social welfare was shown to be np hard gilboa and zemel games econom behav 1 1989 pp 80 93 however this np hardness is unlikely to extend to approximate equilibria since the latter admits a quasi polynomial time algorithm lipton markakis and mehta in proceedings of the 4th acm conference on electronic commerce acm new york 2003 pp 36 41 we show that this optimization problem namely finding in a two player game an approximate equilibrium achieving a large social welfare is unlikely to have a polynomial time algorithm one interpretation of our results is that a ptas for nash equilibrium if it exists should not extend to a ptas for finding the best nash equilibrium technically our result is a reduction from the notoriously difficult problem in modern combinatorics of finding a planted but hidden clique in a random graph g n 1 2 our reduction starts from an instance with planted clique size o log n for comparison the currently known algorithms are effective only for a much larger clique size omega sqrt n

nuclear norm minimization for the planted clique and biclique problems we consider the problems of finding a maximum clique in a graph and finding a maximum edge biclique in a bipartite graph both problems are np hard we write both problems as matrix rank minimization and then relax them using the nuclear norm this technique which may be regarded as a generalization of compressive sensing has recently been shown to be an effective way to solve rank optimization problems in the special case that the input graph has a planted clique or biclique i e a single large clique or biclique plus diversionary edges our algorithm successfully provides an exact solution to the original instance for each problem we provide two analyses of when our algorithm succeeds in the first analysis the diversionary edges are placed by an adversary in the second they are placed at random in the case of random edges for the planted clique problem we obtain the same bound as alon krivelevich and sudakov as well as feige and krauthgamer but we use different techniques

on combinatorial testing problems we study a class of hypothesis testing problems in which upon observing the realization of an n dimensional gaussian vector one has to decide whether the vector was drawn from a standard normal distribution or alternatively whether there is a subset of the components belonging to a certain given class of sets whose elements have been contaminated that is have a mean different from zero we establish some general conditions under which testing is possible and others under which testing is hopeless with a small risk the combinatorial and geometric structure of the class of sets is shown to play a crucial role the bounds are illustrated on various examples

on the limit of the largest eigenvalue of the large dimensional sample covariance matrix in this paper the authors show that the largest eigenvalue of the sample covariance matrix tends to a limit under certain conditions when both the number of variables and the sample size tend to infinity the above result is proved under the mild restriction that the fourth moment of the elements of the sample sums of squares and cross products sp matrix exist

higher criticism for detecting sparse heterogeneous mixtures higher criticism or second level significance testing is a multiplecomparisons concept mentioned in passing by tukey it concerns a situation where there are many independent tests of significance and one is interested in rejecting the joint null hypothesis tukey suggested comparing the fraction of observed significances at a given level to the expected fraction under the joint null in fact he suggested standardizing the difference of the two quantities and forming a z score the resulting z score tests the significance of the body of significance tests we consider a generalization where we maximize this z score over a range of significance levels 0 0 we are able to show that the resulting higher criticism statistic is effective at resolving a very subtle testing problem testing whether n normal means are all zero versus the alternative that a small fraction is nonzero the subtlety of this sparse normal means testing problem can be seen from work of ingster and jin who studied such problems in great detail in their studies they identified an interesting range of cases where the small fraction of nonzero means is so small that the alternative hypothesis exhibits little noticeable effect on the distribution of the p values either for the bulk of the tests or for the few most highly significant tests in this range when the amplitude of nonzero means is calibrated with the fraction of nonzero means the likelihood ratio test for a precisely specified alternative would still succeed in separating the two hypotheses we show that the higher criticism is successful throughout the same region of amplitude sparsity where the likelihood ratio test

an augmented lagrangian approach for sparse principal component analysis principal component analysis pca is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering however the standard pca suffers from the fact that the principal components pcs are usually linear combinations of all the original variables and it is thus often difficult to interpret the pcs to alleviate this drawback various sparse pca approaches were proposed in the literature cadima and jolliffe in j appl stat 22 203 214 1995 d aspremont et al in j mach learn res 9 1269 1294 2008 d aspremont et al siam rev 49 434 448 2007 jolliffe in j appl stat 22 29 35 1995 journee et al in j mach learn res 11 517 553 2010 jolliffe et al in j comput graph stat 12 531 547 2003 moghaddam et al in advances in neural information processing systems 18 915 922 mit press cambridge 2006 shen and huang in j multivar anal 99 6 1015 1034 2008 zou et al in j comput graph stat 15 2 265 286 2006 despite success in achieving sparsity some important properties enjoyed by the standard pca are lost in these methods such as uncorrelation of pcs and orthogonality of loading vectors also the total explained variance that they attempt to maximize can be too optimistic in this paper we propose a new formulation for sparse pca aiming at finding sparse and nearly uncorrelated pcs with orthogonal loading vectors while explaining as much of the total variance as possible we also develop a novel augmented lagrangian method for solving a class of nonsmooth constrained optimization problems which is well suited for our formulation of sparse pca we show that it converges to a feasible point and moreover under some regularity assumptions it converges to a stationary point additionally we propose two nonmonotone gradient methods for solving the augmented lagrangian subproblems and establish their global and local convergence finally we compare our sparse pca approach with several existing methods on synthetic zou et al in j comput graph stat 15 2 265 286 2006 pitprops jeffers in appl stat 16 225 236 1967 and gene expression data chin et al in cancer cell 10 529c 541c 2006 respectively the computational results demonstrate that the sparse pcs produced by our approach substantially outperform those by other methods in terms of total explained variance correlation of pcs and orthogonality of loading vectors moreover the experiments on random data show that our method is capable of solving large scale problems within a reasonable amount of time

weakly supervised part proposal segmentation from multiple images weakly supervised local part segmentation is challenging due to the difficulty of modeling multiple local parts from image level prior in this paper we propose a new weakly supervised local part proposal segmentation method based on the observation that local parts will keep fixed along the object pose variations hence the local part can be segmented by capturing object pose variations based on such observation a new local part proposal segmentation model is proposed three aspects such as shape similarity based cosegmentation shape matching based part detection and segmentation and graph matching based part assignment are considered a part segmentation energy function is first proposed four terms such as mrf based single image segmentation term shape feature based foreground consistency term ncuts based part segmentation term and two order graphs matching based part consistency term are contained then a three sub minimization based energy minimization method is proposed to accomplish approximation solution finally we verify our method based on three image data sets pascal voc 2008 part data set ucb bird data set and cat dog data set and one video data set ucf sports data set the experimental results demonstrate a better segmentation performance compared with the existing object cosegmentation and part proposal generation methods

sparse principal component analysis and iterative thresholding principal component analysis pca is a classical dimension reduction method which projects data onto the principal subspace spanned by the leading eigenvectors of the covariance matrix however it behaves poorly when the number of features p is comparable to or even much larger than the sample size n in this paper we propose a new iterative thresholding approach for estimating principal subspaces in the setting where the leading eigenvectors are sparse under a spiked covariance model we find that the new approach recovers the principal subspace and leading eigenvectors consistently and even optimally in a range of high dimensional sparse settings simulated examples also demonstrate its competitive performance

spectrum estimation for large dimensional covariance matrices using random matrix theory estimating the eigenvalues of a population covariance matrix from a sample covariance matrix is a problem of fundamental importance in multivariate statistics the eigenvalues of covariance matrices play a key role in many widely used techniques in particular in principal component analysis pca in many modern data analysis problems statisticians are faced with large datasets where the sample size n is of the same order of magnitude as the number of variables p random matrix theory predicts that in this context the eigenvalues of the sample covariance matrix are not good estimators of the eigenvalues of the population covariance we propose to use a fundamental result in random matrix theory the marcenko pastur equation to better estimate the eigenvalues of large dimensional covariance matrices the marcenko pastur equation holds in very wide generality and under weak assumptions the estimator we obtain can be thought of as shrinking in a nonlinear fashion the eigenvalues of the sample covariance matrix to estimate the population eigenvalues inspired by ideas of random matrix theory we also suggest a change of point of view when thinking about estimation of high dimensional vectors we do not try to estimate directly the vectors but rather a probability measure that describes them we think this is a theoretically more fruitful way to think about these problems our estimator gives fast and good or very good results in extended simulations our algorithmic approach is based on convex optimization we also show that the proposed estimator is consistent

the largest eigenvalues of sample covariance matrices for a spiked population diagonal case we consider large complex random sample covariance matrices obtained from spiked populations that is when the true covariance matrix is diagonal with all but finitely many eigenvalues equal to 1 we investigate the limiting behavior of the largest eigenvalues when the population and the sample sizes both become large under some conditions on moments of the sample distribution we prove that the asymptotic fluctuations of the largest eigenvalues are the same as for a complex gaussian sample with the same true covariance the real setting is also considered

finite sample approximation results for principal component analysis a matrix perturbation approach principal component analysis pca is a standard tool for dimensional reduction of a set of n observations samples each with p variables in this paper using a matrix perturbation approach we study the nonasymptotic relation between the eigenvalues and eigenvectors of pca computed on a finite sample of size n and those of the limiting population pca as n oo as in machine learning we present a finite sample theorem which holds with high probability for the closeness between the leading eigenvalue and eigenvector of sample pca and population pca under a spiked covariance model in addition we also consider the relation between finite sample pca and the asymptotic results in the joint limit p n with p n c we present a matrix perturbation view of the phase transition phenomenon and a simple linear algebra based derivation of the eigenvalue and eigenvector overlap in this asymptotic limit moreover our analysis also applies for finite p n where we show that although there is no sharp phase transition as in the infinite case either as a function of noise level or as a function of sample size n the eigenvector of sample pca may exhibit a sharp loss of tracking suddenly losing its relation to the true eigenvector of the population pca matrix this occurs due to a crossover between the eigenvalue due to the signal and the largest eigenvalue due to noise whose eigenvector points in a random direction