information theoretic determination of minimax rates of convergence we present some general results determining minimax bounds on statistical risk for density estimation based on certain information theoretic considerations these bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence

covariance regularization by thresholding this paper considers regularizing a covariance matrix of p variables estimated from n observations by hard thresholding we show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense the variables are gaussian or sub gaussian and log p n 0 and obtain explicit rates the results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity we discuss an intuitive resampling scheme for threshold selection and prove a general cross validation result that justifies this approach we also compare thresholding to other covariance estimators in simulations and on an example from climate data 1 introduction estimation of covariance matrices is important in a number of areas of statistical analysis including dimension reduction by principal component analysis pca classification by linear or quadratic discriminant analysis lda and qda establishing independence and conditional independence relations in the context of graphical models and setting confidence intervals on linear functions of the means of the components in recent years many application areas where these tools are used have been dealing with very high dimensional datasets and sample sizes can be very small relative to dimension examples include genetic data brain imaging spectroscopic imaging climate data and many others it is well known by now that the empirical covariance matrix for samples of size n from a p variate gaussian distribution np p is not a good estimator of the population covariance if p is large many results in random matrix theory illustrate this from the classical mary law 29 to the more recent work of johnstone and his students on the theory of the largest eigenvalues 12 23 30 and associated eigenvectors 24 however with the exception of a method for estimating the covariance spectrum 11 these probabilistic results do not offer alternatives to the sample covariance matrix alternative estimators for large covariance matrices have therefore attracted a lot of attention recently two broad classes of covariance estimators have emerged those that rely on a natural ordering among variables and assume that variables

a nonlinear lower bound on the practical combinational complexity an infinite sequence f fn n 1 of one output boolean functions with the following three properties is constructed 1 f n can be computed by a boolean circuit with o n gates 2 for any positive nondecreasing and unbounded function h n r each boolean circuit having an n h n separator requires nonlinear number of gates to compute f n 3 each planar boolean circuit requires n2 gates to compute f n

the multiplicative complexity of quadratic boolean forms abstract let the multiplicative complexity l f of a boolean function f be the minimal number of gates that are sufficient to evaluate f by circuits over the basis 1 we characterize the multiplicative complexity l f of quadratic boolean forms f it is half the rank of an associated matrix two quadratic boolean forms f g have the same complexity l f l g iff they are isomorphic by a linear isomorphism we characterize computational independence of two quadratic boolean forms f 1 f 2 in the sense that l f 1 f 2 l f 1 l f 2

gate elimination circuit size lower bounds and sat upper bounds abstract most of the known lower bounds for binary boolean circuits with unrestricted depth are proved by the gate elimination method the most efficient known algorithms for the sat problem on binary boolean circuits use similar case analyses to the ones in gate elimination chen and kabanets recently showed that the known case analyses can also be used to prove average case circuit lower bounds that is lower bounds on the size of approximations of an explicit function in this paper we provide a general framework for proving worst average case lower bounds for circuits and upper bounds for sat that is built on ideas of chen and kabanets a proof in such a framework goes as follows one starts by fixing three parameters a class of circuits a circuit complexity measure and a set of allowed substitutions the main ingredient of a proof goes as follows by going through a number of cases one shows that for any circuit from the given class one can find an allowed substitution such that the given measure of the circuit reduces by a sufficient amount this case analysis immediately implies an upper bound for sat to obtain worst average case circuit complexity lower bounds one needs to present an explicit construction of a function that is a disperser extractor for the class of sources defined by the set of substitutions under consideration we show that many known proofs of circuit size lower bounds and upper bounds for sat fall into this framework using this framework we prove the following new bounds average case lower bounds of 3 24 n and 2 59 n for circuits over u 2 and b 2 respectively though the lower bound for the basis b 2 is given for a quadratic disperser whose explicit construction is not currently known and faster than 2 n sat algorithms for circuits over u 2 and b 2 of size at most 3 24 n and 2 99 n respectively here by b 2 we mean the set of all bivariate boolean functions and by u 2 the set of all bivariate boolean functions except for parity and its complement

optimal selection of reduced rank estimators of high dimensional matrices we introduce a new criterion the rank selection criterion rsc for selecting the optimal reduced rank estimator of the coefficient matrix in multivariate response regression models the corresponding rsc estimator minimizes the frobenius norm of the fit plus a regularization term proportional to the number of parameters in the reduced rank model the rank of the rsc estimator provides a consistent estimator of the rank of the coefficient matrix in general the rank of our estimator is a consistent estimate of the effective rank which we define to be the number of singular values of the target matrix that are appropriately large the consistency results are valid not only in the classic asymptotic regime when n the number of responses and p the number of predictors stay bounded and m the number of observations grows but also when either or both n and p grow possibly much faster than m we establish minimax optimal bounds on the mean squared errors of our estimators our finite sample performance bounds for the rsc estimator show that it achieves the optimal balance between the approximation error and the penalty term furthermore our procedure has very low computational complexity linear in the number of candidate models making it particularly appealing for large scale problems we contrast our estimator with the nuclear norm penalized least squares nnp estimator which has an inherently higher computational complexity than rsc for multivariate regression models we show that nnp has estimation properties similar to those of rsc albeit under stronger conditions however it is not as parsimonious as rsc we offer a simple correction of the nnp estimator which leads to consistent rank estimation we verify and illustrate our theoretical findings via an extensive simulation study

nonlinear lower bounds on the number of processors of circuits with sublinear separators it is proved that each boolean circuit with o na separator for an a 1 must have n1a processors to compute some specific one output boolean functions the above stated result holds also for unbounded fan in fan out boolean circuits a nonlinear lower bound on the number of processors is achieved also for planar vlsi circuits computing some one output boolean functions in time o nb for b 12

optimal rates of convergence for sparse covariance matrix estimation this paper considers estimation of sparse covariance matrices and establishes the optimal rate of convergence under a range of matrix operator norm and bregman divergence losses a major focus is on the derivation of a rate sharp minimax lower bound the problem exhibits new features that are significantly different from those that occur in the conventional nonparametric function estimation problems standard techniques fail to yield good results and new tools are thus needed we first develop a lower bound technique that is particularly well suited for treating two directional problems such as estimating sparse covariance matrices the result can be viewed as a generalization of le cam s method in one direction and assouad s lemma in another this lower bound technique is of independent interest and can be used for other matrix estimation problems we then establish a rate sharp minimax lower bound for estimating sparse covariance matrices under the spectral norm by applying the general lower bound technique a thresholding estimator is shown to attain the optimal rate of convergence under the spectral norm the results are then extended to

evolution equations for phase separation and ordering in binary alloys we explore two phenomenological approaches leading to systems of coupled cahn hilliard and cahn allen equations for describing the dynamics of systems which can undergo first order phase separation and order disorder transitions simultaneously starting from the same discrete lattice free energy function in the first approach a quasicontinuum limit is taken for this discrete energy and the evolution of the system is then assumed to be given by gradient flow in the second approach a discrete set of gradient flow evolution equations is derived for the lattice dynamics and a quasicontinuum limit is then taken we demonstrate in the context of bcc fe al binary alloys that it is important that variables be chosen that accommodate the variations in the average concentration as well as the underlying ordered structure of the possible coexistent phases only then will the two approaches lead to roughly the same continuum descriptions we conjecture that in general the number of variables necessary to describe the dynamics of such systems is equal ton 1 n 2 1 wheren 1 is given by the dimension of the span of the bases of the irreducible representations needed to describe the symmetry groups of the possible equilibrium phases andn 2 is the number of chemical components n 1 of these variables are nonconserved and the remaining are conserved and represent the average concentrations for the fe al alloys this implies a description of one conserved order parameter and one nonconserved order parameter the resultant description is given by a cahn hilliard equation coupled to a cahn allen equation via the lower order nonlinear terms the rough equivalence of the two phenomenological methods adds credibility to the validity of the resulting evolution equations a similar description should also be valid for alloy systems in which the structure of the competing phases is more complicated

correlation bounds and sat algorithms for small linear size circuits we revisit the gate elimination method generalize it to prove correlation bounds of boolean circuits with parity and also derive deterministic satisfiability counting algorithms for small linear size circuits let b2 be the full binary basis and let u2 b2 we prove that for circuits over u2 of size 3nn for any constant 0 5 the correlation with parity is at most 2n 1 and there is a sat algorithm which counts the number of satisfying assignments running in time 2nn 1 for circuit size 3nn for 0 the correlation with parity is at most 2 n and there is a sat algorithm running in time 2n n similar correlation bounds and algorithms are also proved for circuits over b2 of size almost 2 5n

operator norm consistent estimation of large dimensional sparse covariance matrices estimating covariance matrices is a problem of fundamental importance in multivariate statistics in practice it is increasingly frequent to work with data matrices x of dimension n x p where p and n are both large results from random matrix theory show very clearly that in this setting standard estimators like the sample covariance matrix perform in general very poorly in this large n large p setting it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0 and hence this matrix is sparse we develop an estimator to handle this situation the estimator is shown to be consistent in operator norm when for instance we have p n as n oo in other words the largest singular value of the difference between the estimator and the population covariance matrix goes to zero this implies consistency of all the eigenvalues and consistency of eigenspaces associated to isolated eigenvalues we also propose a notion of sparsity for matrices that is compatible with spectral analysis and is independent of the ordering of the variables

a boolean function requiring 3n network size abstract paul 1977 has proved a 2 5 n lower bound for the network complexity of an explicit boolean function we modify the definition of paul s function slightly and prove a 3 n lower bound for the network complexity of that function

approximation dans les espaces metriques et theorie de l estimation we investigate the relations between the speed of estimation and the metric structure of the parameter space especially in the case when its metric dimension is infinite given some distance d on generally hellinger distance in the case of n i i d variables we consider the minimax risk for n observations r n q mathop inf sup limits t n theta in theta text ie theta d q theta t n t n being any estimate of we shall look for functions r such that for positive constants c 1 q and c 2 q c 1 r q n r n q c2 r q n r n is the speed of estimation and we shall show under fairly general conditions including i i d variables and regular cases of markov chains and stationnary gaussian processes that r n is determined up to multiplicative constants by the metric structure of we shall also give a construction for some sort of universal estimates the risk of which is bounded by c 2 r q n in all cases where the preceding theory applies

regularized estimation of large covariance matrices this paper considers estimating a covariance matrix of p variables from n observations by either banding the sample covariance matrix or estimating a banded version of the inverse of the covariance we show that these estimates are consistent in the operator norm as long as logp 2 n 0 and obtain explicit rates the results are uniform over some fairly natural well conditioned families of covariance matrices we also introduce an analogue of the gaussian white noise model and show that if the population covariance is embeddable in that model and well conditioned then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix the results can be extended to smooth versions of banding and to non gaussian distributions with su ciently short tails a resampling approach is proposed for choosing the banding parameter in practice this approach is illustrated numerically on both simulated and real data

sparse pca optimal rates and adaptive estimation principal component analysis pca is one of the most commonly used statistical procedures with a wide range of applications this paper considers both minimax and adaptive estimation of the principal subspace in the high dimensional setting under mild technical conditions we first establish the optimal rates of convergence for estimating the principal subspace which are sharp with respect to all the parameters thus providing a complete characterization of the difficulty of the estimation problem in term of the convergence rate the lower bound is obtained by calculating the local metric entropy and an application of fano s lemma the rate optimal estimator is constructed using aggregation which however might not be computationally feasible we then introduce an adaptive procedure for estimating the principal subspace which is fully data driven and can be computed efficiently it is shown that the estimator attains the optimal rates of convergence simultaneously over a large collection of the parameter spaces a key idea in our construction is a reduction scheme which reduces the sparse pca problem to a high dimensional multivariate regression problem this method is potentially also useful for other related problems

the complexity of central slice functions let f be a monotone boolean function over x x1 xn the k slice of f is the function fk f tkn vtk 1n where tkn is the kth threshold function berkowitz has shown that sufficiently large superlinear lower bounds on the monotone network complexity of kk imply lower bounds of the same order on the combinational complexity of f and that if f has large combinational complexity then some slice of f must have large monotone complexity however this latter result does not specify any particular slice and it is known that some nontrivial slice functions of np complete predicates have linear complexity in this paper we consider the 12n slice functions and show that for three basic monotone boolean np complete functions this slice is also np complete in addition the 12n slice of some boolean matrix functions f is studied and is proved to be no easier to compute than f

platelets a multiscale approach for recovering edges and surfaces in photon limited medical imaging the nonparametric multiscale platelet algorithms presented in this paper unlike traditional wavelet based methods are both well suited to photon limited medical imaging applications involving poisson data and capable of better approximating edge contours this paper introduces platelets localized functions at various scales locations and orientations that produce piecewise linear image approximations and a new multiscale image decomposition based on these functions platelets are well suited for approximating images consisting of smooth regions separated by smooth boundaries for smoothness measured in certain holder classes it is shown that the error of m term platelet approximations can decay significantly faster than that of m term approximations in terms of sinusoids wavelets or wedgelets this suggests that platelets may outperform existing techniques for image denoising and reconstruction fast platelet based maximum penalized likelihood methods for photon limited image denoising deblurring and tomographic reconstruction problems are developed because platelet decompositions of poisson distributed images are tractable and computationally efficient existing image reconstruction methods based on expectation maximization type algorithms can be easily enhanced with platelet techniques experimental results suggest that platelet based methods can outperform standard reconstruction methods currently in use in confocal microscopy image restoration and emission tomography

lower bounds for the minimax risk using f divergences and applications lower bounds involving f divergences between the underlying probability measures are proved for the minimax risk in estimation problems our proofs just use simple convexity facts special cases and straightforward corollaries of our bounds include well known inequalities for establishing minimax lower bounds such as fano s inequality pinsker s inequality and inequalities based on global entropy conditions two applications are provided a new minimax lower bound for the reconstruction of convex bodies from noisy support function measurements and a different proof of a recent minimax lower bound for the estimation of a covariance matrix

optimal rates of convergence for covariance matrix estimation covariance matrix plays a central role in multivariate statistical analysis significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices however a minimax theory has yet been developed in this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and frobenius norm it is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation the minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties a key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound the technical analysis requires new ideas that are quite different from those used in the more conventional function sequence estimation problems

minimax risk over l p balls for l p error consider estimating the mean vector from datan n 2 i withl q norm loss q 1 when is known to lie in ann dimensionall p ball p 0 for largen the ratio of minimaxlinear risk to minimax risk can bearbitrarily large ifp q obvious exceptions aside the limiting ratio equals 1 only ifp q 2 our arguments are mostly indirect involving a reduction to a univariate bayes minimax problem whenp q simple non linear co ordinatewise threshold rules are asymptotically minimax at small signal to noise ratios and within a bounded factor of asymptotic minimaxity in general we also give asymptotic evaluations of the minimax linear risk our results are basic to a theory of estimation in besov spaces using wavelet bases to appear elsewhere

rate optimal denoising of simultaneously sparse and low rank matrices we study minimax rates for denoising simultaneously sparse and low rank matrices in high dimensions we show that an iterative thresholding algorithm achieves near optimal rates adaptively under mild conditions for a large class of loss functions numerical experiments on synthetic datasets also demonstrate the competitive performance of the proposed method

lower bounds for depth restricted branching programs abstract we present a new method for proving lower bounds on the complexity of branching programs and consider k times only branching programs while exponential and nearly exponential lower bounds on the complexity of one time only branching programs were proved for many problems there are still missing methods of proving lower bounds for k times only programs k 1 we prove exponential lower bounds for k times only branching programs which have the additional restriction that the input bits are read k times yet blockwise and in each block in the same order this is done both for the algebraic decision problem polyn d n n prime d n whether a given mapping g f n f n is a polynomial over f n of degree at most d and for the corresponding monotone problem over quadratic boolean matrices as a consequence we obtain a sharp bound of order n log n on the communication complexity of polyn n 0 1 2

finite element approximation of a degenerate allen cahn cahn hilliard system we consider a fully practical finite element approximation of an allen cahn cahn hilliard system with a degenerate mobility and a logarithmic free energy this system arises in the modeling of phase separation and ordering in binary alloys in addition to showing well posedness and stability bounds for our approximation we prove convergence in one space dimension finally some numerical experiments are presented

pattern formations and spatial entropy for spatially discrete diffusion equations formation of mosaic patterns for spatially discrete diffusion equations with cubic nonlinearity is investigated we construct feasible basic patterns in each parameter region and combine these basic patterns into large patterns on one and two dimensional lattices the basic patterns are characterized and constructed through formulating parameter conditions based on a geometrical setting spatial entropy associated with these patterns are computed or estimated we also consider three typical boundary conditions and investigate their influences on pattern formations and spatial entropy several numerical computations are performed to illustrate such a formation of patterns

volume ratio sparsity and minimaxity under unitarily invariant norms this paper studies non asymptotic minimax estimation of high dimensional matrices and provides tight minimax rates for a large collection of loss functions in a variety of problems via information theoretic methods based on the convex geometry of finite dimensional banach spaces we first develop a volume ratio approach for determining minimax estimation rates of unconstrained mean matrices under all unitarily invariant norm losses which turn out to only depend on the norm of identity matrix in addition we establish the minimax rates for estimating normal mean matrices with submatrix sparsity where the sparsity constraint introduces an additional term in the rate which in contrast to the unconstrained case is determined by the smoothness lipschitz constant of the norm this method is also applicable to the low rank matrix completion problem and extends well beyond the additive noise model in particular it yields tight rates in covariance matrix estimation and poisson rate matrix estimation problems for all unitarily invariant norms

mutual information metric entropy and cumulative relative entropy risk assume p theta theta epsilon theta is a set of probability distributions with a common dominating measure on a complete separable metric space y a state theta epsilon theta is chosen by nature a statistician obtains n independent observations y 1 dots y n from y distributed according to p theta for each time t between 1 and n based on the observations y 1 dots y t 1 the statistician produces an estimated distribution hat p t for p theta and suffers a loss l p theta hat p t the cumulative risk for the statistician is the average total loss up to time n of special interest in information theory data compression mathematical finance computational learning theory and statistical mechanics is the special case when the loss l p theta hat p t is the relative entropy between the true distribution p theta and the estimated distribution hat p t here the cumulative bayes risk from time 1 to n is the mutual information between the random parameter theta and the observations y 1 dots y n new bounds on this mutual information are given in terms of the laplace transform of the hellinger distance between pairs of distributions indexed by parameters in theta from these bounds on the cumulative minimax risk are given in terms of the metric entropy of theta with respect to the hellinger distance the assumptions required for these bounds are very general and do not depend on the choice of the dominating measure they apply to both finite and infinite dimensional theta they apply in some cases where y is infinite dimensional in some cases where y is not compact in some cases where the distributions are not smooth and in some parametric cases where asymptotic normality of the posterior distribution fails

wavelets ridgelets and curvelets for poisson noise removal in order to denoise poisson count data we introduce a variance stabilizing transform vst applied on a filtered discrete poisson process yielding a near gaussian process with asymptotic constant variance this new transform which can be deemed as an extension of the anscombe transform to filtered data is simple fast and efficient in very low count situations we combine this vst with the filter banks of wavelets ridgelets and curvelets leading to multiscale vsts ms vsts and nonlinear decomposition schemes by doing so the noise contaminated coefficients of these ms vst modified transforms are asymptotically normally distributed with known variances a classical hypothesis testing framework is adopted to detect the significant coefficients and a sparsity driven iterative scheme reconstructs properly the final estimate a range of examples show the power of this ms vst approach for recovering important structures of various morphologies in very low count images these results also demonstrate that the ms vst approach is competitive relative to many existing denoising methods

mosiac solutions and entropy for discrete coupled phase transition equations abstract we consider arrays of coupled scalar differential equations organized on a spatial lattice one component of this system is analogous to the allen cahn partial differential equation which on the integer lattice has interactions of nearest neighbor type and the other to the cahn hilliard partial differential equation which has interactions of nearest and next nearest neighbor type our coupling functions are forms of the so called double obstacle nonlinearity the interaction strengths of both equations are not restricted in magnitude or in sign and need not be near a continuum limit we prove existence and uniqueness results for the initial value problem and consider the existence and stability of a class of equilibrium solutions called mosaic solutions these equilibrium solutions take only the values 1 1 and 0 at each lattice point using the notion of a weakly forward invariant set we provide criteria for weak lyapunov and weak asymptotic stability rigorous results are then obtained for the spatial entropy of these stable mosaic solutions and it is shown that the existence and stability results obtained on the integer lattice can be used to obtain similar results on an arbitrary lattice numerical results are presented that illustrate the importance of the analytical results

computational barriers in minimax submatrix detection this paper studies the minimax detection of a small submatrix of elevated mean in a large matrix contaminated by additive gaussian noise to investigate the tradeoff between statistical performance and computational cost from a complexity theoretic perspective we consider a sequence of discretized models which are asymptotically equivalent to the gaussian model under the hypothesis that the planted clique detection problem cannot be solved in randomized polynomial time when the clique size is of smaller order than the square root of the graph size the following phase transition phenomenon is established when the size of the large matrix p to infty if the submatrix size k theta p alpha for any alpha in 0 2 3 computational complexity constraints can incur a severe penalty on the statistical performance in the sense that any randomized polynomial time test is minimax suboptimal by a polynomial factor in p if k theta p alpha for any alpha in 2 3 1 minimax optimal detection can be attained within constant factors in linear time using schatten norm loss as a representative example we show that the hardness of attaining the minimax estimation rate can crucially depend on the loss function implications on the hardness of support recovery are also obtained

minimax estimation in sparse canonical correlation analysis canonical correlation analysis is a widely used multivariate statistical technique for exploring the relation between two sets of variables this paper considers the problem of estimating the leading canonical correlation directions in high dimensional settings recently under the assumption that the leading canonical correlation directions are sparse various procedures have been proposed for many high dimensional applications involving massive data sets however there has been few theoretical justification available in the literature in this paper we establish rate optimal nonasymptotic minimax estimation with respect to an appropriate loss function for a wide range of model spaces two interesting phenomena are observed first the minimax rates are not affected by the presence of nuisance parameters namely the covariance matrices of the two sets of random variables though they need to be estimated in the canonical correlation analysis problem second we allow the presence of the residual canonical correlation directions however they do not influence the minimax rates under a mild condition on eigengap a generalized sin theta theorem and an empirical process bound for gaussian quadratic forms under rank constraint are used to establish the minimax upper bounds which may be of independent interest

optimal estimation of low rank density matrices the density matrices are positively semi definite hermitian matrices of unit trace that describe the state of a quantum system the goal of the paper is to develop minimax lower bounds on error rates of estimation of low rank density matrices in trace regression models used in quantum state tomography in particular in the case of pauli measurements with explicit dependence of the bounds on the rank and other complexity parameters such bounds are established for several statistically relevant distances including quantum versions of kullback leibler divergence relative entropy distance and of hellinger distance so called bures distance and schatten p norm distances sharp upper bounds and oracle inequalities for least squares estimator with von neumann entropy penalization are obtained showing that minimax lower bounds are attained up to logarithmic factors for these distances