minimax estimation of linear and quadratic functionals on sparsity classes for the gaussian sequence model we obtain nonasymptotic minimax rates of estimation of the linear quadratic and the ell 2 norm functionals on classes of sparse vectors and construct optimal estimators that attain these rates the main object of interest is the class b 0 s of s sparse vectors theta theta 1 dots theta d for which we also provide completely adaptive estimators independent of s and of the noise variance sigma having logarithmically slower rates than the minimax ones furthermore we obtain the minimax rates on the ell q balls b q r theta in mathbb r d theta q le r where 0 q le2 and theta q sum i 1 d theta i q 1 q this analysis shows that there are in general three zones in the rates of convergence that we call the sparse zone the dense zone and the degenerate zone while a fourth zone appears for estimation of the quadratic functional we show that as opposed to estimation of theta the correct logarithmic terms in the optimal rates for the sparse zone scale as log d s 2 and not as log d s for the class b 0 s the rates of estimation of the linear functional and of the ell 2 norm have a simple elbow at s sqrt d boundary between the sparse and the dense zones and exhibit similar performances whereas the estimation of the quadratic functional q theta reveals more complex effects the minimax risk on b 0 s is infinite and the sparseness assumption needs to be combined with a bound on the ell 2 norm finally we apply our results on estimation of the ell 2 norm to the problem of testing against sparse alternatives in particular we obtain a nonasymptotic analog of the ingster donoho jin theory revealing some effects that were not captured by the previous asymptotic analysis

minimax rates in permutation estimation for feature matching the problem of matching two sets of features appears in various tasks of computer vision and can be often formalized as a problem of permutation estimation we address this problem from a statistical point of view and provide a theoretical analysis of the accuracy of several natural estimators to this end the minimax rate of separation is investigated and its expression is obtained as a function of the sample size noise level and dimension of the features we consider the cases of homoscedastic and heteroscedastic noise and establish in each case tight upper bounds on the separation distance of several estimators these upper bounds are shown to be unimprovable both in the homoscedastic and heteroscedastic settings interestingly these bounds demonstrate that a phase transition occurs when the dimension d of the features is of the order of the logarithm of the number of features n for d o log n the rate is dimension free and equals log n 1 2 where is the noise level in contrast when d is larger than c log n for some constant c 0 the minimax rate increases with d and is of the order of d log n 1 4 we also discuss the computational aspects of the estimators and provide empirical evidence of their consistency on synthetic data finally we show that our results extend to more general matching criteria

small delay approximation of stochastic delay differential equations delay differential equations evolve in an infinite dimensional phase space in this paper we consider the effect of external fluctuations noise on delay differential equations involving one variable thus leading to univariate stochastic delay differential equations sdde s for small delays a univariate nondelayed stochastic differential equation approximating such a sdde is presented another approximation complementary to the first is also obtained using an average of the sdde s drift term over the delayed dynamical variable which defines a conditional average drift this second approximation is characterized by the fact that the diffusion term is identical to that of the original sdde for small delays our approach yields a steady state probability density and a conditional average drift which are in close agreement with numerical simulations of the original sdde we illustrate this scheme with the delayed linear langevin equation and a stochastic version of the delayed logistic equation the technique can be used with any type of noise and is easily generalized to multiple delays s1063 651x 99 08304 x

consistent distribution free k sample and independence tests for univariate random variables a popular approach for testing if two univariate random variables are statistically independent consists of partitioning the sample space into bins and evaluating a test statistic on the binned data the partition size matters and the optimal partition size is data dependent while for detecting simple relationships coarse partitions may be best for detecting complex relationships a great gain in power can be achieved by considering finer partitions we suggest novel consistent distribution free tests that are based on summation or maximization aggregation of scores over all partitions of a fixed size we show that our test statistics based on summation can serve as good estimators of the mutual information moreover we suggest regularized tests that aggregate over all partition sizes and prove those are consistent too we provide polynomial time algorithms which are critical for computing the suggested test statistics efficiently we show that the power of the regularized tests is excellent compared to existing tests and almost as powerful as the tests based on the optimal yet unknown in practice partition size in simulations as well as on a real data example

splitting np complete sets we show that a set is m autoreducible if and only if it is m mitotic this solves a long standing open question in a surprising way as a consequence of this unconditional result and recent work by glaser et al complete sets for all of the following complexity classes are m mitotic mathrm np mathrm conp oplus mathrm p mathrm pspace and mathrm nexp as well as all levels of mathrm ph mathrm modph and the boolean hierarchy over mathrm np in the cases of mathrm np mathrm pspace mathrm nexp and mathrm ph this at once answers several well studied open questions these results tell us that complete sets share a redundancy that was not known before in particular every mathrm np complete set a splits into two mathrm np complete sets a 1 and a 2 we disprove the equivalence between autoreducibility and mitoticity for all polynomial time bounded reducibilities between 3 tt reducibility and turing reducibility there exists a sparse set in mathrm exp that is polynomial time 3 tt autoreducible but not weakly polynomial time t mitotic in particular polynomial time t autoreducibility does not imply polynomial time weak t mitoticity which solves an open question by buhrman and torenvliet

realigning random states we study how the realignment criterion also called computable cross norm criterion succeeds asymptotically in detecting whether random states are separable or entangled we consider random states on cd cd obtained by partial tracing a haar distributed random pure state on cd cd cs over an ancilla space cs we show that for large d typically the realignment criterion is successful in detecting entanglement if and only if s 83 2d2 in this sense the realignment criterion is asymptotically weaker than the partial transposition criterion

asymptotically optimal discrete time nonlinear filters from stochastically convergent state process approximations we consider the problem of approximating optimal in the mmse sense nonlinear filters in a discrete time setting exploiting properties of stochastically convergent state process approximations more specifically we consider a class of nonlinear partially observable stochastic systems comprised by a possibly nonstationary hidden stochastic process the state observed through another conditionally gaussian stochastic process the observations under general assumptions we show that given an approximating process which for each time step is stochastically convergent to the state process in some appropriate sense an approximate filtering operator can be defined which converges to the true optimal nonlinear filter of the state in a strong and well defined sense i e compactly in time and uniformly in a completely characterized measurable set of probability measure almost unity also providing a purely quantitative justification of egoro s theorem for the problem at hand the results presented in this paper can form a common basis for the analysis and characterization of a number of heuristic approaches for approximating a large class of optimal nonlinear filters such as approximate grid based techniques known to perform well in a variety of applications

a unified theorem on sdp rank reduction we consider the problem of finding a low rank approximate solution to a system of linear equations in symmetric positive semidefinite matrices where the approximation quality of a solution is measured by its maximum relative deviation both above and below from the prescribed quantities we show that a simple randomized polynomial time procedure produces a low rank solution that has provably good approximation qualities our result provides a unified treatment of and generalizes several well known results in the literature in particular it contains as special cases the johnson lindenstrauss lemma on dimensionality reduction results on low distortion embeddings into low dimensional euclidean space and approximation results on certain quadratic optimization problems

a tight degree 4 sum of squares lower bound for the sherrington kirkpatrick hamiltonian we show that if varvec w is an n times n matrix drawn from the gaussian orthogonal ensemble then with high probability the degree 4 sum of squares relaxation cannot certify an upper bound on the objective n 1 cdot varvec x top varvec w varvec x under the constraints x i 2 1 0 i e varvec x in pm 1 n that is asymptotically smaller than lambda max varvec w approx 2 we also conjecture a proof technique for lower bounds against sum of squares relaxations of any degree held constant as n rightarrow infty by proposing an approximate pseudomoment construction

on the rank of extreme matrices in semidefinite programs and the multiplicity of optimal eigenvalues we derive some basic results on the geometry of semidefinite programming sdp and eigenvalue optimization i e the minimization of the sum of the k largest eigenvalues of a smooth matrix valued function we provide upper bounds on the rank of extreme matrices in sdps and the first theoretically solid explanation of a phenomenon of intrinsic interest in eigenvalue optimization in the spectrum of an optimal matrix the kth and k 1 st largest eigenvalues tend to be equal and frequently have multiplicity greater than two this clustering is intuitively plausible and has been observed as early as 1975 when the matrix valued function is affine we prove that clustering must occur at extreme points of the set of optimal solutions if the number of variables is sufficiently large we also give a lower bound on the multiplicity of the critical eigenvalue these results generalize to the case of a general matrix valued function under appropriate conditions

maximum likelihood estimation for linear gaussian covariance models summary we study parameter estimation in linear gaussian covariance models which are p dimensional gaussian models with linear constraints on the covariance matrix maximum likelihood estimation for this class of models leads to a non convex optimization problem which typically has many local maxima using recent results on the asymptotic distribution of extreme eigenvalues of the wishart distribution we provide sufficient conditions for any hill climbing method to converge to the global maximum although we are primarily interested in the case in which n p the proofs of our results utilize large sample asymptotic theory under the scheme n p 1 remarkably our numerical simulations indicate that our results remain valid for p as small as 2 an important consequence of this analysis is that for sample sizes n 14p maximum likelihood estimation for linear gaussian covariance models behaves as if it were a convex optimization problem

weak mitoticity of bounded disjunctive and conjunctive truth table autoreducible sets abstract glaser et al siamjcomp 2008 and tcs 2009 2 proved existence of two sparse sets a and b in exp where a is 3 tt truth table polynomial time autoreducible but not weakly polynomial time turing mitotic and b is polynomial time 2 tt autoreducible but not weakly polynomial time 2 tt mitotic we unify and strengthen both of those results by showing that there is a sparse set in exp that is polynomial time 2 tt autoreducible but not even weakly polynomial time turing mitotic all these results indicate that polynomial time autoreducibilities in general do not imply polynomial time mitoticity at all with the only exceptions of the many one and 1 tt reductions on the other hand however we proved that every autoreducible set for the polynomial time bounded disjunctive or conjunctive tt reductions is weakly mitotic for the polynomial time tt reduction that makes logarithmically many queries only this shows that autoreducible sets for reductions making more than one query could still be mitotic in some way if they possess certain special properties

adaptive estimation of a quadratic functional by model selection we consider the problem of estimating s 2 when s belongs to some separable hilbert space and one observes the gaussian process y t langles t rangle sigmal t for all t epsilon mathbb h where l is some gaussian isonormal process this framework allows us in particular to consider the classical gaussian sequence model for which mathbb h l 2 mathbb n and l t sum lambda geq1 t lambda varepsilon lambda where varepsilon lambda lambda geq1 is a sequence of i i d standard normal variables our approach consists in considering some at most countable families of finite dimensional linear subspaces of mathbb h the models and then using model selection via some conveniently penalized least squares criterion to build new estimators of s 2 we prove a general nonasymptotic risk bound which allows us to show that such penalized estimators are adaptive on a variety of collections of sets for the parameter s depending on the family of models from which they are built in particular in the context of the gaussian sequence model a convenient choice of the family of models allows defining estimators which are adaptive over collections of hyperrectangles ellipsoids l p bodies or besov bodies we take special care to describe the conditions under which the penalized estimator is efficient when the level of noise sigma tends to zero our construction is an alternative to the one by efroimovich and low for hyperrectangles and provides new results otherwise

unraveling the rank one solution mystery of robust miso downlink transmit optimization a verifiable sufficient condition via a new duality result this paper concentrates on a robust transmit optimization problem for the multiuser multi input single output miso downlink scenario and under inaccurate channel state information csi this robust problem deals with a general rank transmit covariance design and follows a safe rate constrained formulation under spherically bounded csi uncertainties curiously simulation results in previous works suggested that the robust problem admits rank one optimal transmit covariances in most cases such a numerical finding is appealing because transmission with rank one covariances can be easily realized by single stream transmit beamforming this gives rise to a fundamentally important question namely whether we can theoretically identify conditions under which the robust problem admits a rank one solution in this paper we identify one such condition simply speaking we show that the robust problem is guaranteed to admit a rank one solution if the csi uncertainties are not too large and the multiuser channel is not too poorly conditioned to establish the aforementioned condition we develop a novel duality framework through which an intimate relationship between the robust problem and a related maximin problem is revealed our condition involves only a simple expression with respect to the multiuser channel and other system parameters in particular unlike other sufficient rank one conditions that have appeared in the literature ours is verifiable the application of our analysis framework to several other csi uncertainty models is also discussed

sparse phase retrieval via truncated amplitude flow this paper develops a novel algorithm termed sparse truncated amplitude flow sparta to reconstruct a sparse signal from a small number of magnitude only measurements it deals with what is also known as sparse phase retrieval pr which is np hard in general and emerges in many science and engineering applications upon formulating sparse pr as an amplitude based nonconvex optimization task sparta works iteratively in two stages in stage one the support of the underlying sparse signal is recovered using an analytically well justified rule and subsequently a sparse orthogonality promoting initialization is obtained via power iterations restricted on the support and in the second stage the initialization is successively refined by means of hard thresholding based gradient type iterations sparta is a simple yet effective scalable and fast sparse pr solver on the theoretical side for any n dimensional k sparse k ll n signal boldsymbol x with minimum in modulus nonzero entries on the order of 1 sqrt k vert boldsymbol x vert 2 sparta recovers the signal exactly up to a global unimodular constant from about k 2 log n random gaussian measurements with high probability furthermore sparta incurs computational complexity on the order of k 2n log n with total runtime proportional to the time required to read the data which improves upon the state of the art by at least a factor of k finally sparta is robust against additive noise of bounded support extensive numerical tests corroborate markedly improved recovery performance and speedups of sparta relative to existing alternatives

separating complexity classes using autoreducibility a set is autoreducible if it can be reduced to itself by a turing machine that does not ask its own input to the oracle we use autoreducibility to separate the polynomial time hierarchy from exponential space by showing that all turing complete sets for certain levels of the exponential time hierarchy are autoreducible but there exists some turing complete set for doubly exponential space that is not although we already knew how to separate these classes using diagonalization our proofs separate classes solely by showing they have different structural properties thus applying post s program to complexity theory we feel such techniques may prove unknown separations in the future in particular if we could settle the question as to whether all turing complete sets for doubly exponential time are autoreducible we would separate either polynomial time from polynomial space and nondeterministic logarithmic space from nondeterministic polynomial time or else the polynomial time hierarchy from exponential time we also look at the autoreducibility of complete sets under nonadaptive bounded query probabilistic and nonuniform reductions we show how settling some of these autoreducibility questions will also lead to new complexity class separations

high dimensional analysis of semidefinite relaxations for sparse principal components principal component analysis pca is a classical method for dimensionality reduction based on extracting the dominant eigenvectors of the sample covariance matrix however pca is well known to behave poorly in the large p small n setting in which the problem dimension p is comparable to or larger than the sample size n this paper studies pca in this high dimensional regime but under the additional assumption that the maximal eigenvector is sparse say with at most k nonzero components we consider a spiked covariance model in which a base matrix is perturbed by adding a k sparse maximal eigenvector and we analyze two computationally tractable methods for recovering the support set of this maximal eigenvector as follows a a simple diagonal thresholding method which transitions from success to failure as a function of the rescaled sample size dia n p k n k 2 log p k and b a more sophisticated semidefinite programming sdp relaxation which succeeds once the rescaled sample size sdp n p k n klog p k is larger than a critical threshold in addition we prove that no method including the best method which has exponential time complexity can succeed in recovering the support if the order parameter sdp n p k is below a threshold our results thus highlight an interesting trade off between computational and statistical efficiency in high dimensional inference

adiabatic reduction near a bifurcation in stochastically modulated systems we reexamine the procedure of adiabatic elimination of fast relaxing variables near a bifurcation point when some of the parameters of the system are stochastically modulated approximate stationary solutions of the fokker planck equation are obtained near threshold for the pitchfork and transcritical bifurcations correlations between fast variables and random modulation may shift the effective bifurcation point by an amount proportional to the intensity of the fluctuations we also find that fluctuations of the fast variables above threshold are not always gaussian and centered around the deterministic center manifold as was previously believed numerical solutions obtained for a few illustrative examples support these conclusions copyright ital 1998 ital the american physical society

approximation dans les espaces metriques et theorie de l estimation we investigate the relations between the speed of estimation and the metric structure of the parameter space especially in the case when its metric dimension is infinite given some distance d on generally hellinger distance in the case of n i i d variables we consider the minimax risk for n observations r n q mathop inf sup limits t n theta in theta text ie theta d q theta t n t n being any estimate of we shall look for functions r such that for positive constants c 1 q and c 2 q c 1 r q n r n q c2 r q n r n is the speed of estimation and we shall show under fairly general conditions including i i d variables and regular cases of markov chains and stationnary gaussian processes that r n is determined up to multiplicative constants by the metric structure of we shall also give a construction for some sort of universal estimates the risk of which is bounded by c 2 r q n in all cases where the preceding theory applies

estimation of integral functionals of a density let be a smooth function of k 2 variables we shall investigate in this paper the rates of convergence of estimators of t f f f x f x f k x x dx when f belongs to some class of densities of smoothness s we prove that when 2k 1 4 one can define an estimator t n of t f based on n i i d observations of density f on the real line which converges at the semiparametric rate 1 n on the other hand when s 2k 1 4 t f cannot be estimated at a rate faster than n with 4 s k 4s 1 we shall also provide some extensions to the multidimensional case those results extend previous works of levit of bickel and ritov and of donoho and nussbaum on estimation of quadratic functionals

approximation algorithms for homogeneous polynomial optimization with quadratic constraints in this paper we consider approximation algorithms for optimizing a generic multi variate homogeneous polynomial function subject to homogeneous quadratic constraints such optimization models have wide applications e g in signal processing magnetic resonance imaging mri data training approximation theory and portfolio selection since polynomial functions are non convex the problems under consideration are all np hard in general in this paper we shall focus on polynomial time approximation algorithms in particular we first study optimization of a multi linear tensor function over the cartesian product of spheres we shall propose approximation algorithms for such problem and derive worst case performance ratios which are shown to be dependent only on the dimensions of the model the methods are then extended to optimize a generic multi variate homogeneous polynomial function with spherical constraint likewise approximation algorithms are proposed with provable approximation performance ratios furthermore the constraint set is relaxed to be an intersection of co centered ellipsoids namely we consider maximization of a homogeneous polynomial over the intersection of ellipsoids centered at the origin and propose polynomial time approximation algorithms with provable worst case performance ratios numerical results are reported illustrating the effectiveness of the approximation algorithms studied

optimal adaptive estimation of a quadratic functional minimax mean squared error estimates of quadratic functionals of smooth functions have been constructed for a variety of smoothness classes in contrast to many nonparametric function estimation problems there are both regular and irregular cases in the regular cases the minimax mean squared error converges at a rate proportional to the inverse of the sample size whereas in the irregular case much slower rates are the rule we investigate the problem of adaptive estimation of a quadratic functional of a smooth function when the degree of smoothness of the underlying function is not known it is shown that estimators cannot achieve the minimax rates of convergence simultaneously over two parameter spaces when at least one of these spaces corresponds to the irregular case a lower bound for the mean squared error is given which shows that any adaptive estimator which is rate optimal for the regular case must lose a logarithmic factor in the irregular case on the other hand we give a rather simple adaptive estimator which is sharp for the regular case and attains this lower bound in the irregular case moreover we explicitly describe a subset of functions where our adaptive estimator loses the logarithmic factor and show that this subset is relatively small

non mitotic sets we study the question of the existence of non mitotic sets in np we show under various hypotheses that 1 tt mitoticity and m mitoticity differ on np t autoreducibility and t mitoticity differ on np this contrasts the situation in the recursion theoretic setting where ladner showed that autoreducibility and mitoticity coincide 2 tt autoreducibility does not imply weak 2 tt mitoticity from this it follows that autoreducibility and mitoticity are not equivalent for all reducibilities between 2 tt and t although the notions coincide for m and 1 tt reducibility

moment inequalities for sums of random matrices and their applications in optimization in this paper we consider various moment inequalities for sums of random matrices which are well studied in the functional analysis and probability theory literature and demonstrate how they can be used to obtain the best known performance guarantees for several problems in optimization first we show that the validity of a recent conjecture of nemirovski is actually a direct consequence of the so called non commutative khintchine s inequality in functional analysis using this result we show that an sdp based algorithm of nemirovski which is developed for solving a class of quadratic optimization problems with orthogonality constraints has a logarithmic approximation guarantee this improves upon the polynomial approximation guarantee established earlier by nemirovski furthermore we obtain improved safe tractable approximations of a certain class of chance constrained linear matrix inequalities secondly we consider a recent result of delage and ye on the so called data driven distributionally robust stochastic programming problem one of the assumptions in the delage ye result is that the underlying probability distribution has bounded support however using a suitable moment inequality we show that the result in fact holds for a much larger class of probability distributions given the close connection between the behavior of sums of random matrices and the theoretical properties of various optimization problems we expect that the moment inequalities discussed in this paper will find further applications in optimization

quality gain analysis of the weighted recombination evolution strategy on general convex quadratic functions quality gain is the expected relative improvement of the function value in a single step of a search algorithm quality gain analysis reveals the dependencies of the quality gain on the parameters of a search algorithm based on which one can derive the optimal values for the parameters in this paper we investigate evolution strategies with weighted recombination on general convex quadratic functions we derive a bound for the quality gain and two limit expressions of the quality gain from the limit expressions we derive the optimal recombination weights and the optimal step size and find that the optimal recombination weights are independent of the hessian of the objective function moreover the dependencies of the optimal parameters on the dimension and the population size are revealed differently from previous works where the population size is implicitly assumed to be smaller than the dimension our results cover the population size proportional to or greater than the dimension simulation results show the optimal parameters derived in the limit approximates the optimal values in non asymptotic scenarios

l 1 regularized least squares for support recovery of high dimensional single index models with gaussian designs it is known that for a certain class of single index models sims y f xp 1t 0 e support recovery is impossible when x n 0 ip p and a model complexity adjusted sample size is below a critical threshold recently optimal algorithms based on sliced inverse regression sir were suggested these algorithms work provably under the assumption that the design x comes from an i i d gaussian distribution in the present paper we analyze algorithms based on covariance screening and least squares with l1 penalization i e lasso and demonstrate that they can also enjoy optimal up to a scalar rescaled sample size in terms of support recovery albeit under slightly different assumptions on f and e compared to the sir based algorithms furthermore we show more generally that lasso succeeds in recovering the signed support of 0 if x n 0 and the covariance satisfies the irrepresentable condition our work extends existing results on the support recovery of lasso for the linear model to a more general class of sims

joint covariance estimation with mutual linear structure we consider the problem of joint estimation of structured covariance matrices assuming the structure is unknown estimation is achieved using heterogeneous training sets namely given groups of measurements coming from centered populations with different covariances our aim is to determine the mutual structure of these covariance matrices and estimate them supposing that the covariances span a low dimensional affine subspace in the space of symmetric matrices we develop a new efficient algorithm discovering the structure and using it to improve the estimation our technique is based on the application of principal component analysis in the matrix space we also derive an upper performance bound of the proposed algorithm in the gaussian scenario and compare it with the cramer rao lower bound numerical simulations are presented to illustrate the performance benefits of the proposed method

laplace deconvolution on the basis of time domain data and its application to dynamic contrast enhanced imaging in the present paper we consider the problem of laplace deconvolution with noisy discrete non equally spaced observations on a finite time interval which appears in many different contexts we propose a new method for laplace deconvolution which is based on expansions of the convolution kernel the unknown function and the observed signal over laguerre functions basis which acts as a surrogate eigenfunction basis using regression setting the expansion results in a small system of linear equations with the matrix of the system being triangular and toeplitz the number m of the terms in the expansion of the estimator is controlled via complexity penalty the advantage of this methodology is that it leads to very fast computations produces no boundary effects due to extension at zero and cut off at t and provides an estimator with the risk within a logarithmic factor of m of the oracle risk under no assumptions on the model and within a constant factor of the oracle risk under mild assumptions we emphasize that in the present paper we consider the true observational model with possibly non equispaced observations which are available on a finite interval of length t and account for the bias associated with this model the study is motivated by perfusion imaging using a short injection of contrast agent a procedure which is widely applied for medical assessment of blood flows within tissues such as cancerous tumors presence of a tuning parameter a allows to choose the most advantageous time units so that both the kernel and the unknown right hand side of the equation are well represented for the further deconvolution in addition absence of boundary effects allows for an accurate estimator of the tissue blood flow parameter which is of great interest to medical doctors the methodology is illustrated by an extensive simulation study and a real data example which confirms that the proposed technique is fast efficient accurate usable from a practical point of view and competitive

pitchfork and hopf bifurcation thresholds in stochastic equations with delayed feedback the bifurcation diagram of a model stochastic differential equation with delayed feedback is presented we are motivated by recent research on stochastic effects in models of transcriptional gene regulation we start from the normal form for a pitchfork bifurcation and add multiplicative or parametric noise and linear delayed feedback the latter is sufficient to originate a hopf bifurcation in that region of parameters in which there is a sufficiently strong negative feedback we find a sharp bifurcation in parameter space and define the threshold as the point in which the stationary distribution function p x changes from a delta function at the trivial state x 0 to p x approximately x alpha at small x with alpha 1 exactly at threshold we find that the bifurcation threshold is shifted by fluctuations relative to the deterministic limit by an amount that scales linearly with the noise intensity analytic calculations of the bifurcation threshold are also presented in the limit of small delay tau 0 that compare quite favorably with the numerical solutions even for moderate values of tau

a semidefinite relaxation scheme for multivariate quartic polynomial optimization with quadratic constraints we present a general semidefinite relaxation scheme for general n variate quartic polynomial optimization under homogeneous quadratic constraints unlike the existing sum of squares approach which relaxes the quartic optimization problems to a sequence of typically large linear semidefinite programs sdp our relaxation scheme leads to a possibly nonconvex quadratic optimization problem with linear constraints over the semidefinite matrix cone in mathbb r n times n it is shown that each alpha factor approximate solution of the relaxed quadratic sdp can be used to generate in randomized polynomial time an o alpha factor approximate solution for the original quartic optimization problem where the constant in o cdot depends only on problem dimension in the case where only one positive definite quadratic constraint is present in the quartic optimization problem we present a randomized polynomial time approximation algorithm which can provide a guaranteed relative approximation ratio of 1 o n 2

stochastic differential delay equation moment stability and application to hematopoietic stem cell regulation system we study the moment stability of the trivial solution of a linear differential delay equation in the presence of additive and multiplicative white noise the results established here are applied to examining the local stability of the hematopoietic stem cell hsc regulation system in the presence of noise the stability of the first moment for the solutions of a linear differential delay equation under stochastic perturbation is identical to that of the unperturbed system however the stability of the second moment is altered by the perturbation we obtain using laplace transform techniques necessary and sufficient conditions for the second moment to be bounded in applying the results to the hsc system we find that the system stability is sensitive to perturbations in the stem cell differentiation and death rates but insensitive to perturbations in the proliferation rate

shapefit exact location recovery from corrupted pairwise directions let t1 tn r and consider the location recovery problem given a subset of pairwise direction observations ti tj ti tj 2 i j n n where a constant fraction of these observations are arbitrarily corrupted find ti i 1 up to a global translation and scale we propose a novel algorithm for the location recovery problem which consists of a simple convex program over dn real variables we prove that this program recovers a set of n i i d gaussian locations exactly and with high probability if the observations are given by an erdos renyi graph d is large enough and provided that at most a constant fraction of observations involving any particular location are adversarially corrupted

minimax estimation via wavelet shrinkage we attempt to recover an unknown function from noisy sampled data using orthonormal bases of compactly supported wavelets we develop a nonlinear method which works in the wavelet domain by simple nonlinear shrinkage of the empirical wavelet coefficients the shrinkage can be tuned to be nearly minimax over any member of a wide range of triebel and besov type smoothness constraints and asymptotically minimax over besov bodies with p q linear estimates cannot achieve even the minimax rates over triebel and besov classes with p 2 so the method can significantly outperform every linear method e g kernel smoothing spline sieve in a minimax sense variants of our method based on simple threshold nonlinear estimators are nearly minimax our method possesses the interpretation of spatial adaptivity it reconstructs using a kernel which may vary in shape and bandwidth from point to point depending on the data least favorable distributions for certain of the triebel and besov scales generate objects with sparse wavelet transforms many real objects have similarly sparse transforms which suggests that these minimax results are relevant for practical problems sequels to this paper which was first drafted in november 1990 discuss practical implementation spatial adaptation properties universal near minimaxity and applications to inverse problems

near optimal sample complexity bounds for robust learning of gaussian mixtures via compression schemes we introduce a novel technique for distribution learning based on a notion of sample compression any class of distributions that allows such a compression scheme can be learned with few samples m

detecting anomalous activity on networks with the graph fourier scan statistic we consider the problem of deciding based on a single noisy measurement at each vertex of a given graph whether the underlying unknown signal is constant over the graph or there exists a cluster of vertices with anomalous activation this problem is relevant to several applications such as surveillance disease outbreak detection biomedical imaging environmental monitoring etc since the activations in these problems often tend to be localized to small groups of vertices in the graphs we model such activity by a class of signals that are elevated over a possibly disconnected cluster with low cut size relative to its size we analyze the corresponding generalized likelihood ratio glr statistics and relate it to the problem of finding a sparsest cut in the graph we develop a convex relaxation of the glr statistic based on spectral graph theory which we call the graph fourier scan statistic gfss in our main theoretical result we show that the performance of the gfss depends explicitly on the spectral properties of the graph to assess the optimality of the gfss we prove an information theoretic lower bound for the detection of anomalous activity on graphs because the gfss requires the specification of a tuning parameter we develop an adaptive version of the gfss using these results we are able to characterize in a very explicit form the performance of the gfss on a few notable graph topologies we demonstrate that the gfss can efficiently detect a simulated arsenic contamination in groundwater

delay fokker planck equations novikov s theorem and boltzmann distributions as small delay approximations we study time delayed stochastic systems that can be described by means of so called delay fokker planck equations using novikov s theorem we first show that the theory of delay fokker planck equations is on an equal footing with the theory of ordinary fokker planck equations subsequently we derive stationary distributions in the case of small time delays in the case of additive noise systems these distributions can be cast into the form of boltzmann distributions involving effective potential functions

optimal bounds for aggregation of affine estimators we study the problem of aggregation of estimators when the estimators are not independent of the data used for aggregation and no sample splitting is allowed if the estimators are deterministic vectors it is well known that the minimax rate of aggregation is of order log m where m is the number of estimators to aggregate it is proved that for affine estimators the minimax rate of aggregation is unchanged it is possible to handle the linear dependence between the affine estimators and the data used for aggregation at no extra cost the minimax rate is not impacted either by the variance of the affine estimators or any other measure of their statistical complexity the minimax rate is attained with a penalized procedure over the convex hull of the estimators for a penalty that is inspired from the q aggregation procedure the results follow from the interplay between the penalty strong convexity and concentration

optimal estimation of genetic relatedness in high dimensional linear models estimating the genetic relatedness between two traits based on the genome wide association data is an important problem in genetics research in the framework of high dimensional linear models we

on maximization of quadratic form over intersection of ellipsoids with common center we demonstrate that if a1 a m are symmetric positive semidefinite n n matrices with positive definite sum and a is an arbitrary symmetric n n matrix then the relative accuracy in terms of the optimal value of the semidefinite relaxation max x tr ax mid tr a ix le1 i 1 m x succeq0 eqno hbox sdp of the optimization program x tax to max mid x ta ix le 1 i 1 m eqno hbox p is not worse than 1 frac 1 2 ln 2m 2 it is shown that this bound is sharp in order as far as the dependence on m is concerned and that a feasible solution x to p with x tax ge frac opt hbox rm sdp 2 ln 2m 2 eqno can be found efficiently this somehow improves one of the results of nesterov 4 where bound similar to is established for the case when all ai are of rank 1

semidefinite relaxation bounds for indefinite homogeneous quadratic optimization this paper studies the relationship between the optimal value of a homogeneous quadratic optimization problem and its semidefinite programming sdp relaxation we consider two quadratic optimization models 1 min x c x mid x a k x ge 1 k 0 1 ldots m x in mathbb f n and 2 max x c x mid x a k x le 1 k 0 1 ldots m x in mathbb f n where mathbb f is either the real field mathbb r or the complex field mathbb c and a k c are symmetric matrices for the minimization model 1 we prove that if the matrix c and all but one of the a k s are positive semidefinite then the ratio between the optimal value of 1 and its sdp relaxation is upper bounded by o m 2 when mathbb f mathbb r and by o m when mathbb f mathbb c moreover when two or more of the a k s are indefinite this ratio can be arbitrarily large for the maximization model 2 we show that if c and at most one of the a k s are indefinite while other a k s are positive semidefinite then the ratio between the optimal value of 2 and its sdp relaxation is bounded from below by o 1 log m for both the real and the complex case this result improves the bound based on the so called approximate s lemma of ben tal nemirovski and roos siam j optim 13 2002 pp 535 560 when two or more of the a k s in 2 are indefinite we derive a general bound in terms of the problem data and the sdp solution for both optimization models we present examples to show that the derived approximation bounds are essentially tight

model selection for regression on a fixed design we deal with the problem of estimating some unknown regression function involved in a regression framework with deterministic design points for this end we consider some collection of finite dimensional linear spaces models and the least squares estimator built on a data driven selected model among this collection this data driven choice is performed via the minimization of some penalized model selection criterion that generalizes on mallows c p we provide non asymptotic risk bounds for the so defined estimator from which we deduce adaptivity properties our results hold under mild moment conditions on the errors the statement and the use of a new moment inequality for empirical processes is at the heart of the techniques involved in our approach

deterministic approximation algorithms for sphere constrained homogeneous polynomial optimization problems due to their fundamental nature and numerous applications sphere constrained polynomial optimization problems have received a lot of attention lately in this paper we consider three such problems i maximizing a homogeneous polynomial over the sphere ii maximizing a multilinear form over a cartesian product of spheres and iii maximizing a multiquadratic form over a cartesian product of spheres since these problems are generally intractable our focus is on designing polynomial time approximation algorithms for them by reducing the above problems to that of determining the l 2 diameters of certain convex bodies we show that they can all be approximated to within a factor of log n n d 2 1 deterministically where n is the number of variables and d is the degree of the polynomial this improves upon the currently best known approximation bound of 1 n d 2 1 in the literature we believe that our approach will find further applications in the design of approximation algorithms for polynomial optimization problems with provable guarantees

low rank positive semidefinite matrix recovery from corrupted rank one measurements we study the problem of estimating a low rank positive semidefinite psd matrix from a set of rank one measurements using sensing vectors composed of i i d standard gaussian entries which are possibly corrupted by arbitrary outliers this problem arises from applications such as phase retrieval covariance sketching quantum space tomography and power spectrum estimation we first propose a convex optimization algorithm that seeks the psd matrix with the minimum ell 1 norm of the observation residual the advantage of our algorithm is that it is free of parameters therefore eliminating the need for tuning parameters and allowing easy implementations we establish that with high probability a low rank psd matrix can be exactly recovered as soon as the number of measurements is large enough even when a fraction of the measurements are corrupted by outliers with arbitrary magnitudes moreover the recovery is also stable against bounded noise with the additional information of an upper bound of the rank of the psd matrix we propose another nonconvex algorithm based on subgradient descent that demonstrates excellent empirical performance in terms of computational efficiency and accuracy

efficient learning and planning with compressed predictive states predictive state representations psrs offer an expressive framework for modelling partially observable systems by compactly representing systems as functions of observable quantities the psr learning approach avoids using local minima prone expectation maximization and instead employs a globally optimal moment based algorithm moreover since psrs do not require a predetermined latent state structure as an input they offer an attractive framework for model based reinforcement learning when agents must plan without a priori access to a system model unfortunately the expressiveness of psrs comes with significant computational cost and this cost is a major factor inhibiting the use of psrs in applications in order to alleviate this shortcoming we introduce the notion of compressed psrs cpsrs the cpsr learning approach combines recent advancements in dimensionality reduction incremental matrix decomposition and compressed sensing we show how this approach provides a principled avenue for learning accurate approximations of psrs drastically reducing the computational costs associated with learning while also providing effective regularization going further we propose a planning framework which exploits these learned models and we show that this approach facilitates model learning and planning in large complex partially observable domains a task that is infeasible without the principled use of compression

geometrizing rates of convergence ii consider estimating a functional t f of an unknown distribution f in mathbf f from data x 1 cdots x n i i d f let omega varepsilon denote the modulus of continuity of the functional t over mathbf f computed with respect to hellinger distance for well behaved loss functions l t we show that inf t n sup mathbf f e fl t n t f is equivalent to l omega n 1 2 to within constants whenever t is linear and mathbf f is convex the same conclusion holds in three nonlinear cases estimating the rate of decay of a density estimating the mode and robust nonparametric regression we study the difficulty of testing between the composite infinite dimensional hypotheses h 0 t f leq t and h 1 t f geq t delta our results hold in the cases studied because the difficulty of the full infinite dimensional composite testing problem is comparable to the difficulty of the hardest simple two point testing subproblem

almost universal codes achieving ergodic mimo capacity within a constant gap this paper addresses the question of achieving capacity with lattice codes in multi antenna block fading channels when the number of fading blocks tends to infinity a design criterion based on the normalized minimum determinant is proposed for division algebra multi block space time codes over fading channels this plays a similar role to the hermite invariant for gaussian channels under maximum likelihood decoding it is shown that this criterion is sufficient to guarantee transmission rates within a constant gap from capacity both for deterministic channels and ergodic fading channels moreover if the number of receive antennas is greater than or equal to the number of transmit antennas the same constant gap is achieved under naive lattice decoding as well in the case of independent identically distributed rayleigh fading the error probability vanishes exponentially fast in contrast to the standard approach in the literature which employs random lattice ensembles the existence results in this paper are derived from the number theory first the gap to capacity is shown to depend on the discriminant of the chosen division algebra then class field theory is applied to build families of algebras with small discriminants the key element in the construction is the choice of a sequence of division algebras whose centers are number fields with small root discriminants

approximation bounds for quadratic optimization with homogeneous quadratic constraints we consider the np hard problem of finding a minimum norm vector in n dimensional real or complex euclidean space subject to m concave homogeneous quadratic constraints we show that a semidefinite programming sdp relaxation for this nonconvex quadratically constrained quadratic program qp provides an o m 2 approximation in the real case and an o m approximation in the complex case moreover we show that these bounds are tight up to a constant factor when the hessian of each constraint function is of rank 1 namely outer products of some given so called steering vectors and the phase spread of the entries of these steering vectors are bounded away from pi 2 we establish a certain constant factor approximation depending on the phase spread but independent of m and n for both the sdp relaxation and a convex qp restriction of the original np hard problem finally we consider a related problem of finding a maximum norm vector subject to m convex homogeneous quadratic constraints we show that an sdp relaxation for this nonconvex qp provides an o 1 ln m approximation which is analogous to a result of nemirovski e math program 86 1999 pp 463 473 for the real case

converse bounds on modulation estimation performance for the gaussian multiple access channel this paper focuses on the problem of separately modulating and jointly estimating two independent continuous valued parameters sent over a gaussian multiple access channel mac under the mean square error mse criterion without bandwidth constraints to this end we first improve an existing lower bound on the mse that is obtained using the parameter modulation estimation techniques for the single user additive white gaussian noise awgn channel as for the main contribution of this paper this improved modulation estimation analysis is generalized to the model of the two user gaussian mac we present outer bounds to the achievable region in the plane of the mse s of the two user parameters which provides a trade off between the mse s where we used zero rate lower bounds on the error probability of gaussian channels by shannon and polyanskiy et al numerical results showed that the multi user adaptation of the zero rate lower bound by polyanskiy et al provides a tighter overall lower bound on the mse pairs than the classical shannon bound in addition we introduced upper bounds on the mse exponents namely the exponential decay rates of these mse s in the asymptotic regime of long blocks that could make use of any bound on the error exponent of a single user awgn channel the obtained results are numerically evaluated for three different bounds on the reliability function of the gaussian channel it is shown that the adaptation of the reliability function by ashikhmin et al to the mac provides a significantly tighter characterization than shannon s sphere packing bound and the divergence bound

estimating the intensity of a random measure by histogram type estimators the purpose of this paper is to estimate the intensity of some random measure n on a set mathcal x by a piecewise constant function on a finite partition of mathcal x given a possibly large family mathcal m of candidate partitions we build a piecewise constant estimator histogram on each of them and then use the data to select one estimator in the family choosing the square of a hellinger type distance as our loss function we show that each estimator built on a given partition satisfies an analogue of the classical squared bias plus variance risk bound moreover the selection procedure leads to a final estimator satisfying some oracle type inequality with as usual a possible loss corresponding to the complexity of the family mathcal m when this complexity is not too high the selected estimator has a risk bounded up to a universal constant by the smallest risk bound obtained for the estimators in the family for suitable choices of the family of partitions we deduce uniform risk bounds over various classes of intensities our approach applies to the estimation of the intensity of an inhomogenous poisson process among other counting processes or the estimation of the mean of a random vector with nonnegative components