effective homology of bicomplexes formalized in coq in this paper we present a complete formalization in the coq theorem prover of an important algorithm in computational algebra namely the calculation of the effective homology of a bicomplex as a necessary tool we encode a hierarchy of algebraic structures in constructive type theory including graded and infinite data structures the experience shows how some limitations of the coq proof assistant to deal with this kind of algebraic data can be overcome by applying a separation of concerns principle more concretely we propose to distinguish in the representation of an algebraic structure such as a group or a module a behavioural part containing operation signatures and axioms and a structural part determining if the algebraic data is free of finite type and so on

design of signature sequences for overloaded cdma and bounds on the sum capacity with arbitrary symbol alphabets in this paper we explore some of the fundamentals of synchronous code division multiple access cdma as applied to wireless and optical communication systems under very general settings of any size for the user symbols and the signature matrix entries the channel is modeled by real complex additive noise of arbitrary distribution two problems are addressed the first problem concerns whether uniquely detectable overloaded matrices exist in the absence of additive noise under these general settings and if so whether there are any practical optimum detection algorithms the second one is about the bounds for the sum channel capacity when user data and signature matrices employ any real or complex alphabets finite or infinite in response to the first problem we have developed practical maximum likelihood detection algorithms for overloaded cdma systems for a large class of alphabets in response to the second problem a general theorem has been developed in which the sum capacity lower bounds with respect to the number of users spreading gain and signal to noise ratio can be derived to show the power and utility of the main theorem a number of sum capacity bounds for special cases are evaluated an important conclusion of this paper is that the lower and upper bounds of the sum capacity for small medium size cdma systems depend on both the input and the signature symbols this is contrary to the asymptotic results for large scale systems reported in the literature also confirmed in this paper where the signature symbols and statistics disappear for signature matrices and input vectors with i i d entries furthermore upper and asymptotic bounds are derived and compared to other derivations

on the stability of the quenched state in mean field spin glass models while the gibbs states of spin glass models have been noted to have an erratic dependence on temperature one may expect the mean over the disorder to produce a continuously varying quenched state the assumption of such continuity in temperature implies that in the infinite volume limit the state is stable under a class of deformations of the gibbs measure the condition is satisfied by the parisi ansatz along with an even broader stationarity property the stability conditions have equivalent expressions as marginal additivity of the quenched free energy implications of the continuity assumption include constraints on the overlap distribution which are expressed as the vanishing of the expectation value for an infinite collection of multi overlap polynomials the polynomials can be computed with the aid of a real replica calculation in which the number of replicas is taken to zero

on the structure of quasi stationary competing particle systems we study point processes on the real line whose configurations x are locally finite have a maximum and evolve through increments which are functions of correlated gaussian variables the correlations are intrinsic to the points and quantified by a matrix q q ij i j a probability measure on the pair x q is said to be quasi stationary if the joint law of the gaps of x and of q is invariant under the evolution a known class of universally quasi stationary processes is given by the ruelle probability cascades rpc which are based on hierarchically nested poisson dirichlet processes it was conjectured that up to some natural superpositions these processes exhausted the class of laws which are robustly quasi stationary the main result of this work is a proof of this conjecture for the case where q ij assume only a finite number of values the result is of relevance for mean field spin glass models where the evolution corresponds to the cavity dynamics and where the hierarchical organization of the gibbs measure was first proposed as an ansatz

rigorous solution of the gardner problem we prove rigorously the well known result of gardner about the typical fractional volume of interactions between n spins which solve the problem of storing a given set of p random patterns the gardner formula for this volume in the limit n p p n is proven for all values of besides we prove a useful criterion for the factorisation of all correlation functions for a class of models of classical statistical mechanics

homotopy groups of suspended classifying spaces an experimental approach when the results of a computer program are confronted to some theorems proved on a theoretical basis three situations can occur there can be an agreement between both approaches the computer program can obtain calculations not covered by the theorems or a discrepancy can be found between both methods in this note we report on a work where the three above mentioned situations happen we have enhanced the computer algebra called kenzo to deal with the computation of homotopy groups of suspended classifying spaces a problem tackled by mikhailov and wu in a paper published in the journal algebraic and geometric topology our experimental approach based on completely different methods from those by mikhailov and wu has allowed us in particular to detect an error in one of their published theorems 1 motivation and statement of the problem it is well known that the suspension functor applied to a topological space x shifts its homology groups that is to say hn x hn 1 x however in the case of homotopy groups the situation is not so favorable and in general there is not a direct relation between x and x let us consider for instance a circle s whose homotopy type is very simple the homology groups of the suspension s s are obvious but the problem of computing the homotopy groups of s is a difficult problem in algebraic topology given a group g its classifying space that is to say the eilenberg maclane spacek g 1 10 has trivial homotopy groups 1 k g 1 g and n k g 1 0 for each n 6 1 but when applying the suspension functor the new homotopy groups k g 1 are in general unknown in 11 several groups n k g 1 are obtained for some particular cases of g and n making use of different results and techniques from group theory and homotopy theory more concretely the main results of the article by mikhailov and wu are descriptions of the groups 4 k a 1 and 5 k a 1 when a is any finitely generated abelian group as applications they also determine n k g 1 with n 4 and 5 for some nonabelian groups as g 3 the 3 th symmetric group and g sl z the standard linear group and 4 k a4 1 for a4 the 4 th alternating group the goal of this work consists in trying to obtain a computer program calculating k g 1 for a general group g which will allow one to obtain as particular computations some theoretical results of 11 to this aim we have developed 2010 mathematics subject classification 68w05 55 04 55q99 partially supported by ministerio de ciencia e innovacion spain project mtm2009 13842c02 01 c xxxx american mathematical society

approximate ultrametricity for random measures and applications to spin glasses in this paper we introduce a notion called approximate ultrametricity which encapsulates the phenomenology of a sequence of random probability measures having supports that behave like ultrametric spaces insofar as they decompose into nested balls we provide a sufficient condition for a sequence of random probability measures on the unit ball of an infinite dimensional separable hilbert space to admit such a decomposition whose elements we call clusters we also characterize the laws of the measures of the clusters by showing that they converge in law to the weights of a ruelle probability cascade these results apply to a large class of classical models in mean field spin glasses we illustrate the notion of approximate ultrametricity by proving a conjecture of talagrand regarding mixed p spin glasses that is known to imply a prediction of dotsenko franz mezard 2016 wiley periodicals inc

absence of self averaging of the order parameter in the sherrington kirkpatrick model we prove that if n is the sherrington kirkpatrick sk hamiltonian and the quantity bar q n n 1 sum left langle s l right rangle h 2 converges in the variance to a nonrandom limit asn then the mean free energy of the model converges to the expression obtained by sk since this expression is known not to be correct in the low temperature region our result implies the non self averaging of the order parameter of the sk model this fact is an important ingredient of the parisi theory which is widely believed to be exact we also prove that the variance of the free energy of the sk model converges to zero asn i e the free energy has the self averaging property

the free energy of a class of hopfield models we show that in the limitp n 0 p n 0 the limit free energy of the hopfield model equals in probability the curie weiss free energy we prove also that the free energy of the hopfield model is self averaging for any finite

large system analysis of joint channel and data estimation for mimo ds cdma systems this paper presents a large system analysis of the performance of joint channel estimation multiuser detection and per user decoding ce mudd for randomly spread multiple input multiple output mimo direct sequence code division multiple access ds cdma systems a suboptimal receiver based on successive decoding in conjunction with linear minimum mean squared error lmmse channel estimation is investigated the replica method developed in statistical mechanics is used to evaluate the performance in the large system limit where the number of users and the spreading factor tend to infinity while their ratio and the number of transmit and receive antennas are kept constant joint ce mudd based on lmmse channel estimation is compared to several methods of one shot channel estimation i e methods that do not utilize decoded data symbols to refine the initial channel estimates in terms of spectral efficiency joint ce mudd is found to significantly reduce the rate loss caused by transmission of pilot signals when compared to the receivers based on one shot channel estimation this holds particularly for multiple antenna systems as a result joint ce mudd can provide significant performance gains compared to the receivers based on one shot channel estimation

right convergence of sparse random graphs the paper is devoted to the problem of establishing right convergence of sparse random graphs this concerns the convergence of the logarithm of number of homomorphisms from graphs or hyper graphs mathbb g n n ge 1 to some target graph w the theory of dense graph convergence including random dense graphs is now well understood borgs et al in ann math 176 151 219 2012 borgs et al in adv math 219 1801 1851 2008 chatterjee and varadhan in eur j comb 32 1000 1017 2011 lovasz and szegedy in j comb theory ser b 96 933 957 2006 but its counterpart for sparse random graphs presents some fundamental difficulties phrased in the statistical physics terminology the issue is the existence of the limits of appropriately normalized log partition functions also known as free energy limits for the gibbs distribution associated with w in this paper we prove that the sequence of sparse erdos renyi graphs is right converging when the tensor product associated with the target graph w satisfies a certain convexity property we treat the case of discrete and continuous target graphs w the latter case allows us to prove a special case of talagrand s recent conjecture more accurately stated as level iii research problem 6 7 2 in his recent book talagrand in mean field models for spin glasses volume i basic examples springer berlin 2010 concerning the existence of the limit of the measure of a set obtained from mathbb r n by intersecting it with linearly in n many subsets generated according to some common probability law our proof is based on the interpolation technique introduced first by guerra and toninelli commun math phys 230 71 79 2002 and developed further in abbe and montanari in on the concentration of the number of solutions of random satisfiability formulas 2013 bayati et al in ann probab conference version in proceedings of 42nd ann symposium on the theory of computing stoc 2010 contucci et al in antiferromagnetic potts model on the erdos renyi random graph 2011 franz and leone in j stat phys 111 3 4 535 564 2003 franz et al in j phys a math gen 36 10967 10985 2003 montanari in ieee trans inf theory 51 9 3221 3246 2005 panchenko and talagrand in probab theory relat fields 130 312 336 2004 specifically bayati et al ann probab conference version in proceedings of 42nd ann symposium on the theory of computing stoc 2010 establishes the right convergence property for erdos renyi graphs for some special cases of w in this paper most of the results in bayati et al ann probab conference version in proceedings of 42nd ann symposium on the theory of computing stoc 2010 follow as a special case of our main theorem

the simplest model of jamming we study a well known neural network model the perceptron as a simple statistical physics model of jamming of hard objects we exhibit two regimes 1 a convex optimization regime where jamming is hypostatic and non critical 2 a non convex optimization regime where jamming is isostatic and critical we characterize the critical jamming phase through exponents describing the distribution laws of forces and gaps surprisingly we find that these exponents coincide with the corresponding ones recently computed in high dimensional hard spheres in addition modifying the perceptron to a random linear programming problem we show that isostaticity is not a sufficient condition for singular force and gap distributions for that fragmentation of the space of solutions replica symmetry breaking appears to be a crucial ingredient we hypothesize universality for a large class of non convex constrained satisfaction problems with continuous variables

tight bounds on the capacity of binary input random cdma systems in this paper we consider code division multiple access cdma communication over a binary input additive white gaussian noise awgn channel using random spreading for a general class of symmetric distributions for spreading sequences in the limit of a large number of users we prove an upper bound to the capacity the bound matches the formula obtained by tanaka using the replica method we also show concentration of various relevant quantities including mutual information and free energy the mathematical methods are quite general and allow us to discuss extensions to other multiuser scenarios

the parisi formula in this chapter we prove the parisi formula which gives the limiting value of the free energy per site for the sherrington kirkpatrick model at each temperature starting with the famous result of guerra that the parisi formula provided a bound for this free energy we make full use of poisson dirichlet cascades we then use the same techniques to prove in a strong sense that in presence of an external field the overlaps are positive

asymptotic and finite sample properties of estimators based on stochastic gradients stochastic gradient descent procedures have gained popularity for parameter estimation from large data sets however their statistical properties are not well understood in theory and in practice avoiding numerical instability requires careful tuning of key parameters here we introduce implicit stochastic gradient descent procedures which involve parameter updates that are implicitly defined intuitively implicit updates shrink standard stochastic gradient descent updates the amount of shrinkage depends on the observed fisher information matrix which does not need to be explicitly computed thus implicit procedures increase stability without increasing the computational burden our theoretical analysis provides the first full characterization of the asymptotic behavior of both standard and implicit stochastic gradient descent based estimators including finite sample error bounds importantly analytical expressions for the variances of these stochastic gradient based estimators reveal their exact loss of efficiency we also develop new algorithms to compute implicit stochastic gradient descent based estimators for generalized linear models cox proportional hazards m estimators in practice and perform extensive experiments our results suggest that implicit stochastic gradient descent procedures are poised to become a workhorse for approximate inference from large data sets

random overlap structures properties and applications to spin glasses random overlap structures rost s are random elements on the space of probability measures on the unit ball of a hilbert space where two measures are identified if they differ by an isometry in spin glasses they arise as natural limits of gibbs measures under the appropriate algebra of functions we prove that the so called cavity mapping on the space of rost s is continuous leading to a proof of the stochastic stability conjecture for the limiting gibbs measures of a large class of spin glass models similar arguments yield the proofs of a number of other properties of rost s that may be useful in future attempts at proving the ultrametricity conjecture lastly assuming that the ultrametricity conjecture holds the setup yields a constructive proof of the parisi formula for the free energy of the sherrington kirkpatrick model by making rigorous a heuristic of aizenman sims and starr

which bridge estimator is the best for variable selection we study the problem of variable selection for linear models under the high dimensional asymptotic setting where the number of observations n grows at the same rate as the number of predictors p we consider two stage variable selection techniques tvs in which the first stage uses bridge estimators to obtain an estimate of the regression coefficients and the second stage simply thresholds this estimate to select the important predictors the asymptotic false discovery proportion operatorname afdp and true positive proportion atpp of these tvs are evaluated we prove that for a fixed atpp in order to obtain a smaller operatorname afdp one should pick a bridge estimator with smaller asymptotic mean square error in the first stage of tvs based on such principled discovery we present a sharp comparison of different tvs via an in depth investigation of the estimation properties of bridge estimators rather than orderwise error bounds with loose constants our analysis focuses on precise error characterization various interesting signal to noise ratio and sparsity settings are studied our results offer new and thorough insights into high dimensional variable selection for instance we prove that a tvs with ridge in its first stage outperforms tvs with other bridge estimators in large noise settings two stage lasso becomes inferior when the signal is rare and weak as a by product we show that two stage methods outperform some standard variable selection techniques such as operatorname lasso and sure independence screening under certain conditions

an iterative construction of solutions of the tap equations for the sherrington kirkpatrick model we propose an iterative scheme for the solutions of the tap equations in the sherrington kirkpatrick model which is shown to converge up to and including the de almeida thouless line the main tool is a representation of the iterations which reveals an interesting structure of them this representation does not depend on the temperature parameter but for temperatures below the de almeida thouless line it contains a part which does not converge to zero in the limit

applications of the lindeberg principle in communications and statistical learning we use a generalization of the lindeberg principle developed by s chatterjee to prove universality properties for various problems in communications statistical learning and random matrix theory we also show that these systems can be viewed as the limiting case of a properly defined sparse system the latter result is useful when the sparse systems are easier to analyze than their dense counterparts the list of problems we consider is by no means exhaustive we believe that the ideas can be used in many other problems relevant for information theory

a deterministic equivalent for the analysis of non gaussian correlated mimo multiple access channels using large dimensional random matrix theory rmt we conduct mutual information analysis of a multiple input multiple output mimo multiple access channel mac our channel model reflects the characteristics in small cell networks where antenna correlations line of sight components and general type of fading distributions have to be included the mutual information expression can be expressed as functionals of the stieltjes transform through the so called shannon transform ideally if the stieltjes transform is known in the context of the large dimensional rmt then the problem is solved however it is difficult to derive the stieltjes transform of the considered channel models directly especially when the transmit correlation matrices are generally nonnegative definite and the channel entries are non gaussian to overcome this we use the generalized lindeberg principle to show that the stieltjes transforms of this class of random matrices with gaussian or non gaussian independent entries coincide in the large dimensional regime this result permits to derive the deterministic equivalents e g the stieltjes transform and the ergodic mutual information for non gaussian mimo channels from the known results developed for gaussian mimo channels as an application we determine the capacity achieving input covariance matrices for the mimo macs and prove that the capacity achieving input covariance matrices are asymptotically independent of the fading distribution

criss cross methods a fresh view on pivot algorithms criss cross methods are pivot algorithms that solve linear programming problems in one phase starting with any basic solution the first finite criss cross method was invented by chang terlaky and wang independently unlike the simplex method that follows a monotonic edge path on the feasible region the trace of a criss cross method is neither monotonic with respect to the objective function nor feasibility preserving the main purpose of this paper is to present mathematical ideas and proof techniques behind finite criss cross pivot methods a recent result on the existence of a short admissible pivot path to an optimal basis is given indicating shortest pivot paths from any basis might be indeed short for criss cross type algorithms the origins and the history of criss cross methods are also touched upon

pivot versus interior point methods pros and cons linear optimization lo is the fundamental problem of mathematical optimization it admits an enormous number of applications in economics engineering science and many other fields the three most significant classes of algorithms for solving lo problems are pivot ellipsoid and interior point methods because ellipsoid methods are not efficient in practice we will concentrate on the computationally successful simplex and primal dual interior point methods only and summarize the pros and cons of these algorithm classes

neural networks retrieving boolean patterns in a sea of gaussian ones restricted boltzmann machines are key tools in machine learning and are described by the energy function of bipartite spin glasses from a statistical mechanical perspective they share the same gibbs measure of hopfield networks for associative memory in this equivalence weights in the former play as patterns in the latter as boltzmann machines usually require real weights to be trained with gradient descent like methods while hopfield networks typically store binary patterns to be able to retrieve the investigation of a mixed hebbian network equipped with both real e g gaussian and discrete e g boolean patterns naturally arises we prove that in the challenging regime of a high storage of real patterns where retrieval is forbidden an additional load of boolean patterns can still be retrieved as long as the ratio between the overall load and the network size does not exceed a critical threshold that turns out to be the same of the standard amit gutfreund sompolinsky theory assuming replica symmetry we study the case of a low load of boolean patterns combining the stochastic stability and hamilton jacobi interpolating techniques the result can be extended to the high load by a non rigorous but standard replica computation argument

performance improvement of iterative multiuser detection for large sparsely spread cdma systems by spatial coupling kudekar et al proved that the belief propagation bp performance for low density parity check codes can be boosted up to the maximum a posteriori map performance by spatial coupling in this paper spatial coupling is applied to sparsely spread code division multiple access systems to improve the performance of iterative multiuser detection based on bp two iterative receivers based on bp are considered 1 one receiver is based on exact bp and 2 the other on an approximate bp with gaussian approximation the performance of the two bp receivers is evaluated via density evolution de in the dense limit after taking the large system limit in which the number of users and the spreading factor tend to infinity while their ratio is kept constant the two bp receivers are shown to achieve the same performance as each other in these limits furthermore taking a continuum limit for the obtained de equations implies that the performance of the two bp receivers can be improved up to the performance achieved by the symbol wise map detection called individually optimal detection via spatial coupling numerical simulations show that spatial coupling can provide a significant improvement in bit error rate for finite sized systems especially in the region of high system loads

asymptotics for high dimensional regression m estimates fixed design results we investigate the asymptotic distributions of coordinates of regression m estimates in the moderate p n regime where the number of covariates p grows proportionally with the sample size n under appropriate regularity conditions we establish the coordinate wise asymptotic normality of regression m estimates assuming a fixed design matrix our proof is based on the second order poincare inequality and leave one out analysis some relevant examples are indicated to show that our regularity conditions are satisfied by a broad class of design matrices we also show a counterexample namely an anova type design to emphasize that the technical assumptions are not just artifacts of the proof finally numerical experiments confirm and complement our theoretical results

generalized entropy power inequalities and monotonicity properties of information new families of fisher information and entropy power inequalities for sums of independent random variables are presented these inequalities relate the information in the sum of n independent random variables to the information contained in sums over subsets of the random variables for an arbitrary collection of subsets as a consequence a simple proof of the monotonicity of information in central limit theorems is obtained both in the setting of independent and identically distributed i i d summands as well as in the more general setting of independent summands with variance standardized sums

construction of pure states in mean field models for spin glasses if a mean field model for spin glasses is generic in the sense that it satisfies the extended ghirlanda guerra identities and if the law of the overlaps has a point mass at the largest point q of its support we prove that one can decompose the configuration space into a sequence of sets ak such that generically the overlap of two configurations is equal to q if and only if they belong to the same set ak for the study of the overlaps each set ak can be replaced by a single point combining this with a recent result of panchenko a connection between ghirlanda guerra identities and ultrametricity ann probab 2008 to appear this proves that if the overlaps take only finitely many values ultrametricity occurs we give an elementary self contained proof of this result based on simple inequalities and an averaging argument

storage capacity in symmetric binary perceptrons we study the problem of determining the capacity of the binary perceptron for two variants of the problem where the corresponding constraint is symmetric we call these variants the rectangle binary perceptron rpb and the u function binary perceptron ubp we show that unlike for the usual step function binary perceptron the critical capacity in these symmetric cases is given by the annealed computation in a large region of parameter space for all rectangular constraints and for narrow enough u function constraints k k we conclude that full step replica symmetry breaking would have to be evaluated in order to obtain the exact capacity in this case

intersecting random half spaces toward the gardner derrida formula gardner and derrida have introduced a natural version of the problem of the capacity of the binary perceptron with temperature and they proposed based on physical methods remarkable formulas for this model we give a complete rigorous proof that these formulas are correct at sufficiently high temperature for a much larger class of models

a connection between the ghirlanda guerra identities and ultrametricity we consider a symmetric positive definite weakly exchangeable infinite random matrix and show that under the technical condition that its elements take a finite number of values the ghirlanda guerra identities imply ultrametricity

hierarchical exchangeability of pure states in mean field spin glass models the main result in this paper is motivated by the mezard parisi ansatz which predicts a very special structure for the distribution of spins in diluted mean field spin glass models such as the random k sat model at positive temperature using the fact that one can safely assume the validity of the ghirlanda guerra identities in these models we prove hierarchical exchangeability of pure states for the asymptotic gibbs measures which allows us to apply a representation result for hierarchically exchangeable arrays recently proved in austin and panchenko in probab theory relat fields 2013 comparing this representation with the predictions of the mezard parisi ansatz one can see that the key property still missing is that the multi overlaps between pure states depend only on their overlaps

an assumption free exact test for fixed design linear models with exchangeable errors we propose the cyclic permutation test cpt to test general linear hypotheses for linear models this test is non randomized and valid in finite samples with exact type i error alpha for an arbitrary fixed design matrix and arbitrary exchangeable errors whenever 1 alpha is an integer and n p ge 1 alpha 1 the test involves applying the marginal rank test to 1 alpha linear statistics of the outcome vector where the coefficient vectors are determined by solving a linear system such that the joint distribution of the linear statistics is invariant with respect to a non standard cyclic permutation group under the null hypothesis the power can be further enhanced by solving a secondary non linear travelling salesman problem for which the genetic algorithm can find a reasonably good solution extensive simulation studies show that the cpt has comparable power to existing tests when testing for a single contrast of coefficients an exact confidence interval can be obtained by inverting the test furthermore we provide a selective yet extensive literature review of the century long efforts on this problem highlighting the novelty of our test

high dimensional robust m estimation asymptotic variance via approximate message passing in a recent article el karoui et al proc natl acad sci 110 36 14557 14562 2013 study the distribution of robust regression estimators in the regime in which the number of parameters p is of the same order as the number of samples n using numerical simulations and highly plausible heuristic arguments they unveil a striking new phenomenon namely the regression coefficients contain an extra gaussian noise component that is not explained by classical concepts such as the fisher information matrix we show here that that this phenomenon can be characterized rigorously using techniques that were developed by the authors for analyzing the lasso estimator under high dimensional asymptotics we introduce an approximate message passing amp algorithm to compute m estimators and deploy state evolution to evaluate the operating characteristics of amp and so also m estimates our analysis clarifies that the extra gaussian noise encountered in this problem is fundamentally similar to phenomena already studied for regularized least squares in the setting n p

on the finiteness of the criss cross method abstract in this short paper we prove the finiteness of the criss cross method by showing a certain binary number of bounded digits associated with each iteration increases monotonically this new proof immediately suggests the possibility of relaxing the pivoting selection in the criss cross method without sacrificing the finiteness

rigorous results for the hopfield model with many patterns we perform a thorough investigation of the main aspects of the hopfield model with many patterns advances are made toward the validity of the replica symmetric solution strong evidence of the validity of this solution is given over the entire domain where this validity is conjectured complete proof is given in a subregion that contains strictly the ergodic region

on the impact of predictor geometry on the performance on high dimensional ridge regularized generalized robust regression estimators we study ridge regularized generalized robust regression estimators i e begin aligned widehat beta mathrm argmin beta in mathbb r p frac 1 n sum i 1 n rho i y i x i beta frac tau 2 beta 2 quad text where quad y i epsilon i x i beta 0 end aligned in the situation where p n tends to a finite non zero limit our study here focuses on the situation where the errors epsilon i s are heavy tailed and x i s have an elliptical like distribution our assumptions are quite general and we do not require homoskedasticity of epsilon i s for instance we obtain a characterization of the limit of widehat beta beta 0 as well as several other results including central limit theorems for the entries of widehat beta

free energy of the spherical mean field model we compute at any temperature the free energy of the multi p spin spherical model when only terms for p even are considered

the parisi ultrametricity conjecture in this paper we prove that the support of a random measure on the unit ball of a separable hilbert space that satises the ghirlanda guerra identities must be ultrametric with probability one this implies the parisi ultrametricity conjecture in mean eld spin glass models such as the sherrington kirkpatrick and mixed p spin models for which gibbs measures are known to satisfy the ghirlanda guerra identities in the thermodynamic limit

a generalization of the lindeberg principle we generalize lindeberg s proof of the central limit theorem to an invariance principle for arbitrary smooth functions of independent and weakly dependent random variables the result is applied to get a similar theorem for smooth functions of exchangeable random variables this theorem allows us to identify for the first time the limiting spectral distributions of wigner matrices with exchangeable entries

a statistical mechanics approach to large system analysis of cdma multiuser detectors we present a theory based on statistical mechanics to evaluate analytically the performance of uncoded fully synchronous randomly spread code division multiple access cdma multiuser detectors with additive white gaussian noise awgn channel under perfect power control and in the large system limit application of the replica method a tool developed in the literature of statistical mechanics allows us to derive analytical expressions for the bit error rate as well as the multiuser efficiency of the individually optimum io and jointly optimum jo multiuser detectors over the whole range of noise levels the information theoretic capacity of the randomly spread cdma channel and the performance of decorrelating and linear minimum mean square error mmse detectors are also derived in the same replica formulation thereby demonstrating validity of the statistical mechanical approach

large system analysis of multiuser detection with an unknown number of users a high snr approach we analyze multiuser detection under the assumption that the number of users accessing the channel is unknown by the receiver in this environment users activity must be estimated along with any other parameters such as data power and location our main goal is to determine the performance loss caused by the need for estimating the identities of active users which are not known a priori to prevent a loss of optimality we assume that identities and data are estimated jointly rather than in two separate steps we examine the performance of multiuser detectors when the number of potential users is large statistical physics methodologies are used to determine the macroscopic performance of the detector in terms of its multiuser efficiency special attention is paid to the fixed point equation whose solution yields the multiuser efficiency of the optimal maximum a posteriori detector in the large signal to noise ratio regime our analysis yields closed form approximate bounds to the minimum mean squared error in this regime these illustrate the set of solutions of the fixed point equation and their relationship with the maximum system load next we study the maximum load that the detector can support for a given quality of service specified by error probability

the space of interactions in neural network models the typical fraction of the space of interactions between each pair of n ising spins which solve the problem of storing a given set of p random patterns as n bit spin configurations is considered the volume is calculated explicitly as a function of the storage ratio alpha p n of the value kappa 0 of the product of the spin and the magnetic field at each site and of the magnetisation m here m may vary between 0 no correlation and 1 completely correlated the capacity increases with the correlation between patterns from alpha 2 for correlated patterns with kappa 0 and tends to infinity as m tends to 1 the calculations use a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is shown to be locally stable a local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist

phase diagram of restricted boltzmann machines and generalized hopfield networks with arbitrary priors restricted boltzmann machines are described by the gibbs measure of a bipartite spin glass which in turn can be seen as a generalized hopfield network this equivalence allows us to characterize the state of these systems in terms of their retrieval capabilities both at low and high load of pure states we study the paramagnetic spin glass and the spin glass retrieval phase transitions as the pattern i e weight distribution and spin i e unit priors vary smoothly from gaussian real variables to boolean discrete variables our analysis shows that the presence of a retrieval phase is robust and not peculiar to the standard hopfield model with boolean patterns the retrieval region becomes larger when the pattern entries and retrieval units get more peaked and conversely when the hidden units acquire a broader prior and therefore have a stronger response to high fields moreover at low load retrieval always exists below some critical temperature for every pattern distribution ranging from the boolean to the gaussian case

rigorous low temperature results for the mean field p spins interaction model we prove that just below the critical temperature the mean field p spins interaction model for p suitably large spontaneously decomposes into different states the asymptotic overlaps between any two different states are zero under a mild unproven hypothesis on the weight distribution of these states we prove that they are pure states this situation is called in physics one level of symmetry breaking

on robust regression with high dimensional predictors we study regression m estimates in the setting where p the number of covariates and n the number of observations are both large but p n we find an exact stochastic representation for the distribution of argmin p i 1 n y i x i at fixed p and n under various assumptions on the objective function and our statistical model a scalar random variable whose deterministic limit r can be studied when p n 0 plays a central role in this representation we discover a nonlinear system of two deterministic equations that characterizes r interestingly the system shows that r depends on through proximal mappings of as well as various aspects of the statistical model underlying our study several surprising results emerge in particular we show that when p n is large enough least squares becomes preferable to least absolute deviations for double exponential errors

non negative principal component analysis message passing algorithms and sharp asymptotics principal component analysis pca aims at estimating the direction of maximal variability of a high dimensional data set a natural question is does this task become easier and estimation more accurate when we exploit additional knowledge on the principal vector we study the case in which the principal vector is known to lie in the positive orthant similar constraints arise in a number of applications ranging from the analysis of gene expression data to spike sorting in neural signal processing in the unconstrained case the estimation performances of pca have been precisely characterized using the random matrix theory under a statistical model known as the spiked model it is known that the estimation error undergoes a phase transition as the signal to noise ratio crosses a certain threshold unfortunately tools from the random matrix theory have no bearing on the constrained problem despite this challenge we develop an analogous characterization in the constrained case within a one spike model in particular 1 we prove that the estimation error undergoes a similar phase transition albeit at a different thresholds in signal to noise ratio that we determine exactly 2 we prove that unlike in the unconstrained case the estimation error depends on the spike vector and characterize the least favorable vectors and 3 we show that a non negative principal component can be approximately computed under the spiked model in nearly linear time this despite the fact that the problem is non convex and in general np hard to solve exactly

a modern maximum likelihood theory for high dimensional logistic regression students in statistics or data science usually learn early on that when the sample size n is large relative to the number of variables p fitting a logistic model by the method of maximum likelihood produces estimates that are consistent and that there are well known formulas that quantify the variability of these estimates which are used for the purpose of statistical inference we are often told that these calculations are approximately valid if we have 5 to 10 observations per unknown parameter this paper shows that this is far from the case and consequently inferences produced by common software packages are often unreliable consider a logistic model with independent features in which n and p become increasingly large in a fixed ratio we prove that i the maximum likelihood estimate mle is biased ii the variability of the mle is far greater than classically estimated and iii the likelihood ratio test lrt is not distributed as a 2 the bias of the mle yields wrong predictions for the probability of a case based on observed values of the covariates we present a theory which provides explicit expressions for the asymptotic bias and variance of the mle and the asymptotic distribution of the lrt we empirically demonstrate that these results are accurate in finite samples our results depend only on a single measure of signal strength which leads to concrete proposals for obtaining accurate inference in finite samples through the estimate of this measure

fundamental limits of symmetric low rank matrix estimation we consider the high dimensional inference problem where the signal is a low rank symmetric matrix which is corrupted by an additive gaussian noise given a probabilistic model for the low rank matrix we compute the limit in the large dimension setting for the mutual information between the signal and the observations as well as the matrix minimum mean squared error while the rank of the signal remains constant we also show that our model extends beyond the particular case of additive gaussian noise and we prove an universality result connecting the community detection problem to our gaussian framework we unify and generalize a number of recent works on pca sparse pca submatrix localization or community detection by computing the information theoretic limits for these problems in the high noise regime in addition we show that the posterior distribution of the signal given the observations is characterized by a parameter of the same dimension as the square of the rank of the signal i e scalar in the case of rank one this allows to locate precisely the information theoretic thresholds for the above mentioned problems finally we connect our work with the hard but detectable conjecture in statistical physics

finite sample analysis of approximate message passing algorithms approximate message passing amp refers to a class of efficient algorithms for statistical estimation in high dimensional problems such as compressed sensing and low rank matrix estimation this paper analyzes the performance of amp in the regime where the problem dimension is large but finite for concreteness we consider the setting of high dimensional regression where the goal is to estimate a high dimensional vector beta 0 from a noisy measurement y a beta 0 w amp is a low complexity scalable algorithm for this problem under suitable assumptions on the measurement matrix a amp has the attractive feature that its performance can be accurately characterized in the large system limit by a simple scalar iteration called state evolution previous proofs of the validity of state evolution have all been asymptotic convergence results in this paper we derive a concentration inequality for amp with gaussian matrices with independent and identically distributed i i d entries and finite dimension n times n the result shows that the probability of deviation from the state evolution prediction falls exponentially in n this provides theoretical support for empirical findings that have demonstrated excellent agreement of amp performance with state evolution predictions for moderately large dimensions the concentration inequality also indicates that the number of amp iterations t can grow no faster than order log n log log n for the performance to be close to the state evolution predictions with high probability the analysis can be extended to obtain similar non asymptotic results for amp in other settings such as low rank matrix estimation

the replica symmetric solution without replica trick for the hopfield model we derive the saddle point equations for the order parameters of the hopfield model in the case of replica symmetry without using the replica trick but assuming that the edwards anderson parameter is a self averaging quantity

new variants of the criss cross method for linearly constrained convex quadratic programming abstract in this paper s zhang s eur j oper res 116 1999 607 new and more flexible criss cross type algorithms with lifo and most often selected variable pivot rules are generalized for linearly constrained convex primal dual quadratic programming problems these criss cross type algorithms are different from the one described in klafszky and terlaky math oper und stat ser optim 24 1992 127 even though the finiteness proof of these new criss cross type algorithms is similar to the original one for the algorithm of klafszky and terlaky in the sense that both these proofs are based on the orthogonality theorem more cases have to be considered due to the flexibility of pivot lifo most often selected variable rules which requires a deeper and more careful analysis when the primal dual problem is a linear programming problem no quadratic terms in the objective function the structure of the corresponding linear complementarity problem is simpler i e the matrix of the problem is skew symmetric for such problem pairs our proof of finiteness simplifies to the proof of illes and meszaros yugoslav j oper res 11 2001 17 and provides a new finiteness proof for s zhang s criss cross type algorithms

new variants of finite criss cross pivot algorithms for linear programming abstract in this paper we generalize the so called first in last out pivot rule and the most often selected variable pivot rule for the simplex method as proposed in s zhang operations research letters 10 1991 189 192 to the criss cross pivot setting where neither the primal nor the dual feasibility is preserved the finiteness of the new criss cross pivot variants is proven

the geometry of the gibbs measure of pure spherical spin glasses we analyze the statics for pure p spin spherical spin glass models with p ge 3 at low enough temperature with f n beta denoting the free energy we compute the logarithmic second order term of nf n beta and prove that for an appropriate centering c n beta nf n beta c n beta is a tight sequence we further establish the absence of temperature chaos those results follow from the following geometric picture we prove for the gibbs measure of interest by itself asymptotically the measure splits into infinitesimal spherical bands centered at deep minima playing the role of pure states for the pure models the latter makes precise the picture of many valleys separated by high mountains and significant parts of the tap analysis from the physics literature