on degrees of freedom of projection estimators with applications to multivariate nonparametric regression abstract in this article we consider the nonparametric regression problem with multivariate predictors we provide a characterization of the degrees of freedom and divergence for estimators of the

symmetry and phaselocking in chains of weakly coupled oscillators weakly coupled chains of oscillators with nearest neighbor interactions are analyzed for phaselocked solutions it is shown that the symmetry properties of the coupling affect the qualitative form of the phaselocked solutions and the scaling behavior of the system as the number of oscillators grows without bound it is also shown that qualitative behavior of these solutions depends on whether the coupling is diffusive or i synaptic terms defined in the paper the methods include the demonstration that the equations for phaselocked solutions can be approximated by a singularly perturbed two point continuum boundary value problem that is easier to analyze the issue of convergence of the phaselocked solutions to solutions of the continuum equation is closely related to questions involving numerical entropy in computation schemes for a conservation law an application to the neurophysiology of motor behavior is discussed briefly

covariance regularization by thresholding this paper considers regularizing a covariance matrix of p variables estimated from n observations by hard thresholding we show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense the variables are gaussian or sub gaussian and log p n 0 and obtain explicit rates the results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity we discuss an intuitive resampling scheme for threshold selection and prove a general cross validation result that justifies this approach we also compare thresholding to other covariance estimators in simulations and on an example from climate data 1 introduction estimation of covariance matrices is important in a number of areas of statistical analysis including dimension reduction by principal component analysis pca classification by linear or quadratic discriminant analysis lda and qda establishing independence and conditional independence relations in the context of graphical models and setting confidence intervals on linear functions of the means of the components in recent years many application areas where these tools are used have been dealing with very high dimensional datasets and sample sizes can be very small relative to dimension examples include genetic data brain imaging spectroscopic imaging climate data and many others it is well known by now that the empirical covariance matrix for samples of size n from a p variate gaussian distribution np p is not a good estimator of the population covariance if p is large many results in random matrix theory illustrate this from the classical mary law 29 to the more recent work of johnstone and his students on the theory of the largest eigenvalues 12 23 30 and associated eigenvectors 24 however with the exception of a method for estimating the covariance spectrum 11 these probabilistic results do not offer alternatives to the sample covariance matrix alternative estimators for large covariance matrices have therefore attracted a lot of attention recently two broad classes of covariance estimators have emerged those that rely on a natural ordering among variables and assume that variables

synchronization in lattices of coupled oscillators abstract we consider coupled nonlinear oscillators with external periodic forces and the dirichlet boundary conditions we prove that synchronization occurs provided that the coupling is dissipative and the coupling coefficients are sufficiently large the synchronization here is of an obvious type the size of an attractor is comparable to the difference of the subsystems

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

model evaluation based on the sampling distribution of estimated absolute prediction error the construction of a reliable practically useful prediction rule for future responses is heavily dependent on the adequacy of the fitted regression model in this article we consider the absolute prediction error the expected value of the absolute difference between the future and predicted responses as the model evaluation criterion this prediction error is easier to interpret than the average squared error and is equivalent to the misclassification error for a binary outcome we show that the prediction error can be consistently estimated via the resubstitution and crossvalidation methods even when the fitted model is not correctly specified furthermore we show that the resulting estimators are asymptotically normal when the prediction rule is nonsmooth the variance of the above normal distribution can be estimated well with a perturbation resampling method with two real examples and an extensive simulation study we demonstrate that the interval estimates obtained from the above normal approximation for the prediction errors provide much more information about model adequacy than their point estimate counterparts

analysis of variance of cross validation estimators of the generalization error this paper brings together methods from two different disciplines statistics and machine learning we address the problem of estimating the variance of cross validation cv estimators of the generalization error in particular we approach the problem of variance estimation of the cv estimators of generalization error as a problem in approximating the moments of a statistic the approximation illustrates the role of training and test sets in the performance of the algorithm it provides a unifying approach to evaluation of various methods used in obtaining training and test sets and it takes into account the variability due to different training and test sets for the simple problem of predicting the sample mean and in the case of smooth loss functions we show that the variance of the cv estimator of the generalization error is a function of the moments of the random variables y card sj sj and y card sjc sj c where sj sj are two training sets and sjc sj c are the corresponding test sets we prove that the distribution of y and y is hypergeometric and we compare our estimator with the one proposed by nadeau and bengio 2003 we extend these results in the regression case and the case of absolute error loss and indicate how the methods can be extended to the classification case we illustrate the results through simulation

on the dynamical behaviour of fitzhugh nagumo systems revisited the purpose of this paper is to analyse a general form of the fitzhugh nagumo model as completely as possible the main result is that no more than two limit cycles can be bifurcated from the unique fixed point via hopf bifurcation and there exist parameters such that this upper bound is attained for these parameters the stability of the inner and outer cycle together with the unique fixed point is also established the results are approached through lyapunov coefficients and rely on a theorem by andronov and aleksandrovic a a andronov a a aleksandrovic theory of bifurcations of dynamical system on a plane wiley 1971 based on singular perturbation theory a sufficient condition for existence of a unique stable limit cycle is given under certain assumptions

threshold selection for covariance estimation thresholding is a regularization method commonly used for covariance estimation which provides consistent estimators if the population covariance satisfies certain sparsity condition bickel and levina 2008a cai and liu 2011 however the performance of the thresholding estimators heavily depends on the threshold level by minimizing the frobenius risk of the adaptive thresholding estimator for covariances we conduct a theoretical study for the optimal threshold level and obtain its analytical expression a consistent estimator based on this expression is proposed for the optimal threshold level which is easy to implement in practice and efficient in computation numerical simulations and a case study on gene expression data are conducted to illustrate the proposed method

dynamics of two strongly coupled relaxation oscillators this paper concerns the dynamics of a pair of identical linearly coupled van der pol relaxation oscillators we study the stability of the in phase and out of phase modes of vibration the stability of both modes is shown to be governed by the behavior of a linear variational equation with periodic coefficients approximate analytical solutions are obtained by the method of matched asymptotic expansions these analytical results are supplemented by numerical integrations based on floquet theory it is shown that previous work based on the sinusoidal nonrelaxation limit fails to predict a significant region of instability for both modes

on the distribution of the largest eigenvalue in principal components analysis let x 1 denote the square of the largest singular value of an n p matrix x all of whose entries are independent standard gaussian variates equivalently x 1 is the largest principal component variance of the covariance matrix x x or the largest eigenvalue of a p variate wishart distribution on n degrees of freedom with identity covariance consider the limit of large p and n with n p gamma ge 1 when centered by mu p sqrt n 1 sqrt p 2 and scaled by sigma p sqrt n 1 sqrt p 1 sqrt n 1 1 sqrt p 1 3 the distribution of x 1 approaches the tracey widom law of order 1 which is defined in terms of the painleve ii differential equation and can be numerically evaluated and tabulated in software simulations show the approximation to be informative for n and p as small as 5 the limit is derived via a corresponding result for complex wishart matrices using methods from random matrix theory the result suggests that some aspects of large p multivariate distribution theory may be easier to apply in practice than their fixed p counterparts

operator norm consistent estimation of large dimensional sparse covariance matrices estimating covariance matrices is a problem of fundamental importance in multivariate statistics in practice it is increasingly frequent to work with data matrices x of dimension n x p where p and n are both large results from random matrix theory show very clearly that in this setting standard estimators like the sample covariance matrix perform in general very poorly in this large n large p setting it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0 and hence this matrix is sparse we develop an estimator to handle this situation the estimator is shown to be consistent in operator norm when for instance we have p n as n oo in other words the largest singular value of the difference between the estimator and the population covariance matrix goes to zero this implies consistency of all the eigenvalues and consistency of eigenspaces associated to isolated eigenvalues we also propose a notion of sparsity for matrices that is compatible with spectral analysis and is independent of the ordering of the variables

sure estimates for a heteroscedastic hierarchical model hierarchical models are extensively studied and widely used in statistics and many other scientific areas they provide an effective tool for combining information from similar resources and achieving partial pooling of inference since the seminal work by james and stein 1961 and stein 1962 shrinkage estimation has become one major focus for hierarchical models for the homoscedastic normal model it is well known that shrinkage estimators especially the james stein estimator have good risk properties the heteroscedastic model though more appropriate for practical applications is less well studied and it is unclear what types of shrinkage estimators are superior in terms of the risk we propose in this article a class of shrinkage estimators based on stein s unbiased estimate of risk sure we study asymptotic properties of various common estimators as the number of means to be estimated grows p we establish the asymptotic optimality property for the sure estimators we then extend our

some large primes and amicable numbers some new large primes of the form 3 2 1 and 9 r 1 related to amicable numbers are given two new large amicable number pairs are found by the method of so called thabit rules 1 the numbers 3 2126761 and 9 211547 1 are primes the only larger known primes seem to be the four recent mersenne primes 2 1 for n 19937 21701 23209 and 44497 see 3 the only primes of the form 3 2n 1 resp 9 2n 1 with n 1 the two numbers q u l p 1 and q2 p u u l p 1 are simultaneously prime then aup fql and ap nq2 are amicable the constants a u p have to be specifically chosen for instance a 23 a 37 59 u 23 52391 p 2462401 resp a 34 72 11 19 u 233 3821 p 1784641 give rules we generated 37 new such rules by the procedure described in 1 theorem 4 the two rules mentioned above actually give amicable numbers both for n 2 the numbers have 42 resp 43 digits we also confirmed and extended te riele s computations 5 who found three large amicable pairs by means of thabit rules derived from known amicable pairs applying theorem 3 of 1 4 for all computations on primes mentioned we used primality tests of lucasian type with certain new simplifications adopting them to the specific type of numbers considered in section 1 resp 3 these slight improvements in theory are to be published in some detail elsewhere along with more material on the received february 21 1980 1980 mathematics subject classification primary ioa40 ioa25 secondary 10a35 10 04 1981 american mathematical society 0025 5718 81 0000 0031 01 50 303 this content downloaded from 157 55 39 116 on sun 18 sep 2016 04 19 51 utc all use subject to http about jstor org terms

when does more regularization imply fewer degrees of freedom sufficient conditions and counterexamples regularization aims to improve prediction performance by trading an increase in training error for better agreement between training and prediction errors which is often captured through decreased degrees of freedom in this paper we give examples which show that regularization can increase the degrees of freedom in common models including the lasso and ridge regression in such situations both training error and degrees of freedom increase making the regularization inherently without merit two important scenarios are described where the expected reduction in degrees of freedom is guaranteed all symmetric linear smoothers and convex constrained linear regression models like ridge regression and the lasso when compared to unconstrained linear regression

synchronization rates in classes of relaxation oscillators relaxation oscillators arise frequently in physics electronics mathematics and biology their mathematical definitions possess a high degree of flexibility in the sense that through appropriate parameter choices relaxation oscillators can be made to exhibit qualitatively different kinds of oscillations we study numerically four different classes of relaxation oscillators through their synchronization rates in one dimensional chains with a heaviside step function interaction and obtain the following results relaxation oscillators in the sinusoidal and relaxation regime both exhibit an average time to synchrony spl sim n where n is the chain length relaxation oscillators in the singular limit exhibit spl sim n sup p where p is a numerically obtained value less than 0 5 relaxation oscillators in the singular limit with parameters modified so that they resemble spike oscillations exhibit spl sim log n in chains and spl sim log l in two dimensional square networks of length l finally using a sigmoid interaction results in spl sim n sup 2 for relaxation oscillators in the sinusoidal and relaxation regimes indicating that the form of the coupling is a controlling factor in the synchronization rate

convex banding of the covariance matrix abstractwe introduce a new sparse estimator of the covariance matrix for high dimensional models in which the variables have a known ordering our estimator which is the solution to a convex optimization problem is equivalently expressed as an estimator that tapers the sample covariance matrix by a toeplitz sparsely banded data adaptive matrix as a result of this adaptivity the convex banding estimator enjoys theoretical optimality properties not attained by previous banding or tapered estimators in particular our convex banding estimator is minimax rate adaptive in frobenius and operator norms up to log factors over commonly studied classes of covariance matrices and over more general classes furthermore it correctly recovers the bandwidth when the true covariance is exactly banded our convex formulation admits a simple and efficient algorithm empirical studies demonstrate its practical effectiveness and illustrate that our exactly banded estimator works well even when the true covariance

the estimation of prediction error having constructed a data based estimation rule perhaps a logistic regression or a classification tree the statistician would like to know its performance as a predictor of future cases there are two main theories concerning prediction error 1 penalty methods such as cp akaike s information criterion and stein s unbiased risk estimate that depend on the covariance between data points and their corresponding predictions and 2 cross validation and related nonparametric bootstrap techniques this article concerns the connection between the two theories a rao blackwell type of relation is derived in which nonparametric methods such as cross validation are seen to be randomized versions of their covariance penalty counterparts the model based penalty methods offer substantially better accuracy assuming that the model is believable

regularized estimation of large covariance matrices this paper considers estimating a covariance matrix of p variables from n observations by either banding the sample covariance matrix or estimating a banded version of the inverse of the covariance we show that these estimates are consistent in the operator norm as long as logp 2 n 0 and obtain explicit rates the results are uniform over some fairly natural well conditioned families of covariance matrices we also introduce an analogue of the gaussian white noise model and show that if the population covariance is embeddable in that model and well conditioned then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix the results can be extended to smooth versions of banding and to non gaussian distributions with su ciently short tails a resampling approach is proposed for choosing the banding parameter in practice this approach is illustrated numerically on both simulated and real data

the little bootstrap and other methods for dimensionality selection in regression x fixed prediction error abstract when a regression problem contains many predictor variables it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor variables usually a regression equation based on a few variables will be more accurate and certainly simpler there are various methods for picking good subsets of variables and programs that do such procedures are part of every widely used statistical package the most common methods are based on stepwise addition or deletion of variables and on best subsets the latter refers to a search method that given the number of variables to be in the equation say five locates that regression equation based on five variables that has the lowest residual sum of squares among all five variable equations all of these procedures generate a sequence of regression equations the first based on one variable the next based on two variables and so on each member of this sequence is called a submodel and the number of variables in the e

improvements on cross validation the 632 bootstrap method abstract a training set of data has been used to construct a rule for predicting future responses what is the error rate of this rule this is an important question both for comparing models and for assessing a final selected model the traditional answer to this question is given by cross validation the cross validation estimate of prediction error is nearly unbiased but can be highly variable here we discuss bootstrap estimates of prediction error which can be thought of as smoothed versions of cross validation we show that a particular bootstrap method the 632 rule substantially outperforms cross validation in a catalog of 24 simulation experiments besides providing point estimates we also consider estimating the variability of an error rate estimate all of the results here are nonparametric and apply to any possible prediction rule however we study only classification problems with 0 1 loss in detail our simulations include smooth prediction rules like fisher s linear discriminant fun

spatial shrinkage estimation of diffusion tensors on diffusion weighted imaging data diffusion tensor imaging dti based on the diffusion weighted imaging dwi data acquired from magnetic resonance experiments has been widely used to analyze the physical structure of white matter fibers in the human brain in vivo the raw dwi data however carry noise this contaminates the diffusion tensor dt estimates and introduces systematic bias into the induced eigenvalues these bias components affect the effectiveness of fiber tracking algorithms in this article we propose a two stage spatial shrinkage estimation spske procedure to accommodate the spatial information carried in dwi data in dt estimation and to reduce the bias components in the corresponding derived eigenvalues to this end in the framework of the heteroscedastic linear model spske incorporates l 1 type penalization and the locally weighted least square function the theoretical properties of spske are explored the effectiveness of spske is further illustrated by simulation and real data examples supplementary materi

distributed output regulation of nonlinear multi agent systems via host internal model this note presents a host internal model h im approach to distributed output regulation of reaching nonlinear leader following consensus focusing on a basic nonlinear scenario of identical or homogeneous agents of a normal form it is shown that the problem is solvable as long as its interaction digraph contains a certain directed spanning tree a constructive lyapunov protocol is proposed by incorporating a single h im the result is also demonstrated by fitzhugh nagumo fhn and lorenz type nonlinear dynamic networks

computation of all the amicable paris below 10 10 an efficient exhaustive numerical search method for amicable pairs is described with the aid of this method all 1427 amicable pairs with smaller member below 1010 have been computed more than 800 pairs being new this extends previous exhaustive work below 108 by h cohen in three appendices contained in the supplements section of this issue various statistics are given including an ordered list of all the gcd s of the 1427 amicable pairs below 1010 which may be useful in further amicable pair research suggested by the numerical results a theorem of borho and hoffmann for constructing aps has been extended

quantile regression in reproducing kernel hilbert spaces in this article we consider quantile regression in reproducing kernel hilbert spaces which we call kernel quantile regression kqr we make three contributions 1 we propose an efficient algorithm that computes the entire solution path of the kqr with essentially the same computational cost as fitting one kqr model 2 we derive a simple formula for the effective dimension of the kqr model which allows convenient selection of the regularization parameter and 3 we develop an asymptotic theory for the kqr model

optimal shrinkage estimation of mean parameters in family of distributions with quadratic variance this paper discusses the simultaneous inference of mean parameters in a family of distributions with quadratic variance function we first introduce a class of semi parametric parametric shrinkage estimators and establish their asymptotic optimality properties two specific cases the location scale family and the natural exponential family with quadratic variance function are then studied in detail we conduct a comprehensive simulation study to compare the performance of the proposed methods with existing shrinkage estimators we also apply the method to real data and obtain encouraging results

adaptive covariance matrix estimation through block thresholding estimation of large covariance matrices has drawn considerable recent attention and the theoretical focus so far has mainly been on developing a minimax theory over a fixed parameter space in this paper we consider adaptive covariance matrix estimation where the goal is to construct a single procedure which is minimax rate optimal simultaneously over each parameter space in a large collection a fully data driven block thresholding estimator is proposed the estimator is constructed by carefully dividing the sample covariance matrix into blocks and then simultaneously estimating the entries in a block by thresholding the estimator is shown to be optimally rate adaptive over a wide range of bandable covariance matrices a simulation study is carried out and shows that the block thresholding estimator performs well numerically some of the technical tools developed in this paper can also be of independent interest

test for bandedness of high dimensional covariance matrices and bandwidth estimation motivated by the latest effort to employ banded matrices to estimate a high dimensional covariance we propose a test for being banded with possible diverging bandwidth the test is adaptive to the large p small n situations without assuming a specific parametric distribution for the data we also formulate a consistent estimator for the bandwidth of a banded high dimensional covariance matrix the properties of the test and the bandwidth estimator are investigated by theoretical evaluations and simulation studies as well as an empirical analysis on a protein mass spectroscopy data

multimodal regimes in a compartmental model of the dopamine neuron we study chains of strongly electrically coupled relaxation oscillators modeling dopamine neurons when individual oscillators are in the regime close to an andronov hopf bifurcation ahb the coupled system exhibits a variety of oscillatory behavior we show that the proximity of individual oscillators to the ahb has a significant impact on the system dynamics in a wide range of parameters it manifests itself through a family of stable multimodal periodic solutions that are composed out of large amplitude relaxation oscillations and small amplitude oscillations this family of solutions has a rich bifurcation structure the waveform and the period vary greatly across the family the structure and bifurcations of the stable periodic solutions of the coupled system are investigated using numerical and analytic techniques

jump information criterion for statistical inference in estimating discontinuous curves nonparametric regression analysis when the regression function is discontinuous has many applications existing methods for estimating a discontinuous regression curve usually assume that the number of jumps in the regression curve is known beforehand which is unrealistic in some situations although there has been research on estimation of a discontinuous regression curve when the number of jumps is unknown the problem remains mostly open because such research often requires assumptions on other related quantities such as a known minimum jump size in this paper we propose a jump information criterion which consists of a term measuring the fidelity of the estimated regression curve to the observed data and a penalty related to the number of jumps and the jump sizes the number of jumps can then be determined by minimizing our criterion theoretical and numerical studies show that our method works well

evaluating prediction rules for t year survivors with censored regression models suppose that we are interested in establishing simple but reliable rules for predicting future t year survivors through censored regression models in this article we present inference procedures for evaluating such binary classification rules based on various prediction precision measures quantified by the overall misclassification rate sensitivity and specificity and positive and negative predictive values specifically under various working models we derive consistent estimators for the above measures through substitution and cross validation estimation procedures furthermore we provide large sample approximations to the distributions of these nonsmooth estimators without assuming that the working model is correctly specified confidence intervals for example for the difference of the precision measures between two competing rules can then be constructed all of the proposals are illustrated with real examples and their finite sample properties are evaluated through a simulation study

sure information criteria for large covariance matrix estimation and their asymptotic properties consider n independent and identically distributed p dimensional gaussian random vectors with covariance matrix sigma the problem of estimating sigma when p is much larger than n has received a lot of attention in recent years yet little is known about the information criterion for covariance matrix estimation how to properly define such a criterion and what are the statistical properties we attempt to answer these questions in this paper by focusing on the estimation of bandable covariance matrices when p n but log p o n motivated by the deep connection between stein s unbiased risk estimation sure and akaike information criterion aic in regression models we propose a family of generalized sure sure c indexed by c for covariance matrix estimation where c is some constant when c is 2 sure2 provides an unbiased estimator of the frobenius risk of the covariance matrix estimator furthermore we show that by minimizing sure2 over all possible banding covariance matrix estimators we attain the minimax optimal rate of convergence under the frobenius norm and the resulting estimator behaves like the covariance matrix estimator obtained by the so called oracle tuning when the true covariance matrix is exactly banded we prove that by minimizing text sure log n we select the true bandwidth with probability tending to one therefore our analysis indicates that sure2 and text sure log n can be regarded as the aic and bayesian information criterion for large covariance matrix estimation respectively

the covariance inflation criterion for adaptive model selection we propose a new criterion for model selection in prediction problems the covariance inflation criterion adjusts the training error by the average covariance of the predictions and responses when the prediction rule is applied to permuted versions of the data set this criterion can be applied to general prediction problems e g regression or classification and to general prediction rules e g stepwise regression tree based models and neural nets as a by product we obtain a measure of the effective number of parameters used by an adaptive procedure we relate the covariance inflation criterion to other model selection procedures and illustrate its use in some regression and classification problems we also revisit the conditional bootstrap approach to model selection

phase equations for relaxation oscillators we use the malkin theorem to derive phase equations for networks of weakly connected relaxation oscillators we find an explicit formula for the connection functions when the oscillators have one dimensional slow variables the functions are discontinuous in the relaxation limit mu rightarrow 0 which provides a simple alternative illustration to the major conclusion of the fast threshold modulation ftm theory by somers and kopell biological cybernetics 68 1993 pp 393 407 that synchronization of relaxation oscillators has properties that are quite different from those of smooth nonrelaxation oscillators we use bonhoeffer van der pol relaxation oscillators to illustrate the theory numerically

on the degrees of freedom of the lasso we study the effective degrees of freedom of the lasso in the framework of stein s unbiased risk estimation sure we show that the number of nonzero coefficients is an unbiased estimate forthe degrees of freedom of the lasso a conclusion that requires no special assumption on the predictors in addition the unbiased estimator is shown to be asymptotically consistent with these results on hand various model selection criteria cp aic and bic are available which along with the lars algorithm provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least squares fit

from stein s unbiased risk estimates to the method of generalized cross validation this paper concerns the method of generalized cross validation gcv a promising way of choosing between linear estimates based on stein estimates and the associated unbiased risk estimates stein 1981 a new approach to gcv is developed many consistency results are obtained for the cross validated steinized estimates in the contexts of nearest neighbor nonparametric regression model selection ridge regression and smoothing splines moreover the associated stein s unbiased risk estimate is shown to be uniformly consistent in assessing the true loss not the risk consistency properties are examined as well when the sampling error is unknown finally we propose a variant of gcv to handle the case that the dimension of the raw data is known to be greater than that of their expected values

adaptive thresholding for sparse covariance matrix estimation in this article we consider estimation of sparse covariance matrices and propose a thresholding procedure that is adaptive to the variability of individual entries the estimators are fully data driven and demonstrate excellent performance both theoretically and numerically it is shown that the estimators adaptively achieve the optimal rate of convergence over a large class of sparse covariance matrices under the spectral norm in contrast the commonly used universal thresholding estimators are shown to be suboptimal over the same parameter spaces support recovery is discussed as well the adaptive thresholding estimators are easy to implement the numerical performance of the estimators is studied using both simulated and real data simulation results demonstrate that the adaptive thresholding estimators uniformly outperform the universal thresholding estimators the method is also illustrated in an analysis on a dataset from a small round blue cell tumor microarray experiment a supplement to this ar

how biased is the apparent error rate of a prediction rule abstract a regression model is fitted to an observed set of data how accurate is the model for predicting future observations the apparent error rate tends to underestimate the true error rate because the data have been used twice both to fit the model and to check its accuracy we provide simple estimates for the downward bias of the apparent error rate the theory applies to general exponential family linear models and general measures of prediction error special attention is given to the case of logistic regression on binary data with error rates measured by the proportion of misclassified cases several connected ideas are compared mallows s cp cross validation generalized cross validation the bootstrap and akaike s information criterion

global robust distributed output consensus of multi agent nonlinear systems an internal model approach abstract this paper studies the problem of global robust distributed output consensus of heterogeneous leader follower multi agent nonlinear systems by general directed output interactions for a class of minimum phase single input single output nonlinear agents having unity relative degree it is shown that the problem is solvable by an internal model approach under certain mild conditions a lyapunov function based output feedback control law is developed by converting the global output consensus into a global distributed stabilization problem for an augmented network

discussion local rademacher complexities and oracle inequalities in risk minimization in this magnificent paper professor koltchinskii offers general and powerful performance bounds for empirical risk minimization a fundamental principle of statistical learning theory since the elegant pioneering work of vapnik and chervonenkis in the early 1970s various such bounds have been known that relate the performance of empirical risk minimizers to combinatorial and geometrical features of the class over which the minimization is performed this area of research has been a rich source of motivation and a major field of applications of empirical process theory the appearance of advanced concentration inequalities in the 1990s primarily thanks to talagrand s influential work provoked major advances in both empirical process theory and statistical learning theory and led to a much deeper understanding of some of the basic phenomena in the discussed paper professor koltchinskii develops a powerful new methodology iterative localization which with the help of concentration inequalities is able to explain most of the recent results and go significantly beyond them in many cases the main motivation behind professor koltchinskii s paper is based on classical problems of statistical learning theory such as binary classification and regression in which given a sample xi y i i 1 n of independent and identically distributed pairs of random variables where the xi take their values in some feature space x and the yi are say real valued the goal is to find a function f x r whose risk defined in terms of the expected value of an appropriately chosen loss function is as small as possible in the remaining part of this discussion we point out how the performance bounds of professor koltchinskii s paper can be used to study a seemingly different model motivated by nonparametric ranking problems which has received increasing attention both in the statistical and machine learning literature indeed in several applications such as the search engine problem or credit risk screening the goal is to learn how to rank or to score observations rather than just classify them in this case performance measures involve pairs of observations as can be seen for instance with the auc area under an roc curve criterion in this

on measuring and correcting the effects of data mining and model selection abstract in the theory of linear models the concept of degrees of freedom plays an important role this concept is often used for measurement of model complexity for obtaining an unbiased estimate of the error variance and for comparison of different models i have developed a concept of generalized degrees of freedom gdf that is applicable to complex modeling procedures the definition is based on the sum of the sensitivity of each fitted value to perturbation in the corresponding observed value the concept is nonasymptotic in nature and does not require analytic knowledge of the modeling procedures the concept of gdf offers a unified framework under which complex and highly irregular modeling procedures can be analyzed in the same way as classical linear models by using this framework many difficult problems can be solved easily for example one can now measure the number of observations used in a variable selection process different modeling procedures such as a tree based regression and a

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

excess optimism how biased is the apparent error of an estimator tuned by sure abstractnearly all estimators in statistical prediction come with an associated tuning parameter in one way or another common practice given data is to choose the tuning parameter value that minimizes a constructed estimate of the prediction error of the estimator we focus on stein s unbiased risk estimator or sure which forms an unbiased estimate of the prediction error by augmenting the observed training error with an estimate of the degrees of freedom of the estimator parameter tuning via sure minimization has been advocated by many authors in a wide variety of problem settings and in general it is natural to ask what is the prediction error of the sure tuned estimator an obvious strategy would be simply use the apparent error estimate as reported by sure that is the value of the sure criterion at its minimum to estimate the prediction error of the sure tuned estimator but this is no longer unbiased in fact we would expect that the minimum of the sure criterion is systematically bias

optimal rates of convergence for covariance matrix estimation covariance matrix plays a central role in multivariate statistical analysis significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices however a minimax theory has yet been developed in this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and frobenius norm it is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation the minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties a key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound the technical analysis requires new ideas that are quite different from those used in the more conventional function sequence estimation problems

can the strengths of aic and bic be shared a conflict between model indentification and regression estimation a traditional approach to statistical inference is to identify the true or best model first with little or no consideration of the specific goal of inference in the model identification stage can the pursuit of the true model also lead to optimal regression estimation in model selection it is well known that bic is consistent in selecting the true model and aic is minimax rate optimal for estimating the regression function a recent promising direction is adaptive model selection in which in contrast to aic and bic the penalty term is data dependent some theoretical and empirical results have been obtained in support of adaptive model selection but it is still not clear if it can really share the strengths of aic and bic model combining or averaging has attracted increasing attention as a means to overcome the model selection uncertainty can bayesian model averaging be optimal for estimating the regression function in a minimax sense we show that the answers to these questions are basically in the negative for any model selection criterion to be consistent it must behave suboptimally for estimating the regression function in terms of minimax rate of covergence and bayesian model averaging cannot be minimax rate optimal for regression estimation

large margin semi supervised learning in classification semi supervised learning occurs when a large amount of unlabeled data is available with only a small number of labeled data in such a situation how to enhance predictability of classification through unlabeled data is the focus in this article we introduce a novel large margin semi supervised learning methodology using grouping information from unlabeled data together with the concept of margins in a form of regularization controlling the interplay between labeled and unlabeled data based on this methodology we develop two specific machines involving support vector machines and learning denoted as ssvm and spsi through difference convex programming in addition we estimate the generalization error using both labeled and unlabeled data for tuning regularizers finally our theoretical and numerical analyses indicate that the proposed methodology achieves the desired objective of delivering high performance in generalization particularly against some strong performers

differential geometric least angle regression a differential geometric approach to sparse generalized linear models sparsity is an essential feature of many contemporary data problems remote sensing various forms of automated screening and other high throughput measurement devices collect a large amount of information typically about few independent statistical subjects or units in certain cases it is reasonable to assume that the underlying process generating the data is itself sparse in the sense that only a few of the measured variables are involved in the process we propose an explicit method of monotonically decreasing sparsity for outcomes that can be modelled by an exponential family in our approach we generalize the equiangular condition in a generalized linear model although the geometry involves the fisher information in a way that is not obvious in the simple regression setting the equiangular condition turns out to be equivalent with an intuitive condition imposed on the rao score test statistics in certain special cases the method can be tweaked to obtain l1 penalized generalized linear model solution paths but the method itself defines sparsity more directly although the computation of the solution paths is not trivial the method compares favourably with other path following algorithms

grouping pursuit through a regularization solution surface extracting grouping structure or identifying homogenous subgroups of predictors in regression is crucial for high dimensional data analysis a low dimensional structure in particular grouping when captured in a regression model enables to enhance predictive performance and to facilitate a model s interpretability grouping pursuit extracts homogenous subgroups of predictors most responsible for outcomes of a response this is the case in gene network analysis where grouping reveals gene functionalities with regard to progression of a disease to address challenges in grouping pursuit we introduce a novel homotopy method for computing an entire solution surface through regularization involving a piecewise linear penalty this nonconvex and overcomplete penalty permits adaptive grouping and nearly unbiased estimation which is treated with a novel concept of grouped subdifferentials and difference convex programming for efficient computation finally the proposed method not only achieves high performanc

simultaneous pursuit of sparseness and rank structures for matrix decomposition in multi response regression pursuit of two different types of structures is essential to battle the curse of dimensionality in this paper we seek a sparsest decomposition representation of a parameter matrix in terms of a sum of sparse and low rank matrices among many overcomplete decompositions on this basis we propose a constrained method subject to two nonconvex constraints respectively for sparseness and low rank properties computationally obtaining an exact global optimizer is rather challenging to overcome the difficulty we use an alternating directions method solving a low rank subproblem and a sparseness subproblem alternatively where we derive an exact solution to the low rank subproblem as well as an exact solution in a special case and an approximated solution generally through a surrogate of the l0 constraint and difference convex programming for the sparse subproblem theoretically we establish convergence rates of a global minimizer in the hellinger distance providing an insight into why pursuit of two different types of decomposed structures is expected to deliver higher estimation accuracy than its counterparts based on either sparseness alone or low rank approximation alone numerical examples are given to illustrate these aspects in addition to an application to facial imagine recognition and multiple time series analysis

a general class of exponential inequalities for martingales and ratios in this paper we introduce a technique for obtaining exponential inequalities with particular emphasis placed on results involving ratios our main applications consist of approximations to the tail probability of the ratio of a martingale over its conditional variance or its quadratic variation for continuous martingales we provide examples that strictly extend several of the classical exponential inequalities for sums of independent random variables and martingales the spirit of this application is that when going from results for sums of independent random variables to martingales one should replace the variance by the conditional variance and the exponential of a function of the variance by the expectation of the exponential of the same function of the conditional variance the decoupling inequalities used to attain our goal are of independent interest they include a new exponential decoupling inequality with constraints and a sharp inequality for the probability of the intersection of a fixed number of dependent sets finally we also present an exponential inequality that does not require any integrability conditions involving the ratio of the sum of conditionally symmetric variables to its sum of squares

optimal model assessment selection and combination central to statistical theory and application is statistical modeling which typically involves choosing a single model or combining a number of models of different sizes and from different sources whereas model selection seeks a single best modeling procedure model combination combines the strength of different modeling procedures in this article we look at several key issues and argue that model assessment is the key to model selection and combination most important we introduce a general technique of optimal model assessment based on data perturbation thus yielding optimal selection in particular model selection and combination from a frequentist perspective we advocate model combination over a selected subset of modeling procedures because it controls bias while reducing variability hence yielding better performance in terms of the accuracy of estimation and prediction to realize the potential of model combination we develop methodologies for determining the optimal tuning parameter such

linear model selection when covariates contain errors prediction precision is arguably the most relevant criterion of a model in practice and is often a sought after property a common difficulty with covariates measured with errors is the impossibility of performing prediction evaluation on the data even if a model is completely given without any unknown parameters we bypass this inherent difficulty by using special properties on moment relations in linear regression models with measurement errors the end product is a model selection procedure that achieves the same optimality properties that are achieved in classical linear regression models without covariate measurement error asymptotically the procedure selects the model with the minimum prediction error in general and selects the smallest correct model if the regression relation is indeed linear our model selection procedure is useful in prediction when future covariates without measurement error become available e g due to improved technology or better management and design of data collection procedures

synchronization and transient dynamics in the chains of electrically coupled fitzhugh nagumo oscillators chains of n fitzhugh nagumo oscillators with a gradient in natural frequencies and strong diffusive coupling are analyzed in this paper we study the system s dynamics in the limit of infinitely large coupling and then treat the case when the coupling is large but finite as a perturbation of the former case in the large coupling limit the 2n dimensional phase space has an unexpected structure there is an n 1 dimensional cylinder foliated by periodic orbits with an integral that is constant on each orbit when the coupling is large but finite this cylinder becomes an analog of an inertial manifold the phase trajectories approach the cylinder on the fast time scale and then slowly drift along it toward a unique limit cycle we analyze these dynamics using geometric theory for singularly perturbed dynamical systems asymptotic expansions of solutions rigorously justified and lyapunov s method

a proximal iteration for deconvolving poisson noisy images using sparse representations we propose an image deconvolution algorithm when the data is contaminated by poisson noise the image to restore is assumed to be sparsely represented in a dictionary of waveforms such as the wavelet or curvelet transforms our key contributions are as follows first we handle the poisson noise properly by using the anscombe variance stabilizing transform leading to a nonlinear degradation equation with additive gaussian noise second the deconvolution problem is formulated as the minimization of a convex functional with a data fidelity term reflecting the noise properties and a nonsmooth sparsity promoting penalty over the image representation coefficients e g lscr1 norm an additional term is also included in the functional to ensure positivity of the restored image third a fast iterative forward backward splitting algorithm is proposed to solve the minimization problem we derive existence and uniqueness conditions of the solution and establish convergence of the iterative algorithm finally a gcv based model selection procedure is proposed to objectively select the regularization parameter experimental results are carried out to show the striking benefits gained from taking into account the poisson statistics of the noise these results also suggest that using sparse domain regularization may be tractable in many deconvolution applications with poisson noise such as astronomy and microscopy

model selection for generalized estimating equations accommodating dropout missingness the generalized estimating equation gee has been a popular tool for marginal regression analysis with longitudinal data and its extension the weighted gee approach can further accommodate data that are missing at random mar model selection methodologies for gee however have not been systematically developed to allow for missing data we propose the missing longitudinal information criterion mlic for selection of the mean model and the mlic for correlation mlicc for selection of the correlation structure in gee when the outcome data are subject to dropout monotone missingness and are mar our simulation results reveal that the mlic and mlicc are effective for variable selection in the mean model and selecting the correlation structure respectively we also demonstrate the remarkable drawbacks of naively treating incomplete data as if they were complete and applying the existing gee model selection method the utility of proposed method is further illustrated by two real applications involving missing longitudinal outcome data

spatial weights matrix selection and model averaging for spatial autoregressive models abstract spatial econometrics relies on the spatial weights matrix to specify the cross sectional dependence however the candidate spatial weights matrices might not be unique this paper proposes a model selection procedure to choose a weights matrix from several candidates by using a mallows type criterion we prove that when the true weights matrix is not in the candidates the procedure is asymptotically optimal in the sense of minimizing the squared loss otherwise the procedure can select the true weights matrix consistently we then propose a model averaging procedure to reduce the squared loss we also provide procedures for the spatial model with heteroscedasticity and endogenous regressors and the model with both spatial lag and spatial error monte carlo experiments show that proposed procedures have satisfactory finite sample performances we apply the model selection and model averaging procedures to study the market integration in china using historical rice prices

generalized thresholding of large covariance matrices we propose a new class of generalized thresholding operators that combine thresholding with shrinkage and study generalized thresholding of the sample covariance matrix in high dimensions generalized thresholding of the covariance matrix has good theoretical properties and carries almost no computational burden we obtain an explicit convergence rate in the operator norm that shows the tradeoff between the sparsity of the true model dimension and the sample size and shows that generalized thresholding is consistent over a large class of models as long as the dimension p and the sample size n satisfy log p n 0 in addition we show that generalized thresholding has the sparsistency property meaning it estimates true zeros as zeros with probability tending to 1 and under an additional mild condition is sign consistent for nonzero elements we show that generalized thresholding covers as special cases hard and soft thresholding smoothly clipped absolute deviation and adaptive lasso and comp

global competition and local cooperation in a network of neural oscillators abstract an architecture of locally excitatory globally inhibitory oscillator networks is proposed and investigated both analytically and by computer simulation the model for each oscillator corresponds to a standard relaxation oscillator with two time scales oscillators are locally coupled by a scheme that resembles excitatory synaptic coupling and each oscillator also inhibits other oscillators through a common inhibitor oscillators are driven to be oscillatory by external stimulation the network exhibits a mechanism of selective gating whereby an oscillator jumping up to its active phase rapidly recruits the oscillators stimulated by the same pattern while preventing the other oscillators from jumping up we show analytically that with the selective gating mechanism the network rapidly achieves both synchronization within blocks of oscillators that are stimulated by connected regions and desynchronization between different blocks computer simulations demonstrate the model s promising ability for segmenting multiple input patterns in real time this model lays a physical foundation for the oscillatory correlation theory of feature binding and may provide an effective computational framework for scene segmentation and figure ground segregation

on p amicable pairs let o n denote euler s totient function i e the number of positive integers 1 we call these numbers o amicable pairs with multiplier k analogously to carmichael s multiply amicable pairs for the function which sums all the divisors of n we have computed all the o amicable pairs with larger member 10 9 the largest pair consisting of two 46 digit numbers

adaptive model selection most model selection procedures use a fixed penalty penalizing an increase in the size of a model these nonadaptive selection procedures perform well only in one type of situation for instance bayesian information criterion bic with a large penalty performs well for small models and poorly for large models and akaike s information criterion aic does just the opposite this article proposes an adaptive model selection procedure that uses a data adaptive complexity penalty based on a concept of generalized degrees of freedom the proposed procedure combining the benefit of a class of nonadaptive procedures approximates the best performance of this class of procedures across a variety of different situations this class includes many well known procedures such as aic bic mallows s cp and risk inflation criterion ric the proposed procedure is applied to wavelet thresholding in nonparametric regression and variable selection in least squares regression simulation results and an asymptotic

nonparametric estimation of large covariance matrices of longitudinal data summary estimation of an unstructured covariance matrix is difficult because of its positivedefiniteness constraint this obstacle is removed by regressing each variable on its predecessors so that estimation of a covariance matrix is shown to be equivalent to that of estimating a sequence of varying coefficient and varying order regression models our framework is similar to the use of increasing order autoregressive models in approximating the covariance matrix or the spectrum of a stationary time series as an illustration we adopt fan zhang s 2000 two step estimation of functional linear models and propose nonparametric estimators of covariance matrices which are guaranteed to be positive definite for parsimony a suitable order for the sequence of auto regression models is found using penalised likelihood criteria like aic and bic some asymptotic results for the local polynomial estimators of components of a covariance matrix are established two longitudinal datasets are analysed to illustrate the methodology a simulation study reveals the advantage of the nonparametric covariance estimator over the sample covariance matrix for large covariance matrices

asymptotic optimality for c_p c_l cross validation and generalized cross validation discrete index set c p c l cross validation and generalized cross validation are useful data driven techniques for selecting a good estimate from a proposed class of linear estimates the asymptotic behaviors of these procedures are studied some easily interpretable conditions are derived to demonstrate the asymptotic optimality it is argued that cross validation and generalized cross validation can be viewed as some special ways of applying c l applications in nearest neighbor nonparametric regression and in model selection are discussed in detail

more comments on c p we discuss the interpretation of c p plots and show how they can be calibrated in several ways we comment on the practice of using the display as a basis for formal selection of a subset regression model and extend the range of application of the device to encompass arbitrary linear estimates of the regression coefficients for example ridge estimates

on the predictive risk in misspecified quantile regression abstract in the present paper we investigate the predictive risk of possibly misspecified quantile regression functions the in sample risk is well known to be an overly optimistic estimate of the predictive risk and we provide two relatively simple asymptotic characterizations of the associated bias also called expected optimism we propose estimates for the expected optimism and the predictive risk and establish their uniform consistency under mild conditions our results hold for models of moderately growing size and allow the quantile function to be incorrectly specified empirical evidence from our estimates is encouraging as it compares favorably with cross validation