asymptotic optimality theory for decentralized sequential hypothesis testing in sensor networks the decentralized sequential hypothesis testing problem is studied in sensor networks where a set of sensors receive independent observations and send summary messages to the fusion center which makes a final decision in the scenario where the sensors have full access to their past observations the first asymptotically bayes sequential test is developed having the same asymptotic performance as the optimal centralized test that has access to all sensor observations next in the scenario where the sensors do not have full access to their past observations a simple but asymptotically bayes sequential tests is developed in which sensor message functions are what we call tandem quantizer where each sensor only uses two different sensor quantizers with at most one switch between these two possibilities moreover a new minimax formulation of optimal stationary sensor quantizers is proposed and is studied in detail in the case of additive gaussian sensor noise finally our results show that in the simplest models feedback from the fusion center does not improve asymptotic performance in the scenario with full local memory however even a one shot one bit feedback can significantly improve performance in the case of limited local memory

least squares model averaging this paper considers the problem of selection of weights for averaging across least squares estimates obtained from a set of models existing model average methods are based on exponential akaike information criterion aic and bayesian information criterion bic weights in distinction this paper proposes selecting the weights by minimizing a mallows criterion the latter an estimate of the average squared error from the model average fit we show that our new mallows model average mma estimator is asymptotically optimal in the sense of achieving the lowest possible squared error in a class of discrete model average estimators in a simulation experiment we show that the mma estimator compares favorably with those based on aic and bic weights the proof of the main result is an application of the work of li 1987

information bounds and quick detection of parameter changes in stochastic systems by using information theoretic bounds and sequential hypothesis testing theory this paper provides a new approach to optimal detection of abrupt changes in stochastic systems this approach not only generalizes previous work in the literature on optimal detection far beyond the relatively simple models treated but also suggests alternative performance criteria which are more tractable and more appropriate for general stochastic systems in addition it leads to detection rules which have manageable computational complexity for on line implementation and yet are nearly optimal under the different performance criteria considered

quantization effect on the log likelihood ratio and its application to decentralized sequential detection it is well known that quantization cannot increase the kullback leibler divergence which can be thought of as the expected value or first moment of the log likelihood ratio in this paper we investigate the quantization effects on the second moment of the log likelihood ratio it is shown via the convex domination technique that quantization may result in an increase in the case of the second moment but the increase is bounded above by 2 e the result is then applied to decentralized sequential detection problems not only to provide simpler sufficient conditions for asymptotic optimality theories in the simplest models but also to shed new light on more complicated models in addition some brief remarks on other higher order moments of the log likelihood ratio are also provided

sequential sensor installation for wiener disorder detection we consider a centralized multisensor online quickest disorder detection problem where the observation from each sensor is a wiener process gaining a constant drift at a common unobservable disorder time the objective is to detect the disorder time as quickly as possible with small probability of false alarms unlike the earlier work on multisensor change detection problems we assume that the observer can apply a sequential sensor installation policy at any time before a disorder alarm is raised the observer can install new sensors to collect additional signals the sensors are statistically identical and there is a fixed installation cost per sensor we propose a bayesian formulation of the problem we identify an optimal policy consisting of a sequential sensor installation strategy and an alarm time which minimize a linear bayes risk of detection delay false alarm and new sensor installations we also provide a numerical algorithm and illustrate it on examples our numerical examples show that

information bounds and quickest change detection in decentralized decision systems the quickest change detection problem is studied in decentralized decision systems where a set of sensors receive independent observations and send summary messages to the fusion center which makes a final decision in the system where the sensors do not have access to their past observations the previously conjectured asymptotic optimality of a procedure with a monotone likelihood ratio quantizer mlrq is proved in the case of additive gaussian sensor noise if the signal to noise ratios snr at some sensors are sufficiently high this procedure can perform as well as the optimal centralized procedure that has access to all the sensor observations even if all snrs are low its detection delay will be at most pi 2 1ap 57 larger than that of the optimal centralized procedure next in the system where the sensors have full access to their past observations the first asymptotically optimal procedure in the literature is developed surprisingly the procedure has the same asymptotic performance as the optimal centralized procedure although it may perform poorly in some practical situations because of slow asymptotic convergence finally it is shown that neither past message information nor the feedback from the fusion center improves the asymptotic performance in the simplest model

jackknife model averaging abstract we consider the problem of obtaining appropriate weights for averaging m approximate misspecified models for improved estimation of an unknown conditional mean in the face of non nested model uncertainty in heteroskedastic error settings we propose a jackknife model averaging jma estimator which selects the weights by minimizing a cross validation criterion this criterion is quadratic in the weights so computation is a simple application of quadratic programming we show that our estimator is asymptotically optimal in the sense of achieving the lowest possible expected squared error monte carlo simulations and an illustrative application show that jma can achieve significant efficiency gains over existing model selection and averaging methods in the presence of heteroskedasticity

optimal stopping times for detecting changes in distributions it is shown that page s stopping time is optimal for the detection of changes in distributions in a well defined sense this work is a generalization of an existing result where it was shown that page s stopping time is optimal asymptotically

asymptotic optimality theory for decentralized sequential multihypothesis testing problems the bayesian formulation of sequentially testing m 3 hypotheses is studied in the context of a decentralized sensor network system in such a system local sensors observe raw observations and send quantized sensor messages to a fusion center which makes a final decision when stopping taking observations asymptotically optimal decentralized sequential tests are developed from a class of two stage tests that allows the sensor network system to make a preliminary decision in the first stage and then optimize each local sensor quantizer accordingly in the second stage it is shown that the optimal local quantizer at each local sensor in the second stage can be defined as a maximin quantizer which turns out to be a randomization of at most m 1 unambiguous likelihood quantizers ulq we first present in detail our results for the system with a single sensor and binary sensor messages and then extend to more general cases involving any finite alphabet sensor messages multiple sensors or composite hypotheses

adaptive regression by mixing adaptation over different procedures is of practical importance different procedures perform well under different conditions in many practical situations it is rather hard to assess which conditions are approximately satisfied so as to identify the best procedure for the data at hand thus automatic adaptation over various scenarios is desirable a practically feasible method named adaptive regression by mixing arm is proposed to convexly combine general candidate regression procedures under mild conditions the resulting estimator is theoretically shown to perform optimally in rates of convergence without knowing which of the original procedures work the best simulations are conducted in several settings including comparing a parametric model with nonparametric alternatives comparing a neural network with a projection pursuit in multidimensional regression and combining bandwidths in kernel regression the results clearly support the theoretical property of arm the arm algorithm assigns we

frequentist model average estimators the traditional use of model selection methods in practice is to proceed as if the final selected model had been chosen in advance without acknowledging the additional uncertainty introduced by model selection this often means underreporting of variability and too optimistic confidence intervals we build a general large sample likelihood apparatus in which limiting distributions and risk properties of estimators post selection as well as of model average estimators are precisely described also explicitly taking modeling bias into account this allows a drastic reduction in complexity as competing model averaging schemes may be developed discussed and compared inside a statistical prototype experiment where only a few crucial quantities matter in particular we offer a frequentist view on bayesian model averaging methods and give a link to generalized ridge estimators our work also leads to new model selection criteria the methods are illustrated with real data applications

semiparametric model averaging prediction for dichotomous response abstract model averaging has attracted abundant attentions in the past decades as it emerges as an impressive forecasting device in econometrics social sciences and medicine so far most developed model averaging methods focus only on either parametric models or nonparametric models with a continuous response in this paper we propose a semiparametric model averaging prediction smap method for a dichotomous response the idea is to approximate the unknown score function by a linear combination of one dimensional marginal score functions the weight parameters involved in the approximation are obtained by first smoothing the nonparametric marginal scores and then applying the parametric model averaging via a maximum likelihood estimation the proposed smap provides greater flexibility than parametric models while being more stable than a fully nonparametric approach theoretical properties are investigated in two practical scenarios i covariates are conditionally independent given the response and ii the conditional independence assumption does not hold in the first scenario we show that smap puts weight one to the true model and hence the model averaging estimators are consistent in the second scenario in which a true model may not exist smap is shown to be asymptotically optimal in the sense that its kullback leibler loss is asymptotically identical to that of the best but infeasible model averaging estimator empirical evidences from simulation studies and a real data analysis are presented to support and illustrate our methods

nonconcave penalized likelihood with a diverging number of parameters a class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by fan and li to simultaneously estimate parameters and select important variables they demonstrated that this class of procedures has an oracle property when the number of parameters is finite however in most model selection problems the number of parameters should be large and grow with the sample size in this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to as the sample size increases under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators furthermore the consistency of the sandwich formula of the covariance matrix is demonstrated nonconcave penalized likelihood ratio statistics are discussed and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions the asymptotic results are augmented by a simulation study and the newly developed methodology is illustrated by an analysis of a court case on the sexual discrimination of salary

model averaging and weight choice in linear mixed effects models this article studies model averaging for linear mixed effects models we establish an unbiased estimator of the squared risk for the model averaging and use the estimator as a criterion for choosing weights the resulting model average estimator is proved to be asymptotically optimal under some regularity conditions simulation experiments show it is superior or comparable to estimators based on the final models selected by the commonly used methods and some existing averaging procedures the proposed procedure is applied to data from an aids clinic trial

bayesian quickest change point detection with sampling right constraints in this paper bayesian quickest change detection problems with sampling right constraints are considered specifically there is a sequence of random vari ables whose probability density function will change at an unknown time the goal is to detect such a change in a way such that a linear combination of the average detection delay and the false alarm probability is minimized two types of sampling right constrains are discussed the first one is a li mited sampling right constraint in which the observer can take at most n observations from this random sequence under this setup we show that the cost function can be written as a set of iterative fun ctions which can be solved by markov optimal stopping theory the optimal stopping rule is shown to be a threshold rule an asymptotic upper bound of the average detection delay is developed as the false alarm probability goes to zero this upper bound indicates that the performance of the limited sampling right problem is close to that of the classic bayesian quickest detection for several scenar ios of practical interest the second constraint discussed in this paper is a stochastic sampling right const raint in which sampling rights are consumed by taking observations and are replenished randomly the observer cannot take observations if there are no sampling rights left we characterize the optimal solution which has a very complex structure for practical applications we propose a low complexity algorithm in which the sampling rule is to take observations as long as the observer has sampling rights left and the detection scheme is a threshold rule we show that this low complexity scheme is first order as ymptotically optimal as the false alarm probability goes to zero

sequential changepoint detection in quality control and dynamical systems after a brief survey of a large variety of sequential detection procedures that are widely scattered in statistical references on quality control and engineering references on fault detection and signal processing we study some open problems concerning these procedures and introduce a unified theory of sequential changepoint detection this theory leads to a class of sequential detection rules which are not too demanding in computational and memory requirements for on line implementation and yet are nearly optimal under several performance criteria

model free feature screening for ultrahigh dimensional discriminant analysis this work is concerned with marginal sure independence feature screening for ultrahigh dimensional discriminant analysis the response variable is categorical in discriminant analysis this enables us to use the conditional distribution function to construct a new index for feature screening in this article we propose a marginal feature screening procedure based on empirical conditional distribution function we establish the sure screening and ranking consistency properties for the proposed procedure without assuming any moment condition on the predictors the proposed procedure enjoys several appealing merits first it is model free in that its implementation does not require specification of a regression model second it is robust to heavy tailed distributions of predictors and the presence of potential outliers third it allows the categorical response having a diverging number of classes in the order of o n with some 0 we assess the finite sample property of the proposed procedure by mont

a model averaging approach for high dimensional regression this article considers high dimensional regression problems in which the number of predictors p exceeds the sample size n we develop a model averaging procedure for high dimensional regression problems unlike most variable selection studies featuring the identification of true predictors our focus here is on the prediction accuracy for the true conditional mean of y given the p predictors our method consists of two steps the first step is to construct a class of regression models each with a smaller number of regressors to avoid the degeneracy of the information matrix the second step is to find suitable model weights for averaging to minimize the prediction error we estimate the model weights using a delete one cross validation procedure departing from the literature of model averaging that requires the weights always sum to one an important improvement we introduce is to remove this constraint we derive some theoretical results to justify our procedure a theorem is proved showing that del

optimal detection of a change in distribution abstract document discusses a stopping rule that is a limit of bayes rules derived then an almost minimax rule is presented

model selection an integral part of inference we argue that model selection uncertainty should be fully incorporated into statistical inference whenever estimation is sensitive to model choice and that choice is made with reference to the data we consider different philosophies for achieving this goal and suggest strategies for data analysis we illustrate our methods through three examples the first is a poisson regression of bird counts in which a choice is to be made between inclusion of one or both of two covariates the second is a line transect data set for which different models yield substantially different estimates of abundance the third is a simulated example in which truth is known

multihypothesis sequential probability ratio tests ii accurate asymptotic expansions for the expected sample size for pt i see ibid vol 45 p 2448 61 1999 we proved in pt i that two specific constructions of multihypothesis sequential tests which we refer to as multihypothesis sequential probability ratio tests msprts are asymptotically optimal as the decision risks or error probabilities go to zero the msprts asymptotically minimize not only the expected sample size but also any positive moment of the stopping time distribution under very general statistical models for the observations in this paper based on nonlinear renewal theory we find accurate asymptotic approximations up to a vanishing term for the expected sample size that take into account the overshoot over the boundaries of decision statistics the approximations are derived for the scenario where the hypotheses are simple the observations are independent and identically distributed i i d according to one of the underlying distributions and the decision risks go to zero simulation results for practical examples show that these approximations are fairly accurate not only for large but also for moderate sample sizes the asymptotic results given here complete the analysis initiated by baum and veeravalli 1994 where first order asymptotics were obtained for the expected sample size under a specific restriction on the kullback leibler distances between the hypotheses

efficient scalable schemes for monitoring a large number of data streams the sequential changepoint detection problem is studied in the context of global online monitoring of a large number of independent data streams we are interested in detecting an occurring event as soon as possible but we do not know when the event will occur nor do we know which subset of data streams will be affected by the event a family of scalable schemes is proposed based on the sum of the local cumulative sum cusum statistics from each individual data stream and is shown to asymptotically minimize the detection delays for each and every possible combination of affected data streams subject to the global false alarm constraint the usefulness and limitations of our asymptotic optimality results are illustrated by numerical simulations and heuristic arguments the appendices contain a probabilistic result on the first epoch to simultaneous record values for multiple independent random walks

optimal weight choice for frequentist model average estimators there has been increasing interest recently in model averaging within the frequentist paradigm the main benefit of model averaging over model selection is that it incorporates rather than ignores the uncertainty inherent in the model selection process one of the most important yet challenging aspects of model averaging is how to optimally combine estimates from different models in this work we suggest a procedure of weight choice for frequentist model average estimators that exhibits optimality properties with respect to the estimator s mean squared error mse as a basis for demonstrating our idea we consider averaging over a sequence of linear regression models building on this base we develop a model weighting mechanism that involves minimizing the trace of an unbiased estimator of the model average estimator s mse we further obtain results that reflect the finite sample as well as asymptotic optimality of the proposed mechanism a monte carlo study based on simulated and real data evaluates

extremal properties of likelihood ratio quantizers m hypotheses and a random variable y with a different probability distribution under each hypothesis are considered a quantizer is applied to form a quantized random variable gamma y the extreme points of the set of possible probability distributions of gamma y as gamma ranges over all quantizers is characterized optimality properties of likelihood ratio quantizers are established for a very broad class of quantization problems including problems involving the maximization of an ali silvey 1966 distance measure and the neyman pearson variant of the decentralized detection problem

information theory and mixing least squares regressions for gaussian regression we develop and analyze methods for combining estimators from various models for squared error loss an unbiased estimator of the risk of the mixture of general estimators is developed special attention is given to the case that the component estimators are least squares projections into arbitrary linear subspaces such as those spanned by subsets of explanatory variables in a given design we relate the unbiased estimate of the risk of the mixture estimator to estimates of the risks achieved by the components this results in simple and accurate bounds on the risk and its estimate in the form of sharp and exact oracle inequalities that is without advance knowledge of which model is best the resulting performance is comparable to or perhaps even superior to what is achieved by the best of the individual models furthermore in the case that the unknown parameter has a sparse representation our mixture estimator adapts to the underlying sparsity simulations show that the performance of these mixture estimators is better than that of a related model selection estimator which picks a model with the highest weight also the connection between our mixtures with bayes procedures is discussed

an asymptotic theory for model selection inference in general semiparametric problems summary hjort claeskens 2003 developed an asymptotic theory for model selection model averaging and subsequent inference using likelihood methods in parametric models along with associated confidence statements in this article we consider a semiparametric version of this problem wherein the likelihood depends on parameters and an unknown function and model selection averaging is to be applied to the parametric parts of the model we show that all the results of hjort claeskens hold in the semiparametric context if the fisher information matrix for parametric models is replaced by the semiparametric information bound for semiparametric models and if maximum likelihood estimators for parametric models are replaced by semiparametric efficient profile estimators our methods of proof employ le cam s contiguity lemmas leading to transparent results the results also describe the behaviour of semiparametric model estimators when the parametric component is misspecified and also have implications for pointwise consistent model selectors somekeywords akaike information criterion bayes information criterion efficient semiparametric estimation frequentist model averaging model averaging model selection profile likelihood semiparametric model

using the generalized likelihood ratio statistic for sequential detection of a change point we study sequential detection of a change point using the generalized likelihood ratio statistic for the special case of detecting a change in a normal mean with known variance we give approximations to the average run lengths and compare our procedure to standard cusum tests and combined cusum shewhart tests several examples indicating extensions to problems involving multiple parameters are discussed

a computational study with a new algorithm for the three machine permutation flow shop problem with release times abstract we consider the three machine permutation flow shop scheduling problem with release times where the objective is to minimize the maximum completion time a special solvable case is found for the f 2 r j c max problem which sharpens the boundary between easy and hard cases and can be used to compute a tight lower bound for our problem two dominance rules are generalized and applied to generating initial schedules directing the search strategy and decomposing the problem into smaller ones the branch and bound algorithm proposed here combines an adaptive branching rule with a fuzzy search strategy to narrow the search tree and lead the search to an optimal solution as early as possible our extensive numerical experiments have led to a classification of easy vs hard problems dependent only on the relative size of the release times the algorithm has quickly solved approximately 90 of the hardest test problem instances with up to 200 jobs and 100 of the large problems classified as easy

the dimensionality reduction principle for generalized additive models soit x y une paire de variables aleatoires telles que x x 1 x j s etende sur c 0 1 j la distribution conditionnelle de y etant donne x x est supposee appartenir a une famille exponentielle appropriee ayant pour parametre r f x denote la dependance de vis a vis de x et f l approximation additive de f ayant la log vraisemblance esperee maximum possible

least squares model averaging by mallows criterion abstract this paper is in response to a recent paper by hansen 2007 who proposed an optimal model average estimator with weights selected by minimizing a mallows criterion the main contribution of hansen s paper is a demonstration that the mallows criterion is asymptotically equivalent to the squared error so the model average estimator that minimizes the mallows criterion also minimizes the squared error in large samples we are concerned with two assumptions that accompany hansen s approach the first is the assumption that the approximating models are strictly nested in a way that depends on the ordering of regressors often there is no clear basis for the ordering and the approach does not permit non nested models which are more realistic from a practical viewpoint second for the optimality result to hold the model weights are required to lie within a special discrete set in fact hansen noted both difficulties and called for extensions of the proof techniques we provide an alternative proof which shows that the result on the optimality of the mallows criterion in fact holds for continuous model weights and under a non nested set up that allows any linear combination of regressors in the approximating models that make up the model average estimator these results provide a stronger theoretical basis for the use of the mallows criterion in model averaging by strengthening existing findings

the kolmogorov filter for variable screening in high dimensional binary classification variable screening techniques have been proposed to mitigate the impact of high dimensionality in classification problems including t test marginal screening fan fan 2008 and maximum marginal likelihood screening fan song 2010 however these methods rely on strong modelling assumptions that are easily violated in real applications to circumvent the parametric modelling assumptions we propose a new variable screening technique for binary classification based on the kolmogorov smirnov statistic we prove that this so called kolmogorov filter enjoys the sure screening property under much weakened model assumptions we supplement our theoretical study by a simulation study copyright 2013 oxford university press