ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

convex aggregative modelling of infinite memory nonlinear systems abstractthe convex aggregation technique is applied for modelling general class of nonlinear systems with unknown structure and infinite memory the finite sample size properties of the algorithm are formally established and compared to the standard least squares counterpart of the method the proposed algorithm demonstrates its advantages when the a priori knowledge and the measurement data are both scarce that is when the information about the actual system structure is unknown or uncertain and the measurement set is small and disturbed by a noise numerical experiments illustrate application and practical benefits of the method for various nonlinear systems

adapting to unknown sparsity by controlling the false discovery rate we attempt to recover an n dimensional vector observed in white noise where n is large and the vector is known to be sparse but the degree of sparsity is unknown we consider three different ways of defining sparsity of a vector using the fraction of nonzero terms imposing power law decay bounds on the ordered entries and controlling the lp norm for p small we obtain a procedure which is asymptotically minimax for l r loss simultaneously throughout a range of such sparsity classes the optimal procedure is a data adaptive thresholding scheme driven by control of the false discovery rate fdr fdr control is a relatively recent innovation in simultaneous testing ensuring that at most a certain fraction of the rejected null hypotheses will correspond to false rejections in our treatment the fdr control parameter qn also plays a determining role in asymptotic minimaxity if q lim qn 0 1 2 and also qn log n we get sharp asymptotic minimaxity simultaneously over a wide range of sparse parameter spaces and loss functions on the other hand q lim qn 1 2 1 forces the risk to exceed the minimax risk by a factor growing with q to our knowledge this relation between ideas in simultaneous inference and asymptotic decision theory is new our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors these new rules impose complexity penalization of the form 2 log potential model size actual model size we exhibit a close connection with fdr controlling procedures under stringent control of the false discovery rate

fisher information and stochastic complexity by taking into account the fisher information and removing an inherent redundancy in earlier two part codes a sharper code length as the stochastic complexity and the associated universal process are derived for a class of parametric processes the main condition required is that the maximum likelihood estimates satisfy the central limit theorem the same code length is also obtained from the so called maximum likelihood code

inference after model selection typical modeling strategies involve model selection which has a significant effect on inference of estimated parameters common practice is to use a selected model ignoring uncertainty introduced by the process of model selection this could yield overoptimistic inferences resulting in false discovery in this article we develop a general methodology via optimal approximation for estimating the mean and variance of complex statistics that involve the process of model selection this allows us to make approximately unbiased inferences taking into account the selection process we examine the operating characteristics of the proposed methodology via asymptotic analyses and simulations these results show that the proposed methodology yields correct inferences and outperforms common alternatives

bayesian measures of model complexity and fit summary we consider the problem of comparing complex hierarchical models in which the number of parameters is not clearly defined using an information theoretic argument we derive a measure pd for the effective number of parameters in a model as the difference between the posterior mean of the deviance and the deviance at the posterior means of the parameters of interest in general pd approximately corresponds to the trace of the product of fisher s information and the posterior covariance which in normal models is the trace of the hat matrix projecting observations onto fitted values its properties in exponential families are explored the posterior mean deviance is suggested as a bayesian measure of fit or adequacy and the contributions of individual observations to the fit and complexity can give rise to a diagnostic plot of deviance residuals against leverages adding pd to the posterior mean deviance gives a deviance information criterion for comparing models which is related to other information criteria and has an approximate decision theoretic justification the procedure is illustrated in some examples and comparisons are drawn with alternative bayesian and classical proposals throughout it is emphasized that the quantities required are trivial to compute in a markov chain monte carlo analysis

adaptive regression by mixing adaptation over different procedures is of practical importance different procedures perform well under different conditions in many practical situations it is rather hard to assess which conditions are approximately satisfied so as to identify the best procedure for the data at hand thus automatic adaptation over various scenarios is desirable a practically feasible method named adaptive regression by mixing arm is proposed to convexly combine general candidate regression procedures under mild conditions the resulting estimator is theoretically shown to perform optimally in rates of convergence without knowing which of the original procedures work the best simulations are conducted in several settings including comparing a parametric model with nonparametric alternatives comparing a neural network with a projection pursuit in multidimensional regression and combining bandwidths in kernel regression the results clearly support the theoretical property of arm the arm algorithm assigns we

the estimation of prediction error having constructed a data based estimation rule perhaps a logistic regression or a classification tree the statistician would like to know its performance as a predictor of future cases there are two main theories concerning prediction error 1 penalty methods such as cp akaike s information criterion and stein s unbiased risk estimate that depend on the covariance between data points and their corresponding predictions and 2 cross validation and related nonparametric bootstrap techniques this article concerns the connection between the two theories a rao blackwell type of relation is derived in which nonparametric methods such as cross validation are seen to be randomized versions of their covariance penalty counterparts the model based penalty methods offer substantially better accuracy assuming that the model is believable

frequentist model average estimators the traditional use of model selection methods in practice is to proceed as if the final selected model had been chosen in advance without acknowledging the additional uncertainty introduced by model selection this often means underreporting of variability and too optimistic confidence intervals we build a general large sample likelihood apparatus in which limiting distributions and risk properties of estimators post selection as well as of model average estimators are precisely described also explicitly taking modeling bias into account this allows a drastic reduction in complexity as competing model averaging schemes may be developed discussed and compared inside a statistical prototype experiment where only a few crucial quantities matter in particular we offer a frequentist view on bayesian model averaging methods and give a link to generalized ridge estimators our work also leads to new model selection criteria the methods are illustrated with real data applications

the little bootstrap and other methods for dimensionality selection in regression x fixed prediction error abstract when a regression problem contains many predictor variables it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor variables usually a regression equation based on a few variables will be more accurate and certainly simpler there are various methods for picking good subsets of variables and programs that do such procedures are part of every widely used statistical package the most common methods are based on stepwise addition or deletion of variables and on best subsets the latter refers to a search method that given the number of variables to be in the equation say five locates that regression equation based on five variables that has the lowest residual sum of squares among all five variable equations all of these procedures generate a sequence of regression equations the first based on one variable the next based on two variables and so on each member of this sequence is called a submodel and the number of variables in the e

discussion of from fixed x to random x regression bias variance decompositions covariance penalties and prediction error estimation rosset and tibshirani are to be congratulated for addressing an important yet under studied topic of modern statistics so called random x prediction in regression analysis in statistics random x

a mass conservative characteristic fe scheme for optimal control problems governed by convection diffusion equations abstract in this paper we develop a mass conservative characteristic finite element scheme for optimal control problems governed by linear singularly perturbed convection diffusion equations we discuss the case that the velocity fields are non divergence free the space discretization of the state variable is done by piecewise linear continuous functions whereas the control variable is approximated by piecewise constant functions due to the limitation of the regularity the scheme preserves the mass balance for the original state equation we derive a priori error estimates for both the control and state approximations some numerical examples are presented to show the efficiency of the proposed scheme

optimal geostatistical model selection in many fields of science predicting variables of interest over a study region based on noisy data observed at some locations is an important problem two popular methods for the problem are kriging and smoothing splines the former assumes that the underlying process is stochastic whereas the latter assumes it is purely deterministic kriging performs better than smoothing splines in some situations but is outperformed by smoothing splines in others however little is known regarding selecting between kriging and smoothing splines in addition how to perform variable selection in a geostatistical model has not been well studied in this article we propose a general methodology for selecting among arbitrary spatial prediction methods based on approximately unbiased estimation of mean squared prediction errors using a data perturbation technique the proposed method accounts for estimation uncertainty in both kriging and smoothing spline predictors and is shown to be optimal in terms of two mean sq

calibrating the degrees of freedom for automatic data smoothing and effective curve checking curve fitting and curve checking based on the local polynomial regression technique are commonly used data analytic methods in statistics this article examines in nonparametric settings both the asymptotic expressions and empirical formulas for degrees of freedom df a notion introduced by hastie and tibshirani of linear smoothers the asymptotic results give useful insights into the nonparametric modeling complexity meanwhile by substituting the exact dfs by the empirical formula an empirical version of the generalized cross validation egcv is obtained an automatic bandwidth selection method based on minimizing egcv is proposed for conducting local smoothing this procedure preserves full benefits of the ordinary and generalized cross validation but offers a substantial reduction in computational burden furthermore the egcv minimizing bandwidth can be extended in a very simple manner to fit multivariate models such as the varying coefficient models applications of calibrating dfs to impo

regression and time series model selection in small samples summary a bias correction to the akaike information criterion aic is derived for regression and autoregressive time series models the correction is of particular use when the sample size is small or when the number of fitted parameters is a moderate to large fraction of the sample size the corrected method called aicc is asymptotically efficient if the true model is infinite dimensional furthermore when the true model is of finite dimension aicc is found to provide better model order choices than any other asymptotically efficient method applications to nonstationary autoregressive and mixed autoregressive moving average time series models are also discussed

turbo quantization of oversampled signals we present a new iterative quantization procedure for oversampled frames with guaranteed convergence and stability the procedure uses similar concepts to that of turbo decoding in the context of error control coding the quantization process is modeled as a quadratic integer programming problem a trellis structure is introduced to solve the problem where the quantized coefficients represent a path through the trellis diagram the quantization metric is iteratively improved through turbo iterations by exchanging quantization extrinsic information the proposed procedure is shown to provide significant improvement over prior art in quantization and integer programming

a reference bayesian test for nested hypotheses and its relationship to the schwarz criterion abstract to compute a bayes factor for testing h 0 0 in the presence of a nuisance parameter priors under the null and alternative hypotheses must be chosen as in bayesian estimation an important problem has been to define automatic or reference methods for determining priors based only on the structure of the model in this article we apply the heuristic device of taking the amount of information in the prior on equal to the amount of information in a single observation then after transforming to be null orthogonal to we take the marginal priors on to be equal under the null and alternative hypotheses doing so and taking the prior on to be normal we find that the log of the bayes factor may be approximated by the schwarz criterion with an error of order o p n rather than the usual error of order o p 1 this result suggests the schwarz criterion should provide sensible approximate solutions to bayesian testing problems at least when the hypotheses are nested when

the covariance inflation criterion for adaptive model selection we propose a new criterion for model selection in prediction problems the covariance inflation criterion adjusts the training error by the average covariance of the predictions and responses when the prediction rule is applied to permuted versions of the data set this criterion can be applied to general prediction problems e g regression or classification and to general prediction rules e g stepwise regression tree based models and neural nets as a by product we obtain a measure of the effective number of parameters used by an adaptive procedure we relate the covariance inflation criterion to other model selection procedures and illustrate its use in some regression and classification problems we also revisit the conditional bootstrap approach to model selection

on the degrees of freedom of the lasso we study the effective degrees of freedom of the lasso in the framework of stein s unbiased risk estimation sure we show that the number of nonzero coefficients is an unbiased estimate forthe degrees of freedom of the lasso a conclusion that requires no special assumption on the predictors in addition the unbiased estimator is shown to be asymptotically consistent with these results on hand various model selection criteria cp aic and bic are available which along with the lars algorithm provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least squares fit

from stein s unbiased risk estimates to the method of generalized cross validation this paper concerns the method of generalized cross validation gcv a promising way of choosing between linear estimates based on stein estimates and the associated unbiased risk estimates stein 1981 a new approach to gcv is developed many consistency results are obtained for the cross validated steinized estimates in the contexts of nearest neighbor nonparametric regression model selection ridge regression and smoothing splines moreover the associated stein s unbiased risk estimate is shown to be uniformly consistent in assessing the true loss not the risk consistency properties are examined as well when the sampling error is unknown finally we propose a variant of gcv to handle the case that the dimension of the raw data is known to be greater than that of their expected values

calibration and empirical bayes variable selection for the problem of variable selection for the normal linear model selection criteria such as aic c p bic and ric have fixed dimensionality penalties such criteria are shown to correspond to selection of maximum posterior models under implicit hyperparameter choices for a particular hierarchical bayes formulation based on this calibration we propose empirical bayes selection criteria that use hyperparameter estimates instead of fixed choices for obtaining these estimates both marginal and conditional maximum likelihood methods are considered as opposed to traditional fixed penalty criteria these empirical bayes criteria have dimensionality penalties that depend on the data their performance is seen to approximate adaptively the performance of the best fixed penalty criterion across a variety of orthogonal and nonorthogonal set ups including wavelet regression empirical bayes shrinkage estimators of the selected coefficients are also proposed

some comments on cp i study the typical configuration of a cp plot when the number of variables in the regression problem is large and there are many weak effects i show that a particular configuration that is very commonly seen can arise in a simple way i give a formula by means of which the risk incurred by the minimum cp rule can be estimated

how biased is the apparent error rate of a prediction rule abstract a regression model is fitted to an observed set of data how accurate is the model for predicting future observations the apparent error rate tends to underestimate the true error rate because the data have been used twice both to fit the model and to check its accuracy we provide simple estimates for the downward bias of the apparent error rate the theory applies to general exponential family linear models and general measures of prediction error special attention is given to the case of logistic regression on binary data with error rates measured by the proportion of misclassified cases several connected ideas are compared mallows s cp cross validation generalized cross validation the bootstrap and akaike s information criterion

the risk inflation criterion for multiple regression a new criterion is proposed for the evaluation of variable selection procedures in multiple regression this criterion which we call the risk inflation is based on an adjustment to the risk essentially the risk inflation is the maximum increase in risk due to selecting rather than knowing the correct predictors a new variable selection procedure is obtained which in the case of orthogonal predictors substantially improves on aic c p and bic and is close to optimal in contrast to aic c p and bic which use dimensionality penalties of 2 2 and log n respectively this new procedure uses a penalty 2 log p where p is the number of available predictors for the case of nonorthogonal predictors bounds for the optimal penalty are obtained

on measuring and correcting the effects of data mining and model selection abstract in the theory of linear models the concept of degrees of freedom plays an important role this concept is often used for measurement of model complexity for obtaining an unbiased estimate of the error variance and for comparison of different models i have developed a concept of generalized degrees of freedom gdf that is applicable to complex modeling procedures the definition is based on the sum of the sensitivity of each fitted value to perturbation in the corresponding observed value the concept is nonasymptotic in nature and does not require analytic knowledge of the modeling procedures the concept of gdf offers a unified framework under which complex and highly irregular modeling procedures can be analyzed in the same way as classical linear models by using this framework many difficult problems can be solved easily for example one can now measure the number of observations used in a variable selection process different modeling procedures such as a tree based regression and a

discussion local rademacher complexities and oracle inequalities in risk minimization in this magnificent paper professor koltchinskii offers general and powerful performance bounds for empirical risk minimization a fundamental principle of statistical learning theory since the elegant pioneering work of vapnik and chervonenkis in the early 1970s various such bounds have been known that relate the performance of empirical risk minimizers to combinatorial and geometrical features of the class over which the minimization is performed this area of research has been a rich source of motivation and a major field of applications of empirical process theory the appearance of advanced concentration inequalities in the 1990s primarily thanks to talagrand s influential work provoked major advances in both empirical process theory and statistical learning theory and led to a much deeper understanding of some of the basic phenomena in the discussed paper professor koltchinskii develops a powerful new methodology iterative localization which with the help of concentration inequalities is able to explain most of the recent results and go significantly beyond them in many cases the main motivation behind professor koltchinskii s paper is based on classical problems of statistical learning theory such as binary classification and regression in which given a sample xi y i i 1 n of independent and identically distributed pairs of random variables where the xi take their values in some feature space x and the yi are say real valued the goal is to find a function f x r whose risk defined in terms of the expected value of an appropriately chosen loss function is as small as possible in the remaining part of this discussion we point out how the performance bounds of professor koltchinskii s paper can be used to study a seemingly different model motivated by nonparametric ranking problems which has received increasing attention both in the statistical and machine learning literature indeed in several applications such as the search engine problem or credit risk screening the goal is to learn how to rank or to score observations rather than just classify them in this case performance measures involve pairs of observations as can be seen for instance with the auc area under an roc curve criterion in this

model selection an integral part of inference we argue that model selection uncertainty should be fully incorporated into statistical inference whenever estimation is sensitive to model choice and that choice is made with reference to the data we consider different philosophies for achieving this goal and suggest strategies for data analysis we illustrate our methods through three examples the first is a poisson regression of bird counts in which a choice is to be made between inclusion of one or both of two covariates the second is a line transect data set for which different models yield substantially different estimates of abundance the third is a simulated example in which truth is known

large margin semi supervised learning in classification semi supervised learning occurs when a large amount of unlabeled data is available with only a small number of labeled data in such a situation how to enhance predictability of classification through unlabeled data is the focus in this article we introduce a novel large margin semi supervised learning methodology using grouping information from unlabeled data together with the concept of margins in a form of regularization controlling the interplay between labeled and unlabeled data based on this methodology we develop two specific machines involving support vector machines and learning denoted as ssvm and spsi through difference convex programming in addition we estimate the generalization error using both labeled and unlabeled data for tuning regularizers finally our theoretical and numerical analyses indicate that the proposed methodology achieves the desired objective of delivering high performance in generalization particularly against some strong performers

nonparametric regression with basis selection from multiple libraries new nonparametric regression procedures called bsml basis selection from multiple libraries are proposed in this article for estimating a complex function by a linear combination of basis functions adaptively selected from multiple libraries different classes of basis functions are chosen to model various features of the function for example truncated constants can model change points in the function while polynomial spline representers may be used to model smooth components the generalized cross validation gcv and covariance inflation criteria are used to balance goodness of fit and model complexity where the model complexity is estimated adaptively by either the generalized degrees of freedom or covariance penalty the cross validation cv method is also considered for model selection spatially adaptive regression and model selection in multivariate nonparametric regression will be used to illustrate the flexibility and efficiency of the bsml procedures extensive simulations show that the bs

minimax multiple shrinkage estimation on presente des nouveaux estimateurs qui incorporent l information partielle a priori en utilisant les donnees comme emulation du comportement et du risque des estimateurs consideres les plus efficaces

sigma delta spl sigma spl delta quantization and finite frames the k level sigma delta spl sigma spl delta scheme with step size spl delta is introduced as a technique for quantizing finite frame expansions for spl ropf sup d error estimates for various quantized frame expansions are derived and in particular it is shown that spl sigma spl delta quantization of a unit norm finite frame expansion in spl ropf sup d achieves approximation error where n is the frame size and the frame variation spl sigma f p is a quantity which reflects the dependence of the spl sigma spl delta scheme on the frame here spl par spl middot spl par is the d dimensional euclidean 2 norm lower bounds and refined upper bounds are derived for certain specific cases as a direct consequence of these error bounds one is able to bound the mean squared error mse by an order of 1 n sup 2 when dealing with sufficiently redundant frame expansions this represents a significant improvement over classical pulse code modulation pcm quantization which only has mse of order 1 n under certain nonrigorous statistical assumptions spl sigma spl delta also achieves the optimal mse order for pcm with consistent reconstruction

grouping pursuit through a regularization solution surface extracting grouping structure or identifying homogenous subgroups of predictors in regression is crucial for high dimensional data analysis a low dimensional structure in particular grouping when captured in a regression model enables to enhance predictive performance and to facilitate a model s interpretability grouping pursuit extracts homogenous subgroups of predictors most responsible for outcomes of a response this is the case in gene network analysis where grouping reveals gene functionalities with regard to progression of a disease to address challenges in grouping pursuit we introduce a novel homotopy method for computing an entire solution surface through regularization involving a piecewise linear penalty this nonconvex and overcomplete penalty permits adaptive grouping and nearly unbiased estimation which is treated with a novel concept of grouped subdifferentials and difference convex programming for efficient computation finally the proposed method not only achieves high performanc

trellis coded quantization of memoryless and gauss markov sources trellis coded quantization tcq is developed and applied to the encoding of memoryless and gauss markov sources the theoretical justification for the approach is alphabet constrained rate distortion theory which is a dual to the channel capacity argument that motivates trellis coded modulation tcm the authors adopt the notions of signal set expansion set partitioning and branch labeling of tcm but modify the techniques to account for the source distribution to design tcq coders of low complexity with excellent mean squared error mse performance for a memoryless uniform source tcq provides an mse within 0 21 db of the distortion rate bound at all positive integral rates the performance is superior to that promised by the coefficient of quantization for all of the best lattices known in dimensions 24 or less for a memoryless gaussian source the tcq performance at rates of 0 5 1 and 2 b sample is superior to all previous results the authors found in the literature the encoding complexity of tcq is very modest tcq is incorporated into a predictive coding structure for the encoding of gauss markov sources simulation results for first second and third order gauss markov sources are presented

parametric or nonparametric a parametricness index for model selection in model selection literature two classes of criteria perform well asymptotically in different situations bayesian information criterion bic as a representative is consistent in selection when the true model is finite dimensional parametric scenario akaike s information criterion aic performs well in an asymptotic efficiency when the true model is infinite dimensional nonparametric scenario but there is little work that addresses if it is possible and how to detect the situation that a specific model selection problem is in in this work we differentiate the two scenarios theoretically under some conditions we develop a measure parametricness index pi to assess whether a model selected by a potentially consistent procedure can be practically treated as the true model which also hints on aic or bic is better suited for the data for the goal of estimating the regression function a consequence is that by switching between aic and bic based on the pi the resulting regression estimator is simultaneously asymptotically efficient for both parametric and nonparametric scenarios in addition we systematically investigate the behaviors of pi in simulation and real data and show its usefulness

optimal weight choice for frequentist model average estimators there has been increasing interest recently in model averaging within the frequentist paradigm the main benefit of model averaging over model selection is that it incorporates rather than ignores the uncertainty inherent in the model selection process one of the most important yet challenging aspects of model averaging is how to optimally combine estimates from different models in this work we suggest a procedure of weight choice for frequentist model average estimators that exhibits optimality properties with respect to the estimator s mean squared error mse as a basis for demonstrating our idea we consider averaging over a sequence of linear regression models building on this base we develop a model weighting mechanism that involves minimizing the trace of an unbiased estimator of the model average estimator s mse we further obtain results that reflect the finite sample as well as asymptotic optimality of the proposed mechanism a monte carlo study based on simulated and real data evaluates

optimal model assessment selection and combination central to statistical theory and application is statistical modeling which typically involves choosing a single model or combining a number of models of different sizes and from different sources whereas model selection seeks a single best modeling procedure model combination combines the strength of different modeling procedures in this article we look at several key issues and argue that model assessment is the key to model selection and combination most important we introduce a general technique of optimal model assessment based on data perturbation thus yielding optimal selection in particular model selection and combination from a frequentist perspective we advocate model combination over a selected subset of modeling procedures because it controls bias while reducing variability hence yielding better performance in terms of the accuracy of estimation and prediction to realize the potential of model combination we develop methodologies for determining the optimal tuning parameter such

performance of sigma delta quantizations in finite frames in this paper we consider sigma delta sd quantization of geometrically uniform gu finite frames in the first part we prove that under some conditions the variant i and ii permutation modulation pm codes first introduced by slepian 1968 1965 can belong to the class of gu frames then we focus essentially on a subclass of a gu frame namely cyclic geometrically uniform cgu frame family of frames containing finite harmonic frames both in bbc m and bbr m for first and second order sd quantizers we establish that the reconstruction minimum squares error mse behaves as 1 r 2 where r denotes the frame redundancy this result is shown to be true both under the deterministic quantization model used in benedetto 2004 benedetto 2006 and yilmaz 2001 as well as under the widely used additive white quantization noise assumption for the widely used l th order noise shaping filter g z 1 z 1 l we show that the mse behaves as 1 r 2 irrespectively of the filter order l more importantly we prove also that in the case of tight and normalized cgu frame when the frame length is too large compared to the filter order and under some conditions on the quantizer the reconstruction mse can decay as fast as o 1 r 2 l 1 finally it is shown that degrees of freedom in cgu frames when compared to harmonic frames result in a smaller mse albeit mse prop 1 r 2 in both cases

efficient quantization for overcomplete expansions in spl ropf sup n we study construction of structured regular quantizers for overcomplete expansions in spl ropf sup n our goal is to design structured quantizers which allow simple reconstruction algorithms with low complexity and which have good performance in terms of accuracy most related work to date in quantized redundant expansions has assumed that the same uniform scalar quantizer was used on all the expansion coefficients several approaches have been proposed to improve the reconstruction accuracy with some of these methods having significant complexity instead we consider the joint design of the overcomplete expansion and the scalar quantizers allowing different step sizes in such a way as to produce an equivalent vector quantizer evq with periodic structure the construction of a periodic quantizer is based on lattices in spl ropf sup n and the concept of geometrically scaled similar sublattices the periodicity makes it possible to achieve good accuracy using simple reconstruction algorithms e g linear reconstruction or a small lookup table

the focused information criterion a variety of model selection criteria have been developed of general and specific types most of these aim at selecting a single model with good overall properties for example formulated via average prediction quality or shortest estimated overall distance to the true model the akaike the bayesian and the deviance information criteria along with many suitable variations are examples of such methods these methods are not concerned however with the actual use of the selected model which varies with context and application the present article takes the view that the model selector should instead focus on the parameter singled out for interest in particular a model that gives good precision for one estimand may be worse when used for inference for another estimand we develop a method that for a given focus parameter estimates the precision of any submodel based estimator the framework is that of large sample likelihood inference using an unbiased estimate of limiting risk we propose a foc

adaptive model selection most model selection procedures use a fixed penalty penalizing an increase in the size of a model these nonadaptive selection procedures perform well only in one type of situation for instance bayesian information criterion bic with a large penalty performs well for small models and poorly for large models and akaike s information criterion aic does just the opposite this article proposes an adaptive model selection procedure that uses a data adaptive complexity penalty based on a concept of generalized degrees of freedom the proposed procedure combining the benefit of a class of nonadaptive procedures approximates the best performance of this class of procedures across a variety of different situations this class includes many well known procedures such as aic bic mallows s cp and risk inflation criterion ric the proposed procedure is applied to wavelet thresholding in nonparametric regression and variable selection in least squares regression simulation results and an asymptotic

more comments on c p we discuss the interpretation of c p plots and show how they can be calibrated in several ways we comment on the practice of using the display as a basis for formal selection of a subset regression model and extend the range of application of the device to encompass arbitrary linear estimates of the regression coefficients for example ridge estimates