random rates in anisotropic regression in the context of minimax theory we propose a new kind of risk normalized by a random variable measurable with respect to the data we present a notion of optimality and a method to construct optimal procedures accordingly we apply this general setup to the problem of selecting significant variables in gaussian white noise in particular we show that our method essentially improves the accuracy of estimation in the sense of giving explicit improved confidence sets in l2 norm links to adaptive estimation are discussed 1 introduction searching for significant variables is certainly one of the oldest and most popular problems in statistics one of the simplest models where the issue of selecting significant variables was first stated mathematically is linear regression a vast literature has been devoted to this topic since and different approaches have been proposed over the last forty years both for estimation and for hypothesis testing among many authors we refer to akaike 1 breiman and freedman 3 chernoff 5 csiszar and korner 6 dychakov 10 patel 42 renyi 46 freidlina 13 meshalkin 35 malyutov and tsitovich 34 schwarz 47 and stone 48 in classical parametric regression if we consider a linear model we first have to measure the possible gain of searching for a limited number of significant variables if the model comes from a specific field of application then only an adequate description together with its solution is relevant however from a mathematical point of view a theory of selecting significant variables does not lead at least asymptotically to a substantial improvement of the accuracy of estimation in a regular parametric model the classical n rate of convergence is not affected by the number of significant variables however even in this setup let us emphasize that asymptotically has to be understood as up to a constant and that the correct choice of significant variables may possibly improve this constant if instead of a linear model we consider a nonparametric regression model the search for significant variables becomes crucial for estimating the regression function the rate of convergence explicitly depends on the set of significant variables let us develop this statement with the following example of multivariate regression suppose we observe z n xi y i i 1 n in the model

density estimation by wavelet thresholding density estimation is a commonly used test case for nonparametric estimation methods we explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients minimax rates of convergence are studied over a large range of besov function classes b pq and for a range of global l p error measures 1 p p some form of nonlinearity is essential since the minimax linear estimators are suboptimal by polynomial powers of n a second approach using an approximation of a gaussian white noise model in a mallows metric is used to attain exactly optimal rates of convergence for quadratic error p 2

on coverage and local radial rates of credible sets in the mildly ill posed inverse signal in white noise model we construct confidence sets as credible balls with respect to the empirical bayes posterior resulting from a certain two level hierarchical prior the quality of the posterior is characterized by the contraction rate which we allow to be local that is depending on the parameter the issue of optimality of the constructed confidence sets is addressed via a trade off between its size the local radial rate and its coverage probability we introduce excessive bias restriction ebr more general than self similarity and polished tail condition recently studied in the literature under ebr we establish the confidence optimality of our credible set with some local oracle radial rate we also derive the oracle estimation inequality and the oracle posterior contraction rate the obtained local results are more powerful than global adaptive minimax results for a number of smoothness scales follow as consequence in particular the ones considered by szabo et al ann statist 43 2015 1391 1428

design of observers for linear systems with unknown inputs with a straightforward treatment a design of reduced order observers is presented for linear systems with unknown inputs the observers are derived with a physical meaning some explanations are given within the framework of descriptor system observer design principles the conditions for the existence of the observers are presented some illustrative examples are included

global uniform risk bounds for wavelet deconvolution estimators we consider the statistical deconvolution problem where one observes n replications from the model y x where x is the unobserved random signal of interest and is an independent random error with distribution under weak assumptions on the decay of the fourier transform of we derive upper bounds for the finite sample sup norm risk of wavelet deconvolution density estimators f n for the density f of x where f r r is assumed to be bounded we then derive lower bounds for the minimax sup norm risk over besov balls in this estimation problem and show that wavelet deconvolution density estimators attain these bounds we further show that linear estimators adapt to the unknown smoothness of f if the fourier transform of decays exponentially and that a corresponding result holds true for the hard thresholding wavelet estimator if decays polynomially we also analyze the case where f is a supersmooth analytic density we finally show how our results and recent techniques from rademacher processes can be applied to construct global confidence bands for the density f

rates of contraction of posterior distributions based on gaussian process priors we derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on gaussian processes the rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel hilbert space of the gaussian process and the small ball probabilities of the gaussian process we determine these quantities for a range of examples of gaussian priors and in several statistical settings for instance we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a fractionally integrated brownian motion we also consider regression with gaussian errors and smooth classification under a logistic or probit link function combined with various priors

fuzzy support vector machines a support vector machine svm learns the decision surface from two distinct classes of the input points in many applications each input point may not be fully assigned to one of these two classes in this paper we apply a fuzzy membership to each input point and reformulate the svms such that different input points can make different contributions to the learning of decision surface we call the proposed method fuzzy svms fsvms

the smoothness test for a density function abstract the problem of testing hypothesis that a density function has no more than derivatives versus it has more than derivatives is considered for a solution the l 2 norms of wavelet orthogonal projections on some orthogonal differences of spaces from a multiresolution analysis is used for the construction of the smoothness test an asymptotic distribution of a smoothness estimator is used to analyze that asymptotic distribution a new technique of enrichment procedure is proposed the finite sample behavior of the smoothness test is demonstrated in a numerical experiment in case of determination if a density function is continuous or discontinuous

generalized k nearest neighbor rules abstract this paper discusses a suitable framework for generalizing the k nearest neighbor k nnr algorithms to cases where the design labels are not necessarily crisp i e not binary valued the proposed framework imbeds all crisp k nnr s into a larger structure of fuzzy k nnr s the resultant model enables neighborhood voting to be a continuous function of local labels at a point to be classified we emphasize that the decision itself may be crisp even when a fuzzy k nnr is utilized the usefulness of this extension of the conventional technique is illustrated by comparing the observed error rates of four classifiers the hard k nnr two fuzzy k nnr s and a fuzzy 1 nearest prototype rule 1 npr on three data sets anderson s iris data and samples from synthetic univariate and bivariate normal mixtures our conclusions all four designs yield comparable usually within 4 error rates the fuzzy c means fcm based k nnr is usually the best design the fcm 1 npr is the most efficient and perhaps most useful of the four designs and finally that generalized nnr s are an important and useful extension of the conventional ones

disturbance decoupled observer design a unified viewpoint this paper is concerned with the observer design for linear systems with unknown inputs an equivalent system which is free of unknown inputs is derived for the purpose of the observer design based on the equivalent system description one can design an observer full order reduced order minimal order or functional for linear systems with unknown inputs using well known techniques it is shown that the equivalent system exists as long as there exists a disturbance decoupled observer two sets of the known existence conditions of disturbance decoupled observers are proved to be equivalent

statistical inference for the optimal approximating model in the setting of high dimensional linear models with gaussian noise we investigate the possibility of confidence statements connected to model selection although there exist numerous procedures for adaptive point estimation the construction of adaptive confidence regions is severely limited cf li in ann stat 17 1001 1008 1989 the present paper sheds new light on this gap we develop exact and adaptive confidence regions for the best approximating model in terms of risk one of our constructions is based on a multiscale procedure and a particular coupling argument utilizing exponential inequalities for noncentral 2 distributions we show that the risk and quadratic loss of all models within our confidence region are uniformly bounded by the minimal risk times a factor close to one

a simple bootstrap method for constructing nonparametric confidence bands for functions standard approaches to constructing nonparametric confidence bands for functions are frustrated by the impact of bias which generally is not estimated consistently when using the bootstrap and conventionally smoothed function estimators to overcome this problem it is common practice to either undersmooth so as to reduce the impact of bias or oversmooth and thereby introduce an explicit or implicit bias estimator however these approaches and others based on nonstandard smoothing methods complicate the process of inference for example by requiring the choice of new unconventional smoothing parameters and in the case of undersmoothing producing relatively wide bands in this paper we suggest a new approach which exploits to our advantage one of the difficulties that in the past has prevented an attractive solution to the problem the fact that the standard bootstrap bias estimator suffers from relatively high frequency stochastic error the high frequency together with a technique based on quantiles can be exploited to dampen down the stochastic error term leading to relatively narrow simple to construct confidence bands

adaptive confidence bands for nonparametric regression functions this article proposes a new formulation for the construction of adaptive confidence bands cbs in nonparametric function estimation problems cbs which have size that adapts to the smoothness of the function while guaranteeing that both the relative excess mass of the function lying outside the band and the measure of the set of points where the function lies outside the band are small it is shown that the bands adapt over a maximum range of lipschitz classes the adaptive cb can be easily implemented in standard statistical software with wavelet support we investigate the numerical performance of the procedure using both simulated and real datasets the numerical results agree well with the theoretical analysis the procedure can be easily modified and used for other nonparametric function estimation models supplementary materials for this article are available online

an exponential inequality for the distribution function of the kernel density estimator with applications to adaptive estimation it is shown that the uniform distance between the distribution function f n k h of the usual kernel density estimator based on an i i d sample from an absolutely continuous law on mathbb r with bandwidth h and the empirical distribution function fn satisfies an exponential inequality this inequality is used to obtain sharp almost sure rates of convergence of f n k h n f n infty under mild conditions on the range of bandwidths hn including the usual mise optimal choices another application is a dvoretzky kiefer wolfowitz type inequality for f n k h f infty where f is the true distribution function the exponential bound is also applied to show that an adaptive estimator can be constructed that efficiently estimates the true distribution function f in sup norm loss and at the same time estimates the density of f if it exists but without assuming it does at the best possible rate of convergence over holder balls again in sup norm loss

adaptive confidence sets in l 2 the problem of constructing confidence sets that are adaptive in l 2 loss over a continuous scale of sobolev classes of probability densities is considered adaptation holds where possible with respect to both the radius of the sobolev ball and its smoothness degree and over maximal parameter spaces for which adaptation is possible two key regimes of parameter constellations are identified one where full adaptation is possible and one where adaptation requires critical regions be removed techniques used to derive these results include a general nonparametric minimax test for infinite dimensional null and alternative hypotheses and new lower bounds for l 2 adaptive confidence sets

observers for descriptor systems abstract the design of observers is investigated for the estimation of the descriptor vector x t of descriptor systems of the form ex t ax t bu t y t cx t such systems are divided into two categories non causal impulsive systems and causal impulse free systems for non causal systems an observer of order rank e is proposed and termed a full order observer since the dynamical order of a descriptor system is regarded as rank e for causal systems it is shown that the number of independent outputs m does not exceed rank e and an observer of order rank e m is introduced and termed a minimal order observer the approach is based on the transformation of the system into particular design coordinates via a singular value decomposition of the matrix e illustrative examples are included

frequentist coverage of adaptive nonparametric bayesian credible sets we investigate the frequentist coverage of bayesian credible sets in a nonparametric setting we consider a scale of priors of varying regularity and choose the regularity by an empirical bayes method next we consider a central set of prescribed posterior probability in the posterior distribution of the chosen regularity we show that such an adaptive bayes credible set gives correct uncertainty quantification of polished tail parameters in the sense of high probability of coverage of such parameters on the negative side we show by theory and example that adaptation of the prior necessarily leads to gross and haphazard uncertainty quantification for some true parameters that are still within the hyperrectangle regularity scale

on adaptive posterior concentration rates tion of holder balls and that moreover achieve our lower bound we analyse the consequences in terms of asymptotic behaviour of poste rior credible balls as well as frequentist minimax adaptive estimation our results are appended with an upper bound for the contraction rate under an arbitrary loss in a generic regular experiment the up per bound is attained for certain sieve priors and enables to extend our results to density estimation

multiscale change point inference summary we introduce a new estimator the simultaneous multiscale change point estimator smuce for the change point problem in exponential family regression an unknown step function is estimated by minimizing the number of change points over the acceptance region of a multiscale test at a level the probability of overestimating the true number of change points k is controlled by the asymptotic null distribution of the multiscale test statistic further we derive exponential bounds for the probability of underestimating k by balancing these quantities will be chosen such that the probability of correctly estimating k is maximized all results are even non asymptotic for the normal case on the basis of these bounds we construct asymptotically honest confidence sets for the unknown step function and its change points at the same time we obtain exponential bounds for estimating the change point locations which for example yield the minimax rate up to a log term finally the simultaneous multiscale change point estimator achieves the optimal detection rate of vanishing signals as n even for an unbounded number of change points we illustrate how dynamic programming techniques can be employed for efficient computation of estimators and confidence regions the performance of the multiscale approach proposed is illustrated by simulations and in two cutting edge applications from genetic engineering and photoemission spectroscopy

anti concentration and honest adaptive confidence bands modern construction of uniform condence e and nickl 2010 this condition requires the existence of a limit distribution of an extreme value type for the supremum of a studentized empirical process equivalently for the supremum of a gaussian process with the same covariance function as that of the studentized empirical process the principal contribution of this paper is to remove the need for this classical condition we show that a considerably weaker sucient condi tion is derived from an anti concentration property of the supremum of the approximating gaussian process and we derive an inequality lead ing to such a property for separable gaussian processes we refer to the new condition as a generalized sbr condition our new result shows that the supremum does not concentrate too fast around any value we then apply this result to derive a gaussian multiplier bootstrap procedure for constructing honest condence bands for nonparametric density estimators this result can be applied in other nonparametric problems as well an essential advantage of our approach is that it ap plies generically even in those cases where the limit distribution of the supremum of the studentized empirical process does not exist or is un known this is of particular importance in problems where resolution levels or other tuning parameters have been chosen in a data driven fash ion which is needed for adaptive constructions of the condence bands furthermore our approach is asymptotically honest at a polynomial rate namely the error in coverage level converges to zero at a fast polynomial speed with respect to the sample size in sharp contrast the approach based on extreme value theory is asymptotically honest only at a logarithmic rate the error converges to zero at a slow loga rithmic speed finally of independent interest is our introduction of a new practical version of lepski s method which computes the optimal non conservative resolution levels via a gaussian multiplier bootstrap method

a unifying approach of lti discrete time state observers by means of complementary matrices this paper deals with the generalization of state observers in the framework of lti deterministic discrete time systems using complementary matrix formalism four necessary and sufficient general conditions have been established in the design of any type of state observer furthermore necessary sufficient intrinsic conditions have been deduced respecting these intrinsic conditions guarantees satisfaction of the preceding general conditions an illustrative example is also included

support vector based fuzzy neural network for pattern classification fuzzy neural networks fnns for pattern classification usually use the backpropagation or c cluster type learning algorithms to learn the parameters of the fuzzy rules and membership functions from the training data however such kinds of learning algorithms usually cannot minimize the empirical risk training error and expected risk testing error simultaneously and thus cannot reach a good classification performance in the testing phase to tackle this drawback a support vector based fuzzy neural network svfnn is proposed for pattern classification in this paper the svfnn combines the superior classification power of support vector machine svm in high dimensional data spaces and the efficient human like reasoning of fnn in handling uncertainty information a learning algorithm consisting of three learning phases is developed to construct the svfnn and train its parameters in the first phase the fuzzy rules and membership functions are automatically determined by the clustering principle in the second phase the parameters of fnn are calculated by the svm with the proposed adaptive fuzzy kernel function in the third phase the relevant fuzzy rules are selected by the proposed reducing fuzzy rule method to investigate the effectiveness of the proposed svfnn classification it is applied to the iris vehicle dna satimage ijcnn1 datasets from the uci repository statlog collection and ijcnn challenge 2001 respectively experimental results show that the proposed svfnn for pattern classification can achieve good classification performance with drastically reduced number of fuzzy kernel functions

can one estimate the conditional distribution of post model selection estimators we consider the problem of estimating the conditional distribution of a post model selection estimator where the conditioning is on the selected model the notion of a post model selection estimator here refers to the combined procedure resulting from first selecting a model e g by a model selection criterion such as aic or by a hypothesis testing procedure and then estimating the parameters in the selected model e g by least squares or maximum likelihood all based on the same data set we show that it is impossible to estimate this distribution with reasonable accuracy even asymptotically in particular we show that no estimator for this distribution can be uniformly consistent not even locally this follows as a corollary to local minimax lower bounds on the performance of estimators for this distribution similar impossibility results are also obtained for the conditional distribution of linear functions e g predictors of the post model selection estimator

convergence rates of posterior distributions we consider the asymptotic behavior of posterior distributions and bayes estimators for infinite dimensional statistical models we give general results on the rate of convergence of the posterior measure these are applied to several examples including priors on finite sieves log spline models dirichlet processes and interval censoring

uniform limit theorems for wavelet density estimators let pn y k y k jn 1 l 0 k lk 2l 2 2 l y k be the linear wavelet density estimator where are a father and a mother wavelet with compact support are the empirical wavelet coefficients based on an i i d sample of random variables distributed according to a density p 0 on r and j n z j n several uniform limit theorems are proved first the almost sure rate of convergence of sup y r pn y ep n y is obtained and a law of the logarithm for a suitably scaled version of this quantity is established this implies that sup y r p n y po y attains the optimal almost sure rate of convergence for estimating p 0 if j n is suitably chosen second a uniform central limit theorem as well as strong invariance principles for the distribution function of p n that is for the stochastic processes n f w n s f s n s pn p 0 s r are proved and more generally uniform central limit theorems for the processes n p n p 0 f f for other donsker classes f of interest are considered as a statistical application it is shown that essentially the same limit theorems can be obtained for the hard thresholding wavelet estimator introduced by donoho et al ann statist 24 1996 508 539

the class of all stable unbiased state estimators abstract recent literature on robust control has made extensive use of the characterization of the class of all stabilizing compensators this note shows that a similar result holds for the class of all stable unbiased state estimators this result is believed to have major implications for the design of robust state estimators to deal with non standard noise sources modelling errors etc