the lagrangian relaxation method for solving integer programming problems this article originally appeared in management science january 1981 volume 27 number 1 pp 1 18 published by the institute of management sciences one of the most computationally useful ideas of the 1970s is the observation that many hard integer programming problems can be viewed as easy problems complicated by a relatively small set of side constraints dualizing the side constraints produces a lagrangian problem that is easy to solve and whose optimal value is a lower bound for minimization problems on the optimal value of the original problem the lagrangian problem can thus be used in place of a linear programming relaxation to provide bounds in a branch and bound algorithm this approach has led to dramatically improved algorithms for a number of important problems in the areas of routing location scheduling assignment and set covering this paper is a review of lagrangian relaxation based on what has been learned in the last decade

adaptive estimation of the dynamics of a discrete time stochastic volatility model abstract this paper is concerned with the discrete time stochastic volatility model y i exp x i 2 i x i 1 b x i x i i 1 where only y i is observed the model is rewritten as a particular hidden model z i x i e i x i 1 b x i x i i 1 where i and e i are independent sequences of i i d noise moreover the sequences x i and e i are independent and the distribution of e is known then our aim is to estimate the functions b and 2 when only observations z 1 z n are available we propose to estimate b f and b 2 2 f and study the integrated mean square error of projection estimators of these functions on automatically selected projection spaces by ratio strategy estimators of b and 2 are then deduced the mean square risk of the resulting estimators are studied and their rates are discussed lastly simulation experiments are provided constants in the penalty functions defining the estimators are calibrated and the quality of the estimators is checked on several examples

on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

lagrangean decomposition a model yielding stronger lagrangean bounds given a mixed integer programming problem with two matrix constraints it is possible to define a lagrangean relaxation such that the relaxed problem decomposes additively into two subproblems each having one of the two matrices of the original problem as its constraints there is one lagrangean multiplier per variable we prove that the optimal value of this new lagrangean dual dominates the optimal value of the lagrangean dual obtained by relaxing one set of constraints and give a necessary condition for a strict improvement we show on an example that the resulting bound improvement can be substantial we show on a complex practical problem how lagrangean decomposition may help uncover hidden special structures and thus yield better solution methodology

on estimating a density using hellinger distance and some other strange facts on s interesse ici aux possibles vitesses d estimation d une densite a support compact dans m sous des hypotheses de regularite lorsque la perte est mesuree par le carre de la distance de hellinger on regardera aussi le cas connu des normes mathbb l q pour 1 q 2 et le risque est le risque minimax sur la famille on donne une methode generale permettant de traiter les problemes dans le cadre de la theorie de l approximation sous des conditions concernant l entropie metrique et l e capacite des familles a estimer les rapports entre regularite et entropie metrique etant bien connus nous pourrons aussi traiter les cas classiques et d autres qui le sont moins sous des conditions de bornes inferieures les vitesses sont celles observees pour la norme mathbb l q mais elles different dans le cas general on montre aussi que les restrictions sur la compacite du support ou la regularite sont indispensables et que leur absence mene a l impossibilite d obtenir une estimation raisonnable en ce sens que n importe quelle suite d estimateurs sera arbitrairement mauvaise en un point au moins un resultat analogue est vrai sous des conditions de regularite

minimum complexity regression estimation with weakly dependent observations the minimum complexity regression estimation framework barron 1991 barron and cover 1991 and rissanen 1989 is a general data driven methodology for estimating a regression function from a given list of parametric models using independent and identically distributed i i d observations we extend barron s regression estimation framework to m dependent observations and to strongly mixing observations in particular we propose abstract minimum complexity regression estimators for dependent observations which may be adapted to a particular list of parametric models and establish upper bounds on the statistical risks of the proposed estimators in terms of certain deterministic indices of resolvability assuming that the regression function satisfies a certain fourier transform type representation we examine minimum complexity regression estimators adapted to a list of parametric models based on neural networks and by using the upper bounds for the abstract estimators we establish rates of convergence for the statistical risks of these estimators also as a key tool we extend the classical bernstein inequality from i i d random variables to m dependent processes and to strongly mixing processes

rates of convergence for minimum contrast estimators we shall present here a general study of minimum contrast estimators in a nonparametric setting although our results are also valid in the classical parametric case for independent observations these estimators include many of the most popular estimators in various situations such as maximum likelihood estimators least squares and other estimators of the regression function estimators for mixture models or deconvolution the main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters optimal rates depending on entropy conditions are already known at least for some of the models involved and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large but under some circumstances large entropies or changes in the entropy structure due to local perturbations the resulting the rates are only suboptimal counterexamples are constructed which show that the phenomenon is real for non parametric maximum likelihood or regression this proves that under purely metric assumptions our theorem is optimal and that minimum contrast estimators happen to be suboptimal

density estimation by wavelet thresholding density estimation is a commonly used test case for nonparametric estimation methods we explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients minimax rates of convergence are studied over a large range of besov function classes b pq and for a range of global l p error measures 1 p p some form of nonlinearity is essential since the minimax linear estimators are suboptimal by polynomial powers of n a second approach using an approximation of a gaussian white noise model in a mallows metric is used to attain exactly optimal rates of convergence for quadratic error p 2

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

a universal prior for integers and estimation by minimum description length of the number of bits required to write down the observed data has been reformulated to extend the classical maximum likelihood principle the principle permits estimation of the number of the parameters in statistical models in addition to their values and even of the way the parameters appear in the models i e of the model structures the principle rests on a new way to interpret and construct a universal prior distribution for the integers which makes sense even when the parameter is an individual object truncated realvalued parameters are converted to integers by dividing them by their precision and their prior is determined from the universal prior for the integers by optimizing the precision 1 introduction in this paper we study estimation based upon the principle of minimizing the total number of binary digits required to rewrite the observed data when each observation is given with some precision instead of attempting at an absolutely shortest description which would be futile we look for the optimum relative to a class of parametrically given distributions this minimum description length mdl principle which we introduced in a less comprehensive form in 25 turns out to degenerate to the more familiar maximum likelihood ml principle in case the number of parameters in the models is fixed so that the description length of the parameters themselves can be ignored in another extreme case where the parameters determine the data it similarly degenerates to jaynes s principle of maximum entropy 14 but the main power of the new criterion is that it permits estimates of the entire model its parameters their number and even the way the parameters appear in the model i e the model structure hence there will be no need to supplement the estimated parameters with a separate hypothesis test to decide whether a model is adequately parameterized or perhaps over parameterized

asymptotic strong duality for bounded integer programming a logarithmic exponential dual formulation a logarithmic exponential dual formulation is proposed in this paper for bounded integer programming problems this new dual formulation possesses an asymptotic strong duality property and guarantees the identification of an optimal solution of the primal problem these prominent features are achieved by exploring a novel nonlinear lagrangian function deriving an asymptotic zero duality gap investigating the unimodality of the associated dual function and ensuring the primal feasibility of optimal solutions in the dual formulation one other feature of the logarithmicexponential dual formulation is that no actual dual search is needed when parameters are set above certain threshold values

degree of approximation results for feedforward networks approximating unknown mappings and their derivatives recently barron 1993 has given rates for hidden layer feedforward networks with sigmoid activation functions approximating a class of functions satisfying a certain smoothness condition these rates do not depend on the dimension of the input space we extend barron s results to feedforward networks with possibly nonsigmoid activation functions approximating mappings and their derivatives simultaneously our conditions are similar but not identical to barron s but we obtain the same rates of approximation showing that the approximation error decreases at rates as fast as n 1 2 where n is the number of hidden units the dimension of the input space appears only in the constants of our bounds

nonlinear 0 1 programming i linearization techniques any real valued nonlinear function in 0 1 variables can be rewritten as a multilinear function we discuss classes of lower and upper bounding linear expressions for multilinear functions in 0 1 variables for any multilinear inequality in 0 1 variables we define an equivalent family of linear inequalities this family contains the well known system of generalized covering inequalities as well as other linear equivalents of the multilinear inequality that are more compact i e of smaller cardinality in a companion paper 7 we discuss dominance relations between various linear equivalents of a multilinear inequality and describe a class of algorithms for multilinear 0 1 programming based on these results

smooth discrimination analysis ties f and g can be based on the estimation of the set g x fx gx we consider applications where it is appropriate to assume that the region g has a smooth boundary or belongs to another nonparametric class of sets in particular this assumption makes sense if discrimination is used as a data analytic tool decision rules based on minimization of empirical risk over the whole class of sets and over sieves are considered their rates of convergence are obtained we show that these rules achieve optimal rates for estimation of g and optimal rates of convergence for bayes risks an interesting conclusion is that the optimal rates for bayes risks can be very fast in particular faster than the parametric root n rate these fast rates cannot be guaranteed for plug in rules

a simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training a general convergence criterion for certain iterative sequences in hilbert space is presented for an important subclass of these sequences estimates of the rate of convergence are given under very mild assumptions these results establish an 0 1 f4n nonsampling convergence rate for projection pursuit regression and neural network training where n represents the number of ridge functions neurons or coefficients in a greedy basis expansion

approximation dans les espaces metriques et theorie de l estimation we investigate the relations between the speed of estimation and the metric structure of the parameter space especially in the case when its metric dimension is infinite given some distance d on generally hellinger distance in the case of n i i d variables we consider the minimax risk for n observations r n q mathop inf sup limits t n theta in theta text ie theta d q theta t n t n being any estimate of we shall look for functions r such that for positive constants c 1 q and c 2 q c 1 r q n r n q c2 r q n r n is the speed of estimation and we shall show under fairly general conditions including i i d variables and regular cases of markov chains and stationnary gaussian processes that r n is determined up to multiplicative constants by the metric structure of we shall also give a construction for some sort of universal estimates the risk of which is bounded by c 2 r q n in all cases where the preceding theory applies

new concentration inequalities in product spaces we introduce three new ways to measure the distance from a point to a subset of a product space and we prove corresponding concentration inequalities each of them allows to control the fluctuation of a new class of random variables

nonlinear 0 1 programming ii dominance relations and algorithms a nonlinear 0 1 program can be restated as a multilinear 0 1 program which in turn is known to be equivalent to a linear 0 1 program with generalized covering g c inequalities in a companion paper 6 we have defined a family of linear inequalities that contains more compact smaller cardinality linearizations of a multilinear 0 1 program than the one based on the g c inequalities in this paper we analyze the dominance relations between inequalities of the above family in particular we give a criterion that can be checked in linear time for deciding whether a g c inequality can be strengthened by extending the cover from which it was derived we then describe a class of algorithms based on these results and discuss our computational experience we conclude that the g c inequalities can be strengthened most of the time to an extent that increases with problem density in particular the algorithm using the strengthening procedure outperforms the one using only g c inequalities whenever the number of nonlinear terms per constraint exceeds about 12 15 and the difference in their performance grows with the number of such terms

geometrical and performance analysis of gmd and chase decoding algorithms the overall number of nearest neighbors in bounded distance decoding bdd algorithms is given by n sub 0 eff n sub 0 n sub bdd where nbdd denotes the number of additional non codeword neighbors that are generated during the suboptimal decoding process we identify and enumerate the nearest neighbors associated with the original generalized minimum distance gmd and chase 1972 decoding algorithms after careful examination of the decision regions of these algorithms we derive an approximated probability ratio between the error contribution of a noncodeword neighbor one of n sub bdd points and a codeword nearest neighbor for chase algorithm 1 it is shown that the contribution to the error probability of a noncodeword nearest neighbor is a factor of 2 sup d 1 less than the contribution of a codeword while for chase algorithm 2 the factor is 2 sup d 2 1 d being the minimum hamming distance of the code for chase algorithm 3 and gmd a recursive procedure for calculating this ratio which turns out to be nonexponential in d is presented this procedure can also be used for specifically identifying the error patterns associated with chase algorithm 3 and gmd utilizing the probability ratio we propose an improved approximated upper bound on the probability of error based on the union bound approach simulation results are given to demonstrate and support the analytical derivations

lagrangian decomposition for integer nonlinear programming with linear constraints we present a lagrangean decomposition to study integer nonlinear programming problems solving the dual lagrangean relaxation we have to obtain at each iteration the solution of a nonlinear programming with continuous variables and an integer linear programming decreasing iteratively the primal dual gap we propose two algorithms to treat the integer nonlinear programming

an invitation to quantum tomography we describe quantum tomography as an inverse statistical problem in which the quantum state of a light beam is the unknown parameter and the data are given by results of measurements performed on identical quantum systems the state can be represented as an infinite dimensional density matrix or equivalently as a density on the plane called the wigner function we present consistency results for pattern function projection estimators and for sieve maximum likelihood estimators for both the density matrix of the quantum state and its wigner function we illustrate the performance of the estimators on simulated data an em algorithm is proposed for practical implementation there remain many open problems e g rates of convergence adaptation and studying other estimators a main purpose of the paper is to bring these to the attention of the statistical community

an algebraic approach to integer portfolio problems abstract integer variables allow the treatment of some portfolio optimization problems in a more realistic way and introduce the possibility of adding some natural features to the model we propose an algebraic approach to maximize the expected return under a given admissible level of risk measured by the covariance matrix to reach an optimal portfolio it is an essential ingredient the computation of different test sets via grobner basis of linear subproblems that are used in a dual search strategy

risk bounds for model selection via penalization performance bounds for criteria for model selection are devel oped using recent theory for sieves the model selection criteria are based on an empirical loss or contrast function with an added penalty term moti vated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of ob servations most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function we show that the quadratic risk of the minimum penal ized empirical contrast estimatoris bounded by an index of the accuracy of the sieve this accuracy index quanties the trade off among the candidate models between the approximation error and parameter dimension relative to sample size if we choose a list of models which exhibit good approximation prop erties with respect to different classes of smoothness the estimator can be simultaneously minimax rate optimal in each of those classes this is what is usually called adaptation the type of classes of smoothness in which one gets adaptation depends heavily on the list of models if too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously it may happen that the estimator is only approx

a distributed computation algorithm for solving portfolio problems with integer variables a portfolio problem with integer variables can facilitate the use of complex models including models containing discrete asset values transaction costs and logical constraints this study proposes a distributed algorithm for solving a portfolio program to obtain a global optimum for a portfolio problem with n integer variables the objective function first is converted into an ellipse function containing n separated quadratic terms next the problem is decomposed into m equal size separable programming problems solvable by a distributed computation system composed of m personal computers linked via the internet the numerical examples illustrate that the proposed method can obtain the global optimum effectively for large scale portfolio problems involving integral variables

adaptive confidence sets in l 2 the problem of constructing confidence sets that are adaptive in l 2 loss over a continuous scale of sobolev classes of probability densities is considered adaptation holds where possible with respect to both the radius of the sobolev ball and its smoothness degree and over maximal parameter spaces for which adaptation is possible two key regimes of parameter constellations are identified one where full adaptation is possible and one where adaptation requires critical regions be removed techniques used to derive these results include a general nonparametric minimax test for infinite dimensional null and alternative hypotheses and new lower bounds for l 2 adaptive confidence sets

on the mixed integer signomial programming problems this paper proposes an approximate method to solve the mixed integer signomial programming problem for which the objective function and the constraints may contain product terms with exponents and decision variables which could be continuous or integral a linear programming relaxation is derived for the problem based on piecewise linearization techniques which first convert a signomial term into the sum of absolute terms these absolute terms are then linearized by linearization strategies in addition a novel approach is included for solving integer and undefined problems in the logarithmic piecewise technique which leads to more usefulness of the proposed method the proposed method could reach a solution as close as possible to the global optimum

a survey for the quadratic assignment problem the quadratic assignment problem qap one of the most difficult problems in the np hard class models many real life problems in several areas such as facilities location parallel and distributed computing and combinatorial data analysis combinatorial optimization problems such as the traveling salesman problem maximal clique and graph partitioning can be formulated as a qap in this paper we present some of the most important qap formulations and classify them according to their mathematical sources we also present a discussion on the theoretical resources used to define lower bounds for exact and heuristic algorithms we then give a detailed discussion of the progress made in both exact and heuristic solution methods including those formulated according to metaheuristic strategies finally we analyze the contributions brought about by the study of different approaches

approximation of density functions by sequences of exponential families 1 introduction consider the estimation of a probability density func tion p x defined on a bounded interval we approximate the logarithm of the density by a basis function expansion consisting of polynomials splines or trigonometric series the expansion yields a regular exponential family within which we estimate the density by the method of maximum likelihood this method of density estimation arises by application of the principle of maxi mum entropy or minimum relative entropy subject to empirical constraints we show that if the logarithm of the density has r square integrable deriva tives f idr log p12 ox then the sequence of density estimators p converges to p in the sense of relative entropy kullback leibler fp log p pn at rate opr 1 m2r m n as m x c and m2 n 0 in the spline and trigonometric cases and m3 n o 0 in the polynomial case where m is the dimension of the family and n is the sample size boundary conditions are assumed for the density in the trigonometric case this convergence rate specializes to op r n 2r 2r 1 by setting m nl 2r 1 when the log density is known to have degree of smoothness at least r analogous convergence results for the relative entropy are shown to hold in general for any class of log density functions and sequence of finite dimensional linear spaces having l2 and l approximation properties the approximation of log densities using polynomials has previously been considered by neyman 1937 to define alternatives for goodness of fit tests by good 1963 as an application of the method of maximum entropy or minimum relative entropy by crain 1974 1976a b 1977 who demonstrates existence and consistency of the maximum likelihood estimator and by mead and

minimax estimation via wavelet shrinkage we attempt to recover an unknown function from noisy sampled data using orthonormal bases of compactly supported wavelets we develop a nonlinear method which works in the wavelet domain by simple nonlinear shrinkage of the empirical wavelet coefficients the shrinkage can be tuned to be nearly minimax over any member of a wide range of triebel and besov type smoothness constraints and asymptotically minimax over besov bodies with p q linear estimates cannot achieve even the minimax rates over triebel and besov classes with p 2 so the method can significantly outperform every linear method e g kernel smoothing spline sieve in a minimax sense variants of our method based on simple threshold nonlinear estimators are nearly minimax our method possesses the interpretation of spatial adaptivity it reconstructs using a kernel which may vary in shape and bandwidth from point to point depending on the data least favorable distributions for certain of the triebel and besov scales generate objects with sparse wavelet transforms many real objects have similarly sparse transforms which suggests that these minimax results are relevant for practical problems sequels to this paper which was first drafted in november 1990 discuss practical implementation spatial adaptation properties universal near minimaxity and applications to inverse problems

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

approximation of least squares regression on nested subspaces on presente une methode generale pour approximer le biais et la variance dans une echelle de normes hilbertiennes

minimax risk over l p balls for l p error consider estimating the mean vector from datan n 2 i withl q norm loss q 1 when is known to lie in ann dimensionall p ball p 0 for largen the ratio of minimaxlinear risk to minimax risk can bearbitrarily large ifp q obvious exceptions aside the limiting ratio equals 1 only ifp q 2 our arguments are mostly indirect involving a reduction to a univariate bayes minimax problem whenp q simple non linear co ordinatewise threshold rules are asymptotically minimax at small signal to noise ratios and within a bounded factor of asymptotic minimaxity in general we also give asymptotic evaluations of the minimax linear risk our results are basic to a theory of estimation in besov spaces using wavelet bases to appear elsewhere

hinging hyperplanes for regression classification and function approximation a hinge function y h x consists of two hyperplanes continuously joined together at a hinge in regression prediction classification pattern recognition and noiseless function approximation use of sums of hinge functions gives a powerful and efficient alternative to neural networks with computation times several orders of magnitude less than is obtained by fitting neural networks with a comparable number of parameters a simple and effective method for finding good hinges is presented

quadratic assignment problems abstract this paper surveys quadratic assignment problems qap at first several applications of this problem class are described and mathematical formulations of qaps are given then some exact solution methods and good heuristics are outlined their computational behaviour is illustrated by numerical results further recent results on the asymptotic probabilistic behaviour of qaps are outlined

new dependence coefficients examples and applications to statistics to measure the dependence between a real valued random variable x and a algebra we consider four distances between the conditional distribution function of x given and the distribution function of x the coefficients obtained are weaker than the corresponding mixing coefficients and may be computed in many situations in particular we show that they are well adapted to functions of mixing sequences iterated random functions and dynamical systems starting from a new covariance inequality we study the mean integrated square error for estimating the unknown marginal density of a stationary sequence we obtain optimal rates for kernel estimators as well as projection estimators on a well localized basis under a minimal condition on the coefficients using recent results we show that our coefficients may be also used to obtain various exponential inequalities a concentration inequality for lipschitz functions and a berry esseen type inequality

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

wavelet shrinkage asymptopia much recent effort has sought asymptotically minimax methods for recovering infinite dimensional objects curves densities spectral densities images from noisy data a now rich and complex body of work develops nearly or exactly minimax estimators for an array of interesting problems unfortunately the results have rarely moved into practice for a variety of reasons among them being similarity to known methods computational intractability and lack of spatial adaptivity we discuss a method for curve estimation based on n noisy data translate the empirical wavelet coefficients towards the origin by an amount 2 log n n the proposal differs from those in current use is computationally practical and is spatially adaptive it thus avoids several of the previous objections further the method is nearly minimax both for a wide variety of loss functions pointwise error global error measured in l p norms pointwise and global error in estimation of derivatives and for a wide range of smoothness classes including standard holder and sobolev classes and bounded variation this is a much broader near optimality than anything previously proposed we draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves finally the theory underlying the method is interesting as it exploits a correspondence between statistical questions and questions of optimal recovery and information based complexity

large sample inference for log spline models let f be a continuous and positive unknown density on a known compact interval mathscr y let f denote the distribution function of f and let q f 1 denote its quantile function a finite parameter exponential family model based on b splines is constructed maximum likelihood estimation of the parameters of the model based on a random sample of size n from f yields estimates hat f f and hat q of f f and q respectively under mild conditions if the number of parameters tends to infinity in a suitable manner as n rightarrow infty these estimates achieve the optimal rate of convergence the asymptotic behavior of the corresponding confidence bounds is also investigated in particular it is shown that the standard errors of hat f and hat q are asymptotically equal to those of the usual empirical distribution function and empirical quantile function

optimal spatial adaptation to inhomogeneous smoothness an approach based on kernel estimates with variable bandwidth selectors a new v ria e 1 s e 0r estimation is proposed the application of this bandwidth selector leads to kernel estimates that achieve optimal rates of convergence over b 3 sses this implies that the procedure adapts to spatially inhomogeneous s noothness in particu lar the estimates share optimality properties with wavelet estimates based on thresholding of empirical wavelet coefficients 1 introduction in nonparametric curve estimation the statistical anal ysis may focus on the inference of the qualitative structure of the analyzed curve often interesting features of the curve are connected with spatially inhomogeneous smoothness in this case curve estimates that are spatially adaptlve ar e appropriaie a variety of such procedures have been proposed in the literature in breiman friedman olshen and stone 1983 piecewise constant least squares estimates are considered with a data adaptive choice of the pieces cart more generally friedman and silverman 1989 friedman 1991 and luo and wahba 1995 use variable knot splines mars knot points are added removed and allocated recursively using cross validation techniques these methods have shown good performance in simulations and real data examples however no asymptotic theory is available mammen and van de geer 1997 discuss penalized least squares curve estimation for spatially inhomogeneous curves they propose penalty terms which allow more spatial inhomogeneity than the usual l2 norms of deriva tives of the curve the estimates turn out to be variable knot splines see also mammen 1991 results on rates of convergence and a pointwise asymptotic distribution theory are given miiller and stadtmiiller 1987 staniswalis 1989 and brockmann gasser and hermann 1993 propose kernel estimation with locally variable band width selectors the calculation of local bandwidths is based on pilot estima tion of local smoothness characteristics an asymptotic analysis is available

mixed integer quadratic programming this paper considers mixed integer quadratic programs in which the objective function is quadratic in the integer and in the continuous variables and the constraints are linear in the variables of both types the generalized benders decomposition is a suitable approach for solving such programs however the program does not become more tractable if this method is used since benders cuts are quadratic in the integer variables a new equivalent formulation that renders the program tractable is developed under which the dual objective function is linear in the integer variables and the dual constraint set is independent of these variables benders cuts that are derived from the new formulation are linear in the integer variables and the original problem is decomposed into a series of integer linear master problems and standard quadratic subproblems the new formulation does not introduce new primary variables or new constraints into the computational steps of the decomposition algorithm

model selection for regression on a fixed design we deal with the problem of estimating some unknown regression function involved in a regression framework with deterministic design points for this end we consider some collection of finite dimensional linear spaces models and the least squares estimator built on a data driven selected model among this collection this data driven choice is performed via the minimization of some penalized model selection criterion that generalizes on mallows c p we provide non asymptotic risk bounds for the so defined estimator from which we deduce adaptivity properties our results hold under mild moment conditions on the errors the statement and the use of a new moment inequality for empirical processes is at the heart of the techniques involved in our approach

decision theoretic generalizations of the pac model for neural net and other learning applications we describe a generalization of the pac learning model that is based on statistical decision theory in this model the learner receives randomly drawn examples each example consisting of an instance x in x and an outcome y in y and tries to find a hypothesis h x a where h in h that specifies the appropriate action a in a to take for each instance x in order to minimize the expectation of a loss l y a here x y and a are arbitrary sets l is a real valued function and examples are generated according to an arbitrary joint distribution on x times y special cases include the problem of learning a function from x into y the problem of learning the conditional probability distribution on y given x regression and the problem of learning a distribution on x density estimation we give theorems on the uniform convergence of empirical loss estimates to true expected loss rates for certain hypothesis spaces h and show how this implies learnability with bounded sample size disregarding computational complexity as an application we give distribution independent upper bounds on the sample size needed for learning with feedforward neural networks our theorems use a generalized notion of vc dimension that applies to classes of real valued functions adapted from pollard s work and a notion of capacity and metric dimension for classes of functions that map into a bounded metric space supersedes 89 30 and 90 52 also in information and computation vol 100 no 1 september 1992

multiscale likelihood analysis and complexity penalized estimation we describe here a framework for a certain class of multiscale likelihood factorizations wherein in analogy to a wavelet decomposition of an l 2 function a given likelihood function has an alternative representation as a product of conditional densities reflecting information in both the data and the parameter vector localized in position and scale the framework is developed as a set of sufficient conditions for the existence of such factorizations formulated in analogy to those underlying a standard multiresolution analysis for wavelets and hence can be viewed as a multiresolution analysis for likelihoods we then consider the use of these factorizations in the task of nonparametric complexity penalized likelihood estimation we study the risk properties of certain thresholding and partitioning estimators and demonstrate their adaptivity and near optimality in a minimax sense over a broad range of function spaces based on squared hellinger distance as a loss function in particular our results provide an illustration of how properties of classical wavelet based estimators can be obtained in a single unified framework that includes models for continuous count and categorical data types

an asymptotic property of model selection criteria probability models are estimated by use of penalized log likelihood criteria related to akaike 1973 information criterion aic and minimum description length mdl the accuracies of the density estimators are shown to be related to the tradeoff between three terms the accuracy of approximation the model dimension and the descriptive complexity of the model classes the asymptotic risk is determined under conditions on the penalty term and is shown to be minimax optimal for some cases as an application we show that the optimal rate of convergence is simultaneously achieved for log densities in sobolev spaces w sub 2 sup s u without knowing the smoothness parameter s and norm parameter u in advance applications to neural network models and sparse density function estimation are also provided

an optimal selection of regression variables summary an asymptotically optimal selection of regression variables is proposed the key assumption is that the number of control variables is infinite or increases with the sample size it is also shown that mallows s qp akaike s fpe and aic methods are all asymptotically equivalent to this method

sup norm approximation bounds for networks through probabilistic methods we consider the problem of approximating a smooth target function and its derivatives by networks involving superpositions and translations of a fixed activation function the approximation is with respect to the sup norm and the rate is shown to be of order o n sup 1 2 that is the rate is independent of the dimension d the results apply to neural and wavelet networks and extend the work of barren see proc 7th yale workshop on adaptive and learning systems may 1992 and ibid vol 39 p 930 1993 the approach involves probabilistic methods based on central limit theorems for empirical processes indexed by classes of functions

generalized benders decomposition for one class of minlps with vector conic constraint in this paper we mainly study one class of mixed integer nonlinear programming problems minlps with vector conic constraint in banach spaces duality theory of convex vector optimization problems applied to this class of minlps is deeply investigated with the help of duality we use the generalized benders decomposition method to establish an algorithm for solving this minlp several convergence theorems on the algorithm are also presented the convergence theorems generalize and extend the existing results on minlps in finite dimensional spaces

asymptotic optimality for c_p c_l cross validation and generalized cross validation discrete index set c p c l cross validation and generalized cross validation are useful data driven techniques for selecting a good estimate from a proposed class of linear estimates the asymptotic behaviors of these procedures are studied some easily interpretable conditions are derived to demonstrate the asymptotic optimality it is argued that cross validation and generalized cross validation can be viewed as some special ways of applying c l applications in nearest neighbor nonparametric regression and in model selection are discussed in detail