probability inequalities for empirical processes and a law of the iterated logarithm given a class f of functions on x we can view vn as a stochastic process indexed by f and consider limit theorems for this process to prove such theorems it is often helpful to have bounds on the tail of the r v sup w i n f i see for example dudley 1978 kuelbs and dudley 1980 dudley and phillip 1983 or gine and zinn 1983 our main question of interest is for what classes f can best possible bounds be obtained so first we must ask what are these best possible bounds of course

minimax nonparametric classification ii model selection for adaptation for pt i see ibid vol 45 no 7 p 2271 84 1999 we study nonparametric estimation of a conditional probability for classification based on a collection of finite dimensional models for the sake of flexibility different types of models linear or nonlinear are allowed as long as each satisfies a dimensionality assumption we show that with a suitable model selection criterion the penalized maximum likelihood estimator has a risk bounded by an index of resolvability expressing a good tradeoff among approximation error estimation error and model complexity the bound does not require any assumption on the target conditional probability and can be used to demonstrate the adaptivity of estimators based on model selection examples are given with both splines and neural nets and problems of high dimensional estimation are considered the resulting adaptive estimator is shown to behave optimally or near optimally over sobolev classes with unknown orders of interaction and smoothness and classes of integrable fourier transform of gradient in terms of rates of convergence the performance is the same as if one knew which of them contains the true conditional probability in advance the corresponding classifier also converges optimally or nearly optimally simultaneously over these classes

an optimal global nearest neighbor metric a quadratic metric dao x y x y t ao x y is proposed which minimizes the mean squared error between the nearest neighbor asymptotic risk and the finite sample risk under linearity assumptions a heuristic argument is given which indicates that this metric produces lower mean squared error than the euclidean metric a nonparametric estimate of ao is developed if samples appear to come from a gaussian mixture an alternative parametrically directed distance measure is suggested for nearness decisions within a limited region of space examples of some two class gaussian mixture distributions are included

the optimal distance measure for nearest neighbor classification a local distance measure is shown to optimize the performance of the nearest neighbor two class classifier for a finite number of samples the difference between the finite sample error and the asymptotic error is used as the criterion of improvement this new distance measure is compared to the well known euclidean distance an algorithm for practical implementation is introduced this algorithm is shown to be computationally competitive with the present nearest neighbor procedures and is illustrated experimentally a closed form for the corresponding second order moment of this criterion is found finally the above results are extended to

on the computation of the eigenproblems of hydrogen helium in strong magnetic and electric fields with the sparse grid combination technique abstract we introduce the combination technique for the numerical solution of d dimensional eigenproblems on sparse grids here o d log n d 1 different problems each of size o n have to be solved independently this is in contrast to the one problem of size o n d for a conventional finite element discretization where n denotes the number of grid points in one coordinate direction therefore also higher dimensional eigenvalue problems can be treated by our sparse grid combination approach we apply this method to solve the three dimensional schrodinger equation for hydrogen one electron problem and the six dimensional schrodinger equation for helium two electron problem in strong magnetic and electric fields

nonparametric estimation and classification using radial basis function nets and empirical risk minimization studies convergence properties of radial basis function rbf networks for a large class of basis functions and reviews the methods and results related to this topic the authors obtain the network parameters through empirical risk minimization the authors show the optimal nets to be consistent in the problem of nonlinear function approximation and in nonparametric classification for the classification problem the authors consider two approaches the selection of the rbf classifier via nonlinear function estimation and the direct method of minimizing the empirical error probability the tools used in the analysis include distribution free nonasymptotic probability inequalities and covering numbers for classes of functions

asymptotic efficiency of classifying procedures using the hermite series estimate of multivariate probability densities corresp pattern recognition procedures derived from a nonparametric estimate of multivariate probability density functions using the orthogonal hermite system are examined for sufficiently regular densities the convergence rate of the mean integrated square error mise is o n l epsilon epsilon 0 where n is the number of observations and is independent of the dimension as a consequence the rate at which the probability of misclassification converges to the bayes probability of error as the length n of the learning sequence tends to infinity is also independent of the dimension of the class densities and equals o n 1 2 delta delta o

the optimal strategy of a tree classifier abstract this paper deals with the decision rules of a tree classifier for performing the classification at each nonterminal node under the assumption of complete probabilistic information for given tree structure and feature subsets to be used the optimal decision rules strategy are derived which minimize the overall probability of misclassification the primary result is illustrated by an example

machine learning with data dependent hypothesis classes we extend the vc theory of statistical learning to data dependent spaces of classifiers this theory can be viewed as a decomposition of classifier design into two components the first component is a restriction to a data dependent hypothesis class and the second is empirical risk minimization within that class we define a measure of complexity for data dependent hypothesis classes and provide data dependent versions of bounds on error deviance and estimation error we also provide a structural risk minimization procedure over data dependent hierarchies and prove consistency we use this theory to provide a framework for studying the trade offs between performance and computational complexity in classifier design as a consequence we obtain a new family of classifiers with dimension independent performance bounds and efficient learning procedures

optimal bandwidth selection in nonparametric regression function estimation kernel estimators of an unknown multivariate regression function are investigated a bandwidth selection rule is considered which can be formulated in terms of cross validation under mild assumptions on the kernel and the unknown regression function it is seen that this rule is asymptotically optimal

on an asymptotically optimal adaptive classifier design criterion a new approach for estimating classification errors is presented in the model there are two types of classification error empirical and generalization error the first is the error observed over the training samples and the second is the discrepancy between the error probability and empirical error in this research the vapnik and chervonenkis dimension vcdim is used as a measure for classifier complexity based on this complexity measure an estimate for generalization error is developed an optimal classifier design criterion the generalized minimum empirical error criterion gmee is used the gmee criterion consists of two terms the empirical and the estimate of generalization error as an application the criterion is used to design the optimal neural network classifier a corollary to the gamma optimality of neural network based classifiers is proven thus the approach provides a theoretic foundation for the connectionist approach to optimal classifier design experimental results to validate this approach

estimation of a regression function by maxima of minima of linear functions in this paper estimation of a regression function from independent and identically distributed random variables is considered estimates are defined by minimization of the empirical l2 risk over a class of functions which are defined as maxima of minima of linear functions results concerning the rate of convergence of the estimates are derived in particular it is shown that for smooth regression functions satisfying the assumption of single index models the estimate is able to achieve up to some logarithmic factor the corresponding optimal one dimensional rate of convergence hence under these assumptions the estimate is able to circumvent the so called curse of dimensionality the small sample behavior of the estimates is illustrated by applying them to simulated data

automatic pattern recognition a study of the probability of error a test sequence is used to select the best rule from a class of discrimination rules defined in terms of the training sequence the vapnik chervonenkis and related inequalities are used to obtain distribution free bounds on the difference between the probability of error of the selected rule and the probability of error of the best rule in the given class the bounds are used to prove the consistency and asymptotic optimality for several popular classes including linear discriminators nearest neighbor rules kernel based rules histogram rules binary tree classifiers and fourier series classifiers in particular the method can be used to choose the smoothing parameter in kernel based rules to choose k in the k nearest neighbor rule and to choose between parametric and nonparametric rules

a classification procedure using the multiple fourier series abstract a nonparametric procedure using the multiple fourier series is presented and examined conditions for the weak and strong bayes risk consistency are given and the rates at which the probability of misclassification converges to the bayes probability of error are presented the rates for the convergence in mean as well as w p 1 are independent of the dimension

minkowski decomposition of okounkov bodies on surfaces abstract we prove that the okounkov body of a big divisor with respect to a general flag on a smooth projective surface whose pseudo effective cone is rational polyhedral decomposes as the minkowski sum of finitely many simplices and line segments arising as okounkov bodies of nef divisors

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

lower bounds in pattern recognition and learning lower bounds are derived for the performance of any pattern recognition algorithm which using training data selects a discrimination rule from a certain class of rules the bounds involve the vapnik chervonenkis dimension of the class and l the minimal error probability within the class we provide lower bounds when l 0 the usual assumption in valiant s theory of learning and l 0

the rates of convergence of kernel regression estimates and classification rules both nonrecursive and recursive nonparametric regression estimates are studied the rates of weak and strong convergence of kernel estimates as well as corresponding multiple classification errors are derived without assuming the existence of the density of the measurements an application of the obtained results to a nonparametric bayes predication is presented

automatic classification of cervical cells using a binary tree classifier abstract a k means clustering algorithm for designing binary tree classifiers is introduced for the classification of cervical cells at each nonterminal node of the designed binary tree classifier two sets of effective feature are selected one is based on the bhattacharyya distance a measure of separability between two classes the other is based on the merits of classification accuracy the classification result has shown the effectiveness of the features and the binary tree classifier used