information theoretic determination of minimax rates of convergence we present some general results determining minimax bounds on statistical risk for density estimation based on certain information theoretic considerations these bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence

the painleve property for partial differential equations in this paper we define the painleve property for partial differential equations and show how it determines in a remarkably simple manner the integrability the backlund transforms the linearizing transforms and the lax pairs of three well known partial differential equations burgers equation kdv equation and the modified kdv equation this indicates that the painleve property may provide a unified description of integrable behavior in dynamical systems ordinary and partial differential equations while at the same time providing an efficient method for determining the integrability of particular systems

refined error bounds for several learning algorithms this article studies the achievable guarantees on the error rates of certain learning algorithms with particular focus on refining logarithmic factors many of the results are based on a general technique for obtaining bounds on the error rates of sample consistent classifiers with monotonic error regions in the realizable case we prove bounds of this type expressed in terms of either the vc dimension or the sample compression size this general technique also enables us to derive several new bounds on the error rates of general sample consistent learning algorithms as well as refined bounds on the label complexity of the cal active learning algorithm additionally we establish a simple necessary and sufficient condition for the existence of a distribution free bound on the error rates of all sample consistent learning rules converging at a rate inversely proportional to the sample size we also study learning in the presence of classification noise deriving a new excess error rate guarantee for general vc classes under tsybakov s noise condition and establishing a simple and general necessary and sufficient condition for the minimax excess risk under bounded noise to converge at a rate inversely proportional to the sample size

aggregation of svm classifiers using sobolev spaces this paper investigates statistical performances of support vector machines svm and considers the problem of adaptation to the margin parameter and to complexity in particular we provide a classifier with no tuning parameter it is a combination of svm classifiers our contribution is two fold 1 we propose learning rates for svm using sobolev spaces and build a numerically realizable aggregate that converges with same rate 2 we present practical experiments of this method of aggregation for svm using both sobolev spaces and gaussian kernels

risk bounds for embedded variable selection in classification trees the problems of model and variable selections for classification trees are jointly considered a penalized criterion is proposed which explicitly takes into account the number of variables and a risk bound inequality is provided for the tree classifier minimizing this criterion this penalized criterion is compared to the one used during the pruning step of the cart algorithm it is shown that the two criteria are similar under some specific margin assumptions in practice the tuning parameter of the cart penalty has to be calibrated by hold out or cross validation a simulation study is performed to compare the form of the theoretical penalized criterion we propose with the form obtained after tuning the regularization parameter via cross validation

approachability and fixed points for non convex set valued maps this paper studies the approachability of upper semicontinuous nonconvex set valued maps by single valued continuous functions and its applications to fixed point theory the use of approximate selections was initiated by j von neumann 41 in the proof of his well known minimax principle this remarkable property was extended to the normed space setting by cellina s and cellina and lasota 9 in the context of a degree theory for upper semicontinuous convex compact valued maps it also holds for maps with compact contractible values defined on finite polyhedra of iw ma cole11 40 and more generally defined on compact anrs mclennan 39 it was recently taken up by gbrniewicz granas and kryszewski 22 241 in the context of an index theory for non convex maps defined on compact anrs the content of this paper is divided into three parts the first part is devoted to background material concerning the types of spaces and set valued maps studied herein in the second part we define the abstract class d of approachable set valued maps that is the class of maps a x y

minimax analysis of active learning this work establishes distribution free upper and lower bounds on the minimax label complexity of active learning with general hypothesis classes under various noise models the results reveal a number of surprising facts in particular under the noise model of tsybakov 2004 the minimax label complexity of active learning with a vc class is always asymptotically smaller than that of passive learning and is typically signi cantly smaller than the best previously published upper bounds in the active learning literature in high noise regimes it turns out that all active learning problems of a given vc dimension have roughly the same minimax label complexity which contrasts with well known results for bounded noise in low noise regimes we find that the label complexity is well characterized by a simple combinatorial complexity measure we call the star number interestingly we find that almost all of the complexity measures previously explored in the active learning literature have worst case values exactly equal to the star number we also propose new active learning strategies that nearly achieve these minimax label complexities

risk bounds for statistical learning we propose a general theorem providing upper bounds for the risk of an empirical risk minimizer erm we essentially focus on the binary classification framework we extend tsybakov s analysis of the risk of an erm under margin type conditions by using concentration inequalities for conveniently weighted empirical processes this allows us to deal with ways of measuring the size of a class of classifiers other than entropy with bracketing as in tsybakov s work in particular we derive new risk bounds for the erm when the classification rules belong to some vc class under margin conditions and discuss the optimality of these bounds in a minimax sense

a direct method for constructing the traveling wave solutions of a modified generalized vakhnenko equation abstract proposing a direct method namely the traveling wave reduction combined with a smart variable transformation a series of traveling wave solutions are constructed for a modified generalized vakhnenko equation these solutions include the jacobi elliptic function solution the weierstrass elliptic function solution and the trigonometric function solution

a new lower bound for multiple hypothesis testing the purpose of this correspondence is to give a new easily tractable and sharp lower bound for the maximal error in multiple hypothesis testing with an application to nonasymptotic lower bounds for the minimax risk of estimators

a new method for estimation and model selection rho estimation the aim of this paper is to present a new estimation procedure that can be applied in various statistical frameworks including density and regression and which leads to both robust and optimal or nearly optimal estimators in density estimation they asymptotically coincide with the celebrated maximum likelihood estimators at least when the statistical model is regular enough and contains the true density to estimate for very general models of densities including non compact ones these estimators are robust with respect to the hellinger distance and converge at optimal rate up to a possible logarithmic factor in all cases we know in the regression setting our approach improves upon the classical least squares in many respects in simple linear regression for example it provides an estimation of the coefficients that are both robust to outliers and simultaneously rate optimal or nearly rate optimal for a large class of error distributions including gaussian laplace cauchy and uniform among others

concentration inequalities and asymptotic results for ratio type empirical processes let be a class of measurable functions on a measurable space s mathcal s with values in 0 1 and let pn n 1 i 1n xi be the empirical measure based on an i i d sample x1 xn from a probability distribution p on s mathcal s we study the behavior of suprema of the following type sup r n textless sigma p f leq delta n frac p n f pf phi sigma p f where pf var1 2pf and is a continuous strictly increasing function with 0 0 using talagrand s concentration inequality for empirical processes we establish concentration inequalities for such suprema and use them to derive several results about their asymptotic behavior expressing the conditions in terms of expectations of localized suprema of empirical processes we also prove new bounds for expected values of sup norms of empirical processes in terms of the largest pf and the l2 p norm of the envelope of the function class which are especially suited for estimating localized suprema with this technique we extend to function classes most of the known results on ratio type suprema of empirical processes including some of alexander s results for vc classes of sets we also consider applications of these results to several important problems in nonparametric statistics and in learning theory including general excess risk bounds in empirical risk minimization and their versions for l2 regression and classification and ratio type bounds for margin distributions in classification

optimal aggregation of classifiers in statistical learning classification can be considered as nonparametric estimation of sets where the risk is defined by means of a specific distance between sets associated with misclassification error it is shown that the rates of convergence of classifiers depend on two parameters the complexity of the class of candidate sets and the margin parameter the dependence is explicitly given indicating that optimal fast rates approaching o n 1 can be attained where n is the sample size and that the proposed classifiers have the property of robustness to the margin the main result of the paper concerns optimal aggregation of classifiers we suggest a classifier that automatically adapts both to the complexity and to the margin and attains the optimal fast rates up to a logarithmic factor

smooth discrimination analysis ties f and g can be based on the estimation of the set g x fx gx we consider applications where it is appropriate to assume that the region g has a smooth boundary or belongs to another nonparametric class of sets in particular this assumption makes sense if discrimination is used as a data analytic tool decision rules based on minimization of empirical risk over the whole class of sets and over sieves are considered their rates of convergence are obtained we show that these rules achieve optimal rates for estimation of g and optimal rates of convergence for bayes risks an interesting conclusion is that the optimal rates for bayes risks can be very fast in particular faster than the parametric root n rate these fast rates cannot be guaranteed for plug in rules

statistical performance of support vector machines the support vector machine svm algorithm is well known to the computer learning community for its very good practical results the goal of the present paper is to study this algorithm from a statistical perspective using tools of concentration theory and empirical processes our main result builds on the observation made by other authors that the svm can be viewed as a statistical regularization procedure from this point of view it can also be interpreted as a model selection principle using a penalized criterion it is then possible to adapt general methods related to model selection in this framework to study two important points 1 what is the minimum penalty and how does it compare to the penalty actually used in the svm algorithm 2 is it possible to obtain oracle inequalities in that setting for the specific loss function used in the svm algorithm we show that the answer to the latter question is positive and provides relevant insight to the former our result shows that it is possible to obtain fast rates of convergence for svms

adaptive estimation of the intensity of inhomogeneous poisson processes via concentration inequalities in this paper we establish oracle inequalities for penalized projection estimators of the intensity of an inhomogeneous poisson process we study consequently the adaptive properties of penalized projection estimators at first we provide lower bounds for the minimax risk over various sets of smoothness for the intensity and then we prove that our estimators achieve these lower bounds up to some constants the crucial tools to obtain the oracle inequalities are new concentration inequalities for suprema of integral functionals of poisson processes which are analogous to talagrand s inequalities for empirical processes

classification algorithms using adaptive partitioning algorithms for binary classification based on adaptive tree partitioning are formulated and analyzed for both their risk performance and their friendliness to numerical implementation the algorithms can be viewed as generating a set approximation to the bayes set and thus fall into the general category of set estimators in contrast with the most studied tree based algorithms which utilize piecewise constant approximation on the generated partition ieee trans inform theory52 2006 1335 1353 mach learn 66 2007 209 242 we consider decorated trees which allow us to derive higher order methods convergence rates for these methods are derived in terms the parameter alpha of margin conditions and a rate s of best approximation of the bayes set by decorated adaptive partitions they can also be expressed in terms of the besov smoothness beta of the regression function that governs its approximability by piecewise polynomials on adaptive partition the execution of the algorithms does not require knowledge of the smoothness or margin conditions besov smoothness conditions are weaker than the commonly used holder conditions which govern approximation by nonadaptive partitions and therefore for a given regression function can result in a higher rate of convergence this in turn mitigates the compatibility conflict between smoothness and margin parameters

new concentration inequalities in product spaces we introduce three new ways to measure the distance from a point to a subset of a product space and we prove corresponding concentration inequalities each of them allows to control the fluctuation of a new class of random variables

a further improved extended fan sub equation method for 2 1 dimensional breaking soliton equations in this paper a further improved extended fan sub equation method is used to construct exact solutions of the 2 1 dimensional breaking soliton equations as a result many new and more general non travelling wave and coefficient function solutions are obtained including soliton like solutions triangular like solutions single and combined non degenerate jacobi elliptic wave function like solutions weierstrass elliptic doubly like periodic solutions each of which contains an arbitrary function of two variables the results show the proposed method give new and more general exact solutions more importantly the method with the aid of symbolic computation provides a very effective and powerful mathematical tool for solving a great many non linear partial differential equations in mathematical physics

rates of convergence in active learning we study the rates of convergence in generalization error achievable by active learning under various types of label noise additionally we study the general problem of model selection for active learning with a nested hierarchy of hypothesis classes and propose an algorithm whose error rate provably converges to the best achievable error among classifiers in the hierarchy at a rate adaptive to both the complexity of the optimal classifier and the noise conditions in particular we state sufficient conditions for these rates to be dramatically faster than those achievable by passive learning

rademacher complexities and bounding the excess risk in active learning sequential algorithms of active learning based on the estimation of the level sets of the empirical risk are discussed in the paper localized rademacher complexities are used in the algorithms to estimate the sample sizes needed to achieve the required accuracy of learning in an adaptive way probabilistic bounds on the number of active examples have been proved and several applications to binary classification problems are considered

ranking the best instances we formulate a local form of the bipartite ranking problem where the goal is to focus on the best instances we propose a methodology based on the construction of real valued scoring functions we study empirical risk minimization of dedicated statistics which involve empirical quantiles of the scores we first state the problem of finding the best instances which can be cast as a classification problem with mass constraint next we develop special performance measures for the local ranking problem which extend the area under an roc curve auc criterion and describe the optimal elements of these new criteria we also highlight the fact that the goal of ranking the best instances cannot be achieved in a stage wise manner where first the best instances would be tentatively identified and then a standard auc criterion could be applied eventually we state preliminary statistical results for the local ranking problem

fast rates in statistical and online learning the speed with which a learning algorithm converges as it is presented with more data is a central problem in machine learning a fast rate of convergence means less data is needed for the same level of performance the pursuit of fast rates in online and statistical learning has led to the discovery of many conditions in learning theory under which fast learning is possible we show that most of these conditions are special cases of a single unifying condition that comes in two forms the central condition for proper learning algorithms that always output a hypothesis in the given model and stochastic mixability for online algorithms that may make predictions outside of the model we show that under surprisingly weak assumptions both conditions are in a certain sense equivalent the central condition has a re interpretation in terms of convexity of a set of pseudoprobabilities linking it to density estimation under misspecification for bounded losses we show how the central condition enables a direct proof of fast rates and we prove its equivalence to the bernstein condition itself a generalization of the tsybakov margin condition both of which have played a central role in obtaining fast rates in statistical learning yet while the bernstein condition is two sided the central condition is one sided making it more suitable to deal with unbounded losses in its stochastic mixability form our condition generalizes both a stochastic exp concavity condition identified by juditsky rigollet and tsybakov and vovk s notion of mixability our unifying conditions thus provide a substantial step towards a characterization of fast rates in statistical learning similar to how classical mixability characterizes constant regret in the sequential prediction with expert advice setting

nonasymptotic bounds for vector quantization in hilbert spaces recent results in quantization theory show that the convergence rate for the mean squared expected distortion of the empirical risk minimizer strategy for any fixed probability distribution satisfying some regularity conditions is o 1 n where n is the sample size however the dependency of the average distortion on other parameters is not known this paper offers more general conditions which may be thought of as margin conditions under which a sharp upper bound on the expected distortion rate of the empirically optimal quantizer is derived this upper bound is also proved to be sharp with respect to the dependency of the distortion on other natural parameters of the quantization issue

finite dimensional projection for classification and statistical learning in this paper a new method for the binary classification problem is studied it relies on empirical minimization of the hinge risk over an increasing sequence of finite dimensional spaces a suitable dimension is picked by minimizing the regularized risk where the regularization term is proportional to the dimension an oracle type inequality is established for the excess generalization risk i e regret to bayes of the procedure which ensures adequate convergence properties of the method we suggest to select the considered sequence of subspaces by applying kernel principal components analysis kpca in this case the asymptotical convergence rate of the method can be better than what is known for the support vector machine svm exemplary experiments are presented on benchmark data sets where the practical results of the method are comparable to the svm

localization of vc classes beyond local rademacher complexities abstract in statistical learning the excess risk of empirical risk minimization erm is controlled by comp n f n where n is a size of a learning sample comp n f is a complexity term associated with a given class f and 1 2 1 interpolates between slow and fast learning rates in this paper we introduce an alternative localization approach for binary classification that leads to a novel complexity measure fixed points of the local empirical entropy we show that this complexity measure gives a tight control over comp n f in the upper bounds under bounded noise our results are accompanied by a minimax lower bound that involves the same quantity in particular we practically answer the question of optimality of erm under bounded noise for general vc classes

risk bounds for model selection via penalization performance bounds for criteria for model selection are devel oped using recent theory for sieves the model selection criteria are based on an empirical loss or contrast function with an added penalty term moti vated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of ob servations most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function we show that the quadratic risk of the minimum penal ized empirical contrast estimatoris bounded by an index of the accuracy of the sieve this accuracy index quanties the trade off among the candidate models between the approximation error and parameter dimension relative to sample size if we choose a list of models which exhibit good approximation prop erties with respect to different classes of smoothness the estimator can be simultaneously minimax rate optimal in each of those classes this is what is usually called adaptation the type of classes of smoothness in which one gets adaptation depends heavily on the list of models if too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously it may happen that the estimator is only approx

the extended g g expansion method and its applications to the whitham broer kaup like equations and coupled hirota satsuma kdv equations in this paper based on a new general ansatze the extended g g expansion method is proposed to seek exact solutions of nonlinear evolution equations being concise and straightforward this method is applied to construct travelling wave solutions of whitham broer kaup like equations and coupled hirota satsuma kdv equations by using this method new exact solutions involving parameters expressed by three types of functions which are the hyperbolic functions the trigonometric functions and the rational functions are obtained when the parameters are taken as special values some solitary wave solutions are derived from the hyperbolic function solutions

symbolic computation and new families of exact non travelling wave solutions to 3 1 dimensional kadomstev petviashvili equation abstract in this paper the improved tanh function method in f d xie y zhang z s lu chaos solitons fractals 24 2005 257 is further improved by constructing new ansatz solution of the considered equation as its application the 3 1 dimensional kadomstev petviashvili equation is considered and abundant new exact non travelling wave solutions are obtained

discussion local rademacher complexities and oracle inequalities in risk minimization in this magnificent paper professor koltchinskii offers general and powerful performance bounds for empirical risk minimization a fundamental principle of statistical learning theory since the elegant pioneering work of vapnik and chervonenkis in the early 1970s various such bounds have been known that relate the performance of empirical risk minimizers to combinatorial and geometrical features of the class over which the minimization is performed this area of research has been a rich source of motivation and a major field of applications of empirical process theory the appearance of advanced concentration inequalities in the 1990s primarily thanks to talagrand s influential work provoked major advances in both empirical process theory and statistical learning theory and led to a much deeper understanding of some of the basic phenomena in the discussed paper professor koltchinskii develops a powerful new methodology iterative localization which with the help of concentration inequalities is able to explain most of the recent results and go significantly beyond them in many cases the main motivation behind professor koltchinskii s paper is based on classical problems of statistical learning theory such as binary classification and regression in which given a sample xi y i i 1 n of independent and identically distributed pairs of random variables where the xi take their values in some feature space x and the yi are say real valued the goal is to find a function f x r whose risk defined in terms of the expected value of an appropriately chosen loss function is as small as possible in the remaining part of this discussion we point out how the performance bounds of professor koltchinskii s paper can be used to study a seemingly different model motivated by nonparametric ranking problems which has received increasing attention both in the statistical and machine learning literature indeed in several applications such as the search engine problem or credit risk screening the goal is to learn how to rank or to score observations rather than just classify them in this case performance measures involve pairs of observations as can be seen for instance with the auc area under an roc curve criterion in this

on the rate of convergence of local averaging plug in classification rules under a margin condition the rates of convergence of plug in kernel partitioning and nearest neighbors classification rules are analyzed a margin condition which measures how quickly the a posteriori probabilities cross the decision boundary smoothness conditions on the a posteriori probabilities and boundedness of the feature vector are imposed the rates of convergence of the plug in classifiers shown in this paper are faster than previously known

the improved g g expansion method and its applications to the broer kaup equations and approximate long water wave equations by introducing a new general ansatze the improved g g expansion method is proposed to construct exact solutions of both broer kaup equations and approximate long water wave equations as a result some new travelling wave solutions involving parameters expressed by three types of functions which are the hyperbolic functions the trigonometric functions and the rational functions are obtained when the parameters are taken as special values the solitary wave solutions are derived from the hyperbolic function solutions the proposed method is straightforward concise and effective and can be applied to other nonlinear evolution equations in mathematical physics

generalized multivalued quasi variational inequalities in this paper we introduce and study a new class of variational inequalities known as generalized multivalued quasi variational inequalities this class is the most general and unifying one using essentially the projection technique we establish the equivalence between this class and the implicit wiener hopf equations this equivalence is used to suggest a number of iterative algorithms for solving the generalized multivalued quasi variational inequalities the convergence analysis of these algorithms is also studied the results proved in this paper represent a significant improvement and refinement of the previously known results in this area

a generalized g g expansion method and its application to the 2 1 dimensional broer kaup equations abstract in this paper a generalized g g expansion method is used to seek more general exact solutions of the 2 1 dimensional broer kaup equations as a result non travelling wave solutions with three arbitrary functions are obtained including hyperbolic function solutions trigonometric function solutions and rational solutions the proposed method is more powerful than wang et al s method in m l wang x z li j l zhang the g g expansion method and travelling wave solutions of nonlinear evolution equations in mathematical physics phys lett a 372 2008 417 423 and it can be used for many other nonlinear evolution equations in mathematical physics

fast learning rates for plug in classifiers it has been recently shown that under the margin or low noise assumption there exist classifiers attaining fast rates of convergence of the excess bayes risk that is rates faster than n 1 2 the work on this subject has suggested the following two conjectures i the best achievable fast rate is of the order n 1 and ii the plug in classifiers generally converge more slowly than the classifiers based on empirical risk minimization we show that both conjectures are not correct in particular we construct plug in classifiers that can achieve not only fast but also super fast rates that is rates faster than n 1 we establish minimax lower bounds showing that the obtained rates cannot be improved

risk bounds for cart classifiers under a margin condition non asymptotic risk bounds for classification and regression trees cart classifiers are obtained in the binary supervised classification framework under a margin assumption on the joint distribution of the covariates and the labels these risk bounds are derived conditionally on the construction of the maximal binary tree and allow to prove that the linear penalty used in the cart pruning algorithm is valid under the margin condition it is also shown that conditionally on the construction of the maximal tree the final selection by test sample does not alter dramatically the estimation accuracy of the bayes classifier

the compound g g expansion method and double non traveling wave solutions of 2 1 dimensional nonlinear partial differential equations abstract to seek the exact double non traveling wave solutions of nonlinear partial differential equations the compound g g expansion method is firstly proposed in this paper with the aid of symbolic computation this new method is applied to construct double non traveling wave solutions of 2 1 dimensional painleve integrable burgers equation and 2 1 dimensional breaking soliton equation as a result abundant double non traveling wave solutions including double hyperbolic function solutions double trigonometric function solutions double rational solutions and a series of complexiton solutions of these two equations are obtained via the proposed method these exact solutions contain arbitrary functions which may be helpful to explain some complex phenomena when the parameters are taken as special values the double solitary like wave solutions can be derived from double hyperbolic function solutions furthermore the time evolutions of double solitary like wave solutions are discussed in detail

spaces and maps approximation and fixed points abstract a general approximation property for topological spaces is studied in relation with fixed point theory for set valued maps a particular instance of this property is the admissibility in the sense of klee examples of convex sets of topological spaces equipped with a local topological convexity structure as well as general classes of approximative neighborhood retracts are shown to have this approximation property a general topological principle on the preservation of the fixed point property under this space approximation is proved it allows the passage from basic classes of spaces to more elaborate ones for general classes of nonconvex set valued maps

a series of abundant exact travelling wave solutions for a modified generalized vakhnenko equation using auxiliary equation method in this paper a series of abundant exact travelling wave solutions is established for a modified generalized vakhnenko equation by using auxiliary equation method these solutions can be expressed by jacobi elliptic function when jacobi elliptic functions modulus m 1 or 0 the travelling wave solutions degenerate to four types of solutions namely the soliton solutions the hyperbolic function solutions the trigonometric function solutions constant solutions

lower bounds in pattern recognition and learning lower bounds are derived for the performance of any pattern recognition algorithm which using training data selects a discrimination rule from a certain class of rules the bounds involve the vapnik chervonenkis dimension of the class and l the minimal error probability within the class we provide lower bounds when l 0 the usual assumption in valiant s theory of learning and l 0

ranking and empirical minimization of u statistics the problem of ranking ordering instances instead of simply classifying them has recently gained much attention in machine learning in this paper we formulate the ranking problem in a rigorous statistical framework the goal is to learn a ranking rule for deciding among two instances which one is better with minimum ranking risk since the natural estimates of the risk are of the form of a u statistic results of the theory of u processes are required for investigating the consistency of empirical risk minimizers we establish in particular a tail inequality for degenerate u processes and apply it for showing that fast rates of convergence may be achieved under specific noise assumptions just like in classification convex risk minimization methods are also studied

auxiliary equation method for the mkdv equation with variable coefficients abstract by using solutions of an ordinary differential equation an auxiliary equation method is described to seek exact solutions of nonlinear evolution equations with variable coefficients being concise and straightforward this method is applied to the mkdv equation with variable coefficients as a result new explicit solutions including solitary wave solutions and trigonometric function solutions are obtained with the aid of symbolic computation

a high dimensional wilks phenomenon a theorem by wilks asserts that in smooth parametric density estimation the difference between the maximum likelihood and the likelihood of the sampling distribution converges toward a chi square distribution where the number of degrees of freedom coincides with the model dimension this observation is at the core of some goodness of fit testing procedures and of some classical model selection methods this paper describes a non asymptotic version of the wilks phenomenon in bounded contrast optimization procedures using concentration inequalities for general functions of independent random variables it proves that in bounded contrast minimization as for example in statistical learning theory the difference between the empirical risk of the minimizer of the true risk in the model and the minimum of the empirical risk the excess empirical risk satisfies a bernstein like inequality where the variance term reflects the dimension of the model and the scale term reflects the noise conditions from a mathematical statistics viewpoint the significance of this result comes from the recent observation that when using model selection via penalization the excess empirical risk represents a minimum penalty if non asymptotic guarantees concerning prediction error are to be provided from the perspective of empirical process theory this paper describes a concentration inequality for the supremum of a bounded non centered actually non positive empirical process combining the now classical analysis of m estimation building on talagrand s inequality for suprema of empirical processes and versatile moment inequalities for functions of independent random variables this paper develops a genuine bernstein like inequality that seems beyond the reach of traditional tools

predicting 0 1 functions on randomly drawn points we consider the problem of predicting 0 1 valued functions on rn and smaller domains based on their values on randomly drawn points our model is related to valiant s pac learning model but does not require the hypotheses used for prediction to be represented in any specified form in our main result we show how to construct prediction strategies that are optimal to within a constant factor for any reasonable class f of target functions this result is based on new combinatorial results about classes of functions of finite vc dimension we also discuss more computationally efficient algorithms for predicting indicator functions of axis parallel rectangles more general intersection closed concept classes and halfspaces in rn these are also optimal to within a constant factor finally we compare the general performance of prediction strategies derived by our method to that of those derived from methods in pac learning theory

simultaneous adaptation to the margin and to complexity in classification we consider the problem of adaptation to the margin and to complexity in binary classification we suggest a learning method with a numerically easy aggregation step adaptivity both to the margin and complexity in classification usually involves empirical risk minimization or rademacher complexities which lead to numerical difficulties on the other hand there exist classifiers that are easy to compute and that converge with fast rates but are not adaptive combining these classifiers by our aggregation procedure we get numerically realizable adaptive classifiers that converge with fast rates