high dimensional nonconvex stochastic optimization by doubly stochastic successive convex approximation in this paper we consider supervised learning problems over training sets in which the number of training examples and the dimension of feature vectors are both large we focus on the case where the loss function defining the quality of the parameter we wish to estimate may be non convex but also has a convex regularization we propose a doubly stochastic successive convex approximation scheme dssc able to handle non convex regularized expected risk minimization the method operates by decomposing the decision variable into blocks and operating on random subsets of blocks at each step fusing the merits of stochastic approximation with block coordinate methods and then implements successive convex approximation in contrast to many stochastic convex methods whose almost sure behavior is not guaranteed in non convex settings dssc attains almost sure convergence to a stationary solution of the problem moreover we show that the proposed dssc algorithm achieves stationarity at a rate of mathcal o log t t 1 4 numerical experiments on a non convex variant of a lasso regression problem show that dssc performs favorably in this setting we then apply this method to the task of dictionary learning from high dimensional visual data collected from a ground robot and observe reliable convergence behavior for a difficult non convex stochastic program

scale sensitive dimensions uniform convergence and learnability learnability in valiant s pac learning model has been shown to be strongly related to the existence of uniform laws of large numbers these laws define a distribution free convergence property of means to expectations uniformly over classes of random variables classes of real valued functions enjoying such a property are also known as uniform glivenko cantelli classes in this paper we prove through a generalization of sauer s lemma that may be interesting in its own right a new characterization of uniform glivenko cantelli classes our characterization yields dudley gine and zinn s previous characterization as a corollary furthermore it is the first based on a gine and zinn s previous characterization as a corollary furthermore it is the first based on a simple combinatorial quantity generalizing the vapnik chervonenkis dimension we apply this result to obtain the weakest combinatorial condition known to imply pac learnability in the statistical regression or agnostic framework furthermore we find a characterization of learnability in the probabilistic concept model solving an open problem posed by kearns and schapire these results show that the accuracy parameter plays a crucial role in determining the effective complexity of the learner s hypothesis class

multiclass learnability and the erm principle we study the sample complexity of multiclass prediction in several learning settings for the pac setting our analysis reveals a surprising phenomenon in sharp contrast to binary classification we show that there exist multiclass hypothesis classes for which some empirical risk minimizers erm learners have lower sample complexity than others furthermore there are classes that are learnable by some erm learners while other erm learners will fail to learn them we propose a principle for designing good erm learners and use this principle to prove tight bounds on the sample complexity of learning symmetric multiclass hypothesis classes classes that are invariant under permutations of label names we further provide a characterization of mistake and regret bounds for multiclass learning in the online setting and the bandit setting using new generalizations of littlestone s dimension

composite media and asymptotic dirichlet forms we introduce a metric topology on the space of dirichlet forms and study compactness and closure properties of families of local and non local forms

reflected diffusion processes with jumps a stochastic differential equation of wiener poisson type is considered in a d dimensional bounded region by using a penalization argument on the domain we are able to prove the existence and uniqueness of solutions in the strong sense the main assumptions are lipschitzian coefficients either convex or smooth domains and a regular outward reflecting direction as a direct consequence it is verified that the reflected diffusion process with jumps depends on the initial date in a lipschitz fashion

distribution free prediction bands for non parametric regression summary we study distribution free non parametric prediction bands with a focus on their finite sample behaviour first we investigate and develop different notions of finite sample coverage guarantees then we give a new prediction band by combining the idea of conformal prediction with non parametric conditional density estimation the proposed estimator called cops conformal optimized prediction set always has a finite sample guarantee under regularity conditions the estimator converges to an oracle band at a minimax optimal rate a fast approximation algorithm and a data driven method for selecting the bandwidth are developed the method is illustrated in simulated and real data examples

learnability stability and uniform convergence the problem of characterizing learnability is the most basic question of statistical learning theory a fundamental and long standing answer at least for the case of supervised classification and regression is that learnability is equivalent to uniform convergence of the empirical risk to the population risk and that if a problem is learnable it is learnable via empirical risk minimization in this paper we consider the general learning setting introduced by vapnik which includes most statistical learning problems as special cases we show that in this setting there are non trivial learning problems where uniform convergence does not hold empirical risk minimization fails and yet they are learnable using alternative mechanisms instead of uniform convergence we identify stability as the key necessary and sufficient condition for learnability moreover we show that the conditions for learnability in the general setting are significantly more complex than in supervised classification and regression

on the generalization ability of on line learning algorithms in this paper it is shown how to extract a hypothesis with small risk from the ensemble of hypotheses generated by an arbitrary on line learning algorithm run on an independent and identically distributed i i d sample of data using a simple large deviation argument we prove tight data dependent bounds for the risk of this hypothesis in terms of an easily computable statistic m sub n associated with the on line performance of the ensemble via sharp pointwise bounds on m sub n we then obtain risk tail bounds for kernel perceptron algorithms in terms of the spectrum of the empirical kernel matrix these bounds reveal that the linear hypotheses found via our approach achieve optimal tradeoffs between hinge loss and margin size over the class of all linear functions an issue that was left open by previous results a distinctive feature of our approach is that the key tools for our analysis come from the model of prediction of individual sequences i e a model making no probabilistic assumptions on the source generating the data in fact these tools turn out to be so powerful that we only need very elementary statistical facts to obtain our final risk bounds

a direct formulation for sparse pca using semidefinite programming given a covariance matrix we consider the problem of maximizing the variance explained by a particular linear combination of the input variables while constraining the number of nonzero coefficients in this combination this problem arises in the decomposition of a covariance matrix into sparse factors or sparse principal component analysis pca and has wide applications ranging from biology to finance we use a modification of the classical variational representation of the largest eigenvalue of a symmetric matrix where cardinality is constrained and derive a semidefinite programming based relaxation for our problem we also discuss nesterov s smooth minimization technique applied to the semidefinite program arising in the semidefinite relaxation of the sparse pca problem the method has complexity o n 4 sqrt log n epsilon where n is the size of the underlying covariance matrix and epsilon is the desired absolute accuracy on the optimal value of the problem

on nonparametric confidence intervals an inequality is given for the expected length of a confidence interval given that a particular distribution generated the data and assuming that the confidence interval has a given coverage probability over a family of distributions as a corollary attempts to adapt to the regularity of the true density within derivative smoothness classes cannot improve the rate of convergence of the length of the confidence interval over minimax fixed length intervals and still maintain uniform coverage probability however adaptive confidence intervals can attain improved rates of convergence in some other classes of densities such as those satisfying a shape restriction

weak convergence of symmetric diffusion processes in this paper we show the convergence of forms in the sense of mosco associated with the part form on relatively compact open set of dirichlet forms with locally uniform ellipticity and the locally uniform boundedness of ground states under regular dirichlet space setting we also get the same assertion under dirichlet space in infinite dimensional setting as a result of this we get the weak convergence under some conditions on initial distributions and the growth order of the volume of the balls defined by modified pseudo metric used in k th sturm

the solution path of the generalized lasso we present a path algorithm for the generalized lasso problem this problem penalizes the l1 norm of a matrix d times the coefficient vector and has a wide range of applications dictated by the choice of d our algorithm is based on solving the dual of the generalized lasso which greatly facilitates computation of the path for d i the usual lasso we draw a connection between our approach and the well known lars algorithm for an arbitrary d we derive an unbiased estimate of the degrees of freedom of the generalized lasso fit this estimate turns out to be quite intuitive in many applications

mosco convergence of dirichlet forms in infinite dimensions with changing reference measures let e be an infinite dimensional locally convex space let n be a weakly convergent sequence of probability measures on e and let en be a sequence of dirichlet forms on e such that en is defined on l2 n general sufficient conditions for mosco convergence of the gradient dirichlet forms are obtained applications to gibbs states on a lattice and to the gaussian case are given weak convergence of the associated processes is discussed

prediction and tolerance intervals with transformation and or weighting we consider estimation of quantiles and construction of prediction and tolerance intervals for a new response following a possibly nonlinear regression fit with transformation and or weighting we consider the case of normally distributed errors and to a lesser extent the nonparametric case in which the error distribution is unknown quantile estimation here follows standard theory although we introduce a simple computational device for likelihood ratio testing and confidence intervals prediction and tolerance intervals are somewhat more difficult to obtain we show that the effect of estimating parameters when constructing tolerance intervals can be expected to be greater than the effect in the prediction problem improved prediction and tolerance intervals are constructed based on resampling techniques in the tolerance interval case a simple analytical correction is introduced we apply these methods to the prediction of automobile stopping distances and salmon production using respectively a he

on neumann type problems for nonlocal equations set in a half space we study neumann type boundary value problems for nonlocal equations related to levy processes since these equations are nonlocal neumann type problems can be obtained in many ways depending on the kind of reflection we impose on the outside jumps to focus on the new phenomenas and ideas we consider different models of reflection and rather general non symmetric levy measures but only simple linear equations in half space domains we derive the neumann reflection problems through a truncation procedure on the levy measure and then we develop a viscosity solution theory which includes comparison existence and some regularity results for problems involving fractional laplacian type operators like e g delta alpha 2 we prove that solutions of all our nonlocal neumann problems converge as alpha goes to 2 to the solution of a classical neumann problem the reflection models we consider include cases where the underlying levy processes are reflected projected and or censored upon exiting the domain

mosco convergence of nonlocal to local quadratic forms abstract we study sequences of nonlocal quadratic forms and function spaces that are related to markov jump processes in bounded domains with a lipschitz boundary our aim is to show the convergence of these forms to local quadratic forms of gradient type under suitable conditions we establish the convergence in the sense of mosco our framework allows bounded and unbounded nonlocal operators to be studied at the same time moreover we prove that smooth functions with compact support are dense in the nonlocal function spaces under consideration

structure of indicator function classes with finite vapnik chervonenkis dimensions the vapnik chervonenkis vc dimension is used to measure the complexity of a function class and plays an important role in a variety of fields including artificial neural networks and machine learning one major concern is the relationship between the vc dimension and inherent characteristics of the corresponding function class according to sauer s lemma if the vc dimension of an indicator function class f is equal to d the cardinality of the set fs1n will not be larger than d 0dcnd therefore there naturally arises a question about the vc dimension of an indicator function class what kinds of elements will be contained in the function class f if f has a finite vc dimension in this brief we answer the above question first we investigate the structure of the function class f when the cardinality of the set fs1n reaches the maximum value d 0dcnd based on the derived result we then figure out what kinds of elements will be contained in f if f has a finite vc dimension

the dirichlet problem for nonlocal operators in this note we set up the elliptic and the parabolic dirichlet problem for linear nonlocal operators as opposed to the classical case of second order differential operators here the boundary data are prescribed on the complement of a given bounded set we formulate the problem in the classical framework of hilbert spaces and prove unique solvability using standard techniques like the fredholm alternative

on line predictive linear regression gauss linear model independent identically distributed observations multivariate analysis on line protocol prequential statistics regression we consider the on line predictive version of the standard problem of linear regression the goal is to predict each consecutive response given the corresponding explanatory variables and all the previous observations the standard treatment of prediction in linear regression analysis has two drawbacks 1 the usual prediction intervals guarantee that the probability of error is equal to the nominal significance level but this property per se does not imply that the long run frequency of error is close to 2 it is not suitable for prediction of complex systems as it assumes that the number of observations exceeds the number of parameters we state a general result showing that in the on line protocol the frequency of error does equal the nominal significance level up to statistical fluctuations and we describe alternative regression models in which informative prediction intervals can be found before the number of observations exceeds the number of parameters one of these models which only assumes that the observations are independent and identically distributed is popular in machine learning but greatly underused in the statistical theory of regression

nonlocal diffusion and peridynamic models with neumann type constraints and their numerical approximations abstract this paper studies nonlocal diffusion models associated with a finite nonlocal horizon parameter that characterizes the range of nonlocal interactions the focus is on the variational formulation associated with neumann type constraints and its numerical approximations we establish the well posedness for some variational problems associated and study their local limit as 0 a main contribution is to derive a second order convergence to the local limit we then discuss the numerical approximations including standard finite element methods and quadrature based finite difference methods we study their convergence in the nonlocal setting and in the local limit

k vectors an alternating minimization algorithm for learning regression functions the k vectors algorithm for learning regression functions proposed here is akin to the well known k means algorithm both algorithms partition the feature space but unlike the k means algorithm the k vectors algorithm aims to reconstruct the response rather than the feature the partitioning rule of the algorithm is based on maximizing the correlation inner product of the feature vector with a set of k vectors and generates polyhedral cells similar to the ones generated by the nearest neighbor rule of the k means algorithm similarly to k means the learning algorithm alternates between two types of steps in the first type of steps k labels are determined via a centroid type rule in the response space which uses a surrogate hinge type loss function to the mean squared error loss function in the second type of steps the k vectors which determine the partition are updated according to a multiclass classification rule in the spirit of support vector machines it is proved that both steps of the algorithm only require solving convex optimization problems and that the algorithm is empirically consistent as the length of the training sequence increases to infinity fixed points of the empirical version of the algorithm tend to fixed points of the population version of the algorithm learnability of the predictor class posit by the algorithm is also established

stability of density based clustering high density clusters can be characterized by the connected components of a level set l x p x of the underlying probability density function p generating the data at some appropriate level 0 the complete hierarchical clustering can be characterized by a cluster tree t l in this paper we study the behavior of a density level set estimate l and cluster tree estimate t based on a kernel density estimator with kernel bandwidth h we define two notions of instability to measure the variability of l and t as a function of h and investigate the theoretical properties of these instability measures

learnability and the vapnik chervonenkis dimension valiant s learnability model is extended to learning classes of concepts defined by regions in euclidean space e n the methods in this paper lead to a unified treatment of some of valiant s results along with previous results on distribution free convergence of certain pattern recognition algorithms it is shown that the essential condition for distribution free learnability is finiteness of the vapnik chervonenkis dimension a simple combinatorial parameter of the class of concepts to be learned using this parameter the complexity and closure properties of learnable classes are analyzed and the necessary and sufficient conditions are provided for feasible learnability

discretized vapnik chervonenkis dimension for analyzing complexity of real function classes in this paper we introduce the discretized vapnik chervonenkis vc dimension for studying the complexity of a real function class and then analyze properties of real function classes and neural networks we first prove that a countable traversal set is enough to achieve the vc dimension for a real function class whereas its classical definition states that the traversal set is the output range of the function class based on this result we propose the discretized vc dimension defined by using a countable traversal set consisting of rational numbers in the range of a real function class by using the discretized vc dimension we show that if a real function class has a finite vc dimension only a finite traversal set is needed to achieve the vc dimension we then point out that the real function classes which have the infinite vc dimension can be grouped into two categories type a and type b subsequently based on the obtained results we discuss the relationship between the vc dimension of an indicator output network and that of the real output network when both networks have the same structure except for the output activation functions finally we present the risk bound based on the discretized vc dimension for a real function class that has infinite vc dimension and is of type a we prove that with such a function class the empirical risk minimization erm principle for the function class is still consistent with overwhelming probability this is a development of the existing knowledge that the erm learning is consistent if and only if the function class has a finite vc dimension

non local dirichlet forms and symmetric jump processes we consider the non local symmetric dirichlet form e f given by with f the closure with respect to e 1 of the set of c 1 functions on r d with compact support where e 1 f f e f f f rd f x 2 dx and where the jump kernel j satisfies for 0 2 x y 1 this assumption allows the corresponding jump process to have jump intensities whose sizes depend on the position of the process and the direction of the jump we prove upper and lower estimates on the heat kernel we construct a strong markov process corresponding to e f we prove a parabolic harnack inequality for non negative functions that solve the heat equation with respect to e finally we construct an example where the corresponding harmonic functions need not be continuous

stability and generalization we define notions of stability for learning algorithms and show how to use these notions to derive generalization error bounds based on the empirical error and the leave one out error the methods we use can be applied in the regression framework as well as in the classification one when the classifier is obtained by thresholding a real valued function we study the stability properties of large classes of learning algorithms such as regularization based algorithms in particular we focus on hilbert space regularization and kullback leibler regularization we demonstrate how to apply the results to svm for regression and classification

function spaces and extension results for nonlocal dirichlet problems abstract we study function spaces and extension results in relation with dirichlet problems involving integrodifferential operators for such problems data are prescribed on the complement of a given domain r d we introduce a function space that serves as a trace space for nonlocal dirichlet problems and study related extension results

on nonparametric estimation of density level sets let x 1 x n be independent identically distributed observations from an unknown probability density f consider the problem of estimating the level set g g f x r 2 f x from the sample x 1 x n under the assumption that the boundary of g has a certain smoothness we propose piecewise polynomial estimators of g based on the maximization of local empirical excess masses we show that the estimators have optimal rates of convergence in the asymptotically minimax sense within the studied classes of densities we find also the optimal convergence rates for estimation of convex level sets a generalization to the n dimensional case where n 2 is given

a coverage theory for least squares summary a sensible use of an estimation method requires that assessment criteria for the quality of the estimate be available we present a coverage theory for the least squares estimate by suitably modifying the empirical costs one constructs statistics that are guaranteed to cover with known probability the cost associated with a next still unseen member of the population all results of this paper are distribution free and can be applied to least squares problems in use across a variety of fields

variance function estimation abstract heteroscedastic regression models are used in fields including economics engineering and the biological and physical sciences often the heteroscedasticity is modeled as a function of the covariates or the regression and other structural parameters standard asymptotic theory implies that how one estimates the variance function in particular the structural parameters has no effect on the first order properties of the regression parameter estimates there is evidence however both in practice and higher order theory to suggest that how one estimates the variance function does matter further in some settings estimation of the variance function is of independent interest or plays an important role in estimation of other quantities in this article we study variance function estimation in a unified way focusing on common methods proposed in the statistical and other literature to make both general observations and compare different estimation schemes we show that there are significant di

the selection of prior distributions by formal rules abstract subjectivism has become the dominant philosophical foundation for bayesian inference yet in practice most bayesian analyses are performed with so called noninformative priors that is priors constructed by some formal rule we review the plethora of techniques for constructing such priors and discuss some of the practical and philosophical issues that arise when they are used we give special emphasis to jeffreys s rules and discuss the evolution of his viewpoint about the interpretation of priors away from unique representation of ignorance toward the notion that they should be chosen by convention we conclude that the problems raised by the research on priors chosen by formal rules are serious and may not be dismissed lightly when sample sizes are small relative to the number of parameters being estimated it is dangerous to put faith in any default solution but when asymptotics take over jeffreys s rules and their variants remain reasonable choices we also provide an annotated b

penalized local polynomial regression for spatial data when performing spatial regression analysis in environmental data applications spatial heterogeneity in the regression coefficients is often observed spatially varying coefficient models including geographically weighted regression and spline models are standard tools for quantifying such heterogeneity in this paper we propose a spatially varying coefficient model that represents the spatially varying parameters as a mixture of local polynomials at selected locations the local polynomial parameters have attractive interpretations indicating various types of spatial heterogeneity instead of estimating the spatially varying regression coefficients directly we develop a penalized least squares regression procedure for the local polynomial parameter estimation which both shrinks the parameter estimation and penalizes the differences among parameters that are associated with neighboring locations we develop confidence intervals for the varying regression coefficients and prediction intervals for the response we apply the proposed method to characterize the spatially varying association between particulate matter concentrations pm2 5 and pollutant gases related to the secondary aerosol formulation in china the identified regression coefficients show distinct spatial patterns for nitrogen dioxide sulfur dioxide and carbon monoxide during different seasons

localized functional principal component analysis we propose localized functional principal component analysis lfpca looking for orthogonal basis functions with localized support regions that explain most of the variability of a random process the lfpca is formulated as a convex optimization problem through a novel deflated fantope localization method and is implemented through an efficient algorithm to obtain the global optimum we prove that the proposed lfpca converges to the original functional principal component analysis fpca when the tuning parameters are chosen appropriately simulation shows that the proposed lfpca with tuning parameters chosen by cross validation can almost perfectly recover the true eigenfunctions and significantly improve the estimation accuracy when the eigenfunctions are truly supported on some subdomains in the scenario that the original eigenfunctions are not localized the proposed lfpca also serves as a nice tool in finding orthogonal basis functions that balance between interpretability and the capability of exp

algorithmic stability and sanity check bounds for leave one out cross validation in this article we prove sanity check bounds for the error of the leave one out cross validation estimate of the generalization error that is bounds showing that the worst case error of this estimate is not much worse than that of the training error estimate the name sanity check refers to the fact that although we often expect the leave one out estimate to perform considerably better than the training error estimate we are here only seeking assurance that its performance will not be considerably worse perhaps surprisingly such assurance has been given only for limited cases in the prior literature on cross validation any nontrivial bound on the error of leave one out must rely on some notion of algorithmic stability previous bounds relied on the rather strong notion of hypothesis stability whose application was primarily limited to nearest neighbor and other local algorithms here we introduce the new and weaker notion of error stability and apply it to obtain sanity check bounds for leave one out for other classes of learning algorithms including training error minimization procedures and bayesian algorithms we also provide lower bounds demonstrating the necessity of some form of error stability for proving bounds on the error of the leave one out estimate and the fact that for training error minimization algorithms in the worst case such bounds must still depend on the vapnik chervonenkis dimension of the hypothesis class

distribution free prediction sets this article introduces a new approach to prediction by bringing together two different nonparametric ideas distribution free inference and nonparametric smoothing specifically we consider the problem of constructing nonparametric tolerance prediction sets we start from the general conformal prediction approach and we use a kernel density estimator as a measure of agreement between a sample point and the underlying distribution the resulting prediction set is shown to be closely related to plug in density level sets with carefully chosen cutoff values under standard smoothness conditions we get an asymptotic efficiency result that is near optimal for a wide range of function classes but the coverage is guaranteed whether or not the smoothness conditions hold and regardless of the sample size the performance of our method is investigated through simulation studies and illustrated in a real data example

principal eigenvalue of mixed problem for the fractional laplacian moving the boundary conditions abstract we analyze the behavior of the eigenvalues of the following nonlocal mixed problem s u 1 d u in u 0 in d n s u 0 in n our goal is to construct different sequences of problems by modifying the configuration of the sets d and n and to provide sufficient and necessary conditions on the size and the location of these sets in order to obtain sequences of eigenvalues that in the limit recover the eigenvalues of the dirichlet or neumann problem we will see that the nonlocality plays a crucial role here since the sets d and n can have infinite measure a phenomenon that does not appear in the local case see for example 6 7 8

adaptive confidence bands we show that there do not exist adaptive confidence bands for curve estimation except under very restrictive assumptions we propose instead to construct adaptive bands that cover a surrogate function f which is close to but simpler than f the surrogate captures the significant features in f we establish lower bounds on the width for any confidence band for f and construct a procedure that comes within a small constant factor of attaining the lower bound for finite samples

fast learning rates for plug in classifiers it has been recently shown that under the margin or low noise assumption there exist classifiers attaining fast rates of convergence of the excess bayes risk that is rates faster than n 1 2 the work on this subject has suggested the following two conjectures i the best achievable fast rate is of the order n 1 and ii the plug in classifiers generally converge more slowly than the classifiers based on empirical risk minimization we show that both conjectures are not correct in particular we construct plug in classifiers that can achieve not only fast but also super fast rates that is rates faster than n 1 we establish minimax lower bounds showing that the obtained rates cannot be improved

exact minimax estimation of the predictive density in sparse gaussian models we consider estimating the predictive density under kullback leibler loss in an l0 sparse gaussian sequence model explicit expressions of the first order minimax risk along with its exact constant asymptotically least favorable priors and optimal predictive density estimates are derived compared to the sparse recovery results involving point estimation of the normal mean new decision theoretic phenomena are seen suboptimal performance of the class of plug in density estimates reflects the predictive nature of the problem and optimal strategies need diversification of the future risk we find that minimax optimal strategies lie outside the gaussian family but can be constructed with threshold predictive density estimates novel minimax techniques involving simultaneous calibration of the sparsity adjustment and the risk diversification mechanisms are used to design optimal predictive density estimates

measuring mass concentrations and estimating density contour clusters an excess mass approach by using empirical process theory the so called excess mass approach is studied it can be applied to various statistical problems especially in higher dimensions such as testing for multimodality estimating density contour clusters estimating nonlinear functionals of a density density estimation regression problems and spectral analysis we mainly consider the problems of testing for multimodality and estimating density contour clusters but the other problems also are discussed the excess mass over mathbb c is defined as a supremum of a certain functional defined on mathbb c where mathbb c is a class of subsets of the d dimensional euclidean space comparing excess masses over different classes mathbb c yields information about the modality of the underlying probability measure f this can be used to construct tests for multimodality if f has a density f the maximizing sets of the excess mass are level sets or density contour clusters of f provided they lie in mathbb c the excess mass and the density contour clusters can be estimated from the data asymptotic properties of these estimators and of the test statistics are studied for general classes mathbb c including the classes of balls ellipsoids and convex sets

decision theoretic generalizations of the pac model for neural net and other learning applications we describe a generalization of the pac learning model that is based on statistical decision theory in this model the learner receives randomly drawn examples each example consisting of an instance x in x and an outcome y in y and tries to find a hypothesis h x a where h in h that specifies the appropriate action a in a to take for each instance x in order to minimize the expectation of a loss l y a here x y and a are arbitrary sets l is a real valued function and examples are generated according to an arbitrary joint distribution on x times y special cases include the problem of learning a function from x into y the problem of learning the conditional probability distribution on y given x regression and the problem of learning a distribution on x density estimation we give theorems on the uniform convergence of empirical loss estimates to true expected loss rates for certain hypothesis spaces h and show how this implies learnability with bounded sample size disregarding computational complexity as an application we give distribution independent upper bounds on the sample size needed for learning with feedforward neural networks our theorems use a generalized notion of vc dimension that applies to classes of real valued functions adapted from pollard s work and a notion of capacity and metric dimension for classes of functions that map into a bounded metric space supersedes 89 30 and 90 52 also in information and computation vol 100 no 1 september 1992

censored stable processes we present several constructions of a censored stable process in an open set d rn i e a symmetric stable process which is not allowed to jump outside d we address the question of whether the process will approach the boundary of d in a finite time we give sharp conditions for such approach in terms of the stability index and the thickness of the boundary as a corollary new results are obtained concerning besov spaces on non smooth domains including the critical exponent case we also study the decay rate of the corresponding harmonic functions which vanish on a part of the boundary we derive a boundary harnack principle in c1 1 open sets

algorithm dependent generalization bounds for multi task learning often tasks are collected for multi task learning mtl because they share similar feature structures based on this observation in this paper we present novel algorithm dependent generalization bounds for mtl by exploiting the notion of algorithmic stability we focus on the performance of one particular task and the average performance over multiple tasks by analyzing the generalization ability of a common parameter that is shared in mtl when focusing on one particular task with the help of a mild assumption on the feature structures we interpret the function of the other tasks as a regularizer that produces a specific inductive bias the algorithm for learning the common parameter as well as the predictor is thereby uniformly stable with respect to the domain of the particular task and has a generalization bound with a fast convergence rate of order mathcal o 1 n where n is the sample size of the particular task when focusing on the average performance over multiple tasks we prove that a similar inductive bias exists under certain conditions on the feature structures thus the corresponding algorithm for learning the common parameter is also uniformly stable with respect to the domains of the multiple tasks and its generalization bound is of the order mathcal o 1 t where t is the number of tasks these theoretical analyses naturally show that the similarity of feature structures in mtl will lead to specific regularizations for predicting which enables the learning algorithms to generalize fast and correctly from a few examples

improving coverage accuracy of nonparametric prediction intervals summary methods are suggested for improving the coverage accuracy of intervals for predicting future values of a random variable drawn from a sampled distribution it is shown that properties of solutions to such problems may be quite unexpected for example the bootstrap and the jackknife perform very poorly when used to calibrate coverage although the jackknife estimator of the true coverage is virtually unbiased a version of the smoothed bootstrap can be employed for successful calibration however interpolation among adjacent order statistics can also be an effective way of calibrating although even there the results are unexpected in particular whereas the coverage error can be reduced from o n 1 to orders o n 2 and o n 3 where n denotes the sample size by interpolating among two and three order statistics respectively the next two orders of reduction require interpolation among five and eight order statistics respectively

discrete approximation of symmetric jump processes on metric measure spaces in this paper we give general criteria on tightness and weak convergence of discrete markov chains to symmetric jump processes on metric measure spaces under mild conditions as an application we investigate discrete approximation for a large class of symmetric jump processes we also discuss some application of our results to the scaling limit of random walk in random conductance

boundary fluxes for nonlocal diffusion abstract we study a nonlocal diffusion operator in a bounded smooth domain prescribing the flux through the boundary this problem may be seen as a generalization of the usual neumann problem for the heat equation first we prove existence uniqueness and a comparison principle next we study the behavior of solutions for some prescribed boundary data including blowing up ones finally we look at a nonlinear flux boundary condition