on the galerkin and collocation methods for a cauchy singular integral equation a convergence analysis of the galerkin and collocation methods is given for a scalar cauchy singular integral equation of index zero on the arc 1 1 when the inner products and integral transforms are approximated by interpolatory quadrature rules

on consistent nonparametric order determination and chaos summary we give a brief introduction to deterministic chaos and a link between chaotic deterministic models and stochastic time series models we argue that it is often natural to determine the embedding dimension in a noisy environment first in any systematic study of chaos setting the stochastic models within the framework of non linear autoregression we introduce the notion of a generalized partial autocorrelation and an order we approach the estimation of the embedding dimension via order determination of an unknown non linear autoregression by cross validation and give justification by proving its consistency under global boundedness as a by product we provide a theoretical justification of the final prediction error approach of auestad and tj0stheim some illustrations based on the henon map and several real data sets are given the bias of the residual sum of squares as essentially a noise variance estimator is quantified

flexible discriminant analysis by optimal scoring abstract fisher s linear discriminant analysis is a valuable tool for multigroup classification with a large number of predictors one can find a reduced number of discriminant coordinate functions that are optimal for separating the groups with two such functions one can produce a classification map that partitions the reduced space into regions that are identified with group membership and the decision boundaries are linear this article is about richer nonlinear classification schemes linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups in this paper we obtain nonparametric versions of discriminant analysis by replacing linear regression by any nonparametric regression method in this way any multiresponse regression technique such as mars or neural networks can be postprocessed to improve its classification performance

the ii p method for estimating multivariate functions from noisy data the method for estimating an underlying smooth function of m variables x l xm using noisy data is based on approximating it by a sum of products of the form m m x m the problem is then reduced to estimating the univariate functions in the products a convergent algorithm is described the method keeps tight control on the degrees of freedom used in the fit many examples are given the quality of fit given by the method is excellent usually only a few products are enough to fit even fairly complicated functions the coding into products of univariate functions allows a relatively understandable interpretation of the multivariate fit

tree structured classification via generalized discriminant analysis abstract the problem of constructing classification rules that can be represented as decision trees is considered each object to be classified has an associated x vector containing possibly incomplete covariate information tree construction is based on the information provided in a learning sample of objects with known class identities the x vectors in the learning sample may have missing values as well procedures are proposed for each of the components of classifier construction such as split selection tree size determination treatment of missing values and ranking of variables the main idea is recursive application of linear discriminant analysis with the variables at each stage being appropriately chosen according to the data and the type of splits desired standard statistical techniques used as basic building blocks include analysis of variance linear and canonical discriminant analysis and principal component analysis a new method of tree structured classification is obtained by assem

optimal partitioning for classification and regression trees an iterative algorithm that finds a locally optimal partition for an arbitrary loss function in time linear in n for each iteration is presented the algorithm is a k means like clustering algorithm that uses as its distance measure a generalization of kullback s information divergence moreover it is proven that the globally optimal partition must satisfy a nearest neighbour condition using divergence as the distance measure these results generalize similar results of l breiman et al 1984 to an arbitrary number of classes or regression variables and to an arbitrary number of bills experimental results on a text to speech example are provided and additional applications of the algorithm including the design of variable combinations surrogate splits composite nodes and decision graphs are suggested

regularized discriminant analysis abstract linear and quadratic discriminant analysis are considered in the small sample high dimensional setting alternatives to the usual maximum likelihood plug in estimates for the covariance matrices are proposed these alternatives are characterized by two parameters the values of which are customized to individual situations by jointly minimizing a sample based estimate of future misclassification risk computationally fast implementations are presented and the efficacy of the approach is examined through simulation studies and application to data these studies indicate that in many circumstances dramatic gains in classification accuracy can be achieved

asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects global minimization via monte carlo the asymptotic behavior of the systems x n 1 x n a n b x n xi n a n sigma x n psi n and dy bar b y dt sqrt a t sigma y dw is studied where psi n is i i d gaussian xi n is a correlated bounded sequence of random variables and a n approx a 0 log a 1 n without xi n such algorithms are versions of the simulated annealing method for global optimization when the objective function values can only be sampled via monte carlo the discrete algorithm is a combination of stochastic approximation and simulated annealing our forms are appropriate the psi n are the annealing variables and xi n is the sampling noise for large a 0 a full asymptotic analysis is presented via the theory of large deviations mean escape time after arbitrary time n from neighborhoods of stable sets of the algorithm mean transition times after arbitrary time n from a neighborhood of one stable set to another a

on visualization and aggregation of nearest neighbor classifiers nearest neighbor classification is one of the simplest and most popular methods for statistical pattern recognition a major issue in k nearest neighbor classification is how to find an optimal value of the neighborhood parameter k in practice this value is generally estimated by the method of cross validation however the ideal value of k in a classification problem not only depends on the entire data set but also on the specific observation to be classified instead of using any single value of k this paper studies results for a finite sequence of classifiers indexed by k along with the usual posterior probability estimates a new measure called the bayesian measure of strength is proposed and investigated in this paper as a measure of evidence for different classes the results of these classifiers and their corresponding estimated misclassification probabilities are visually displayed using shaded strips these plots provide an effective visualization of the evidence in favor of different classes when a given data point is to be classified we also propose a simple weighted averaging technique that aggregates the results of different nearest neighbor classifiers to arrive at the final decision based on the analysis of several benchmark data sets the proposed method is found to be better than using a single value of k

new error estimates for galerkin method to an airfoil equation in this paper we consider new error estimates under the weighted chebyshev norm for galerkin method to an airfoil equation galerkin discretisation is discussed the smoothness conditions of the input functions are improved i e they need to be holder continuous with 12 m 1

hierarchical mixtures of experts and the em algorithm we present a tree structured architecture for supervised learning the statistical model underlying the architecture is a hierarchical mixture model in which both the mixture coefficients and the mixture components are generalized linear models glim s learning is treated as a maximum likelihood problem in particular we present an expectation maximization em algorithm for adjusting the parameters of the architecture we also develop an on line learning algorithm in which the parameters are updated incrementally comparative simulation results are presented in the robot dynamics domain

modified quadratic discriminant functions and the application to chinese character recognition issues in the quadratic discriminant functions qdf are discussed and two types of modified quadratic disriminant functions mqdf1 mqdf2 which are less sensitive to the estimation error of the covariance matrices are proposed the mqdf1 is a function which employs a kind of a pseudo bayesian estimate of the covariance matrix instead of the maximum likelihood estimate ordinarily used in the qdf the mqdf2 is a variation of the mqdf1 to save the required computation time and storage two discriminant functions were applied to chinese character recognition to evaluate their effectiveness and remarkable improvement was observed in their performance

model selection and the principle of minimum description length this article reviews the principle of minimum description length mdl for problems of model selection by viewing statistical modeling as a means of generating descriptions of observed data the mdl framework discriminates between competing models based on the complexity of each description this approach began with kolmogorov s theory of algorithmic complexity matured in the literature on information theory and has recently received renewed attention within the statistics community here we review both the practical and the theoretical aspects of mdl as a tool for model selection emphasizing the rich connections between information theory and statistics at the boundary between these two disciplines we find many interesting interpretations of popular frequentist and bayesian procedures as we show mdl provides an objective umbrella under which rather disparate approaches to statistical modeling can coexist and be compared we illustrate the mdl principle by considering problems in regression nonpar

improved wavelet based watermarking through pixel wise masking a watermarking algorithm operating in the wavelet domain is presented performance improvement with respect to existing algorithms is obtained by means of a new approach to mask the watermark according to the characteristics of the human visual system hvs in contrast to conventional methods operating in the wavelet domain masking is accomplished pixel by pixel by taking into account the texture and the luminance content of all the image subbands the watermark consists of a pseudorandom sequence which is adaptively added to the largest detail bands as usual the watermark is detected by computing the correlation between the watermarked coefficients and the watermarking code and the detection threshold is chosen in such a way that the knowledge of the watermark energy used in the embedding phase is not needed thus permitting one to adapt it to the image at hand experimental results and comparisons with other techniques operating in the wavelet domain prove the effectiveness of the new algorithm

improving the generalization properties of radial basis function neural networks an important feature of radial basis function neural networks is the existence of a fast linear learning algorithm in a network capable of representing complex nonlinear mappings satisfactory generalization in these networks requires that the network mapping be sufficiently smooth we show that a modification to the error functional allows smoothing to be introduced explicitly without significantly affecting the speed of training a simple example is used to demonstrate the resulting improvement in the generalization properties of the network

a simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training a general convergence criterion for certain iterative sequences in hilbert space is presented for an important subclass of these sequences estimates of the rate of convergence are given under very mild assumptions these results establish an 0 1 f4n nonsampling convergence rate for projection pursuit regression and neural network training where n represents the number of ridge functions neurons or coefficients in a greedy basis expansion

likelihood inference in nearest neighbour classification models summary traditionally the neighbourhood size k in the k nearest neighbour algorithm is either fixed at the first nearest neighbour or is selected on the basis of a crossvalidation study in this paper we present an alternative approach that develops the k nearest neighbour algorithm using likelihood based inference our method takes the form of a generalised linear regression on a set of k nearest neighbour autocovariates by defining the k nearestneighbour algorithm in this way we are able to extend the method to accommodate the original predictor variables as possible linear effects as well as allowing for the inclusion of multiple nearest neighbour terms the choice of the final model proceeds via a stepwise regression procedure it is shown that our method incorporates a conventional generalised linear model and a conventional k nearest neighbour algorithm as special cases empirical results suggest that the method out performs the standard k nearest neighbour method in terms of misclassification rate on a wide variety of datasets

stochastic relaxation gibbs distributions and the bayesian restoration of images we make an analogy between images and statistical mechanics systems pixel gray levels and the presence and orientation of edges are viewed as states of atoms or molecules in a lattice like physical system the assignment of an energy function in the physical system determines its gibbs distribution because of the gibbs distribution markov random field mrf equivalence this assignment also determines an mrf image model the energy function is a more convenient and natural mechanism for embodying picture attributes than are the local characteristics of the mrf for a range of degradation mechanisms including blurring nonlinear deformations and multiplicative or additive noise the posterior distribution is an mrf with a structure akin to the image model by the analogy the posterior distribution defines another imaginary physical system gradual temperature reduction in the physical system isolates low energy states annealing or what is the same thing the most probable states under the gibbs distribution the analogous operation under the posterior distribution yields the maximum a posteriori map estimate of the image given the degraded observations the result is a highly parallel relaxation algorithm for map estimation we establish convergence properties of the algorithm and we experiment with some simple pictures for which good restorations are obtained at low signal to noise ratios

flexible parsimonious smoothing and additive modeling a simple method is presented for fitting regression models that are nonlinear in the explanatory variables despite its simplicity or perhaps because of it the method has some powerful characteristics that cause it to be competitive with and often superior to more sophisticated techniques especially for small data sets in the presence of high noise

a bayesian reassessment of nearest neighbor classification the k nearest neighbor knn procedure is a well known deterministic method used in supervised classification this article proposes a reassessment of this approach as a statistical technique derived from a proper probabilistic model in particular we modify the assessment found in holmes and adams and evaluated by manocha and girolami where the underlying probabilistic model is not completely well defined once provided with a clear probabilistic basis for the knn procedure we derive computational tools for bayesian inference on the parameters of the corresponding model in particular we assess the difficulties inherent to both pseudo likelihood and path sampling approximations of an intractable normalizing constant we implement a correct mcmc sampler based on perfect sampling when perfect sampling is not available we use instead a gibbs sampling approximation illustrations of the performance of the corresponding bayesian classifier are provided for benchmark datasets demonstrating in particula

how tight are the vapnik chervonenkis bounds we describe a series of numerical experiments that measure the average generalization capability of neural networks trained on a variety of simple functions these experiments are designed to test the relationship between average generalization performance and the worst case bounds obtained from formal learning theory using the vapnik chervonenkis vc dimension blumer et al 1989 haussler et al 1990 recent statistical learning theories tishby et al 1989 schwartz et al 1990 suggest that surpassing these bounds might be possible if the spectrum of possible generalizations has a gap near perfect performance we indeed find that in some cases the average generalization is significantly better than the vc bound the approach to perfect performance is exponential in the number of examples m rather than the 1 m result of the bound however in these cases we have not found evidence of the gap predicted by the above statistical theories in other cases we do find the 1 m behavior of the vc bound

projection based approximation and a duality with kernel methods projection pursuit regression and kernel regression are methods for estimating a smooth function of several variables from noisy data obtained at scattered sites methods based on local averaging can perform poorly in high dimensions curse of dimensionality intuition and examples have suggested that projection based approaches can provide better fits for what sorts of regression functions is this true when and by how much do projection methods reduce the curse of dimensionality we make a start by focusing on the two dimensional problem and study the l 2 approximation error bias of the two procedures with respect to gaussian measure let ra stand for a certain ppr type approximation and ka for a particular kernel type approximation building on a simple but striking duality for polynomials we show that ra behaves significantly better than the minimax rate of approximation for radial functions while ka performs significantly better than the minimax rate for harmonic functions in fact the rate improvements carry over to large classes ra behaving very well for functions with enough angular smoothness oscillating slowly with angle while ka behaves very well for functions with enough laplacian smoothness oscillations averaging out locally the rate improvements matter they are equivalent to lowering the dimensionality of the problem for example for functions with nice tail behavior ra behaves as if the dimensionality of the problem were 1 5 rather than its nominal value 2 also ra and ka are complementary for a given function if one method offers a dimensionality reduction the other does not

on the interpretation of regression plots abstract a framework is developed for the interpretation of regression plots including plots of the response against selected covariates residual plots added variable plots and detrended added variable plots it is shown that many of the common interpretations associated with these plots can be misleading the framework also allows for the generalization of standard plots and the development of new plotting paradigms a paradigm for graphical exploration of regression problems is sketched

on the statistical analysis of dirty pictures may 7th 1986 professor a f m smith in the chair summary a continuous two dimensional region is partitioned into a fine rectangular array of sites or pixels each pixel having a particular colour belonging to a prescribed finite set the true colouring of the region is unknown but associated with each pixel there is a possibly multivariate record which conveys imperfect information about its colour according to a known statistical model the aim is to reconstruct the true scene with the additional knowledge that pixels close together tend to have the same or similar colours in this paper it is assumed that the local characteristics of the true scene can be represented by a nondegenerate markov random field such information can be combined with the records by bayes theorem and the true scene can be estimated according to standard criteria however the computational burden is enormous and the reconstruction may reflect undesirable largescale properties of the random field thus a simple iterative method of reconstruction is proposed which does not depend on these large scale characteristics the method is illustrated by computer simulations in which the original scene is not directly related to the assumed random field some complications including parameter estimation are discussed potential applications are mentioned briefly

modelling complexity applications of gibbs sampling in medicine summary we review applications of gibbs sampling in medicine involving longitudinal spatial covariate measurement and survival models applications in immunology pharmacology transplantation cancer screening industrial epidemiology and genetic epidemiology are discussed

on nonparametric discrimination using density differences summary we propose a technique for nonparametric discrimination in which smoothing parameters are chosen jointly according to a criterion based on the difference between two densities the approach is suitable for categorical continuous and mixed data and uses information from both populations to determine the smoothing parameter for any one population in the case of categorical data optimal performance is sometimes achieved using negative smoothing parameters a property which does not emerge if the smoothing parameters are chosen individually

an image adaptive wavelet based watermarking of digital images in digital management multimedia content and data can easily be used in an illegal way being copied modified and distributed again copyright protection intellectual and material rights protection for authors owners buyers distributors and the authenticity of content are crucial factors in solving an urgent and real problem in such scenario digital watermark techniques are emerging as a valid solution in this paper we describe an algorithm called wm2 0 for an invisible watermark private strong wavelet based and developed for digital images protection and authenticity using discrete wavelet transform dwt is motivated by good time frequency features and well matching with human visual system directives these two combined elements are important in building an invisible and robust watermark wm2 0 works on a dual scheme watermark embedding and watermark detection the watermark is embedded into high frequency dwt components of a specific sub image and it is calculated in correlation with the image features and statistic properties watermark detection applies a re synchronization between the original and watermarked image the correlation between the watermarked dwt coefficients and the watermark signal is calculated according to the neyman pearson statistic criterion experimentation on a large set of different images has shown to be resistant against geometric filtering and stirmark attacks with a low rate of false alarm

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

a note on a superconvergence result for the generalized airfoil equation we obtain convergence results and error estimates for calculating linear functionals of solutions of a class of cauchy singular integral equations via galerkin s method these results generalize the superconvergence estimates obtained by miel 7 and involve extending the class of functionals relaxing the smoothness conditions imposed and determining the effects of quadrature errors in the numerical implementation of this method in particular it is shown that full superconvergence can be restored by evaluating the integrals sufficiently accurately

a new approach to pre processing digital image for wavelet based watermark the growth of the internet has increased the phenomenon of digital piracy in multimedia objects like software image video audio and text therefore it is strategic to individualize and to develop methods and numerical algorithms which are stable and have low computational cost that will allow us to find a solution to these problems we describe a digital watermarking algorithm for color image protection and authenticity robust not blind and wavelet based the use of discrete wavelet transform is motivated by good time frequency features and a good match with human visual system directives these two combined elements are important for building an invisible and robust watermark moreover our algorithm can work with any image thanks to the step of pre processing of the image that includes resize techniques that adapt to the size of the original image for wavelet transform the watermark signal is calculated in correlation with the image features and statistic properties in the detection step we apply a re synchronization between the original and watermarked image according to the neyman pearson statistic criterion experimentation on a large set of different images has been shown to be resistant against geometric filtering and stirmark attacks with a low rate of false alarm

classification based on hybridization of parametric and nonparametric classifiers parametric methods of classification assume specific parametric models for competing population densities e g gaussian population densities can lead to linear and quadratic discriminant analysis and they work well when these model assumptions are valid violation in one or more of these parametric model assumptions often leads to a poor classifier on the other hand nonparametric classifiers e g nearest neighbor and kernel based classifiers are more flexible and free from parametric model assumptions but the statistical instability of these classifiers may lead to poor performance when we have small numbers of training sample observations nonparametric methods however do not use any parametric structure of population densities therefore even when one has some additional information about population densities that important information is not used to modify the nonparametric classification rule this paper makes an attempt to overcome these limitations of parametric and nonparametric approaches and combines their strengths to develop some hybrid classification methods we use some simulated examples and benchmark data sets to examine the performance of these hybrid discriminant analysis tools asymptotic results on their misclassification rates have been derived under appropriate regularity conditions

decision theoretic generalizations of the pac model for neural net and other learning applications we describe a generalization of the pac learning model that is based on statistical decision theory in this model the learner receives randomly drawn examples each example consisting of an instance x in x and an outcome y in y and tries to find a hypothesis h x a where h in h that specifies the appropriate action a in a to take for each instance x in order to minimize the expectation of a loss l y a here x y and a are arbitrary sets l is a real valued function and examples are generated according to an arbitrary joint distribution on x times y special cases include the problem of learning a function from x into y the problem of learning the conditional probability distribution on y given x regression and the problem of learning a distribution on x density estimation we give theorems on the uniform convergence of empirical loss estimates to true expected loss rates for certain hypothesis spaces h and show how this implies learnability with bounded sample size disregarding computational complexity as an application we give distribution independent upper bounds on the sample size needed for learning with feedforward neural networks our theorems use a generalized notion of vc dimension that applies to classes of real valued functions adapted from pollard s work and a notion of capacity and metric dimension for classes of functions that map into a bounded metric space supersedes 89 30 and 90 52 also in information and computation vol 100 no 1 september 1992

an iterative growing and pruning algorithm for classification tree design a critical issue in classification tree design obtaining right sized trees i e trees which neither underfit nor overfit the data is addressed instead of stopping rules to halt partitioning the approach of growing a large tree with pure terminal nodes and selectively pruning it back is used a new efficient iterative method is proposed to grow and prune classification trees this method divides the data sample into two subsets and iteratively grows a tree with one subset and prunes it with the other subset successively interchanging the roles of the two subsets the convergence and other properties of the algorithm are established theoretical and practical considerations suggest that the iterative free growing and pruning algorithm should perform better and require less computation than other widely used tree growing and pruning algorithms numerical results on a waveform recognition problem are presented to support this view

a probabilistic nearest neighbour method for statistical pattern recognition summary nearest neighbour algorithms are among the most popular methods used in statistical pattern recognition the models are conceptually simple and empirical studies have shown that their performance is highly competitive against other techniques however the lack of a formal framework for choosing the size of the neighbourhood k is problematic furthermore the method can only make discrete predictions by reporting the relative frequency of the classes in the neighbourhood of the prediction point we present a probabilistic framework for the k nearest neighbour method that largely overcomes these difficulties uncertainty is accommodated via a prior distribution on k as well as in the strength of the interaction between neighbours these prior distributions propagate uncertainty through to proper probabilistic predictions that have continuous support on 0 1 the method makes no assumptions about the distribution of the predictor variables the method is also fully automatic with no user set parameters and empirically it proves to be highly accurate on many bench mark data sets

logistic regression diagnostics a maximum likelihood fit of a logistic regression model and other similar models is extremely sensitive to outlying responses and extreme points in the design space we develop diagnostic measures to aid the analyst in detecting such observations and in quantifying their effect on various aspects of the maximum likelihood fit the elements of the fitting process which constitute the usual output parameter estimates standard errors residuals etc will be used for this purpose with a properly designed computing package for fitting the usual maximum likelihood model the diagnostics are essentially free for the asking in particular good data analysis for logistic regression models need not be expensive or time consuming

penalized discriminant analysis fisher s linear discriminant analysis lda is a popular data analytic tool for studying the relationship between a set of predictors and a categorical response in this paper we describe a penalized version of lda it is designed for situations in which there are many highly correlated predictors such as those obtained by discretizing a function or the grey scale values of the pixels in a series of images in cases such as these it is natural efficient and sometimes essential to impose a spatial smoothness constraint on the coefficients both for improved prediction performance and interpretability we cast the classification problem into a regression framework via optimal scoring using this our proposal facilitates the use of any penalized regression technique in the classification setting the technique is illustrated with examples in speech recognition and handwritten character recognition

resistant fits for some commonly used logistic models with medical application logistic regression type models are used in many applications some examples include the classical dose response experiment prospective and retrospective studies of disease incidence with and without matching and the analysis of ordinal data in most instances the model is fitted by the method of maximum likelihood which like least squares is sensitive to atypical observations an alternative to maximum likelihood is proposed and illustrated by examples

density estimation with confidence sets exemplified by superclusters and voids in the galaxies abstract a method is presented for forming both a point estimate and a confidence set of semiparametric densities the final product is a three dimensional figure that displays a selection of density estimates for a plausible range of smoothing parameters the boundaries of the smoothing parameter are determined by a nonparametric goodness of fit test that is based on the sample spacings for each value of the smoothing parameter our estimator is selected by choosing the normal mixture that maximizes a function of the sample spacings a point estimate is selected from this confidence set by using the method of cross validation an algorithm to find the mixing distribution that maximizes the spacings functional is presented these methods are illustrated with a data set from the astronomy literature the measurements are velocities at which galaxies in the corona borealis region are moving away from our galaxy if the galaxies are clustered the velocity density will be multimodal with clusters correspond

on robustness in the logistic regression model summary we investigate robustness in the logistic regression model copas has studied two forms of robust estimator a robust resistant estimate of pregibon and an estimate based on a misclassification model he concluded that robust resistant estimates are much more biased in small samples than the usual logistic estimate is and recommends a bias corrected version of the misclassification estimate we show that there are other versions of robust resistant estimates which have bias often approximately the same as and sometimes even less than the logistic estimate these estimates belong to the mallows class in addition the corrected misclassification estimate is inconsistent at the logistic model we develop a simple consistent modification the modified estimate is a member of the mallows class but unlike most robust estimates it has an interpretable tuning constant the results are illustrated on data sets featuring different kinds of outliers

on a conjecture of huber concerning the convergence of projection pursuit regression we generalize the projection pursuit procedure of friedman and stuetzle abstract version and prove strong convergence this answers a question of huber

statistical theory of learning curves under entropic loss criterion the present paper elucidates a universal property of learning curves which shows how the generalization error training error and the complexity of the underlying stochastic machine are related and how the behavior of a stochastic machine is improved as the number of training examples increases the error is measured by the entropic loss it is proved that the generalization error converges to h0 the entropy of the conditional distribution of the true machine as h0 m 2t while the training error converges as h0 m 2t where t is the number of examples and m shows the complexity of the network when the model is faithful implying that the true machine is in the model m is reduced to m the number of modifiable parameters this is a universal law because it holds for any regular machine irrespective of its structure under the maximum likelihood estimator similar relations are obtained for the bayes and gibbs learning algorithms these learning curves show the relation among the accuracy of learning the complexity of a model and the number of training examples

superconvergence phenomena in the finite element method abstract we give a brief survey of superconvergence phenomena in finding a numerical solution of differential equations by finite elements several new results and open problems are introduced