matrix momentum for practical natural gradient learning an on line learning rule based on the introduction of a matrix momentum term is presented aimed at alleviating the computational costs of standard natural gradient learning the new rule natural gradient matrix momentum is analysed in the case of two layer feed forward neural network learning via methods of statistical physics it appears to provide a practical algorithm that performs as well as standard natural gradient descent in both the transient and asymptotic regimes but with a hugely reduced complexity

homogeneous asymptotic limits of haar measures of semisimple linear groups and their lattices we show that haar measures of connected semisimple groups embedded via a representation into a matrix space have a homogeneous asymptotic limit when viewed from far away and appropriately rescaled this is still true if the haar measure of the semisimple group is replaced by the haar measure of a irreducible lattice of the group and the asymptotic measure is the same in the case of an almost simple group of rank greater than 2 a remainder term is also obtained this extends and makes precise anterior results of duke rudnick and sarnak and eskin mcmullen in the case of a group variety

balanced line bundles on fano varieties a conjecture of batyrev and manin relates arithmetic properties of varieties with ample anticanonical class to geometric invariants in particular counting functions defined by metrized ample line bundles and the corresponding asymptotics of rational points of bounded height are interpreted in terms of cones of effective divisors and certain thresholds with respect to these cones this framework leads to the notion of balanced line bundles whose counting functions conjecturally capture generic distributions of rational points we investigate balanced line bundles in the context of the minimal model program with special regard to the classification of fano threefolds

varieties with too many rational points we investigate fano varieties defined over a number field that contain subvarieties whose number of rational points of bounded height is comparable to the total number on the variety

prime fano threefolds and integrable systems for a general k3 surface s of genus g with 2 g 10 we prove that the intermediate jacobians of the family of prime fano threefolds of genus g containing s as a hyperplane section form generically an algebraic completely integrable hamiltonian system

height uniformity for integral points on elliptic curves we recall the result of d abramovich and its generalization by p pacelli on the uniformity for stably integral points on elliptic curves it says that the lang vojta conjecture on the distribution of integral points on a variety of logarithmic general type implies the uniformity for the numbers of stably integral points on elliptic curves in this paper we will investigate its analogue for their heights under the assumption of the vojta conjecture basically we will show that the vojta conjecture gives a naturally expected simple uniformity for their heights

inhomogeneous cubic congruences and rational points on del pezzo surfaces for given non zero integers a b q we investigate the density of solutions x y 2 z 2 to the binary cubic congruence ax 2 by 3 0 mod q and use it to establish the manin conjecture for a singular del pezzo surface of degree 2 dened over q

nombre de points de hauteur bornee sur les surfaces de del pezzo de degre 5 we state manin s conjecture in the particular case of the split del pezzo s surfaces of degree 5 over q we show that for an open set u v nu q b card p u q h p b c b log b 4 b the constant c is the one conjectured by e peyre sommaire

how initial conditions affect generalization performance in large networks generalization is one of the most important problems in neural network research it is influenced by several factors in the network design such as network size weight decay factor and others we show here that the initial weight distribution for gradient decent training algorithms is one other factor that influences generalization the initial conditions guide the training algorithm to search particular places of the weight space for instance small initial weights tend to result in low complexity networks and therefore can effectively act as a regularization factor we propose a novel network complexity measure which is helpful in shedding insight into the phenomenon as well as in studying other aspects of generalization

generalization errors of the simple perceptron to find an exact form for the generalization error of a learning machine is an open problem even in the simplest case simple perceptron learning we introduce a new approach to tackle the problem the generalization error of the simple perceptron is expressed as a linear combination of extreme values of inputs with the help of extreme value theory in statistics we then obtain an exact form of the generalization error of the simple perceptron in the case of the worst learning generalization errors of the higher order perceptron taking the form of an inverse power law in the number of examples are also considered

asymptotic statistical theory of overtraining and cross validation a statistical theory for overtraining is proposed the analysis treats general realizable stochastic neural networks trained with kullback leibler divergence in the asymptotic case of a large number of training examples it is shown that the asymptotic gain in the generalization error is small if we perform early stopping even if we have access to the optimal stopping time based on the cross validation stopping we consider the ratio the examples should be divided into training and cross validation sets in order to obtain the optimum performance although cross validated early stopping is useless in the asymptotic region it surely decreases the generalization error in the nonasymptotic region our large scale simulations done on a cm5 are in good agreement with our analytical findings

linear growth for châtelet surfaces an upper bound of the expected order of magnitude is established for the number of mathbb q rational points of bounded height on ch telet surfaces defined over mathbb q

on the distribution of points of bounded height on equivariant compactifications of vector groups we prove asymptotic formulas for the number of rational points of bounded height on smooth equivariant compactifications of the affine space

singularities affect dynamics of learning in neuromanifolds the parameter spaces of hierarchical systems such as multilayer perceptrons include singularities due to the symmetry and degeneration of hidden units a parameter space forms a geometrical manifold called the neuromanifold in the case of neural networks such a model is identified with a statistical model and a riemannian metric is given by the fisher information matrix however the matrix degenerates at singularities such a singular structure is ubiquitous not only in multilayer perceptrons but also in the gaussian mixture probability densities arma time series model and many other cases the standard statistical paradigm of the cramer rao theorem does not hold and the singularity gives rise to strange behaviors in parameter estimation hypothesis testing bayesian inference model selection and in particular the dynamics of learning from examples prevailing theories so far have not paid much attention to the problem caused by singularity relying only on ordinary statistical theories developed for regular nonsingular models only recently have researchers remarked on the effects of singularity and theories are now being developed this article gives an overview of the phenomena caused by the singularities of statistical manifolds related to multilayer perceptrons and gaussian mixtures we demonstrate our recent results on these problems simple toy models are also used to show explicit solutions we explain that the maximum likelihood estimator is no longer subject to the gaussian distribution even asymptotically because the fisher information matrix degenerates that the model selection criteria such as aic bic and mdl fail to hold in these models that a smooth bayesian prior becomes singular in such models and that the trajectories of dynamics of learning are strongly affected by the singularity causing plateaus or slow manifolds in the parameter space the natural gradient method is shown to perform well because it takes the singular geometrical structure into account the generalization error and the training error are studied in some examples

four types of learning curves if machines are learning to make decisions given a number of examples the generalization error e t is defined as the average probability that an incorrect decision is made for a new example by a machine when trained with t examples the generalization error decreases as t increases and the curve e t is called a learning curve the present paper uses the bayesian approach to show that given the annealed approximation learning curves can be classified into four asymptotic types if the machine is deterministic with noiseless teacher signals then 1 e at 1 when the correct machine parameter is unique and 2 e at 2 when the set of the correct parameters has a finite measure if the teacher signals are noisy then 3 e at 1 2 for a deterministic machine and 4 e c at 1 for a stochastic machine

a numerical study on learning curves in stochastic multilayer feedforward networks the universal asymptotic scaling laws proposed by amari et al are studied in large scale simulations using a cm5 small stochastic multilayer feedforward networks trained with backpropagation are investigated in the range of a large number of training patterns t the asymptotic generalization error scales as 1 t as predicted for a medium range t a faster 1 t2 scaling is observed this effect is explained by using higher order corrections of the likelihood expansion it is shown for small t that the scaling law changes drastically when the network undergoes a transition from strong overfitting to effective learning

universal torsors of del pezzo surfaces and homogeneous spaces let cox sr be the homogeneous coordinate ring of the blow up sr of in r general points i e a smooth del pezzo surface of degree 9 r we prove that for r 6 7 proj cox sr can be embedded into gr pr where gr is an algebraic group with root system given by the primitive picard lattice of sr and pr gr is a certain maximal parabolic subgroup

construction of minimal n 2 n encoders for any n the encoding problem rumelhart and mcclelland 1986 is an important canonical problem it has been widely used as a benchmark here we have analytically derived minimal sized nets necessary and sufficient to solve encoding problems of arbitrary size the proofs are constructive we construct n 2 n encoders and show that two hidden units are also necessary for n 2 moreover the geometric approach employed is general and has much wider applications for example this method has also helped us derive lower bounds on the redundancy necessary for achieving complete fault tolerance phatak and koren 1992a b

counting algebraic numbers with large height ii let denote the field of rational numbers open image in new window an algebraic closure of and h open image in new window the absolute multiplicative weil height for each positive integer d and real number mathcal h geqslant 1 it is well known that the number open image in new window of points in open image in new window having degree d over and satisfying h left alpha right leqslant mathcal h is finite this is the one dimensional case of northcott s theorem 8 see also 5 page 59 the systematic study of the counting function open image in new window and that of related functions in higher dimensions was begun by schmidt 10 it is relatively easy to prove the existence of a positive constant c c d such that open image in new window 1 and also the existence of positive constants c c d and mathcal h 0 mathcal h 0 left d right such that open image in new window 2

stochastic complexity and modeling on demontre un theoreme fondamental qui donne une borne inferieure pour la longueur de code et donc pour les erreurs de prediction on definit les notions d information a priori et d information utile dans les donnees

neural representation of space using sinusoidal arrays o keefe 1991 has proposed that spatial information in rats might be represented as phasors phase and amplitude of a sine wave encoding angle and distance to a landmark we describe computer simulations showing that operations on phasors can be efficiently realized by arrays of spiking neurons that re code the temporal dimension of the sine wave spatially some cells in motor and parietal cortex exhibit response properties compatible with this proposal

on an unsupervised learning rule for scalar quantization following the maximum entropy principle in this paper we observe that a particular class of rational function rf approximations may be viewed as feedforward networks like the radial basis function rbf network the training of the rf network may be performed using a linear adaptive filtering algorithm we illustrate the application of the rf network by considering two nonlinear signal processing problems the first problem concerns the one step prediction of a time series consisting of a pair of complex sinusoid in the presence of colored non gaussian noise simulated data were used for this problem in the second problem we use the rf network to build a nonlinear dynamic model of sea clutter radar backscattering from a sea surface here real life data were used for the study

learning from examples in large neural networks a statistical mechanical theory of learning from examples in layered networks at finite temperature is studied when the training error is a smooth function of continuously varying weights the generalization error falls off asymptotically as the inverse number of examples by analytical and numerical studies of single layer perceptrons we show that when the weights are discrete the generalization error can exhibit a discontinuous transition to perfect generalization for intermediate sizes of the example set the state of perfect generalization coexists with a metastable spin glass state

parameter convergence and learning curves for neural networks we revisit the oft studied asymptotic in sample size behavior of the parameter or weight estimate returned by any member of a large family of neural network training algorithms by properly accounting for the characteristic property of neural networks that their empirical and generalization errors possess multiple minima we rigorously establish conditions under which the parameter estimate converges strongly into the set of minima of the generalization error convergence of the parameter estimate to a particular value cannot be guaranteed under our assumptions we then evaluate the asymptotic distribution of the distance between the parameter estimate and its nearest neighbor among the set of minima of the generalization error results on this question have appeared numerous times and generally assert asymptotic normality the conclusion expected from familiar statistical arguments concerned with maximum likelihood estimators these conclusions are usually reached on the basis of somewhat informal calculations although we shall see that the situation is somewhat delicate the preceding results then provide a derivation of learning curves for generalization and empirical errors that leads to bounds on rates of convergence

error bounds for functional approximation and estimation using mixtures of experts we examine some mathematical aspects of learning unknown mappings with the mixture of experts model mem specifically we observe that the mem is at least as powerful as a class of neural networks in a sense that will be made precise upper bounds on the approximation error are established for a wide class of target functions the general theorem states that spl par f f sub n spl par sub p spl les c n sup r d for f spl isin w sub p sup r l a sobolev class over 1 1 sup d and f sub n belongs to an n dimensional manifold of normalized ridge functions the same bound holds for the mem as a special case of the above the stochastic error in the context of learning from independent and identically distributed i i d examples is also examined an asymptotic analysis establishes the limiting behavior of this error in terms of certain pseudo information matrices these results substantiate the intuition behind the mem and motivate applications

attraction radii in binary hopfield nets are hard to compute we prove that it is an np hard problem to determine the attraction radius of a stable vector in a binary hopfield memory network and even that the attraction radius is hard to approximate under synchronous updating the problems are already np hard for two step attraction radii direct one step attraction radii can be computed in polynomial time

density of rational points on a quadric bundle in mathbb p 3 times mathbb p 3 we establish an asymptotic formula for the number of rational points of bounded anticanonical height which lie on a certain zariski dense subset of the biprojective hypersurface x1y12 x4y42 0 in p3 p3 this confirms the modified manin conjecture for this variety in which the removal of a thin set of rational points is allowed

prediction intervals for artificial neural networks abstract the artificial neural network ann is becoming a very popular model for engineering and scientific applications inspired by brain architecture artificial neural networks represent a class of nonlinear models capable of learning from data neural networks have been applied in many areas including pattern matching classification prediction and process control this article focuses on the construction of prediction intervals previous statistical theory for constructing confidence intervals for the parameters or the weights in an ann is inappropriate because the parameters are unidentifiable we show in this article that the problem disappears in prediction we then construct asymptotically valid prediction intervals and also show how to use the prediction intervals to choose the number of nodes in the network we then apply the theory to an example for predicting the electrical load

on dimensionality sample size and classification error of nonparametric linear classification algorithms this paper compares two nonparametric linear classification algorithms the zero empirical error classifier and the maximum margin classifier with parametric linear classifiers designed to classify multivariate gaussian populations formulae and a table for the mean expected probability of misclassification mep sub n are presented they show that the classification error is mainly determined by n p a learning set size dimensionality ratio however the influences of learning set size on the generalization error of parametric and nonparametric linear classifiers are quite different under certain conditions the nonparametric approach allows us to obtain reliable rules even in cases where the number of features is larger than the number of training vectors

algebraic analysis for nonidentifiable learning machines this article clarifies the relation between the learning curve and the algebraic geometrical structure of a nonidentifiable learning machine such as a multilayer neural network whose true parameter set is an analytic set with singular points by using a concept in algebraic analysis we rigorously prove that the bayesian stochastic complexity or the free energy is asymptotically equal to 1 log n m1 1 log log n constant where n is the number of training samples and 1 and m1 are the rational number and the natural number which are determined as the birational invariant values of the singularities in the parameter space also we show an algorithm to calculate 1 and m1 based on the resolution of singularities in algebraic geometry in regular statistical models 2 1 is equal to the number of parameters and m1 1 whereas in nonregular models such as multilayer networks 2 1 is not larger than the number of parameters and m1 1 since the increase of the stochastic complexity is equal to the learning curve or the generalization error the nonidentifiable learning machines are better models than the regular ones if bayesian ensemble learning is applied

redundancy reduction as a strategy for unsupervised learning a redundancy reduction strategy which can be applied in stages is proposed as a way to learn as efficiently as possible the statistical properties of an ensemble of sensory messages the method works best for inputs consisting of strongly correlated groups that is features with weaker statistical dependence between different features this is the case for localized objects in an image or for words in a text a local feature measure determining how much a single feature reduces the total redundancy is derived which turns out to depend only on the probability of the feature and of its components but not on the statistical properties of any other features the locality of this measure makes it ideal as the basis for a neural implementation of redundancy reduction and an example of a very simple non hebbian algorithm is given the effect of noise on learning redundancy is also discussed

on the geometry of thin exceptional sets in manin s conjecture manin s conjecture predicts the rate of growth of rational points of a bounded height after removing those lying on an exceptional set we study whether the exceptional set in manin s conjecture is a thin set using the minimal model program and boundedness of log fano varieties

rational points of bounded height on general conic bundle surfaces a conjecture of manin predicts the asymptotic distribution of rational points of bounded height on fano varieties in this paper we use conic bundles to obtain correct lower bounds for a wide class of surfaces over number fields for which the conjecture is still far from being proved for example we obtain the conjectured lower bound of manin s conjecture for any del pezzo surface whose picard rank is sufficiently large or for arbitrary del pezzo surfaces after possibly an extension of the ground field of small degree

natural gradient works efficiently in learning when a parameter space has a certain underlying structure the ordinary gradient of a function does not represent its steepest direction but the natural gradient does information geometry is used for calculating the natural gradients in the parameter space of perceptrons the space of matrices for blind source separation and the space of linear dynamical systems for blind source deconvolution the dynamical behavior of natural gradient online learning is analyzed and is proved to be fisher efficient implying that it has asymptotically the same performance as the optimal batch estimation of parameters this suggests that the plateau phenomenon which appears in the backpropagation learning algorithm of multilayer perceptrons might disappear or might not be so serious when the natural gradient is used an adaptive method of updating the learning rate is proposed and analyzed

manin s conjecture for quartic del pezzo surfaces with a conic fibration an asymptotic formula is established for the number of q rational points of bounded height on a non singular quartic del pezzo surface with a conic bundle structure

brief steering of a class of nonholonomic systems with drift terms in the present paper nonholonomic systems with drift terms are studied the discussion is focused on a class of lagrangian systems with a cyclic coordinate we present an approach to open loop path planning in which the system evolution is studied on manifolds of dimension equal to the number of control inputs a control procedure is derived and it is applied to the planar diver

tamagawa numbers of diagonal cubic surfaces numerical evidence a refined version of manin sconjecture aboutthe asymptotics of points of bounded height on fano varieties has been developped by batyrev and the authors we test numerically this refined conjecture for some diagonal cubic surfaces

counting rational points on biquadratic hypersurfaces abstract an asymptotic formula is established for the number of rational points of bounded anticanonical height which lie on a certain zariski open subset of an arbitrary smooth biquadratic hypersurface in sufficiently many variables the proof uses the hardy littlewood circle method

a polynomial time algorithm for generating neural networks for pattern classification its stability properties and some test results polynomial time training and network design are two major issues for the neural network community a new algorithm has been developed that can learn in polynomial time and also design an appropriate network the algorithm is for classification problems and uses linear programing models to design and train the network this paper summarizes the new algorithm proves its stability properties and provides some computational results to demonstrate its potential

sieving rational points on varieties an upper bound sieve for rational points on suitable varieties isdeveloped together with applications tocounting rational points in thin sets to local solubility in families and to the notion of friable rational pointswith respect to divisors in the special case of quadrics sharper estimates areobtained by developing a version of the selberg sieve for rational points

stochastic complexity and generalization error of a restricted boltzmann machine in bayesian estimation in this paper we consider the asymptotic form of the generalization error for the restricted boltzmann machine in bayesian estimation it has been shown that obtaining the maximum pole of zeta functions is related to the asymptotic form of the generalization error for hierarchical learning models watanabe 2001a b the zeta function is defined by using a kullback function we use two methods to obtain the maximum pole a new eigenvalue analysis method and a recursive blowing up process we show that these methods are effective for obtaining the asymptotic form of the generalization error of hierarchical learning models

asymptotic properties of the fisher kernel this letter analyzes the fisher kernel from a statistical point of view the fisher kernel is a particularly interesting method for constructing a model of the posterior probability that makes intelligent use of unlabeled data i e of the underlying data density it is important to analyze and ultimately understand the statistical properties of the fisher kernel to this end we first establish sufficient conditions that the constructed posterior model is realizable i e it contains the true distribution realizability immediately leads to consistency results subsequently we focus on an asymptotic analysis of the generalization error which elucidates the learning curves of the fisher kernel and how unlabeled data contribute to learning we also point out that the squared or log loss is theoretically preferable because both yield consistent estimators to other losses such as the exponential loss when a linear classifier is used together with the fisher kernel therefore this letter underlines that the fisher kernel should be viewed not as a heuristics but as a powerful statistical tool with well controlled statistical properties

on the geometry of feedforward neural network error surfaces many feedforward neural network architectures have the property that their overall input output function is unchanged by certain weight permutations and sign flips in this paper the geometric structure of these equioutput weight space transformations is explored for the case of multilayer perceptron networks with tanh activation functions similar results hold for many other types of neural networks it is shown that these transformations form an algebraic group isomorphic to a direct product of weyl groups results concerning the root spaces of the lie algebras associated with these weyl groups are then used to derive sets of simple equations for minimal sufficient search sets in weight space these sets which take the geometric forms of a wedge and a cone occupy only a minute fraction of the volume of weight space a separate analysis shows that large numbers of copies of a network performance function optimum weight vector are created by the action of the equioutput transformation group and that these copies all lie on the same sphere some implications of these results for learning are discussed

the density of rational points on non singular hypersurfaces ii for any integers d n geq 2 let x subset mathbb p n be a non singular hypersurface of degree d that is defined over the rational numbers the main result in this paper is a proof that the number of rational points on x which have height at most b is o b n 1 varepsilon for any varepsilon 0 the implied constant in this estimate depends at most upon d varepsilon and n

mutual information metric entropy and cumulative relative entropy risk assume p theta theta epsilon theta is a set of probability distributions with a common dominating measure on a complete separable metric space y a state theta epsilon theta is chosen by nature a statistician obtains n independent observations y 1 dots y n from y distributed according to p theta for each time t between 1 and n based on the observations y 1 dots y t 1 the statistician produces an estimated distribution hat p t for p theta and suffers a loss l p theta hat p t the cumulative risk for the statistician is the average total loss up to time n of special interest in information theory data compression mathematical finance computational learning theory and statistical mechanics is the special case when the loss l p theta hat p t is the relative entropy between the true distribution p theta and the estimated distribution hat p t here the cumulative bayes risk from time 1 to n is the mutual information between the random parameter theta and the observations y 1 dots y n new bounds on this mutual information are given in terms of the laplace transform of the hellinger distance between pairs of distributions indexed by parameters in theta from these bounds on the cumulative minimax risk are given in terms of the metric entropy of theta with respect to the hellinger distance the assumptions required for these bounds are very general and do not depend on the choice of the dominating measure they apply to both finite and infinite dimensional theta they apply in some cases where y is infinite dimensional in some cases where y is not compact in some cases where the distributions are not smooth and in some parametric cases where asymptotic normality of the posterior distribution fails

lie algebraic stability criteria for switched systems it was recently shown that a family of exponentially stable linear systems whose matrices generate a solvable lie algebra possesses a quadratic common lyapunov function which implies that the corresponding switched linear system is exponentially stable for arbitrary switching in this paper we prove that the same properties hold under the weaker condition that the lie algebra generated by given matrices can be decomposed into a sum of a solvable ideal and a subalgebra with a compact lie group the corresponding local stability result for nonlinear switched systems is also established moreover we demonstrate that if a lie algebra fails to satisfy the above condition then it can be generated by a family of stable matrices such that the corresponding switched linear system is not stable relevant facts from the theory of lie algebras are collected at the end of the paper for easy reference

geometry of lie integrability by quadratures in this paper we extend the lie theory of integration by quadratures of systems of ordinary differential equations in two different ways first we consider a finite dimensional lie algebra of vector fields and discuss the most general conditions under which the integral curves of one of the fields can be obtained by quadratures in a prescribed way it turns out that the conditions can be expressed in a purely algebraic way in the second step we generalize the construction to the case in which we substitute the lie algebra of vector fields by a module generalized distribution we obtain a much larger class of explicitly integrable systems replacing standard concepts of solvable or nilpotent lie algebra with distributional solvability nilpotency

constraints on synchronizing oscillator networks this paper investigates the constraints placed on some synchronized oscillator models by their underlying dynamics phase response graphs are used to determine the phase locking behaviors of three oscillator models these results are compared with idealized phase response graphs for single phase and multiple phase systems we find that all three oscillators studied are best suited to operate in a single phase system and that the requirements placed on oscillatory models for operation in a multiple phase system are not compatible with the underlying dynamics of oscillatory behavior for these types of oscillator models

on the asymptotic normality of hierarchical mixtures of experts for generalized linear models in the class of hierarchical mixtures of experts hme models experts in the exponential family with generalized linear mean functions of the form spl psi spl alpha x sup t spl beta are mixed according to a set of local weights called the gating functions depending on the predictor x here spl psi spl middot is the inverse link function we provide regularity conditions on the experts and on the gating functions under which the maximum likelihood method in the large sample limit produces a consistent and asymptotically normal estimator of the mean response the regularity conditions are validated for poisson gamma normal and binomial experts

recurrent and feedforward polynomial modeling of coupled time series we present two methods for the prediction of coupled time series the first one is based on modeling the series by a dynamic system with a polynomial format this method can be formulated in terms of learning in a recurrent network for which we give a computationally effective algorithm the second method is a purely feedforward network procedure whose architecture derives from the recurrence relations for the derivatives of the trajectories of a ricatti format dynamic system it can also be used for the modeling of discrete series in terms of nonlinear mappings both methods have been tested successfully against chaotic series

an asymptotic statistical theory of polynomial kernel methods the generalization properties of learning classifiers with a polynomial kernel function are examined in kernel methods input vectors are mapped into a high dimensional feature space where the mapped vectors are linearly separated it is well known that a linear dichotomy has an average generalization error or a learning curve proportional to the dimension of the input space and inversely proportional to the number of given examples in the asymptotic limit however it does not hold in the case of kernel methods since the feature vectors lie on a submanifold in the feature space called the input surface in this letter we discuss how the asymptotic average generalization error depends on the relationship between the input surface and the true separating hyperplane in the feature space where the essential dimension of the true separating polynomial named the class is important we show its upper bounds in several cases and confirm these using computer simulations

rational points on quartic hypersurfaces let x be a projective non singular quartic hypersurface of dimension 39 or more which is defined over we show that x is non empty provided that x is non empty and x has p adic points for every prime p 2009 walter de gruyter

rational points of bounded height on fano varieties a prime pe7z let v be an algebraic variety defined over f and li a metrized line bundle on v i e a system l consisting of a line bundle l and a family of banach v adic metrics on l f for all v s satisfying well known conditions cf 1 chap vi then we can define a height function hl x f i r by the formula h x is x 1 where s is a local section of l non vanishing

learning coefficient of generalization error in bayesian estimation and vandermonde matrix type singularity the term algebraic statistics arises from the study of probabilistic models and techniques for statistical inference using methods from algebra and geometry sturmfels 2009 the purpose of our study is to consider the generalization error and stochastic complexity in learning theory by using the log canonical threshold in algebraic geometry such thresholds correspond to the main term of the generalization error in bayesian estimation which is called a learning coefficient watanabe 2001a 2001b the learning coefficient serves to measure the learning efficiencies in hierarchical learning models in this letter we consider learning coefficients for vandermonde matrix type singularities by using a new approach focusing on the generators of the ideal which defines singularities we give tight new bound values of learning coefficients for the vandermonde matrix type singularities and the explicit values with certain conditions by applying our results we can show the learning coefficients of three layered neural networks and normal mixture models

convergent algorithm for sensory receptive field development an unsupervised developmental algorithm for linear maps is derived which reduces the pixel entropy using the measure introduced in previous work at every update and thus removes pairwise correlations between pixels since the measure of pixel entropy has only a global minimum the algorithm is guaranteed to converge to the minimum entropy map such optimal maps have recently been shown to possess cognitively desirable properties and are likely to be used by the nervous system to organize sensory information the algorithm derived here turns out to be one proposed by goodall for pairwise decorrelation it is biologically plausible since in a neural network implementation it requires only data available locally to a neuron in training over ensembles of two dimensional input signals with the same spatial power spectrum as natural scenes networks develop output neurons with center surround receptive fields similar to those of ganglion cells in the retina some technical issues pertinent to developmental algorithms of this sort such as symmetry fixing are also discussed

height zeta functions of equivariant compactifications of unipotent groups we prove manin s conjecture for bi equivariant compactifications of unipotent groups 2015 wiley periodicals inc

statistical theory of learning curves under entropic loss criterion the present paper elucidates a universal property of learning curves which shows how the generalization error training error and the complexity of the underlying stochastic machine are related and how the behavior of a stochastic machine is improved as the number of training examples increases the error is measured by the entropic loss it is proved that the generalization error converges to h0 the entropy of the conditional distribution of the true machine as h0 m 2t while the training error converges as h0 m 2t where t is the number of examples and m shows the complexity of the network when the model is faithful implying that the true machine is in the model m is reduced to m the number of modifiable parameters this is a universal law because it holds for any regular machine irrespective of its structure under the maximum likelihood estimator similar relations are obtained for the bayes and gibbs learning algorithms these learning curves show the relation among the accuracy of learning the complexity of a model and the number of training examples

eisenstein series on loop groups maass selberg relations 3 in this paper we prove the maass selberg relations for the inner product of truncated pseudo eisenstein series one can then extract the maass selberg relations for truncated eisenstein series themselves from those for truncated pseudo eisenstein series a task which we shall complete in a subsequent paper