structure of metastable states in the hopfield model an upper bound for the number of metastable states in the hopfield model is calculated as a function of the hamming fraction from an input pattern for all finite values of alpha the ratio of number of patterns to nodes the hamming fraction from the input pattern to the nearest metastable state is infinite when alpha 0 113 the bound also implies that there is a gap between a set of states close to the input pattern and another set centred around the hamming fraction 0 5 from it

learning the reversed wedge problem using a multi interacting perceptron with correlated weights abstract we consider the task of learning the so called reversed wedge problem using a multi interacting perceptron with first and third order synap ses where the third order synaptic couplings are expressed as products of the first order synapses associated to the neurons involved in the corresponding multi interaction this correlation condition allows the training of the multi interacting perceptron to be achieved by adjusting the set of first order weights in such a way that the learning rates scales with the dimensionality of a simple perceptron remarkably if the width of the reversed inner region wedge is smaller than 2 3 the high temperature approach predicts a transition from a poor generalization regime to a state with good performance where the generalization error is identical to the results for the problem of a simple perceptron learning a linearly separable rule the simulation results are in excellent agreement with the analytical predictions

learning and retrieval in attractor neural networks above saturation the authors investigate neural networks in the range of parameters when the ground state energy is positive namely when a synaptic matrix which satisfies all the desired constraints cannot be found by the learning algorithm in particular they calculate the typical distribution functions of local stabilities obtained for a number of algorithms in this region these functions are used to investigate the retrieval properties as reflected by the size of the basins of attraction this is done analytically in sparsely connected networks and numerically in fully connected networks the main conclusion is that the retrieval behaviour of attractor neural networks can be improved by learning above saturation

the category of f modules has finite global dimension abstract let r be a regular ring of characteristic p 0 in 4 hochster showed that the category of lyubeznik s f r modules has enough injectives so that every f r module has an injective resolution in this category we show in this paper that under mild conditions on r for example when r is essentially of finite type over an f finite regular local ring the category of f modules has finite global dimension d 1 where d dim r in 4 hochster also showed that when m and n are f r finite f r modules hom f r m n is finite we show that in general ext f r 1 m n is not necessarily finite

a uniformly accurate finite element method for a singularly perturbed one dimensional reaction diffusion application d une methode a elements finis avec des elements de base exponentiels a un probleme autoadjoint a perturbation singuliere et a deux points limites generation d un schema aux differences tridiagonal uniformement precis du second ordre

sensitivity of spin glass order to temperature changes by a loop expansion around parisi s mean field theory for an ising spin glass it is shown that the overlap of the magnetization patterns belonging to two different temperatures t and t vanishes to any order si t si t 0 while the correlation overlap sisj t sisj t calculated to first loop order and for technical reasons for dimensions d 8 only is found to decay exponentially with a characteristic length approximately t t 1

heterogeneous domain decomposition for singularly perturbed elliptic boundary value problems a heterogeneous domain decomposition method is presented for the numerical solution of singularly perturbed elliptic boundary value problems the method which is parallelizable at various levels uses several ideas of asymptotic analysis the subdomains match the domains of validity of the local inner and outer asymptotic expansions and cut off functions are used to match solutions in neighboring subdomains the positions of the interfaces as well as the mesh widths depend on the small parameter varepsilon on the subdomains iterative solution techniques are used which may vary from one subdomain to another the global convergence rate depends on varepsilon it generally increases like some power of log varepsilon 1 1 as varepsilon downarrow 0 the method is illustrated on several two dimensional singular perturbation problems

finite size effects and bounds for perceptron models ablncl in lhis paper we consider two main aspsln of the binary perccptron problem the maximal capacity when random paltem are stored model a and its generalhation ability model b we have extended previous numerical estimates of critical capacilies and studied thermal properties of systems of small sizes to lest recent replica predictions we have also considered some simpler versions of thest models the discrete spherical vemions can be salved exactly using gardner s replica calculation for the spherical model and are shown to give a rigorous upper bound and lower bound on the capacities of models a and b respectively iby vemions of models a and b are soivcd in detail and provide information which is useful for interpreting the bnile size eeeciects present in the numerical studies of models a and b

from statistical mechanics to neural networks and back abstract the neural network models which have attracted so much attention in the physics community consist of a large assembly of two state elements highly interconnected by ferromagnetic and antiferromagnetic interactions methods of statistical mechanics have been applied to these models bringing new discoveries and surprises to neural network science at the same time neural network models have posed new problems and incentives to statistical mechanics one of the basic problems is that of learning which in the context of these models means a choice of the interactions for which the network dynamics leads to specific attractors the main emphasis in this paper is on the essential developments and results on this problem derived from methods of statistical mechanics

large systems of random linear equations with nonnegative solutions characterizing the solvable and the unsolvable phase large systems of linear equations are ubiquitous in science quite often e g when considering population dynamics or chemical networks the solutions must be nonnegative recently it has been shown that large systems of random linear equations exhibit a sharp transition from a phase where a nonnegative solution exists with probability one to one where typically no such solution may be found the critical line separating the two phases was determined by combining farkas lemma with the replica method here we show that the same methods remain viable to characterize the two phases away from criticality to this end we analytically determine the residual norm of the system in the unsolvable phase and a suitable measure of robustness of solutions in the solvable one our results are in very good agreement with numerical simulations

learning and generalization in a two layer neural network the role of the vapnik chervonvenkis dimension bounds for the generalization ability of neural networks based on vapnik chervonenkis vc theory are compared with statistical mechanics results for the case of the parity machine for fixed phase space dimension the vc dimension can grows arbitrarily by increasing the number k of hidden units generalization is impossible up to a critical number of training examples that grows with the vc dimension the asymptotic decrease of the generalization error mathrm ensuremath varepsilon mathit g comes out independent of k and the vc bounds strongly overestimate mathrm ensuremath varepsilon mathit g this shows that phase space dimension and vc dimension can play independent and different roles for the generalization process

the failure distribution in percolation models of breakdown the probability of finding a largest defect cluster of size n in a percolation network is calculated analytically using a new distribution function scaling equation from this result and the stress voltage enhancement at the tip of the most critical defect in the network the probability of failure in percolation models of breakdown is calculated for defect fractions less than the percolation point this distribution is found to be of the form exponential of an exponential numerical simulations on the two dimensional random fuse network confirm the new distribution function and convincingly distinguish between it and the weibull 1951 form most often used in the fitting of breakdown data

applications of quintic hermite collocation with time discretization to singularly perturbed problems abstract singular perturbation problems have been discussed using collocation technique with quintic hermite interpolating polynomials as base functions these polynomials have the property to interpolate the function as well as its tangent at node points to discretize the problem in temporal direction forward difference operator has been applied the given technique is a combination of collocation and difference scheme parameter uniform convergence has been studied using the method given by farrell and hegarty 1991 rate of convergence of quintic hermite difference scheme has been found to depend upon node points applicability and computational effect of the scheme has been examined through numerical examples results have been presented graphically through surface plots as well as in tabular form

a second order parameter uniform overlapping schwarz method for reaction diffusion problems with boundary layers the problem of constructing a parameter uniform numerical method for a singularly perturbed self adjoint ordinary differential equation is considered it is shown that a suitably designed discrete schwarz method based on a standard finite difference operator with a uniform mesh on each subdomain gives numerical approximations which converge in the maximum norm to the exact solution uniformly with respect to the singular perturbation parameter this parameter uniform convergence is shown to be essentially second order that this new discrete schwarz method is efficient in practice is demonstrated by numerical experiments

learning a rule in a multilayer neural network the problem of learning from examples in multilayer networks is studied within the framework of statistical mechanics using the replica formalism we calculate the average generalization error of a fully connected committee machine in the limit of a large number of hidden units if the number of training examples is proportional to the number of inputs in the network the generalization error as a function of the training set size approaches a finite value if the number of training examples is proportional to the number of weights in the network we find first order phase transitions with a discontinuous drop in the generalization error for both binary and continuous weights

weight space structure and generalization in the reversed wedge perceptron the generalization ability of the reversed wedge perceptron serving as a toy model for multilayer neural networks is investigated we analyse the decomposition of the version space into disjoint cells belonging to different internal representations defined by the signs of the aligning fields the version space is characterized by the number and size of these cells and their typical overlap with the teacher network for a small training set the system is unable to detect the structure of the patterns induced by the teacher accordingly it performs as if storing random input output patterns with very low generalization ability and a large misfit in the internal representation with increasing training set size cells with large misfit are eliminated at a much higher rate than those with internal representation similar to that of the teacher this results eventually in the discontinuous phase transition to good generalization typical for multilayer neural networks

generalization ability of perceptrons with continuous outputs in this paper we examine the influence of different input output relations on the generalization ability of a single layer perceptron the input output relations can be linear binary or sigmoid with this choice we take into account most of the cases which are of present interest the generalization problem will be realizable or unrealizable if the input output relations for teacher and student are identical or not we show that sometimes it can have a positive effect on the generalization ability if one learns with errors

uniform convergence of the schwarz alternating method for solving singularly perturbed advection diffusion equations our purpose in this paper is to study the convergence rate of some variants of the schwarz alternating method for solving two types of singularly perturbed advection diffusion equations one in which the advection term first order term dominates and the other in which the lowest order term zeroth order term dominates for such problems boundary or other layer regions may be present for singular perturbation problems in which the advection dominates we choose two overlapping subregions one corresponding to an outer or elliptic region and the other corresponding to an inner or hyperbolic region our main result for this problem shows that if the subdomains can be chosen to follow the flow i e if the boundary interface of one of the subdomains corresponds to an outflow boundary for the streamlines of the flow then the schwarz iterates converge in the maximum norm with an error reduction factor per iteration that exponentially decays with increasing overlap or decreasing diffusion for the singular perturbation problem in which the lowest order term dominates we consider many overlapping subregions with no special requirement on outflow or inflow boundaries our main result here which is valid for two variants of the classical schwarz algorithm shows that the error reduction factor per iteration decays exponentially in the maximum norm with increasing overlap and increasing dominance of the zeroth order term for both problems our proofs rely on the maximum principle and the construction of barrier functions

a robust overlapping schwarz domain decomposition algorithm for time dependent singularly perturbed reaction diffusion problems in this work we consider a singularly perturbed parabolic problem of reaction diffusion type to solve this problem numerically we develop an overlapping schwarz domain decomposition method where we use the asymptotic behaviour of the exact solution for domain partitioning we prove that the method gives uniform numerical approximations of first order in time and almost second order in space furthermore we address the much faster convergence of the algorithm for small perturbation parameter e to be more specific we prove that when e is small just one iteration is required to achieve the desired accuracy we then extend the method to a system of singularly perturbed parabolic problems of reaction diffusion type numerical experiments support the theoretical results and demonstrate the effectiveness of the method

a parameter uniform schwarz method for a coupled system of reaction diffusion equations we consider an arbitrarily sized coupled system of one dimensional reaction diffusion problems that are singularly perturbed in nature we describe an algorithm that uses a discrete schwarz method on three overlapping subdomains extending the method in h macmullen j j h miller e o riordan g i shishkin a second order parameter uniform overlapping schwarz method for reaction diffusion problems with boundary layers j comput appl math 130 2001 231 244 to a coupled system on each subdomain we use a standard finite difference operator on a uniform mesh we prove that when appropriate subdomains are used the method produces e uniform results furthermore we improve upon the analysis of the above mentioned reference to show that for small e just one iteration is required to achieve the expected accuracy

phase transitions in the generalization behaviour of multilayer neural networks we study the generalization ability of multilayer neural networks with tree architecture for the case of random training sets the first layer consists of k spherical perceptrons with binary output a boolean function b computes the final output from the k values produced by the first layer we first calculate the learning behaviour of gibbs learning in the case of learnable rules where teacher and student have the same architecture and the boolean function b is permutation symmetric with respect to the hidden units in the asymptotic case of high loading alpha to infinity as usual alpha is the loading parameter we find that the generalization error vanishes in the same way for all b the reason being is that there are two effects cancelling each other in the opposite limit for small alpha we find qualitatively different behaviour i e some networks undergo a phase transition we show how these differences in the behaviour depend on certain characteristics of the boolean function b we then study the bayes algorithm which generalizes according to the majority decision of a certain ensemble of machines and find that the parity function plays a special role if the teacher is a parity machine there always exists a single student parity machine that generalizes as well as the bayes algorithm

a multi interacting perceptron model with continuous outputs abstract we consider learning and generalization of real functions by a multi interacting feed forward network model with continuous outputs with invertible transfer functions the expansion in different multi interacting orders provides a classification for the functions to be learnt and suggests the learning rules that reduce to the hebb learning rule only for the second order linear perceptron the over sophistication problem is straightforwardly overcome by a natural cutoff in the multi interacting synapses the student is able to learn the architecture of the target rule that is the simpler a rule is the faster the multi interacting perception may learn simulation results are in excellent agreement with analytical calculations

on a domain decomposition algorithm for a singularly perturbed reaction diffusion problem this paper deals with an iterative algorithm for domain decomposition applied to the solution of a singularly perturbed reaction diffusion problem this algorithm is based on finite difference domain decomposition approach and suitable for parallel computing convergence properties of the algorithm are established numerical results for a test singularly perturbed problem are presented

a schwarz alternating procedure for singular perturbation problems we show that the schwarz alternating procedure offers a good algorithm for the numerical solution of singular perturbation problems provided the domain decomposition is properly designed to resolve the boundary and transition layers we give sharp estimates for the optimal position of the domain boundaries and study the convergence rates of the algorithm for various linear second order singular perturbation problems we report on implementation results for a turning point problem and a combustion problem

a second order overlapping schwarz method for a 2d singularly perturbed semilinear reaction diffusion problem an overlapping schwarz domain decomposition is applied to a semilinear reaction diffusion equation posed in a smooth two dimensional domain the problem may exhibit multiple solutions its diffusion parameter e2 is arbitrarily small which induces boundary layers the schwarz method invokes a boundary layer subdomain and an interior subdomain the narrow subdomain overlap being of width o e ln h where h is the maximum side length of mesh elements and the global number of mesh nodes does not exceed o h 2 we employ finite differences on layer adapted meshes of bakhvalov and shishkin types in the boundary layer subdomain and lumped mass linear finite elements on a quasiuniform delaunay triangulation in the interior subdomain for this iterative method we present maximum norm error estimates for e 0 1 it is shown in particular that when e c ln h 1 one iteration is sufficient to get second order convergence with in the case of the shishkin mesh a logarithmic factor in the maximum norm uniformly in e numerical results are presented to support our theoretical conclusions

learning and generalization in multineuron interacting feed forward neural networks we consider learning and generalization in the multi interacting feed forward network model recently proposed by h o carmesin 1994 with an a priori definition of the net architecture based on symmetries presented by the function to be learnt we define a generalized hebb rule extend the maximum stability learning algorithm to multi interactions and obtain training and generalization curves for rules where different orders of synapses are not correlated the results obtained for the simple perceptron concerning the hebb rule and through replica calculations in the space of couplings may be straightforwardly adapted to multi interactions through a simple renormalization of the total number of independent couplings analytical and numerical simulation results are compared and show excellent agreement

symmetry breaking in nonmonotonic neural networks the optimal performance of a nonmonotonic neural network is studied by the replica method in analogy to what happens in multi layered networks the authors show that replica symmetry breaking rsb is required the distribution of the patterns stabilities the correlations in the distribution of the internal representation and the optimal capacity per synapse alpha c approximately 4 8 are computed with one step of rsb

optimal storage properties of neural network models the authors calculate the number p alpha n of random n bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter k for each value of alpha and k there is a minimum fraction fmin of wrong bits they find a critical line alpha c k with alpha c 0 2 the minimum fraction of wrong bits vanishes for alpha alpha c k the calculations are done using a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is locally stable in a finite region of the k alpha plane including the line alpha c k but there is a line above which the solution becomes unstable and replica symmetry must be broken

learning algorithms with optimal stability in neural networks to ensure large basins of attraction in spin glass like neural networks of two state elements xi imu or 1 the authors propose to study learning rules with optimal stability delta where delta is the largest number satisfying delta or sigma j jij xi jmu xi imu mu 1 p i 1 n where n is the number of neurons and p is the number of patterns they motivate this proposal and provide optimal stability learning rules for two different choices of normalisation for the synaptic matrix jij in addition numerical work is presented which gives the value of the optimal stability for random uncorrelated patterns

the space of interactions in neural network models the typical fraction of the space of interactions between each pair of n ising spins which solve the problem of storing a given set of p random patterns as n bit spin configurations is considered the volume is calculated explicitly as a function of the storage ratio alpha p n of the value kappa 0 of the product of the spin and the magnetic field at each site and of the magnetisation m here m may vary between 0 no correlation and 1 completely correlated the capacity increases with the correlation between patterns from alpha 2 for correlated patterns with kappa 0 and tends to infinity as m tends to 1 the calculations use a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is shown to be locally stable a local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist

an almost fourth order uniformly convergent domain decomposition method for a coupled system of singularly perturbed reaction diffusion equations we consider a system of m 2 singularly perturbed equations of reaction diffusion type coupled through the reaction term a high order schwarz domain decomposition method is developed to solve the system numerically the method splits the original domain into three overlapping subdomains on two boundary layer subdomains we use a compact fourth order difference scheme on a uniform mesh while on the interior subdomain we use a hybrid scheme on a uniform mesh we prove that the method is almost fourth order e uniformly convergent furthermore we prove that when e is small one iteration is sufficient to get almost fourth order e uniform convergence numerical experiments are performed to support the theoretical results

cartier modules finiteness results on a locally noetherian scheme x over a field of positive characteristic p we study the category of coherent ox modules m equipped with a pe linear map i e an additive map c ox ox satisfying rc m c r p e m for all m a m r a ox the notion of nilpotence meaning that some power of the map c is zero is used to rigidify this cate gory the resulting quotient category called cartier crystals satisfies some strong finiteness conditions the main result in this paper states that if the frobenius morphism on x is a finite map i e if x is f finite then all cartier crystals have finite length we further show how this and related results can be used to recover and generalize other finiteness results of hartshorne speiser 19 lyubeznik 27 sharp 30 enescu hochster 14 and hochster 20 about the structure of modules with a left action of the frobenius for example we show that over any regular f finite scheme x lyubeznik s f finite modules 27 have finite length

an analysis of overlapping domain decomposition methods for singularly perturbed reaction diffusion problems in this paper we present an analysis of overlapping domain decomposition methods for singularly perturbed reaction diffusion problems for this purpose we consider a model problem for coupled system of singularly perturbed reaction diffusion equations with distinct small positive parameters exhibiting overlapping boundary layers at both ends of the domain a discrete overlapping schwarz domain decomposition method is considered to solve the model problem numerically the analysis is based on defining some auxiliary problems that allow to prove the uniform convergence of the method in two steps splitting the discretization error and the iteration error the numerical approximations generated from the method are shown to be almost second order uniformly convergent the presented idea has a general significance and can be used for the analysis of overlapping domain decomposition methods for other classes of singularly perturbed reaction diffusion problems

computational method for a singular perturbation problem via domain decomposition and its parallel implementation abstract in this paper a computation method for the nonlinear problem with a small perturbation parameter via domain decomposition is presented we apply a combination of the two approaches to solving the problem iterative algorithms for domain decomposition and a grid refinement technique on subdomains two iterative algorithms are examined the first one is the schwarz alternating procedure and the second algorithm is highly suitable for parallel computing numerical examples are provided

statistical mechanics of neural networks what are the differences between wide and narrow basins abstract we consider training noise in neural networks as a means of tuning the structure of retrieval basins and study how learning and retrieving properties depend on it the stability of the replica symmetric solution and the correlation in the weight space indicate that neural networks can be roughly classified into hebbian like and msn like msn meaning the maximally stable network re entrant retrieval noise robustness selectivity damage spreading and activity distribution all illustrate the differences in retrieval behaviours arising from the different basin structures