structure of metastable states in the hopfield model an upper bound for the number of metastable states in the hopfield model is calculated as a function of the hamming fraction from an input pattern for all finite values of alpha the ratio of number of patterns to nodes the hamming fraction from the input pattern to the nearest metastable state is infinite when alpha 0 113 the bound also implies that there is a gap between a set of states close to the input pattern and another set centred around the hamming fraction 0 5 from it

storage capacity of a neural network with state dependent synapses the storage capacity of the hopfield network is limited to p n c 0 138 p number of patterns n number of neurons beyond which the contribution of weakly correlated patterns surpasses that of the desired pattern this contribution can be eliminated by introducing a threshold a pattern correlation below this threshold is simply set to zero in the synapses we solve the mean field equations and derive the critical value of the threshold required to stabilize p n patterns with c

index concepts for linear mixed systems of differential algebraic and hyperbolic type equations for many technical systems the use of a refined network modeling approach leads to hyperbolic type initial boundary value problems of partial differential algebraic equations pdaes the boundary conditions of these systems are governed by time dependent differential algebraic equations daes that couple the pdae system with the network elements that are modeled by daes in time only in order to classify these systems we extend some index notions that have already been introduced to treat parabolic type problems a perturbation index is considered that reflects the sensitivity of the mixed system to slight perturbations in the right hand side of the pdaes as well as in the input signals of the dae systems and initial values in order to make an a posteriori analysis of semidiscretization in space and time we introduce additionally a space and a method of lines mol index here one is especially interested in whether the semidiscretized systems properly reflect the properties of the underlying systems we will show that these indices may detect an artificial sensitivity with respect to perturbations e g if the semidiscretization does not consider the information transport along the characteristics

hierarchical neural networks for the storage of correlated memories a class of hierarchical neural network models introduced by dotsenko for the storage and associative recall of strongly correlated memories is studied analytically and numerically in these models patterns stored in higher levels of the hierarchy represent generalized categories and those stored in lower levels describe finer details we first show that the models originally proposed by dotsenko have a serious flaw they are not able to detect or correct errors in categorization which may be present in the input we then describe three different models which attempt to overcome this shortcoming of the original models in the first model the interaction between different levels of the hierarchy has the form of an external field conjugate to memories stored in the lower level in the second model a three spin interaction term is included in addition to the usual binary interactions of the hopfield type the third model makes use of a time delay mechanism to induce if necessary transitions between memory states and their complements detailed analytical and numerical studies of the performance of these models are presented our analysis shows that all three models are able to detect and also to correct in varying degrees any error in categorization that may be present in the input pattern

categorization and effective perceptron learning in feed forward neural networks in the initial stages of back propagation learning weights in the first synaptic layer are small and hidden neurons operate in the linear regime so the input output map is then the same as that of a perceptron as the weights increase the hidden neurons begin to saturate each saturation pattern represents a particular category of input vectors for which the neural network nn locally behaves as a perceptron the effective perceptron weights for each category are functions of the synaptic weights of the nn we define an internal temperature as the inverse of the norm of the first synaptic matrix and an entropy function that measures the disorder of the hidden neuron layer the learning curves are recast in the language of a condensation process the entropy drops abruptly when the hidden neurons begin to saturate marking a clear transition from the global perceptron state to an ordered state characterized by different local effective perceptrons in different categories the problem of optimizing the effective perceptrons in different categories is frustrated

a differentiation index for partial differential algebraic equations a differentiation index for nonlinear partial differential algebraic equations is presented determination of the differentiation index with respect to a direction in the space of independent variables uncovers all equations that must be satisfied by cauchy data on the hyperplane orthogonal to that direction this index is thus a generalization of the differentiation index of differential algebraic equations the motivation for this work is consistent initialization of partial differential algebraic systems the analysis is demonstrated through several examples including some taken from fluid dynamics and chemical engineering

index of an implicit time varying linear differential equation a noncommutative linear algebraic approach abstract an intrinsic definition of the index for implicit linear time varying differential systems is proposed this definition relies on linear algebraic techniques over the noncommutative principal ideal ring of nonconstant linear differential operators noncommutative transfer function or state variable representation accordance with the existing definition for constant coefficient linear systems is demonstrated

global observability of a class of nonlinear discrete time systems abstract it is shown that discrete time systems whose dynamics are governed by piecewise monotone maps of the interval are globally observable under a general class of observations the results rest on the fact that the dynamics are ergodic

adjoint sensitivity analysis for time dependent partial differential equations with adaptive mesh refinement a new adjoint sensitivity analysis approach is presented for time dependent partial differential equations with adaptive mesh refinement the new approach called adda combines the best features of both the adjoint of the discretization ad and discretization of the adjoint da approaches it removes the obstacles of applying ad to adaptive methods and in contrast to da requires for its use only a minimal amount of knowledge about the formulation of adjoint pdes and their boundary conditions the effectiveness and efficiency of adda are demonstrated for several numerical examples

index and characteristic analysis of linear pdae systems automatable analyses of the index and well posedness of two general classes of linear pdaes are presented for use in equation based dynamic simulation packages these analyses determine whether a problem possesses a unique solution trajectory the potential effect of noise in the initial and boundary conditions on the calculated solution and the index of the model equations measures of the degeneracy of the system rather than a differentiation index bound the order of derivatives of forcing functions that appear in the solution

classification and generalization of pseudo random number generators abstract in this paper we give a classification of pseudo random number generators up to a certain group action by giving a complete invariant for the action a realization theory is developed based on this invariant

on the multi neuron interaction model without truncating the interaction a replica symmetric mean field theory approach is presented to the multi neuron interaction model introduced by de almeida and iglesias 1990 phys lett 146a 239 fixed point equations are derived for the relevant order parameters of the model extended to include biased patterns without truncating the interaction the capacity bias and the temperature capacity phase diagrams are discussed compared with the truncated version of the model it is found that the capacity at zero temperature is infinite and that the retrieval states satisfy the de almeida thouless stability condition

the index of general nonlinear daes in the last few years there has been considerable research on differential algebraic equations daes f t y y 0 where f y is identically singular much of the mathematical effort has focused on computing a solution that is assumed to exist more recently there has been some discussion of solvability of daes there has historically been some imprecision in the use of the two key concepts of solvability and index for daes the index is also important in control and systems theory but with different terminology the consideration of increasingly complex nonlinear daes makes a clear and correct development necessary this paper will try to clarify several points concerning the index after establishing some new and more precise terminology that we need some inaccuracies in the literature will be corrected the two types of indices most frequently used the differentiation index and the perturbation index are defined with respect to solutions of unperturbed problems examples are given to show that these indices can be very different for the same problem we define new maximum indices which are the maxima of earlier indices in a neighborhood of the solution over a set of perturbations and show that these indices are simply related to each other these indices are also related to an index defined in terms of jacobians

on impasse points of quasilinear differential algebraic equations abstract this paper presents a mathematical characterization of the impasse points of quasilinear dae s a x g x where then a x is a nxn matrix having constant but not full r n in the domain of interest we show that a reduction procedure permits to reduce the dae to a quasilinear ode al xi xi g1 xi rr and that impasse points correspond to special singularities of the reduced ode impasse points of higher index dae s can also be defined the only difference is that the above reduction procedure takes more than one step

space time domain decomposition methods for diffusion problems in mixed formulations this paper is concerned with global in time nonoverlapping domain decomposition methods for the mixed formulation of the diffusion problem two approaches are considered one uses the time dependent steklov poincare operator and the other uses optimized schwarz waveform relaxation oswr based on robin transmission conditions for each method a mixed formulation of an interface problem on the space time interfaces between subdomains is derived and different time grids are employed to adapt to different time scales in the subdomains demonstrations of the well posedness of the robin subdomain problems involved in the oswr method and a convergence proof of the oswr algorithm are given for the mixed formulation numerical results for two dimensional problems with strong heterogeneities are presented to illustrate the performance of the two methods

optimal basins of attraction in randomly sparse neural network models the size of the basin of attraction for randomly sparse neural networks with optimal interactions is calculated for all values of the storage ratio alpha p c 2 where p is the number of random uncorrelated patterns and c is the connectivity the basin of attraction is finite while for alpha 0 42 the basin of attraction is almost 100

global observability of morse smale vectorfields abstract a procedure for investigating the global observability of a class of vectorfields is proposed the method derives from given qualitative properties of the flow it is shown that for morse smale flows local observability criteria can be tied together leading to a global theorem

higher order neural networks polya polynomials and fermi cluster diagrams the problem of controlling higher order interactions in neural networks is addressed with techniques commonly applied in the cluster analysis of quantum many particle systems for multineuron synaptic weights chosen according to a straightforward extension of the standard hebbian learning rule we show that higher order contributions to the stimulus felt by a given neuron can be readily evaluated via polya s combinatoric group theoretical approach or equivalently by exploiting a precise formal analogy with fermion diagrammatics

learning and generalization in multineuron interacting feed forward neural networks we consider learning and generalization in the multi interacting feed forward network model recently proposed by h o carmesin 1994 with an a priori definition of the net architecture based on symmetries presented by the function to be learnt we define a generalized hebb rule extend the maximum stability learning algorithm to multi interactions and obtain training and generalization curves for rules where different orders of synapses are not correlated the results obtained for the simple perceptron concerning the hebb rule and through replica calculations in the space of couplings may be straightforwardly adapted to multi interactions through a simple renormalization of the total number of independent couplings analytical and numerical simulation results are compared and show excellent agreement

differential algebraic equations indices and integral algebraic equations in a recent report hairer lubich and roche report ch 1211 dept de mathematiques universite de geneve geneve switzerland 1988 define the index of differential algebraic equations daes by considering the effect of perturbations of the equations on the solutions this index which will be called the perturbation index p i is one more than the number of derivatives of the perturbation that must appear in any estimate of the bound of the change in the solution an earlier form of index used by a number of authors including gear and petzold siam j numer anal 21 1984 pp 716 728 is determined by the number of differentiations of the daes that are required to generate an ordinary differential equation ode satisfied by the solution this will be called the differential index d i hairer lubich and roche give an example whose differential index is one and perturbation index is two and other examples where they are identical it will be shown that d i leqq p i leqq d i 1 and th

generalized state variable representation for a simplified crane description recent theoretical advances in nonlinear systems theory insist on the relevance of dynamics where control derivatives appear we discuss here a simplified model of a crane which exhibits such a description

chaotic dynamics of high order neural networks the dynamics of an extremely diluted neural network with high order synapses acting as corrections to the hopfield model is investigated the learning rules for the high order connections contain mixing of memories different from all the previous generalizations of the hopfield model the dynamics may display fixed points or periodic and chaotic orbits depending on the weight of the high order connections e the noise levelt and the network load defined as the ratio between the number of stored patterns and the mean connectivity per neuron p c as in the related fully connected case there is an optimal value of the weight e that improves the storage capacity of the system the capacity diverges

neural network modeling of associative memory beyond the hopfield model a number of neural network models in which fixed point and limit cycle attractors of the underlying dynamics are used to store and associatively recall information are described in the first class of models a hierarchical structure is used to store an exponentially large number of strongly correlated memories the second class of models uses limit cycles to store and retrieve individual memories a neurobiologically plausible network that generates low amplitude periodic variations of activity similar to the oscillations observed in electroencephalographic recordings is also described results obtained from analytic and numerical studies of the properties of these networks are discussed

multiconnected neural network models a generalisation of the hopfield model which includes interactions between p 2 ising spins is considered the exact storage capacity behaves as np 1 2 p 1 ln n when the number of nodes n is large in the limit p to infinity the thermodynamics of the model can be solved exactly without using the replica method at zero temperature a solution which is completely correlated with the input pattern exists for alpha alpha c where alpha c to infinity as p to infinity and this solution has lower energy than the spin glass solution if alpha alpha 1 1 4 ln 2 where the number of patterns n 2 alpha p np 1 for finite values of p the correlation with the input pattern is not complete for p 3 and 4 approximate values of alpha c and alpha 1 are obtained and for p to infinity the replica symmetric approximation gives alpha c approximately p 4 ln p

three unfinished works on the optimal storage capacity of networks the optimal storage properties of three different neural network models are studied for two of these models the architecture of the network is a perceptron with or j interactions whereas for the third model the output can be an arbitrary function of the inputs analytic bounds and numerical estimates of the optimal capacities and of the minimal fraction of errors are obtained for the first two models the third model can be solved exactly and the exact solution is compared to the bounds and to the results of numerical simulations used for the two other models