spike timing dependent plasticity the relationship to rate based learning for models with weight dynamics determined by a stable fixed point experimental evidence indicates that synaptic modification depends on the timing relationship between the presynaptic inputs and the output spikes that they generate in this letter results are presented for models of spike timing dependent plasticity stdp whose weight dynamics is determined by a stable fixed point four classes of stdp are identified on the basis of the time extent of their input output interactions the effect on the potentiation of synapses with different rates of input is investigated to elucidate the relationship of stdp with classical studies of long term potentiation and depression and rate based hebbian learning the selective potentiation of higher rate synaptic inputs is found only for models where the time extent of the input output interactions is input restricted i e restricted to time domains delimited by adjacent synaptic inputs and that have a time asymmetric learning window with a longer time constant for depression than for potentiation the analysis provides an account of learning dynamics determined by an input selective stable fixed point the effect of suppressive interspike interactions on stdp is also analyzed and shown to modify the synaptic dynamics

fixed point attractor analysis for a class of neurodynamics nearly all models in neural networks start from the assumption that the input output characteristic is a sigmoidal function on parameter space we present a systematic and feasible method for analyzing the whole spectrum of attractors all saturated all but one saturated all but twosaturated and so on of a neurodynamical system with a saturated sigmoidal function as its input output characteristic we present an argument that claims under a mild condition that only all saturated or all but one saturated attractors are observable for the neurodynamics for any given all saturated configuration all but one saturated configuration the article shows how to construct an exact parameter region r such that if and only if the parameters fall within r then is an attractor a fixed point of the dynamics the parameter region for an all saturated fixed point attractor is independent of the specific choice of a saturated sigmoidal function whereas for an all but one saturated fixed point it

an analysis of synaptic normalization in a general class of hebbian models in standard hebbian models of developmental synaptic plasticity synaptic normalization must be introduced in order to constrain synaptic growth and ensure the presence of activity dependent competitive dynamics in such models multiplicative normalization cannot segregate afferents whose patterns of electrical activity are positively correlated while subtractive normalization can it is now widely believed that multiplicative normalization cannot segregate positively correlated afferents in any hebbian model however we recently provided a counterexample to this belief by demonstrating that our own neurotrophic model of synaptic plasticity which can segregate positively correlated afferents can be reformulated as a nonlinear hebbian model with competition implemented through multiplicative normalization we now perform an analysis of a general class of hebbian models under general forms of synaptic normalization in particular we extract conditions on the forms of these rules that guarantee that such models possess a fixed point structure permitting the segregation of all but perfectly correlated afferents we find that the failure of multiplicative normalization to segregate positively correlated afferents in a standard hebbian model is quite atypical

hebbian learning of recurrent connections a geometrical perspective we show how a hopfield network with modifiable recurrent connections undergoing slow hebbian learning can extract the underlying geometry of an input space first we use a slow and fast analysis to derive an averaged system whose dynamics derives from an energy function and therefore always converges to equilibrium points the equilibria reflect the correlation structure of the inputs a global object extracted through local recurrent interactions only second we use numerical methods to illustrate how learning extracts the hidden geometrical structure of the inputs indeed multidimensional scaling methods make it possible to project the final connectivity matrix onto a euclidean distance matrix in a high dimensional space with the neurons labeled by spatial position within this space the resulting network structure turns out to be roughly convolutional the residual of the projection defines the nonconvolutional part of the connectivity which is minimized in the process finally we show how restricting the dimension of the space where the neurons live gives rise to patterns similar to cortical maps we motivate this using an energy efficiency argument based on wire length minimization finally we show how this approach leads to the emergence of ocular dominance or orientation columns in primary visual cortex via the self organization of recurrent rather than feedforward connections in addition we establish that the nonconvolutional or long range connectivity is patchy and is co aligned in the case of orientation learning

practical aspects of formulation and solution of moving mesh partial differential equations moving mesh partial differential equations mmpdes are used in the mmpde moving mesh method to generate adaptive moving meshes for the numerical solution of time dependent problems how mmpdes are formulated and solved is crucial to the efficiency and robustness of the method in this paper several practical aspects of formulating and solving mmpdes are studied they include spatial balance scaling invariance effective control of mesh concentration bounds on time steps multiple sub steps and two level mesh movement numerical results are also given c 2001 academic press

metric tensors for anisotropic mesh generation it has been amply demonstrated that significant improvements in accuracy and efficiency can be gained when a properly chosen anisotropic mesh is used in the numerical solution for a large class of problems which exhibit anisotropic solution features in practice an anisotropic mesh is commonly generated as a quasi uniform mesh in the metric determined by a tensor specifying the shape size orientation of elements thus it is crucial to choose an appropriate metric tensor for anisotropic mesh generation and adaptation in this paper we develop a general formula for the metric tensor for use in any spatial dimension the formulation is based on error estimates for polynomial preserving interpolation on simiplicial elements numerical results in two dimensions are presented to demonstrate the ability of the metric tensor to produce anisotropic meshes with correct mesh concentration and good overall quality the procedure developed in this paper for defining the metric tensor can also be applied to other types of error estimates

on neurodynamics with limiter function and linsker s developmental model the limiter function is used in many learning and retrieval models as the constraint controlling the magnitude of the weight or state vectors in this paper we developed a new method to relate the set of saturated fixed points to the set of system parameters of the models that use the limiter function and then as a case study applied this method to linsker s hebbian learning network we derived a necessary and sufficient condition to test whether a given saturated weight or state vector is stable or not for any given set of system parameters and used this condition to determine the whole regime in the parameter space over which the given state is stable this approach allows us to investigate the relative stability of the major receptive fields reported in linsker s simulations and to demonstrate the crucial role played by the synaptic density functions

hebbian learning and spiking neurons a correlation based hebbian learning rule at a spike level with millisecond resolution is formulated mathematically analyzed and compared with learning in a firing rate description the relative timing of presynaptic and postsynaptic spikes influences synaptic weights via an asymmetric learning window a differential equation for the learning dynamics is derived under the assumption that the time scales of learning and neuronal spike dynamics can be separated the differential equation is solved for a poissonian neuron model with stochastic spike arrival it is shown that correlations between input and output spikes tend to stabilize structure formation with an appropriate choice of parameters learning leads to an intrinsic normalization of the average weight and the output firing rate noise generates diffusion like spreading of synaptic weights s1063 651x 99 02804 4

hebb type dynamics is sufficient to account for the inverse magnification rule in cortical somatotopy the inverse magnification rule in cortical somatotopy is the experimentally derived inverse relationship between cortical magnification area of somatotopic map representing a unit area of skin surface and receptive field size area of restricted skin surface driving a cortical neuron we show by computer simulation of a simple multilayer model that hebb type synaptic modification subject to competitive constraints is sufficient to account for the inverse magnification rule

simple modification of oja rule limits l1 norm of weight vector and leads to sparse connectivity this letter describes a simple modification of the oja learning rule which asymptotically constrains the l1 norm of an input weight vector instead of the l2 norm as in the original rule this constraining is local as opposed to commonly used instant normalizations which require the knowledge of all input weights of a neuron to update each one of them individually the proposed rule converges to a weight vector that is sparser has more zero weights than the vector learned by the original oja rule with or without the zero bound which could explain the developmental synaptic pruning

comment on structural damage identification using assigned partial eigenstructure n a recent interesting paper cobb and liebst1 propose to identify the damage of a structure by using the partial measurements of the mode shapes without adding their missing parts the problem is that by doing so the measured modes cannot be orthogonalized2 3 and hence they cannot represent a linear dynamic structure a closed form solution for the missing measurements can be found in refs 2 6 in this way the modes can be orthogonalized in every step of the iterations hence the missing measurements are treated as hidden functions of the physical parameters of the structure as they really are to some extent the real measurements are also treated as functions of the parameters every iteration represents some possible linear dynamic structure and brings the model closer to the real structure

axonal processes and neural plasticity a reply we examine the claim that a class of sprouting and retraction models is mathematically equivalent to a fixed anatomy model we accept subject to important caveats a narrow mathematical equivalence of the energy functions in both classes of model we argue that this narrow equivalence of energy functions does not however entail equivalence of the models indeed the claim of complete model equivalence hides significant dynamical differences between the approaches which we discuss we also disagree that our work demonstrates that subtractive constraint enforcement is natural in fixed anatomy models

reduced representation by neural networks with restricted receptive fields model neural networks can perform dimensional reductions of input data sets using correlation based learning rules to adjust their weights simple hebbian learning rules lead to an optimal reduction at the single unit level but result in highly redundant network representations more complex rules designed to reduce or remove this redundancy can develop optimal principal component representations but they are not very compelling from a biological perspective neurons in biological networks have restricted receptive fields limiting their access to the input data space we find that within this restricted receptive field architecture simple correlation based learning rules can produce surprisingly efficient reduced representations when noise is present the size of the receptive fields can be optimally tuned to maximize the accuracy of reconstructions of input data from a reduced representation

the role of weight normalization in competitive learning the effect of different kinds of weight normalization on the outcome of a simple competitive learning rule is analyzed it is shown that there are important differences in the representation formed depending on whether the constraint is enforced by dividing each weight by the same amount divisive enforcement or subtracting a fixed amount from each weight subtractive enforcement for the divisive cases weight vectors spread out over the space so as to evenly represent typical inputs whereas for the subtractive cases the weight vectors tend to the axes of the space so as to represent extreme inputs the consequences of these differences are examined

variational mesh adaptation ii error estimates and monitor functions the key to the success of a variational mesh adaptation method is to define a proper monitor function which controls mesh adaptation in this paper we study the choice of the monitor function for the variational adaptive mesh method developed in the previous work j comput phys 174 2001 924 two types of monitor functions scalar matrix and non scalar matrix ones are defined based on asymptotic estimates of interpolation error obtained using the interpolation theory of finite element methods the choice of the adaptation intensity parameter is also discussed for each of these monitor functions asymptotic bounds on interpolation error are obtained for adaptive meshes that satisfy the regularity and equidistribution conditions two dimensional numerical results are given to verify the theoretical findings

the role of constraints in hebbian learning models of unsupervised correlation based hebbian synaptic plasticity are typically unstable either all synapses grow until each reaches the maximum allowed strength or all synapses decay to zero strength a common method of avoiding these outcomes is to use a constraint that conserves or limits the total synaptic strength over a cell we study the dynamic effects of such constraints two methods of enforcing a constraint are distinguished multiplicative and subtractive for otherwise linear learning rules multiplicative enforcement of a constraint results in dynamics that converge to the principal eigenvector of the operator determining unconstrained synaptic development subtractive enforcement in contrast typically leads to a final state in which almost all synaptic strengths reach either the maximum or minimum allowed value this final state is often dominated by weight configurations other than the principal eigenvector of the unconstrained operator multiplicative enforcement yields a graded receptive field in which most mutually correlated inputs are represented whereas subtractive enforcement yields a receptive field that is sharpened to a subset of maximally correlated inputs if two equivalent input populations e g two eyes innervate a common target multiplicative enforcement prevents their segregation ocular dominance segregation when the two populations are weakly correlated whereas subtractive enforcement allows segregation under these circumstances these results may be used to understand constraints both over output cells and over input cells a variety of rules that can implement constrained dynamics are discussed

grid adaption based on modified anisotropic diffusion equations formulated in the parametric domain a new grid adaption algorithm for problems in computational fluid dynamics is presented the basic equations are derived from a variational problem formulated in the parametric domain of the mapping that defines the existing grid modification of the basic equations provides desirable properties in boundary layers the resulting modified anisotropic diffusion equations are solved for the computational coordinates as functions of the parametric coordinates and these functions are numerically inverted numerical examples show that the algorithm is robust that shocks and boundary layers are well resolved on the adapted grid and that the flow solution becomes a globally smooth function of the computational coordinates

a three dimensional adaptive method based on the iterative grid redistribution in this paper we develop a three dimensional adaptive method based on iterative grid redistribution technique introduced in j comput phys 159 2000 246 the key step for the successful implementation is a fast algorithm for grid generation which is composed of solving the linear grid equation systems and an efficient method for inverting a map by computing iso surface intersections to carry out the three dimensional calculations the whole procedure is also parallelized on a pc cluster the improved and extended three dimensional adaptive method is applied to solve pdes with singular solutions that have three dimensional structures numerical experiments have demonstrated the method s effectiveness

removing time variation with the anti hebbian differential synapse i describe a local synaptic learning rule that can be used to remove the effects of certain types of systematic temporal variation in the inputs to a unit according to this rule changes in synaptic weight result from a conjunction of short term temporal changes in the inputs and the output formally this is like the differential rule proposed by klopf 1986 and kosko 1986 except for a change of sign which gives it an anti hebbian character by itself this rule is insufficient a weight conservation condition is needed to prevent the weights from collapsing to zero and some further constraint implemented here by a biasing term to select particular sets of weights from the subspace of those which give minimal variation as an example i show that this rule will generate center surround receptive fields that remove temporally varying linear gradients from the inputs

constrained optimization for neural map formation a unifying framework for weight growth and normalization computational models of neural map formation can be considered on at least three different levels of abstraction detailed models including neural activity dynamics weight dynamics that abstract from the neural activity dynamics by an adiabatic approximation and constrained optimization from which equations governing weight dynamics can be derived constrained optimization uses an objective function from which a weight growth rule can be derived as a gradient flow and some constraints from which normalization rules are derived in this article we present an example of how an optimization problem can be derived from detailed nonlinear neural dynamics a systematic investigation reveals how different weight dynamics introduced previously can be derived from two types of objective function terms and two types of constraints this includes dynamic link matching as a special case of neural map formation we focus in particular on the role of coordinate transformations to derive different weight dynamics from the same optimization problem several examples illustrate how the constrained optimization framework can help in understanding generating and comparing different models of neural map formation the techniques used in this analysis may also be useful in investigating other types of neural dynamics