spike timing dependent plasticity the relationship to rate based learning for models with weight dynamics determined by a stable fixed point experimental evidence indicates that synaptic modification depends on the timing relationship between the presynaptic inputs and the output spikes that they generate in this letter results are presented for models of spike timing dependent plasticity stdp whose weight dynamics is determined by a stable fixed point four classes of stdp are identified on the basis of the time extent of their input output interactions the effect on the potentiation of synapses with different rates of input is investigated to elucidate the relationship of stdp with classical studies of long term potentiation and depression and rate based hebbian learning the selective potentiation of higher rate synaptic inputs is found only for models where the time extent of the input output interactions is input restricted i e restricted to time domains delimited by adjacent synaptic inputs and that have a time asymmetric learning window with a longer time constant for depression than for potentiation the analysis provides an account of learning dynamics determined by an input selective stable fixed point the effect of suppressive interspike interactions on stdp is also analyzed and shown to modify the synaptic dynamics

neural associative memory and the willshaw palm probability distribution previous asymptotic analyses of binary neural associative networks of willshaw or steinbuch type relied on a binomial approximation of the neurons dendritic potentials this approximation has been proven to be good only if the stored patterns are extremely sparse for example when the mean number of active units k per pattern vector scales with the logarithm of the vector size n recent promising results concerning storage capacity and retrieval efficiency for larger pattern activities k log n have been doubted because here the binomial approximation can lead to a massive overestimation of performance in this work i compute and characterize the exact willshaw palm distribution of the dendritic potentials for hetero association auto association and fixed and random pattern activity comparing the raw and central moments of the willshaw palm distribution to the moments of the corresponding binomial probability reveals that asymptotically the binomial approximation becomes exact for almost any su

extracting oscillations neuronal coincidence detection with noisy periodic spike input how does a neuron vary its mean output firing rate if the input changes from random to oscillatory coherent but noisy activity what are the critical parameters of the neuronal dynamics and input statistics to answer these questions we investigate the coincidence detection properties of an integrate and fire neuron we derive an expression indicating how coincidence detection depends on neuronal parameters specifically we show how coincidence detection depends on the shape of the postsynaptic response function the number of synapses and the input statistics and we demonstrate that there is an optimal threshold our considerations can be used to predict from neuronal parameters whether and to what extent a neuron can act as a coincidence detector and thus can convert a temporal code into a rate code

sqp methods for solving optimal control problems with control and state constraints adjoint variables sensitivity analysis and real time control parametric nonlinear optimal control problems subject to control and state constraints are studied two discretization methods are discussed that transcribe optimal control problems into nonlinear programming problems for which sqp methods provide efficient solution methods it is shown that sqp methods can be used also for a check of second order sufficient conditions and for a postoptimal calculation of adjoint variables in addition sqp methods lead to a robust computation of sensitivity differentials of optimal solutions with respect to perturbation parameters numerical sensitivity analysis is the basis for real time control approximations of perturbed solutions which are obtained by evaluating a first order taylor expansion with respect to the parameter the proposed numerical methods are illustrated by the optimal control of a low thrust satellite transfer to geosynchronous orbit and a complex control problem from aquanautics the examples illustrate the robustness accuracy and efficiency of the proposed numerical algorithms

generalized insertion region guides for delaunay mesh refinement mesh generation by delaunay refinement is a widely used technique for constructing guaranteed quality triangular and tetrahedral meshes the quality guarantees are usually provided in terms of the bounds on circumradius to shortest edge ratio and on the grading of the resulting mesh traditionally circumcenters of skinny elements and middle points of boundary faces and edges are used for the positions of inserted points however recently variations of the traditional algorithms are being proposed that are designed to achieve certain optimization objectives by inserting new points in neighborhoods of the center points in this paper we propose a general approach to the selection of point positions by defining one two and three dimensional selection regions such that any point insertion strategy based on these regions is automatically endowed with the theoretical guarantees proven here in particular for the input models defined by planar linear complexes under the assumption that no input angle is le

memory capacity for sequences in a recurrent network with biological constraints the ca3 region of the hippocampus is a recurrent neural network that is essential for the storage and replay of sequences of patterns that represent behavioral events here we present a theoretical framework to calculate a sparsely connected network s capacity to store such sequences as in ca3 only a limited subset of neurons in the network is active at any one time pattern retrieval is subject to error and the resources for plasticity are limited our analysis combines an analytical mean field approach stochastic dynamics and cellular simulations of a time discrete mcculloch pitts network with binary synapses to maximize the number of sequences that can be stored in the network we concurrently optimize the number of active neurons that is pattern size and the firing threshold we find that for one step associations i e minimal sequences the optimal pattern size is inversely proportional to the mean connectivity c whereas the optimal firing threshold is independent of the connectivity if the number of synapses per neuron is fixed the maximum number p of stored sequences in a sufficiently large nonmodular network is independent of its number n of cells on the other hand if the number of synapses scales as the network size to the power of 3 2 the number of sequences p is proportional to n in other words sequential memory is scalable furthermore we find that there is an optimal ratio r between silent and nonsilent synapses at which the storage capacity p c 1 r n assumes a maximum for long sequences the capacity of sequential memory is about one order of magnitude below the capacity for minimal sequences but otherwise behaves similar to the case of minimal sequences in a biologically inspired scenario the information content per synapse is far below theoretical optimality suggesting that the brain trades off error tolerance against information content in encoding sequential memories

learning and retrieval in attractor neural networks above saturation the authors investigate neural networks in the range of parameters when the ground state energy is positive namely when a synaptic matrix which satisfies all the desired constraints cannot be found by the learning algorithm in particular they calculate the typical distribution functions of local stabilities obtained for a number of algorithms in this region these functions are used to investigate the retrieval properties as reflected by the size of the basins of attraction this is done analytically in sparsely connected networks and numerically in fully connected networks the main conclusion is that the retrieval behaviour of attractor neural networks can be improved by learning above saturation

a neural network for storing individual patterns in limit cycles a neural network model in which individual memories are stored in limit cycles is studied analytically and numerically in this model there are two kinds of interactions a hopfield like term that tends to stabilize the system in a memorized state and a second term with a time delay that acts to induce transitions between a memorized state and its complement state for a proper choice of the values of the parameters this model exhibits limit cycle behaviour in which the overlap with a target pattern oscillates in time an asymmetrically diluted version of the model is studied analytically in the limit of extreme dilution we find that the model with cycles performs better than a similarly diluted version of the hopfield model the performance of the fully connected model is studied by numerical simulations we find a behaviour qualitatively similar to that of the dilute model the model with cycles is found to perform better than the hopfield model as a pattern classifier if the memory loading level and the degree of corruption of the input patterns are high

generating well shaped d dimensional delaunay meshes a d dimensional simplicial mesh is a delaunay triangulation if the circumsphere of each of its simplices does not contain any vertices inside a mesh is well shaped if the maximum aspect ratio of all its simplices is bounded from above by a constant it is a long term open problem to generate well shaped d dimensional delaunay meshes for a given polyhedral domain in this paper we present a refinement based method that generates well shaped d dimensional delaunay meshes for any plc domain with no small input angles furthermore we show that the generated well shaped mesh has o n d simplices where n is the smallest number of d simplices of any almost good meshes for the same domain here a mesh is almost good if each of its simplices has a bounded circumradius to the shortest edge length ratio

optimised network for sparsely coded patterns the performance of attractor neural networks storing sparsely coded patterns has been shown to be greatly improved on shifting from the 1 1 representation of neural states to the 0 1 representation here the authors show that when this shift is considered as a special case of the transformation of the dynamical variables which depends on a continuous parameter the value of the parameter can be chosen to improve the performance of the network even further for every value of the bias in the patterns

convergence under dynamical thresholds with delays necessary and sufficient conditions are obtained for the existence of a globally asymptotically stable equilibrium of a class of delay differential equations modeling the action of a neuron with dynamical threshold effects

survey constrained model predictive control stability and optimality model predictive control is a form of control in which the current control action is obtained by solving at each sampling instant a finite horizon open loop optimal control problem using the current state of the plant as the initial state the optimization yields an optimal control sequence and the first control in this sequence is applied to the plant an important advantage of this type of control is its ability to cope with hard constraints on controls and states it has therefore been widely applied in petro chemical and related industries where satisfaction of constraints is particularly important because efficiency demands operating points on or close to the boundary of the set of admissible states and controls in this review we focus on model predictive control of constrained systems both linear and nonlinear and discuss only briefly model predictive control of unconstrained nonlinear and or time varying systems we concentrate our attention on research dealing with stability and optimality in these areas the subject has developed in our opinion to a stage where it has achieved sufficient maturity to warrant the active interest of researchers in nonlinear control we distill from an extensive literature essential principles that ensure stability and use these to present a concise characterization of most of the model predictive controllers that have been proposed in the literature in some cases the finite horizon optimal control problem solved on line is exactly equivalent to the same problem with an infinite horizon in other cases it is equivalent to a modified infinite horizon optimal control problem in both situations known advantages of infinite horizon optimal control accrue

three dimensional unstructured mesh generation part 1 fundamental aspects of triangulation and point creation abstract the present paper introduces an alternative approach for delaunay triangulation in which the triangulation is mapped from an equivalent convex hull in a higher dimension furthermore some fundamental aspects of a point creation algorithm for unstructured mesh generation have been addressed based on numerical experiments a point spacing tensor and point insertion criterion have been introduced aiming to deal with anisotropic meshes

dynamic hebbian learning in adaptive frequency oscillators nonlinear oscillators are widely used in biology physics and engineering for modeling and control they are interesting because of their synchronization properties when coupled to other dynamical systems in this paper we propose a learning rule for oscillators which adapts their frequency to the frequency of any periodic or pseudo periodic input signal learning is done in a dynamic way it is part of the dynamical system and not an offline process an interesting property of our model is that it is easily generalizable to a large class of oscillators from phase oscillators to relaxation oscillators and strange attractors with a generic learning rule one major feature of our learning rule is that the oscillators constructed can adapt their frequency without any signal processing or the need to specify a time window or similar free parameters all the processing is embedded in the dynamics of the adaptive oscillator the convergence of the learning is proved for the hopf oscillator then numerical experiments are carried out to explore the learning capabilities of the system finally we generalize the learning rule to non harmonic oscillators like relaxation oscillators and strange attractors

linear stability analysis of retrieval state in associative memory neural networks of spiking neurons we study associative memory neural networks of the hodgkin huxley type of spiking neurons in which multiple periodic spatiotemporal patterns of spike timing are memorized as limit cycle type attractors in encoding the spatiotemporal patterns we assume the spike timing dependent synaptic plasticity with the asymmetric time window analysis for periodic solution of retrieval state reveals that if the area of the negative part of the time window is equivalent to the positive part then crosstalk among encoded patterns vanishes phase transition due to the loss of the stability of periodic solution is observed when we assume fast function for direct interaction among neurons in order to evaluate the critical point of this phase transition we employ floquet theory in which the stability problem of the infinite number of spiking neurons interacting with function is reduced to the eigenvalue problem with the finite size of matrix numerical integration of the single body dynamics yields the explicit value of the matrix which enables us to determine the critical point of the phase transition with a high degree of precision

a non markovian random walk underlies a stochastic model of spike timing dependent plasticity a stochastic model of spike timing dependent plasticity stdp proposes that spike timing influences the probability but not the amplitude of synaptic strength change at single synapses the classic biphasic stdp profile emerges as a spatial average over many synapses presented with a single spike pair or as a temporal average over a single synapse presented with many spike pairs we have previously shown that the model accounts for a variety of experimental data including spike triplet results and has a number of desirable theoretical properties including being entirely self stabilizing in all regions of parameter space our earlier analyses of the model have employed cumbersome spike to spike averaging arguments to derive results here we show that the model can be reformulated as a non markovian random walk in synaptic strength the step sizes being fixed as postulated this change of perspective greatly simplifies earlier calculations by integrating out the proposed switch mechanism by which changes in strength are driven and instead concentrating on the changes in strength themselves moreover this change of viewpoint is generative facilitating further calculations that would be intractable if not impossible with earlier approaches we prepare the machinery here for these later calculations but also briefly indicate how this machinery may be used by considering two particular applications

lossless convexification of non convex optimal control problems for state constrained linear systems this paper analyzes a class of finite horizon optimal control problems with mixed non convex and convex control constraints and linear state constraints a convex relaxation of the problem is proposed and it is proved that a solution of the relaxed problem is also a solution of the original problem this process is called lossless convexification and its generalization for problems with state constraints is the primary contribution of the paper doing so enables the use of interior point methods of convex optimization to obtain global optimal solutions of the original non convex problem the approach is also demonstrated on example problems

global convergence of a nonsmooth newton method for control state constrained optimal control problems we investigate a nonsmooth newton method for the numerical solution of optimal control problems subject to mixed control state constraints the necessary conditions are stated in terms of a local minimum principle by use of the fischer burmeister function the local minimum principle is transformed into an equivalent nonlinear and nonsmooth equation in appropriate banach spaces this nonlinear and nonsmooth equation is solved by a nonsmooth newton s method we prove the global convergence and the locally superlinear convergence under certain regularity conditions the globalized method is based on the minimization of the squared residual norm numerical examples for the rayleigh problem conclude the article

three dimensional unstructured mesh generation part 2 surface meshes abstract this paper deals with surface patches and surface meshing triangular and quadrilateral patches in linear and quadratic forms and non uniform rational b spline nurbs patch have been used to define surface geometry from the real application of view data conversion between the mesh generator and some existing cad packages has been considered as a result of this study converters from a graphics data standard and several cad data formats have been implemented the quality improvement of surface meshes has been discussed in terms of parametric plane stretching diagonal swapping and smoothing procedures furthermore a scheme of visual representation is introduced to utilize colour effect in validating the geometry and its surface meshes

brief paper optimal control problems with a continuous inequality constraint on the state and the control we consider an optimal control problem with a nonlinear continuous inequality constraint both the state and the control are allowed to appear explicitly in this constraint by discretizing the control space and applying a novel transformation a corresponding class of semi infinite programming problems is derived a solution of each problem in this class furnishes a suboptimal control for the original problem furthermore we show that such a solution can be computed efficiently using a penalty function method on the basis of these two ideas an algorithm that computes a sequence of suboptimal controls for the original problem is proposed our main result shows that the cost of these suboptimal controls converges to the minimum cost for illustration an example problem is solved

pattern retrieval in an asymmetric neural network with embedded limit cycles a neural network model in which the connection matrix is formed by summing direct products of successive patterns in cyclic sequences is studied it is found that the noise elimination performance of this model can be better than that of an autocorrelation model due to a reduced tendency for trapping in spurious attractors

storage capacity of neural networks effect of the fluctuations of the number of active neurons per memory the storage capacity in an attractor neural network with excitatory couplings is shown to depend not only on the fraction of active neurons per pattern or coding rate but also on the fluctuations around this value in the thermodynamical limit the capacity is calculated in the case of exactly the same number of active neurons in every pattern for every coding level the capacity is increased with respect to the case of random patterns results are supported by numerical simulations done with an exhaustive search algorithm and partly solve in the sparse coding limit the paradox of the discrepancy of the capacity of the willshaw model with optimal capacity

fast generation of 2 d node distributions for mesh free pde discretizations abstract many applications require that nodes be scattered locally quasi uniformly within 2 d regions or on curved surfaces in 3 d space while obeying some prescribed spatially varying density function this study focuses on creating variable density node layouts that are suitable for mesh free discretizations of pdes another application is dithering to simulate half tone images the method is an advancing front type scheme inspired by the physical process of dropping of variable sized grains into a bucket

what can a neuron learn with spike timing dependent plasticity spiking neurons are very flexible computational modules which can implement with different values of their adjustable synaptic parameters an enormous variety of different transformations f from input spike trains to output spike trains we examine in this letter the question to what extent a spiking neuron with biologically realistic models for dynamic synapses can be taught via spike timing dependent plasticity stdp to implement a given transformation f we consider a supervised learning paradigm where during training the output of the neuron is clamped to the target signal teacher forcing the well known perceptron convergence theorem asserts the convergence of a simple supervised learning algorithm for drastically simplified neuron models mcculloch pitts neurons we show that in contrast to the perceptron convergence theorem no theoretical guarantee can be given for the convergence of stdp with teacher forcing that holds for arbitrary input spike patterns on the other hand we prove that average case versions of the perceptron convergence theorem hold for stdp in the case of uncorrelated and correlated poisson input spike trains and simple models for spiking neurons for a wide class of cross correlation functions of the input spike trains the resulting necessary and sufficient condition can be formulated in terms of linear separability analogously as the well known condition of learnability by perceptrons however the linear separability criterion has to be applied here to the columns of the correlation matrix of the poisson input we demonstrate through extensive computer simulations that the theoretically predicted convergence of stdp with teacher forcing also holds for more realistic models for neurons dynamic synapses and more general input distributions in addition we show through computer simulations that these positive learning results hold not only for the common interpretation of stdp where stdp changes the weights of synapses but also for a more realistic interpretation suggested by experimental data where stdp modulates the initial release probability of dynamic synapses

on curvature element size control in metric surface mesh generation summary a new procedure is suggested for controlling the element size distribution of surface meshes during automatic adaptive surface mesh generation in order to ensure that the geometry of the surface can be accurately captured the curvature properties of the surface are first analyzed based on the principal curvatures and principal directions of the surface the curvature element size requirement is defined in form of a metric tensor field this element size controlling metric tensor field which can either be isotopic or anisotopic depends on the user requirement is then employed to control the element size distribution during mesh generation the suggested procedure is local adaptive and can be easily used with many parametric surface mesh generators as the proposed scheme defines the curvature element size requirement in an implicit manner it can be combined with any other user defined element size specification using the standard metric intersection procedure this eventually leads to a simple implementation procedure and a high computational efficiency numerical examples indicate that the new procedure can effectively control the element size of surface meshes in the cost of very little additional computational effort

robust receding horizon control of constrained nonlinear systems we present a method for the construction of a robust dual mode receding horizon controller which can be employed for a wide class of nonlinear systems with state and control constraints and model error the controller is dual mode in a neighborhood of the origin the control action is generated by a linear feedback controller designed for the linearized system outside this neighborhood receding horizon control is employed existing receding horizon controllers for nonlinear continuous time systems which are guaranteed to stabilize the nonlinear system to which they are applied require the exact solution at every instant of an optimal control problem with terminal equality constraints these requirements are considerably relaxed in the dual mode receding horizon controller presented in this paper stability is achieved by imposing a terminal inequality rather than an equality constraint only approximate minimization is required a variable time horizon is permitted robustness is achieved by employing conservative state and stability constraint sets thereby permitting a margin of error the resultant dual mode controller requires considerably less online computation than existing receding horizon controllers for nonlinear constrained systems

a chebyshev technique for solving nonlinear optimal control problems a numerical technique for solving nonlinear optimal control problems is introduced the state and control variables are expanded in the chebyshev series and an algorithm is provided for approximating the system dynamics boundary conditions and performance index application of this method results in the transformation of differential and integral expressions into systems of algebraic or transcendental expressions in the chebyshev coefficients the optimum condition is obtained by applying the method of constrained extremum for linear quadratic optimal control problems the state and control variables are determined by solving a set of linear equations in the chebyshev coefficients applicability is illustrated with the minimum time and maximum radius orbit transfer problems

evolution of network structure by temporal learning we study the effect of learning dynamics on network topology a network of discrete dynamical systems is considered for this purpose and the coupling strengths are made to evolve according to a temporal learning rule that is based on the paradigm of spike time dependent plasticity this incorporates necessary competition between different edges the final network we obtain is robust and has a broad degree distribution

anisotropic centroidal voronoi tessellations and their applications in this paper we introduce a novel definition of the anisotropic centroidal voronoi tessellation acvt corresponding to a given riemann metric tensor a directional distance function is used in the definition to simplify the computation we provide algorithms to approximate the acvt using the lloyd iteration and the construction of anisotropic delaunay triangulation under the given riemannian metric the acvt is applied to the optimization of two dimensional anisotropic delaunay triangulation to the generation of surface cvt and high quality triangular mesh on general surfaces various numerical examples demonstrating the effectiveness of the proposed method are presented

gardner derrida neural networks with correlated patterns the storage properties of an optimal neural network with correlated patterns is studied allowing for a finite fraction of errors determined by the gardner derrida cost function a discontinuity in the probability distribution of the local stabilities is seen as a drastic decrease in the minimal fraction of errors there is also an enlargement of the domain in which the replica symmetric solution is stable allowing for higher storage capacities

stability against fluctuations scaling bifurcations and spontaneous symmetry breaking in stochastic models of synaptic plasticity in stochastic models of synaptic plasticity based on a random walk the control of fluctuations is imperative we have argued that synapses could act as low pass filters filtering plasticity induction steps before expressing a step change in synaptic strength earlier work showed in simulation that such a synaptic filter tames fluctuations very well leading to patterns of synaptic connectivity that are stable for long periods of time here we approach this problem analytically we explicitly calculate the lifetime of meta stable states of synaptic connectivity using a fokker planck formalism in order to understand the dependence of this lifetime on both the plasticity step size and the filtering mechanism we find that our analytical results agree very well with simulation results despite having to make two approximations our analysis reveals however a deeper significance to the filtering mechanism and the plasticity step size we show that a filter scales the step size into a smaller effective step size this scaling suggests that the step size may itself play the role of a temperature parameter so that a filter cools the dynamics thereby reducing the influence of fluctuations using the master equation we explicitly demonstrate a bifurcation at a critical step size confirming this interpretation at this critical point spontaneous symmetry breaking occurs in the class of stochastic models of synaptic plasticity that we consider

quality mesh generation in higher dimensions we consider the problem of triangulating a d dimensional region our mesh generation algorithm called qmg is a quadtree based algorithm that can triangulate any polyhedral region including nonconvex regions with holes furthermore our algorithm guarantees a bounded aspect ratio triangulation provided that the input domain itself has no sharp angles finally our algorithm is guaranteed never to overrefine the domain in the sense that the number of simplices produced by qmg is bounded above by a factor times the number produced by any competing algorithm where the factor depends on the aspect ratio bound satisfied by the competing algorithm the qmg algorithm has been implemented in c and is used as a mesh generator for the finite element method

closed form expressions for the moments of the binomial probability distribution this work develops closed form expressions for the raw and central moments of the binomial probability distribution for this i first derive a recursive formula for the raw moments from the moment generating function then it is shown that the recursion involved is essentially the same as for the stirling numbers of the second kind from this fact it is then possible to derive the closed formulae finally i discuss an application of these formulae to the analysis of neural associative memory

training the integrate and fire model with the informax principle i in terms of the informax principle and the input output relationship of the integrate and fire if model if neuron learning rules are developed for supervised learning and with uniform weight of synapses the theoretically tractable case we show that the derived learning rule is stable and the stable state is unique for unsupervised learning within physiologically reasonable parameter regions both long term potentiation ltp and long term depression ltd could happen when the inhibitory input is weak but ltd cannot be observed when inhibitory input is strong enough when both ltp and ltd occur ltd is observable when the output of the postsynaptic neuron is faster than pre synaptic inputs otherwise ltp is observable as observed in recent experiments learning rules of general cases are also studied and numerical examples show that the derived learning rule tends to equalize the contribution of different inputs to the output firing rates

self organizing dual coding based on spike time dependent plasticity it has been a matter of debate how firing rates or spatiotemporal spike patterns carry information in the brain recent experimental and theoretical work in part showed that these codes especially a population rate code and a synchronous code can be dually used in a single architecture however we are not yet able to relate the role of firing rates and synchrony to the spatiotemporal structure of inputs and the architecture of neural networks in this article we examine how feedforward neural networks encode multiple input sources in the firing patterns we apply spiketime dependent plasticity as a fundamental mechanism to yield synaptic competition and the associated input filtering we use the fokker planck formalism to analyze the mechanism for synaptic competition in the case of multiple inputs which underlies the formation of functional clusters in downstream layers in a self organizing manner depending on the types of feedback coupling and shared connectivity clusters are independently engaged in population rate coding or synchronous coding or they interact to serve as input filters classes of dual codings and functional roles of spike time dependent plasticity are also discussed

dynamics of temporal learning rules the changes of synaptic strength are analyzed on two time scales the fast local field dynamics and the slow synaptic modification dynamics the fast dynamics are determined by the synaptic strengths and background noise in the system the slow dynamics are determined by the functional form of a temporal learning rule temporal learning rules are defined to be functions yielding state dependent changes in synaptic strengths depending on the timing of pre and postsynaptic states in the network the evolution of local field dynamics that result from various learning rules are analyzed for a stochastic discrete time neural model with no relative refractory period that receives a series of delayed adaptive inputs a fixed point is found in the learning dynamics and conditions for two types of instabilities are analyzed four universality classes of dynamics are found that are independent of the details of the temporal learning rules examples are given of biological systems in which these temporal learning rules have been identified and their functional consequences are discussed

information capacity of a perceptron the authors study the information storage capacity of a simple perceptron in the error regime for random unbiased patterns the geometrical analysis gives a logarithmic dependence for the information content in the asymptotic limit in this case the statistical physics approach when used at the simplest level of replica theory does not give satisfactory results however for perceptrons with finite stability the information content can be simply calculated with statistical physics methods in a region above the critical storage level for biased as well as for unbiased patterns

random cellular froths in spaces of any dimension and curvature froth is a random partition of a d dimensional space by cells this assembly of cells obeys two fundamental laws euler s relation and the condition of maximum vertex figure imposed by geometry and by topological stability respectively these two conditions generate a set of relations between the variables that fully characterize the system topologically the number of degrees of freedom of the system and a set of useful independent variables the even valences have been found the influence of the space dimension and curvature on the range of variability of these valences is discussed and up to d 5 the regions in valence space corresponding to differently curved froths are calculated explicitly

random walks for spike timing dependent plasticity random walk methods are used to calculate the moments of negative image equilibrium distributions in synaptic weight dynamics governed by spike timing dependent plasticity stdp the neural architecture of the model is based on the electrosensory lateral line lobe ell of mormyrid electric fish which forms a negative image of the reafferent signal from the fish s own electric discharge to optimize detection of sensory electric fields of particular behavioral importance to the fish is the variance of the equilibrium postsynaptic potential in the presence of noise which is determined by the variance of the equilibrium weight distribution recurrence relations are derived for the moments of the equilibrium weight distribution for arbitrary postsynaptic potential functions and arbitrary learning rules for the case of homogeneous network parameters explicit closed form solutions are developed for the covariances of the synaptic weight and postsynaptic potential distributions

fully anisotropic goal oriented mesh adaptation for 3d steady euler equations this paper studies the coupling between anisotropic mesh adaptation and goal oriented error estimate the former is very well suited to the control of the interpolation error it is generally interpreted as a local geometric error estimate on the contrary the latter is preferred when studying approximation errors for pdes it generally involves non local error contributions consequently a full and strong coupling between both is hard to achieve due to this apparent incompatibility this paper shows how to achieve this coupling in three steps first a new a priori error estimate is proved in a formal framework adapted to goal oriented mesh adaptation for output functionals this estimate is based on a careful analysis of the contributions of the implicit error and of the interpolation error second the error estimate is applied to the set of steady compressible euler equations which are solved by a stabilized galerkin finite element discretization a goal oriented error estimation is derived it involves the interpolation error of the euler fluxes weighted by the gradient of the adjoint state associated with the observed functional third rewritten in the continuous mesh framework the previous estimate is minimized on the set of continuous meshes thanks to a calculus of variations the optimal continuous mesh is then derived analytically thus it can be used as a metric tensor field to drive the mesh adaptation from a numerical point of view this method is completely automatic intrinsically anisotropic and does not depend on any a priori choice of variables to perform the adaptation 3d examples of steady flows around supersonic and transsonic jets are presented to validate the current approach and to demonstrate its efficiency

hebbian learning and spiking neurons a correlation based hebbian learning rule at a spike level with millisecond resolution is formulated mathematically analyzed and compared with learning in a firing rate description the relative timing of presynaptic and postsynaptic spikes influences synaptic weights via an asymmetric learning window a differential equation for the learning dynamics is derived under the assumption that the time scales of learning and neuronal spike dynamics can be separated the differential equation is solved for a poissonian neuron model with stochastic spike arrival it is shown that correlations between input and output spikes tend to stabilize structure formation with an appropriate choice of parameters learning leads to an intrinsic normalization of the average weight and the output firing rate noise generates diffusion like spreading of synaptic weights s1063 651x 99 02804 4

method of evolving junctions a new approach to optimal control with constraints abstract we propose a new strategy called method of evolving junctions mej to compute the solutions for a class of optimal control problems with constraints on both state and control variables our main idea is that by leveraging the geometric structures of the optimal solutions we recast the infinite dimensional optimal control problem into an optimization problem depending on a finite number of points called junctions then using a modified gradient flow method whose dimension can change dynamically we find local solutions for the optimal control problem we also employ intermittent diffusion a global optimization method based on stochastic differential equations to obtain the global optimal solution we demonstrate via a numerical example that mej can effectively solve path planning problems in dynamical environments

dynamical partitions of space in any dimension topologically stable cellular partitions of d dimensional spaces are studied a complete statistical description of the average structural properties of such partitions is given in terms of a sequence of or variables for d even or odd these variables are the average coordination numbers of the 2k dimensional polytopes which make up the cellular structure a procedure to produce d dimensional space partitions through cell division and cell coalescence transformations is presented classes of structures which are invariant under these transformations are found and the average properties of such structures are illustrated homogeneous partitions are constructed and compared with the known structures obtained by voronoi partitions and sphere packings in high dimensions

storage capacity of a diluted neural network with ising couplings the capacity for storing random patterns in a diluted neural network is determined following the method of gardner 1988 the non zero coupling coefficients are restricted to take on the ising values or 1 different degrees of dilution are considered the maximum values alpha c 1 17 for the storage capacity is obtained when each neuron is connected to 63 of the other neurons

a three dimensional parametric mesher with surface boundary layer capability abstract a novel parametric surface meshing technique is presented its distinctive feature relies on successive approximations of the cad geometry through a hierarchical process where geometric information is gathered incrementally a detailed review of zero and first order surface approximations and their impact on parametric surface meshing algorithms is performed the proposed approach emphasizes the use of three dimensional information in order to be as independent as possible of the parametrization to overcome limitations of meshing purely in the parametric plane the presented technique includes semi structured boundary layer surface mesh generation which is a critical capability for accurate solutions to flows around geometries that have leading edge features numerous examples illustrate the method s robustness and ability to high quality meshes for complex cad geometries

brief paper lossless convexification of a class of optimal control problems with non convex control constraints we consider a class of finite time horizon optimal control problems for continuous time linear systems with a convex cost convex state constraints and non convex control constraints we propose a convex relaxation of the non convex control constraints and prove that the optimal solution of the relaxed problem is also an optimal solution for the original problem which is referred to as the lossless convexification of the optimal control problem the lossless convexification enables the use of interior point methods of convex optimization to obtain globally optimal solutions of the original non convex optimal control problem the solution approach is demonstrated on a number of planetary soft landing optimal control problems

duality between learning machines a bridge between supervised and unsupervised learning we exhibit a duality between two perceptrons that allows us to compare the theoretical analysis of supervised and unsupervised learning tasks the first perceptron has one output and is asked to learn a classification of p patterns the second dual perceptron has p outputs and is asked to transmit as much information as possible on a distribution of inputs we show in particular that the maximum information that can be stored in the couplings for the supervised learning task is equal to the maximum information that can be transmitted by the dual perceptron

statistical mechanics of reward modulated learning in decision making networks the neural substrates of decision making have been intensively studied using experimental and computational approaches alternative choice tasks accompanying reinforcement have often been employed in investigations into decision making choice behavior has been empirically found in many experiments to follow herrnstein s matching law a number of theoretical studies have been done on explaining the mechanisms responsible for matching behavior various learning rules have been proved in these studies to achieve matching behavior as a steady state of learning processes the models in the studies have consisted of a few parameters however a large number of neurons and synapses are expected to participate in decision making in the brain we investigated learning behavior in simple but large scale decision making networks we considered the covariance learning rule which has been demonstrated to achieve matching behavior as a steady state loewenstein seung 2006 we analyzed model behavior in a thermodynamic limit where the number of plastic synapses went to infinity by means of techniques of the statistical mechanics we can derive deterministic differential equations in this limit for the order parameters which allow an exact calculation of the evolution of choice behavior as a result we found that matching behavior cannot be a steady state of learning when the fluctuations in input from individual sensory neurons are so large that they affect the net input to value encoding neurons this situation naturally arises when the synaptic strength is sufficiently strong and the excitatory input and the inhibitory input to the value encoding neurons are balanced the deviation from matching behavior is caused by increasing variance in the input potential due to the diffusion of synaptic efficacies this effect causes an undermatching phenomenon which has been often observed in behavioral experiments

precise capacity analysis in binary networks with multiple coding level inputs we compute retrieval probabilities as a function of pattern age for networks with binary neurons and synapses updated with the simple hebbian learning model studied in amit and fusi 1994 the analysis depends on choosing a neural threshold that enables patterns to stabilize in the neural dynamics in contrast to most earlier work where selective neurons for each pattern are drawn independently with fixed probability f here we analyze the situation where f is drawn from some distribution on a range of coding levels in order to set a workable threshold in this setting it is necessary to introduce a simple inhibition in the neural dynamics whose magnitude depends on the total activity of the network proper choice of the threshold depends on the value of the covariances between the synapses for which we provide an explicit formula retrieval probabilities depend on the distribution of the fields induced by a learned pattern we show that the field induced by the first learned pattern evolves as a markov chain during subsequent learning epochs leading to a recursive formula for the distribution alternatively the distribution can be computed using a normal approximation which involves the value of the synaptic covariances capacity is computed as the sum of the retrival probabilities over all ages we show through simulation that the chosen threshold enables retrieval with asynchronous dynamics even in the presence of significant noise in the initial state of the pattern the computed probabilities with both methods are shown to be very close to probabilities estimated from simulation the analysis is extended to randomly connected networks

parametric surface meshing using a combined advancing front generalized delaunay approach an indirect method for meshing parametric surfaces conforming to a user specifiable size map is presented first from this size specification a riemannian metric is defined so that the desired mesh is one with unit length edges with respect to the related riemannian space the so called unit mesh then based on the intrinsic properties of the surface the riemannian structure is induced into the parametric space finally a unit mesh is generated completely inside the parametric space such that it conforms to the metric of the induced riemannian structure this mesh is constructed using a combined advancing front delaunay approach applied within a riemannian context the proposed method can be applied to mesh composite parametric surfaces several examples illustrate the efficiency of our approach copyright 2000 john wiley sons ltd

optimal storage properties of neural network models the authors calculate the number p alpha n of random n bit patterns that an optimal neural network can store allowing a given fraction f of bit errors and with the condition that each right bit is stabilised by a local field at least equal to a parameter k for each value of alpha and k there is a minimum fraction fmin of wrong bits they find a critical line alpha c k with alpha c 0 2 the minimum fraction of wrong bits vanishes for alpha alpha c k the calculations are done using a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is locally stable in a finite region of the k alpha plane including the line alpha c k but there is a line above which the solution becomes unstable and replica symmetry must be broken

the space of interactions in neural network models the typical fraction of the space of interactions between each pair of n ising spins which solve the problem of storing a given set of p random patterns as n bit spin configurations is considered the volume is calculated explicitly as a function of the storage ratio alpha p n of the value kappa 0 of the product of the spin and the magnetic field at each site and of the magnetisation m here m may vary between 0 no correlation and 1 completely correlated the capacity increases with the correlation between patterns from alpha 2 for correlated patterns with kappa 0 and tends to infinity as m tends to 1 the calculations use a saddle point method and the order parameters at the saddle point are assumed to be replica symmetric this solution is shown to be locally stable a local iterative learning algorithm for updating the interactions is given which will converge to a solution of given kappa provided such solutions exist

frequency analysis with coupled nonlinear oscillators we present a method to obtain the frequency spectrum of a signal with a nonlinear dynamical system the dynamical system is composed of a pool of adaptive frequency oscillators with negative mean field coupling for the frequency analysis the synchronization and adaptation properties of the component oscillators are exploited the frequency spectrum of the signal is reflected in the statistics of the intrinsic frequencies of the oscillators the frequency analysis is completely embedded in the dynamics of the system thus no pre processing or additional parameters such as time windows are needed representative results of the numerical integration of the system are presented it is shown that the oscillators tune to the correct frequencies for both discrete and continuous spectra due to its dynamic nature the system is also capable to track non stationary spectra further we show that the system can be modeled in a probabilistic manner by means of a nonlinear fokker planck equation the probabilistic treatment is in good agreement with the numerical results and provides a useful tool to understand the underlying mechanisms leading to convergence

memory nearly on a spring a mean first passage time approach to memory lifetimes we study memory lifetimes in a perceptron based framework with binary synapses using the mean first passage time for the perceptron s total input to fall below firing threshold to define memory lifetimes working with the simplest memory related model of synaptic plasticity we may obtain exact results for memory lifetimes or working in the continuum limit good analytical approximations that afford either much qualitative insight or extremely good quantitative agreement in one particular limit we find that memory dynamics reduce to the well understood ornstein uhlenbeck process we show that asymptotically the lifetimes of memories grow logarithmically in the number of synapses when the perceptron s firing threshold is zero reproducing standard results from signal to noise ratio analyses however this is only an asymptotically valid result and we show that extending its application outside the range of its validity leads to a massive overestimate of the minimum number of synapses required for suc

anisotropic diagrams labelle shewchuk approach revisited f labelle and j shewchuk have proposed a discrete definition of anisotropic voronoi diagrams these diagrams are parametrized by a metric field under mild hypotheses on the metric field such voronoi diagrams can be refined so that their dual is a triangulation with elements shaped according to the specified anisotropic metric field we propose an alternative view of the construction of these diagrams and a variant of labelle and shewchuk s meshing algorithm this variant computes the voronoi vertices using a higher dimensional power diagram and refines the diagram as long as dual triangles overlap we see this variant as a first step toward a 3 dimensional anisotropic meshing algorithm

minimum time speed optimisation over a fixed path in this paper we investigate the problem of optimising the speed of a vehicle over a fixed path for minimum time traversal we utilise a change of variables that has been known since the 1980s although the resulting convexity of the problem was not noted until recently the contributions of this paper are three fold first we extend the convexification of the problem to a more general framework second we identify a wide range of vehicle models and constraints which can be included in this expanded framework third we develop and implement an algorithm that allows these problems to be solved in real time on embedded systems with a high degree of accuracy

regridding surface triangulations an advancing front surface gridding technique that operates on discretely defined surfaces i e triangulations is presented different aspects that are required to make the procedure reliable for complex geometries are discussed notable among these are a the recovery of surface features and discrete surface patches from the discrete data b filtering based on point and side normals to remove undesirable data close to cusps and corners c the proper choice of host faces for ridges and d fast interpolation procedures suitable for complex geometries post generation surface recovery or repositioning techniques are discussed several examples ranging from academic to industrial demonstrate the utility of the proposed procedure forab initiosurface meshing from discrete data such as those encountered when the surface description is already given as discrete the improvement of existing surface triangulations as well as remeshing applications during runs exhibiting significant change of domain

the role of constraints in hebbian learning models of unsupervised correlation based hebbian synaptic plasticity are typically unstable either all synapses grow until each reaches the maximum allowed strength or all synapses decay to zero strength a common method of avoiding these outcomes is to use a constraint that conserves or limits the total synaptic strength over a cell we study the dynamic effects of such constraints two methods of enforcing a constraint are distinguished multiplicative and subtractive for otherwise linear learning rules multiplicative enforcement of a constraint results in dynamics that converge to the principal eigenvector of the operator determining unconstrained synaptic development subtractive enforcement in contrast typically leads to a final state in which almost all synaptic strengths reach either the maximum or minimum allowed value this final state is often dominated by weight configurations other than the principal eigenvector of the unconstrained operator multiplicative enforcement yields a graded receptive field in which most mutually correlated inputs are represented whereas subtractive enforcement yields a receptive field that is sharpened to a subset of maximally correlated inputs if two equivalent input populations e g two eyes innervate a common target multiplicative enforcement prevents their segregation ocular dominance segregation when the two populations are weakly correlated whereas subtractive enforcement allows segregation under these circumstances these results may be used to understand constraints both over output cells and over input cells a variety of rules that can implement constrained dynamics are discussed

capacity of neural networks with discrete synaptic couplings the authors study the optimal storage capacity of neural networks with discrete local constraints on the synaptic couplings jij models with such constraints include those with binary couplings jij or 1 or jij 0 1 quantised couplings with larger synaptic range e g jij or 1 l or 2 l or 1 and in the limit continuous couplings confined to the hypercube mod jij mod or 1 box confinement they find that the optimal storage capacity alpha kappa is best determined by the vanishing of a suitably defined entropy as calculated in the replica symmetric approximation they also extend their results to cases with biased memories and make contact with sparse coding models

optimal anisotropic meshes for minimizing interpolation errors in l p norm in this paper we present a new optimal interpolation error estimate in l p norm 1 p for finite element simplicial meshes in any spatial dimension a sufficient condition for a mesh to be nearly optimal is that it is quasi uniform under a new metric defined by a modified hessian matrix of the function to be interpolated we also give new functionals for the global moving mesh method and obtain optimal monitor functions from the viewpoint of minimizing interpolation error in the l p norm some numerical examples are also given to support the theoretical estimates

anisotropic delaunay mesh generation anisotropic meshes are triangulations of a given domain in the plane or in higher dimensions with elements elongated along prescribed directions anisotropic triangulations are known to be well suited for interpolation of functions or solving pdes assuming that the anisotropic shape requirements for mesh elements are given through a metric field varying over the domain we propose a new approach to anisotropic mesh generation relying on the notion of anisotropic delaunay meshes an anisotropic delaunay mesh is defined as a mesh in which the star of each vertex v consists of simplices that are delaunay for the metric associated to vertex v this definition works in any dimension and allows us to define a simple refinement algorithm the algorithm takes as input a domain and a metric field and provides after completion an anisotropic mesh whose elements are sized and shaped according to the metric field

neural network modeling of associative memory beyond the hopfield model a number of neural network models in which fixed point and limit cycle attractors of the underlying dynamics are used to store and associatively recall information are described in the first class of models a hierarchical structure is used to store an exponentially large number of strongly correlated memories the second class of models uses limit cycles to store and retrieve individual memories a neurobiologically plausible network that generates low amplitude periodic variations of activity similar to the oscillations observed in electroencephalographic recordings is also described results obtained from analytic and numerical studies of the properties of these networks are discussed

associative memory on the puzzling sparse coding limit recent studies of the information capacity in a sparsely coded memory net has led to some contradictory results in the willshaw model where the couplings are binary 0 or 1 the maximal quantity of information that can be stored is 1n 2 approximately 0 69 bits per synapse on the other hand a calculation a la gardner 1988 for 0 1 couplings gives an upper bound for the maximal capacity of about 0 29 bits per synapse in this study the author considers two possible sources for this discrepancy the first one is that the criteria for defining the maximal capacity are different with or without a constraint of perfect errorless storage the second one is a difference in the choice of the probability distribution of the random patterns used to compute this capacity this analysis shows in particular that for the willshaw model the maximal information capacity is much larger when the number of active neurons is exactly the same in every stored pattern than when it is given only in average in addition he gives an argument showing that this result may be generic e g valid for any activity level and independent of the learning rule

stability of negative image equilibria in spike timing dependent plasticity we investigate the stability of negative image equilibria in mean synaptic weight dynamics governed by spike timing dependent plasticity stdp the model architecture closely follows the anatomy and physiology of the electrosensory lateral line lobe ell of mormyrid electric fish the ell uses a spike timing dependent learning rule to form a negative image of the reafferent signal from the fish s own electric discharge thus improving detectability of external electric fields we derive sufficient conditions for existence of the negative image and necessary and sufficient conditions for stability for arbitrary postsynaptic potential functions and arbitrary learning rules this significantly generalizes earlier investigations we then apply the general result to several examples of biological interest including a class of learning rules consistent with the rule observed experimentally in the mormyrid ell

an extended advancing front technique for closed surfaces mesh generation an extended advancing front technique aft with shift operations and riemann metric named as shifting aft is presented for finite element mesh generation on 3d surfaces especially 3d closed surfaces riemann metric is used to govern the size and shape of the triangles in the parametric space the shift operators are employed to insert a floating space between real space and parametric space during the 2d parametric space mesh generation in the previous work of closed surface mesh generation the virtual boundaries are adopted when mapping the closed surfaces into 2d open parametric domains however it may cause the mesh quality worsening problem in order to overcome this problem the aft kernel is combined with the shift operator in this paper the shifting aft can generate high quality meshes and guarantee convergence in both open and closed surfaces for the shifting aft it is not necessary to introduce virtual boundaries while meshing a closed surface hence the boundary discretization procedure is largely simplified and moreover better shaped triangles will be generated because there are no additional interior constraints yielded by virtual boundaries comparing with direct methods the shifting aft avoids costly and unstable 3d geometrical computations in the real space some examples presented in this paper have demonstrated the advantages of shift aft in 3d surface mesh generation especially for the closed surfaces copyright 2007 john wiley sons ltd

gaussian process approach to spiking neurons for inhomogeneous poisson inputs this article presents a new theoretical framework to consider the dynamics of a stochastic spiking neuron model with general membrane response to input spike we assume that the input spikes obey an inhomogeneous poisson process the stochastic process of the membrane potential then becomes a gaussian process when a general type of the membrane response is assumed the stochastic process becomes a markov gaussian process we present a calculation method for the membrane potential density and the firing probability density our new formulation is the extension of the existing formulation based on diffusion approximation although the single markov assumption of the diffusion approximation simplifies the stochastic process analysis the calculation is inaccurate when the stochastic process involves a multiple markov property we find that the variation of the shape of the membrane response which has often been ignored in existing stochastic process studies significantly affects the firing probability our approach can consider the reset effect which has been difficult to deal with by analysis based on the first passage time density

constrained optimization for neural map formation a unifying framework for weight growth and normalization computational models of neural map formation can be considered on at least three different levels of abstraction detailed models including neural activity dynamics weight dynamics that abstract from the neural activity dynamics by an adiabatic approximation and constrained optimization from which equations governing weight dynamics can be derived constrained optimization uses an objective function from which a weight growth rule can be derived as a gradient flow and some constraints from which normalization rules are derived in this article we present an example of how an optimization problem can be derived from detailed nonlinear neural dynamics a systematic investigation reveals how different weight dynamics introduced previously can be derived from two types of objective function terms and two types of constraints this includes dynamic link matching as a special case of neural map formation we focus in particular on the role of coordinate transformations to derive different weight dynamics from the same optimization problem several examples illustrate how the constrained optimization framework can help in understanding generating and comparing different models of neural map formation the techniques used in this analysis may also be useful in investigating other types of neural dynamics

on the computational capability of a perceptron the minimal fraction of errors a perceptron can make on p random dichotomies is investigated an exact upper bound is presented as well as comparisons between the replica theory estimates and a rigorous lower bound