bluuebunny/crossref_metadata_2025 huggingface dataset:
Natural History of Dural Arteriovenous Shunts
Authors: Söderman Michael , Pavic Ladislav , Edner Göran , Holmin Staffan , Andersson Tommy
Published: April 2008

Background and Purpose—

Dural arteriovenous shunts with cortical venous reflux or drainage may cause neurological symptoms and death with or without intracranial hemorrhage. Present knowledge about the natural history of these lesions is limited, however. We investigated the incidences of intracranial hemorrhage, progressive dementia syndrome, and death in patients diagnosed in our neurovascular center.

Methods—

We evaluated the records of 85 patients with dural arteriovenous shunts with cortical venous drainage or reflux hospitalized in our institution from 1978 to 2007. The annual incidences of intracranial hemorrhage, progressive dementia syndrome, and death were calculated.

Results—

Fifty-three patients did not have an intracranial hemorrhage as the presenting event. One of these patients bled after diagnosis. Thirty-two patients had an intracranial hemorrhage as the presenting event. Three patients bled after diagnosis. One of these patients died. Apart from deficits caused by hemorrhage, no patient reported adverse neurological symptoms. In patients presenting with an intracranial hemorrhage the annual risk for hemorrhage is approximately 7.4% and in those not presenting with a hemorrhage it is approximately 1.5%.

Conclusion—

The risk of intracranial hemorrhage from a dural arteriovenous shunt with cortical venous drainage is most likely smaller than previously proposed. Presentation with hemorrhage is a risk factor for hemorrhage. The risks of developing neurological symptoms not related to hemorrhage are also less than previously reported.
Conversion of Temporal Correlations Between Stimuli to Spatial Correlations Between Attractors
Authors: Griniasty M. , Tsodyks M. V. , Amit Daniel J.
Published: April 2008

It is shown that a simple modification of synaptic structures (of the Hopfield type) constructed to produce autoassociative attractors, produces neural networks whose attractors are correlated with several (learned) patterns used in the construction of the matrix. The modification stores in the matrix a fixed sequence of uncorrelated patterns. The network then has correlated attractors, provoked by the uncorrelated stimuli. Thus, the network converts the temporal order (or temporal correlation) expressed by the sequence of patterns, into spatial correlations expressed in the distributions of neural activities in attractors. The model captures phenomena observed in single electrode recordings in performing monkeys by Miyashita et al. The correspondence is as close as to reproduce the fact that given uncorrelated patterns as sequentially learned stimuli, the attractors produced are significantly correlated up to a separation of 5 (five) in the sequence. This number 5 is universal in a range of parameters, and requires essentially no tuning. We then discuss learning scenarios that could lead to this synaptic structure as well as experimental predictions following from it. Finally, we speculate on the cognitive utility of such an arrangement.
Training Periodic Sequences Using Fourier Series Error Criterion
Authors: Kottas James A.
Published: April 2008

Training a network to learn a set of periodic input/output sequences effectively makes the network learn a mapping between amplitudes and phases in Fourier space. The spectral backpropagation (SBP) training algorithm is a different way of doing this task. It measures the Fourier series components of the output error sequences and minimizes the total spectral energy as an adaptation criterion. This approach can train not only the weights but also time delays associated with the interconnects. Furthermore, the cells can have finite bandwidth via a first-order low-pass filter. Having adaptable time delays gives the SBP algorithm a powerful way to control the phase characteristics of the network.
Generalization and Approximation Capabilities of Multilayer Networks
Authors: Takahashi Yoshikane
Published: April 2008

This paper develops a theory for constructing 3-layered networks. The theory allows one to specify a finite discrete set of training data and a network structure (minimum intermediate units, synaptic weights and biases) that generalizes and approximates any given continuous mapping between sets of contours on a plane within any given permissible error.
Statistical Mechanics for a Network of Spiking Neurons
Authors: Kruglyak Leonid , Bialek William
Published: April 2008

We show that a simple statistical mechanics model can capture the collective behavior of large networks of spiking neurons. Qualitative arguments suggest that regularly firing neurons should be described by a planar "spin" of unit length. We extract these spins from spike trains and then measure the interaction Hamiltonian using simulations of small clusters of cells. Correlations among spike trains obtained from simulations of large arrays of cells are in quantitative agreement with the predictions from these Hamiltonians. We comment on the novel computational abilities of these "XY networks."
Convergent Algorithm for Sensory Receptive Field Development
Authors: Atick Joseph J. , Redlich A. Norman
Published: April 2008

An unsupervised developmental algorithm for linear maps is derived which reduces the pixel-entropy (using the measure introduced in previous work) at every update and thus removes pairwise correlations between pixels. Since the measure of pixel-entropy has only a global minimum the algorithm is guaranteed to converge to the minimum entropy map. Such optimal maps have recently been shown to possess cognitively desirable properties and are likely to be used by the nervous system to organize sensory information. The algorithm derived here turns out to be one proposed by Goodall for pairwise decorrelation. It is biologically plausible since in a neural network implementation it requires only data available locally to a neuron. In training over ensembles of two-dimensional input signals with the same spatial power spectrum as natural scenes, networks develop output neurons with center-surround receptive fields similar to those of ganglion cells in the retina. Some technical issues pertinent to developmental algorithms of this sort, such as “symmetry fixing,” are also discussed.
Clustering Data by Melting
Authors: Wong Yiu-fai
Published: April 2008

We derive a new clustering algorithm based on information theory and statistical mechanics, which is the only algorithm that incorporates scale. It also introduces a new concept into clustering: cluster independence. The cluster centers correspond to the local minima of a thermodynamic free energy, which are identified as the fixed points of a one-parameter nonlinear map. The algorithm works by melting the system to produce a tree of clusters in the scale space. Melting is also insensitive to variability in cluster densities, cluster sizes, and ellipsoidal shapes and orientations. We tested the algorithm successfully on both simulated data and a Synthetic Aperture Radar image of an agricultural site with 12 attributes for crop identification.
Neural Networks and Nonlinear Adaptive Filtering: Unifying Concepts and New Algorithms
Authors: Nerrand O. , Roussel-Ragot P. , Personnaz L. , Dreyfus G. , Marcos S.
Published: April 2008

The paper proposes a general framework that encompasses the training of neural networks and the adaptation of filters. We show that neural networks can be considered as general nonlinear filters that can be trained adaptively, that is, that can undergo continual training with a possibly infinite number of time-ordered examples. We introduce the canonical form of a neural network. This canonical form permits a unified presentation of network architectures and of gradient-based training algorithms for both feedforward networks (transversal filters) and feedback networks (recursive filters). We show that several algorithms used classically in linear adaptive filtering, and some algorithms suggested by other authors for training neural networks, are special cases in a general classification of training algorithms for feedback networks.
A Neural Network Model of Inhibitory Information Processing in Aplysia
Authors: Blazis Diana E. J. , Fischer Thomas M. , Carew Thomas J.
Published: April 2008

Recent cellular studies have revealed a novel form of inhibitory information processing in the siphon withdrawal reflex of the marine mollusc Aplysia: Motorneuronal output is significantly reduced by activity-dependent potentiation of recurrent inhibition within the siphon withdrawal network (Fischer and Carew 1991, 1993). This inhibitory modulation is mediated by two types of identified interneurons, L29s and L30s. In an effort to describe and analyze this and other forms of inhibitory information processing in Aplysia, and to compare it with similar processing in other nervous systems, we have constructed a neural network model that incorporates many empirically observed features of these interneurons. The model generates important aspects of the interactions of cells L29 and L30, and with no further modification, exhibits many network level phenomena that were not explicitly incorporated into the model.
Hints and the VC Dimension
Authors: Abu-Mostafa Yaser S.
Published: April 2008

Learning from hints is a generalization of learning from examples that allows for a variety of information about the unknown function to be used in the learning process. In this paper, we use the VC dimension, an established tool for analyzing learning from examples, to analyze learning from hints. In particular, we show how the VC dimension is affected by the introduction of a hint. We also derive a new quantity that defines a VC dimension for the hint itself. This quantity is used to estimate the number of examples needed to "absorb" the hint. We carry out the analysis for two types of hints, invariances and catalysts. We also describe how the same method can be applied to other types of hints.
Redundancy Reduction as a Strategy for Unsupervised Learning
Authors: Redlich A. Norman
Published: April 2008

A redundancy reduction strategy, which can be applied in stages, is proposed as a way to learn as efficiently as possible the statistical properties of an ensemble of sensory messages. The method works best for inputs consisting of strongly correlated groups, that is features, with weaker statistical dependence between different features. This is the case for localized objects in an image or for words in a text. A local feature measure determining how much a single feature reduces the total redundancy is derived which turns out to depend only on the probability of the feature and of its components, but not on the statistical properties of any other features. The locality of this measure makes it ideal as the basis for a "neural" implementation of redundancy reduction, and an example of a very simple non-Hebbian algorithm is given. The effect of noise on learning redundancy is also discussed.
Neural Networks for Optimization Problems with Inequality Constraints: The Knapsack Problem
Authors: Ohlsson Mattias , Peterson Carsten , Söderberg Bo
Published: April 2008

A strategy for finding approximate solutions to discrete optimization problems with inequality constraints using mean field neural networks is presented. The constraints x ≤ 0 are encoded by x⊖(x) terms in the energy function. A careful treatment of the mean field approximation for the self-coupling parts of the energy is crucial, and results in an essentially parameter-free algorithm. This methodology is extensively tested on the knapsack problem of size up to 103items. The algorithm scales like NM for problems with N items and M constraints. Comparisons are made with an exact branch and bound algorithm when this is computationally possible (N ≤ 30). The quality of the neural network solutions consistently lies above 95% of the optimal ones at a significantly lower CPU expense. For the larger problem sizes the algorithm is compared with simulated annealing and a modified linear programming approach. For "nonhomogeneous" problems these produce good solutions, whereas for the more difficult "homogeneous" problems the neural approach is a winner with respect to solution quality and/or CPU time consumption. The approach is of course also applicable to other problems of similar structure, like set covering.
A Model for Motor Endplate Morphogenesis: Diffusible Morphogens, Transmembrane Signaling, and Compartmentalized Gene Expression
Authors: Kerszberg Michel , Changeux Jean-Pierre
Published: April 2008

A mathematical model for the formation and maintenance of synaptic contacts at the motor endplate is proposed. It is based on diffusion between sarcoplasmic nuclei of limiting amounts of a morphogen substance. The morphogen is postulated to act on genetic switch-like intranuclear units and to regulate positively both the transcription of its own gene and that of acetylcholine receptor (AChR) subunit genes. The efficacy of autoregulation is assumed to be depressed by electrical activity; while AChR genes transcription is enhanced by anterograde neural factors. Thus the model involves Turing's classical ingredients: autocatalysis and short range activation by the morphogen, and long range inhibition by electrical activity. Our predictions include: the stabilization of a single, transcriptionally active nucleus located in the central region of the developing muscle fiber (or myotube); the frequent occurrence of transcriptional activity in nuclei at the tendinous ends; and the onset, upon denervation of adult muscle, of transcription waves, starting from both the central site and the tendinous nuclei. In noninnervated fibers, the calculations show that spontaneous, irregular electrical activity leads to a variety of near-periodic spatial patterns of transcription; these are also predicted in innervated fibers when the depressing effect of electrical activity is weak, giving rise to the stabilization of multiple endplates as occurs in muscles with distributed innervation.
Universal Approximation by Phase Series and Fixed-Weight Networks
Authors: Cotter Neil E. , Conwell Peter R.
Published: April 2008

In this note we show that weak (specified energy bound) universal approximation by neural networks is possible if variable synaptic weights are brought in as network inputs rather than being embedded in a network. We illustrate this idea with a Fourier series network that we transform into what we call a phase series network. The transformation only increases the number of neurons by a factor of two.
Backpropagation with Homotopy
Authors: Yang Liping , Yu Wanzhen
Published: April 2008

When training a feedforward neural network with backpropagation (Rumelhart et al. 1986), local minima are always a problem because of the nonlinearity of the system. There have been several ways to attack this problem: for example, to restart the training by selecting a new initial point, to perform the preprocessing of the input data or the neural network. Here, we propose a method which is efficient in computation to avoid some local minima.
A Neural Network for the Processing of Optic Flow from Ego-Motion in Man and Higher Mammals
Authors: Lappe Markus , Rauschecker Josef P.
Published: April 2008

Interest in the processing of optic flow has increased recently in both the neurophysiological and the psychophysical communities. We have designed a neural network model of the visual motion pathway in higher mammals that detects the direction of heading from optic flow. The model is a neural implementation of the subspace algorithm introduced by Heeger and Jepson (1990). We have tested the network in simulations that are closely related to psychophysical and neurophysiological experiments and show that our results are consistent with recent data from both fields. The network reproduces some key properties of human ego-motion perception. At the same time, it produces neurons that are selective for different components of ego-motion flow fields, such as expansions and rotations. These properties are reminiscent of a subclass of neurons in cortical area MSTd, the triple-component neurons. We propose that the output of such neurons could be used to generate a computational map of heading directions in or beyond MST.
Arbitrary Elastic Topologies and Ocular Dominance
Authors: Dayan Peter
Published: April 2008

The elastic net, which has been used to produce accounts of the formation of topology-preserving maps and ocular dominance stripes (OD), embodies a nearest neighbor topology. A Hebbian account of OD is not so restricted—and indeed makes the prediction that the width of the stripes depends on the nature of the (more general) neighborhood relations. Elastic and Hebbian accounts have recently been unified, raising a question mark about their different determiners of stripe widths. This paper considers this issue, and demonstrates theoretically that it is possible to use more general topologies in the elastic net, including those effectively adopted in the Hebbian model.
Surface Interpolation Networks
Authors: Pentland Alex P.
Published: April 2008

Orthogonal wavelets can be used as models for receptive fields in the human visual system. They may also be used to solve spatial interpolation problems formulated either as regularization or 2-D Kalman filtering. The solutions take the form of simple feedback networks, and only a few iterations are required for convergence.
Combining Exploratory Projection Pursuit and Projection Pursuit Regression with Application to Neural Networks
Authors: Intrator Nathan
Published: April 2008

We present a novel classification and regression method that combines exploratory projection pursuit (unsupervised training) with projection pursuit regression (supervised training), to yield a new family of cost/complexity penalty terms. Some improved generalization properties are demonstrated on real-world problems.
A Simplified Gradient Algorithm for IIR Synapse Multilayer Perceptrons
Authors: Back Andrew D. , Tsoi Ah Chung
Published: April 2008

A network architecture with a global feedforward local recurrent construction was presented recently as a new means of modeling nonlinear dynamic time series (Back and Tsoi 1991a). The training rule used was based on minimizing the least mean square (LMS) error and performed well, although the amount of memory required for large networks may become significant if a large number of feedback connections are used. In this note, a modified training algorithm based on a technique for linear filters is presented, simplifying the gradient calculations significantly. The memory requirements are reduced from O[na(na + nb)Ns] to O[(2na + nb)Ns], where na is the number of feedback delays, and Ns is the total number of synapses. The new algorithm reduces the number of multiply-adds needed to train each synapse by na at each time step. Simulations indicate that the algorithm has almost identical performance to the previous one.
The Characteristics of the Convergence Time of Associative Neural Networks
Authors: Tanaka Toshiaki , Yamada Miki
Published: April 2008

The authors have analyzed the dynamics of associative neural networks based on macroscopic state equations and have shown that both a layered associative net and an autocorrelation type net have the same convergence property: If a recalling process succeeds, the network converges very fast to one of the memorized patterns. But if a recalling process fails, it converges very slowly to a spurious state or does not converge. This property was also checked by computer simulations on a large scale (N = 1000) neural network. Moreover, it is shown that the convergence time for a successful recall is of order log(N). If this convergence time difference is used, execution time and memory can be saved and it can be determined whether a recalling process succeeds or fails without any additional procedure.
Pattern Discrimination Using Feedforward Networks: A Benchmark Study of Scaling Behavior
Authors: Rögnvaldsson Thorsteinn
Published: April 2008

The discrimination powers of multilayer perceptron (MLP) and learning vector quantization (LVQ) networks are compared for overlapping gaussian distributions. It is shown, both analytically and with Monte Carlo studies, that the MLP network handles high-dimensional problems in a more efficient way than LVQ. This is mainly due to the sigmoidal form of the MLP transfer function, but also to the fact that the MLP uses hyperplanes more efficiently. Both algorithms are equally robust to limited training sets and the learning curves fall off like 1/M, where M is the training set size, which is compared to theoretical predictions from statistical estimates and Vapnik-Chervonenkis bounds.
A Neural Network That Learns to Interpret Myocardial Planar Thallium Scintigrams
Authors: Rosenberg Charles , Erel Jacob , Atlan Henri
Published: April 2008

The planar thallium-201 (201Tl) myocardial perfusion scintigram is a widely used diagnostic technique for detecting and estimating the risk of coronary artery disease. Interpretation is currently based on visual scoring of myocardial defects combined with image quantitation and is known to have a significant subjective component. Neural networks learned to interpret thallium scintigrams as determined by both individual and multiple (consensus) expert ratings. Four different types of networks were explored: single-layer, two-layer backpropagation (BP), BP with weight smoothing, and two-layer radial basis function (RBF). The RBF network was found to yield the best performance (94.8% generalization by region) and compares favorably with human experts. We conclude that this network is a valuable clinical tool that can be used as a reference "diagnostic support system" to help reduce inter- and intraobserver variability. This system is now being further developed to include other variables that are expected to improve the final clinical diagnosis.
Dynamics of Populations of Integrate-and-Fire Neurons, Partial Synchronization and Memory
Authors: Usher Marius , Schuster Heinz Georg , Niebur Ernst
Published: April 2008

We study the dynamics of completely connected populations of refractory integrate-and-fire neurons in the presence of noise. Solving the master equation based on a mean-field approach, and by computer simulations, we find sustained states of activity that correspond to fixed points and show that for the same value of external input, the system has one or two attractors. The dynamic behavior of the population under the influence of external input and noise manifests hysteresis effects that might have a functional role for memory. The temporal dynamics at higher temporal resolution, finer than the transmission delay times and the refractory period, are characterized by synchronized activity of subpopulations. The global activity of the population shows aperiodic oscillations analogous to experimentally found field potentials.
The Effects of Cell Duplication and Noise in a Pattern Generating Network
Authors: Ferrar Catherine H. , Williams Thelma L. , Bowtell Graham
Published: April 2008

Stability against stochastic variation is an important property for biological systems. This paper investigates the robustness of the rhythmic activity produced by a model of the segmental rhythm generator for locomotion in the lamprey, by introducing stochastic properties into the network. In addition, since neuronal models for vertebrate systems often use a single neuron to represent a large class of cells, this paper explores one of the consequences of such reduction, by investigating the effects of duplicating all the cells of the network on its stability against stochastic variation. We have found the basic model network to be very stable, and have found that this stability is increased by doubling the number of cells in the network.
Emergence of Position-Independent Detectors of Sense of Rotation and Dilation with Hebbian Learning: An Analysis
Authors: Zhang Kechen , Sereno Martin I. , Sereno Margaret E.
Published: April 2008

We previously demonstrated that it is possible to learn position-independent responses to rotation and dilation by filtering rotations and dilations with different centers through an input layer with MT-like speed and direction tuning curves and connecting them to an MST-like layer with simple Hebbian synapses (Sereno and Sereno 1991). By analyzing an idealized version of the network with broader, sinusoidal direction-tuning and linear speed-tuning, we show analytically that a Hebb rule trained with arbitrary rotation, dilation/contraction, and translation velocity fields yields units with weight fields that are a rotation plus a dilation or contraction field, and whose responses to a rotating or dilating/contracting disk are exactly position independent. Differences between the performance of this idealized model and our original model (and real MST neurons) are discussed.
Discovering Predictable Classifications
Authors: Schmidhuber Jürgen , Prelinger Daniel
Published: April 2008

Prediction problems are among the most common learning problems for neural networks (e.g., in the context of time series prediction, control, etc.). With many such problems, however, perfect prediction is inherently impossible. For such cases we present novel unsupervised systems that learn to classify patterns such that the classifications are predictable while still being as specific as possible. The approach can be related to the IMAX method of Becker and Hinton (1989) and Zemel and Hinton (1991). Experiments include a binary stereo task proposed by Becker and Hinton, which can be solved more readily by our system.
A Kinetic Model of Short- and Long-Term Potentiation
Authors: Migliore M. , Ayala G. F.
Published: April 2008

We present a kinetic model that can account for several experimental findings on short- and long-term potentiation (STP and LTP) and their pharmacological modulation. The model, which is consistent with Hebb's postulate, uses the hypothesis that part of the origin of LTP may be a consequence of an increased release of neurotransmitter due to a retrograde signal. The operation of the model is expressed by a set of irreversible reactions, each of which should be thought of as equivalent to a set of more complex reactions. We show that a retrograde signal alone is not sufficient to maintain LTP unless long-term change of the rate constant of some of the reactions is caused by high-frequency stimulation. Pharmacological manipulation of LTP is interpreted as modifications of the rate constants of one or more of the reactions that express a given mechanism. The model, because of its simplicity, can be useful to test more specific mechanisms by expanding one or more reactions as suggested by new experimental evidence.
Sensitivity of Synaptic Plasticity to the Ca2+ Permeability of NMDA Channels: A Model of Long-Term Potentiation in Hippocampal Neurons
Authors: Schutter Erik De , Bower James M.
Published: April 2008

We have examined a model by Holmes and Levy (1990) of the induction of associative long-term potentiation (LTP) by a rise in the free Ca2+ concentration ([Ca2+]) after synaptic activation of dendritic spines. The previously reported amplification of the change in [Ca2+] caused by coactivation of several synapses was found to be quite sensitive to changes in the permeability of the N-methyl-D-aspartate (NMDA) receptor channels to Ca2+. Varying this parameter indicated that maximum amplification is obtained at values that are close to Ca2+ permeabilities reported in the literature. However, amplification failed if permeability is reduced by more than 50%. We also found that the maximum free [Ca2+] reached in an individual spine during synaptic coactivation of several spines depended on the location of that spine on the dendritic tree. Distal spines attained a higher [Ca2+] than proximal ones, with differences of up to 80%. The implications of this result for the uniformity of induction of associative LTP in spines in different regions of the dendrite are discussed.
Models of Perceptual Learning in Vernier Hyperacuity
Authors: Weiss Yair , Edelman Shimon , Fahle Manfred
Published: April 2008

Performance of human subjects in a wide variety of early visual processing tasks improves with practice. HyperBF networks (Poggio and Girosi 1990) constitute a mathematically well-founded framework for understanding such improvement in performance, or perceptual learning, in the class of tasks known as visual hyperacuity. The present article concentrates on two issues raised by the recent psychophysical and computational findings reported in Poggio et al. (1992b) and Fahle and Edelman (1992). First, we develop a biologically plausible extension of the HyperBF model that takes into account basic features of the functional architecture of early vision. Second, we explore various learning modes that can coexist within the HyperBF framework and focus on two unsupervised learning rules that may be involved in hyperacuity learning. Finally, we report results of psychophysical experiments that are consistent with the hypothesis that activity-dependent presynaptic amplification may be involved in perceptual learning in hyperacuity.
Learning to Generalize from Single Examples in the Dynamic Link Architecture
Authors: Konen Wolfgang , Malsburg Christoph von der
Published: April 2008

A large attraction of neural systems lies in their promise of replacing programming by learning. A problem with many current neural models is that with realistically large input patterns learning time explodes. This is a problem inherent in a notion of learning that is based almost entirely on statistical estimation. We propose here a different learning style where significant relations in the input pattern are recognized and expressed by the unsupervised self-organization of dynamic links. The power of this mechanism is due to the very general a priori principle of conservation of topological structure. We demonstrate that style with a system that learns to classify mirror symmetric pixel patterns from single examples.
Construction of Minimal n-2-n Encoders for Any n
Authors: Phatak D. S. , Choi H. , Koren I.
Published: April 2008

The encoding problem (Rumelhart and McClelland 1986) is an important canonical problem. It has been widely used as a benchmark. Here, we have analytically derived minimal-sized nets necessary and sufficient to solve encoding problems of arbitrary size. The proofs are constructive: we construct n-2-n encoders and show that two hidden units are also necessary for n > 2. Moreover, the geometric approach employed is general and has much wider applications. For example, this method has also helped us derive lower bounds on the redundancy necessary for achieving complete fault tolerance (Phatak and Koren 1992a,b).
Limitations of the Hodgkin-Huxley Formalism: Effects of Single Channel Kinetics on Transmembrane Voltage Dynamics
Authors: Strassberg Adam F. , DeFelice Louis J.
Published: April 2008

A standard membrane model, based on the continuous deterministic Hodgkin-Huxley equations, is compared to an alternative membrane model, based on discrete stochastic ion channel populations represented through Markov processes. Simulations explore the relationship between these two levels of description: the behavior predicted by the macroscopic membrane currents versus the behavior predicted by their microscopic ion channels. Discussion considers the extent to which these random events underlying neural signals mediate random events in neural computation.
Two-Dimensional Motion Perception in Flies
Authors: Borst A. , Egelhaaf M. , Seung H. S.
Published: April 2008

We study two-dimensional motion perception in flies using a semicircular visual stimulus. Measurements of both the H1-neuron and the optomotor response are consistent with a simple model supposing spatial integration of the outputs of correlation-type motion detectors. In both experiment and model, there is substantial H1 and horizontal (yaw) optomotor response to purely vertical motion of the stimulus. We conclude that the fly's optomotor response to a two-dimensional pattern, depending on its structure, may deviate considerably from the direction of pattern motion.
Neural Representation of Space Using Sinusoidal Arrays
Authors: Touretzky David S. , Redish A. David , Wan Hank S.
Published: April 2008

O'Keefe (1991) has proposed that spatial information in rats might be represented as phasors: phase and amplitude of a sine wave encoding angle and distance to a landmark. We describe computer simulations showing that operations on phasors can be efficiently realized by arrays of spiking neurons that recode the temporal dimension of the sine wave spatially. Some cells in motor and parietal cortex exhibit response properties compatible with this proposal.
Rational Function Neural Network
Authors: Leung Henry , Haykin Simon
Published: April 2008

In this paper we observe that a particular class of rational function (RF) approximations may be viewed as feedforward networks. Like the radial basis function (RBF) network, the training of the RF network may be performed using a linear adaptive filtering algorithm. We illustrate the application of the RF network by considering two nonlinear signal processing problems. The first problem concerns the one-step prediction of a time series consisting of a pair of complex sinusoid in the presence of colored non-gaussian noise. Simulated data were used for this problem. In the second problem, we use the RF network to build a nonlinear dynamic model of sea clutter (radar backscattering from a sea surface); here, real-life data were used for the study.
A Function Estimation Approach to Sequential Learning with Neural Networks
Authors: Kadirkamanathan Visakan , Niranjan Mahesan
Published: April 2008

In this paper, we investigate the problem of optimal sequential learning, viewed as a problem of estimating an underlying function sequentially rather than estimating a set of parameters of the neural network. First, we arrive at a suboptimal solution to the sequential estimate that can be mapped by a growing gaussian radial basis function (GaRBF) network. This network adds hidden units for each observation. The function space approach in which the estimates are represented as vectors in a function space is used in developing a growth criterion to limit its growth. A simplification of the criterion leads to two joint criteria on the distance of the present pattern and the existing unit centers in the input space and on the approximation error of the network for the given observation to be satisfied together. This network is similar to the resource allocating network (RAN) (Platt 1991a) and hence RAN can be interpreted from a function space approach to sequential learning. Second, we present an enhancement to the RAN. The RAN either allocates a new unit based on the novelty of an observation or adapts the network parameters by the LMS algorithm. The function space interpretation of the RAN lends itself to an enhancement of the RAN in which the extended Kalman filter (EKF) algorithm is used in place of the LMS algorithm. The performance of the RAN and the enhanced network are compared in the experimental tasks of function approximation and time-series prediction demonstrating the superior performance of the enhanced network with fewer number of hidden units. The approach adopted here has led us toward the minimal network required for a sequential learning problem.
Learning Finite State Machines With Self-Clustering Recurrent Networks
Authors: Zeng Zheng , Goodman Rodney M. , Smyth Padhraic
Published: April 2008

Recent work has shown that recurrent neural networks have the ability to learn finite state automata from examples. In particular, networks using second-order units have been successful at this task. In studying the performance and learning behavior of such networks we have found that the second-order network model attempts to form clusters in activation space as its internal representation of states. However, these learned states become unstable as longer and longer test input strings are presented to the network. In essence, the network “forgets” where the individual states are in activation space. In this paper we propose a new method to force such a network to learn stable states by introducing discretization into the network and using a pseudo-gradient learning rule to perform training. The essence of the learning rule is that in doing gradient descent, it makes use of the gradient of a sigmoid function as a heuristic hint in place of that of the hard-limiting function, while still using the discretized value in the feedback update path. The new structure uses isolated points in activation space instead of vague clusters as its internal representation of states. It is shown to have similar capabilities in learning finite state automata as the original network, but without the instability problem. The proposed pseudo-gradient learning rule may also be used as a basis for training other types of networks that have hard-limiting threshold activation functions.
The Role of Constraints in Hebbian Learning
Authors: Miller Kenneth D. , MacKay David J. C.
Published: April 2008

Models of unsupervised, correlation-based (Hebbian) synaptic plasticity are typically unstable: either all synapses grow until each reaches the maximum allowed strength, or all synapses decay to zero strength. A common method of avoiding these outcomes is to use a constraint that conserves or limits the total synaptic strength over a cell. We study the dynamic effects of such constraints.

Two methods of enforcing a constraint are distinguished, multiplicative and subtractive. For otherwise linear learning rules, multiplicative enforcement of a constraint results in dynamics that converge to the principal eigenvector of the operator determining unconstrained synaptic development. Subtractive enforcement, in contrast, typically leads to a final state in which almost all synaptic strengths reach either the maximum or minimum allowed value. This final state is often dominated by weight configurations other than the principal eigenvector of the unconstrained operator. Multiplicative enforcement yields a “graded” receptive field in which most mutually correlated inputs are represented, whereas subtractive enforcement yields a receptive field that is “sharpened” to a subset of maximally correlated inputs. If two equivalent input populations (e.g., two eyes) innervate a common target, multiplicative enforcement prevents their segregation (ocular dominance segregation) when the two populations are weakly correlated; whereas subtractive enforcement allows segregation under these circumstances.

These results may be used to understand constraints both over output cells and over input cells. A variety of rules that can implement constrained dynamics are discussed.
Toward a Theory of the Striate Cortex
Authors: Li Zhaoping , Atick Joseph J.
Published: April 2008

We explore the hypothesis that linear cortical neurons are concerned with building a particular type of representation of the visual world—one that not only preserves the information and the efficiency achieved by the retina, but in addition preserves spatial relationships in the input—both in the plane of vision and in the depth dimension. Focusing on the linear cortical cells, we classify all transforms having these properties. They are given by representations of the scaling and translation group and turn out to be labeled by rational numbers ‘(p + q)/p’ (p, q integers). Any given (p, q) predicts a set of receptive fields that comes at different spatial locations and scales (sizes) with a bandwidth of log2 [(p + q)/p] octaves and, most interestingly, with a diversity of ‘q’ cell varieties. The bandwidth affects the trade-off between preservation of planar and depth relations and, we think, should be selected to match structures in natural scenes. For bandwidths between 1 and 2 octaves, which are the ones we feel provide the best matching, we find for each scale a minimum of two distinct cell types that reside next to each other and in phase quadrature, that is, differ by 90° in the phases of their receptive fields, as are found in the cortex, they resemble the “even-symmetric” and “odd-symmetric” simple cells in special cases. An interesting consequence of the representations presented here is that the pattern of activation in the cells in response to a translation or scaling of an object remains the same but merely shifts its locus from one group of cells to another. This work also provides a new understanding of color coding changes from the retina to the cortex.
A Neural Network for Coding of Trajectories by Time Series of Neuronal Population Vectors
Authors: Lukashin Alexander V. , Georgopoulos Apostolos P.
Published: April 2008

The neuronal population vector is a measure of the combined directional tendency of the ensemble of directionally tuned cells in the motor cortex. It has been found experimentally that a trajectory of limb movement can be predicted by adding together population vectors, tip-to-tail, calculated for successive instants of time to construct a neural trajectory. In the present paper we consider a model of the dynamic evolution of the population vector. The simulated annealing algorithm was used to adjust the connection strengths of a feedback neural network so that it would generate a given trajectory by a sequence of population vectors. This was repeated for different trajectories. Resulting sets of connection strengths reveal a common feature regardless of the type of trajectories generated by the network: namely, the mean connection strength was negatively correlated with the angle between the preferred directions of neuronal pair involved in the connection. The results are discussed in the light of recent experimental findings concerning neuronal connectivity within the motor cortex.
Neural Network Model of the Cerebellum: Temporal Discrimination and the Timing of Motor Responses
Authors: Buonomano Dean V. , Mauk Michael D.
Published: April 2008

Substantial evidence has established that the cerebellum plays an important role in the generation of movements. An important aspect of motor output is its timing in relation to external stimuli or to other components of a movement. Previous studies suggest that the cerebellum plays a role in the timing of movements. Here we describe a neural network model based on the synaptic organization of the cerebellum that can generate timed responses in the range of tens of milliseconds to seconds. In contrast to previous models, temporal coding emerges from the dynamics of the cerebellar circuitry and depends neither on conduction delays, arrays of elements with different time constants, nor populations of elements oscillating at different frequencies. Instead, time is extracted from the instantaneous granule cell population vector. The subset of active granule cells is time-varying due to the granule—Golgi—granule cell negative feedback. We demonstrate that the population vector of simulated granule cell activity exhibits dynamic, nonperiodic trajectories in response to a periodic input. With time encoded in this manner, the output of the network at a particular interval following the onset of a stimulus can be altered selectively by changing the strength of granule → Purkinje cell connections for those granule cells that are active during the target time window. The memory of the reinforcement at that interval is subsequently expressed as a change in Purkinje cell activity that is appropriately timed with respect to stimulus onset. Thus, the present model demonstrates that a network based on cerebellar circuitry can learn appropriately timed responses by encoding time as the population vector of granule cell activity.
Elastic Net Model of Ocular Dominance: Overall Stripe Pattern and Monocular Deprivation
Authors: Goodhill Geoffrey J. , Willshaw David J.
Published: April 2008

The elastic net (Durbin and Willshaw 1987) can account for the development of both topography and ocular dominance in the mapping from the lateral geniculate nucleus to primary visual cortex (Goodhill and Willshaw 1990). Here it is further shown for this model that (1) the overall pattern of stripes produced is strongly influenced by the shape of the cortex: in particular, stripes with a global order similar to that seen biologically can be produced under appropriate conditions, and (2) the observed changes in stripe width associated with monocular deprivation are reproduced in the model.
Segmentation by a Network of Oscillators with Stored Memories
Authors: Sompolinsky H. , Tsodyks M.
Published: April 2008

We propose a model of coupled oscillators with noise that performs segmentation of stimuli using a set of stored images, each consisting of objects and a background. The oscillators' amplitudes encode the spatial and featural distribution of the external stimulus. The coherence of their phases signifies their belonging to the same object. In the learning stage, the couplings between phases are modified in a Hebb-like manner. By mean-field analysis and simulations, we show that an external stimulus whose local features resemble those of one or several of the stored objects generates a selective phase coherence that represents the stored pattern of segmentation.
Dimension Reduction of Biological Neuron Models by Artificial Neural Networks
Authors: Doya Kenji , Selverston Allen I.
Published: April 2008

An artificial neural network approach to dimension reduction of dynamical systems is proposed and applied to conductance-based neuron models. Networks with bottleneck layers of continuous-time dynamical units could make a two-dimensional model from the trajectories of the Hodgkin-Huxley model and a three-dimensional model from the trajectories of a six-dimensional bursting neuron model. Nullcline analysis of these reduced models revealed the bifurcations of the dynamical system underlying firing and bursting behaviors.
Neural Network Process Models Based on Linear Model Structures
Authors: Scott Gary M. , Ray W. Harmon
Published: April 2008

The KBANN (Knowledge-Based Artificial Neural Networks) approach uses neural networks to refine knowledge that can be written in the form of simple propositional rules. This idea is extended by presenting the MANNIDENT (Multivariable Artificial Neural Network Identification) algorithm by which the mathematical equations of linear dynamic process models determine the topology and initial weights of a network, which is further trained using backpropagation. This method is applied to the task of modeling a nonisothermal chemical reactor in which a first-order exothermic reaction is occurring. This method produces statistically significant gains in accuracy over both a standard neural network approach and a linear model. Furthermore, using the approximate linear model to initialize the weights of the network produces statistically less variation in model fidelity. By structuring the neural network according to the approximate linear model, the model can be readily interpreted.
Why Some Feedforward Networks Cannot Learn Some Polynomials
Authors: Cardell N. Scott , Joerding Wayne , Li Ying
Published: April 2008

It seems natural to test feedforward networks on deterministic functions. Yet, some simple functions, notably polynomials, present some difficult problems for approximation by feedforward networks. The estimated parameters become unbounded and fail to follow any unique pattern. Furthermore, as the fit to the specified functions becomes closer, numerical problems may develop in the algorithm. This paper explains why these problems occur for polynomials of order less than or equal to the number of hidden units of a feedforward network. We show that other examples occur for functions mathematically related to the network's squashing function. These difficulties do not indicate problems with the training algorithm, but occur as an inherent consequence of the role of the connection weights in feedforward networks.