mtybilly/Instruct2DS huggingface dataset:
An Object-Oriented Framework for the Simulation of Neural Nets
Authors: A. Linden, Th. Sudbrak, Ch. Tietz, F. Weber
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The field of software simulators for neural networks has been ex(cid:173) panding very rapidly in the last years but their importance is still  being underestimated. They must provide increasing levels of as(cid:173) sistance for the design, simulation and analysis of neural networks.  With our object-oriented framework (SESAME) we intend to show  that very high degrees of transparency, manageability and flexibil(cid:173) ity for complex experiments can be obtained. SESAME's basic de(cid:173) sign philosophy is inspired by the natural way in which researchers  explain their computational models. Experiments are performed  with networks of building blocks, which can be extended very eas(cid:173) ily. Mechanisms have been integrated to facilitate the construction  and analysis of very complex architectures. Among these mech(cid:173) anisms are t.he automatic configuration of building blocks for an  experiment and multiple inheritance at run-time.
Modeling Consistency in a Speaker Independent Continuous Speech Recognition System
Authors: Yochai Konig, Nelson Morgan, Chuck Wooters, Victor Abrash, Michael Cohen, Horacio Franco
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We  would like to  incorporate speaker-dependent consistencies, such  as  gender, in an otherwise speaker-independent speech recognition system.  In this paper we discuss a Gender Dependent Neural Network (GDNN)  which can  be tuned for each  gender,  while sharing most of the speaker  independent parameters. We use a classification network to help generate  gender-dependent phonetic probabilities for a statistical (HMM) recogni(cid:173) tion system.  The gender classification net predicts the gender with high  accuracy,  98.3% on a Resource Management test set.  However,  the in(cid:173) tegration of the GDNN into our hybrid HMM-neural network recognizer  provided an improvement in the recognition score that is not statistically  significant on a Resource Management test set.
Learning Curves, Model Selection and Complexity of Neural Networks
Authors: Noboru Murata, Shuji Yoshizawa, Shun-ichi Amari
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Learning  curves  show  how  a  neural  network  is  improved  as  the  number  of  t.raiuing  examples  increases  and  how  it  is  related  to  the  network  complexity.  The  present  paper  clarifies  asymptotic  properties and  their relation of t.wo  learning curves, one concerning  the predictive loss  or generalization  loss  and  the other the training  loss.  The  result  gives  a  natural  definition  of the  complexity  of  a  neural  network.  Moreover,  it  provides  a  new  criterion  of  model  selection.
Optimal Depth Neural Networks for Multiplication and Related Problems
Authors: Kai-Yeung Siu, Vwani Roychowdhury
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: An artificial neural network (ANN) is commonly modeled by a threshold  circuit, a network of interconnected processing units called linear threshold  gates. The depth of a network represents the number of unit delays or the  time for parallel computation. The SIze of a circuit is the number of gates  and measures the amount of hardware . It was known that traditional logic  circuits consisting of only unbounded fan-in AND, OR, NOT gates would  require at least O(log n/log log n) depth to compute common arithmetic  functions such as the product or the quotient of two n-bit numbers, unless  we allow the size (and fan-in) to increase exponentially (in n). We show in  this paper that ANNs can be much more powerful than traditional logic  circuits. In particular, we prove that that iterated addition can be com(cid:173) puted by depth-2 ANN, and multiplication and division can be computed  by depth-3 ANNs with polynomial size and polynomially bounded integer  weights, respectively. Moreover, it follows from known lower bound re(cid:173) sults that these ANNs are optimal in depth. We also indicate that these  techniques can be applied to construct polynomial-size depth-3 ANN for  powering, and depth-4 ANN for mUltiple product.
Time Warping Invariant Neural Networks
Authors: Guo-Zheng Sun, Hsing-Hen Chen, Yee-Chun Lee
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We proposed a model of Time Warping Invariant Neural Networks (TWINN) 
to handle the time warped continuous signals. Although TWINN is a simple modifica(cid:173) tion of well known recurrent neural  network, analysis  has  shown  that TWINN com(cid:173) pletely removes time warping and is  able to handle difficult classification problem. It  is also shown that TWINN has certain advantages over the current available sequential  processing  schemes:  Dynamic  Programming(DP)[I],  Hidden  Markov  Model((cid:173) HMM)[2],  Time  Delayed  Neural  Networks(TDNN)  [3]  and  Neural  Network  Finite  Automata(NNFA)[4]. 
We also analyzed the time continuity employed in TWINN and pointed out that 
this  kind of structure can memorize longer input history compared with Neural Net(cid:173) work Finite Automata (NNFA).  This  may  help  to  understand  the  well  accepted fact  that for  learning  grammatical reference with NNF A one had to start with  very short  strings in training set. 
The  numerical  example  we  used  is  a  trajectory  classification  problem.  This 
problem, making a feature of variable sampling rates, having internal states, continu(cid:173) ous  dynamics,  heavily  time-warped  data  and  deformed  phase  space  trajectories,  is  shown to be difficult to other schemes. With TWINN this problem has been learned in  100 iterations. For benchmark we also trained the exact same problem with TDNN and  completely failed as expected.
Connected Letter Recognition with a Multi-State Time Delay Neural Network
Authors: Hermann Hild, Alex Waibel
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The  Multi-State  Time  Delay  Neural  Network  (MS-TDNN)  inte(cid:173) grates  a nonlinear time alignment procedure  (DTW) and the high(cid:173) accuracy  phoneme spotting capabilities of a TDNN into  a  connec(cid:173) tionist speech recognition system with word-level classification and  error  backpropagation.  We  present  an  MS-TDNN  for  recognizing  continuously  spelled  letters,  a  task  characterized  by  a  small  but  highly confusable vocabulary.  Our MS-TDNN  achieves  98.5/92.0%  word  accuracy  on  speaker  dependent/independent  tasks,  outper(cid:173) forming previously reported results on the same databases.  We pro(cid:173) pose training techniques  aimed at improving sentence  level  perfor(cid:173) mance, including free  alignment  across  word  boundaries,  word  du(cid:173) ration  modeling and error backpropagation on  the sentence  rather  than the word level.  Architectures  integrating submodules special(cid:173) ized  on  a  subset  of speakers  achieved further  improvements.
Mapping Between Neural and Physical Activities of the Lobster Gastric Mill
Authors: Kenji Doya, Mary E. T. Boyle, Allen I. Selverston
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A computer model of the musculoskeletal system of the lobster  gastric mill was constructed in order to provide a behavioral in(cid:173) terpretation of the rhythmic patterns obtained from isolated stom(cid:173) atogastric ganglion. The model was based on Hill's muscle model  and quasi-static approximation of the skeletal dynamics and could  simulate the change of chewing patterns by the effect of neuromod(cid:173) ulators. 
1 THE STOMATOGASTRIC NERVOUS SYSTEM 
The crustacean stomatogastric ganglion (STG) is a circuit of 30 neurons that con(cid:173) trols rhythmic movement of the foregut. It is one of the best elucidated neural  circuits. All the neurons and the synaptic connections between them are identi(cid:173) fied and the effects of neuromodulators on the oscillation patterns and neuronal  characteristics have been extensively studied (Selverston and Moulins 1987, H arris(cid:173) Warrick et al. 1992). However, STG's function as a controller of ingestive behavior  is not fully understood in part because of our poor understanding of the controlled  object: the musculoskeletal dynamics of the foregut. We constructed a mathemat(cid:173) ical model of the gastric mill, three teeth in the stomach, in order to predict motor  patterns from the neural oscillation patterns which are recorded from the isolated  ganglion. 
The animal we used was the Californian spiny lobster (Panulirus interruptus), which
Nets with Unreliable Hidden Nodes Learn Error-Correcting Codes
Authors: Stephen Judd, Paul W. Munro
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: In a multi-layered neural network, anyone of the hidden layers can be  viewed as computing a distributed representation of the input. Several  "encoder" experiments have shown that when the representation space is  small it can be fully used. But computing with such a representation  requires completely dependable nodes. In the case where the hidden  nodes are noisy and unreliable, we find that error correcting schemes  emerge simply by using noisy units during training; random errors in(cid:173) jected during backpropagation result in spreading representations apart.  Average and minimum distances increase with misfire probability, as  predicted by coding-theoretic considerations. Furthennore, the effect of  this noise is to protect the machine against permanent node failure,  thereby potentially extending the useful lifetime of the machine.
Unsmearing Visual Motion: Development of Long-Range Horizontal Intrinsic Connections
Authors: Kevin E. Martin, Jonathan A. Marshall
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Human VlSlon systems integrate information nonlocally, across long  spatial ranges.  For example, a moving stimulus appears smeared  when viewed briefly (30 ms), yet sharp when viewed for a longer  exposure (100 ms) (Burr, 1980). This suggests that visual systems  combine information along a trajectory that matches the motion of  the stimulus. Our self-organizing neural network model shows how  developmental exposure to moving stimuli can direct the formation of  horizontal trajectory-specific motion integration pathways that unsmear  representations of moving stimuli. These results account for Burr's data  and can potentially also model ot.her phenomena, such as visual inertia.
Remote Sensing Image Analysis via a Texture Classification Neural Network
Authors: Hayit K. Greenspan, Rodney Goodman
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: In this work we apply a texture classification network to remote sensing im(cid:173) age analysis. The goal is to extract the characteristics of the area depicted  in the input image, thus achieving a segmented map of the region. We have  recently proposed a combined neural network and rule-based framework  for texture recognition. The framework uses unsupervised and supervised  learning, and provides probability estimates for the output classes. We  describe the texture classification network and extend it to demonstrate  its application to the Landsat and Aerial image analysis domain .
Filter Selection Model for Generating Visual Motion Signals
Authors: Steven J. Nowlan, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Neurons  in  area  MT of primate visual  cortex  encode  the  velocity  of moving objects.  We present  a  model of how  MT cells  aggregate  responses  from  VI  to  form  such  a  velocity  representation.  Two  different  sets  of units,  with  local  receptive  fields,  receive  inputs  from motion energy filters.  One set of units forms estimates of local  motion, while the second set computes the utility of these estimates.  Outputs from  this second set  of units  "gate"  the outputs from  the  first  set  through  a  gain  control  mechanism.  This  active  process  for  selecting  only  a  subset  of local  motion  responses  to  integrate  into more global  responses  distinguishes  our  model from  previous  models of velocity estimation.  The model yields  accurate  velocity  estimates  in  synthetic  images  containing multiple moving  targets  of varying size,  luminance,  and spatial frequency  profile and  deals  well  with a  number of transparency  phenomena.
Non-Linear Dimensionality Reduction
Authors: David DeMers, Garrison W. Cottrell
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A method for creating a non-linear encoder-decoder for multidimensional data  with  compact  representations  is  presented.  The commonly  used  technique of  autoassociation is extended  to allow non-linear representations,  and an  objec(cid:173) tive  function  which  penalizes activations  of individual  hidden  units  is  shown  to  result in minimum dimensional encodings with respect to allowable error in  reconstruction.
Feudal Reinforcement Learning
Authors: Peter Dayan, Geoffrey E. Hinton
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: One way to speed up reinforcement learning is to enable learning to  happen simultaneously at multiple resolutions in space and time.  This paper shows how to create a Q-Iearning managerial hierarchy  in which high level managers learn how to set tasks to their sub(cid:173) managers who, in turn, learn how to satisfy them.  Sub-managers  need  not initially understand  their managers' commands.  They  simply learn to maximise their reinforcement in the context of the  current command.  We illustrate the system using a simple maze task ..  As the system  learns  how to get around,  satisfying commands at the multiple  levels, it explores more efficiently than standard, flat,  Q-Iearning  and builds a more comprehensive map.
A Neural Network that Learns to Interpret Myocardial Planar Thallium Scintigrams
Authors: Charles Rosenberg, Jacob Erel, Henri Atlan
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The planar thallium-201 myocardial perfusion scintigram is a widely used  diagnostic technique for detecting and estimating the risk of coronary  artery disease. Neural networks learned to interpret 100 thallium scinti(cid:173) grams as determined by individual expert ratings. Standard error back(cid:173) propagation was compared to standard LMS, and LMS combined with  one layer of RBF units. Using the "leave-one-out" method, generaliza(cid:173) tion was tested on all 100 cases. Training time was determined automati(cid:173) cally from cross-validation perfonnance. Best perfonnance was attained  by the RBF/LMS network with three hidden units per view and compares  favorably with human experts.
Learning Cellular Automaton Dynamics with Neural Networks
Authors: N. H. Wulff, J A Hertz
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We have trained networks of E - II units with short-range connec(cid:173) tions to simulate simple cellular automata that exhibit complex or  chaotic behaviour. Three levels of learning are possible (in decreas(cid:173) ing order of difficulty): learning the underlying automaton rule,  learning asymptotic dynamical behaviour, and learning to extrap(cid:173) olate the training history. The levels of learning achieved with and  without weight sharing for different automata provide new insight  into their dynamics.
Probability Estimation from a Database Using a Gibbs Energy Model
Authors: John W. Miller, Rodney M. Goodman
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We present an algorithm for creating a neural network which pro(cid:173) duces accurate probability estimates as outputs. The network im(cid:173) plements a Gibbs probability distribution model of the training  database. This model is created by a new transformation relating  the joint probabilities of attributes in the database to the weights  (Gibbs potentials) of the distributed network model. The theory  of this transformation is presented together with experimental re(cid:173) sults. One advantage of this approach is the network weights are  prescribed without iterative gradient descent. Used as a classifier  the network tied or outperformed published results on a variety of  databases.
An Analog VLSI Chip for Radial Basis Functions
Authors: Janeen Anderson, John C. Platt, David B. Kirk
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We have designed, fabricated, and tested an analog VLSI chip  which computes radial basis functions in parallel. We have de(cid:173) veloped a synapse circuit that approximates a quadratic function.  We aggregate these circuits to form radial basis functions. These  radial basis functions are then averaged together using a follower  aggregator.
Predicting Complex Behavior in Sparse Asymmetric Networks
Authors: Ali A. Minai, William B. Levy
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Recurrent  networks  of threshold  elements  have  been  studied  inten(cid:173) sively  as  associative  memories and pattern-recognition devices. While  most  research  has  concentrated  on  fully-connected  symmetric  net(cid:173) works.  which  relax  to  stable  fixed  points.  asymmetric  networks  show  richer dynamical behavior. and can be used as  sequence  generators or  flexible  pattern-recognition  devices.  In  this  paper.  we  approach  the  problem of predicting  the  complex global behavior of a  class  of ran(cid:173) dom asymmetric networks in terms of network parameters.  These net(cid:173) works can show fixed-point.  cyclical or effectively aperiodic  behavior.  depending  on  parameter  values.  and  our approach  can  be used  to  set  parameters.  as  necessary.  to  obtain a  desired complexity of dynamics.  The  approach  also  provides  qualitative  insight  into  why  the  system  behaves as it does and suggests possible applications.
Destabilization and Route to Chaos in Neural Networks with Random Connectivity
Authors: Bernard Doyon, Bruno Cessac, Mathias Quoy, Manuel Samuelides
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The occurence of chaos  in recurrent neural  networks  is  supposed to  depend on the architecture and on the synaptic coupling strength. It is  studied here for a randomly diluted architecture.  By normalizing  the  variance  of synaptic  weights,  we  produce  a  bifurcation  parameter,  dependent on this variance and on the slope of the transfer function but  independent of the connectivity, that allows a sustained activity and the  occurence  of  chaos  when  reaching  a  critical  value.  Even  for  weak  connectivity  and small size, we find  numerical results in accordance  with the theoretical  ones  previously  established  for  fully  connected  infinite  sized  networks.  Moreover  the  route  towards  chaos  is  numerically checked to be a quasi-periodic one, whatever the type of the  first bifurcation is (Hopf bifurcation, pitchfork or flip).
Recognition-based Segmentation of On-Line Hand-printed Words
Authors: M. Schenkel, H. Weissman, I. Guyon, C. Nohl, D. Henderson
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: This paper reports on the performance of two methods for  recognition-based segmentation of strings of on-line hand-printed  capital Latin characters. The input strings consist of a time(cid:173) ordered sequence of X-Y coordinates, punctuated by pen-lifts. The  methods were designed to work in "run-on mode" where there is no  constraint on the spacing between characters. While both methods  use a neural network recognition engine and a graph-algorithmic  post-processor, their approaches to segmentation are quite differ(cid:173) ent. The first method, which we call IN SEC (for input segmen(cid:173) tation), uses a combination of heuristics to identify particular pen(cid:173) lifts as tentative segmentation points. The second method, which  we call OUTSEC (for output segmentation), relies on the empiri(cid:173) cally trained recognition engine for both recognizing characters and  identifying relevant segmentation points.
Context-Dependent Multiple Distribution Phonetic Modeling with MLPs
Authors: Michael Cohen, Horacio Franco, Nelson Morgan, David E. Rumelhart, Victor Abrash
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A  number  of  hybrid  multilayer  perceptron  (MLP)/hidden  Markov  model  (HMM:)  speech  recognition  systems  have  been  developed  in  recent  years  (Morgan  and Bourlard.  1990).  In  this  paper.  we  present  a  new  MLP  architecture  and  training  algorithm  which  allows  the  modeling  of  context-dependent  phonetic  classes  in  a  hybrid  MLP/HMM:  framework.  The  new  training  procedure  smooths  MLPs  trained  at  different  degrees  of context  dependence  in  order  to obtain  a  robust  estimate  of the  cootext-dependent  probabilities.  Tests  with  the  DARPA  Resomce  Management  database  have  shown  substantial  advantages  of  the  context-dependent  MLPs  over  earlier  cootext(cid:173) independent  MLPs.  and  have  shown  substantial  advantages  of  this  hybrid approach over a pure  HMM approach.
Automatic Capacity Tuning of Very Large VC-Dimension Classifiers
Authors: I. Guyon, B. Boser, V. Vapnik
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Large VC-dimension classifiers can learn difficult tasks, but are usually  impractical because they generalize well only if they are trained with huge  quantities of data. In this paper we show that even high-order polynomial  classifiers in high dimensional spaces can be trained with a small amount  of training data and yet generalize better than classifiers with a smaller  VC-dimension. This is achieved with a maximum margin algorithm (the  Generalized Portrait). The technique is applicable to a wide variety of  classifiers, including Perceptrons, polynomial classifiers (sigma-pi unit net(cid:173) works) and Radial Basis Functions. The effective number of parameters is  adjusted automatically by the training algorithm to match the complexity  of the problem. It is shown to equal the number of those training patterns  which are closest patterns to the decision boundary (supporting patterns).  Bounds on the generalization error and the speed of convergence of the al(cid:173) gorithm are given. Experimental results on handwritten digit recognition  demonstrate good generalization compared to other algorithms.
Biologically Plausible Local Learning Rules for the Adaptation of the Vestibulo-Ocular Reflex
Authors: Olivier Coenen, Terrence J. Sejnowski, Stephen G. Lisberger
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The vestibulo-ocular reflex (VOR) is a compensatory eye movement that  stabilizes images on the retina during head turns. Its magnitude, or gain,  can be modified by visual experience during head movements. Possible  learning mechanisms for this adaptation have been explored in a model  of the oculomotor system based on anatomical and physiological con(cid:173) straints. The local correlational learning rules in our model reproduce the  adaptation and behavior of the VOR under certain parameter conditions.  From these conditions, predictions for the time course of adaptation at  the learning sites are made.
Combining Neural and Symbolic Learning to Revise Probabilistic Rule Bases
Authors: J. Jeffrey Mahoney, Raymond J. Mooney
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: a system for  revising probabilis(cid:173)
This paper describes  RAPTURE  - tic  knowledge  bases  that  combines  neural  and  symbolic  learning  methods.  RAPTURE  uses  a  modified  version  of backpropagation  to refine  the  certainty factors  of a  MYCIN-style  rule base  and  uses  ID3's  information gain  heuristic  to  add  new  rules.  Results on  re(cid:173) fining  two  actual  expert  knowledge  bases  demonstrate  that  this  combined approach performs better  than previous methods.
Transient Signal Detection with Neural Networks: The Search for the Desired Signal
Authors: José Carlos Príncipe, Abir Zahalka
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Matched  filtering  has  been  one  of the  most  powerful  techniques  employed for transient detection. Here we will show that a dynamic  neural  network  outperforms  the  conventional  approach.  When  the  artificial neural  network (ANN) is  trained with supervised learning  schemes  there  is  a  need  to  supply  the  desired  signal  for  all  time,  although  we  are  only  interested  in  detecting  the  transient.  In  this  paper  we  also  show  the  effects  on  the  detection  agreement  of  different strategies to construct the desired signal. The extension of  the  Bayes  decision  rule  (011  desired  signal),  optimal  in  static  classification,  performs  worse  than  desired  signals  constructed by  random noise or prediction during the background.
Information Theoretic Analysis of Connection Structure from Spike Trains
Authors: Satoru Shiono, Satoshi Yamada, Michio Nakashima, Kenji Matsumoto
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We have attempted to use information theoretic quantities for  ana(cid:173) lyzing neuronal connection  structure from spike  trains.  Two point  mu tual information  and  its maximum value,  channel capacity,  be(cid:173) tween  a  pair of neurons  were  found  to  be  useful  for  sensitive  de(cid:173) tection  of crosscorrelation  and  for  estimation  of synaptic strength,  respectively.  Three point mutual information among three neurons  could give their interconnection structure.  Therefore, our informa(cid:173) tion  theoretic analysis  was  shown  to be a  very  powerful technique  for  deducing  neuronal  connection  structure.  Some  concrete exam(cid:173) ples  of its  application  to simulated  spike  trains  are presented.
Comparing the Performance of Connectionist and Statistical Classifiers on an Image Segmentation Problem
Authors: Sheri L. Gish, W. E. Blanz
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: In  this  study,  we  test  the  suitability  of a  connection(cid:173)
In the development  of an image segmentation system for  real time  image processing applications,  we  apply the classical decision anal(cid:173) ysis  paradigm by  viewing image segmentation as a  pixel classifica.(cid:173) tion  task.  We  use  supervised  training  to derive  a  classifier for  our  system  from  a  set  of  examples  of a  particular  pixel  classification  problem.  ist  method  against  two  statistical  methods,  Gaussian  maximum  likelihood  classifier  and first,  second,  and third degree  polynomial  classifiers,  for  the  solution  of a  "real  world"  image  segmentation  problem  taken  from  combustion  research.  Classifiers  are  derived  using  all  three  methods,  and  the  performance  of all  of the  classi(cid:173) fiers  on  the  training  data set  as  well  as  on  3  separate  entire  test  images  is  measured.
Can Simple Cells Learn Curves? A Hebbian Model in a Structured Environment
Authors: William R. Softky, Daniel M. Kammen
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: In the mammalian visual cortex,  orientation-selective 'simple cells'  which detect  straight lines  may be adapted  to detect  curved  lines  instead.  We  test  a  biologically  plausible,  Hebbian,  single-neuron  model,  which learns oriented receptive fields  upon exposure  to un(cid:173) structured (noise) input and maintains orientation selectivity upon  exposure  to  edges  or  bars  of all  orientations  and  positions.  This  model  can  also  learn  arc-shaped  receptive  fields  upon  exposure  to an environment of only  circular  rings.  Thus,  new  experiments  which try to induce an abnormal (curved) receptive field  may pro(cid:173) vide insight into the plasticity of simple cells.  The model suggests  that exposing  cells  to only  a  single  spatial  frequency  may induce  more  striking spatial frequency  and  orientation  dependent effects  than heretofore observed.
Development and Regeneration of Eye-Brain Maps: A Computational Model
Authors: Jack D. Cowan, A. E. Friedman
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We outline a computational model  of the development and regenera(cid:173) tion of specific eye-brain circuits. The model comprises a self-organiz(cid:173) ing map-forming network which uses local Hebb rules. constrained by  molecular markers.  Various  simulations of the development of eye(cid:173) brain maps in fish and frogs are described.
Neural Network Analysis of Distributed Representations of Dynamical Sensory-Motor Transformations in the Leech
Authors: Shawn R. Lockery, Yan Fang, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: Interneurons in leech ganglia receive multiple sensory inputs and make  synaptic contacts with many motor neurons. These "hidden" units  coordinate several different behaviors. We used physiological and  anatomical constraints to construct a model of the local bending reflex.  Dynamical networks were trained on experimentally derived input-output  patterns using recurrent back-propagation. Units in the model were  modified to include electrical synapses and multiple synaptic time  constants. The properties of the hidden units that emerged in the  simulations matched those in the leech. The model and data support  distributed rather than localist representations in the local bending reflex.  These results also explain counterintuitive aspects of the local bending  circuitry.
A Computer Modeling Approach to Understanding the Inferior Olive and Its Relationships to the Cerebellar Cortex in Rats
Authors: Maurice Lee, James M. Bower
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: This paper presents the results of a simulation of the spatial relationship  between the inferior olivary nucleus and  folium  crus IIA of the lateral  hemisphere  of  the  rat  cerebellum.  The  principal  objective  of  this  modeling effort was to resolve an apparent conflict between a proposed  zonal organization of olivary projections to cerebellar cortex suggested  by  anatomical  tract-tracing  experiments  (Brodal  &  Kawamura  1980;  Campbell & Armstrong  1983) and a more patchy organization apparent  with physiological mapping (Robertson  1987).  The results suggest that  several unique features  of the olivocerebellar circuit may contribute to  the appearance of zonal organization using anatomical  techniques, but  that  the  detailed  patterns  of  patchy  tactile  projections  seen  with  physiological  techniques  are  a  more  accurate  representation  of  the  afferent organization of this region of cortex.
Neural Network Weight Matrix Synthesis Using Optimal Control Techniques
Authors: O. Farotimi, Amir Dembo, Thomas Kailath
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: T.  Kailath 
Given a set  of input-output  training samples,  we  describe  a  proce(cid:173) dure  for  determining  the  time sequence  of weights  for  a  dynamic  neural  network  to  model  an  arbitrary  input-output  process.  We  formulate  the  input-output  mapping  problem  as  an  optimal con(cid:173) trol  problem,  defining  a  performance  index  to  be  minimized  as  a  function  of time-varying  weights.  We  solve  the  resulting  nonlin(cid:173) ear two-point-boundary-value problem, and this yields the  training  rule.  For the performance index chosen,  this rule  turns out  to be  a  continuous time generalization of the outer product rule earlier sug(cid:173) gested heuristically by Hopfield for designing associative memories.  Learning curves for  the new  technique are  presented.
Collective Oscillations in the Visual Cortex
Authors: Daniel M. Kammen, Christof Koch, Philip J. Holmes
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The  firing  patterns  of populations  of cells  in  the  cat  visual  cor(cid:173) tex can  exhibit  oscillatory  responses  in  the  range  of 35  - 85  Hz.  Furthermore,  groups  of neurons  many  mm's  apart  can  be  highly  synchronized  as  long  as  the  cells  have  similar  orientation  tuning.  We investigate two basic network architectures that incorporate ei(cid:173) ther nearest-neighbor or global feedback interactions and conclude  that non-local feedback plays a  fundamental role in the initial syn(cid:173) chronization and dynamic stability of the oscillations.
Operational Fault Tolerance of CMAC Networks
Authors: Michael J. Carter, Franklin J. Rudolph, Adam J. Nucci
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The performance sensitivity of Albus' CMAC network was studied for  the scenario in which faults are introduced into the adjustable weights  after training has been accomplished.  It was found that fault sensitivity  was reduced with increased generalization when "loss of weight" faults  were considered,  but sensitivity  was increased for  "saturated weight"  faults.
Effects of Firing Synchrony on Signal Propagation in Layered Networks
Authors: G. T. Kenyon, Eberhard E. Fetz, R. D. Puff
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: Spiking  neurons  which  integrate  to  threshold  and  fire  were  used  to  study  the  transmission  of  frequency  modulated  (FM)  signals  through  layered networks.  Firing correlations  between cells  in  the  input  layer  were  found  to  modulate  the  transmission  of  FM  sig(cid:173) nals  under  certain dynamical  conditions.  A  tonic  level  of activity  was  maintained  by  providing  each  cell  with  a  source  of Poisson(cid:173) distributed  synaptic  input.  When  the  average  membrane  depo(cid:173) larization  produced  by  the  synaptic  input  was  sufficiently  below  threshold,  the  firing  correlations  between  cells  in  the  input  layer  could greatly amplify the signal present in subsequent layers.  When  the  depolarization was sufficiently close  to threshold,  however,  the  firing  synchrony  between cells  in the  initial layers could  no  longer  effect  the propagation of FM  signals.  In this latter case,  integrate(cid:173) and-fire  neurons  could  be  effectively  modeled  by  simpler  analog  elements governed by a  linear input-output relation.
Neural Network Visualization
Authors: Jakub Wejchert, Gerald Tesauro
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We  have developed graphics  to visualize  static and dynamic infor(cid:173) mation in layered neural network learning systems.  Emphasis  was  placed  on  creating new  visuals  that  make  use  of spatial arrange(cid:173) ments,  size  information,  animation  and  color.  We  applied  these  tools  to  the  study  of back-propagation learning of simple  Boolean  predicates,  and  have  obtained  new  insights  into  the  dynamics  of  the learning process.
Discovering the Structure of a Reactive Environment by Exploration
Authors: Michael Mozer, Jonathan Bachrach
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: Consider a robot wandering around an unfamiliar environment. performing ac(cid:173) tions and sensing the resulting environmental states.  The robot's task is to con(cid:173) struct an internal model of its environment. a model that will allow it to predict  the consequences of its actions  and to determine what sequences of actions  to  take  to  reach  particular  goal  states.  Rivest  and  Schapire  (1987&,  1987b;  Schapire.  1988) have studied this problem and have designed a symbolic algo(cid:173) rithm  to  strategically explore  and infer the  structure of "finite state"  environ(cid:173) ments.  The heart of this algorithm is a clever representation of the environment  called an update graph.  We have developed a connectionist implementation of  the update  graph using  a highly-specialized network  architecture.  With  back  propagation learning and a trivial exploration strategy - tions - gorithm on simple problems.  The network has  the  additional  strength  that it  can accommodate stochastic environments.  Perhaps the  greatest virtue of the  connectionist  approach  is  that  it suggests  generalizations of the update graph  representation that do not arise from a traditional, symbolic perspective. 
choosing random ac(cid:173) the connectionist network can outperfonn the Rivest and Schapire al(cid:173)
Complexity of Finite Precision Neural Network Classifier
Authors: Amir Dembo, Kai-Yeung Siu, Thomas Kailath
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: A rigorous analysis on the finite  precision computational <)Spects  of  neural  network as  a  pattern  classifier  via a  probabilistic  approach  is  presented.  Even though there exist negative results on  the capa(cid:173) bility of perceptron,  we  show  the following  positive results:  Given  n  pattern vectors each represented by en bits where  e > 1,  that are  uniformly  distributed,  with  high  probability  the  perceptron  can  perform  all  possible  binary  classifications  of the  patterns.  More(cid:173) over, the resulting neural network requires a  vanishingly small pro(cid:173) portion O(log n/n) of the memory that would be required for  com(cid:173) plete  storage  of the  patterns.  Further,  the  perceptron  algorithm  takes  O(n2)  arithmetic operations  with  high  probability,  whereas  other  methods  such  as  linear  programming  takes  O(n3 .5 )  in  the  worst  case.  We also  indicate some mathematical connections  with  VLSI  circuit  testing and the theory  of random matrices.
Dimensionality Reduction and Prior Knowledge in E-Set Recognition
Authors: Kevin J. Lang, Geoffrey E. Hinton
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: It is  well known  that  when an  automatic  learning algorithm  is  applied  to a  fixed  corpus  of data,  the size of the corpus  places  an  upper bound  on  the  number  of degrees  of freedom  that  the  model  can  contain  if  it  is  to generalize  well.  Because  the  amount  of hardware  in  a  neural  network  typically  increases  with  the  dimensionality  of  its  inputs,  it  can be challenging to build a high-performance network for classifying  large input patterns.  In this paper, several techniques for addressing this  problem  are  discussed  in  the context of an  isolated  word  recognition  task.
Adjoint Operator Algorithms for Faster Learning in Dynamical Neural Networks
Authors: Jacob Barhen, Nikzad Benny Toomarian, Sandeep Gulati
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: A  methodology for  faster supervised learning in  dynamical nonlin(cid:173) ear neural networks is  presented.  It exploits the concept  of adjoint  operntors  to  enable  computation  of changes  in  the  network's  re(cid:173) sponse due to perturbations in all system parameters,  using the so(cid:173) lution of a single set of appropriately constructed  linear equations.  The  lower  bound  on  speedup  per  learning  iteration  over  conven(cid:173) tional methods for  calculating the neuromorphic energy gradient is  O(N2),  where  N  is  the number  of neurons  in  the network.
Computer Simulation of Oscillatory Behavior in Cerebral Cortical Networks
Authors: Matthew A. Wilson, James M. Bower
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: It has been known for  many years that specific regions of the work(cid:173) ing cerebral cortex display periodic variations in correlated cellular  activity.  While the olfactory system has  been the focus  of much of  this  work,  similar  behavior  has  recently  been observed  in  primary  visual  cortex.  We  have  developed  models  of  both  the  olfactory  and  visual  cortex which  replicate  the observed  oscillatory  proper(cid:173) ties of these  networks.  Using  these  models  we  have  examined  the  dependence of oscillatory behavior on single cell properties and net(cid:173) work  architectures.  We  discuss the idea that  the oscillatory  events  recorded from  cerebral cortex may  be intrinsic  to the architecture  of cerebral  cortex  as  a  whole,  and  that  these  rhythmic  patterns  may be important in coordinating neuronal activity during sensory  processmg.
Coupled Markov Random Fields and Mean Field Theory
Authors: Davi Geiger, Federico Girosi
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: Federico  Girosi  Artificial Intelligence  Laboratory,  MIT  545  Tech.  Sq.  #  788  Cambridge, MA 02139 
In recent years many researchers have investigated the use of Markov  Random Fields (MRFs)  for  computer vision.  They can be  applied  for  example  to  reconstruct  surfaces  from  sparse  and  noisy  depth  data  coming from  the  output  of a  visual  process,  or  to  integrate  early vision  processes  to label  physical  discontinuities.  In  this  pa(cid:173) per  we  show  that  by  applying  mean field  theory  to  those  MRFs  models a  class of neural  networks is  obtained.  Those networks can  speed  up  the  solution  for  the  MRFs  models.  The  method  is  not  restricted  to computer vision.
Pulse-Firing Neural Chips for Hundreds of Neurons
Authors: Michael Brownlow, Lionel Tarassenko, Alan F. Murray, Alister Hamilton, Il Song Han, H. Martin Reekie
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We  announce  new  CMOS  synapse  circuits  using  only  three  and four  MOSFETsisynapse.  Neural states are asynchronous  pulse  streams,  upon  which  arithmetic  is  performed  directly.  Chips  implementing  over  100  fully  programmable  synapses  are  described  and  projections  to  networks  of  hundreds  of  neurons are made. 
1 OVERVIEW OF PULSE FIRING NEURAL  VLSI  The  inspiration  for  the  use  of  pulse  firing  in  silicon  neural  networks  is  clearly  the  electrical/chemical  pulse  mechanism  in  "real"  biological  neurons.  Asynchronous,  digital  voltage  pulses  are  used  to  signal  states  t Si  )  through  synapse  weights  {  Tij  }  to  emulate  neural  dynamics.  Neurons  fire  voltage  pulses  of a frequency  determined  by  their  level  of activity  but of a  constant  magnitude  (usually  5  Volts)  [Murray,1989a].  As  indicated  in  Fig.  1,  to  synapses  perform  arithmetic  directly  on  these  asynchronous  pulses,  increment  or  decrement  the  receiving  neuron's  activity.  The  activity  of  a  receiving  neuron  i,  Xi  is  altered  at  a  frequency  controlled  by  the  sending  neuron  j,  with  state  Sj  by  an  amount  determined  by  the  synapse  weight  (here,  T ij ). 
1 On secondment from  the Korean Telecommunications Authority 
786 
Brownlow, Tarassenko, Murray, Hamilton, Han and Reekie
Rule Representations in a Connectionist Chunker
Authors: David S. Touretzky, Gillette Elvgreen III
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We  present  two  connectionist  architectures  for  chunking  of symbolic  rewrite rules.  One uses backpropagation learning, the other competitive  learning.  Although  they  were  developed  for  chunking  the  same  sorts  of rules,  the  two  differ  in  their  representational  abilities  and  learning  behaviors.
Speaker Independent Speech Recognition with Neural Networks and Speech Knowledge
Authors: Yoshua Bengio, Renato de Mori, Régis Cardin
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: We  attempt to  combine neural  networks with  knowledge  from  speech  science to build  a speaker independent speech recogni(cid:173) tion  system.  This  knowledge  is  utilized  in  designing  the  preprocessing,  input coding, output coding,  output supervision  and  architectural  constraints.  To  handle  the  temporal  aspect  of speech we  combine  delays,  copies  of activations  of hidden  and  output  units  at  the  input  level,  and  Back-Propagation  for  Sequences  (BPS),  a  learning algorithm for networks with  local  self-loops.  This  strategy  is  demonstrated  in  several  experi(cid:173) ments,  in  particular  a  nasal  discrimination  task  for  which  the  application  of  a  speech  theory  hypothesis  dramatically  im(cid:173) proved generalization.
An Analog VLSI Model of Adaptation in the Vestibulo-Ocular Reflex
Authors: Stephen P. DeWeerth, Carver Mead
Published in: Advances in Neural Information Processing Systems 2  (NIPS 1989) (Main Conference Track), 1989

Abstract: The vestibulo-ocular reflex (VOR)  is  the primary mechanism that  controls the compensatory eye movements that stabilize retinal im(cid:173) ages during rapid head motion.  The primary pathways of this sys(cid:173) tem are feed-forward,  with inputs from  the semicircular canals and  outputs  to  the  oculomotor system.  Since  visual  feedback  is  not  used  directly  in  the  VOR  computation,  the  system  must  exploit  motor learning to perform correctly.  Lisberger(1988) has proposed  a  model for  adapting the  VOR gain  using  image-slip  information  from  the  retina.  We  have  designed  and  tested  analog  very  large(cid:173) scale integrated (VLSI) circuitry that implements a  simplified ver(cid:173) sion of Lisberger's adaptive VOR model.