mtybilly/Instruct2DS huggingface dataset:
A Cortico-Cerebellar Model that Learns to Generate Distributed Motor Commands to Control a Kinematic Arm
Authors: S. P. Singh, A. G. Barto, J. C. Houk N. E. Berthier
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A neurophysiologically-based model is presented that controls a simulated  kinematic arm during goal-directed reaches. The network generates a  quasi-feedforward motor command that is learned using training signals  generated by corrective movements. For each target, the network selects  and sets the output of a subset of pattern generators. During the move(cid:173) ment, feedback from proprioceptors turns off the pattern generators. The  task facing individual pattern generators is to recognize when the arm  reaches the target and to turn off. A distributed representation of the mo(cid:173) tor command that resembles population vectors seen in vivo was produced  naturally by these simulations.
Linear Operator for Object Recognition
Authors: Shimon Ullman Ronen Basri
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Visual  object  recognition  involves  the  identification of images of 3-D  ob(cid:173) jects seen  from  arbitrary  viewpoints.  We  suggest  an  approach  to object  recognition  in  which  a  view  is  represented  as  a  collection  of points  given  by their location in the image.  An object is  modeled by a set of 2-D views  together  with  the  correspondence  between  the  views.  We show that  any  novel  view  of the  object  can  be expressed  as a  linear  combination  of the  stored  views.  Consequently,  we  build  a  linear operator that  distinguishes  between views of a specific object and views of other objects.  This opera(cid:173) tor can be implemented using neural network architectures with relatively  simple structures.
Learning Unambiguous Reduced Sequence Descriptions
Authors: Jürgen Schmidhuber
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Do you  want  your  neural  net  algorithm  to learn  sequences?  Do not lim(cid:173) it  yourself to  conventional  gradient  descent  (or  approximations  thereof).  Instead,  use  your sequence  learning  algorithm  (any will  do)  to implement  the  following  method  for  history  compression.  No  matter  what  your  fi(cid:173) nal goals  are,  train  a  network  to predict  its next input from  the  previous  ones.  Since  only  unpredictable  inputs  convey  new  information, ignore  all  predictable  inputs  but let  all  unexpected  inputs  (plus  information  about  the  time  step  at  which  they  occurred)  become  inputs  to  a  higher-level  network of the same kind  (working on a  slower,  self-adjusting  time scale).  Go on  building  a  hierarchy  of such  networks.  This  principle  reduces  the  descriptions  of event  sequences  without  1088  of information,  thus  easing  supervised  or  reinforcement  learning  tasks.  Alternatively,  you  may  use  two recurrent  networks  to collapse  a  multi-level  predictor hierarchy  into a  single  recurrent  net.  Experiments  show  that systems based on  these prin(cid:173) ciples  can require less  computation per  time step and many fewer  training  sequences  than conventional training algorithms for  recurrent  nets.  Final(cid:173) ly you can modify the above method such that predictability is not defined  in  a  yes-or-no fashion  but in a  continuous fashion.
Dynamically-Adaptive Winner-Take-All Networks
Authors: Trent E. Lange
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Winner-Take-All  (WTA)  networks.  in  which  inhibitory  interconnec(cid:173) tions are used to determine the most highly-activated of a pool of unilS.  are an important part of many  neural network models.  Unfortunately,  convergence of normal  WT A networks  is  extremely  sensitive  to  the  magnitudes of their weights, which must be hand-tuned and which gen(cid:173) erally  only  provide  the  right amount of inhibition across a relatively  small  range of initial  conditions.  This  paper  presents Dynamjcally(cid:173) Adaptive Winner-Telke-All  (DA WTA)  netw
Some Approximation Properties of Projection Pursuit Learning Networks
Authors: Christopher G. Atkeson Ying Zhao
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: This paper will address an important question in machine learning:  What  kind  of network architectures  work  better on  what  kind  of problems?  A  projection pursuit  learning  network  has  a  very similar  structure to a  one  hidden  layer  sigmoidal  neural  network.  A  general  method  based  on  a  continuous  version  of projection  pursuit  regression  is  developed  to  show  that  projection  pursuit  regression  works  better  on angular  smooth func(cid:173) tions  than on  Laplacian  smooth functions.  There exists  a  ridge  function  approximation  scheme  to  avoid  the  curse  of dimensionality  for  approxi(cid:173) mating functions  in  L2(¢d).
Unsupervised learning of distributions on binary vectors using two layer networks
Authors: David Haussler Yoav Freund
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We study a particular type of Boltzmann machine with a bipartite graph structure called a harmo(cid:173) nium. Our interest is in using such a machine to model a probability distribution on binary input  vectors. We analyze the class of probability distributions that can be modeled by such machines.  showing that for each n ~ 1 this class includes arbitrarily good appwximations to any distribution  on the set of all n-vectors of binary inputs. We then present two learning algorithms for these  machines .. The first learning algorithm is the standard gradient ascent heuristic for computing  maximum likelihood estimates for the parameters (i.e. weights and thresholds) of the modeL Here  we give a closed form for this gradient that is significantly easier to compute than the corresponding  gradient for the general Boltzmann machine . The second learning algorithm is a greedy method  that creates the hidden units and computes their weights one at a time. This method is a variant  of the standard method for projection pursuit density estimation . We give experimental results for  these learning methods on synthetic data and natural data from the domain of handwritten digits.
Best-First Model Merging for Dynamic Learning and Recognition
Authors: Stephen M. Omohundro
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: "Best-first  model  merging"  is  a  general  technique  for  dynamically  choosing  the  structure of a  neural  or related  architecture while avoid(cid:173) ing  overfitting.  It is  applicable  to both  leaming  and  recognition  tasks  and often generalizes significantly better than fixed structures. We dem(cid:173) onstrate the approach applied to the tasks of choosing radial basis func(cid:173) tions for function  learning, choosing  local  affine models  for  curve and  constraint surface modelling, and choosing the structure of a balltree or  bumptree to maximize efficiency of access. 
1  TOWARD MORE COGNITIVE LEARNING  Standard backpropagation neural networks learn in a way which appears to be quite differ(cid:173) ent from  human leaming. Viewed as a cognitive system, a standard network always main(cid:173) tains  a  complete  model  of its  domain.  This  model  is  mostly  wrong  initially,  but  gets  gradually better and better as data appears. The net deals with all data in  much  the same  way and has no representation for the strength of evidence behind a certain conclusion. The  network architecture is usually chosen before any data is seen and the processing is much  the same in the early phases of learning as in the late phases.  Human and animalleaming appears to proceed in quite a different manner. When an organ(cid:173) ism has not had many experiences in a domain of importance to it, each individual experi(cid:173) ence is critical. Rather than use such an experience to slightly modify the parameters of a  global model, a better strategy is to remember the experience in detail. Early in learning. an  organism doesn't know which features of an experience are important unless it has a strong
Neural Computing with Small Weights
Authors: Jehoshua Bruck Kai-Yeung Siu
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: An important issue in neural computation is the dynamic range of weights  in  the  neural  networks.  Many  experimental  results  on  learning  indicate  that  the  weights  in  the  networks  can  grow  prohibitively  large  with  the  size  of the  inputs.  Here  we  address  this  issue  by  studying  the  tradeoffs  between  the  depth  and  the  size  of weights  in  polynomial-size  networks  of linear  threshold  elements  (LTEs).  We  show  that  there  is  an  efficient  way  of simulating  a  network  of LTEs  with  large  weights  by  a  network  of LTEs  with  small weights.  In  particular,  we  prove  that  every  depth-d,  polynomial-size network  of LTEs  with  exponentially  large  integer  weights  can  be  simulated  by  a  depth-(2d + 1),  polynomial-size network  of LTEs  with  polynomially  bounded  integer  weights.  To  prove  these  results,  we  use  tools from  harmonic analysis of Boolean  functions.  Our  technique  is  quite  general,  it  provides  insights  to some other  problems.  For  example,  we  are  able to improve the  best  known  results  on  the  depth  of a  network  of linear  threshold  elements  that  computes the  COM PARI SO N,  SUM  and  PRO DU CT of two  n-bits  numbers,  and  the  MAX 1M U M  and  the  SORTING of n  n-bit numbers.
A Segment-Based Automatic Language Identification System
Authors: Ronald A. Cole Yeshwant K. Muthusamy
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We have developed  a four-language automatic language identification sys(cid:173) tem  for  high-quality  speech.  The  system  uses  a  neural  network-based  segmentation algorithm to segment speech  into seven  broad phonetic cat(cid:173) egories.  Phonetic  and prosodic features  computed on  these  categories  are  then  input to a  second  network that performs the language classification.  The system was trained  and tested on separate sets  of speakers of Ameri(cid:173) can English, Japanese, Mandarin Chinese and Tamil.  It currently performs  with  an accuracy  of 89.5% on  the utterances  of the test set.
A Topographic Product for the Optimization of Self-Organizing Feature Maps
Authors: Klaus Pawelzik, Theo Geisel Hans-Ulrich Bauer
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Optimizing the performance of self-organizing feature maps like the Ko(cid:173) honen map involves the choice of the output space topology. We present  a topographic product which measures the preservation of neighborhood  relations as a criterion to optimize the output space topology of the map  with regard to the global dimensionality DA as well as to the dimensi(cid:173) ons in the individual directions. We test the topographic product method  not only on synthetic mapping examples, but also on speech data. In the  latter application our method suggests an output space dimensionality of  DA = 3, in coincidence with recent recognition results on the same data  set.
Learning How to Teach or Selecting Minimal Surface Data
Authors: Ricardo A. Marques Pereira Davi Geiger
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Learning a  map from an input set  to an output set is similar to the prob(cid:173) lem of reconstructing  hypersurfaces  from sparse  data (Poggio and  Girosi,  1990).  In this framework,  we  discuss  the  problem of automatically select(cid:173) ing  "minimal"  surface  data.  The objective  is  to be  able to approximately  reconstruct  the  surface  from the selected  sparse  data.  We  show  that this  problem is  equivalent  to  the  one  of compressing  information by  data re(cid:173) moval and the one oflearning how to teach.  Our key step is to introduce a  process  that statistically selects  the data according  to the  model.  During  the process  of data selection  (learning how  to teach)  our system  (teacher)  is  capable  of predicting  the  new  surface,  the  approximated  one  provided  by  the  selected  data.  We  concentrate  on  piecewise  smooth surfaces,  e.g.  images,  and  use  mean field  techniques  to  obtain a  deterministic  network  that is shown to compress  image data. 
1  Learning and  surface  reconstruction 
Given a dense input data that represents a hypersurface, how could we automatically  select  very few  data points such  as  to be able to use  these fewer  data points (sparse  data)  to approximately reconstruct  the hypersurface  ?  We  will be  using the term surface  to refer  to hypersurface  (surface  in multidimen-
Visual Grammars and Their Neural Nets
Authors: Eric Mjolsness
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: I  exhibit  a  systematic  way  to  derive  neural  nets  for  vision  problems.  It  involves  formulating  a  vision  problem  as  Bayesian  inference  or  decision  on  a  comprehensive  model  of the  visual  domain given  by  a  probabilistic  grammar.
Hierarchical Transformation of Space in the Visual System
Authors: Stephen A. Fisher, Terrence J. Sejnowski Alexandre Pouget
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Neurons  encoding  simple  visual  features  in  area  VI  such  as  orientation,  direction  of motion  and  color  are  organized  in  retinotopic  maps.  How(cid:173) ever,  recent  physiological  experiments have  shown  that  the  responses  of  many  neurons  in  VI  and  other  cortical  areas  are  modulated  by  the  di(cid:173) rection of gaze.  We  have developed a  neural  network model of the  visual  cortex to explore the hypothesis that visual  features are encoded in  head(cid:173) centered coordinates at early stages of visual processing.  New experiments  are suggested for  testing this hypothesis using  electrical stimulations and  psychophysical observations.
Illumination and View Position in 3D Visual Recognition
Authors: Amnon Shashua
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: It is  shown  that  both  changes  in  viewing  position  and  illumination  con(cid:173) ditions  can  be  compensated for,  prior  to recognition,  using  combinations  of images  taken  from  different  viewing  positions  and  different  illumina(cid:173) tion  conditions.  It is  also  shown  that,  in  agreement  with  psychophysical  findings,  the  computation  requires  at  least  a  sign-bit  image  as  input  - contours alone  are not sufficient.
Constant-Time Loading of Shallow 1-Dimensional Networks
Authors: Stephen Judd
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: The complexity of learning in shallow I-Dimensional neural networks has  been shown elsewhere to be linear in the size of the network. However,  when the network has a huge number of units (as cortex has) even linear  time might be unacceptable. Furthermore, the algorithm that was given to  achieve this time was based on a single serial processor and was biologically  implausible.  In this work we consider the more natural parallel model of processing  and demonstrate an expected-time complexity that is constant (i.e.  dependent of the size of the network). This holds even when inter-node  communication channels are short and local, thus adhering to more bio(cid:173) logical and VLSI constraints.
Combined Neural Network and Rule-Based Framework for Probabilistic Pattern Recognition and Discovery
Authors: Rodney Goodman, Rama Chellappa Hayit K. Greenspan
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A combined neural network and rule-based approach is suggested as a  general framework for pattern recognition. This approach enables unsu(cid:173) pervised and supervised learning, respectively, while providing probability  estimates for the output classes. The probability maps are utilized for  higher level analysis such as a feedback for smoothing over the output la(cid:173) bel maps and the identification of unknown patterns (pattern "discovery").  The suggested approach is presented and demonstrated in the texture - analysis task. A correct classification rate in the 90 percentile is achieved  for both unstructured and structured natural texture mosaics. The advan(cid:173) tages of the probabilistic approach to pattern analysis are demonstrated.
Experimental Evaluation of Learning in a Neural Microsystem
Authors: Anthony Jayakumar, Stephan Luna Joshua Alspector
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We report learning measurements from a system composed of a cascadable  learning chip, data generators and analyzers for training pattern presentation,  and an X-windows based software interface. The 32 neuron learning chip has  496 adaptive synapses and can perform Boltzmann and mean-field learning  using separate noise and gain controls. We have used this system to do learning  experiments on the parity and replication problem. The system settling time  limits the learning speed to about 100,000 patterns per second roughly  independent of system size.
Recurrent Eye Tracking Network Using a Distributed Representation of Image Motion
Authors: Stephen G. Lisberger, Terrence J. Sejnowski Paul A. Viola
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: This  paper  briefly  describes  an  artificial  neural  network  for  preattentive  visual processing.  The network is  capable of determiuing image motioll in  a type of stimulus which defeats most popular methods of motion detect.ion  - a  subset  of second-order  visual  motion stimuli known  as  drift-balanced  stimuli(DBS). The processing st.ages of the network described in this paper  are  integratable into  a  model  capable  of simultaneous motion extractioll.  edge  detection,  and  the determination of occlusion.
Against Edges: Function Approximation with Multiple Support Maps
Authors: Alex Pentland Trevor Darrell
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Networks for reconstructing a sparse or noisy function often use an edge  field to segment the function into homogeneous regions, This approach  assumes that these regions do not overlap or have disjoint parts, which is  often false. For example, images which contain regions split by an occlud(cid:173) ing object can't be properly reconstructed using this type of network. We  have developed a network that overcomes these limitations, using support  maps to represent the segmentation of a signal. In our approach, the sup(cid:173) port of each region in the signal is explicitly represented. Results from  an initial implementation demonstrate that this method can reconstruct  images and motion sequences which contain complicated occlusion.
Induction of Multiscale Temporal Structure
Authors: Michael Mozer
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Learning  structure  in  temporally-extended  sequences  is  a  difficult  com(cid:173) putational problem because  only a  fraction  of the relevant  information is  available  at  any  instant.  Although  variants  of back  propagation  can  in  principle  be  used  to find  structure in  sequences,  in  practice  they  are  not  sufficiently  powerful  to  discover  arbitrary  contingencies,  especially  those  spanning  long  temporal  intervals  or  involving  high  order  statistics.  For  example,  in  designing  a  connectionist  network for  music  composition,  we  have encountered  the problem that the net is  able to learn  musical struc(cid:173) ture that occurs locally in time-e.g., relations  among notes within a  mu(cid:173) sical phrase-but not structure that occurs over longer time periods--e.g.,  relations  among  phrases.  To  address  this  problem,  we  require  a  means  of constructing  a  reduced  deacription  of the  sequence  that  makes  global  aspects more explicit  or more readily detectable.  I propose to achieve this  using hidden  units that operate with different  time constants.  Simulation  experiments  indicate  that slower  time-scale  hidden  units  are  able  to pick  up global structure,  structure that simply can not be learned  by standard  back propagation. 
Many patterns in  the world  are intrinsically  temporal,  e.g.,  speech,  music,  the  un(cid:173) folding  of events.  Recurrent  neural net  architectures  have  been devised  to  accom(cid:173) modate  time-varying  sequences.  For  example,  the  architecture  shown  in  Figure  1  can  map  a  sequence  of inputs  to  a  sequence  of outputs.  Learning  structure  in  temporally-extended  sequences is  a difficult  computational problem because  the in(cid:173) put pattern may not contain all the task-relevant information at any instant.  Thus,  275
Locomotion in a Lower Vertebrate: Studies of the Cellular Basis of Rhythmogenesis and Oscillator Coupling
Authors: James T. Buchanan
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: To test  whether  the  known  connectivies  of neurons  in  the lamprey spinal  cord are sufficient to account for locomotor rhythmogenesis, a  CCconnection(cid:173) ist"  neural network simulation was done using identical cells connected ac(cid:173) cording to experimentally established  patterns.  It was  demonstrated  that  the  network  oscillates  in  a  stable  manner  with  the  same  phase  relation(cid:173) ships among the neurons as observed  in the lamprey.  The model was  then  used  to  explore  coupling  between  identical
Single Neuron Model: Response to Weak Modulation in the Presence of Noise
Authors: W. Jacobs A. R. Bulsara
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We consider a noisy bist.able single neuron model driven by a periodic  external modulation. The modulation introduces a correlated switching  between st.ates driven by the noise. The information flow through the sys(cid:173) tem from the modulation to the output switching events, leads to a succes(cid:173) sion of strong peaks in the power spectrum. The signal-to-noise ratio (SNR)  obtained from this power spectrum is a measure of the information content  in the neuron response . With increasing noise intensity, the SNR passes  t.hrough a maximum, an effect which has been called stochastic resonance.  We treat t.he problem wit.hin the framework of a recently developed approx(cid:173) imate theory, valid in the limits of weak noise intensity, weak periodic forc(cid:173) ing and low forcing frequency. A comparison of the results of this theory  with those obtained from a linear syst.em FFT is also presented .
Fast Learning with Predictive Forward Models
Authors: Carlos Brody
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A method for transforming performance evaluation signals distal both in  space and time into proximal signals usable by supervised learning algo(cid:173) rithms, presented in [Jordan & Jacobs 90], is examined. A simple obser(cid:173) vation concerning differentiation through models trained with redundant  inputs (as one of their networks is) explains a weakness in the original  architecture and suggests a modification: an internal world model that  encodes action-space exploration and, crucially, cancels input redundancy  to the forward model is added. Learning time on an example task, cart(cid:173) pole balancing, is thereby reduced about 50 to 100 times.
Nonlinear Pattern Separation in Single Hippocampal Neurons with Active Dendritic Membrane
Authors: Brenda J. Claiborne, Thomas H. Brown Anthony M. Zador
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: The dendritic trees of cortical pyramidal neurons seem ideally suited to  perfonn local processing on inputs.  To explore some of the implications  of this complexity for the computational power of neurons, we simulated  a realistic biophysical model of a hippocampal pyramidal cell in which a  "cold spot"-a high density patch of inhibitory Ca-dependent K channels  and  a  colocalized  patch  of Ca  channels-was  present  at  a  dendritic  branch  point.  The cold spot induced a non monotonic relationship  be(cid:173) tween the strength of the synaptic input and the probability of neuronal  fIring.  This effect could also be interpreted as an analog stochastic XOR.
A Neural Network for Motion Detection of Drift-Balanced Stimuli
Authors: Hilary Tunley
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: This  paper  briefly  describes  an  artificial  neural  network  for  preattentive  visual processing.  The network is  capable of determiuing image motioll in  a type of stimulus which defeats most popular methods of motion detect.ion  - a  subset  of second-order  visual  motion stimuli known  as  drift-balanced  stimuli(DBS). The processing st.ages of the network described in this paper  are  integratable into  a  model  capable  of simultaneous motion extractioll.  edge  detection,  and  the determination of occlusion.
Generalization Performance in PARSEC - A Structured Connectionist Parsing Architecture
Authors: Ajay N. Jain
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: This paper presents PARSEC-a system for generating connectionist  parsing networks from example parses. PARSEC is not based on formal  grammar systems and is geared toward spoken language tasks. PARSEC  networks exhibit three strengths important for application to speech pro(cid:173) cessing:  1) they learn to parse, and generalize well compared to hand(cid:173) coded grammars; 2) they tolerate several types of noise;  3)  they can  learn to use multi-modal input. Presented are the PARSEC architecture  and performance analyses along several dimensions that demonstrate  PARSEC's features. PARSEC's performance is compared to that of tra(cid:173) ditional grammar-based parsing systems.
Hierarchies of adaptive experts
Authors: Robert A. Jacobs Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: In this paper we present a neural network architecture that discovers a  recursive decomposition of its input space. Based on a generalization of the  modular architecture of Jacobs, Jordan, Nowlan, and Hinton (1991), the  architecture uses competition among networks to recursively split the input  space into nested regions and to learn separate associative mappings within  each region. The learning algorithm is shown to perform gradient ascent  in a log likelihood function that captures the architecture's hierarchical  structure.
Optical Implementation of a Self-Organizing Feature Extractor
Authors: Claus Benkert, Verena Hebler, Ju-Seog Jang, Don Montgomery, Mark Saffman Dana Z. Anderson
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We demonstrate a self-organizing system based on photorefrac(cid:173) tive ring oscillators.  We  employ the system in two ways that  can  both be thought of as feature extractors; one acts on  a  set  of images exposed repeatedly to the system strictly as a linear  feature extractor, and the other serves as a signal demultiplex(cid:173) er  for  fiber  optic  communications.  Both  systems  implement  unsupervised competitive learning embedded within the mode  interaction dynamics between the modes of a set of ring oscilla(cid:173) tors.  After a training period, the modes of the rings become as(cid:173) sociated  with  frequencies within the incoming data stream. 
image  features  or  carrier
A Neurocomputer Board Based on the ANNA Neural Network Chip
Authors: Bernhard E. Boser, Lawrence D. Jackel Eduard Säckinger
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A  board is described  that contains the ANN A  neural-network  chip,  and a  DSP32C  digital  signal  processor.  The  ANNA  (Analog  Neural  Network  Arithmetic  unit)  chip  performs  mixed  analog/digital  processing.  The  combination  of ANNA  with  the  DSP  allows  high-speed,  end-to-end  ex(cid:173) ecution  of numerous signal-processing  applications,  including the  prepro(cid:173) cessing,  the  neural-net  calculations,  and  the  postprocessing  steps.  The  ANNA  board  evaluates  neural  networks  10  to  100  times  faster  than  the  DSP  alone.  The  board  is  suitable  for  implementing  large  (million  con(cid:173) nections)  networks  with sparse  weight  matrices.  Three  applications  have  been  implemented on  the  board:  a  convolver  network  for  slant  detection  of text  blocks,  a  handwritten  digit  recognizer,  and  a  neural  network  for  recognition-based  segmentation.
Adaptive Synchronization of Neural and Physical Oscillators
Authors: Shuji Yoshizawa Kenji Doya
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Animal  locomotion  patterns  are  controlled  by recurrent  neural  networks  called central pattern generators  (CPGs).  Although  a  CPG  can  oscillate  autonomously,  its  rhythm and  phase  must  be  well  coordinated  with  the  state of the physical system using sensory inputs.  In this paper we propose  a learning algorithm for synchronizing neural and physical oscillators with  specific phase relationships.  Sensory input connections are modified by the  correlation between cellular activities and input signals.  Simulations show  that the learning rule can be used for setting sensory feedback connections  to a  CPG as  well  as coupling connections between CPGs. 
1  CENTRAL AND SENSORY MECHANISMS IN 
LOCOMOTION  CONTROL 
Patterns of animal locomotion, such as walking, swimming, and fiying,  are generated  by recurrent neural networks that are located in segmental ganglia of invertebrates  and spinal  cords  of vertebrates  (Barnes  and  Gladden,  1985).  These networks  can  produce basic rhythms of locomotion without sensory inputs and are  called central  pattern generators  (CPGs).  The physical systems of locomotion, such as  legs, fins,  and  wings  combined  with  physical environments,  have  their  own  oscillatory  char(cid:173) acteristics.  Therefore,  in  order  to  realize  efficient  locomotion,  the  frequency  and  the  phase  of oscillation  of a  CPG must  be  well  coordinated  with  the state  of the  physical system.  For example,  the  bursting patterns  of motoneurons  that  drive  a  leg  muscle must be coordinated  with  the configuration  of the leg,  its contact with  the ground,  and the state of other legs.
Recurrent Networks and NARMA Modeling
Authors: Les E. Atlas, Douglas R. Martin Jerome Connor
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: There exist large classes of time series, such as those with nonlinear moving  average components, that are not well modeled by feedforward networks  or linear models, but can be modeled by recurrent networks. We show that  recurrent neural networks are a type of nonlinear autoregressive-moving  average (N ARMA) model. Practical ability will be shown in the results of  a competition sponsored by the Puget Sound Power and Light Company,  where the recurrent networks gave the best performance on electric load  forecasting.
ANN Based Classification for Heart Defibrillators
Authors: S. Pickard, P. Leong, Z. Chi, B. Flower, Y. Xie M. Jabri
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Current Intra-Cardia defibrillators make use of simple classification algo(cid:173) rithms to determine patient conditions and subsequently to enable proper  therapy. The simplicity is primarily due to the constraints on power dissipa(cid:173) tion and area available for implementation. Sub-threshold implementation  of artificial neural networks offer potential classifiers with higher perfor(cid:173) mance than commercially available defibrillators. In this paper we explore  several classifier architectures and discuss micro-electronic implementation  issues.
Adaptive Development of Connectionist Decoders for Complex Error-Correcting Codes
Authors: Mario Blaum Sheri L. Gish
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: \Ve  present.  an  approach  for  df'velopment  of a  decoder  for  any  complex  binary error-correct.ing  code- (ECC) via training from examples of decoded  received  words.  Our  decoder  is  a  connectionist  architecture.  We  describe  two  sepa.rate  solutions:  A  system-level  solution  (the  Cascaded  Networks  Decoder);  and  the  ECC-Enhanced  Decoder,  a  solution  which  simplifies  the  mapping problem  which  must  be  solved  for  decoding.  Although  both  solutions  meet  our  basic  approach  constraint  for  simplicity and  compact(cid:173) ness.  only  the  ECC-Enhanced  Decoder  meet.s  our second  basic  constraint  of being  a  generic  solution.
Self-organization in real neurons: Anti-Hebb in 'Channel Space'?
Authors: Anthony J. Bell
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Ion channels are the dynamical systems of the nervous system. Their  distribution within the membrane governs not only communication of in(cid:173) formation between neurons, but also how that information is integrated  within the cell. Here, an argument is presented for an 'anti-Hebbian' rule  for changing the distribution of voltage-dependent ion channels in order  to flatten voltage curvatures in dendrites. Simulations show that this rule  can account for the self-organisation of dynamical receptive field properties  such as resonance and direction selectivity. It also creates the conditions  for the faithful conduction within the cell of signals to which the cell has  been exposed. Various possible cellular implementations of such a learn(cid:173) ing rule are proposed, including activity-dependent migration of channel  proteins in the plane of the membrane.
Tangent Prop - A formalism for specifying selected invariances in an adaptive network
Authors: Bernard Victorri, Yann LeCun, John Denker Patrice Simard
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: In many machine learning applications, one has access, not only to training  data, but also to some high-level a priori knowledge about the desired be(cid:173) havior of the system. For example, it is known in advance that the output  of a character recognizer should be invariant with respect to small spa(cid:173) tial distortions of the input images (translations, rotations, scale changes,  etcetera).  We have implemented a scheme that allows a network to learn the deriva(cid:173) tive of its outputs with respect to distortion operators of our choosing.  This not only reduces the learning time and the amount of training data,  but also provides a powerful language for specifying what generalizations  we wish the network to perform.
Human and Machine 'Quick Modeling'
Authors: Karl Gustafson Jakob Bernasconi
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present here an interesting experiment in 'quick modeling' by humans,  performed independently on small samples, in several languages and two  continents, over the last three years. Comparisons to decision tree proce(cid:173) dures and neural net processing are given. From these, we conjecture that  human reasoning is better represented by the latter, but substantially dif(cid:173) ferent from both. Implications for the 'strong convergence hypothesis' be(cid:173) tween neural networks and machine learning are discussed, now expanded  to include human reasoning comparisons.
Practical Issues in Temporal Difference Learning
Authors: Gerald Tesauro
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: This  paper  examines  whether  temporal  difference  methods  for  training  connectionist  networks,  such  as  Suttons's  TO(')  algorithm,  can  be  suc(cid:173) cessfully  applied to complex real-world problems.  A number of important  practical issues are identified and discussed from a general theoretical per(cid:173) spective.  These practical issues are then examined in the context of a case  study  in  which  TO(')  is  applied  to  learning  the  game  of backgammon  from  the  outcome of self-play.  This is  apparently  the  first  application of  this  algorithm  to  a  complex  nontrivial  task.  It is  found  that,  with  zero  knowledge  built  in,  the network  is able  to learn  from  scratch  to  play  the  entire game at a  fairly strong intermediate level of performance,  which  is  clearly  better  than conventional commercial programs, and which  in fact  surpasses  comparable networks  trained  on  a  massive  human expert  data  set.  The hidden  units in  these network  have apparently discovered  useful  features,  a  longstanding  goal of computer  games  research.  Furthermore,  when  a  set  of hand-crafted  features  is  added  to the  input representation,  the resulting networks reach  a  near-expert  level of performance, and have  achieved good results against world-class human play.
A Network of Localized Linear Discriminants
Authors: Martin S. Glassman
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: The localized linear discriminant network (LLDN) has been designed to address  classification problems containing relatively closely  spaced data from  different  classes  (encounter zones  [1], the accuracy problem  [2]).  Locally trained hyper(cid:173) plane segments are an effective way to define the decision boundaries for these  regions [3].  The LLD uses a modified perceptron training algorithm for effective  discovery of separating hyperplane/sigmoid units within narrow boundaries. The  basic unit of the network is the discriminant receptive field (DRF) which combines  the LLD function with Gaussians representing the dispersion of the local training  data with respect to the hyperplane.  The DRF implements a local distance mea(cid:173) sure [4], and obtains the benefits of networks oflocalized units [5].  A constructive  algorithm for the two-class case is described which incorporates DRF's into the  hidden layer to solve local discrimination problems.  The output unit produces a  smoothed, piecewise linear decision boundary.  Preliminary results  indicate the  ability of the LLDN to efficiently achieve separation when boundaries are narrow  and complex,  in  cases where  both the "standard" multilayer perceptron (MLP)  and k-nearest neighbor (KNN) yield high error rates on training data. 
1  The LLD Training Algorithm and DRF Generation 
The LLD is defined by the hyperplane normal vector V and its "midpoint" M (a translated  origin  [1]  near  the  center  of gravity  of the  training  data  in  feature  space).  Incremental  corrections  to  V  and  M  accrue  for  each  training  token  feature  vector  Y j  in  the  training  set,  as  iIlustrated in figure  1 (exaggerated magnitudes).  The surface of the hyperplane is  appropriately moved either towards or away  from  Yj  by  rotating V,  and shifting M  along
Multi-Digit Recognition Using a Space Displacement Neural Network
Authors: Christopher J. C. Burges, Yann LeCun, John S. Denker Ofer Matan
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present a feed-forward network architecture for  recognizing an uncon(cid:173) strained  handwritten  multi-digit  string.  This  is  an extension  of previous  work on recognizing isolated digits.  In this architecture a  single  digit  rec(cid:173) ognizer  is  replicated  over  the  input.  The  output  layer  of the  network  is  coupled to a Viterbi alignment module that chooses the best interpretation  of the input.  Training errors are propagated through the Viterbi module.  The novelty in  this procedure is  that segmentation is  done on the feature  maps developed in the Space Displacement Neural Network (SDNN) rather  than the input  (pixel) space.
JANUS: Speech-to-Speech Translation Using Connectionist and Non-Connectionist Techniques
Authors: Ajay N. Jain, Arthur E. McNair, Joe Tebelskis, Louise Osterholtz, Hiroaki Saito, Otto Schmidbauer, Tilo Sloboda, Monika Woszczyna Alex Waibel
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present JANUS, a speech-to-speech translation system that utilizes  diverse processing strategies, including connectionist learning, tradi(cid:173) tional AI knowledge representation approaches, dynamic programming,  and stochastic techniques. JANUS  translates continuously spoken  English and German into German, English, and Japanese. JANUS cur(cid:173) rently achieves 87%  translation fidelity from English speech and 97%  from German speech. We present the JANUS system along with com(cid:173) parative evaluations of its interchangeable processing components, with  special emphasis on the connectionist modules. 
• Also with University of Karlsruhe, Karlsruhe. Germany.  1N"ow with Alliant Techsystems Research and Technology Center. Hopkins. Minnesota.
A Parallel Analog CCD/CMOS Signal Processor
Authors: Amnon Yariv Charles F. Neugebauer
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A CCO based signal processing IC that computes a fully parallel single  quadrant vector-matrix multiplication has been designed and fabricated with a  2j..un CCO/CMOS process.  The device incorporates  an array of Charge  Coupled Devices (CCO) which hold an analog matrix of charge encoding the  matrix elements.  Input vectors are digital with 1 - 8 bit accuracy.
A Computational Mechanism to Account for Averaged Modified Hand Trajectories
Authors: Tamar Flash Ealan A. Henis
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Using the double-step target displacement paradigm the mechanisms un(cid:173) derlying arm trajectory modification were investigated. Using short (10- 110 msec) inter-stimulus intervals the resulting hand motions were initially  directed in between the first and second target locations. The kinematic  features of the modified motions were accounted for by the superposition  scheme, which involves the vectorial addition of two independent point-to(cid:173) point motion units: one for moving the hand toward an internally specified  location and a second one for moving between that location and the final  target location . The similarity between the inferred internally specified lo(cid:173) cations and previously reported measured end-points of the first saccades  in double-step eye-movement studies may suggest similarities between per(cid:173) ceived target locations in eye and hand motor control.
Learning Global Direct Inverse Kinematics
Authors: Kenneth Kreutz-Delgado David DeMers
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We  introduce  and  demonstrate  a  bootstrap  method  for  construction  of an  in(cid:173) verse function for the robot kinematic mapping using only sample configuration(cid:173) space/workspace data.  Unsupervised learning (clustering) techniques are used on  pre-image neighborhoods in order to  learn  to partition the configuration space  into subsets over which the kinematic mapping is  invertible.  Supervised leam(cid:173) ing is  then  used separately on each  of the partitions to approximate the inverse  function.  The ill-posed inverse kinematics function is thereby regularized, and  a globa1 inverse kinematics solution for the wristless Puma manipulator is devel(cid:173) oped.
VISIT: A Neural Model of Covert Visual Attention
Authors: Subutai Ahmad
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Visual attention is the ability to dynamically restrict processing to a subset  of the visual field.  Researchers  have long argued that such a  mechanism is  necessary  to efficiently perform many intermediate level visual tasks.  This  paper describes  VISIT,  a  novel  neural  network  model  of visual attention.  The current system models the search for target objects in scenes  contain(cid:173) ing  multiple distractors.  This  is  a  natural  task  for  people,  it  is  studied  extensively by psychologists,  and it requires  attention.  The network's be(cid:173) havior  closely  matches  the  known  psychophysical  data  on  visual  search  and visual  attention.  VISIT also  matches much of the  physiological data  on attention and provides a  novel view  of the functionality of a  number of  visual areas.  This paper concentrates  on  the  biological plausibility of the  model and its relationship to the primary visual cortex, pulvinar, superior  colliculus and posterior  parietal areas.
Kernel Regression and Backpropagation Training With Noise
Authors: Lasse Holmström Petri Koistinen
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: One method proposed for improving the generalization capability of a feed(cid:173) forward  network  trained  with  the  backpropagation  algorithm  is  to  use  artificial  training vectors  which  are  obtained by  adding noise  to the orig(cid:173) inal training vectors.  We  discuss  the  connection  of such  backpropagation  training with noise  to kernel  density  and kernel regression  estimation.  We  compare by simulated examples (1)  backpropagation, (2)  backpropagation  with  noise,  and  (3)  kernel  regression  in  mapping estimation  and  pattern  classification  contexts.
Recognition of Manipulated Objects by Motor Learning
Authors: Mitsuo Kawato Hiroaki Gomi
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present two neural network controller learning schemes based on feedback(cid:173) error-learning and  modular architecture for recognition and control  of multiple  manipulated objects. In the first  scheme, a Gating Network is  trained to acquire  object-specific representations for recognition of a number of objects (or sets of  objects).  In  the  second  scheme,  an  Estimation  Network is  trained  to  acquire  function-specific, rather than object-specific, representations which directly estimate  physical parameters. Both  recognition networks are trained to identify manipulated  objects  using  somatic  and/or  visual  information.  After  learning,  appropriate  motor  commands  for  manipulation  of each  object  are  issued by  the  control  networks.