mtybilly/Instruct2DS huggingface dataset:
Hierarchical Transformation of Space in the Visual System
Authors: Alexandre Pouget, Stephen A. Fisher, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Neurons  encoding  simple  visual  features  in  area  VI  such  as  orientation,  direction  of motion  and  color  are  organized  in  retinotopic  maps.  How(cid:173) ever,  recent  physiological  experiments have  shown  that  the  responses  of  many  neurons  in  VI  and  other  cortical  areas  are  modulated  by  the  di(cid:173) rection of gaze.  We  have developed a  neural  network model of the  visual  cortex to explore the hypothesis that visual  features are encoded in  head(cid:173) centered coordinates at early stages of visual processing.  New experiments  are suggested for  testing this hypothesis using  electrical stimulations and  psychophysical observations.
Combined Neural Network and Rule-Based Framework for Probabilistic Pattern Recognition and Discovery
Authors: Hayit K. Greenspan, Rodney Goodman, Rama Chellappa
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A combined neural network and rule-based approach is suggested as a  general framework for pattern recognition. This approach enables unsu(cid:173) pervised and supervised learning, respectively, while providing probability  estimates for the output classes. The probability maps are utilized for  higher level analysis such as a feedback for smoothing over the output la(cid:173) bel maps and the identification of unknown patterns (pattern "discovery").  The suggested approach is presented and demonstrated in the texture - analysis task. A correct classification rate in the 90 percentile is achieved  for both unstructured and structured natural texture mosaics. The advan(cid:173) tages of the probabilistic approach to pattern analysis are demonstrated.
Experimental Evaluation of Learning in a Neural Microsystem
Authors: Joshua Alspector, Anthony Jayakumar, Stephan Luna
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We report learning measurements from a system composed of a cascadable  learning chip, data generators and analyzers for training pattern presentation,  and an X-windows based software interface. The 32 neuron learning chip has  496 adaptive synapses and can perform Boltzmann and mean-field learning  using separate noise and gain controls. We have used this system to do learning  experiments on the parity and replication problem. The system settling time  limits the learning speed to about 100,000 patterns per second roughly  independent of system size.
Recurrent Eye Tracking Network Using a Distributed Representation of Image Motion
Authors: Paul A. Viola, Stephen G. Lisberger, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: This  paper  briefly  describes  an  artificial  neural  network  for  preattentive  visual processing.  The network is  capable of determiuing image motioll in  a type of stimulus which defeats most popular methods of motion detect.ion  - a  subset  of second-order  visual  motion stimuli known  as  drift-balanced  stimuli(DBS). The processing st.ages of the network described in this paper  are  integratable into  a  model  capable  of simultaneous motion extractioll.  edge  detection,  and  the determination of occlusion.
Against Edges: Function Approximation with Multiple Support Maps
Authors: Trevor Darrell, Alex Pentland
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Networks for reconstructing a sparse or noisy function often use an edge  field to segment the function into homogeneous regions, This approach  assumes that these regions do not overlap or have disjoint parts, which is  often false. For example, images which contain regions split by an occlud(cid:173) ing object can't be properly reconstructed using this type of network. We  have developed a network that overcomes these limitations, using support  maps to represent the segmentation of a signal. In our approach, the sup(cid:173) port of each region in the signal is explicitly represented. Results from  an initial implementation demonstrate that this method can reconstruct  images and motion sequences which contain complicated occlusion.
Single Neuron Model: Response to Weak Modulation in the Presence of Noise
Authors: A. R. Bulsara, W. Jacobs
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We consider a noisy bist.able single neuron model driven by a periodic  external modulation. The modulation introduces a correlated switching  between st.ates driven by the noise. The information flow through the sys(cid:173) tem from the modulation to the output switching events, leads to a succes(cid:173) sion of strong peaks in the power spectrum. The signal-to-noise ratio (SNR)  obtained from this power spectrum is a measure of the information content  in the neuron response . With increasing noise intensity, the SNR passes  t.hrough a maximum, an effect which has been called stochastic resonance.  We treat t.he problem wit.hin the framework of a recently developed approx(cid:173) imate theory, valid in the limits of weak noise intensity, weak periodic forc(cid:173) ing and low forcing frequency. A comparison of the results of this theory  with those obtained from a linear syst.em FFT is also presented .
Nonlinear Pattern Separation in Single Hippocampal Neurons with Active Dendritic Membrane
Authors: Anthony M. Zador, Brenda J. Claiborne, Thomas H. Brown
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: The dendritic trees of cortical pyramidal neurons seem ideally suited to  perfonn local processing on inputs.  To explore some of the implications  of this complexity for the computational power of neurons, we simulated  a realistic biophysical model of a hippocampal pyramidal cell in which a  "cold spot"-a high density patch of inhibitory Ca-dependent K channels  and  a  colocalized  patch  of Ca  channels-was  present  at  a  dendritic  branch  point.  The cold spot induced a non monotonic relationship  be(cid:173) tween the strength of the synaptic input and the probability of neuronal  fIring.  This effect could also be interpreted as an analog stochastic XOR.
Hierarchies of adaptive experts
Authors: Michael I. Jordan, Robert A. Jacobs
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: In this paper we present a neural network architecture that discovers a  recursive decomposition of its input space. Based on a generalization of the  modular architecture of Jacobs, Jordan, Nowlan, and Hinton (1991), the  architecture uses competition among networks to recursively split the input  space into nested regions and to learn separate associative mappings within  each region. The learning algorithm is shown to perform gradient ascent  in a log likelihood function that captures the architecture's hierarchical  structure.
Optical Implementation of a Self-Organizing Feature Extractor
Authors: Dana Z. Anderson, Claus Benkert, Verena Hebler, Ju-Seog Jang, Don Montgomery, Mark Saffman
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We demonstrate a self-organizing system based on photorefrac(cid:173) tive ring oscillators.  We  employ the system in two ways that  can  both be thought of as feature extractors; one acts on  a  set  of images exposed repeatedly to the system strictly as a linear  feature extractor, and the other serves as a signal demultiplex(cid:173) er  for  fiber  optic  communications.  Both  systems  implement  unsupervised competitive learning embedded within the mode  interaction dynamics between the modes of a set of ring oscilla(cid:173) tors.  After a training period, the modes of the rings become as(cid:173) sociated  with  frequencies within the incoming data stream. 
image  features  or  carrier
A Neurocomputer Board Based on the ANNA Neural Network Chip
Authors: Eduard Säckinger, Bernhard E. Boser, Lawrence D. Jackel
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A  board is described  that contains the ANN A  neural-network  chip,  and a  DSP32C  digital  signal  processor.  The  ANNA  (Analog  Neural  Network  Arithmetic  unit)  chip  performs  mixed  analog/digital  processing.  The  combination  of ANNA  with  the  DSP  allows  high-speed,  end-to-end  ex(cid:173) ecution  of numerous signal-processing  applications,  including the  prepro(cid:173) cessing,  the  neural-net  calculations,  and  the  postprocessing  steps.  The  ANNA  board  evaluates  neural  networks  10  to  100  times  faster  than  the  DSP  alone.  The  board  is  suitable  for  implementing  large  (million  con(cid:173) nections)  networks  with sparse  weight  matrices.  Three  applications  have  been  implemented on  the  board:  a  convolver  network  for  slant  detection  of text  blocks,  a  handwritten  digit  recognizer,  and  a  neural  network  for  recognition-based  segmentation.
Adaptive Synchronization of Neural and Physical Oscillators
Authors: Kenji Doya, Shuji Yoshizawa
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Animal  locomotion  patterns  are  controlled  by recurrent  neural  networks  called central pattern generators  (CPGs).  Although  a  CPG  can  oscillate  autonomously,  its  rhythm and  phase  must  be  well  coordinated  with  the  state of the physical system using sensory inputs.  In this paper we propose  a learning algorithm for synchronizing neural and physical oscillators with  specific phase relationships.  Sensory input connections are modified by the  correlation between cellular activities and input signals.  Simulations show  that the learning rule can be used for setting sensory feedback connections  to a  CPG as  well  as coupling connections between CPGs. 
1  CENTRAL AND SENSORY MECHANISMS IN 
LOCOMOTION  CONTROL 
Patterns of animal locomotion, such as walking, swimming, and fiying,  are generated  by recurrent neural networks that are located in segmental ganglia of invertebrates  and spinal  cords  of vertebrates  (Barnes  and  Gladden,  1985).  These networks  can  produce basic rhythms of locomotion without sensory inputs and are  called central  pattern generators  (CPGs).  The physical systems of locomotion, such as  legs, fins,  and  wings  combined  with  physical environments,  have  their  own  oscillatory  char(cid:173) acteristics.  Therefore,  in  order  to  realize  efficient  locomotion,  the  frequency  and  the  phase  of oscillation  of a  CPG must  be  well  coordinated  with  the state  of the  physical system.  For example,  the  bursting patterns  of motoneurons  that  drive  a  leg  muscle must be coordinated  with  the configuration  of the leg,  its contact with  the ground,  and the state of other legs.
Recurrent Networks and NARMA Modeling
Authors: Jerome Connor, Les E. Atlas, Douglas R. Martin
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: There exist large classes of time series, such as those with nonlinear moving  average components, that are not well modeled by feedforward networks  or linear models, but can be modeled by recurrent networks. We show that  recurrent neural networks are a type of nonlinear autoregressive-moving  average (N ARMA) model. Practical ability will be shown in the results of  a competition sponsored by the Puget Sound Power and Light Company,  where the recurrent networks gave the best performance on electric load  forecasting.
ANN Based Classification for Heart Defibrillators
Authors: M. Jabri, S. Pickard, P. Leong, Z. Chi, B. Flower, Y. Xie
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Current Intra-Cardia defibrillators make use of simple classification algo(cid:173) rithms to determine patient conditions and subsequently to enable proper  therapy. The simplicity is primarily due to the constraints on power dissipa(cid:173) tion and area available for implementation. Sub-threshold implementation  of artificial neural networks offer potential classifiers with higher perfor(cid:173) mance than commercially available defibrillators. In this paper we explore  several classifier architectures and discuss micro-electronic implementation  issues.
Adaptive Development of Connectionist Decoders for Complex Error-Correcting Codes
Authors: Sheri L. Gish, Mario Blaum
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: \Ve  present.  an  approach  for  df'velopment  of a  decoder  for  any  complex  binary error-correct.ing  code- (ECC) via training from examples of decoded  received  words.  Our  decoder  is  a  connectionist  architecture.  We  describe  two  sepa.rate  solutions:  A  system-level  solution  (the  Cascaded  Networks  Decoder);  and  the  ECC-Enhanced  Decoder,  a  solution  which  simplifies  the  mapping problem  which  must  be  solved  for  decoding.  Although  both  solutions  meet  our  basic  approach  constraint  for  simplicity and  compact(cid:173) ness.  only  the  ECC-Enhanced  Decoder  meet.s  our second  basic  constraint  of being  a  generic  solution.
Tangent Prop - A formalism for specifying selected invariances in an adaptive network
Authors: Patrice Simard, Bernard Victorri, Yann LeCun, John Denker
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: In many machine learning applications, one has access, not only to training  data, but also to some high-level a priori knowledge about the desired be(cid:173) havior of the system. For example, it is known in advance that the output  of a character recognizer should be invariant with respect to small spa(cid:173) tial distortions of the input images (translations, rotations, scale changes,  etcetera).  We have implemented a scheme that allows a network to learn the deriva(cid:173) tive of its outputs with respect to distortion operators of our choosing.  This not only reduces the learning time and the amount of training data,  but also provides a powerful language for specifying what generalizations  we wish the network to perform.
Human and Machine 'Quick Modeling'
Authors: Jakob Bernasconi, Karl Gustafson
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present here an interesting experiment in 'quick modeling' by humans,  performed independently on small samples, in several languages and two  continents, over the last three years. Comparisons to decision tree proce(cid:173) dures and neural net processing are given. From these, we conjecture that  human reasoning is better represented by the latter, but substantially dif(cid:173) ferent from both. Implications for the 'strong convergence hypothesis' be(cid:173) tween neural networks and machine learning are discussed, now expanded  to include human reasoning comparisons.
Multi-Digit Recognition Using a Space Displacement Neural Network
Authors: Ofer Matan, Christopher J. C. Burges, Yann LeCun, John S. Denker
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present a feed-forward network architecture for  recognizing an uncon(cid:173) strained  handwritten  multi-digit  string.  This  is  an extension  of previous  work on recognizing isolated digits.  In this architecture a  single  digit  rec(cid:173) ognizer  is  replicated  over  the  input.  The  output  layer  of the  network  is  coupled to a Viterbi alignment module that chooses the best interpretation  of the input.  Training errors are propagated through the Viterbi module.  The novelty in  this procedure is  that segmentation is  done on the feature  maps developed in the Space Displacement Neural Network (SDNN) rather  than the input  (pixel) space.
JANUS: Speech-to-Speech Translation Using Connectionist and Non-Connectionist Techniques
Authors: Alex Waibel, Ajay N. Jain, Arthur E. McNair, Joe Tebelskis, Louise Osterholtz, Hiroaki Saito, Otto Schmidbauer, Tilo Sloboda, Monika Woszczyna
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present JANUS, a speech-to-speech translation system that utilizes  diverse processing strategies, including connectionist learning, tradi(cid:173) tional AI knowledge representation approaches, dynamic programming,  and stochastic techniques. JANUS  translates continuously spoken  English and German into German, English, and Japanese. JANUS cur(cid:173) rently achieves 87%  translation fidelity from English speech and 97%  from German speech. We present the JANUS system along with com(cid:173) parative evaluations of its interchangeable processing components, with  special emphasis on the connectionist modules. 
• Also with University of Karlsruhe, Karlsruhe. Germany.  1N"ow with Alliant Techsystems Research and Technology Center. Hopkins. Minnesota.
A Parallel Analog CCD/CMOS Signal Processor
Authors: Charles F. Neugebauer, Amnon Yariv
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A CCO based signal processing IC that computes a fully parallel single  quadrant vector-matrix multiplication has been designed and fabricated with a  2j..un CCO/CMOS process.  The device incorporates  an array of Charge  Coupled Devices (CCO) which hold an analog matrix of charge encoding the  matrix elements.  Input vectors are digital with 1 - 8 bit accuracy.
A Computational Mechanism to Account for Averaged Modified Hand Trajectories
Authors: Ealan A. Henis, Tamar Flash
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Using the double-step target displacement paradigm the mechanisms un(cid:173) derlying arm trajectory modification were investigated. Using short (10- 110 msec) inter-stimulus intervals the resulting hand motions were initially  directed in between the first and second target locations. The kinematic  features of the modified motions were accounted for by the superposition  scheme, which involves the vectorial addition of two independent point-to(cid:173) point motion units: one for moving the hand toward an internally specified  location and a second one for moving between that location and the final  target location . The similarity between the inferred internally specified lo(cid:173) cations and previously reported measured end-points of the first saccades  in double-step eye-movement studies may suggest similarities between per(cid:173) ceived target locations in eye and hand motor control.
Learning Global Direct Inverse Kinematics
Authors: David DeMers, Kenneth Kreutz-Delgado
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We  introduce  and  demonstrate  a  bootstrap  method  for  construction  of an  in(cid:173) verse function for the robot kinematic mapping using only sample configuration(cid:173) space/workspace data.  Unsupervised learning (clustering) techniques are used on  pre-image neighborhoods in order to  learn  to partition the configuration space  into subsets over which the kinematic mapping is  invertible.  Supervised leam(cid:173) ing is  then  used separately on each  of the partitions to approximate the inverse  function.  The ill-posed inverse kinematics function is thereby regularized, and  a globa1 inverse kinematics solution for the wristless Puma manipulator is devel(cid:173) oped.
Kernel Regression and Backpropagation Training With Noise
Authors: Petri Koistinen, Lasse Holmström
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: One method proposed for improving the generalization capability of a feed(cid:173) forward  network  trained  with  the  backpropagation  algorithm  is  to  use  artificial  training vectors  which  are  obtained by  adding noise  to the orig(cid:173) inal training vectors.  We  discuss  the  connection  of such  backpropagation  training with noise  to kernel  density  and kernel regression  estimation.  We  compare by simulated examples (1)  backpropagation, (2)  backpropagation  with  noise,  and  (3)  kernel  regression  in  mapping estimation  and  pattern  classification  contexts.
Recognition of Manipulated Objects by Motor Learning
Authors: Hiroaki Gomi, Mitsuo Kawato
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present two neural network controller learning schemes based on feedback(cid:173) error-learning and  modular architecture for recognition and control  of multiple  manipulated objects. In the first  scheme, a Gating Network is  trained to acquire  object-specific representations for recognition of a number of objects (or sets of  objects).  In  the  second  scheme,  an  Estimation  Network is  trained  to  acquire  function-specific, rather than object-specific, representations which directly estimate  physical parameters. Both  recognition networks are trained to identify manipulated  objects  using  somatic  and/or  visual  information.  After  learning,  appropriate  motor  commands  for  manipulation  of each  object  are  issued by  the  control  networks.
Oscillatory Model of Short Term Memory
Authors: David Horn, Marius Usher
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We investigate a model in which excitatory neurons have dynamical thresh(cid:173) olds  which  display  both  fatigue  and  potentiation.  The  fatigue  property  leads to oscillatory  behavior.  It is  responsible  for  the ability of the model  to  perform  segmentation,  i.e.,  decompose  a  mixed  input  into  staggered  oscillations  of the  activities  of the  cell-assemblies  (memories)  affected  by  it.  Potentiation  is  responsible  for  sustaining  these  staggered  oscillations  after  the  input  is  turned  off,  i.e.  the  system  serves  as  a  model for  short  term memory.  It  has  a  limited  STM  capacity, reminiscent  of the  magical  number 7 ± 2.
Estimating Average-Case Learning Curves Using Bayesian, Statistical Physics and VC Dimension Methods
Authors: David Haussler, Michael Kearns, Manfred Opper, Robert Schapire
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: In this paper we investigate an average-case model of concept learning, and  give results that place the popular statistical physics and VC dimension  theories of learning curve behavior in a common framework.
Operators and curried functions: Training and analysis of simple recurrent networks
Authors: Janet Wiles, Anthony Bloesch
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present a framework for programming tbe bidden unit representations of  simple recurrent networks based on the use of hint units (additional targets at  the output layer). We present two ways of analysing a network trained within  this framework: Input patterns act as operators on the information encoded by  the context units; symmetrically, patterns of activation over tbe context units  act as curried functions of the input sequences. Simulations demonstrate that a  network can learn to represent three different functions simultaneously and  canonical discriminant analysis is used to investigate bow operators and curried  functions are represented in the space of bidden unit activations.
CCD Neural Network Processors for Pattern Recognition
Authors: Alice M. Chiang, Michael L. Chuang, Jeffrey R. LaFranchise
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A  CCD-based  processor  that we  call  the  NNC2  is  presented.  The  NNC2  implements a  fully  connected  192-input, 32-output two-layer network  and  can  be  cascaded  to  form  multilayer  networks  or  used  in  parallel  for  ad(cid:173) ditional  input or  output  nodes.  The device  computes  1.92  x  109  connec(cid:173) tions/sec when clocked at 10 MHz.  Network weights can be specified to six  bits of accuracy and are stored on-chip in  programmable digital memories.  A  neural  network  pattern  recognition  system using  NNC2  and  CCD  im(cid:173) age  feature  extractor  (IFE)  devices  is  described.  Additionally,  we  report  a  CCD  output  circuit  that  exploits  inherent  nonlinearities  in  the  charge  injection  process to realize  an  adjustable-threshold sigmoid  in  a  chip  area  of 40  x  80  J.tlU2 .
A Simple Weight Decay Can Improve Generalization
Authors: Anders Krogh, John A. Hertz
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: It has  been observed  in  numerical simulations that a weight decay  can  im(cid:173) prove generalization in a feed-forward  neural network.  This paper explains  why.  It is  proven  that  a  weight  decay  has  two effects  in  a  linear  network.  First,  it  suppresses  any  irrelevant  components  of  the  weight  vector  by  choosing  the smallest  vector  that solves  the learning  problem.  Second,  if  the size  is chosen  right,  a weight  decay  can suppress some of the effects  of  static  noise  on  the  targets,  which  improves  generalization  quite  a  lot.  It  is  then  shown  how  to extend  these  results  to networks  with hidden  layers  and  non-linear  units.  Finally  the  theory  is  confirmed  by  some numerical  simulations using  the  data from  NetTalk.
Neural Network Diagnosis of Avascular Necrosis from Magnetic Resonance Images
Authors: Armando Manduca, Paul Christy, Richard Ehman
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A vascular necrosis (AVN) of the femoral head is a common yet poten(cid:173) tially serious disorder which can be detected in its very early stages with  magnetic resonance imaging. We have developed multi-layer perceptron  networks, trained with conjugate gradient optimization, which diagnose  A VN from single magnetic resonance images of the femoral head with  100% accuracy on training data and 97% accuracy on test data.
Obstacle Avoidance through Reinforcement Learning
Authors: Tony J. Prescott, John E. W. Mayhew
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A  method  is  described  for  generating  plan-like.  reflexive.  obstacle  avoidance behaviour in a mobile robot. The experiments reported here  use  a  simulated  vehicle  with  a  primitive  range  sensor.  Avoidance  behaviour is encoded as a set of continuous functions of the  perceptual  input space. These functions are stored  using CMACs and trained by a  variant of Barto and Sutton's adaptive critic  algorithm.  As  the  vehicle  explores  its  surroundings it adapts  its  responses  to  sensory stimuli  so  as  to  minimise  the  negative  reinforcement  arising  from  collisions.  Strategies  for  local  navigation  are  therefore  acquired  in  an  explicitly  goal-driven  fashion.  The resulting  trajectories  form  elegant collision(cid:173) free paths through the environment
The VC-Dimension versus the Statistical Capacity of Multilayer Networks
Authors: Chuanyi Ji, Demetri Psaltis
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: A  general  relationship  is  developed  between  the  VC-dimension  and  the  statistical lower  epsilon-capacity which shows  that the VC-dimension can  be  lower  bounded  (in  order)  by the statistical  lower epsilon-capacity of a  network  trained  with  random  samples.  This  relationship  explains  quan(cid:173) titatively  how  generalization  takes  place  after  memorization,  and  relates  the concept of generalization (consistency) with the capacity of the optimal  classifier over a class of classifiers with the same structure and the capacity  of the Bayesian classifier.  Furthermore, it provides a general methodology  to evaluate a  lower  bound for  the VC-dimension of feedforward multilayer  neural networks.  This  general  methodology  is  applied  to  two  types  of networks  which  are  important  for  hardware  implementations:  two  layer  (N  - 2L  - 1)  net(cid:173) works  with  binary  weights,  integer  thresholds  for  the  hidden  units  and  zero  threshold  for  the  output  unit,  and  a  single  neuron  ((N - 1)  net(cid:173) works)  with  binary  weigths  and  a  zero  threshold.  Specifically,  we  obtain  OC~L) ::;  d2  ::;  O(W),  and  d1  ""'  O(N).  Here  W  is  the  total  number  of weights  of  the  (N - 2L - 1)  networks.  d1  and  d2  represent  the  VC(cid:173) dimensions  for  the (N - 1)  and  (N - 2L - 1)  networks respectively.
Improving the Performance of Radial Basis Function Networks by Learning Center Locations
Authors: Dietrich Wettschereck, Thomas Dietterich
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Three methods for improving the performance of (gaussian) radial basis  function (RBF) networks were tested on the NETtaik task. In RBF, a  new example is classified by computing its Euclidean distance to a set of  centers chosen by unsupervised methods. The application of supervised  learning to learn a non-Euclidean distance metric was found to reduce the  error rate of RBF networks, while supervised learning of each center's vari(cid:173) ance resulted in inferior performance. The best improvement in accuracy  was achieved by networks called generalized radial basis function (GRBF)  networks. In GRBF, the center locations are determined by supervised  learning. After training on 1000 words, RBF classifies 56.5% of letters  correct, while GRBF scores 73.4% letters correct (on a separate test set).  From these and other experiments, we conclude that supervised learning  of center locations can be very important for radial basis function learning.
Models Wanted: Must Fit Dimensions of Sleep and Dreaming
Authors: J. Allan Hobson, Adam N. Mamelak, Jeffrey P. Sutton
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: During waking and sleep, the brain and mind undergo a  tightly linked and  precisely  specified  set  of changes  in state.  At  the  level  of neurons,  this  process  has  been  modeled  by  variations  of Volterra-Lotka  equations  for  cyclic fluctuations of brainstem cell populations.  However, neural network  models based upon rapidly developing knowledge ofthe specific population  connectivities  and  their  differential responses  to drugs  have  not  yet  been  developed.  Furthermore, only  the  most  preliminary  attempts  have  been  made  to model across states.  Some  of our own attempts  to link rapid eye  movement (REM) sleep neurophysiology and dream cognition using neural  network approaches  are summarized in this  paper.
Neural Control for Rolling Mills: Incorporating Domain Theories to Overcome Data Deficiency
Authors: Martin Röscheisen, Reimar Hofmann, Volker Tresp
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: In  a  Bayesian  framework,  we  give  a  principled  account  of how  domain(cid:173) specific prior knowledge such  as imperfect analytic domain theories can be  optimally  incorporated  into  networks  of locally-tuned  units:  by  choosing  a  specific  architecture  and  by  applying  a  specific  training  regimen.  Our  method  proved  successful  in  overcoming  the  data  deficiency  problem  in  a  large-scale  application  to  devise  a  neural  control  for  a  hot  line  rolling  mill.  It  achieves  in  this  application  significantly  higher  accuracy  than  optimally-tuned standard  algorithms such  as  sigmoidal backpropagation,  and  outperforms the state-of-the-art solution.
Polynomial Uniform Convergence of Relative Frequencies to Probabilities
Authors: Alberto Bertoni, Paola Campadelli, Anna Morpurgo, Sandra Panizza
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We  define  the  concept  of  polynomial  uniform  convergence  of  relative  frequencies  to  probabilities  in  the  distribution-dependent  context.  Let  Xn  = {O, l}n, let Pn be a  probability distribution on Xn  and let Fn  C  2X ,.  be  a  family  of events.  The  family  {(Xn, Pn, Fn)}n~l  has  the  property  of polynomial uniform  convergence  if the  probability  that  the  maximum  difference  (over  Fn)  between  the  relative  frequency  and  the  probabil(cid:173) ity  of  an  event  exceed  a  given  positive  e  be  at  most  6  (0  <  6  <  1),  when  the sample on  which  the frequency  is  evaluated  has size  polynomial  in  n,l/e,l/b.  Given  at-sample  (Xl, ... ,Xt),  let  C~t)(XI, ... ,Xt)  be  the  Vapnik-Chervonenkis dimension of the family {{x}, ... ,xtl n f  I f  E  Fn}  and  M(n, t)  the  expectation  E(C~t) It).  We  show  that  {(Xn, Pn, Fn)}n~l  has  the  property of polynomial uniform convergence  iff there exists  f3  > 0  such  that  M(n, t)  = O(n/t!3).  Applications  to  distribution-dependent  PAC learning are discussed.
Forward Dynamics Modeling of Speech Motor Control Using Physiological Data
Authors: Makoto Hirayama, Eric Vatikiotis-Bateson, Mitsuo Kawato, Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We  propose  a  paradigm  for  modeling  speech  production  based  on  neural  networks.  We  focus  on characteristics  of the  musculoskeletal  system.  Using  real physiological data - articulator movements and EMG from  muscle activity(cid:173) a  neural  network  learns  the  forward  dynamics  relating  motor  commands  to  muscles  and  the  ensuing  articulator  behavior.  After  learning,  simulated  perturbations, were used to asses properties of the acquired model, such as natural  frequency,  damping,  and  interarticulator  couplings.  Finally,  a  cascade neural  network is  used  to  generate  continuous  motor commands from  a  sequence  of  discrete articulatory targets.
Learning to Segment Images Using Dynamic Feature Binding
Authors: Michael Mozer, Richard S. Zemel, Marlene Behrmann
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Despite  the fact  that complex visual  scenes  contain multiple,  overlapping  objects,  people  perform  object  recognition  with  ease  and accuracy.  One  operation  that facilitates  recognition  is  an early  segmentation  process  in  which  features of objects are grouped and labeled  according  to which  ob(cid:173) ject they belong.  Current computational systems  that perform this oper(cid:173) ation  are based  on predefined  grouping heuristics.  We  describe  a  system  called  MAGIC  that  learn.  how  to  group  features  based  on  a  set  of pre(cid:173) segmented  examples.  In many cases,  MAGIC  discovers  grouping heuristics  similar to those previously proposed, but it also has the capability of find(cid:173) ing  nonintuitive  structural regularities  in images.  Grouping is  performed  by  a  relaxation  network  that  aUempts  to  dynamically  bind  related  fea(cid:173) tures.  Features  transmit  a  complex-valued  signal  (amplitude  and  phase)  to one another;  binding  can thus be represented  by phase locking  related  features.  MAGIC'S  training  procedure is  a  generalization  of recurrent  back  propagation to complex-valued units. 
When a  visual image contains multiple,  overlapping objects,  recognition is  difficult  because features in the image are not grouped according to which object they belong.  Without the capability to form such groupings, it would be necessary to undergo a  massive search through all subsets of image features.  For this reason,  most machine  vision  recognition  systems include  a  component  that  performs feature  grouping or  image  .egmentation (e.g.,  Guzman,  1968; Lowe,  1985;  Marr,  1982).
Simulation of Optimal Movements Using the Minimum-Muscle-Tension-Change Model
Authors: Menashe Dornay, Yoji Uno, Mitsuo Kawato, Ryoji Suzuki
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: This  work  discusses  various  optimization  techniques  which  were  proposed  in  models  for  controlling  arm  movements.  In  particular,  the  minimum-muscle-tension-change  model  is  investigated.  A  dynamic  simulator of the  monkey's  arm,  including  seventeen  single  and double  joint muscles,  is  utilized  to  generate  horizontal  hand  movements.  The  hand  trajectories  produced by  this  algorithm  are discussed.
Neural Network Analysis of Event Related Potentials and Electroencephalogram Predicts Vigilance
Authors: Rita Venturini, William W. Lytton, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Automated  monitoring  of  vigilance  in  attention  intensive  tasks  such  as  air  traffic  control  or  sonar  operation  is  highly  desirable.  As  the  opera(cid:173) tor monitors the  instrument,  the  instrument  would  monitor the operator,  insuring against lapses.  We  have taken a first  step  toward this goal by  us(cid:173) ing feedforward neural networks trained with backpropagation to interpret  event  related  potentials  (ERPs)  and electroencephalogram  (EEG)  associ(cid:173) ated with periods of high and low vigilance.  The accuracy of our system on  an ERP data set  averaged  over  28  minutes was  96%,  better than the 83%  accuracy  obtained  using  linear  discriminant  analysis.  Practical  vigilance  monitoring will require prediction over shorter time periods.  We were  able  to  average  the  ERP  over  as  little  as  2  minutes  and  still  get  90%  correct  prediction of a vigilance measure.  Additionally, we  achieved similarly good  performance  using segments of EEG  power  spectrum  as  short  as  56  sec.
HARMONET: A Neural Net for Harmonizing Chorales in the Style of J. S. Bach
Authors: Hermann Hild, Johannes Feulner, Wolfram Menzel
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: HARMONET, a system employing connectionist networks for music pro(cid:173) cessing, is presented. After being trained on some dozen Bach chorales  using error backpropagation, the system is capable of producing four-part  chorales in the style of J .s.Bach, given a one-part melody. Our system  solves a musical real-world problem on a performance level appropriate  for musical practice. HARMONET's power is based on (a) a new coding  scheme capturing musically relevant information and (b) the integration of  backpropagation and symbolic algorithms in a hierarchical system, com(cid:173) bining the advantages of both.
A Self-Organizing Integrated Segmentation and Recognition Neural Net
Authors: Jim Keeler, David E. Rumelhart
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We present a neural network algorithm that simultaneously performs seg(cid:173) mentation and recognition of input patterns that self-organizes to detect  input pattern locations and pattern boundaries. We demonstrate this neu(cid:173) ral network architecture on character recognition using the NIST database  and report on results herein. The resulting system simultaneously seg(cid:173) ments and recognizes touching or overlapping characters, broken charac(cid:173) ters, and noisy images with high accuracy.
A Comparison of Projection Pursuit and Neural Network Regression Modeling
Authors: Jenq-Neng Huang, Hang Li, Martin Maechler, R. Douglas Martin, Jim Schimert
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Two  projection  based  feedforward  network  learning  methods  for  model(cid:173) free  regression  problems  are  studied  and  compared  in  this  paper:  one  is  the  popular  back-propagation learning  (BPL);  the  other  is  the  projection  pursuit learning  (PPL).  Unlike  the  totally  parametric  BPL  method,  the  PPL  non-parametrically  estimates  unknown  nonlinear  functions  sequen(cid:173) tially (neuron-by-neuron and layer-by-Iayer) at each iteration while jointly  estimating  the  interconnection  weights.  In  terms  of  learning  efficiency,  both  methods  have  comparable  training  speed  when  based  on  a  Gauss(cid:173) Newton  optimization  algorithm  while  the  PPL  is  more  parsimonious.  In  terms of learning robustness  toward noise  outliers,  the  BPL is  more sensi(cid:173) tive to  the  outliers.
Unsupervised Classifiers, Mutual Information and 'Phantom Targets
Authors: John S. Bridle, Anthony J. R. Heading, David J. C. MacKay
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: David J.e. MacKay 
California Institute of Technology  139-74 
Pasadena CA 91125  U.S.A 
We derive criteria for training adaptive classifier networks to perform unsu(cid:173) pervised data analysis.  The first  criterion turns a simple Gaussian classifier  into  a  simple  Gaussian  mixture  analyser.  The  second  criterion,  which  is  much more generally applicable, is based on mutual information. It simpli(cid:173) fies  to an  intuitively  reasonable  difference  between  two entropy functions,  one  encouraging  'decisiveness,'  the  other  'fairness'  to  the  alternat.ive  in(cid:173) terpretations  of the  input.  This  'firm  but  fair'  criterion  can  be  applied  to  any  network  that  produces  probability-type  outputs,  but  it  does  not  necessarily  lead to useful  behavior. 
1  Unsupervised Classification 
One of the  main distinctions made in  discussing  neural  network  architectures,  and  pattern analysis algorithms generally, is  between  supervised and unsupervised data  analysis.  We  should  therefore  be  interested  in  any  method  of  building  bridges  between  techniques  in  these  two  categories.  For  instance,  it  is  possible  to  use  an  unsupervised system such as a  Boltzmann machine to learn  the joint distribution of  inputs and a teacher's classificat.ion labels.  The particular type of bridge we seek is a  method of taking a supervised pattern classifier and turning it into an unsupervised  data analyser.  That is,  we  are interested in  methods of "bootstrapping"  classifiers. 
Consider  a  classifier system.  Its input is  a  vector x, and the output is  a  probability  vector y(x).  (That is,  the elements ofy are positive and sum to 1.)  The elements of  y, (Yi (x), i  =  1 ... N c )  are to be taken as the probabilities that x  should be assigned  to  each  of  Nc  classes.  (Note  that  our  definition  of  classifier  does  not  include  a  decision  process.)
Induction of Finite-State Automata Using Second-Order Recurrent Networks
Authors: Raymond L. Watrous, Gary M. Kuhn
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: Second-order  recurrent  networks  that  recognize  simple  finite  state  lan(cid:173) guages over  {0,1}* are  induced  from  positive and negative examples.  Us(cid:173) ing the  complete gradient of the  recurrent  network  and sufficient  training  examples to  constrain  the  definition  of the  language to  be induced,  solu(cid:173) tions  are  obtained  that  correctly  recognize  strings of arbitrary  length.  A  method for  extracting  a  finite  state  automaton corresponding  to  an  opti(cid:173) mized network is demonstrated.
Network generalization for production: Learning and producing styled letterforms
Authors: Igor Grebert, David G. Stork, Ron Keesing, Steve Mims
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We designed and trained a connectionist network to generate  letterfonns in a new font given just a few exemplars from  that font. During learning. our network constructed a  distributed internal representation of fonts as well as letters.  despite the fact that each training instance exemplified both a  font and a letter. It was necessary to have separate but  interconnected hidden units for " letter" and "font"  representations - several alternative architectures were not  successful.
Green's Function Method for Fast On-Line Learning Algorithm of Recurrent Neural Networks
Authors: Guo-Zheng Sun, Hsing-Hen Chen, Yee-Chun Lee
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: The two well known learning algorithms of recurrent neural networks are  the back-propagation (Rumelhart & el al.,  Werbos) and the  forward propa(cid:173) gation (Williams and Zipser). The main drawback of back-propagation is  its  off-line backward path in time for error cumulation. This violates the on-line  requirement in many  practical applications.  Although the  forward propaga(cid:173) tion algorithm can be used in an on-line manner, the annoying drawback is  the heavy computation load required to update the high dimensional sensitiv(cid:173) ity matrix (0( fir) operations for each time step). Therefore, to develop a fast  forward algorithm is a challenging task. In this paper w~ proposed a forward  learning algorithm which is one order faster (only 0(fV3) operations for each  time step) than the sensitivity matrix algorithm. The basic idea is that instead  of integrating  the  high  dimensional  sensitivity dynamic  equation  we  solve  forward in time for its Green's function to avoid the redundant computations,  and then update the weights whenever the error is to be corrected. 
A Numerical  example  for  classifying  state  trajectories  using  a recurrent 
network is  presented. It substantiated the faster speed of the proposed algo(cid:173) rithm than the Williams and Zipser's algorithm.
Constrained Optimization Applied to the Parameter Setting Problem for Analog Circuits
Authors: David Kirk, Kurt Fleischer, Lloyd Watts, Alan Barr
Published in: Advances in Neural Information Processing Systems 4  (NIPS 1991) (Main Conference Track), 1991

Abstract: We use constrained optimization to select operating parameters for two  circuits: a simple 3-transistor square root circuit, and an analog VLSI  artificial cochlea. This automated method uses computer controlled mea(cid:173) surement and test equipment to choose chip parameters which minimize  the difference between the actual circuit's behavior and a specified goal  behavior. Choosing the proper circuit parameters is important to com(cid:173) pensate for manufacturing deviations or adjust circuit performance within  a certain range. As biologically-motivated analog VLSI circuits become  increasingly complex, implying more parameters, setting these parameters  by hand will become more cumbersome. Thus an automated parameter  setting method can be of great value [Fleischer 90]. Automated parameter  setting is an integral part of a goal-based engineering design methodology  in which circuits are constructed with parameters enabling a wide range  of behaviors, and are then "tuned" to the desired behaviors automatically.