mtybilly/Instruct2DS huggingface dataset:
Silicon Auditory Processors as Computer Peripherals
Authors: John Lazzaro, John Wawrzynek, M. Mahowald, Massimo Sivilotti, Dave Gillespie
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Sever

Present address: f\1. Mahowald, f\1H.C ,\natol1lical Ncmophamacology  Unit, Mansfield TId, Oxfc)('d OXI :1'rl£ Ellgland . mam~vax.oxford.ac.uk  t Present address:  f\lass Siviloui. '1'(1111)(,1' H,csearrh, 180 Nort.h Vinedo  Avenue, Pasadena, CA 9Il07. mass~tanner. corn  :I: Present address: Dave Gill,>spiE', SYllapf,ics, :l()!)8 Orchard Parkway, San  Jose CA, 95131. daveg~synaptics. com
Information, Prediction, and Query by Committee
Authors: Yoav Freund, H. Sebastian Seung, Eli Shamir, Naftali Tishby
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We  analyze the  "query  by  committee"  algorithm, a  method for  fil(cid:173) tering  informative  queries  from  a  random  stream  of inputs.  We  show  that if the  two-member committee algorithm achieves  infor(cid:173) mation  gain  with  positive  lower  bound,  then  the  prediction  error  decreases  exponentially with the number of queries.  We show that,  in  particular,  this  exponential  decrease  holds for  query  learning of  thresholded  smooth functions.
Improving Convergence in Hierarchical Matching Networks for Object Recognition
Authors: Joachim Utans, Gene Gindi
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We  are  interested  in  the  use  of analog  neural  networks  for  recog(cid:173) nizing  visual  objects.  Objects  are  described  by  the  set  of parts  they  are  composed  of  and  their  structural  relationship.  Struc(cid:173) tural  models  are  stored  in  a  database  and  the  recognition  prob(cid:173) lem  reduces  to  matching  data  to  models  in  a  structurally  consis(cid:173) tent  way.  The  object  recognition  problem is  in  general  very  diffi(cid:173) cult in that it involves coupled problems of grouping, segmentation  and  matching.  We  limit the  problem here  to  the simultaneous la(cid:173) belling  of the  parts  of a  single  object  and  the  determination  of  analog  parameters.  This  coupled  problem  reduces  to  a  weighted  match  problem in  which  an  optimizing neural  network  must  min(cid:173) imize  E(M, p)  =  LO'i MO'i WO'i(p),  where  the  {MO'd  are  binary  match  variables  for  data parts  i  to  model  parts  a  and  {Wai(P)}  are weights dependent on parameters p .  In this work we  show that  by  first  solving  for  estimates p without  solving  for  M ai , we  may  obtain good initial parameter estimates that yield  better solutions  for  M  and  p. 
*Current  address: 
International  Computer  Science  Institute,  1947  Center  Street, 
Suite  600,  Berkeley,  CA 94704,  utans@icsi.berkeley.edu 
tCurrent  address:  SUNY  Stony  Brook,  Department  of  Electrical  Engineering,  Stony 
Brook,  NY  11784
A dynamical model of priming and repetition blindness
Authors: Daphne Bavelier, Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We describe a model of visual word recognition that accounts for  several aspects of the temporal processing of sequences of briefly  presented words. The model utilizes a new representation for writ(cid:173) ten words, based on dynamic time warping and multidimensional  scaling. The visual input passes through cascaded perceptual, com(cid:173) parison, and detection stages. We describe how these dynamical  processes can account for several aspects of word recognition, in(cid:173) cluding repetition priming and repetition blindness.
Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain
Authors: Nicol N. Schraudolph, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We  present the  information-theoretic derivation  of a learning algorithm  that  clusters  unlabelled  data  with  linear  discriminants.  In  contrast  to  methods  that  try  to  preserve  information  about  the  input patterns,  we  maximize  the  information  gained  from  observing  the  output  of robust  binary discriminators implemented with sigmoid nodes. We deri ve a local  weight adaptation rule via gradient ascent in this objective, demonstrate  its  dynamics on  some simple data sets,  relate our approach  to  previous  work and suggest directions in which it may be extended.
Using Aperiodic Reinforcement for Directed Self-Organization During Development
Authors: P. R. Montague, P. Dayan, S.J. Nowlan, A Pouget, T.J. Sejnowski
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We present a local learning rule in which Hebbian learning is  conditional on an incorrect prediction of a reinforcement signal.  We propose a biological interpretation of such a framework and  display its utility through examples in which the reinforcement  signal is cast as the delivery of a neuromodulator to its target.  Three exam pIes are presented which illustrate how this framework  can be applied to the development of the oculomotor system.
A Connectionist Symbol Manipulator That Discovers the Structure of Context-Free Languages
Authors: Michael Mozer, Sreerupa Das
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We present a neural net architecture that can discover hierarchical and re(cid:173) cursive structure in symbol strings. To detect structure at multiple levels,  the architecture has the capability of reducing symbols substrings to single  symbols, and makes use of an external stack memory. In terms of formal  languages, the architecture can learn to parse strings in an LR(O) context(cid:173) free grammar. Given training sets of positive and negative exemplars,  the architecture has been trained to recognize many different grammars.  The architecture has only one layer of modifiable weights, allowing for a  straightforward interpretation of its behavior. 
Many cognitive domains involve complex sequences that contain hierarchical or  recursive structure, e.g., music, natural language parsing, event perception. To il(cid:173) lustrate, "the spider that ate the hairy fly" is a noun phrase containing the embed(cid:173) ded noun phrase "the hairy fly." Understanding such multilevel structures requires  forming reduced descriptions (Hinton, 1988) in which a string of symbols or states  ("the hairy fly") is reduced to a single symbolic entity (a noun phrase). We present  a neural net architecture that learns to encode the structure of symbol strings via  such red uction transformations. 
The difficult problem of extracting multilevel structure from complex, extended  sequences has been studied by Mozer (1992), Ring (1993), Rohwer (1990), and  Schmidhuber (1992), among others. While these previous efforts have made some
Analogy-- Watershed or Waterloo? Structural alignment and the development of connectionist models of analogy
Authors: Dedre Gentner, Arthur B. Markman
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Neural network models have been criticized for their inability to make  use of compositional representations. In this paper, we describe a series  of psychological phenomena that demonstrate the role of structured  representations in cognition. These findings suggest that people  compare relational representations via a process of structural alignment.  This process will have to be captured by any model of cognition,  symbolic or subsymbolic.
Network Structuring and Training Using Rule-based Knowledge
Authors: Volker Tresp, Jürgen Hollatz, Subutai Ahmad
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We demonstrate in this paper how certain forms of rule-based  knowledge can be used to prestructure a neural network of nor(cid:173) malized basis functions and give a probabilistic interpretation of  the network architecture. We describe several ways to assure that  rule-based knowledge is preserved during training and present a  method for complexity reduction that tries to minimize the num(cid:173) ber of rules and the number of conjuncts. After training the refined  rules are extracted and analyzed.
Hybrid Circuits of Interacting Computer Model and Biological Neurons
Authors: Sylvie Renaud-Le Masson, Gwendal Le Masson, Eve Marder, L. F. Abbott
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We demonstrate the use of a digital signal processing board to construct  hybrid networks consisting of computer model neurons connected to a  biological neural  network.  This system  operates  in  real  time.  and  the  synaptic  connections  are  realistic  effective  conductances.  Therefore.  the  synapses  made  from  the  computer  model  neuron  are  integrated  correctly  by  the postsynaptic biological neuron.  This method provides  us  with  the  ability  to  add  additional.  completely known elements  to a  biological  network  and  study  their  effect  on  network  activity.  Moreover.  by  changing  the  parameters  of  the  model  neuron.  it  is  possible to assess  the role of individual conductances in  the activity of  the  neuron.  and  in  the  network in which it participates. 
"Present  address. 
l-Enserb.  CNRSURA  846.  351  crs  de  la  Liberation.  33405  Talence  Cedex.  France. 
lXL.  Universit6  de  Bordeaux 
'Present address.  LNPC.  CNRS.  Universite de Bordeaux  1.  Place 
de Dr.  Peyneau.  33120 Arcachon.  France
A Knowledge-Based Model of Geometry Learning
Authors: Geoffrey Towell, Richard Lehrer
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We propose a model of the development of geometric reasoning in children that  explicitly involves learning. The model uses a neural network that is initialized  with an understanding of geometry similar to that of second-grade children.  Through the presentation of a series of examples, the model is shown to develop  an understanding of geometry similar to that of fifth-grade children who were  trained using similar materials.
Integration of Visual and Somatosensory Information for Preshaping Hand in Grasping Movements
Authors: Yoji Uno, Naohiro Fukumura, Ryoji Suzuki, Mitsuo Kawato
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The primate brain must solve two important problems in grasping move(cid:173) ments.  The  first  problem  concerns  the  recognition  of grasped objects:  specifically,  how  does  the  brain integrate  visual  and motor information  on a grasped object? The second problem concerns hand shape planning:  specifically, how does the brain design the hand configuration suited to the  shape of the  object and the manipulation task?  A neural network model  that solves these problems has been developed.  The operations of the net(cid:173) work are divided into a learning phase and an optimization phase.  In the  learning phase, internal representations, which depend on the grasped ob(cid:173) jects and the task,  are  acquired by integrating visual and somatosensory  information.  In the optimization phase, the  most suitable hand shape for  grasping an object is determined by using a relaxation computation of the  network. 

Present  Address: 

Parallel  Distributed  Processing  Research  Dept.,  Sony  Corporation, 
6-7-35 Kitashinagawa, Shinagawa-ku, Tokyo 141, Japan
Memory-Based Reinforcement Learning: Efficient Computation with Prioritized Sweeping
Authors: Andrew W. Moore, Christopher G. Atkeson
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We present a  new algorithm, Prioritized Sweeping, for  efficient  prediction  and control of stochastic Markov systems.  Incremental learning methods  such as Temporal Differencing  and  Q-Iearning have fast  real time perfor(cid:173) mance.  Classical  methods  are  slower,  but  more  accurate,  because  they  make full  use  of the observations.  Prioritized Sweeping aims for  the  best  of both worlds.  It uses  all  previous  experiences both to  prioritize  impor(cid:173) tant dynamic  programming sweeps and to guide the exploration of state(cid:173) space.  We compare Prioritized Sweeping with other reinforcement learning  schemes for  a  number of different stochastic optimal control problems.  It  successfully  solves large state-space  real  time  problems  with  which  other  methods have difficulty. 
1  STOCHASTIC PREDICTION 
The paper introduces a memory-based technique, prioritized 6weeping, which is used  both for  stochastic  prediction and  reinforcement  learning.  A fuller  version  of this  paper is in preparation [Moore and Atkeson, 1992].  Consider the  500 state Markov  system depicted in Figure 1.  The system has sixteen absorbing states,  depicted by  white and black circles.  The prediction problem is  to estimate, for  every state, the  long-term  probability  that  it will  terminate  in  a  white,  rather  than  black,  circle.  The data available to the learner is a sequence of observed state transitions.  Let us  consider two existing methods along  with prioritized sweeping.
Learning Fuzzy Rule-Based Neural Networks for Control
Authors: Charles M. Higgins, Rodney M. Goodman
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A  three-step  method for  function  approximation with  a fuzzy  sys(cid:173) tem  is  proposed.  First,  the  membership  functions  and  an  initial  rule  representation  are  learned;  second,  the  rules  are  compressed  as  much as  possible  using  information theory;  and finally,  a  com(cid:173) putational network  is  constructed  to  compute  the  function  value.  This system  is  applied  to  two control examples:  learning the truck  and trailer backer-upper control system, and learning a  cruise  con(cid:173) trol system for  a  radio-controlled model car.
Hidden Markov Model Induction by Bayesian Model Merging
Authors: Andreas Stolcke, Stephen Omohundro
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: This paper describes a technique for learning both the number of states and the  topology of Hidden Markov Models from examples. The induction process starts  with the most specific model consistent with the training data and generalizes  by successively merging states. Both the choice of states to merge and the  stopping criterion are guided by the Bayesian posterior probability. We compare  our algorithm with the Baum-Welch method of estimating fixed-size models, and  find that it can induce minimal HMMs from data in cases where fixed estimation  does not converge or requires redundant parameters to converge. 
1 INTRODUCTION AND OVERVIEW 
Hidden Markov Models (HMMs) are a well-studied approach to the modelling of sequence  data. HMMs can be viewed as a stochastic generalization of finite-state automata, where  both the transitions between states and the generation of output symbols are governed by  probability distributions. HMMs have been important in speech recognition (Rabiner &  Juang, 1986), cryptography, and more recently in other areas such as protein classification  and alignment (Haussler, Krogh, Mian & SjOlander, 1992; Baldi, Chauvin, Hunkapiller &  McClure, 1993). 
Practitioners have typically chosen the HMM topology by hand, so that learning the HMM  from sample data means estimating only a fixed number of model parameters. The standard  approach is to find a maximum likelihood (ML) or maximum a posteriori probability (MAP)  estimate of the HMM parameters. The Baum-Welch algorithm uses dynamic programming
Statistical Mechanics of Learning in a Large Committee Machine
Authors: Holm Schwarze, John A. Hertz
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We  use  statistical  mechanics  to study  generalization in  large com(cid:173) mittee  machines.  For  an  architecture  with  nonoverlapping  recep(cid:173) tive fields a  replica calculation yields the generalization error in the  limit of a  large number of hidden units.  For continuous weights  the  generalization  error  falls  off asymptotically inversely  proportional  to  Q,  the  number  of  training  examples  per  weight.  For  binary  weights  we  find  a  discontinuous  transition  from  poor  to  perfect  generalization  followed  by  a  wide  region  of metastability.  Broken  replica symmetry is  found  within this  region  at low  temperatures.  For  a  fully  connected  architecture  the  generalization  error  is  cal(cid:173) culated  within  the  annealed  approximation.  For  both  binary  and  continuous  weights  we  find  transitions  from  a  symmetric state  to  one  with  specialized  hidden  units,  accompanied  by  discontinuous  drops  in  the generalization error.
An Information-Theoretic Approach to Deciphering the Hippocampal Code
Authors: William E. Skaggs, Bruce L. McNaughton, Katalin M. Gothard
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Information  theory  is  used  to  derive  a  simple  formula  for  the  amount of information conveyed by the firing rate of a neuron about  any  experimentally measured variable or  combination of variables  (e.g.  running speed,  head  direction,  location of the  animal, etc.).  The  derivation  treats  the  cell  as  a  communication channel  whose  input is the measured variable and whose output is the cell's spike  train.  Applying the formula,  we  find  systematic differences  in  the  information  content  of hippocampal  "place  cells"  in  different  ex(cid:173) perimental conditions.
Using hippocampal 'place cells' for navigation, exploiting phase coding
Authors: Neil Burgess, John O'Keefe, Michael Recce
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A model of the hippocampus as a central element in rat naviga(cid:173) tion is presented. Simulations show both the behaviour of single  cells and the resultant navigation of the rat. These are compared  with single unit recordings and behavioural data. The firing of  CAl place cells is simulated as the (artificial) rat moves in an en(cid:173) vironment. This is the input for a neuronal network whose output,  at each theta (0) cycle, is the next direction of travel for the rat.  Cells are characterised by the number of spikes fired and the time  of firing with respect to hippocampal 0 rhythm. 'Learning' occurs  in 'on-off' synapses that are switched on by simultaneous pre- and  post-synaptic activity. The simulated rat navigates successfully to  goals encountered one or more times during exploration in open  fields. One minute of random exploration of a 1m2 environment  allows navigation to a newly-presented goal from novel starting po(cid:173) sitions. A limited number of obstacles can be successfully avoided.
Weight Space Probability Densities in Stochastic Learning: I. Dynamics and Equilibria
Authors: Todd K. Leen, John E. Moody
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The  ensemble  dynamics  of stochastic  learning  algorithms  can  be  studied  using  theoretical  techniques  from  statistical  physics.  We  develop  the equations  of motion for  the  weight  space  probability  densities for  stochastic learning  algorithms.  We  discuss  equilibria  in the diffusion  approximation and provide expressions for  special  cases  of the LMS  algorithm.  The equilibrium  densities  are not in  general thermal (Gibbs) distributions in the objective function be(cid:173) ing minimized, but rather depend upon an effective potential that  includes  diffusion  effects.  Finally  we  present  an  exact  analytical  expression for the time evolution of the density for a learning algo(cid:173) rithm with weight updates proportional to the sign of the gradient. 
1 
Introduction:  Theoretical Framework 
Stochastic learning algorithms involve  weight  updates of the form 
w(n+1)  =  w(n)  + /-l(n)H[w(n),x(n)] 
(1) 
where w E 7£m  is  the vector of m  weights,  /-l  is  the learning rate, H[.]  E 7£m  is  the  update function,  and  x(n)  is  the exemplar  (input or  input/target pair)  presented
A Hybrid Neural Net System for State-of-the-Art Continuous Speech Recognition
Authors: G. Zavaliagkos, Y. Zhao, R. Schwartz, J. Makhoul
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Untill recently, state-of-the-art, large-vocabulary, continuous speech  recognition (CSR) has employed Hidden Markov Modeling (HMM)  to model speech sounds.  In an attempt to improve over HMM we  developed a hybrid system that integrates HMM technology with neu(cid:173) ral networks. We present the concept of a "Segmental Neural Net"  (SNN) for phonetic modeling in CSR. By taking into account all the  frames of a phonetic segment simultaneously, the SNN overcomes the  well-known conditional-independence limitation of HMMs. In several  speaker-independent experiments with the DARPA Resource Manage(cid:173) ment corpus, the hybrid system showed a consistent improvement in  performance over the baseline HMM system.
A Formal Model of the Insect Olfactory Macroglomerulus: Simulations and Analytic Results
Authors: Christiane Linster, David Marsan, Claudine Masson, Michel Kerszberg, Gérard Dreyfus, Léon Personnaz
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: It  is  known  from  biological  data  that  the  response  patterns  of  interneurons  in  the olfactory  macroglomerulus  (MGC) of insects are of  central importance for the coding of the olfactory signal. We propose an  analytically  tractable  model  of the  MGC  which allows us  to  relate  the  distribution of response patterns to the architecture of the network.
Using Prior Knowledge in a NNPDA to Learn Context-Free Languages
Authors: Sreerupa Das, C. Lee Giles, Guo-Zheng Sun
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Although considerable interest has been shown in language inference and  automata induction using recurrent neural networks, success of these  models has mostly been limited to regular languages. We have previ(cid:173) ously demonstrated that Neural Network Pushdown Automaton (NNPDA)  model is capable of learning deterministic context-free languages (e.g.,  anbn and parenthesis languages) from examples. However, the learning  task is computationally intensive. In this paper we discus some ways in  which a priori knowledge about the task and data could be used for efficient  learning. We also observe that such knowledge is often an experimental  prerequisite for learning nontrivial languages (eg. anbncbmam ).
Generalization Abilities of Cascade Network Architecture
Authors: E. Littmann, H. Ritter
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: In [5], a new incremental cascade network architecture has been  presented. This paper discusses the properties of such cascade  networks and investigates their generalization abilities under the  particular constraint of small data sets. The evaluation is done for  cascade networks consisting of local linear maps using the Mackey(cid:173) Glass time series prediction task as a benchmark. Our results in(cid:173) dicate that to bring the potential of large networks to bear on the  problem of extracting information from small data sets without run(cid:173) ning the risk of overjitting, deeply cascaded network architectures  are more favorable than shallow broad architectures that contain  the same number of nodes.
Explanation-Based Neural Network Learning for Robot Control
Authors: Tom M. Mitchell, Sebastian B. Thrun
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: How can artificial neural nets generalize better from fewer examples? In order  to generalize successfully, neural network learning methods typically require  large training data sets. We introduce a neural network learning method that  generalizes rationally from many fewer data points, relying instead on prior  knowledge encoded in previously learned neural networks. For example, in robot  control learning tasks reported here, previously learned networks that model the  effects of robot actions are used to guide subsequent learning of robot control  functions. For each observed training example of the target function (e.g. the  robot control policy), the learner explains the observed example in terms of its  prior knowledge, then analyzes this explanation to infer additional information  about the shape, or slope, of the target function. This shape knowledge is used  to bias generalization when learning the target function. Results are presented  applying this approach to a simulated robot task based on reinforcement learning.
Some Solutions to the Missing Feature Problem in Vision
Authors: Subutai Ahmad, Volker Tresp
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: In visual processing the ability to deal with  missing and noisy informa(cid:173) tion is crucial. Occlusions and unreliable feature detectors often lead to  situations where little or no  direct information about features  is  availa(cid:173) ble.  However  the  available  information  is  usually  sufficient  to  highly  constrain  the  outputs.  We  discuss  Bayesian  techniques  for  extracting  class probabilities given partial data. The optimal solution involves inte(cid:173) grating  over the  missing dimensions  weighted  by  the  local  probability  densities.  We  show  how  to  obtain  closed-form  approximations  to  the  Bayesian  solution  using  Gaussian  basis function  networks.  The frame(cid:173) work  extends  naturally  to  the case of noisy  features.  Simulations on a  complex task (3D  hand gesture recognition)  validate the  theory.  When  both integration and weighting by input densities are used, performance  decreases gracefully with the number of missing or noisy features.  Per(cid:173) formance is substantially degraded if either step is omitted.
A Note on Learning Vector Quantization
Authors: Virginia R. de Sa, Dana H. Ballard
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Vector Quantization is useful for data compression.  Competitive Learn(cid:173) ing which minimizes reconstruction error is an appropriate algorithm for  vector quantization of unlabelled data.  Vector quantization of labelled  data for classification has a different objective, to minimize the number  of misclassifications, and a different algorithm is appropriate.  We show  that a  variant of Kohonen's LVQ2.1  algorithm can  be  seen  as  a  multi(cid:173) class  extension of an  algorithm which in a  restricted  2  class  case  can  be proven to converge to the Bayes optimal classification boundary.  We  compare the performance of the LVQ2.1 algorithm to that of a modified  version having a decreasing window and normalized step size, on a ten  class vowel classification problem.
Analog Cochlear Model for Multiresolution Speech Analysis
Authors: Weimin Liu, Andreas G. Andreou, Moise H. Goldstein Jr.
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: This paper discusses the parameterization of speech by an analog cochlear  model.  The tradeoff between  time and  frequency  resolution  is  viewed  as  the  fundamental  difference  between  conventional  spectrographic  analysis  and cochlear signal processing for  broadband, rapid-changing signals.  The  model's response exhibits a  wavelet-like analysis in the scale domain that  preserves good temporal resolution; the frequency of each spectral compo(cid:173) nent  in  a  broadband signal  can be accurately  determined  from  the  inter(cid:173) peak  intervals  in  the  instantaneous  firing  rates  of auditory  fibers.  Such  properties  of the  cochlear  model  are  demonstrated  with  natural  speech  and synthetic complex signals.
A Boundary Hunting Radial Basis Function Classifier which Allocates Centers Constructively
Authors: Eric I. Chang, Richard P Lippmann
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A  new  boundary  hunting  radial  basis  function  (BH-RBF)  classifier  which  allocates  RBF  centers  constructively  near  class  boundaries  is  described.  This  classifier creates complex  decision  boundaries  only  in  regions  where  confusions  occur  and  corresponding  RBF  outputs  are  similar.  A  predicted  square  error  measure  is  used  to  determine  how  many centers to add and to determine when to stop adding centers. Two  experiments are presented which demonstrate the advantages of the BH(cid:173) RBF classifier.  One uses artificial data with  two classes and  two  input  features  where each class contains  four clusters but only  one cluster is  near a decision region boundary. The other uses a large seismic database  with  seven  classes and  14  input features.  In  both  experiments  the  BH(cid:173) RBF classifier provides a lower error rate  with  fewer  centers  than  are  required  by  more  conventional  RBF,  Gaussian  mixture,  or  MLP  classifiers.
Learning to See Where and What: Training a Net to Make Saccades and Recognize Handwritten Characters
Authors: Gale Martin, Mosfeq Rashid, David Chapman, James A. Pittman
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: to  integrated  segmentation  and  This  paper  describes  an  approach  recognition of hand-printed characters.  The approach, called Saccade,  integrates  ballistic  and  corrective  saccades  (eye  movements)  with  character recognition.  A single backpropagation net  is  trained to make  a classification decision on a character centered  in its input window, as  well as to estimate the distance of the current and next character from the  center of the input window.  The net  learns to accurately estimate these  distances  regardless  of variations  in  character  width,  spacing  between  characters,  writing  style  and  other factors.  During  testing,  the  system  uses  the  net~xtracted classification  and  distance  information,  along  with a set of jumping rules, to jump from character to character. 
The ability to read rests on multiple foundation skills.  In learning how to read, people learn  how to recognize individual characters centered in the visual  field.  They also learn how  to move their eyes along a line of text, sequentially centering the visual field on successive  characters.  We believe that the key to developing optical character recognition (OCR) sys(cid:173) tems that can mimic human reading capabilities, is to develop systems that can learn these  and other skills in an integrated fashion.  In this paper, we demonstrate that a backpropaga(cid:173) tion net can learn to naVigate along a line of handwritten characters, as  well as to recognize  the characters centered in its visual field.  The system, called Saccade, extends the current  state of the art in OCR technology by using a single classifier to accurately and efficiently  locate and recognize characters, regardless of whether they touch each other or are sepa(cid:173) rate.  The Saccade system was described briefly at the last NIPS conference (Martin & Ra(cid:173) shid,  1992).  In this  paper,  we  describe  it  mcx-e  fully  and report on results demonstrating  its accuracy and efficiency in recognizing handwritten digits. 
The Saccade system takes a cue from the ballistic and corrective saccades (eye movements)  of natural vision systems.  Natural saccades make it possible to efficiently move from one  informative area to another by jumping.  The eye  typically  initiates a ballistic saccade to
How Oscillatory Neuronal Responses Reflect Bistability and Switching of the Hidden Assembly Dynamics
Authors: K. Pawelzik, H.-U. Bauer, J. Deppisch, T. Geisel
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A switching between apparently coherent (oscillatory) and stochastic  episodes of activity has been observed in responses from cat and monkey  visual cortex. We describe the dynamics of these phenomena in two paral(cid:173) lel approaches, a phenomenological and a rather microscopic one. On the  one hand we analyze neuronal responses in terms of a hidden state model  (HSM). The parameters of this model are extracted directly from exper(cid:173) imental spike trains. They characterize the underlying dynamics as well  as the coupling of individual neurons to the network. This phenomenolog(cid:173) ical model thus provides a new framework for the experimental analysis  of network dynamics. The application of this method to multi unit ac(cid:173) tivities from the visual cortex of the cat substantiates the existence of  oscillatory and stochastic states and quantifies the switching behaviour  in the assembly dynamics. On the other hand we start from the single  spiking neuron and derive a master equation for the time evolution of the  assembly state which we represent by a phase density. This phase density  dynamics (PDD) exhibits costability of two attractors, a limit cycle, and  a fixed point when synaptic interaction is nonlinear. External fluctuations  can switch the bistable system from one state to the other. Finally we  show, that the two approaches are mutually consistent and therefore both  explain the detailed time structure in the data.
Summed Weight Neuron Perturbation: An O(N) Improvement Over Weight Perturbation
Authors: Barry Flower, Marwan Jabri
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: The algorithm presented performs gradient descent on the weight space  of an Artificial Neural Network (ANN), using a finite difference to  approximate the gradient The method is novel in that it achieves a com(cid:173) putational complexity similar to that of Node Perturbation, O(N3), but  does not require access to the activity of hidden or internal neurons.  This is possible due to a stochastic relation between perturbations at the  weights and the neurons of an ANN. The algorithm is also similar to  Weight Perturbation in that it is optimal in terms of hardware require(cid:173) ments when used for the training ofVLSI implementations of ANN's.
Extended Regularization Methods for Nonconvergent Model Selection
Authors: W. Finnoff, F. Hergert, H. G. Zimmermann
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Many techniques for model selection in the field of neural networks  correspond to well established statistical methods. The method  of 'stopped training', on the other hand, in which an oversized  network is trained until the error on a further validation set of ex(cid:173) amples deteriorates, then training is stopped, is a true innovation,  since model selection doesn't require convergence of the training  process.  In this paper we show that this performance can be significantly  enhanced by extending the 'non convergent model selection method'  of stopped training to include dynamic topology modifications  (dynamic weight pruning) and modified complexity penalty term  methods in which the weighting of the penalty term is adjusted  during the training process.
Synchronization and Grammatical Inference in an Oscillating Elman Net
Authors: Bill Baird, Todd Troyer, Frank Eeckman
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We have designed an architecture to span the gap between bio(cid:173) physics and cognitive science to address and explore issues of how  a discrete symbol processing system can arise from the continuum,  and how complex dynamics like oscillation and synchronization can  then be employed in its operation and affect its learning. We show  how a discrete-time recurrent "Elman" network architecture can  be constructed from recurrently connected oscillatory associative  memory modules described by continuous nonlinear ordinary dif(cid:173) ferential equations. The modules can learn connection weights be(cid:173) tween themselves which will cause the system to evolve under a  clocked "machine cycle" by a sequence of transitions of attractors  within the modules, much as a digital computer evolves by transi(cid:173) tions of its binary flip-flop attractors. The architecture thus em(cid:173) ploys the principle of "computing with attractors" used by macro(cid:173) scopic systems for reliable computation in the presence of noise. We  have specifically constructed a system which functions as a finite  state automaton that recognizes or generates the infinite set of six  symbol strings that are defined by a Reber grammar. It is a symbol  processing system, but with analog input and oscillatory subsym(cid:173) bolic representations. The time steps (machine cycles) of the sys(cid:173) tem are implemented by rhythmic variation (clocking) of a bifurca(cid:173) tion parameter. This holds input and "context" modules clamped  at their attractors while 'hidden and output modules change state,  then clamps hidden and output states while context modules are  released to load those states as the new context for the next cycle of  input. Superior noise immunity has been demonstrated for systems  with dynamic attractors over systems with static attractors, and  synchronization ("binding") between coupled oscillatory attractors  in different modules has been shown to be important for effecting  reliable transitions.
Attractor Neural Networks with Local Inhibition: from Statistical Physics to a Digitial Programmable Integrated Circuit
Authors: E. Pasero, R. Zecchina
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Networks with local inhibition are shown to have enhanced compu(cid:173) tational performance with respect to the classical Hopfield-like net(cid:173) works. In particular the critical capacity of the network is increased  as well as its capability to store correlated patterns. Chaotic dy(cid:173) namic behaviour (exponentially long transients) of the devices in(cid:173) dicates the overloading of the associative memory. An implementa(cid:173) tion based on a programmable logic device is here presented. A 16  neurons circuit is implemented whit a XILINK 4020 device. The  peculiarity of this solution is the possibility to change parts of the  project (weights, transfer function or the whole architecture) with  a simple software download of the configuration into the XILINK  chip.
Metamorphosis Networks: An Alternative to Constructive Models
Authors: Brian V. Bonnlander, Michael Mozer
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Given a set oft raining examples, determining the appropriate num(cid:173) ber  of  free  parameters  is  a  challenging  problem.  Constructive  learning algorithms attempt to solve this problem automatically by  adding  hidden  units,  and  therefore  free  parameters,  during  learn(cid:173) ing.  We  explore  an  alternative  class  of algorithms-called  meta(cid:173) morphosis  algorithms-in  which  the  number  of units  is  fixed,  but  the number of free  parameters gradually increases  during learning.  The architecture we investigate is composed of RBF units on a lat(cid:173) tice,  which  imposes  flexible  constraints  on  the  parameters  of the  network.  Virtues  of this  approach  include  variable  subset  selec(cid:173) tion,  robust  parameter  selection,  multiresolution  processing,  and  interpolation  of sparse training  data.
A Hybrid Linear/Nonlinear Approach to Channel Equalization Problems
Authors: Wei-Tsih Lee, John Pearson
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Channel equalization problem is an important problem in high-speed  communications. The sequences of symbols transmitted are distorted by  neighboring symbols. Traditionally, the channel equalization problem is  considered as a channel-inversion operation. One problem of this  approach is that there is no direct correspondence between error proba(cid:173) bility and residual error produced by the channel inversion operation. In  this paper, the optimal equalizer design is formulated as a classification  problem. The optimal classifier can be constructed by Bayes decision  rule. In general it is nonlinear. An efficient hybrid linear/nonlinear  equalizer approach has been proposed to train the equalizer. The error  probability of new linear/nonlinear equalizer has been shown to be bet(cid:173) ter than a linear equalizer in an experimental channel.
Physiologically Based Speech Synthesis
Authors: Makoto Hirayama, Eric Vatikiotis-Bateson, Kiyoshi Honda, Yasuharu Koike, Mitsuo Kawato
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: This study demonstrates a  paradigm for  modeling speech  produc(cid:173) tion  based  on  neural  networks.  Using  physiological  data  from  speech  utterances,  a  neural  network  learns  the  forward  dynamics  relating  motor  commands  to  muscles  and  the  ensuing  articulator  behavior  that  allows  articulator trajectories  to be generated  from  motor commands constrained by phoneme input strings and global  performance parameters.  From these movement trajectories, a sec(cid:173) ond  neural  network generates  PARCOR parameters  that  are  then  used  to synthesize  the speech  acoustics.
Weight Space Probability Densities in Stochastic Learning: II. Transients and Basin Hopping Times
Authors: Genevieve B. Orr, Todd K. Leen
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: In  stochastic  learning,  weights  are  random  variables  whose  time  evolution  is  governed  by  a  Markov  process.  At  each  time-step,  n,  the weights  can be described by  a  probability density function  pew, n).  We summarize the theory of the time evolution of P, and  give  graphical  examples  of  the  time  evolution  that  contrast  the  behavior  of stochastic learning  with  true gradient  descent  (batch  learning).  Finally, we use the formalism to obtain predictions of the  time required for noise-induced hopping between basins of different  optima.  We  compare the theoretical predictions with simulations  of large  ensembles  of networks  for  simple  problems in  supervised  and unsupervised learning. 
1  Weight-Space Probability  Densities 
Despite the recent application of convergence theorems from stochastic approxima(cid:173) tion  theory to  neural  network  learning  (Oja 1982,  White  1989)  there remain  out(cid:173) standing questions about the search dynamics in stochastic learning.  For example,  the convergence theorems do  not  tell  us  to which  of several  optima the algorithm
Performance Through Consistency: MS-TDNN's for Large Vocabulary Continuous Speech Recognition
Authors: Joe Tebelskis, Alex Waibel
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: Connectionist Rpeech recognition systems are often handicapped by  an  inconsistency  between  training and  testing  criteria.  This  prob(cid:173) lem  is  addressed  by  the  Multi-State  Time Delay  Neural  Network  (MS-TDNN), a hierarchical phonf'mp and word classifier which uses  DTW  to  modulate  its  connectivit.y  pattern,  and  which  is  directly  trained  on  word-level  targets.  The  consistent  use  of word  accu(cid:173) racy  as  a  criterion  during  bot.h  t.raining  and  testing  leads  to  very  high  system  performance,  even  wif II  limited  training  dat.a.  Until  now,  the  MS-TDN N  has  been  appli('d  primarily  to  small  vocabu(cid:173) lary  recognition  and  word  spotting  tasks.  In  this  papf'f  we  apply  the architecture to large vocabulary continuous speech recognition,  and  demonstrate  that  our  MS-TDNN  outperforms  all  ot,hf'r  sys(cid:173) tems  that  have  been  tested  on  tht'  eMU  Conference  Registration  database.
Generic Analog Neural Computation - The EPSILON Chip
Authors: Stephen Churcher, Donald J. Baxter, Alister Hamilton, Alan F. Murray, H. Martin Reekie
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: An analog CMOS VLSI neural processing chip has been designed and fabri(cid:173) cated. The device employs "pulse-stream" neural state signalljng, and is capa(cid:173) ble of computing some 360 million synaptic connections per secood. In addi(cid:173) tion to basic characterisation results. the performance of the chip in solving  "real-world" problems is also demonstrated.
Analog VLSI Implementation of Multi-dimensional Gradient Descent
Authors: David B. Kirk, Douglas Kerns, Kurt Fleischer, Alan H. Barr
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: We describe an analog VLSI implementation of a multi-dimensional  gradient estimation and descent technique for minimizing an on(cid:173) chip scalar function fO. The implementation uses noise injec(cid:173) tion and multiplicative correlation to estimate derivatives, as in  [Anderson, Kerns 92]. One intended application of this technique  is setting circuit parameters on-chip automatically, rather than  manually [Kirk 91]. Gradient descent optimization may be used  to adjust synapse weights for a backpropagation or other on-chip  learning implementation. The approach combines the features of  continuous multi-dimensional gradient descent and the potential  for an annealing style of optimization. We present data measured  from our analog VLSI implementation.
Improving Performance in Neural Networks Using a Boosting Algorithm
Authors: Harris Drucker, Robert Schapire, Patrice Simard
Published in: Advances in Neural Information Processing Systems 5  (NIPS 1992) (Main Conference Track), 1992

Abstract: A boosting algorithm converts a learning machine with error rate less  than 50% to one with an arbitrarily low error rate. However, the  algorithm discussed here depends on having a large supply of  independent training samples. We show how to circumvent this  problem and generate an ensemble of learning machines whose  performance in optical character recognition problems is dramatically  improved over that of a single network. We report the effect of  boosting on four databases (all handwritten) consisting of 12,000 digits  from segmented ZIP codes from the United State Postal Service  (USPS) and the following from the National Institute of Standards and  Testing (NIST): 220,000 digits, 45,000 upper case alphas, and 45,000  lower case alphas. We use two performance measures: the raw error  rate (no rejects) and the reject rate required to achieve a 1% error rate  on the patterns not rejected. Boosting improved performance in some  cases by a factor of three.