mtybilly/Instruct2DS huggingface dataset:
A Multiscale Adaptive Network Model of Motion Computation in Primates
Authors: H. Taichi Wang, Bimal Mathur, Christof Koch
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  demonstrate  a  multiscale  adaptive  network  model  of motion  computation in primate area MT. The model consists of two stages:  (l)  local velocities are measured across multiple spatio-temporal channels,  and (2) the optical flow  field  is computed by a  network of direction(cid:173) selective neurons  at multiple  spatial  resolutions.  This model  embeds  the computational efficiency of Multigrid algorithms within a parallel  network as well as adaptively  computes  the most reliable estimate of  the flow  field across different spatial scales. Our model neurons show  the same nonclassical receptive field properties as Allman's type I MT  neurons.  Since local velocities are measured across multiple channels,  various  channels  often  provide  conflicting  measurements  to  the  network. We have incorporated a  veto scheme for conflict resolution.  This mechanism provides a novel explanation for the spatial frequency  dependency of the psychophysical phenomenon called Motion Capture. 
1  MOTIVATION  We previously developed a two-stage model of motion computation in the visual system  of primates  (Le.  magnocellular pathway from  retina to  V1  and  MT;  Wang,  Mathur &  Koch, 1989). This algorithm has these deficiencies:  (1) the issue of optimal spatial scale  for  velocity measurement, and (2)  the issue optimal spatial scale for  the smoothness of  motion field.  To address these deficiencies, we have implemented a multi-scale motion  network based on multigrid algorithms.  All  methods of estimating optical flow  make a basic assumption about the scale of the  velocity relative to the  spatial  neighborhood and to the  temporal  discretization step of  delay. Thus,  if the velocity of the pattern  is much larger than  the ratio of the spatial to  temporal  sampling step, an incorrect velocity  value will be obtained (Battiti, Amaldi  &  Koch,  1991). Battiti et al. proposed a coarse-to-fine strategy for adaptively detennining
Spherical Units as Dynamic Consequential Regions: Implications for Attention, Competition and Categorization
Authors: Stephen Jose Hanson, Mark A. Gluck
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: to construct dynamic 
Spherical Units can be used  reconfigurable  consequential regions, the geometric bases for Shepard's (1987) theory of  stimulus generalization in animals and humans. We derive from Shepard's  (1987) generalization theory a particular multi-layer network with dynamic  (centers and radii) spherical regions which possesses a specific mass function  (Cauchy). This learning model generalizes the configural-cue network model  (Gluck & Bower 1988): (1) configural cues can be learned and do not require  pre-wiring the power-set of cues, (2) Consequential regions are continuous  rather than discrete and (3) Competition amoungst receptive fields is shown  to be increased by the global extent of a particular mass function (Cauchy).  We compare other common mass functions (Gaussian; used in models of  Moody & Darken; 1989, Krushke, 1990) or just standard backpropogation  networks with hyperplane/logistic hidden units showing that neither fare as  well as models of human generalization and learning. 
1 The Generalization Problem 
Given a favorable or unfavorable consequence, what should an organism assume about  the contingent stimuli? If a moving shadow overhead appears prior to a hawk attack  what should an organism assume about other moving shadows, their shapes and  positions? If a dense food patch is occasioned by a particular density of certain kinds of  shrubbery what should the organism assume about other shurbbery, vegetation or its  spatial density? In an pattern recognition context, given a character of a certain shape,  orientation, noise level etc.. has been recognized correctly what should the system  assume about other shapes, orientations, noise levels it has yet to encounter? 
• Also a member of Cognitive Science Laboratory, Princeton University, Princeton, NJ 08544
Direct memory access using two cues: Finding the intersection of sets in a connectionist model
Authors: Janet Wiles, Michael S. Humphreys, John D. Bain, Simon Dennis
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: For lack of alternative models, search and decision processes have provided the  dominant paradigm for human memory access using two or more cues, despite  evidence against search as an access process (Humphreys, Wiles & Bain, 1990).  We present an alternative process to search, based on calculating the intersection  of sets of targets activated by two or more cues. Two methods of computing  the intersection are presented, one using information about the possible targets,  the other constraining the cue-target strengths in the memory matrix. Analysis  using orthogonal vectors to represent the cues and targets demonstrates the  competence of both processes, and simulations using sparse distributed  representations demonstrate the performance of the latter process for tasks  involving 2 and 3 cues.
Flight Control in the Dragonfly: A Neurobiological Simulation
Authors: William E. Faller, Marvin W. Luttges
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Neural network simulations of the dragonfly flight neurocontrol system  have  been  developed  to  understand  how  this  insect  uses  complex,  unsteady  aerodynamics.  The  simulation  networks  account  for  the  ganglionic  spatial  distribution  of  cells  as  well  as  the  physiologic  operating range and the stochastic cellular fIring history of each neuron.  In  addition  the  motor  neuron  firing  patterns,  "flight  command  sequences", were utilized. Simulation training was targeted against both  the  cellular  and  flight  motor  neuron  firing  patterns.  The  trained  networks  accurately  resynthesized  the  intraganglionic  cellular firing  patterns. These in  tum controlled the  motor neuron fIring patterns that  drive  wing  musculature  during  flight.  Such  networks  provide  both  neurobiological analysis tools and fIrst  generation controls for  the  use  of "unsteady" aerodynamics.
A Framework for the Cooperation of Learning Algorithms
Authors: Léon Bottou, Patrick Gallinari
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We introduce a framework  for  training architectures composed of several  modules. This framework,  which  uses a statistical formulation  of learning  systems,  provides  a  unique  formalism  for  describing  many  classical  connectionist  algorithms  as  well  as  complex  systems  where  several  algorithms interact. It allows to design hybrid systems which combine the  advantages of connectionist algorithms as well as other learning algorithms.
Continuous Speech Recognition by Linked Predictive Neural Networks
Authors: Joe Tebelskis, Alex Waibel, Bojan Petek, Otto Schmidbauer
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We present a large vocabulary, continuous speech recognition system based  on  Linked  Predictive  Neural  Networks  (LPNN's).  The system  uses  neu(cid:173) ral  networks  as  predictors  of speech  frames,  yielding  distortion  measures  which  are  used  by  the  One Stage DTW algorithm to perform  continuous  speech  recognition.  The system,  already  deployed  in  a  Speech  to Speech  Translation system, currently achieves 95%,  58%,  and 39% word accuracy  on  tasks  with  perplexity  5,  111,  and  402  respectively,  outperforming sev(cid:173) eral simple HMMs  that  we  tested.  We  also  found  that  the  accuracy  and  speed of the LPNN can be slightly improved by the judicious use of hidden  control  inputs.  We  conclude  by  discussing  the  strengths  and  weaknesses  of the predictive approach.
Statistical Mechanics of Temporal Association in Neural Networks
Authors: Andreas V. M. Herz, Zhaoping Li, J. Leo van Hemmen
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We study the representation of static patterns and temporal associa(cid:173) tions in  neural networks with  a  broad  distribution of signal  delays.  For a certain class of such systems, a simple intuitive understanding  of the spatia-temporal computation becomes possible with the help  of a  novel  Lyapunov  functional. It allows  a  quantitative  study  of  the  asymptotic  network  behavior  through  a  statistical  mechanical  analysis. We  present  analytic  calculations of both retrieval  quality  and storage capacity and compare  them with  simulation results.
Cholinergic Modulation May Enhance Cortical Associative Memory Function
Authors: Michael E. Hasselmo, Brooke P. Anderson, James M. Bower
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Combining neuropharmacological experiments with computational model(cid:173) ing, we have shown that cholinergic modulation may enhance associative  memory function in piriform (olfactory) cortex. We have shown that the  acetylcholine analogue carbachol selectively suppresses synaptic transmis(cid:173) sion between cells within piriform cortex, while leaving input connections  unaffected. When tested in a computational model of piriform cortex,  this selective suppression, applied during learning, enhances associative  memory performance.
Evolution and Learning in Neural Networks: The Number and Distribution of Learning Trials Affect the Rate of Evolution
Authors: Ron Keesing, David G. Stork
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: and 
*Dept. of Electrical Engineering 
Stanford University  Stanford, CA 94305 
stork@psych.stanford.edu 
Learning can increase the rate of evolution of a population of  biological organisms (the Baldwin effect). Our simulations  show that in a population of artificial neural networks  solving a pattern recognition problem, no learning or too  much learning leads to slow evolution of the genes whereas  an intermediate amount is optimal. Moreover, for a given  total number of training presentations, fastest evoution  occurs if different individuals within each generation receive  different numbers of presentations, rather than equal  numbers. Because genetic algorithms (GAs) help avoid  local minima in energy functions, our hybrid learning-GA  systems can be applied successfully to complex, high(cid:173) dimensional pattern recognition problems.
A Lagrangian Approach to Fixed Points
Authors: Eric Mjolsness, Willard L. Miranker
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  present  a  new  way  to  derive  dissipative,  optimizing  dynamics  from  the  Lagrangian formulation of mechanics.  It can  be  used  to  obtain  both  standard  and  novel  neural  net  dynamics  for  optimization  problems.  To  demonstrate this we  derive standard descent  dynamics as well as nonstan(cid:173) dard variants that introduce  a  computational attention mechanism.
Neural Network Implementation of Admission Control
Authors: Rodolfo A. Milito, Isabelle Guyon, Sara A. Solla
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A feedforward layered network implements a mapping required to control an  unknown stochastic nonlinear dynamical system. Training is based on a  novel approach that combines stochastic approximation ideas with back(cid:173) propagation. The method is applied to control admission into a queueing sys(cid:173) tem operating in a time-varying environment.
VLSI Implementation of TInMANN
Authors: Matt Melton, Tan Phan, Doug Reeves, Dave Van den Bout
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A massively parallel, all-digital, stochastic architecture - TlnMAN N - is  described which performs competitive and Kohonen types of learning. A  VLSI design is shown for a TlnMANN neuron which fits within a small,  inexpensive MOSIS TinyChip frame, yet which can be used to build larger  networks of several hundred neurons. The neuron operates at a speed of  15 MHz which allows the network to process 290,000 training examples  per second. Use of level sensitive scan logic provides the chip with 100%  fault coverage, permitting very reliable neural systems to be built.
SEXNET: A Neural Network Identifies Sex From Human Faces
Authors: B.A. Golomb, D.T. Lawrence, T.J. Sejnowski
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Sex identification in animals has biological importance. Humans are good  at making this determination visually, but machines have not matched  this ability. A neural network was trained to discriminate sex in human  faces, and performed as well as humans on a set of 90 exemplars. Images  sampled at 30x30 were compressed using a 900x40x900 fully-connected  back-propagation network; activities of hidden units served as input to a  back-propagation "SexNet" trained to produce values of 1 for male and  o for female faces. The network's average error rate of 8.1% compared  favorably to humans, who averaged 11.6%. Some SexNet errors mimicked  those of humans.
From Speech Recognition to Spoken Language Understanding: The Development of the MIT SUMMIT and VOYAGER Systems
Authors: Victor Zue, James Glass, David Goodine, Lynette Hirschman, Hong Leung, Michael Phillips, Joseph Polifroni, Stephanie Seneff
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Spoken language is one of the most natural, efficient, flexible, and econom(cid:173) ical means of communication among humans. As computers play an ever  increasing role in our lives, it is important that we address the issue of  providing a graceful human-machine interface through spoken language.  In this paper, we will describe our recent efforts in moving beyond the  scope of speech recognition into the realm of spoken-language understand(cid:173) ing. Specifically, we report on the development of an urban navigation and  exploration system called VOYAGER, an application which we have used as  a basis for performing research in spoken-language understanding.
Generalization by Weight-Elimination with Application to Forecasting
Authors: Andreas S. Weigend, David E. Rumelhart, Bernardo A. Huberman
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Inspired by the information theoretic idea of minimum description length, we add  a term  to the back propagation cost function that penalizes network complexity.  We  give  the  details  of the  procedure,  called  weight-elimination,  describe  its  dynamics, and clarify the meaning of the parameters involved. From a Bayesian  perspective,  the complexity term  can  be usefully interpreted as  an  assumption  about prior distribution of the weights.  We  use  this  procedure  to  predict  the  sunspot time series and the notoriously noisy series of currency exchange rates.
A Novel Approach to Prediction of the 3-Dimensional Structures of Protein Backbones by Neural Networks
Authors: Henrik Fredholm, Henrik Bohr, Jakob Bohr, Søren Brunak, Rodney M. J. Cotterill, Benny Lautrup, Steffen B. Petersen
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Three-dimensional (3D) structures of protein backbones have been pre(cid:173) dicted using neural networks. A feed forward neural network was trained  on a class of functionally, but not structurally, homologous proteins, us(cid:173) ing backpropagation learning. The network generated tertiary structure  information in the form of binary distance constraints for the Co atoms  in the protein backbone. The binary distance between two Co atoms was  o if the distance between them was less than a certain threshold distance,  and 1 otherwise. The distance constraints predicted by the trained neu(cid:173) ral network were utilized to generate a folded conformation of the protein  backbone, using a steepest descent minimization approach.
Asymptotic slowing down of the nearest-neighbor classifier
Authors: Robert R. Snapp, Demetri Psaltis, Santosh S. Venkatesh
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: If patterns are drawn from an n-dimensional feature space according to a  probability distribution that obeys a weak smoothness criterion, we show  that the probability that a random input pattern is misclassified by a  nearest-neighbor classifier using M random reference patterns asymptoti(cid:173) cally satisfies 
PM(error) "" Poo(error) + M2/n' 
a 
for sufficiently large values of M. Here, Poo(error) denotes the probability  of error in the infinite sample limit, and is at most twice the error of a  Bayes classifier. Although the value of the coefficient a depends upon the  underlying probability distributions, the exponent of M is largely distri(cid:173) bution free. We thus obtain a concise relation between a classifier's ability  to generalize from a finite reference sample and the dimensionality of the  feature space, as well as an analytic validation of Bellman's well known  "curse of dimensionality."
Connectionist Approaches to the Use of Markov Models for Speech Recognition
Authors: Hervé Bourlard, Nelson Morgan, Chuck Wooters
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Previous  work  has  shown  the  ability  of  Multilayer  Perceptrons  (MLPs) to estimate emission probabilities for Hidden Markov Mod(cid:173) els  (HMMs).  The advantages of a  speech recognition system incor(cid:173) porating  both  MLPs  and  HMMs  are  the  best  discrimination  and  the  ability  to  incorporate  multiple  sources  of evidence  (features,  temporal context) without restrictive assumptions of distributions  or  statistical  independence.  This  paper  presents  results  on  the  speaker-dependent portion of DARPA's English language Resource  Management  database.  Results  support  the  previously  reported  utility of MLP probability estimation for  continuous speech recog(cid:173) nition.  An additional approach we  are pursuing is to use  MLPs as  nonlinear predictors for autoregressive HMMs.  While this is shown  to  be  more  compatible  with  the  HMM  formalism,  it  still  suffers  from several limitations.  This approach  is  generalized  to  take ac(cid:173) count of time correlation between successive observations, without  any restrictive assumptions about the driving noise.
A Second-Order Translation, Rotation and Scale Invariant Neural Network
Authors: Shelly D. D. Goggin, Kristina M. Johnson, Karl E. Gustafson
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A second-order architecture is presented  here for  translation, rotation and  scale  invariant processing  of 2-D  images  mapped  to  n  input  units.  This  new architecture has a complexity of O( n) weights as opposed to the O( n 3 )  weights usually required  for  a  third-order,  rotation invariant architecture.  The reduction  in complexity is due  to the use  of discrete  frequency  infor(cid:173) mation.  Simulations show favorable  comparisons to other neural  network  architectures.
Lg Depth Estimation and Ripple Fire Characterization Using Artificial Neural Networks
Authors: John L. Perry, Douglas R. Baumgardt
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This srudy has demonstrated how artificial neural networks (ANNs) can  be used to characterize seismic sources using high-frequency regional  seismic data. We have taken the novel approach of using ANNs as a  research tool for obtaining seismic source information, specifically  depth of focus for earthquakes and ripple-fire characteristics for  economic blasts, rather than as just a feature classifier between  earthquake and explosion populations. Overall, we have found that  ANNs have potential applications to seismic event characterization and  identification, beyond just as a feature classifier. In future studies, these  techniques should be applied to actual data of regional seismic events  recorded at the new regional seismic arrays. The results of this study  indicates that an ANN should be evaluated as part of an operational  seismic event identification system.
Adaptive Range Coding
Authors: Bruce E. Rosen, James M. Goodwin, Jacques J. Vidal
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: these 
to  neuron-like  processing  elements.  "neurons" 
This  paper  examines  a  class  of  neuron  based  that  rely  on  learning  systems  for  dynamic  control  adaptive  range  coding  of  sensor  inputs.  Sensors  are  assumed  to  provide  binary  coded  range  vectors  that  coarsely  describe  the  system  state.  These  vectors  are  Output  input  decisions  generated  by  turn  the  system  state,  subsequently  producing  new  affect  inputs.  the  intervals  and  environment  are  evaluated.  The  neural  weights  as  well  as  the  ran g e  b 0  u n dar i e s  determining  the  output  decisions  are  then  altered  with  future  Preliminary  reinforcement  from  the  promise  of  adapting  "neural  experiments  show  receptive  learning  dynamical  control.  The  observed  performance  with  this  method  exceeds  that  of  earlier  approaches. 
the  goal  of  maximizing  the  environment.
RecNorm: Simultaneous Normalisation and Classification applied to Speech Recognition
Authors: John S. Bridle, Stephen J. Cox
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A particular form of neural network is described, which has terminals  for acoustic patterns, class labels and speaker parameters. A method of  training this network to "tune in" the speaker parameters to a particular  speaker is outlined, based on a trick for converting a supervised network  to an unsupervised mode. We describe experiments using this approach  in isolated word recognition based on whole-word hidden Markov models.  The results indicate an improvement over speaker-independent perfor(cid:173) mance and, for unlabelled data, a performance close to that achieved on  labelled data.
A four neuron circuit accounts for change sensitive inhibition in salamander retina
Authors: Jeffrey L. Teeters, Frank H. Eeckman, Frank S. Werblin
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In salamander retina, the response of On-Off ganglion cells to a central  flash is reduced by movement in the receptive field surround. Through  computer simulation of a 2-D model which takes into account their  anatomical and physiological properties, we show that interactions  between four neuron types (two bipolar and two amacrine) may be  responsible for the generation and lateral conductance of this change  sensitive inhibition. The model shows that the four neuron circuit can  account for previously observed movement sensitive reductions in  ganglion cell sensitivity and allows visualization and prediction of the  spatio-temporal pattern of activity in change sensitive retinal cells.
A Method for the Efficient Design of Boltzmann Machines for Classiffication Problems
Authors: Ajay Gupta, Wolfgang Maass
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We introduce a method for the efficient design of a Boltzmann machine (or  a Hopfield net) that computes an arbitrary given Boolean function f . This  method is based on an efficient simulation of acyclic circuits with threshold  gates by Boltzmann machines. As a consequence we can show that various  concrete Boolean functions f that are relevant for classification problems  can be computed by scalable Boltzmann machines that are guaranteed  to converge to their global maximum configuration with high probability  after constantly many steps.
How Receptive Field Parameters Affect Neural Learning
Authors: Bartlett W. Mel, Stephen M. Omohundro
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  identify  the three  principle factors  affecting  the  performance of learn(cid:173) ing by  networks  with  localized  units:  unit noise,  sample density,  and  the  structure of the target function.  We then  analyze the effect  of unit recep(cid:173) tive  field  parameters  on  these  factors  and  use  this  analysis  to  propose  a  new  learning algorithm which  dynamically alters receptive field  properties  during learning. 
1  LEARNING WITH LOCALIZED RECEPTIVE FIELDS 
Locally-tuned  representations  are  common in  both  biological  and  artificial  neural  networks.  Several workers have analyzed the effect of receptive field  size, shape, and  overlap  on  representation  accuracy:  (Baldi,  1988),  (Ballard,  1987),  and  (Hinton,  1986).  This paper investigates the additional interactions introduced by  the task of  function learning.  Previous studies which have considered learning have for the most  part restricted attention to the use of the input probability distribution to determine  receptive field  layout (Kohonen,  1984)  and (Moody and Darken,  1989).  We  will see  that the structure of the function  being learned may also  be  advantageously taken  into account. 
Function learning using  radial basis functions  (RBF's)  is  currently  a  popular tech(cid:173) nique  (Broomhead  and  Lowe,  1988)  and  serves  as  an  adequate framework  for  our  discussion.  Because  we  are interested  in constraints on biological systems,  we  must  explictly  consider  the  effects  of unit  noise.  The  goal  is  to  choose  the  layout  of  receptive fields  so as  to minimize average performance error.  Let  y  = f(x)  be  the  function  the  network  is  attempting  to  learn  from  example
Proximity Effect Corrections in Electron Beam Lithography Using a Neural Network
Authors: Robert C. Frye, Kevin D. Cummings, Edward A. Rietman
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We have used a neural network to compute corrections for images written  by electron beams to eliminate the proximity effects caused by electron  Iterative methods are effective. but require prohibitively  scattering.  computation time. We have instead trained a neural network to perform  equivalent corrections. resulting in a significant speed-up. We have  examined hardware  implementations using both analog and digital  electronic networks. Both had an acceptably small error of 0.5% compared  to the iterative results. Additionally. we verified that the neural network  correctly generalized the solution of the problem to include patterns not  contained in its training set. We have experimentally verified this approach  on a Cambridge Instruments EBMF 10.5 exposure system.
Exploiting Syllable Structure in a Connectionist Phonology Model
Authors: David S. Touretzky, Deirdre W. Wheeler
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In  a  previous  paper  (Touretzky  &  Wheeler,  1990a) we showed  how  adding  a  clustering operation to a connectionist phonology model produced a parallel pro(cid:173) cessing account of certain "iterative" phenomena.  In this paper we show how the  addition of a second structuring primitive, syllabification, greatly increases  the  power of the model.  We present examples from  a non-Indo-European language  that appear to require rule ordering to at least a depth of four.  By adding syllab(cid:173) ification circuitry to structure the model's perception of the input string, we are  able to handle these examples with only two derivational steps.  We conclude that  in phonology, derivation can be largely replaced by structuring.
Analog Computation at a Critical Point: A Novel Function for Neuronal Oscillations?
Authors: Leonid Kruglyak, William Bialek
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: \Ve show that a simple spin system bia.sed at its critical point can en(cid:173) code spatial characteristics of external signals, sHch as the dimensions of  "objects" in the visual field. in the temporal correlation functions of indi(cid:173) vidual spins. Qualit.ative arguments suggest that regularly firing neurons  should be described by a planar spin of unit lengt.h. and such XY models  exhibit critical dynamics over a broad range of parameters. \Ve show how  to extract these spins from spike trains and then mea'3ure t.he interaction  Hamilt.onian using simulations of small dusters of cells. Static correla(cid:173) tions among spike trains obtained from simulations of large arrays of cells  are in agreement with the predictions from these Hamiltonians, and dy(cid:173) namic correlat.ions display the predicted encoding of spatial information.  \Ve suggest that this novel representation of object dinwnsions in temporal  correlations may be relevant t.o recent experiment.s on oscillatory neural  firing in the visual cortex.
Integrated Segmentation and Recognition of Hand-Printed Numerals
Authors: James D. Keeler, David E. Rumelhart, Wee Kheng Leow
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Neural  network  algorithms  have  proven  useful  for  recognition  of individ(cid:173) ual,  segmented  characters.  However,  their recognition  accuracy  has  been  limited by  the  accuracy  of the  underlying  segmentation  algorithm.  Con(cid:173) ventional,  rule-based  segmentation  algorithms  encounter  difficulty  if the  characters  are touching, broken,  or noisy.  The problem in these situations  is  that  often  one  cannot  properly  segment  a  character  until  it  is  recog(cid:173) nized yet  one cannot  properly recognize  a  character until it is  segmented.  We present here  a  neural network algorithm that simultaneously segments  and recognizes  in an  integrated  system.  This  algorithm has several  novel  features:  it uses  a supervised learning algorithm (backpropagation), but is  able to take position-independent information as  targets and self-organize  the  activities  of the units  in  a  competitive fashion  to infer the  positional  information.  We  demonstrate  this  ability with  overlapping  hand-printed  numerals.
Speech Recognition Using Demi-Syllable Neural Prediction Model
Authors: Ken-ichi Iso, Takao Watanabe
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: The  Neural  Prediction  Model  is  the  speech  recognition  model  based  on  pattern  prediction  by  multilayer  perceptrons.  Its  effectiveness  was  con(cid:173) firmed  by  the  speaker-independent  digit  recognition  experiments.  This  paper  presents  an  improvement  in  the model  and  its  application  to  large  vocabulary speech recognition,  based on subword units.  The improvement  involves an introduction  of "backward  prediction,"  which further  improves  the  prediction  accuracy  of the  original  model  with  only  "forward  predic(cid:173) tion".  In  application of the model  to  speaker-dependent large vocabulary  speech recognition,  the demi-syllable unit is  used  as  a subword recognition  unit.  Experimental  results  indicated  a  95.2%  recognition  accuracy  for  a  5000  word  test  set  and  the  effectiveness  was  confirmed  for  the  proposed  model  improvement and  the demi-syllable subword units.
The Devil and the Network: What Sparsity Implies to Robustness and Memory
Authors: Sanjay Biswas, Santosh S. Venkatesh
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Robustness is a commonly bruited property of neural networks; in particu(cid:173) lar, a folk theorem in neural computation asserts that neural networks-in  contexts with large interconnectivity-continue to function efficiently, al(cid:173) beit with some degradation, in the presence of component damage or loss.  A second folk theorem in such contexts asserts that dense interconnectiv(cid:173) ity between neural elements is a sine qua non for the efficient usage of  resources. These premises are formally examined in this communication  in a setting that invokes the notion of the "devil" 1 in the network as an  agent that produces sparsity by snipping connections. 
1 ON REMOVING THE FOLK FROM THE THEOREM 
Robustness in the presence of component damage is a property that is commonly  attributed to neural networks. The content of the following statement embodies  this sentiment. 
Folk Theorem 1: Computation in neural networks is not substantially  affected by damage to network components. 
While such a statement is manifestly not true in general-witness networks with  "grandmother cells" where damage to the critical cells fatally impairs the com(cid:173) putational ability of the network-there is anecdotal evidence in support of it in 
1 Well, maybe an imp.
Development and Spatial Structure of Cortical Feature Maps: A Model Study
Authors: Klaus Obermayer, Helge Ritter, Klaus Schulten
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: K.  Schulten  Beckman -Insti t u te  University  of Illinois  Urbana, IL  61801 
Feature selective cells in  the primary visual cortex of several species are or(cid:173) ganized in hierarchical topographic maps of stimulus features like  "position  in  visual  space",  "orientation"  and" ocular  dominance".  In  order  to  un(cid:173) derstand and describe their spatial structure and their development, we  in(cid:173) vestigate a self-organizing neural network model based on the feature map  algorithm.  The  model  explains  map  formation  as  a  dimension-reducing  mapping  from  a  high-dimensional  feature  space  onto  a  two-dimensional  lattice,  such  that "similarity"  between features  (or feature  combinations)  is  translated  into  "spatial  proximity"  between  the  corresponding  feature  selective cells.  The model is  able to reproduce several aspects of the spatial  structure of cortical maps  in  the visual  cortex.
Connectionist Implementation of a Theory of Generalization
Authors: Roger N. Shepard, Sheila Kannappan
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Empirically,  generalization between  a  training  and  a  test  stimulus  falls  off in  close approximation to  an  exponential decay  function  of distance between  the  two stimuli in the "stimulus space" obtained by multidimensional scaling.  Math(cid:173) ematically, this result is derivable from  the assumption that an  individual takes  the  training  stimulus  to  belong  to  a  "consequential"  region  that includes  that  stimulus but is  otherwise of unknown  location, size, and  shape in the stimulus  space (Shepard, 1987).  As the individual gains additional information about the  consequential region-by finding other stimuli to be consequential or nOl-the  theory  predicts  the shape  of the generalization  function  to  change  toward  the  function relating actual probability of the consequence to location in the stimulus  space.  This paper describes a natural connectionist implementation of the theory,  and illustrates how implications of the theory for generalization, discrimination,  and classification learning can be explored by connectionist simulation. 
1  THE THEORY OF GENERALIZATION 
Because we never confront exactly the same situation twice, anything we have learned in  any previous situation can guide us in deciding which action to take in the present situation  only  to  the  extent  that  the  similarity  between  the  two  situations  is  sufficient  to  justify  generalization of our previous learning to the present situation.  Accordingly, principles of  generalization must be foundational for any theory of behavior. 
In  Shepard  (1987)  nonarbitrary principles  of generalization  were  sought that  would  be  optimum in any world in which an object, however distinct from other objects, is generally  a  member of some class or natural kind  sharing some dispositional property of potential  consequence for the individual.  A newly encountered plant or animal  might be edible or
Discrete Affine Wavelet Transforms For Anaylsis And Synthesis Of Feedfoward Neural Networks
Authors: Y. C. Pati, P. S. Krishnaprasad
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In this paper we show that discrete affine wavelet transforms can provide  a tool for the analysis and synthesis of standard feedforward neural net(cid:173) works. It is shown that wavelet frames for L2(IR) can be constructed based  upon sigmoids. The spatia-spectral localization property of wavelets can  be exploited in defining the topology and determining the weights of a  feedforward network. Training a network constructed using the synthe(cid:173) sis procedure described here involves minimization of a convex cost func(cid:173) tional and therefore avoids pitfalls inherent in standard backpropagation  algorithms. Extension of these methods to L2(IRN) is also discussed.
Convergence of a Neural Network Classifier
Authors: John S. Baras, Anthony LaVigna
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In  this  paper,  we  prove  that  the  vectors  in  the  LVQ  learning  algorithm  converge.  We  do  this  by  showing  that  the  learning  algorithm  performs  stochastic  approximation.  Convergence  is  then  obtained  by  identifying  the  appropriate  conditions  on  the  learning  rate  and  on  the  underlying  statistics of the  classification  problem.  We  also  present  a  modification  to  the  learning algorithm which  we  argue  results  in  convergence  of the  LVQ  error to the Bayesian optimal error as  the appropriate parameters become  large.
Optimal Sampling of Natural Images: A Design Principle for the Visual System
Authors: William Bialek, Daniel L. Ruderman, A. Zee
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We formulate the problem of optimizing the sampling of natural images  using an array of linear filters. Optimization of information capacity is  constrained by the noise levels of the individual channels and by a penalty  for the construction of long-range interconnections in the array. At low  signal-to-noise ratios the optimal filter characteristics correspond to bound  states of a Schrodinger equation in which the signal spectrum plays the  role of the potential. The resulting optimal filters are remarkably similar  to those observed in the mammalian visual cortex and the retinal ganglion  cells of lower vertebrates. The observed scale invariance of natural images  plays an essential role in this construction.
A competitive modular connectionist architecture
Authors: Robert A. Jacobs, Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We describe a multi-network, or modular, connectionist architecture that  captures that fact that many tasks have structure at a level of granularity  intermediate to that assumed by local and global function approximation  schemes. The main innovation of the architecture is that it combines  associative and competitive learning in order to learn task decompositions.  A task decomposition is discovered by forcing the networks comprising the  architecture to compete to learn the training patterns. As a result of the  competition, different networks learn different training patterns and, thus,  learn to partition the input space. The performance of the architecture on  a "what" and "where" vision task and on a multi-payload robotics task  are presented.
A Model of Distributed Sensorimotor Control in the Cockroach Escape Turn
Authors: R.D. Beer, G. J. Kacmarcik, R.E. Ritzmann, H.J. Chiel
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In response to a puff of wind, the American cockroach turns away and runs.  The circuit underlying the initial turn of this escape response consists of  three populations of individually identifiable nerve cells and appears to em(cid:173) ploy distributed representations in its operation. We have reconstructed  several neuronal and behavioral properties of this system using simplified  neural network models and the backpropagation learning algorithm con(cid:173) strained by known structural characteristics of the circuitry. In order to  test and refine the model, we have also compared the model's responses to  various lesions with the insect's responses to similar lesions.
A VLSI Neural Network for Color Constancy
Authors: Andrew W. Moore, John Allman, Geoffrey Fox, Rodney Goodman
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A system for color correction has been designed, built, and tested suc(cid:173) cessfully; the essential components are three custom chips built using sub(cid:173) threshold analog CMOS VLSI. The system, based on Land's Retinex the(cid:173) ory of color constancy, produces colors similar in many respects to those  produced by the visual system. Resistive grids implemented in analog  VLSI perform the smoothing operation central to the algorithm at video  rates. With the electronic system, the strengths and weaknesses of the  algorithm are explored. 
1 A MODEL FOR COLOR CONSTANCY 
Humans have the remarkable ability to perceive object colors as roughly constant  even if the color of the illumination is varied widely. Edwin Land, founder of the  Polaroid Corporation, models the computation that results in this ability as three  identical center-surround operations performed independently in three color planes,  such as red, green, and blue (Land, 1986). The basis for this model is as follows. 
Consider first an array of grey papers with different reflectances. (Land designated  these arrays Mondrians, since they resemble the works of the Dutch painter Piet 
·Present address: Dept. of Physics, Syracuse University, Syracuse, NY 13244
Stereopsis by a Neural Network Which Learns the Constraints
Authors: Alireza Khotanzad, Ying-Wung Lee
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This paper presents a neural network (NN) approach to the problem of  stereopsis. The correspondence problem (finding the correct matches  between the pixels of the epipolar lines of the stereo pair from amongst all  the possible matches) is posed as a non-iterative many-to-one mapping . A  two-layer feed forward NN architecture is developed to learn and code this  nonlinear and complex mapping using the back-propagation learning rule  and a training set. The important aspect of this technique is that none of  the typical constraints such as uniqueness and continuity are explicitly  imposed. All the applicable constraints are learned and internally coded  by the NN enabling it to be more flexible and more accurate than the  existing methods. The approach is successfully tested on several random(cid:173) dot stereograms. It is shown that the net can generalize its learned map(cid:173) ping to cases outside its training set. Advantages over the Marr-Poggio  Algorithm are discussed and it is shown that the NN performance is supe(cid:173) rIOr.
The Tempo 2 Algorithm: Adjusting Time-Delays By Supervised Learning
Authors: Ulrich Bodenhausen, Alex Waibel
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In this work we describe a new method that adjusts time-delays and the widths of  time-windows in artificial neural networks automatically.  The input of the units  are weighted by a gaussian input-window over time which allows the learning  rules for the delays and widths to be derived in the same way as it is used for the  weights.  Our results on a phoneme classification task compare well with results  obtained with the TDNN by Waibel et al., which was manually optimized for the  same task.
Learning Theory and Experiments with Competitive Networks
Authors: Griff L. Bilbro, David E. van den Bout
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  apply  the  theory  of Tishby,  Levin,  and Sol1a  (TLS)  to two problems.  First  we analyze  an elementary problem for  which we find  the predictions  consistent  with  conventional  statistical  results.  Second  we  numerically  examine  the  more realistic  problem of training a  competitive net  to learn  a  probability  density  from  samples.  We  find  TLS  useful  for  predicting  average  training  behavior. 
. 
1  TLS  APPLIED  TO  LEARNING DENSITIES 
Recently  a  theory  of learning  has  been  constructed  which  describes  the  learning  of a  relation  from  examples  (Tishby, Levin,  and Sol1a,  1989),  (Schwarb, Samalan,  Sol1a,  and Denker,  1990).  The original derivation  relies  on  a  statistical  mechanics  treatment  of the  probability  of independent  events  in  a  system  with  a  specified  average value of an additive error function. 
The resulting  theory is  not restricted  to learning  relations  and it is  not  essentially  statistical  mechanical.  The TLS  theory  can be derived  from  the  principle  of maz(cid:173) imum  entropy,  a  general  inference  tool which  produces  probabilities  characterized  by certain values of the averages of specified functions(Jaynes,  1979).  A TLS theory  can be  constructed  whenever  the specified  function  is  additive and associated  with  independent examples.  In this paper we treat the problem of learning a  probability  density from samples.  Consider the model as some function p( z Iw) of fixed form and adjustable parameters  w  which are  to be chosen  to approximate 1'(z)  where  the overline denotes  the  true  density.  All we  know about l' are the elements of a  training set T  which are drawn
Further Studies of a Model for the Development and Regeneration of Eye-Brain Maps
Authors: Jack D. Cowan, A. E. Friedman
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We describe a computational model of the development and regenera(cid:173) tion of specific eye-brain circuits. The model comprises a self-organiz(cid:173) ing map-forming network which uses local Hebb rules, constrained by  (genetically determined) molecular markers. Various simulations of  the development and regeneration of eye-brain maps in fish and frogs  are described, in particular successful simulations of experiments by  Schmidt-Cicerone-Easter; Meyer; and Y oon.