mtybilly/Instruct2DS huggingface dataset:
Neural Network Application to Diagnostics and Control of Vehicle Control Systems
Authors: Kenneth A. Marko
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Diagnosis of faults in complex, real-time control systems is a  complicated task that has resisted solution by traditional methods. We  have shown that neural networks can be successfully employed to  diagnose faults in digitally controlled powertrain systems. This paper  discusses the means we use to develop the appropriate databases for  training and testing in order to select the optimum network architectures  and to provide reasonable estimates of the classification accuracy of  these networks on new samples of data. Recent work applying neural  nets to adaptive control of an active suspension system is presented.
Evolution and Learning in Neural Networks: The Number and Distribution of Learning Trials Affect the Rate of Evolution
Authors: David G. Stork Ron Keesing
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: and 
*Dept. of Electrical Engineering 
Stanford University  Stanford, CA 94305 
stork@psych.stanford.edu 
Learning can increase the rate of evolution of a population of  biological organisms (the Baldwin effect). Our simulations  show that in a population of artificial neural networks  solving a pattern recognition problem, no learning or too  much learning leads to slow evolution of the genes whereas  an intermediate amount is optimal. Moreover, for a given  total number of training presentations, fastest evoution  occurs if different individuals within each generation receive  different numbers of presentations, rather than equal  numbers. Because genetic algorithms (GAs) help avoid  local minima in energy functions, our hybrid learning-GA  systems can be applied successfully to complex, high(cid:173) dimensional pattern recognition problems.
A Lagrangian Approach to Fixed Points
Authors: Willard L. Miranker Eric Mjolsness
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  present  a  new  way  to  derive  dissipative,  optimizing  dynamics  from  the  Lagrangian formulation of mechanics.  It can  be  used  to  obtain  both  standard  and  novel  neural  net  dynamics  for  optimization  problems.  To  demonstrate this we  derive standard descent  dynamics as well as nonstan(cid:173) dard variants that introduce  a  computational attention mechanism.
Neural Network Implementation of Admission Control
Authors: Isabelle Guyon, Sara A. Solla Rodolfo A. Milito
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A feedforward layered network implements a mapping required to control an  unknown stochastic nonlinear dynamical system. Training is based on a  novel approach that combines stochastic approximation ideas with back(cid:173) propagation. The method is applied to control admission into a queueing sys(cid:173) tem operating in a time-varying environment.
Computing with Arrays of Bell-Shaped and Sigmoid Functions
Authors: Pierre Baldi
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We consider feed-forward neural networks with one non-linear hidden layer  and  linear  output units.  The transfer  function  in the hidden  layer  are  ei(cid:173) ther  bell-shaped  or sigmoid.  In the bell-shaped  case,  we  show  how  Bern(cid:173) stein polynomials on one hand and the theory of the heat equation on the  other  are  relevant  for  understanding the  properties  of the  corresponding  networks.  In  particular,  these  techniques  yield  simple proofs of universal  approximation properties, i.e.  of the fact that any reasonable function can  be approximated to any degree of precision by a linear combination of bell(cid:173) shaped functions.  In addition, in  this  framework  the  problem of learning  is equivalent to the problem of reversing the time course of a diffusion pro(cid:173) cess.  The  results  obtained in  the bell-shaped  case  can  then  be  applied  to  the case  of sigmoid  transfer functions  in the hidden layer,  yielding similar  universality  results.  A  conjecture  related  to the problem of generalization  is  briefly  examined.
VLSI Implementation of TInMANN
Authors: Tan Phan, Doug Reeves, Dave Van den Bout Matt Melton
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A massively parallel, all-digital, stochastic architecture - TlnMAN N - is  described which performs competitive and Kohonen types of learning. A  VLSI design is shown for a TlnMANN neuron which fits within a small,  inexpensive MOSIS TinyChip frame, yet which can be used to build larger  networks of several hundred neurons. The neuron operates at a speed of  15 MHz which allows the network to process 290,000 training examples  per second. Use of level sensitive scan logic provides the chip with 100%  fault coverage, permitting very reliable neural systems to be built.
SEXNET: A Neural Network Identifies Sex From Human Faces
Authors: D.T. Lawrence, T.J. Sejnowski B.A. Golomb
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Sex identification in animals has biological importance. Humans are good  at making this determination visually, but machines have not matched  this ability. A neural network was trained to discriminate sex in human  faces, and performed as well as humans on a set of 90 exemplars. Images  sampled at 30x30 were compressed using a 900x40x900 fully-connected  back-propagation network; activities of hidden units served as input to a  back-propagation "SexNet" trained to produce values of 1 for male and  o for female faces. The network's average error rate of 8.1% compared  favorably to humans, who averaged 11.6%. Some SexNet errors mimicked  those of humans.
From Speech Recognition to Spoken Language Understanding: The Development of the MIT SUMMIT and VOYAGER Systems
Authors: James Glass, David Goodine, Lynette Hirschman, Hong Leung, Michael Phillips, Joseph Polifroni, Stephanie Seneff Victor Zue
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Spoken language is one of the most natural, efficient, flexible, and econom(cid:173) ical means of communication among humans. As computers play an ever  increasing role in our lives, it is important that we address the issue of  providing a graceful human-machine interface through spoken language.  In this paper, we will describe our recent efforts in moving beyond the  scope of speech recognition into the realm of spoken-language understand(cid:173) ing. Specifically, we report on the development of an urban navigation and  exploration system called VOYAGER, an application which we have used as  a basis for performing research in spoken-language understanding.
Generalization by Weight-Elimination with Application to Forecasting
Authors: David E. Rumelhart, Bernardo A. Huberman Andreas S. Weigend
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Inspired by the information theoretic idea of minimum description length, we add  a term  to the back propagation cost function that penalizes network complexity.  We  give  the  details  of the  procedure,  called  weight-elimination,  describe  its  dynamics, and clarify the meaning of the parameters involved. From a Bayesian  perspective,  the complexity term  can  be usefully interpreted as  an  assumption  about prior distribution of the weights.  We  use  this  procedure  to  predict  the  sunspot time series and the notoriously noisy series of currency exchange rates.
A Novel Approach to Prediction of the 3-Dimensional Structures of Protein Backbones by Neural Networks
Authors: Henrik Bohr, Jakob Bohr, Søren Brunak, Rodney M. J. Cotterill, Benny Lautrup, Steffen B. Petersen Henrik Fredholm
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Three-dimensional (3D) structures of protein backbones have been pre(cid:173) dicted using neural networks. A feed forward neural network was trained  on a class of functionally, but not structurally, homologous proteins, us(cid:173) ing backpropagation learning. The network generated tertiary structure  information in the form of binary distance constraints for the Co atoms  in the protein backbone. The binary distance between two Co atoms was  o if the distance between them was less than a certain threshold distance,  and 1 otherwise. The distance constraints predicted by the trained neu(cid:173) ral network were utilized to generate a folded conformation of the protein  backbone, using a steepest descent minimization approach.
Back Propagation Implementation on the Adaptive Solutions CNAPS Neurocomputer Chip
Authors: Hal McCartor
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: The Adaptive Solutions CN APS architecture chip is a general purpose  neurocomputer chip. It has 64 processors, each with 4 K bytes of local  memory, running at 25 megahertz. It is capable of implementing most  current neural network algorithms with on chip learning. This paper dis(cid:173) cusses the implementation of the Back Propagation algorithm on an array  of these chips and shows performance figures from a clock accurate hard(cid:173) ware simulator. An eight chip configuration on one board can update 2.3  billion connections per second in learning mode and process 9.6 billion  connections per second in feed forward mode.
Asymptotic slowing down of the nearest-neighbor classifier
Authors: Demetri Psaltis, Santosh S. Venkatesh Robert R. Snapp
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: If patterns are drawn from an n-dimensional feature space according to a  probability distribution that obeys a weak smoothness criterion, we show  that the probability that a random input pattern is misclassified by a  nearest-neighbor classifier using M random reference patterns asymptoti(cid:173) cally satisfies 
PM(error) "" Poo(error) + M2/n' 
a 
for sufficiently large values of M. Here, Poo(error) denotes the probability  of error in the infinite sample limit, and is at most twice the error of a  Bayes classifier. Although the value of the coefficient a depends upon the  underlying probability distributions, the exponent of M is largely distri(cid:173) bution free. We thus obtain a concise relation between a classifier's ability  to generalize from a finite reference sample and the dimensionality of the  feature space, as well as an analytic validation of Bellman's well known  "curse of dimensionality."
Connectionist Approaches to the Use of Markov Models for Speech Recognition
Authors: Nelson Morgan, Chuck Wooters Hervé Bourlard
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Previous  work  has  shown  the  ability  of  Multilayer  Perceptrons  (MLPs) to estimate emission probabilities for Hidden Markov Mod(cid:173) els  (HMMs).  The advantages of a  speech recognition system incor(cid:173) porating  both  MLPs  and  HMMs  are  the  best  discrimination  and  the  ability  to  incorporate  multiple  sources  of evidence  (features,  temporal context) without restrictive assumptions of distributions  or  statistical  independence.  This  paper  presents  results  on  the  speaker-dependent portion of DARPA's English language Resource  Management  database.  Results  support  the  previously  reported  utility of MLP probability estimation for  continuous speech recog(cid:173) nition.  An additional approach we  are pursuing is to use  MLPs as  nonlinear predictors for autoregressive HMMs.  While this is shown  to  be  more  compatible  with  the  HMM  formalism,  it  still  suffers  from several limitations.  This approach  is  generalized  to  take ac(cid:173) count of time correlation between successive observations, without  any restrictive assumptions about the driving noise.
A Second-Order Translation, Rotation and Scale Invariant Neural Network
Authors: Kristina M. Johnson, Karl E. Gustafson Shelly D. D. Goggin
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A second-order architecture is presented  here for  translation, rotation and  scale  invariant processing  of 2-D  images  mapped  to  n  input  units.  This  new architecture has a complexity of O( n) weights as opposed to the O( n 3 )  weights usually required  for  a  third-order,  rotation invariant architecture.  The reduction  in complexity is due  to the use  of discrete  frequency  infor(cid:173) mation.  Simulations show favorable  comparisons to other neural  network  architectures.
Lg Depth Estimation and Ripple Fire Characterization Using Artificial Neural Networks
Authors: Douglas R. Baumgardt John L. Perry
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This srudy has demonstrated how artificial neural networks (ANNs) can  be used to characterize seismic sources using high-frequency regional  seismic data. We have taken the novel approach of using ANNs as a  research tool for obtaining seismic source information, specifically  depth of focus for earthquakes and ripple-fire characteristics for  economic blasts, rather than as just a feature classifier between  earthquake and explosion populations. Overall, we have found that  ANNs have potential applications to seismic event characterization and  identification, beyond just as a feature classifier. In future studies, these  techniques should be applied to actual data of regional seismic events  recorded at the new regional seismic arrays. The results of this study  indicates that an ANN should be evaluated as part of an operational  seismic event identification system.
Adaptive Range Coding
Authors: James M. Goodwin, Jacques J. Vidal Bruce E. Rosen
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: these 
to  neuron-like  processing  elements.  "neurons" 
This  paper  examines  a  class  of  neuron  based  that  rely  on  learning  systems  for  dynamic  control  adaptive  range  coding  of  sensor  inputs.  Sensors  are  assumed  to  provide  binary  coded  range  vectors  that  coarsely  describe  the  system  state.  These  vectors  are  Output  input  decisions  generated  by  turn  the  system  state,  subsequently  producing  new  affect  inputs.  the  intervals  and  environment  are  evaluated.  The  neural  weights  as  well  as  the  ran g e  b 0  u n dar i e s  determining  the  output  decisions  are  then  altered  with  future  Preliminary  reinforcement  from  the  promise  of  adapting  "neural  experiments  show  receptive  learning  dynamical  control.  The  observed  performance  with  this  method  exceeds  that  of  earlier  approaches. 
the  goal  of  maximizing  the  environment.
RecNorm: Simultaneous Normalisation and Classification applied to Speech Recognition
Authors: Stephen J. Cox John S. Bridle
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A particular form of neural network is described, which has terminals  for acoustic patterns, class labels and speaker parameters. A method of  training this network to "tune in" the speaker parameters to a particular  speaker is outlined, based on a trick for converting a supervised network  to an unsupervised mode. We describe experiments using this approach  in isolated word recognition based on whole-word hidden Markov models.  The results indicate an improvement over speaker-independent perfor(cid:173) mance and, for unlabelled data, a performance close to that achieved on  labelled data.
A four neuron circuit accounts for change sensitive inhibition in salamander retina
Authors: Frank H. Eeckman, Frank S. Werblin Jeffrey L. Teeters
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In salamander retina, the response of On-Off ganglion cells to a central  flash is reduced by movement in the receptive field surround. Through  computer simulation of a 2-D model which takes into account their  anatomical and physiological properties, we show that interactions  between four neuron types (two bipolar and two amacrine) may be  responsible for the generation and lateral conductance of this change  sensitive inhibition. The model shows that the four neuron circuit can  account for previously observed movement sensitive reductions in  ganglion cell sensitivity and allows visualization and prediction of the  spatio-temporal pattern of activity in change sensitive retinal cells.
A Method for the Efficient Design of Boltzmann Machines for Classiffication Problems
Authors: Wolfgang Maass Ajay Gupta
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We introduce a method for the efficient design of a Boltzmann machine (or  a Hopfield net) that computes an arbitrary given Boolean function f . This  method is based on an efficient simulation of acyclic circuits with threshold  gates by Boltzmann machines. As a consequence we can show that various  concrete Boolean functions f that are relevant for classification problems  can be computed by scalable Boltzmann machines that are guaranteed  to converge to their global maximum configuration with high probability  after constantly many steps.
How Receptive Field Parameters Affect Neural Learning
Authors: Stephen M. Omohundro Bartlett W. Mel
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  identify  the three  principle factors  affecting  the  performance of learn(cid:173) ing by  networks  with  localized  units:  unit noise,  sample density,  and  the  structure of the target function.  We then  analyze the effect  of unit recep(cid:173) tive  field  parameters  on  these  factors  and  use  this  analysis  to  propose  a  new  learning algorithm which  dynamically alters receptive field  properties  during learning. 
1  LEARNING WITH LOCALIZED RECEPTIVE FIELDS 
Locally-tuned  representations  are  common in  both  biological  and  artificial  neural  networks.  Several workers have analyzed the effect of receptive field  size, shape, and  overlap  on  representation  accuracy:  (Baldi,  1988),  (Ballard,  1987),  and  (Hinton,  1986).  This paper investigates the additional interactions introduced by  the task of  function learning.  Previous studies which have considered learning have for the most  part restricted attention to the use of the input probability distribution to determine  receptive field  layout (Kohonen,  1984)  and (Moody and Darken,  1989).  We  will see  that the structure of the function  being learned may also  be  advantageously taken  into account. 
Function learning using  radial basis functions  (RBF's)  is  currently  a  popular tech(cid:173) nique  (Broomhead  and  Lowe,  1988)  and  serves  as  an  adequate framework  for  our  discussion.  Because  we  are interested  in constraints on biological systems,  we  must  explictly  consider  the  effects  of unit  noise.  The  goal  is  to  choose  the  layout  of  receptive fields  so as  to minimize average performance error.  Let  y  = f(x)  be  the  function  the  network  is  attempting  to  learn  from  example
Proximity Effect Corrections in Electron Beam Lithography Using a Neural Network
Authors: Kevin D. Cummings, Edward A. Rietman Robert C. Frye
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We have used a neural network to compute corrections for images written  by electron beams to eliminate the proximity effects caused by electron  Iterative methods are effective. but require prohibitively  scattering.  computation time. We have instead trained a neural network to perform  equivalent corrections. resulting in a significant speed-up. We have  examined hardware  implementations using both analog and digital  electronic networks. Both had an acceptably small error of 0.5% compared  to the iterative results. Additionally. we verified that the neural network  correctly generalized the solution of the problem to include patterns not  contained in its training set. We have experimentally verified this approach  on a Cambridge Instruments EBMF 10.5 exposure system.
Exploiting Syllable Structure in a Connectionist Phonology Model
Authors: Deirdre W. Wheeler David S. Touretzky
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In  a  previous  paper  (Touretzky  &  Wheeler,  1990a) we showed  how  adding  a  clustering operation to a connectionist phonology model produced a parallel pro(cid:173) cessing account of certain "iterative" phenomena.  In this paper we show how the  addition of a second structuring primitive, syllabification, greatly increases  the  power of the model.  We present examples from  a non-Indo-European language  that appear to require rule ordering to at least a depth of four.  By adding syllab(cid:173) ification circuitry to structure the model's perception of the input string, we are  able to handle these examples with only two derivational steps.  We conclude that  in phonology, derivation can be largely replaced by structuring.
Integrated Modeling and Control Based on Reinforcement Learning and Dynamic Programming
Authors: Richard S. Sutton
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This is a summary of results with Dyna, a class of architectures for intel(cid:173) ligent systems based on approximating dynamic programming methods.  Dyna architectures integrate trial-and-error (reinforcement) learning and  execution-time planning into a single process operating alternately on the  world and on a learned forward model of the world. We describe and  show results for two Dyna architectures, Dyna-AHC and Dyna-Q. Using a  navigation task, results are shown for a simple Dyna-AHC system which  simultaneously learns by trial and error, learns a world model, and plans  optimal routes using the evolving world model. We show that Dyna-Q  architectures (based on Watkins's Q-Iearning) are easy to adapt for use in  changing environments. 
1 
Introduction to Dyna 
Dyna architectures (Sutton, 1990) use learning algorithms to approximate the con(cid:173) ventional optimal control technique known as dynamic programming (DP) (Bell(cid:173) man, 1957; Bertsekas, 1987). DP itself is not a learning method, but rather a  computational method for determining optimal behavior given a complete model of  the task to be solved. It is very similar to state-space search, but differs in that  it is more incremental and never considers actual action sequences explicitly, only  single actions at a time. This makes DP more amenable to incremental planning  at execution time, and also makes it more suitable for stochastic or incompletely  modeled environments, as it need not consider the extremely large number of se(cid:173) quences possible in an uncertain environment. Learned world models are likely  to be stochastic and uncertain, making DP approaches particularly promising for
A Connectionist Learning Control Architecture for Navigation
Authors: Jonathan R. Bachrach
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A novel learning control architecture is used for navigation. A sophisti(cid:173) cated test-bed is used to simulate a cylindrical robot with a sonar belt  in a planar environment. The task is short-range homing in the pres(cid:173) ence of obstacles. The robot receives no global information and assumes  no comprehensive world model. Instead the robot receives only sensory  information which is inherently limited. A connectionist architecture is  presented which incorporates a large amount of a priori knowledge in the  form of hard-wired networks, architectural constraints, and initial weights.  Instead of hard-wiring static potential fields from object models, myarchi(cid:173) tecture learns sensor-based potential fields, automatically adjusting them  to avoid local minima and to produce efficient homing trajectories. It does  this without object models using only sensory information. This research  demonstrates the use of a large modular architecture on a difficult task.
On Stochastic Complexity and Admissible Models for Neural Network Classifiers
Authors: Padhraic Smyth
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Given some  training data how  should we  choose a particular network clas(cid:173) sifier  from  a  family  of networks  of different  complexities?  In  this  paper  we  discuss how  the application of stochastic complexity theory to classifier  design problems can provide some insights into this problem.  In particular  we  introduce  the  notion  of admissible  models  whereby  the  complexity  of  models  under consideration is  affected  by  (among other factors)  the class  entropy,  the  amount  of training  data,  and  our  prior  belief.  In  particular  we  discuss the implications of these results with respect to neural architec(cid:173) tures and demonstrate the approach on real data from  a medical diagnosis  task. 
1 
Introduction  and Motivation 
In this paper we  examine in a general sense the application of Minimum Description  Length  (MDL)  techniques to the  problem of selecting a  good  classifier from  a  large  set  of candidate models  or  hypotheses.  Pattern recognition  algorithms  differ  from  more  conventional  statistical  modeling  techniques  in  the  sense  that  they  typically  choose from a very large number of candidate models to describe the available data.  Hence,  the  problem of searching  through  this set of candidate models  is  frequently  a  formidable  one, often  approached in practice  by  the use of greedy  algorithms.  In  this context, techniques which allow us to eliminate portions of the hypothesis space  are of considerable interest.  We  will show  in  this paper that it is  possible to use  the  intrinsic  structure of the  MDL  formalism  to eliminate large  numbers  of candidate  models given  only minimal information about  the  data.  Our results  depend  on the
Analog Computation at a Critical Point: A Novel Function for Neuronal Oscillations?
Authors: William Bialek Leonid Kruglyak
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: \Ve show that a simple spin system bia.sed at its critical point can en(cid:173) code spatial characteristics of external signals, sHch as the dimensions of  "objects" in the visual field. in the temporal correlation functions of indi(cid:173) vidual spins. Qualit.ative arguments suggest that regularly firing neurons  should be described by a planar spin of unit lengt.h. and such XY models  exhibit critical dynamics over a broad range of parameters. \Ve show how  to extract these spins from spike trains and then mea'3ure t.he interaction  Hamilt.onian using simulations of small dusters of cells. Static correla(cid:173) tions among spike trains obtained from simulations of large arrays of cells  are in agreement with the predictions from these Hamiltonians, and dy(cid:173) namic correlat.ions display the predicted encoding of spatial information.  \Ve suggest that this novel representation of object dinwnsions in temporal  correlations may be relevant t.o recent experiment.s on oscillatory neural  firing in the visual cortex.
Integrated Segmentation and Recognition of Hand-Printed Numerals
Authors: David E. Rumelhart, Wee Kheng Leow James D. Keeler
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Neural  network  algorithms  have  proven  useful  for  recognition  of individ(cid:173) ual,  segmented  characters.  However,  their recognition  accuracy  has  been  limited by  the  accuracy  of the  underlying  segmentation  algorithm.  Con(cid:173) ventional,  rule-based  segmentation  algorithms  encounter  difficulty  if the  characters  are touching, broken,  or noisy.  The problem in these situations  is  that  often  one  cannot  properly  segment  a  character  until  it  is  recog(cid:173) nized yet  one cannot  properly recognize  a  character until it is  segmented.  We present here  a  neural network algorithm that simultaneously segments  and recognizes  in an  integrated  system.  This  algorithm has several  novel  features:  it uses  a supervised learning algorithm (backpropagation), but is  able to take position-independent information as  targets and self-organize  the  activities  of the units  in  a  competitive fashion  to infer the  positional  information.  We  demonstrate  this  ability with  overlapping  hand-printed  numerals.
Speech Recognition Using Demi-Syllable Neural Prediction Model
Authors: Takao Watanabe Ken-ichi Iso
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: The  Neural  Prediction  Model  is  the  speech  recognition  model  based  on  pattern  prediction  by  multilayer  perceptrons.  Its  effectiveness  was  con(cid:173) firmed  by  the  speaker-independent  digit  recognition  experiments.  This  paper  presents  an  improvement  in  the model  and  its  application  to  large  vocabulary speech recognition,  based on subword units.  The improvement  involves an introduction  of "backward  prediction,"  which further  improves  the  prediction  accuracy  of the  original  model  with  only  "forward  predic(cid:173) tion".  In  application of the model  to  speaker-dependent large vocabulary  speech recognition,  the demi-syllable unit is  used  as  a subword recognition  unit.  Experimental  results  indicated  a  95.2%  recognition  accuracy  for  a  5000  word  test  set  and  the  effectiveness  was  confirmed  for  the  proposed  model  improvement and  the demi-syllable subword units.
The Devil and the Network: What Sparsity Implies to Robustness and Memory
Authors: Santosh S. Venkatesh Sanjay Biswas
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Robustness is a commonly bruited property of neural networks; in particu(cid:173) lar, a folk theorem in neural computation asserts that neural networks-in  contexts with large interconnectivity-continue to function efficiently, al(cid:173) beit with some degradation, in the presence of component damage or loss.  A second folk theorem in such contexts asserts that dense interconnectiv(cid:173) ity between neural elements is a sine qua non for the efficient usage of  resources. These premises are formally examined in this communication  in a setting that invokes the notion of the "devil" 1 in the network as an  agent that produces sparsity by snipping connections. 
1 ON REMOVING THE FOLK FROM THE THEOREM 
Robustness in the presence of component damage is a property that is commonly  attributed to neural networks. The content of the following statement embodies  this sentiment. 
Folk Theorem 1: Computation in neural networks is not substantially  affected by damage to network components. 
While such a statement is manifestly not true in general-witness networks with  "grandmother cells" where damage to the critical cells fatally impairs the com(cid:173) putational ability of the network-there is anecdotal evidence in support of it in 
1 Well, maybe an imp.
Development and Spatial Structure of Cortical Feature Maps: A Model Study
Authors: Helge Ritter, Klaus Schulten Klaus Obermayer
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: K.  Schulten  Beckman -Insti t u te  University  of Illinois  Urbana, IL  61801 
Feature selective cells in  the primary visual cortex of several species are or(cid:173) ganized in hierarchical topographic maps of stimulus features like  "position  in  visual  space",  "orientation"  and" ocular  dominance".  In  order  to  un(cid:173) derstand and describe their spatial structure and their development, we  in(cid:173) vestigate a self-organizing neural network model based on the feature map  algorithm.  The  model  explains  map  formation  as  a  dimension-reducing  mapping  from  a  high-dimensional  feature  space  onto  a  two-dimensional  lattice,  such  that "similarity"  between features  (or feature  combinations)  is  translated  into  "spatial  proximity"  between  the  corresponding  feature  selective cells.  The model is  able to reproduce several aspects of the spatial  structure of cortical maps  in  the visual  cortex.
Connectionist Implementation of a Theory of Generalization
Authors: Sheila Kannappan Roger N. Shepard
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Empirically,  generalization between  a  training  and  a  test  stimulus  falls  off in  close approximation to  an  exponential decay  function  of distance between  the  two stimuli in the "stimulus space" obtained by multidimensional scaling.  Math(cid:173) ematically, this result is derivable from  the assumption that an  individual takes  the  training  stimulus  to  belong  to  a  "consequential"  region  that includes  that  stimulus but is  otherwise of unknown  location, size, and  shape in the stimulus  space (Shepard, 1987).  As the individual gains additional information about the  consequential region-by finding other stimuli to be consequential or nOl-the  theory  predicts  the shape  of the generalization  function  to  change  toward  the  function relating actual probability of the consequence to location in the stimulus  space.  This paper describes a natural connectionist implementation of the theory,  and illustrates how implications of the theory for generalization, discrimination,  and classification learning can be explored by connectionist simulation. 
1  THE THEORY OF GENERALIZATION 
Because we never confront exactly the same situation twice, anything we have learned in  any previous situation can guide us in deciding which action to take in the present situation  only  to  the  extent  that  the  similarity  between  the  two  situations  is  sufficient  to  justify  generalization of our previous learning to the present situation.  Accordingly, principles of  generalization must be foundational for any theory of behavior. 
In  Shepard  (1987)  nonarbitrary principles  of generalization  were  sought that  would  be  optimum in any world in which an object, however distinct from other objects, is generally  a  member of some class or natural kind  sharing some dispositional property of potential  consequence for the individual.  A newly encountered plant or animal  might be edible or
Discrete Affine Wavelet Transforms For Anaylsis And Synthesis Of Feedfoward Neural Networks
Authors: P. S. Krishnaprasad Y. C. Pati
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In this paper we show that discrete affine wavelet transforms can provide  a tool for the analysis and synthesis of standard feedforward neural net(cid:173) works. It is shown that wavelet frames for L2(IR) can be constructed based  upon sigmoids. The spatia-spectral localization property of wavelets can  be exploited in defining the topology and determining the weights of a  feedforward network. Training a network constructed using the synthe(cid:173) sis procedure described here involves minimization of a convex cost func(cid:173) tional and therefore avoids pitfalls inherent in standard backpropagation  algorithms. Extension of these methods to L2(IRN) is also discussed.
Convergence of a Neural Network Classifier
Authors: Anthony LaVigna John S. Baras
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In  this  paper,  we  prove  that  the  vectors  in  the  LVQ  learning  algorithm  converge.  We  do  this  by  showing  that  the  learning  algorithm  performs  stochastic  approximation.  Convergence  is  then  obtained  by  identifying  the  appropriate  conditions  on  the  learning  rate  and  on  the  underlying  statistics of the  classification  problem.  We  also  present  a  modification  to  the  learning algorithm which  we  argue  results  in  convergence  of the  LVQ  error to the Bayesian optimal error as  the appropriate parameters become  large.
Optimal Sampling of Natural Images: A Design Principle for the Visual System
Authors: Daniel L. Ruderman, A. Zee William Bialek
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We formulate the problem of optimizing the sampling of natural images  using an array of linear filters. Optimization of information capacity is  constrained by the noise levels of the individual channels and by a penalty  for the construction of long-range interconnections in the array. At low  signal-to-noise ratios the optimal filter characteristics correspond to bound  states of a Schrodinger equation in which the signal spectrum plays the  role of the potential. The resulting optimal filters are remarkably similar  to those observed in the mammalian visual cortex and the retinal ganglion  cells of lower vertebrates. The observed scale invariance of natural images  plays an essential role in this construction.
Time Trials on Second-Order and Variable-Learning-Rate Algorithms
Authors: Richard Rohwer
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: The performance of seven minimization algorithms are compared on five  neural network problems. These include a variable-step-size algorithm,  conjugate gradient, and several methods with explicit analytic or numerical  approximations to the Hessian.
A competitive modular connectionist architecture
Authors: Michael I. Jordan Robert A. Jacobs
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We describe a multi-network, or modular, connectionist architecture that  captures that fact that many tasks have structure at a level of granularity  intermediate to that assumed by local and global function approximation  schemes. The main innovation of the architecture is that it combines  associative and competitive learning in order to learn task decompositions.  A task decomposition is discovered by forcing the networks comprising the  architecture to compete to learn the training patterns. As a result of the  competition, different networks learn different training patterns and, thus,  learn to partition the input space. The performance of the architecture on  a "what" and "where" vision task and on a multi-payload robotics task  are presented.
A Model of Distributed Sensorimotor Control in the Cockroach Escape Turn
Authors: G. J. Kacmarcik, R.E. Ritzmann, H.J. Chiel R.D. Beer
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In response to a puff of wind, the American cockroach turns away and runs.  The circuit underlying the initial turn of this escape response consists of  three populations of individually identifiable nerve cells and appears to em(cid:173) ploy distributed representations in its operation. We have reconstructed  several neuronal and behavioral properties of this system using simplified  neural network models and the backpropagation learning algorithm con(cid:173) strained by known structural characteristics of the circuitry. In order to  test and refine the model, we have also compared the model's responses to  various lesions with the insect's responses to similar lesions.
A VLSI Neural Network for Color Constancy
Authors: John Allman, Geoffrey Fox, Rodney Goodman Andrew W. Moore
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A system for color correction has been designed, built, and tested suc(cid:173) cessfully; the essential components are three custom chips built using sub(cid:173) threshold analog CMOS VLSI. The system, based on Land's Retinex the(cid:173) ory of color constancy, produces colors similar in many respects to those  produced by the visual system. Resistive grids implemented in analog  VLSI perform the smoothing operation central to the algorithm at video  rates. With the electronic system, the strengths and weaknesses of the  algorithm are explored. 
1 A MODEL FOR COLOR CONSTANCY 
Humans have the remarkable ability to perceive object colors as roughly constant  even if the color of the illumination is varied widely. Edwin Land, founder of the  Polaroid Corporation, models the computation that results in this ability as three  identical center-surround operations performed independently in three color planes,  such as red, green, and blue (Land, 1986). The basis for this model is as follows. 
Consider first an array of grey papers with different reflectances. (Land designated  these arrays Mondrians, since they resemble the works of the Dutch painter Piet 
·Present address: Dept. of Physics, Syracuse University, Syracuse, NY 13244
Stereopsis by a Neural Network Which Learns the Constraints
Authors: Ying-Wung Lee Alireza Khotanzad
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This paper presents a neural network (NN) approach to the problem of  stereopsis. The correspondence problem (finding the correct matches  between the pixels of the epipolar lines of the stereo pair from amongst all  the possible matches) is posed as a non-iterative many-to-one mapping . A  two-layer feed forward NN architecture is developed to learn and code this  nonlinear and complex mapping using the back-propagation learning rule  and a training set. The important aspect of this technique is that none of  the typical constraints such as uniqueness and continuity are explicitly  imposed. All the applicable constraints are learned and internally coded  by the NN enabling it to be more flexible and more accurate than the  existing methods. The approach is successfully tested on several random(cid:173) dot stereograms. It is shown that the net can generalize its learned map(cid:173) ping to cases outside its training set. Advantages over the Marr-Poggio  Algorithm are discussed and it is shown that the NN performance is supe(cid:173) rIOr.
The Tempo 2 Algorithm: Adjusting Time-Delays By Supervised Learning
Authors: Alex Waibel Ulrich Bodenhausen
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: In this work we describe a new method that adjusts time-delays and the widths of  time-windows in artificial neural networks automatically.  The input of the units  are weighted by a gaussian input-window over time which allows the learning  rules for the delays and widths to be derived in the same way as it is used for the  weights.  Our results on a phoneme classification task compare well with results  obtained with the TDNN by Waibel et al., which was manually optimized for the  same task.
Learning Theory and Experiments with Competitive Networks
Authors: David E. van den Bout Griff L. Bilbro
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  apply  the  theory  of Tishby,  Levin,  and Sol1a  (TLS)  to two problems.  First  we analyze  an elementary problem for  which we find  the predictions  consistent  with  conventional  statistical  results.  Second  we  numerically  examine  the  more realistic  problem of training a  competitive net  to learn  a  probability  density  from  samples.  We  find  TLS  useful  for  predicting  average  training  behavior. 
. 
1  TLS  APPLIED  TO  LEARNING DENSITIES 
Recently  a  theory  of learning  has  been  constructed  which  describes  the  learning  of a  relation  from  examples  (Tishby, Levin,  and Sol1a,  1989),  (Schwarb, Samalan,  Sol1a,  and Denker,  1990).  The original derivation  relies  on  a  statistical  mechanics  treatment  of the  probability  of independent  events  in  a  system  with  a  specified  average value of an additive error function. 
The resulting  theory is  not restricted  to learning  relations  and it is  not  essentially  statistical  mechanical.  The TLS  theory  can be derived  from  the  principle  of maz(cid:173) imum  entropy,  a  general  inference  tool which  produces  probabilities  characterized  by certain values of the averages of specified functions(Jaynes,  1979).  A TLS theory  can be  constructed  whenever  the specified  function  is  additive and associated  with  independent examples.  In this paper we treat the problem of learning a  probability  density from samples.  Consider the model as some function p( z Iw) of fixed form and adjustable parameters  w  which are  to be chosen  to approximate 1'(z)  where  the overline denotes  the  true  density.  All we  know about l' are the elements of a  training set T  which are drawn
Further Studies of a Model for the Development and Regeneration of Eye-Brain Maps
Authors: A. E. Friedman Jack D. Cowan
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We describe a computational model of the development and regenera(cid:173) tion of specific eye-brain circuits. The model comprises a self-organiz(cid:173) ing map-forming network which uses local Hebb rules, constrained by  (genetically determined) molecular markers. Various simulations of  the development and regeneration of eye-brain maps in fish and frogs  are described, in particular successful simulations of experiments by  Schmidt-Cicerone-Easter; Meyer; and Y oon.