mtybilly/Instruct2DS huggingface dataset:
An Analog VLSI Splining Network
Authors: Vijay K. Samalam Daniel B. Schwartz
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  have produced  a  VLSI  circuit capable of learning  to approximate ar(cid:173) bitrary  smooth  of a  single  variable  using  a  technique  closely  related  to  splines.  The circuit effectively  has  512 knots space on  a  uniform grid and  has full support for  learning.  The circuit  also can be used  to approximate  multi-variable functions as sum of splines. 
An  interesting,  and  as  of yet,  nearly  untapped set of applications for  VLSI  imple(cid:173) mentation of neural network learning systems can be found in adaptive control and  non-linear signal processing.  In  most such  applications,  the  learning  task  consists  of approximating a  real  function  of a  small number  of continuous  variables  from  discrete  data points.  Special purpose hardware is especially  interesting for  applica(cid:173) tions of this  type since  they  generally  require  real  time on-line learning  and  there  can  be stiff constraints on  the power  budget and size  of the hardware.  Frequently,  the already difficult  learning problem is made more complex by the non-stationary  nature of the underlying process.  Conventional feed-forward  networks  with sigmoidal units are clearly  inappropriate  for applications of this type.  Although they have exhibited remarkable performance  in some types of time series  prediction  problems (for example, Wiegend,  1990  and  Atlas, 1990), their learning rates in general are too slow for on-line learning.  On-line  performance can be improved most easily by using networks with more constrained  architecture, effectively making the learning problem easier by giving the network a  hint about the learning task.  Networks that build local representations of the data,  such  as  radial  basis  functions,  are excellent  candidates for  these  type of problems.  One  great  advantage  of such  networks  is  that  they  require  only  a  single  layer  of  units.  If the position and width of the units are fixed,  the learning problem is linear
Designing Linear Threshold Based Neural Network Pattern Classifiers
Authors: Terrence L. Fine
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: The three problems that concern us are identifying a natural domain of  pattern classification applications of feed forward neural networks, select(cid:173) ing an appropriate feedforward network architecture, and assessing the  tradeoff between network complexity, training set size, and statistical reli(cid:173) ability as measured by the probability of incorrect classification. We close  with some suggestions, for improving the bounds that come from Vapnik(cid:173) Chervonenkis theory, that can narrow, but not close, the chasm between  theory and practice. 
1 Speculations on Neural Network Pattern Classifiers 
The goal is to provide rapid, reliable classification of new inputs from a  (1)  pattern source. Neural networks are appropriate as pattern classifiers when the  pattern sources are ones of which we have little understanding, beyond perhaps a  nonparametric statistical model, but we have been provided with classified samples  of features drawn from each of the pattern categories. Neural networks should be  able to provide rapid and reliable computation of complex decision functions. The  issue in doubt is their statistical response to new inputs. 
(2) The pursuit of optimality is misguided in the context of Point (1). Indeed, it  is unclear what might be meant by 'optimality' in the absence of a more detailed  mathematical framework for the pattern source. 
(3) The well-known, oft-cited 'curse of dimensionality' exposed by Richard Bell(cid:173) man may be a 'blessing' to neural networks. Individual network processing nodes  (e.g., linear threshold units) become more powerful as the number of their inputs  increases. For a large enough number n of points in an input space of d dimensions,  the number of dichotomies that can be generated by such a node grows exponen(cid:173) tially in d. This suggests that, unlike all previous efforts at pattern classification  that required substantial effort directed at the selection of low-dimensional feature  vectors so as to make the decision rule calculable, we may now be approaching a
Qualitative structure from motion
Authors: Daphna Weinshall
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Exact structure from motion is an ill-posed computation and therefore  very sensitive to noise. In this work I describe how a qualitative shape  representation, based on the sign of the Gaussian curvature, can be com(cid:173) puted directly from motion disparities, without the computation of an  exact depth map or the directions of surface normals. I show that humans  can judge the curvature sense of three points undergoing 3D motion from  two, three and four views with success rate significantly above chance. A  simple RBF net has been trained to perform the same task.
Learning to See Rotation and Dilation with a Hebb Rule
Authors: Margaret E. Sereno Martin I. Sereno
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Previous work (M.I. Sereno, 1989; cf. M.E. Sereno, 1987) showed that a  feedforward network with area VI-like input-layer units and a Hebb rule  can develop area MT-like second layer units that solve the aperture  problem for pattern motion. The present study extends this earlier work  to more complex motions. Saito et al. (1986) showed that neurons with  large receptive fields in macaque visual area MST are sensitive to  different senses of rotation and dilation, irrespective of the receptive field  location of the movement singularity. A network with an MT-like  second layer was trained and tested on combinations of rotating, dilating,  and translating patterns. Third-layer units learn to detect specific senses  of rotation or dilation in a position-independent fashion, despite having  position-dependent direction selectivity within their receptive fields.
A Comparative Study of the Practical Characteristics of Neural Network and Conventional Pattern Classifiers
Authors: Richard P Lippmann Kenney Ng
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Seven different pattern classifiers were implemented on a serial computer  and compared using artificial and speech recognition tasks. Two neural  network (radial basis function and high order polynomial GMDH network)  and five conventional classifiers (Gaussian mixture, linear tree, K nearest  neighbor, KD-tree, and condensed K nearest neighbor) were evaluated.  Classifiers were chosen to be representative of different approaches to pat(cid:173) tern classification and to complement and extend those evaluated in a  previous study (Lee and Lippmann, 1989). This and the previous study  both demonstrate that classification error rates can be equivalent across  different classifiers when they are powerful enough to form minimum er(cid:173) ror decision regions, when they are properly tuned, and when sufficient  training data is available. Practical characteristics such as training time,  classification time, and memory requirements, however, can differ by or(cid:173) ders of magnitude. These results suggest that the selection of a classifier  for a particular task should be guided not so much by small differences in  error rate, but by practical considerations concerning memory usage, com(cid:173) putational resources, ease of implementation, and restrictions on training  and classification times.
Real-time autonomous robot navigation using VLSI neural networks
Authors: Michael Brownlow, Gillian Marshall, Jan Tombs, Alan Murray Lionel Tarassenko
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We describe a real time robot navigation system based on three VLSI  neural network modules. These are a resistive grid for path planning, a  nearest-neighbour classifier for localization using range data from a time(cid:173) of-flight infra-red sensor and a sensory-motor associative network for dy(cid:173) namic obstacle avoidance .
CAM Storage of Analog Patterns and Continuous Sequences with 3N2 Weights
Authors: Frank Eeckman Bill Baird
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A simple architecture and algorithm for analytically guaranteed associa(cid:173) tive memory storage of analog patterns, continuous sequences, and chaotic  attractors in the same network is described. A matrix inversion determines  network weights, given prototype patterns to be stored. There are N units  of capacity in an N node network with 3N 2 weights. It costs one unit per  static attractor, two per Fourier component of each sequence, and four per  chaotic attractor. There are no spurious attractors, and there is a Lia(cid:173) punov function in a special coordinate system which governs the approach  of transient states to stored trajectories. Unsupervised or supervised incre(cid:173) mental learning algorithms for pattern classification, such as competitive  learning or bootstrap Widrow-Hoff can easily be implemented. The archi(cid:173) tecture can be "folded" into a recurrent network with higher order weights  that can be used as a model of cortex that stores oscillatory and chaotic  attractors by a Hebb rule. Hierarchical sensory-motor control networks  may be constructed of interconnected "cortical patches" of these network  modules. Network performance is being investigated by application to the  problem of real time handwritten digit recognition.
Generalization Properties of Radial Basis Functions
Authors: Christopher G. Atkeson Sherif M. Botros
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We examine the ability of radial basis functions  (RBFs) to generalize.  We  compare the performance of several types of RBFs.  We use the inverse dy(cid:173) namics of an idealized  two-joint  arm as  a  test case.  We find  that without  a  proper  choice of a  norm for  the inputs,  RBFs have  poor  generalization  properties.  A simple global scaling of the input variables greatly improves  performance.  We suggest some efficient methods to approximate this dis(cid:173) tance  metric.
Learning Time-varying Concepts
Authors: Thomas Petsche, Ronald L. Rivest Anthony Kuh
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This work extends computational learning theory to situations in which concepts  vary over time, e.g., system identification of a time-varying plant. We have  extended formal definitions of concepts and learning to provide a framework  in which an algorithm can track a concept as it evolves over time. Given  this framework and focusing on memory-based algorithms, we have derived  some PAC-style sample complexity results that determine, for example, when  tracking is feasible. We have also used a similar framework and focused on  incremental tracking algorithms for which we have derived some bounds on  the mistake or error rates for some specific concept classes.
A Theory for Neural Networks with Time Delays
Authors: José Carlos Príncipe Bert de Vries
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We present a new neural network model for processing of temporal  patterns.  This  model,  the  gamma  neural model,  is as  general  as  a  convolution  delay  model  with  arbitrary  weight  kernels  w(t).  We  show  that  the  gamma  model  can  be  formulated  as  a  (partially  prewired)  additive  model.  A  temporal  hebbian  learning  rule  is  derived  and  we  establish  links  to  related  existing  models  for  temporal processing.
Natural Dolphin Echo Recognition Using an Integrator Gateway Network
Authors: Patrick W. B. Moore, Paul E. Nachtigall, Ralph H. Penner Herbert L. Roitblat
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We have been studying the performance of a bottlenosed dolphin on  a delayed matching-to-sample task to gain insight into the processes and  mechanisms that the animal uses during echolocation. The dolphin  recognizes targets by emitting natural sonar signals and listening to the  echoes that return. This paper describes a novel neural network  architecture, called an integrator gateway network, that we have de(cid:173) veloped to account for this performance. The integrator gateway  network combines information from multiple echoes to classify targets  with about 90% accuracy. In contrast, a standard backpropagation  network performed with only about 63% accuracy.
Compact EEPROM-based Weight Functions
Authors: C. K. Sin, R. Chu, P. K. Ko A. Kramer
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We are focusing on the development of a  highly compact neural net weight  function based on the use of EEPROM devices.  These devices have already  proven  useful  for  analog  weight storage,  but  existing  designs  rely  on  the  use  of conventional voltage multiplication as the weight function,  requiring  additional  transistors  per  synapse.  A  parasitic  capacitance  between  the  floating gate and the drain of the EEPROM structure leads to an unusual  J-V characteristic  which can  be used  to advantage in  designing a compact  synapse.  This  novel  behavior  is  well  characterized  by  a  model  we  have  developed.  A single-device circuit results in a  1-quadrant synapse function  which  is  nonlinear,  though  monotonic.  A  simple  extension  employing  2  EEPROMs  results  in  a  2  quadrant  function  which  is  much  more  linear.  This approach offers  the  potential for  more than a  ten-fold  increase  in  the  density of neural net  implementations. 
1 
INTRODUCTION - ANALOG  WEIGHTING 
The recent surge of interest in neural networks and parallel analog computation has  motivated  the  need  for  compact  analog computing blocks.  Analog  weighting is  an  important computational function  of this class.  Analog  weighting is  the combining  of two analog values, one  of which  is  typically varying (the input) and one of which  is  typically fixed  (the  weight)  or  at  least  varying  more  slowly.  The  varying  value  is  "weighted"  by  the  fixed  value  through  the  "weighting function",  typically  mul(cid:173) tiplication.  Analog  weighting is  most  interesting  when  the  overall computational  task  involves  computing  the  "weighted  sum  of the  inputs."  That  is,  to  compute 
2:7=1  t(lOj, Vi)  where  to  is  the  weighting function  and  ~v =  {lOb W2,  ... , wn}  and
Discovering Viewpoint-Invariant Relationships That Characterize Objects
Authors: Geoffrey E. Hinton Richard S. Zemel
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Using  an  unsupervised  learning  procedure,  a  network  is  trained on  an en(cid:173) semble of images of the same two-dimensional object at different positions,  orientations  and  sizes.  Each  half of  the  network  "sees"  one  fragment  of  the object, and  tries to produce  as  output a set of 4 parameters that have  high mutual information with the 4 parameters output by  the other half of  the network.  Given the ensemble of training patterns, the 4 parameters on  which the two halves of the network can agree are the position, orientation,  and  size  of the  whole  object,  or  some  recoding  of them.  After  training,  the network can reject  instances of other shapes by  using the fact  that the  predictions  made  by  its  two  halves  disagree.  If two  competing  networks  are trained on  an unlabelled mixture of images of two objects, they  cluster  the training cases on the basis of the objects' shapes,  independently of the  position, orientation, and size.
Reinforcement Learning in Markovian and Non-Markovian Environments
Authors: Jürgen Schmidhuber
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This work addresses three problems with reinforcement learning and adap(cid:173) tive  neuro-control:  1.  Non-Markovian interfaces  between  learner and en(cid:173) vironment.  2.  On-line learning  based  on  system  realization.  3.  Vector(cid:173) valued adaptive critics.  An algorithm is described which is based on system  realization and on two interacting fully recurrent  continually running net(cid:173) works  which  may  learn  in  parallel.  Problems  with  parallel  learning  are  attacked  by  'adaptive randomness'.  It is  also  described  how  interacting  model/controller  systems  can  be  combined  with  vector-valued  'adaptive  critics'  (previous  critics have been  scalar).
Second Order Properties of Error Surfaces: Learning Time and Generalization
Authors: Ido Kanter, Sara A. Solla Yann LeCun
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: The learning time of a simple neural network model is obtained through an  analytic computation of the eigenvalue spectrum for the Hessian matrix,  which describes the second order properties of the cost function in the  space of coupling coefficients. The form of the eigenvalue distribution  suggests new techniques for accelerating the learning process, and provides  a theoretical justification for the choice of centered versus biased state  variables.
Connectionist Music Composition Based on Melodic and Stylistic Constraints
Authors: Todd Soukup Michael Mozer
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We describe  a  recurrent  connectionist  network,  called CONCERT,  that  uses  a  set  of  melodies  written  in  a  given  style  to  compose  new  melodies  in  that  style. CONCERT  is  an extension of a traditional algorithmic composition tech(cid:173) nique  in  which  transition  tables specify  the  probability  of the  next  note  as  a  function of previous context.  A central ingredient of CONCERT  is the use of a  psychologically-grounded representation of pitch.
Transforming Neural-Net Output Levels to Probability Distributions
Authors: Yann LeCun John S. Denker
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: (1)  The  outputs  of a  typical  multi-output  classification  network  do  not  satisfy the axioms of probability; probabilities should be positive and sum  to one.  This problem  can  be solved  by  treating  the trained  network  as  a  preprocessor that produces  a  feature  vector that can be further  processed,  for instance by classical statistical estimation techniques.  (2) We present a  method for computing the first two moments ofthe probability distribution  indicating the range of outputs that are  consistent  with the input and the  training  data.  It is  particularly  useful  to  combine  these  two  ideas:  we  implement the  ideas  of section  1 using  Parzen  windows,  where  the  shape  and relative size  of each  window  is  computed  using the ideas of section  2.  This  allows  us  to make  contact  between  important  theoretical ideas  (e.g.  the  ensemble  formalism)  and  practical  techniques  (e.g.  back-prop).  Our  results  also  shed  new  light  on  and  generalize  the  well-known  "soft max"  scheme. 
1  Distribution of Categories in Output Space 
In  many  neural-net  applications,  it  is  crucial  to  produce  a  set  of C  numbers  that  serve  as estimates of the probability of C  mutually exclusive outcomes.  For exam(cid:173) ple,  in  speech  recognition,  these  numbers  represent  the  probability of C  different  phonemes;  the probabilities of successive segments can be combined using a  Hidden  Markov  Model.  Similarly,  in  an  Optical  Character  Recognition  ("OCR")  applica(cid:173) tion,  the numbers  represent  C  possible characters.  Probability information for  the  "best  guess"  category  (and  probable runner-up  categories)  is  combined  with  con(cid:173) text,  cost  information, etcetera,  to produce  recognition of multi-character strings.
Can neural networks do better than the Vapnik-Chervonenkis bounds?
Authors: Gerald Tesauro David Cohn
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: \Ve describe a series of careful llumerical experiments which measure the  average generalization capability of neural networks trained on a variety of  simple functions. These experiments are designed to test whether average  generalization performance can surpass the worst-case bounds obtained  from formal learning theory using the Vapnik-Chervonenkis dimension  (Blumer et al., 1989). We indeed find that, in some cases, the average  generalization is significantly better than the VC bound: the approach to  perfect performance is exponential in the number of examples m, rather  than the 11m result of the bound. In other cases, we do find the 11m  behavior of the VC bound, and in these cases, the numerical prefactor is  closely related to prefactor contained in the bound.
Discovering Discrete Distributed Representations with Iterative Competitive Learning
Authors: Michael Mozer
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Competitive learning is an unsupervised algorithm that classifies input pat(cid:173) terns into mutually exclusive clusters. In a neural net framework, each clus(cid:173) ter is represented by a processing unit that competes with others in a winner(cid:173) take-all pool for an input pattern. I present a simple extension to the algo(cid:173) rithm that allows it to construct discrete, distributed representations. Discrete  representations are useful because they are relatively easy to analyze and  their information content can readily be measured. Distributed representa(cid:173) tions are useful because they explicitly encode similarity. The basic idea is  to apply competitive learning iteratively to an input pattern, and after each  stage to subtract from the input pattern the component that was captured in  the representation at that stage. This component is simply the weight vector  of the winning unit of the competitive pool. The subtraction procedure forces  competitive pools at different stages to encode different aspects of the input.  The algorithm is essentially the same as a traditional data compression tech(cid:173) nique known as multistep vector quantization, although the neural net per(cid:173) spective suggests potentially powerful extensions to that approach.
A Neural Expert System with Automated Extraction of Fuzzy If-Then Rules and Its Application to Medical Diagnosis
Authors: Yoichi Hayashi
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This paper proposes ajuzzy neural expert system (FNES) with the  following two functions: (1) Generalization of the information derived  from the training data and embodiment of knowledge in the form of the  fuzzy neural network; (2) Extraction of fuzzy If-Then rules with  linguistic relative importance of each proposition in an antecedent  (I f -part) from a trained neural network. This paper also gives a  method to extract automatically fuzzy If-Then rules from the trained  neural network. To prove the effectiveness and validity of the proposed  fuzzy neural expert system. a fuzzy neural expert system for medical  diagnosis has been developed.
Leaning by Combining Memorization and Gradient Descent
Authors: John C. Platt
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We have created a radial basis function network that allocates a  new computational unit whenever an unusual pattern is presented  to the network. The network learns by allocating new units and  adjusting the parameters of existing units. If the network performs  poorly on a presented pattern, then a new unit is allocated which  memorizes the response to the presented pattern. If the network  performs well on a presented pattern, then the network parameters  are updated using standard LMS gradient descent. For predicting  the Mackey Glass chaotic time series, our network learns much  faster than do those using back-propagation and uses a comparable  number of synapses.
Distributed Recursive Structure Processing
Authors: Yoshiro Miyata, Paul Smolensky Geraldine Legendre
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Harmonic grammar (Legendre,  et al., 1990) is a connectionist theory of lin(cid:173) guistic  well-formed ness  based on the assumption  that the well-formedness  of a  sentence  can  be  measured  by  the  harmony  (negative  energy)  of the  corresponding  connectionist  state.  Assuming  a  lower-level  connectionist  network that obeys a  few  general connectionist  principles  but is  otherwise  unspecified,  we  construct  a  higher-level  network  with  an  equivalent  har(cid:173) mony function  that captures the most linguistically relevant global aspects  of the  lower  level  network.  In  this  paper,  we  extend  the  tensor  product  representation  (Smolensky  1990)  to fully  recursive  representations  of re(cid:173) cursively  structured objects like  sentences  in  the lower-level  network.  We  show  theoretically  and  with  an  example  the  power  of the  new  technique  for  parallel distributed  structure  processing.
Dynamics of Learning in Recurrent Feature-Discovery Networks
Authors: Todd K. Leen
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: The  self-organization  of  recurrent  feature-discovery  networks  is  studied  from the perspective of dynamical systems.  Bifurcation theory reveals pa(cid:173) rameter regimes in which multiple equilibria or limit cycles coexist with the  equilibrium at which  the networks  perform principal component analysis.
Navigating through Temporal Difference
Authors: Peter Dayan
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Barto, Sutton and Watkins [2] introduced a grid task as a didactic ex(cid:173) ample of temporal difference planning and asynchronous dynamical pre>(cid:173) gramming. This paper considers the effects of changing the coding of the  input stimulus, and demonstrates that the self-supervised learning of a  particular form of hidden unit representation improves performance.
On the Circuit Complexity of Neural Networks
Authors: K. Y. Siu, A. Orlitsky, T. Kailath V. P. Roychowdhury
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: '~le introduce a geometric approach for investigating the power of threshold  circuits. Viewing n-variable boolean functions as vectors in 'R'2", we invoke  tools from linear algebra and linear programming to derive new results on  the realizability of boolean functions using threshold gat.es.  Using this approach, one can obtain: (1) upper-bounds on the number of  spurious memories in HopfielJ networks, and on the number of functions  implementable by a depth-d threshold circuit; (2) a lower bound on the  number of ort.hogonal input. functions required to implement. a threshold  function; (3) a necessary condit.ion for an arbit.rary set of input. functions to  implement a threshold function; (4) a lower bound on the error introduced  in approximating boolean functions using sparse polynomials; (5) a limit  on the effectiveness of the only known lower-bound technique (based on  computing correlations among boolean functions) for the depth of thresh(cid:173) old circuit.s implement.ing boolean functions, and (6) a constructive proof  that every boolean function f of n input variables is a threshold function  of polynomially many input functions, none of which is significantly cor(cid:173) related with f. Some of these results lead t.o genera.lizations of key results  concerning threshold circuit complexity, particularly t.hose that are based  on the so-called spectral or Ha.rmonic analysis approach. Moreover, our  geometric approach yields simple proofs, based on elementary results from  linear algebra, for many of these earlier results.
An Attractor Neural Network Model of Recall and Recognition
Authors: Yehezkel Yeshurun Eytan Ruppin
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This  work  presents  an  Attractor  Neural  Network  (ANN)  model  of Re(cid:173) call  and  Recognition.  It is  shown  that  an  ANN  model  can  qualitatively  account  for  a  wide  range  of experimental  psychological  data  pertaining  to  the  these  two  main  aspects  of memory  access.  Certain  psychological  phenomena  are  accounted  for,  including  the  effects  of list-length,  word(cid:173) frequency,  presentation  time,  context  shift,  and  aging.  Thereafter,  the  probabilities of successful  Recall  and  Recognition  are  estimated,  in  order  to  possibly enable further  quantitative examination of the model.
Training Knowledge-Based Neural Networks to Recognize Genes in DNA Sequences
Authors: Geoffrey G. Towell, Jude W. Shavlik Michiel O. Noordewier
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We describe the application of a hybrid symbolic/connectionist machine  learning algorithm to the task of recognizing important genetic sequences.  The symbolic portion of the KBANN system utilizes inference rules that  provide a roughly-correct method for recognizing a class of DNA sequences  known as eukaryotic splice-junctions. We then map this "domain theory"  into a neural network and provide training examples. Using the samples,  the neural network's learning algorithm adjusts the domain theory so that  it properly classifies these DNA sequences. Our procedure constitutes  a general method for incorporating preexisting knowledge into artificial  neural networks. We present an experiment in molecular genetics that  demonstrates the value of doing so.
ALCOVE: A Connectionist Model of Human Category Learning
Authors: John K. Kruschke
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: ALCOVE  is  a  connectionist  model  of human  category  learning  that  fits  a  broad spectrum of human learning data.  Its architecture is  based on well(cid:173) established  psychological  theory,  and  is  related  to  networks  using  radial  basis functions.  From the perspective of cognitive psychology,  ALCOVE can  be construed as a combination of exemplar-based representation and error(cid:173) driven  learning.  From the perspective of connectionism,  it can  be seen  as  incorporating constraints into back-propagation networks  appropriate for  modelling human learning.
Multi-Layer Perceptrons with B-Spline Receptive Field Functions
Authors: Marshall Flax, David Handelman, Jack Gelfand Stephen H. Lane
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Multi-layer perceptrons are often slow to learn nonlinear functions  with complex local structure due to the global nature of their function  approximations. It is shown that standard multi-layer perceptrons are  actually a special case of a more general network formulation that  incorporates B-splines into the node computations. This allows novel  spline network architectures to be developed that can combine the  generalization capabilities and scaling properties of global multi-layer  feedforward networks with the computational efficiency and learning  speed of local computational paradigms. Simulation results are  presented for the well known spiral problem of Weiland and of Lang  and Witbrock to show the effectiveness of the Spline Net approach. 
1.
Bumptrees for Efficient Function, Constraint and Classification Learning
Authors: Stephen M. Omohundro
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: A  new  class of data  structures called "bumptrees" is described.  These  structures  are  useful  for  efficiently  implementing  a  number  of neural  network related operations.  An empirical comparison with radial  basis  functions  is presented on a  robot ann mapping learning task.  Applica(cid:173) tions to density estimation. classification. and constraint representation  and learning are also outlined. 
1  WHAT IS A BUMPTREE? 
A bumptree is a new geometric data structure which is useful for efficiently learning. rep(cid:173) resenting. and evaluating geometric relationships in a variety of contexts. They are a natural  generalization  of several  hierarchical  geometric  data  structures  including  oct-trees.  k-d  trees. balltrees and boxtrees. They are useful for many geometric learning tasks including  approximating functions. constraint surfaces. classification regions. and probability densi(cid:173) ties from samples. In the function approximation case. the approach is related to radial basis  function neural networks, but supports faster construction, faster access, and more flexible  modification. We provide empirical data comparing bumptrees with radial basis functions  in section 2.  A bumptree is used to provide efficient access to a collection of functions on a Euclidean  space of interest. It is a complete binary tree in which a leaf corresponds to each function  of interest There are also functions  associated with  each internal node and the defining  constraint is that each interior node's function must be everwhere larger than each of the
Planning with an Adaptive World Model
Authors: Knut Möller, Alexander Linden Sebastian Thrun
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We present a new connectionist planning method [TML90].  By interaction  with  an  unknown  environment,  a  world  model  is  progressively  construc(cid:173) ted  using  gradient  descent.  For  deriving  optimal actions  with  respect  to  future  reinforcement,  planning is  applied in two steps:  an experience  net(cid:173) work proposes a  plan which is subsequently optimized by gradient descent  with  a  chain of world  models,  so  that  an  optimal reinforcement  may be  obtained  when  it  is  actually  run.  The  appropriateness  of this  method  is  demonstrated by a  robotics  application and a  pole balancing task.
Simple Spin Models for the Development of Ocular Dominance Columns and Iso-Orientation Patches
Authors: A. E. Friedman J.D. Cowan
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Simple classical spin models well-known to physicists as the ANNNI  and Heisenberg XY Models. in which long-range interactions occur in  a pattern given by the Mexican Hat operator. can generate many of the  structural properties characteristic of the ocular dominance columns  and iso-orientation patches seen in cat and primate visual cortex.
A Multiscale Adaptive Network Model of Motion Computation in Primates
Authors: Bimal Mathur, Christof Koch H. Taichi Wang
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We  demonstrate  a  multiscale  adaptive  network  model  of motion  computation in primate area MT. The model consists of two stages:  (l)  local velocities are measured across multiple spatio-temporal channels,  and (2) the optical flow  field  is computed by a  network of direction(cid:173) selective neurons  at multiple  spatial  resolutions.  This model  embeds  the computational efficiency of Multigrid algorithms within a parallel  network as well as adaptively  computes  the most reliable estimate of  the flow  field across different spatial scales. Our model neurons show  the same nonclassical receptive field properties as Allman's type I MT  neurons.  Since local velocities are measured across multiple channels,  various  channels  often  provide  conflicting  measurements  to  the  network. We have incorporated a  veto scheme for conflict resolution.  This mechanism provides a novel explanation for the spatial frequency  dependency of the psychophysical phenomenon called Motion Capture. 
1  MOTIVATION  We previously developed a two-stage model of motion computation in the visual system  of primates  (Le.  magnocellular pathway from  retina to  V1  and  MT;  Wang,  Mathur &  Koch, 1989). This algorithm has these deficiencies:  (1) the issue of optimal spatial scale  for  velocity measurement, and (2)  the issue optimal spatial scale for  the smoothness of  motion field.  To address these deficiencies, we have implemented a multi-scale motion  network based on multigrid algorithms.  All  methods of estimating optical flow  make a basic assumption about the scale of the  velocity relative to the  spatial  neighborhood and to the  temporal  discretization step of  delay. Thus,  if the velocity of the pattern  is much larger than  the ratio of the spatial to  temporal  sampling step, an incorrect velocity  value will be obtained (Battiti, Amaldi  &  Koch,  1991). Battiti et al. proposed a coarse-to-fine strategy for adaptively detennining
Spherical Units as Dynamic Consequential Regions: Implications for Attention, Competition and Categorization
Authors: Mark A. Gluck Stephen Jose Hanson
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: to construct dynamic 
Spherical Units can be used  reconfigurable  consequential regions, the geometric bases for Shepard's (1987) theory of  stimulus generalization in animals and humans. We derive from Shepard's  (1987) generalization theory a particular multi-layer network with dynamic  (centers and radii) spherical regions which possesses a specific mass function  (Cauchy). This learning model generalizes the configural-cue network model  (Gluck & Bower 1988): (1) configural cues can be learned and do not require  pre-wiring the power-set of cues, (2) Consequential regions are continuous  rather than discrete and (3) Competition amoungst receptive fields is shown  to be increased by the global extent of a particular mass function (Cauchy).  We compare other common mass functions (Gaussian; used in models of  Moody & Darken; 1989, Krushke, 1990) or just standard backpropogation  networks with hyperplane/logistic hidden units showing that neither fare as  well as models of human generalization and learning. 
1 The Generalization Problem 
Given a favorable or unfavorable consequence, what should an organism assume about  the contingent stimuli? If a moving shadow overhead appears prior to a hawk attack  what should an organism assume about other moving shadows, their shapes and  positions? If a dense food patch is occasioned by a particular density of certain kinds of  shrubbery what should the organism assume about other shurbbery, vegetation or its  spatial density? In an pattern recognition context, given a character of a certain shape,  orientation, noise level etc.. has been recognized correctly what should the system  assume about other shapes, orientations, noise levels it has yet to encounter? 
• Also a member of Cognitive Science Laboratory, Princeton University, Princeton, NJ 08544
Speech Recognition Using Connectionist Approaches
Authors: Khalid Choukri
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: This  paper  is  a  summary  of SPRINT  project  aims  and  results.  The  project  focus  on the use  of neuro-computing techniques to tackle various problems that  remain  unsolved  in  speech  recognition.  First  results  concern  the  use  of feed(cid:173) forward  nets  for  phonetic  units  classification,  isolated  word  recognition,  and  speaker  adaptation.
Feedback Synapse to Cone and Light Adaptation
Authors: Josef Skrzypek
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Light  adaptation  (LA)  allows  cone  vIslOn  to  remain  functional  between  twilight  and  the  brightest  time  of day  even  though,  at  anyone  time,  their  intensity-response (I-R)  characteristic  is  limited  to  3  log  units of the  stimu(cid:173) lating  light.  One mechanism  underlying  LA, was localized  in  the outer seg(cid:173) ment of an isolated cone (1,2). We found that by adding annular illhmination,  an  I-R  characteristic  of a  cone  can  be  shifted  along  the  intensity  domain.  Neural network involving feedback  synapse from  horizontal cells to cones is  involved  to  be  in  register  with  ambient  light  level  of  the  periphery.  An  equivalent  electrical  circuit  with  three  different  transmembrane  channels  leakage,  photocurrent  and  feedback  was  used  to  model static  behavior of a  cone.  SPICE simulation showed  that interactions between feedback  synapse  and  the  light  sensitive  conductance  in  the  outer  segment  can  shift  the  I-R  curves along  the  intensity domain, provided  that phototransduction  mechan(cid:173) ism is not saturated during maximally hyperpolarized light response.
Direct memory access using two cues: Finding the intersection of sets in a connectionist model
Authors: Michael S. Humphreys, John D. Bain, Simon Dennis Janet Wiles
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: For lack of alternative models, search and decision processes have provided the  dominant paradigm for human memory access using two or more cues, despite  evidence against search as an access process (Humphreys, Wiles & Bain, 1990).  We present an alternative process to search, based on calculating the intersection  of sets of targets activated by two or more cues. Two methods of computing  the intersection are presented, one using information about the possible targets,  the other constraining the cue-target strengths in the memory matrix. Analysis  using orthogonal vectors to represent the cues and targets demonstrates the  competence of both processes, and simulations using sparse distributed  representations demonstrate the performance of the latter process for tasks  involving 2 and 3 cues.
Flight Control in the Dragonfly: A Neurobiological Simulation
Authors: Marvin W. Luttges William E. Faller
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Neural network simulations of the dragonfly flight neurocontrol system  have  been  developed  to  understand  how  this  insect  uses  complex,  unsteady  aerodynamics.  The  simulation  networks  account  for  the  ganglionic  spatial  distribution  of  cells  as  well  as  the  physiologic  operating range and the stochastic cellular fIring history of each neuron.  In  addition  the  motor  neuron  firing  patterns,  "flight  command  sequences", were utilized. Simulation training was targeted against both  the  cellular  and  flight  motor  neuron  firing  patterns.  The  trained  networks  accurately  resynthesized  the  intraganglionic  cellular firing  patterns. These in  tum controlled the  motor neuron fIring patterns that  drive  wing  musculature  during  flight.  Such  networks  provide  both  neurobiological analysis tools and fIrst  generation controls for  the  use  of "unsteady" aerodynamics.
A Framework for the Cooperation of Learning Algorithms
Authors: Patrick Gallinari Léon Bottou
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We introduce a framework  for  training architectures composed of several  modules. This framework,  which  uses a statistical formulation  of learning  systems,  provides  a  unique  formalism  for  describing  many  classical  connectionist  algorithms  as  well  as  complex  systems  where  several  algorithms interact. It allows to design hybrid systems which combine the  advantages of connectionist algorithms as well as other learning algorithms.
Continuous Speech Recognition by Linked Predictive Neural Networks
Authors: Alex Waibel, Bojan Petek, Otto Schmidbauer Joe Tebelskis
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We present a large vocabulary, continuous speech recognition system based  on  Linked  Predictive  Neural  Networks  (LPNN's).  The system  uses  neu(cid:173) ral  networks  as  predictors  of speech  frames,  yielding  distortion  measures  which  are  used  by  the  One Stage DTW algorithm to perform  continuous  speech  recognition.  The system,  already  deployed  in  a  Speech  to Speech  Translation system, currently achieves 95%,  58%,  and 39% word accuracy  on  tasks  with  perplexity  5,  111,  and  402  respectively,  outperforming sev(cid:173) eral simple HMMs  that  we  tested.  We  also  found  that  the  accuracy  and  speed of the LPNN can be slightly improved by the judicious use of hidden  control  inputs.  We  conclude  by  discussing  the  strengths  and  weaknesses  of the predictive approach.
A B-P ANN Commodity Trader
Authors: Joseph E. Collard
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: An  Artificial  Neural  Network  recognize  a  buy/sell  particular  commodity  Propagation  of  errors  algorithm  was  used  to  encode  the  the  Long/Short  desired  output  and  18  fundamental  variables  plus  6  (or  18)  Trained  on  one  technical  variables  into  year  of  past  data  to  predict  long/short  market  positions  in  the  future  that  would  have  made  $10,301  profit  on  an  investment  of  less  than  $1000. 

relationship  between
Statistical Mechanics of Temporal Association in Neural Networks
Authors: Zhaoping Li, J. Leo van Hemmen Andreas V. M. Herz
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: We study the representation of static patterns and temporal associa(cid:173) tions in  neural networks with  a  broad  distribution of signal  delays.  For a certain class of such systems, a simple intuitive understanding  of the spatia-temporal computation becomes possible with the help  of a  novel  Lyapunov  functional. It allows  a  quantitative  study  of  the  asymptotic  network  behavior  through  a  statistical  mechanical  analysis. We  present  analytic  calculations of both retrieval  quality  and storage capacity and compare  them with  simulation results.
Cholinergic Modulation May Enhance Cortical Associative Memory Function
Authors: Brooke P. Anderson, James M. Bower Michael E. Hasselmo
Published in: Advances in Neural Information Processing Systems 3  (NIPS 1990) (Main Conference Track), 1990

Abstract: Combining neuropharmacological experiments with computational model(cid:173) ing, we have shown that cholinergic modulation may enhance associative  memory function in piriform (olfactory) cortex. We have shown that the  acetylcholine analogue carbachol selectively suppresses synaptic transmis(cid:173) sion between cells within piriform cortex, while leaving input connections  unaffected. When tested in a computational model of piriform cortex,  this selective suppression, applied during learning, enhances associative  memory performance.