mtybilly/Instruct2DS huggingface dataset:
Electronic Receptors for Tactile/Haptic Sensing
Authors: Andreas G. Andreou
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We discuss synthetic receptors for  haptic sensing. These are based on  magnetic field sensors (Hall effect structures) fabricated using standard  CMOS technologies.  These receptors, biased with a small permanent  magnet can detect the presence of ferro or ferri-magnetic objects in the  vicinity of the sensor. They can also detect the magnitude and direction  of the magnetic field.
A Programmable Analog Neural Computer and Simulator
Authors: Jan Van der Spiegel, David Blackman, Timothy Chiu, Thomas Clare, Joseph Dao, Christopher Donham, Tzu-pu Hsieh, Marc Loinaz Paul Mueller
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This  report describes  the  design  of a  programmable general  purpose analog neural computer and simulator.  It is intended  primarily  for  real-world  real-time  computations  such  as  analysis  of visual  or  acoustical patterns, robotics  and the development of  special purpose  neural nets.  The machine is scalable and  composed of interconnected  modules containing arrays of neurons, modifiable synapses and switches.  It runs  entirely in analog  mode but connection architecture, synaptic  gains and time  constants as well as neuron parameters are set digitally.  Each  neuron has a limited number of inputs and can be connected to any  but not all other neurons. For the determination of synaptic gains and the  implementation  of  learning  algorithms  the  neuron  outputs  are  multiplexed, AID  converted and stored in digital  memory.  Even at  moderate size of 1()3 to IDS neurons  computational speed is expected to  exceed that of any current  digital computer.
An Information Theoretic Approach to Rule-Based Connectionist Expert Systems
Authors: John W. Miller, Padhraic Smyth Rodney M. Goodman
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We discuss in this paper architectures for executing probabilistic rule-bases in a par(cid:173) allel manner,  using  as  a theoretical basis recently introduced information-theoretic  models.  We will begin by describing our (non-neural) learning algorithm and theory  of quantitative rule  modelling, followed  by  a discussion on  the exact nature of two  particular models.  Finally we work through an example of our approach, going from  database to rules to inference network, and compare the network's performance with  the theoretical limits for  specific  problems.
ALVINN: An Autonomous Land Vehicle in a Neural Network
Authors: Dean A. Pomerleau
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: ALVINN (Autonomous Land Vehicle In a Neural Network) is a 3-layer  back-propagation network designed for the task of road following. Cur(cid:173) rently ALVINN takes images from a camera and a laser range finder as input  and produces as output the direction the vehicle should travel in order to  follow the road. Training has been conducted using simulated road images.  Successful tests on the Carnegie Mellon autonomous navigation test vehicle  indicate that the network can effectively follow real roads under certain field  conditions. The representation developed to perfOIm the task differs dra(cid:173) matically when the networlc is trained under various conditions, suggesting  the possibility of a novel adaptive autonomous navigation system capable of  tailoring its processing to the conditions at hand.
Fast Learning in Multi-Resolution Hierarchies
Authors: John Moody
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: A class of fast, supervised learning algorithms is presented. They use lo(cid:173)
cal representations, hashing, atld multiple scales of resolution to approximate  functions which are piece-wise continuous. Inspired by Albus's CMAC model,  the algorithms learn orders of magnitude more rapidly than typical imple(cid:173) mentations of back propagation, while often achieving comparable qualities of  generalization. Furthermore, unlike most traditional function approximation  methods, the algorithms are well suited for use in real time adaptive signal  processing. Unlike simpler adaptive systems, such as linear predictive cod(cid:173) ing, the adaptive linear combiner, and the Kalman filter, the new algorithms  are capable of efficiently capturing the structure of complicated non-linear  systems. As an illustration, the algorithm is applied to the prediction of a  chaotic timeseries.
Mapping Classifier Systems Into Neural Networks
Authors: Lawrence Davis
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: Classifier systems  are  machine  learning  systems  incotporating  a  genetic  al(cid:173)
gorithm  as the learning mechanism.  Although they  respond to inputs that neural  networks can respond to,  their internal  structure, representation  fonnalisms,  and  learning mechanisms differ marlcedly from those employed by neural network re(cid:173) searchers in the same sorts of domains.  As a result, one might conclude that these  two types  of machine learning fonnalisms are intrinsically different.  This is  one  of two papers that, taken together, prove instead that classifier systems and neural  networks  are  equivalent.  In this  paper, half of the  equivalence is  demonstrated  through  the  description  of  a  transfonnation  procedure  that  will  map  classifier  systems into neural networks that  are isomotphic in behavior.  Several alterations  on  the  commonly-used paradigms  employed by  neural  networlc  researchers  are  required  in  order to make  the  transfonnation  worlc.  These alterations  are  noted  and their appropriateness is discussed.  The paper concludes with a  discussion  of  the practical import  of these  results,  and with comments on  their extensibility.
A Model for Resolution Enhancement (Hyperacuity) in Sensory Representation
Authors: John P. Miller Jun Zhang
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: Heiligenberg (1987)  recently proposed a  model  to explain how sen(cid:173)
sory  maps  could  enhance  resolution  through  orderly  arrangement  of 
broadly tuned receptors.  We  have  extended  this  model  to the general 
case  of polynomial  weighting  schemes  and  proved  that  the  response 
function  is  also  a  polynomial  of the  same  order.  We  further  demon(cid:173)
strated  that  the  Hermitian  polynomials  are  eigenfunctions  of the  sys(cid:173)
tem.  Finally we suggested  a  biologically  plausible mechanism for  sen(cid:173)
sory representation of external stimuli with resolution far exceeding the 
inter-receptor separation.
Temporal Representations in a Connectionist Speech System
Authors: Erich J. Smythe
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: SYREN  is  a  connectionist model  that uses  temporal  information  in  a  speech signal  for  syllable  recognition.  It classifies  the  rates  and directions of formant center transitions,  and uses an adaptive  method  to  associate  transition  events  with  each  syllable.  The  system  uses  explicit  spatial  temporal  representations through  de(cid:173) lay  lines.  SYREN  uses  implicit  parametric  temporal  representa(cid:173) tions  in  formant  transition  classification  through  node  activation  onset,  decay,  and transition delays  in sub-networks analogous to  visual  motion detector cells.  SYREN  recognizes 79% of six repe(cid:173) titions  of  24  consonant-vowel  syllables  when  tested  on  unseen  data,  and  recognizes  100%  of  its  training  syllables.
Neural Network Star Pattern Recognition for Spacecraft Attitude Determination and Control
Authors: A. Miguel San Martin Phillip Alvelda
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: computational  bottlenecks 
Currently,  the  most  complex  spacecraft  attitude  determination  and  control  tasks  are  ultimately  governed  by  ground-based  systems  and  personnel.  Conventional  on-board  systems  face  severe  serial  microprocessors operating on inherently parallel problems.  New  computer architectures based on the anatomy of the human brain  seem  to  promise  high  speed  and  fault-tolerant  solutions  to  the  limitations  of serial  processing.  This  paper  discusses  the  latest  applications of artificial neural networks to the  problem of star  pattern recognition  for spacecraft attitude determination.
A Massively Parallel Self-Tuning Context-Free Parser
Authors: Eugene Santos Jr.
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: The  Parsing  and  Learning  System(PALS)  is  a  massively  parallel  self-tuning context-free  parser.  It  is capable  of  parsing sentences of unbounded length mainly due to its  parse-tree representation scheme. The system is capable  of  improving  its  parsing  performance  through  the  presentation of training examples.
Neural Net Receivers in Multiple Access-Communications
Authors: Geoffrey Orsak, Mahesh Varanasi, Behnaam Aazhang Bernd-Peter Paris
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: The application of neural networks to the demodulation of  spread-spectrum signals in a multiple-access environment is  considered. This study is motivated in large part by the fact  that, in a multiuser system, the conventional (matched fil(cid:173) ter) receiver suffers severe performance degradation as the  relative powers of the interfering signals become large (the  "near-far" problem). Furthermore, the optimum receiver,  which alleviates the near-far problem, is too complex to be  of practical use. Receivers based on multi-layer perceptrons  are considered as a simple and robust alternative to the opti(cid:173) mum solution. The optimum receiver is used to benchmark  the performance of the neural net receiver; in particular, it is  proven to be instrumental in identifying the decision regions  of the neural networks. The back-propagation algorithm and  a modified version of it are used to train the neural net. An  importance sampling technique is introduced to reduce the  number of simulations necessary to evaluate the performance  of neural nets. In all examples considered the proposed neu(cid:173) ral ~et receiver significantly outperforms the conventional  recelver.
A Computationally Robust Anatomical Model for Retinal Directional Selectivity
Authors: Franklin R. Amthor Norberto M. Grzywacz
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We analyze a mathematical model for  retinal directionally selective  cells  based  on  recent  electrophysiological  data,  and  show  that  its  computation of motion direction is  robust against  noise  and speed.
Statistical Prediction with Kanerva's Sparse Distributed Memory
Authors: David Rogers
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: A  new  viewpoint  of  the  processing  performed  by  Kanerva's  sparse  distributed  memory  (SDM)  is  presented.  In  conditions  of  near- or  over- capacity,  where  the  associative-memory  behavior  of the  mod(cid:173) el  breaks  down,  the  processing  performed by  the  model  can  be  inter(cid:173) preted  as  that  of  a  statistical  predictor.  Mathematical  results  are  presented  which  serve  as  the  framework  for  a  new  statistical  view(cid:173) point  of  sparse  distributed  memory  and  for  which  the  standard  for(cid:173) mulation  of SDM  is  a  special  case.  This  viewpoint  suggests  possi(cid:173) ble  enhancements  to  the  SDM  model,  including  a  procedure  for  improving  the  predictiveness  of  the  system  based  on  Holland's  work  with  'Genetic  Algorithms',  and  a  method  for  improving  the  capacity of SDM even when used as an associative memory.
Learning Sequential Structure in Simple Recurrent Networks
Authors: Axel Cleeremans, James L. McClelland David Servan-Schreiber
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We explore a network architecture introduced by Elman (1988) for  predicting successive elements of a sequence. The network uses the  pattern of activation over a set of hidden units from time-step t-l,  together with element t, to predict element t+ 1. When the network is  trained with strings from a particular finite-state grammar, it can learn  to be a perfect finite-state recognizer for the grammar. Cluster analyses  of the hidden-layer patterns of activation showed that they encode  prediction-relevant information about the entire path traversed through  the network. We illustrate the phases of learning with cluster analyses  performed at different points during training. 
Several connectionist architectures that are explicitly constrained to capture  sequential infonnation have been developed. Examples are Time Delay  Networks (e.g. Sejnowski & Rosenberg. 1986) -- also called 'moving  window' paradigms -- or algorithms such as back-propagation in time  (Rumelhart. Hinton & Williams. 1986), Such architectures use explicit  representations of several consecutive events. if not of the entire history of  past inputs. Recently. Elman (1988) has introduced a simple recurrent  network (SRN) that has the potential to master an infinite corpus of  sequences with the limited means of a learning procedure that is completely  local in time (see Figure I.).
A Back-Propagation Algorithm with Optimal Use of Hidden Units
Authors: Yves Chauvin
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This  paper  presents  a  variation  of  the  back-propagation  algo(cid:173) rithm  that makes  optimal  use  of  a  network  hidden units  by  de(cid:173) cr~asing an  "energy"  term written  as  a  function  of  the  squared  activations  of  these  hidden units.  The  algorithm  can automati(cid:173) cally  find  optimal  or  nearly  optimal  architectures  necessary  to  solve  known  Boolean  functions,  facilitate  the  interpretation  of  the  activation  of  the  remaining  hidden  units  and  automatically  estimate the complexity of architectures appropriate for phonetic  labeling  problems.  The  general  principle  of the  algorithm  can  also be adapted to different tasks:  for  example,  it can be used to  eliminate the  [0,  0]  local minimum  of the  [-1.  +1]  logistic  acti(cid:173) vation  function  while  preserving  a  much  faster  convergence  and  forcing  binary  activations  over the  set of hidden  units.
GEMINI: Gradient Estimation Through Matrix Inversion After Noise Injection
Authors: Conrad C. Galland, Geoffrey E. Hinton Yann Le Cun
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: Learning procedures that measure how random perturbations of unit ac(cid:173) tivities correlate with changes in reinforcement are inefficient but simple  to implement in hardware. Procedures like back-propagation (Rumelhart,  Hinton and Williams, 1986) which compute how changes in activities af(cid:173) fect the output error are much more efficient, but require more complex  hardware. GEMINI is a hybrid procedure for multilayer networks, which  shares many of the implementation advantages of correlational reinforce(cid:173) ment procedures but is more efficient. GEMINI injects noise only at the  first hidden layer and measures the resultant effect on the output error.  A linear network associated with each hidden layer iteratively inverts the  matrix which relates the noise to the error change, thereby obtaining  the error-derivatives. No back-propagation is involved, thus allowing un(cid:173) known non-linearities in the system. Two simulations demonstrate the  effectiveness of GEMINI.
Neural Networks for Model Matching and Perceptual Organization
Authors: Gene Gindi, P. Anandan Eric Mjolsness
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We introduce an optimization approach for solving problems in com(cid:173)
puter vision that involve multiple levels of abstraction. Our objective  functions include compositional and specialization hierarchies. We cast  vision problems as inexact graph matching problems, formulate graph  matching in terms of constrained optimization, and use analog neural  networks to perform the optimization. The method is applicable to per(cid:173) ceptual grouping and model matching. Preliminary experimental results  are shown.
Neuronal Maps for Sensory-Motor Control in the Barn Owl
Authors: John C. Pearson, J. J. Gelfand, R. M. Peterson, W. E. Sullivan Clay D. Spence
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: The bam owl has fused visual/auditory/motor representations of  space in its midbrain which are used to orient the head so that visu(cid:173) al or auditory stimuli are centered in the visual field of view. We  present models and computer simulations of these structures which  address various problems, inclu
Performance of Synthetic Neural Network Classification of Noisy Radar Signals
Authors: F. D. Garber, I. Jouny, Ashok K. Krishnamurthy Stanley C. Ahalt
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This study evaluates the performance of the multilayer-perceptron  and  the frequency-sensitive  competitive  learning network  in  iden(cid:173) tifying  five  commercial  aircraft  from  radar  backscatter  measure(cid:173) ments.  The performance  of the neural network  classifiers  is  com(cid:173) pared  with  that of the  nearest-neighbor  and  maximum-likelihood  classifiers.  Our  results  indicate  that  for  this  problem,  the  neural  network  classifiers  are  relatively  insensitive  to  changes  in  the  net(cid:173) work  topology,  and  to the noise  level  in  the  training data.  While,  for  this  problem,  the traditional algorithms outperform these sim(cid:173) ple neural classifiers,  we  feel  that neural networks show  the poten(cid:173) tial for  improved performance.
Connectionist Learning of Expert Preferences by Comparison Training
Authors: Gerald Tesauro
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: A  new  training  paradigm,  caned  the  "eomparison  pa.radigm,"  is  introduced  for  tasks in which  a. network must  learn  to choose  a  prdcrred  pattern from  a  set of n  alternatives,  based on  examplcs of Imma.n  expert  prderences.  In this  pa.radigm,  the inpu t  to  the network consists of t.wo  uf the  n  alterna tives,  and  the  trained  output is  the expert's judgement of which  pa.ttern is  better.  This  para.digm is  applied  to  the lea,rning  of hackgammon,  a  difficult  board ga.me in  wllieh  the  expert selects  a  move  from  a. set,  of legal  mm·es.  \Vith  compa.rison  training,  much  higher  levels  of performance  can  hc  a.chiew~d, with  networks  that  are  much  smaller,  and  with  coding  sehemes  t.hat  are  much  simpler  and  easier  to  understand.  Furthermorf',  it  is  possible  to  set  up  the  network  so  tha.t  it  always  produces  consisten t  rank-orderings . 
1.
Neural Network Recognizer for Hand-Written Zip Code Digits
Authors: W. R. Gardner, Hans Peter Graf, Donnie Henderson, R. E. Howard, W. Hubbard, L. D. Jackel, Henry S. Baird, Isabelle Guyon John S. Denker
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This paper describes the construction of a system that recognizes hand-printed  digits, using a combination of classical techniques and neural-net methods. The  system has been trained and tested on real-world data, derived from zip codes seen  on actual U.S. Mail. The system rejects a small percentage of the examples as  unclassifiable, and achieves a very low error rate on the remaining examples. The  system compares favorably with other state-of-the art recognizers. While some of  the methods are specific to this task, it is hoped that many of the techniques will  be applicable to a wide range of recognition tasks.
Analog Implementation of Shunting Neural Networks
Authors: Robert B. Darling, Robert B. Pinter Bahram Nabet
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: An  extremely  compact,  all  analog  and  fully  parallel  implementa(cid:173) tion  of a  class  of shunting  recurrent  neural  networks  that  is  ap(cid:173) plicable to a  wide variety of FET-based integration  technologies is  proposed.  While the contrast enhancement, data compression, and  adaptation to mean input intensity capabilities of the network  are  well suited for  processing of sensory information or feature  extrac(cid:173) tion for a content addressable memory (CAM) system, the network  also admits a global Liapunov function  and can thus achieve stable  CAM storage  itself.  In  addition  the model  can  readily function  as  a front-end  processor to an analog adaptive resonance  circuit.
Range Image Restoration Using Mean Field Annealing
Authors: Wesley E. Snyder Griff L. Bilbro
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: A  new  optimization strategy,  Mean  Field  Annealing, is  presented.  Its application to MAP restoration of noisy range images is derived  and experimentally verified.
Analyzing the Energy Landscapes of Distributed Winner-Take-All Networks
Authors: David S. Touretzky
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: DCPS  (the  Distributed  Connectionist  Production System)  is  a  neural  network  with  complex  dynamical  properties.  Visualizing  the  energy  landscapes of some of its component modules leads to a  better intuitive  understanding  of the  model,  and  suggests  ways  in  which  its  dynamics  can be controlled in order to improve performance on difficult  cases.
Neural Approach for TV Image Compression Using a Hopfield Type Network
Authors: Jean-Bernard Theeten Martine Naillon
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: A self-organizing Hopfield network has been  developed in the context of Vector Ouantiza(cid:173) -tion, aiming at compression of  television  images. The metastable states of the spin  glass-like network are used as  an extra  the Minimal Overlap  storage resource using  and Mezard 1987) to  rule (Krauth  learning  the organization of the attractors.  optimize  The sel f-organi zi ng  that we have  scheme  devised  the generation of an  in  adaptive codebook for any qiven TV image.
Convergence and Pattern-Stabilization in the Boltzmann Machine
Authors: Roger Cheng Moshe Kam
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: The Boltzmann Machine has been introduced as a means to perform  global optimization for multimodal objective functions using the  principles of simulated annealing. In this paper we consider its utility  as a spurious-free content-addressable memory, and provide bounds on  its performance in this context. We show how to exploit the machine's  ability to escape local minima, in order to use it, at a constant  temperature, for unambiguous associative pattern-retrieval in noisy  environments. An association rule, which creates a sphere of influence  around each stored pattern, is used along with the Machine's dynamics  to match the machine's noisy input with one of the pre-stored patterns.  Spurious fIxed points, whose regions of attraction are not recognized by  the rule, are skipped, due to the Machine's fInite probability to escape  from any state. The results apply to the Boltzmann machine and to the  asynchronous net of binary threshold elements (Hopfield model'). They  provide the network designer with worst-case and best-case bounds for  the network's performance, and allow polynomial-time tradeoff studies  of design parameters.
Models of Ocular Dominance Column Formation: Analytical and Computational Results
Authors: Joseph B. Keller, Michael P. Stryker Kenneth D. Miller
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We  have  previously  developed  a  simple  mathemati(cid:173)
cal model  for  formation  of ocular  dominance  columns  in  mammalian  visual  cortex.  The  model  provides  a  com(cid:173) mon framework  in  which a  variety  of activity-dependent  biological machanisms can be studied.  Analytic and com(cid:173) putational  results  together  now  reveal  the  following:  if  inputs  specific  to  each eye  are  locally  correlated  in  their  firing,  and are not  anticorrelated within an arbor radius,  monocular  cells  will  robustly  form  and  be  organized  by  intra-cortical  interactions  into  columns.  Broader  corre(cid:173) lations  withln  each  eye,  or anti-correlations  between the  eyes, create a  more purely monocular cortex; positive cor(cid:173) relation  over  an  arbor  radius  yields  an  almost  perfectly  monocular cortex.  Most features of the model can be un(cid:173) derstood  analytically  through  decomposition  into  eigen(cid:173) functions and linear stability analysis.  This allows predic(cid:173) tion of the widths of the columns and other features from  measurable biological parameters.
Further Explorations in Visually-Guided Reaching: Making MURPHY Smarter
Authors: Bartlett W. Mel
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: MURPHY  is  a  vision-based  kinematic  controller  and  path  planner  based  on  a  connectionist  architecture,  and  implemented  with  a  video  camera and  Rhino XR-series robot  arm.  Imitative of the layout of sen(cid:173) sory  and motor maps in  cerebral cortex,  MURPHY'S internal representa(cid:173) tions  consist of four  coarse-coded populations of simple units represent(cid:173) ing both static and  dynamic aspects of the sensory-motor environment.  In previously reported work [4],  MURPHY first  learned a direct kinematic  model of his  camera-arm system during  a  period  of extended  practice,  and  then  used  this  "mental  model"  to  heuristically  guide  his  hand  to  unobstructed  visual  targets.  MURPHY  has  since  been  extended  in  two  ways:  First, he  now  learns the inverse differential-kinematics of his  arm  in  addition to ordinary direct  kinematics, which  allows  him to push  his  hand  directly towards  a  visual  target  without  the need  for  search.  Sec(cid:173) ondly,  he now  deals with the much more difficult problem of reaching in  the presence of obstacles.
Scaling and Generalization in Neural Networks: A Case Study
Authors: Gerald Tesauro Subutai Ahmad
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: The  issues  of scaling  and  generalization  have  emerged  as  key  issues  in  current studies of supervised learning from examples in neural networks.  Questions such  as  how  many  training  patterns  and  training  cycles  are  needed for  a problem of a given size  and difficulty,  how  to represent the  inllUh  and how  to choose useful training exemplars,  are of considerable  theoretical  and  practical  importance.  Several  intuitive  rules  of thumb  have been obtained from empirical studies, but as yet there are few  rig(cid:173) orous  results.  In  this  paper we  summarize  a  study Qf generalization in  the simplest possible case-perceptron networks learning linearly separa(cid:173) ble  functions.  The  task  chosen  was  the majority function  (i.e.  return  a  1  if a  majority  of the  input  units  are  on),  a  predicate  with  a  num(cid:173) ber  of useful  properties.  We  find  that  many  aspects  of.generalization  in  multilayer  networks  learning  large,  difficult  tasks  are  reproduced  in  this simple domain, in which  concrete numerical results and even some  analytic understanding can be achieved.
A Model of Neural Oscillator for a Unified Submodule
Authors: G. N. Borisyuk, R. M. Borisyuk, Ye. I. Kovalenko, V. I. Makarenko, V. A. Chulaevsky, V. I. Kryukov Alexandr B. Kirillov
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: Abstract Unavailable
An Optimality Principle for Unsupervised Learning
Authors: Terence D. Sanger
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: We propose an optimality  principle for  training an unsu(cid:173) pervised feedforward neural network based upon maximal  ability to reconstruct the input data from the network out(cid:173) puts.  We describe an algorithm which can be used to train  either  linear or  nonlinear  networks  with  certain  types  of  nonlinearity.  Examples of applications  to the problems of  image  coding,  feature  detection,  and analysis  of random(cid:173) dot stereograms are presented.
Neural Analog Diffusion-Enhancement Layer and Spatio-Temporal Grouping in Early Vision
Authors: Michael Seibert, Robert K. Cunningham, Jian Wu Allen M. Waxman
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: A  new  class of neural  network  aimed  at  early  visual  processing  is  described; we call it a Neural  Analog Diffusion-Enhancement Layer or  "NADEL." The  network  consists  of two  levels  which  are  coupled  through feedfoward and shunted feedback connections. The lower level  is  a  two-dimensional  diffusion map which  accepts  visual  features  as  input, and spreads activity over larger scales as a function of time. The  upper layer is periodically fed the  activity from  the diffusion layer and  locates local maxima in it (an extreme form  of contrast enhancement)  using a network of local comparators. These local maxima are fed back  to  the  diffusion  layer  using  an  on-center/off-surround  shunting  anatomy. The maxima are also available  as output of the network.  The  network dynamics  serves  to  cluster features  on  multiple  scales  as  a  function of time, and can be used in a variety of early visual processing  tasks such  as:  extraction of comers  and high  curvature  points  along  edge contours, line end detection, gap filling in contours, generation of  fixation points, perceptual grouping on multiple scales, correspondence  and path impletion  in long-range  apparent  motion,  and building  2-D  shape representations that are invariant to  location, orientation, scale,  and small deformation on the visual field.
Optimization by Mean Field Annealing
Authors: Reinhold Mann, Thomas K. Miller, Wesley E. Snyder, David E. van den Bout, Mark White Griff Bilbro
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: Nearly optimal solutions to many combinatorial problems can be  found using stochastic simulated annealing. This paper extends  the concept of simulated annealing from its original formulation  as a Markov process to a new formulation based on mean field  theory. Mean field annealing essentially replaces the discrete de(cid:173) grees of freedom in simulated annealing with their average values  as computed by the mean field approximation. The net result is  that equilibrium at a given temperature is achieved 1-2 orders of  magnitude faster than with simulated annealing. A general frame(cid:173) work for the mean field annealing algorithm is derived, and its re(cid:173) lationship to Hopfield networks is shown. The behavior of MFA is  examined both analytically and experimentally for a generic combi(cid:173) natorial optimization problem: graph bipartitioning. This analysis  indicates the presence of critical temperatures which could be im(cid:173) portant in improving the performance of neural networks. 
STOCHASTIC VERSUS MEAN FIELD 
In combinatorial optimization problems, an objective function or Hamiltonian,  H(s), is presented which depends on a vector of interacting 3pim, S = {81," .,8N},  in some complex nonlinear way. Stochastic simulated annealing (SSA) (S. Kirk(cid:173) patrick, C. Gelatt, and M. Vecchi (1983)) finds a global minimum of H by com(cid:173) bining gradient descent with a random process. This combination allows, under  certain conditions, choices of s which actually increa3e H, thus providing SSA with  a mechanism for escaping from local minima. The frequency and severity of these  uphill moves is reduced by slowly decreasing a parameter T (often referred to as  the temperature) such that the system settles into a global optimum. 
Two conceptual operationo; are involved in simulated annealing: a thermodatic op(cid:173) eration which schedules decreases in the temperature, and a relazation operation 
92
An Application of the Principle of Maximum Information Preservation to Linear Systems
Authors: Ralph Linsker
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This paper addresses the problem of determining the weights for a  set  of  linear  filters  (model  "cells")  so  as  to  maximize  the  ensemble-averaged information that the cells' output values jointly  convey about their input values,  given  the  statistical properties of  the ensemble of input vectors.  The quantity that is maximized is the  Shannon  information  rate,  or  equivalently  the  average  mutual  information between input and output.  Several models for the role  of processing noise are analyzed, and the biological motivation for  considering them is described.  For simple models in which nearby  input  signal  values  (in  space  or  time)  are  correlated,  the  cells  resulting  from  this  optimization  process  include  center-surround  cells and cells sensitive to temporal variations in input signal.
Spreading Activation over Distributed Microfeatures
Authors: James Hendler
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: One att·empt at explaining human inferencing is that of spread(cid:173) ing activat,ion, particularly in the st.ructured connectionist para(cid:173) digm. This has resulted in t.he building of systems with semanti(cid:173) cally nameable nodes which perform inferencing by examining  t.he pat,t.erns of activation spread. In this paper we demonst.rate  t.hat simple structured network infert'ncing can be p(>rformed by  passing art.iva.t.ion over the weights learned by a distributed alga(cid:173) rit,hm. Thus , an account, is provided which explains a well(cid:173) behaved rela t ionship bet.ween structured and distri butt'd conn('c(cid:173) t.ionist. a.pproachrs.
Consonant Recognition by Modular Construction of Large Phonemic Time-Delay Neural Networks
Authors: Alex Waibel
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: In this paperl  we show that neural networks for speech recognition can be constructed in  a  modular  fashion  by  exploiting  the  hidden  structure  of previously  trained  phonetic  subcategory networks.  The performance of resulting larger phonetic nets was found to be  as  good  as  the  performance  of the  subcomponent  nets  by  themselves.  This  approach  avoids the excessive learning times  that would be necessary to  train larger networks and  allows  for  incremental  learning.  Large  time-delay  neural  networks  constructed  incrementally  by  applying  these  modular  training  techniques  achieved  a  recognition  performance of 96.0% for all consonants.
Constraints on Adaptive Networks for Modeling Human Generalization
Authors: M. Pavel, Van Henkle Mark A. Gluck
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: The potential of adaptive  networks  to learn categorization rules and to  model  human  performance  is  studied  by  comparing  how  natural  and  artificial systems respond to new inputs, i.e., how they generalize.  Like  humans,  networks  can  learn  a  detenninistic  categorization  task  by  a  variety  of  alternative  individual  solutions.  An  analysis  of  the  con(cid:173) straints imposed by using networks with the minimal number of hidden  units  shows  that  this  "minimal  configuration"  constraint  is  not  sufficient to explain and predict human performance;  only a few  solu(cid:173) tions  were  found  to be  shared by both  humans and  minimal  adaptive  networks.  A  further  analysis  of human  and  network  generalizations  indicates  that  initial  conditions  may  provide  important constraints  on  generalization.  A new  technique,  which  we  call  "reversed learning",  is described for finding appropriate initial conditions.
Self Organizing Neural Networks for the Identification Problem
Authors: Wei-Tsih Lee Manoel Fernando Tenorio
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This  work  introduces  a  new  method called Self Organizing  Neural  Network  (SONN)  algorithm  and  demonstrates  its  use  in  a  system  identification  task.  The  algorithm  constructs  the  network,  chooses the neuron functions, and adjusts the weights. It is compared to  the Back-Propagation algorithm in the identification of the chaotic time  series.  The  results  shows  that  SONN  constructs  a  simpler,  more  accurate model. requiring less training data and epochs. The algorithm  can be applied and generalized to appilications as a classifier.
Learning with Temporal Derivatives in Pulse-Coded Neuronal Systems
Authors: Mark Gluck, Eric S. Reifsnider David B. Parker
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: A number of learning models have recently been proposed which  involve calculations of temporal differences (or derivatives in  continuous-time models). These models. like most adaptive network  models. are formulated in tenns of frequency (or activation), a useful  abstraction of neuronal firing rates. To more precisely evaluate the  implications of a neuronal model. it may be preferable to develop a  model which transmits discrete pulse-coded information. We point out  that many functions and properties of neuronal processing and learning  may depend. in subtle ways. on the pulse-coded nature of the informa(cid:173) tion coding and transmission properties of neuron systems. When com(cid:173) pared to formulations in terms of activation. computing with temporal  derivatives (or differences) as proposed by Kosko (1986). Klopf  (1988). and Sutton (1988). is both more stable and easier when refor(cid:173) mulated for a more neuronally realistic pulse-coded system. In refor(cid:173) mulating these models in terms of pulse-coding. our motivation has  been to enable us to draw further parallels and connections between  real-time behavioral models of learning and biological circuit models  of the substrates underlying learning and memory.
Using Backpropagation with Temporal Windows to Learn the Dynamics of the CMU Direct-Drive Arm II
Authors: Barak A. Pearlmutter Kenneth Y. Goldberg
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: Computing the inverse dynamics  of a robot ann is an active area of research  in the control literature.  We hope to  learn the  inverse dynamics  by training  a neural network on the  measured  response of a physical ann.  The  input  to  the  network is  a  temporal  window of measured positions;  output is  a vector  of torques.  We  train  the  network on  data measured from  the  first  two joints  of the CMU Direct-Drive Arm II  as  it moves  through a randomly-generated  sample  of "pick-and-place"  trajectories.  We  then  test  generalization  with  a  new  trajectory  and  compare  its  output  with  the  torque  measured  at  the  physical arm.  The network  is  shown  to  generalize with  a root mean  square  error/standard deviation  (RMSS)  of 0.10.  We  interpreted the weights  of the  network in tenns of the velocity and acceleration filters  used in  conventional  control  theory.
Storing Covariance by the Associative Long-Term Potentiation and Depression of Synaptic Strengths in the Hippocampus
Authors: Terrence J. Sejnowski Patric K. Stanton
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: In modeling studies or memory based on neural networks, both the selective  enhancement and depression or synaptic strengths are required ror effident storage  or inrormation (Sejnowski, 1977a,b; Kohonen, 1984; Bienenstock et aI, 1982;  Sejnowski and Tesauro, 1989). We have tested this assumption in the hippocampus,  a cortical structure or the brain that is involved in long-term memory. A brier,  high-frequency activation or excitatory synapses in the hippocampus produces an  increase in synaptic strength known as long-term potentiation, or L TP (BUss and  Lomo, 1973), that can last ror many days. LTP is known to be Hebbian since it  requires the simultaneous release or neurotransmitter from presynaptic terminals  coupled with postsynaptic depolarization (Kelso et al, 1986; Malinow and Miller,  1986; Gustatrson et al, 1987). However, a mechanism ror the persistent reduction or  synaptic strength that could balance LTP has not yet been demonstrated. We stu(cid:173) died the associative interactions between separate inputs onto the same dendritic  trees or hippocampal pyramidal cells or field CAl, and round that a low-frequency  input which, by itselr, does not persistently change synaptic strength, can either  increase (associative L TP) or decrease in strength (associative long-term depression  or LTD) depending upon whether it is positively or negatively correlated in time  with a second, high-frequency bursting input. LTP or synaptic strength is Hebbian,  and LTD is anti-Hebbian since it is elicited by pairing presynaptic firing with post(cid:173) synaptic hyperpolarization sufficient to block postsynaptic activity. Thus, associa(cid:173) tive L TP and associative L TO are capable or storing inrormation contained in the  covariance between separate, converging hippocampal inputs • 
• Present address: Dep~ents of NeW'Oscience and Neurology, Albert Einstein College  of Medicine, 1410 Pelham Parkway South, Bronx, NY 10461 USA. 
tPresent address: Computational Neurobiology Laboratory, The Salk Institute, P.O. Box  85800, San Diego, CA 92138 USA. 
Storing Covariance by Synaptic Strengths in the Hippocampus 
395
Dynamic, Non-Local Role Bindings and Inferencing in a Localist Network for Natural Language Understanding
Authors: Michael G. Dyer Trent E. Lange
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This  paper introduces  a means to  handle the critical problem  of non(cid:173) local  role-bindings  in  localist  spreading-activation  networks.  Every  conceptual node in the network broadcasts a stable, uniquely-identifying  activation pattern, called its signature.  A dynamic role-binding is cre(cid:173) ated  when  a  role's  binding  node  has  an  activation  that  matches  the  bound concept's signature.  Most importantly, signatures are propagated  across long paths of nodes to handle the non-local role-bindings neces(cid:173) sary  for  inferencing.  Our  localist  network  model,  ROBIN  (ROle  Binding  and  Inferencing  Network),  uses  signature  activations  to  ro(cid:173) bustly represent schemata role-bindings and thus perfonn the inferenc(cid:173) ing, plan/goal analysis,  schema instantiation, word-sense disambigua(cid:173) tion, and dynamic re-interpretation portions of the natural language un(cid:173) derstanding process.
Fixed Point Analysis for Recurrent Networks
Authors: Mary B. Ottaway, Dana H. Ballard Patrice Y. Simard
Published in: Advances in Neural Information Processing Systems 1  (NIPS 1988) (Main Conference Track), 1988

Abstract: This paper provides a systematic analysis of the recurrent backpropaga(cid:173) tion (RBP) algorithm, introducing a number of new results. The main  limitation of the RBP algorithm is that it assumes the convergence of  the network to a stable fixed point in order to backpropagate the error  signals. We show by experiment and eigenvalue analysis that this condi(cid:173) tion can be violated and that chaotic behavior can be avoided. Next we  examine the advantages of RBP over the standard backpropagation al(cid:173) gorithm. RBP is shown to build stable fixed points corresponding to the  input patterns. This makes it an appropriate tool for content address(cid:173) able memories, one-to-many function learning, and inverse problems.
From Stochastic Mixability to Fast Rates
Authors: Robert C. Williamson Nishant A. Mehta
Published in: Advances in Neural Information Processing Systems 27  (NIPS 2014) (Main Conference Track), 2014

Abstract: Empirical risk minimization (ERM) is a fundamental learning rule for statistical learning problems where the data is generated according to some unknown distribution $\mathsf{P}$ and returns a hypothesis $f$ chosen from a fixed class $\mathcal{F}$ with small loss $\ell$. In the parametric setting, depending upon $(\ell, \mathcal{F},\mathsf{P})$ ERM can have slow $(1/\sqrt{n})$ or fast $(1/n)$ rates of convergence of the excess risk as a function of the sample size $n$. There exist several results that give sufficient conditions for fast rates in terms of joint properties of $\ell$, $\mathcal{F}$, and $\mathsf{P}$, such as the margin condition and the Bernstein condition. In the non-statistical prediction with expert advice setting, there is an analogous slow and fast rate phenomenon, and it is entirely characterized in terms of the mixability of the loss $\ell$ (there being no role there for $\mathcal{F}$ or $\mathsf{P}$). The notion of stochastic mixability builds a bridge between these two models of learning, reducing to classical mixability in a special case. The present paper presents a direct proof of fast rates for ERM in terms of stochastic mixability of $(\ell,\mathcal{F}, \mathsf{P})$, and in so doing provides new insight into the fast-rates phenomenon. The proof exploits an old result of Kemperman on the solution to the general moment problem. We also show a partial converse that suggests a characterization of fast rates for ERM in terms of stochastic mixability is possible.
Active Regression by Stratification
Authors: Remi Munos Sivan Sabato
Published in: Advances in Neural Information Processing Systems 27  (NIPS 2014) (Main Conference Track), 2014

Abstract: We propose a new active learning algorithm for parametric linear regression with random design. We provide finite sample convergence guarantees for general distributions in the misspecified model. This is the first active learner for this setting that provably can improve over passive learning. Unlike other learning settings (such as classification), in regression the passive learning rate of O(1/epsilon) cannot in general be improved upon. Nonetheless, the so-called `constant' in the rate of convergence, which is characterized by a distribution-dependent risk, can be improved in many cases. For a given distribution, achieving the optimal risk requires prior knowledge of the distribution. Following the stratification technique advocated in Monte-Carlo function integration, our active learner approaches a the optimal risk using piecewise constant approximations.