mtybilly/Instruct2DS huggingface dataset:
Prediction of Beta Sheets in Proteins
Authors: Anders Krogh, Soren Kamaric Riis
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Most current methods for prediction of protein secondary structure  use a small window of the protein sequence to predict the structure  of the central amino acid.  We describe a new method for prediction  of the  non-local  structure  called  ,8-sheet,  which  consists  of two  or  more  ,8-strands  that  are  connected  by  hydrogen  bonds.  Since,8- strands  are  often  widely separated in the  protein  chain, a network  with two windows is introduced.  After training on a set of proteins  the network  predicts  the sheets  well,  but there are many false  pos(cid:173) itives.  By  using  a  global energy  function  the ,8-sheet  prediction is  combined with a  local prediction of the three secondary structures  a-helix, ,8-strand and coil.  The energy function is minimized using  simulated annealing to give  a  final  prediction.
A Smoothing Regularizer for Recurrent Neural Networks
Authors: Lizhong Wu, John E. Moody
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We derive a smoothing regularizer for recurrent network models by  requiring robustness in prediction performance to perturbations of  the training data.  The regularizer can be viewed  as  a  generaliza(cid:173) tion of the first order Tikhonov stabilizer to dynamic models.  The  closed-form  expression  of the  regularizer  covers  both time-lagged  and  simultaneous recurrent  nets,  with  feedforward  nets  and  one(cid:173) layer linear nets as special cases.  We  have successfully tested this  regularizer in a  number of case studies and found that it performs  better than standard quadratic weight  decay. 
1
REMAP: Recursive Estimation and Maximization of A Posteriori Probabilities - Application to Transition-Based Connectionist Speech Recognition
Authors: Yochai Konig, Hervé Bourlard, Nelson Morgan
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In  this  paper,  we  introduce  REMAP,  an  approach for  the training  and estimation of posterior probabilities using a recursive algorithm  that is  reminiscent of the EM-based  Forward-Backward  (Liporace  1982)  algorithm  for  the  estimation  of sequence  likelihoods.  Al(cid:173) though  very  general,  the  method  is  developed  in  the  context  of a  statistical  model for  transition-based speech  recognition  using  Ar(cid:173) tificial  Neural  Networks  (ANN)  to  generate  probabilities for  Hid(cid:173) den  Markov  Models  (HMMs).  In  the  new  approach,  we  use  local  conditional posterior probabilities of transitions to estimate global  posterior  probabilities of word  sequences.  Although  we  still  use  ANNs  to  estimate  posterior  probabilities,  the  network  is  trained  with targets that are themselves estimates of local posterior proba(cid:173) bilities.  An  initial experimental result shows a significant decrease  in error-rate in  comparison to a  baseline system.
Learning Sparse Perceptrons
Authors: Jeffrey C. Jackson, Mark Craven
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  introduce  a  new  algorithm  designed  to  learn  sparse  percep(cid:173) trons over input representations which include high-order features.  Our  algorithm,  which  is  based  on a  hypothesis-boosting  method,  is  able to PAC-learn a  relatively  natural class  of target concepts.  Moreover, the algorithm appears to work well in practice:  on a set  of three  problem  domains,  the algorithm  produces  classifiers  that  utilize  small  numbers  of features  yet  exhibit  good  generalization  performance.  Perhaps most  importantly,  our algorithm generates  concept descriptions that are easy for  humans to understand. 
1
The Role of Activity in Synaptic Competition at the Neuromuscular Junction
Authors: Samuel R. H. Joseph, David J. Willshaw
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: An  extended  version  of the  dual  constraint  model  of motor  end(cid:173) plate morphogenesis is  presented that includes activity dependent  and  independent  competition.  It is  supported by  a  wide  range  of  recent neurophysiological evidence that indicates a strong relation(cid:173) ship  between  synaptic  efficacy  and  survival.  The  computational  model is  justified  at the molecular level  and its predictions match  the developmental and regenerative behaviour of real synapses.
Explorations with the Dynamic Wave Model
Authors: Thomas P. Rebotier, Jeffrey L. Elman
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Following  Shrager  and  Johnson  (1995)  we  study  growth  of  logi(cid:173) cal  function  complexity  in  a  network  swept  by  two  overlapping  waves:  one  of pruning, and  the  other of Hebbian  reinforcement  of  connections.  Results  indicate  a  significant  spatial  gradient  in  the  appearance  of  both  linearly  separable  and  non  linearly  separable  functions of the two inputs of the network; the n.l.s.  cells are  much  sparser  and  their slope of appearance  is  sensitive  to  parameters  in  a  highly  non-linear  way.
A Multiscale Attentional Framework for Relaxation Neural Networks
Authors: Dimitris I. Tsioutsias, Eric Mjolsness
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  investigate  the  optimization  of neural  networks  governed  by  general  objective functions.  Practical formulations of such  objec(cid:173) tives  are  notoriously  difficult  to solve;  a  common problem  is  the  poor  local extrema that  result  by  any  of the  applied  methods.  In  this paper, a novel framework is introduced for the solution oflarge(cid:173) scale  optimization problems.  It assumes  little about  the objective  function and can be applied to general nonlinear, non-convex func(cid:173) tions;  objectives  in thousand  of variables  are  thus efficiently  min(cid:173) imized  by  a  combination of techniques  - deterministic  annealing,  multiscale optimization, attention mechanisms and trust region op(cid:173) timization methods.
VLSI Model of Primate Visual Smooth Pursuit
Authors: Ralph Etienne-Cummings, Jan Van der Spiegel, Paul Mueller
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A  one  dimensional  model  of primate  smooth  pursuit  mechanism  has  been  implemented  in  2  11m  CMOS  VLSI.  The  model  consolidates  Robinson's  negative  feedback  model  with  Wyatt  and  Pola's  positive  feedback scheme, to produce a  smooth  pursuit  system  which  zero's  the  velocity  of a  target on  the  retina.  Furthermore,  the  system  uses  the  current eye  motion  as  a  predictor for  future  target  motion.  Analysis,  stability and biological correspondence of the system are discussed.  For  implementation  at  the  focal  plane,  a  local  correlation  based  visual  motion  detection  technique  is  used.  Velocity  measurements,  ranging  over 4 orders of magnitude with < 15%  variation, provides  the  input  to  the smooth pursuit system.  The  system  performed  successful  velocity  tracking for high contrast scenes.  Circuit design and performance of the  complete smooth pursuit system is presented.
Reinforcement Learning by Probability Matching
Authors: Philip N. Sabes, Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  present  a  new  algorithm  for  associative  reinforcement  learn(cid:173) ing.  The algorithm is based upon the idea of matching a network's  output probability with a probability distribution derived from the  environment's reward signal.  This Probability Matching algorithm  is shown  to perform faster  and  be  less  susceptible to local  minima  than  previously  existing  algorithms.  We  use  Probability  Match(cid:173) ing to train mixture of experts networks,  an architecture for  which  other reinforcement  learning rules fail  to converge reliably on even  simple  problems.  This  architecture  is  particularly  well  suited  for  our  algorithm as it can compute arbitrarily complex functions  yet  calculation of the output probability is  simple.
Generalized Learning Vector Quantization
Authors: Atsushi Sato, Keiji Yamada
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  propose  a  new  learning  method,  "Generalized  Learning  Vec(cid:173) tor Quantization (GLVQ),"  in which reference vectors are updated  based on the steepest descent method in order to minimize the cost  function .  The  cost  function  is  determined  so  that  the  obtained  learning  rule  satisfies  the  convergence  condition.  We  prove  that  Kohonen's  rule  as  used  in  LVQ  does  not  satisfy  the  convergence  condition  and  thus  degrades  recognition  ability.  Experimental re(cid:173) sults  for  printed  Chinese  character recognition  reveal  that  GLVQ  is  superior to  LVQ  in  recognition  ability.
Bayesian Methods for Mixtures of Experts
Authors: Steve R. Waterhouse, David MacKay, Anthony J. Robinson
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  present  a  Bayesian framework  for  inferring  the  parameters of  a  mixture of experts  model based  on  ensemble  learning  by  varia(cid:173) tional free energy minimisation.  The Bayesian approach avoids the  over-fitting and noise level under-estimation problems of traditional  maximum likelihood inference.  We  demonstrate these  methods on  artificial  problems and sunspot  time series  prediction.
High-Speed Airborne Particle Monitoring Using Artificial Neural Networks
Authors: Alistair Ferguson, Theo Sabisch, Paul Kaye, Laurence C. Dixon, Hamid Bolouri
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Current environmental monitoring systems assume particles to be  spherical, and do not attempt to classify them. A laser-based sys(cid:173) tem developed at the University of Hertfordshire aims at classify(cid:173) ing airborne particles through the generation of two-dimensional  scattering profiles. The pedormances of template matching, and  two types of neural network (HyperNet and semi-linear units) are  compared for image classification. The neural network approach is  shown to be capable of comparable recognition pedormance, while  offering a number of advantages over template matching.
Using Pairs of Data-Points to Define Splits for Decision Trees
Authors: Geoffrey E. Hinton, Michael Revow
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Conventional binary classification trees  such  as  CART either split  the data using axis-aligned hyperplanes or  they perform a  compu(cid:173) tationally expensive search  in the continuous space of hyperplanes  with unrestricted orientations.  We show that the limitations of the  former  can  be overcome  without resorting to the latter.  For every  pair of training data-points, there is one hyperplane that is orthog(cid:173) onal  to the line joining the data-points and  bisects  this line.  Such  hyperplanes  are  plausible  candidates  for  splits.  In  a  comparison  on  a suite of 12  datasets  we  found  that this  method of generating  candidate splits outperformed  the standard  methods,  particularly  when  the training sets were  small.
Dynamics of On-Line Gradient Descent Learning for Multilayer Neural Networks
Authors: David Saad, Sara A. Solla
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  consider  the  problem  of on-line  gradient  descent  learning for  general  two-layer  neural  networks.  An  analytic  solution  is  pre(cid:173) sented  and used  to investigate the  role of the learning  rate in con(cid:173) trolling the evolution and  convergence  of the learning process. 
Learning in layered neural networks refers  to the modification of internal parameters  {J} which specify the strength of the interneuron couplings, so  as  to bring the map  fJ  implemented  by  the  network  as  close  as  possible  to  a  desired  map  1.  The  degree  of success  is  monitored  through  the  generalization  error,  a  measure  of the  dissimilarity between  fJ  and 1.  Consider  maps from  an  N-dimensional  input  space e onto  a  scalar  (,  as  arise  in  the formulation of classification  and  regression  tasks.  Two-layer  networks  with  an  arbitrary  number of hidden  units  have  been  shown  to  be  universal  approximators  [1]  for  such  N-to-one  dimensional  maps.  Information about  the  desired  map i  is  provided  through  independent  examples (e, (1'),  with  (I'  =  i(e) for  all  p .  The  examples are used  to train  a student  network with  N  input units,  K  hidden  units,  and  a  single  linear  output  unit;  the  target  map  i  is  defined  through  a  teacher  network  of  similar  architecture  except  for  the  number  M  of hidden  units.  We  investigate  the  emergence  of generalization  ability  in  an  on-line  learning scenario  [2],  in  which  the  couplings  are  modified  after  the  presentation  of each  example so  as  to minimize the corresponding error.  The resulting changes in {J} are described  as  a  dynamical evolution; the  number of examples plays the role  of time.  In  this  paper  we  limit our  discussion  to  the  case  of the  soft-committee  machine  [2],  in  which  all  the  hidden  units  are  connected  to  the  output  unit  with  positive  couplings  of unit  strength,  and  only  the  input-to-hidden  couplings  are  adaptive. 
*D.Saad@aston.ac.uk  tOn leave  from  AT&T  Bell  Laboratories,  Holmdel,  NJ  07733,  USA 
Dynamics of On-line Gradient Descent Learning for Multilayer Neural Networks 
303 
Consider  the  student  network:  hidden  unit  i  receives  information from  input  unit  r  through  the weight  hr, and  its activation under  presentation of an input pattern  ~ =  (6,· .. ,~N) is  Xi  =  J i  .~,  with  J i  =  (hl, ... ,JiN)  defined  as  the  vector  of  incoming weights  onto the  i-th  hidden  unit.  The output of the student  network  is  a(J,~) =  L:~l  9  (Ji . ~), where  9  is  the  activation function  of the  hidden  units,  taken here  to be the error function g(x) ==  erf(x/V2), and J  ==  {Jdl
Using Feedforward Neural Networks to Monitor Alertness from Changes in EEG Correlation and Coherence
Authors: Scott Makeig, Tzyy-Ping Jung, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  report  here  that  changes  in  the  normalized  electroencephalo(cid:173) graphic  (EEG)  cross-spectrum  can  be  used  in  conjunction  with  feedforward  neural networks to monitor changes in alertness of op(cid:173) erators  continuously  and  in  near-real  time.  Previously,  we  have  shown  that EEG spectral  amplitudes covary with changes in alert(cid:173) ness  as indexed by changes in behavioral error rate on an auditory  detection task [6,4].  Here, we report for the first time that increases  in the frequency  of detection errors  in  this task are  also  accompa(cid:173) nied  by  patterns of increased  and  decreased  spectral  coherence  in  several  frequency  bands  and  EEG  channel  pairs.  Relationships  between  EEG  coherence  and  performance vary  between  subjects,  but within subjects,  their topographic and spectral profiles appear  stable  from  session  to  session.  Changes  in  alertness  also  covary  with  changes  in  correlations  among  EEG  waveforms  recorded  at  different  scalp  sites,  and  neural  networks  can  also  estimate  alert(cid:173) ness  from  correlation  changes  in  spontaneous  and  unobtrusively(cid:173) recorded  EEG signals.
A Dynamical Model of Context Dependencies for the Vestibulo-Ocular Reflex
Authors: Olivier J. M. D. Coenen, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: The vestibulo-ocular reflex  (VOR)  stabilizes  images  on  the  retina during  rapid  head motions.  The gain of the VOR (the ratio of eye to head rotation velocity)  is  typically around -1  when the eyes are focused on a distant target.  However, to  stabilize images accurately, the VOR gain must vary  with context (eye position,  eye vergence and head  translation).  We  first  describe  a  kinematic  model of the  VOR which relies solely on sensory information available from the semicircular  canals (head rotation), the otoliths (head translation), and neural correlates of eye  position and vergence angle.  We then propose a dynamical model and compare it  to the eye velocity responses measured in monkeys. The dynamical model repro(cid:173) duces the observed amplitude and time course of the modulation of the VOR and  suggests one way to combine the required neural signals within the cerebellum and  the brain stem.  It also makes predictions for the responses of neurons to multiple  inputs (head rotation and translation, eye position, etc.) in the oculomotor system.
Constructive Algorithms for Hierarchical Mixtures of Experts
Authors: Steve R. Waterhouse, Anthony J. Robinson
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  present  two  additions  to  the  hierarchical  mixture  of experts  (HME)  architecture.  By  applying a  likelihood splitting criteria to  each expert in the HME we  "grow" the tree adaptively during train(cid:173) ing.  Secondly,  by considering only the most probable path through  the tree we  may "prune"  branches away, either temporarily, or per(cid:173) manently  if they  become  redundant.  We  demonstrate  results  for  the  growing  and  path  pruning  algorithms  which  show  significant  speed  ups  and  more efficient  use  of parameters over  the standard  fixed  structure  in  discriminating  between  two  interlocking spirals  and classifying 8-bit parity patterns.
Discovering Structure in Continuous Variables Using Bayesian Networks
Authors: Reimar Hofmann, Volker Tresp
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  study  Bayesian  networks  for  continuous  variables  using  non(cid:173) linear  conditional  density  estimators.  We  demonstrate  that  use(cid:173) ful  structures  can  be extracted  from  a  data set  in  a  self-organized  way and we  present sampling techniques for  belief update based on  Markov  blanket conditional density models.
Temporal coding in the sub-millisecond range: Model of barn owl auditory pathway
Authors: Richard Kempter, Wulfram Gerstner, J. Leo van Hemmen, Hermann Wagner
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Binaural  coincidence  detection  is  essential  for  the  localization  of  external  sounds  and  requires  auditory signal  processing  with  high  temporal precision.  We present an integrate-and-fire model of spike  processing in the auditory pathway of the barn owl.  It is shown that  a temporal precision in the microsecond range can be achieved with  neuronal  time constants  which  are  at least  one  magnitude longer.  An  important  feature  of our  model  is  an  unsupervised  Hebbian  learning rule which leads to a  temporal fine  tuning of the neuronal  connections. 
·email:  kempter.wgerst.lvh@physik.tu-muenchen.de 
Temporal Coding in  the Submillisecond Range:  Model of Bam Owl Auditory Pathway
Optimizing Cortical Mappings
Authors: Geoffrey J. Goodhill, Steven Finch, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: "Topographic"  mappings occur frequently  in the brain.  A  pop(cid:173) ular approach to understanding the structure of such mappings  is  to map points representing input features  in a  space of a  few  dimensions  to points in a  2 dimensional space using some self(cid:173) organizing  algorithm.  We  argue  that a  more general  approach  may be useful,  where similarities between features  are  not con(cid:173) strained to be geometric distances, and the objective function for  topographic matching is chosen explicitly rather than being spec(cid:173) ified implicitly by the self-organizing algorithm.  We  investigate  analytically an example of this more general approach applied to  the  structure of interdigitated mappings,  such as the pattern of  ocular dominance columns in primary visual cortex.
Experiments with Neural Networks for Real Time Implementation of Control
Authors: Peter K. Campbell, Michael Dale, Herman L. Ferrá, Adam Kowalczyk
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: This  paper  describes  a  neural  network  based  controller  for  allocating  capacity in a telecommunications network.  This system was proposed in  order  to  overcome  a  "real  time"  response  constraint.  Two  basic  architectures are evaluated:  1) a feedforward network-heuristic and; 2) a  feedforward  network-recurrent  network.  These  architectures  are  compared against a linear programming (LP) optimiser as  a benchmark.  This LP optimiser was  also used as  a teacher to label  the  data samples  for  the feedforward  neural  network training  algorithm.  It is  found  that  the systems  are  able to provide a traffic  throughput  of 99%  and  95%,  respectively,  of  the  throughput  obtained  by  the  linear  programming  solution. Once trained, the neural network based solutions are found in a  fraction of the time required by the LP optimiser.
Cholinergic suppression of transmission may allow combined associative memory function and self-organization in the neocortex
Authors: Michael E. Hasselmo, Milos Cekic
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Selective  suppression  of  transmission  at  feedback  synapses  during  learning  is  proposed  as  a  mechanism  for  combining  associative feed(cid:173) back  with  self-organization  of feed forward  synapses.  Experimental  data  demonstrates  cholinergic  suppression  of synaptic  transmission  in  layer I (feedback synapses), and a lack of suppression in layer IV (feed(cid:173) forward synapses).  A network with this feature uses local rules to learn  mappings  which  are not  linearly  separable.  During  learning,  sensory  stimuli  and  desired  response  are  simultaneously  presented  as  input.  Feedforward connections  form  self-organized representations  of input,  while suppressed feedback connections learn  the  transpose of feedfor(cid:173) ward connectivity. During recall, suppression is removed, sensory input  activates  the  self-organized  representation,  and  activity  generates  the  learned response.
Dynamics of Attention as Near Saddle-Node Bifurcation Behavior
Authors: Hiroyuki Nakahara, Kenji Doya
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In  consideration  of attention  as  a  means  for  goal-directed  behav(cid:173) ior in non-stationary environments,  we  argue that the dynamics of  attention  should  satisfy  two  opposing  demands:  long-term  main(cid:173) tenance  and  quick  transition.  These  two  characteristics  are  con(cid:173) tradictory  within  the linear domain.  We  propose  the near saddle(cid:173) node  bifurcation behavior of a  sigmoidal unit with self-connection  as  a  candidate of dynamical mechanism that satisfies both of these  demands.  We  further  show  in  simulations  of  the  'bug-eat-food'  tasks  that  the  near  saddle-node  bifurcation  behavior  of recurrent  networks  can  emerge  as  a functional  property  for  survival in  non(cid:173) stationary environments.
Softassign versus Softmax: Benchmarks in Combinatorial Optimization
Authors: Steven Gold, Anand Rangarajan
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A  new  technique,  termed  soft assign,  is  applied  for  the  first  time  to  two  classic  combinatorial  optimization  problems,  the  travel(cid:173) ing  salesman  problem  and  graph  partitioning.  Soft assign ,  which  has emerged from  the recurrent  neural  network/statistical physics  framework, enforces  two-way  (assignment) constraints without the  use  of penalty  terms  in  the  energy  functions.  The  soft assign  can  also  be  generalized  from  two-way  winner-take-all  constraints  to  multiple membership constraints which are required for graph par(cid:173) titioning.  The  soft assign  technique  is  compared  to  the  softmax  (Potts  glass).  Within  the  statistical  physics  framework,  softmax  and a penalty term has been a widely used method for enforcing the  two-way constraints common within many combinatorial optimiza(cid:173) tion  problems.  The  benchmarks  present  evidence  that  soft assign  has clear advantages in accuracy, speed,  parallelizabilityand algo(cid:173) rithmic simplicity over softmax and a penalty term in optimization  problems with two-way constraints.
From Isolation to Cooperation: An Alternative View of a System of Experts
Authors: Stefan Schaal, Christopher G. Atkeson
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We introduce a constructive,  incremental learning system for regression  problems that models data by means of locally linear experts.  In contrast  to  other approaches,  the  experts  are  trained  independently  and  do  not  compete for data during learning.  Only when a prediction for  a query  is  required  do  the  experts  cooperate  by  blending  their  individual  predic(cid:173) tions.  Each expert is trained by  minimizing  a penalized local cross vali(cid:173) dation error using second order methods. In this way, an expert is able to  find a local distance metric by adjusting the size and  shape of the recep(cid:173) tive field in which its predictions are valid, and also to detect relevant in(cid:173) put features  by  adjusting  its bias  on  the  importance of individual input  dimensions. We derive asymptotic results for our method. In a variety of  simulations the properties of the algorithm are demonstrated with respect  to  interference,  learning  speed,  prediction  accuracy,  feature  detection,  and task oriented incremental learning.
Fast Learning by Bounding Likelihoods in Sigmoid Type Belief Networks
Authors: Tommi Jaakkola, Lawrence K. Saul, Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Sigmoid  type  belief networks,  a  class  of probabilistic  neural  net(cid:173) works,  provide  a  natural  framework  for  compactly  representing  probabilistic  information  in  a  variety  of unsupervised  and super(cid:173) vised  learning  problems.  Often  the  parameters used  in  these  net(cid:173) works  need  to  be  learned  from examples.  Unfortunately,  estimat(cid:173) ing  the  parameters  via  exact  probabilistic  calculations  (i.e,  the  EM-algorithm)  is  intractable  even  for  networks  with  fairly  small  numbers of hidden  units.  We  propose  to  avoid the  infeasibility of  the  E step  by  bounding likelihoods instead of computing them ex(cid:173) actly.  We  introduce extended  and  complementary representations  for  these  networks  and  show  that  the  estimation  of the  network  parameters  can  be  made fast  (reduced  to  quadratic optimization)  by  performing the estimation in either of the  alternative domains.  The  complementary networks  can  be  used  for  continuous  density  estimation as  well.
Neural Control for Nonlinear Dynamic Systems
Authors: Ssu-Hsin Yu, Anuradha M. Annaswamy
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A neural network based approach is presented for controlling two distinct  types of nonlinear systems.  The first  corresponds to  nonlinear systems  with  parametric  uncertainties  where  the  parameters occur  nonlinearly.  The second corresponds to  systems  for  which  stabilizing control  struc(cid:173) tures cannot be determined.  The proposed neural controllers are shown  to result in closed-loop system stability under certain conditions.
Sample Complexity for Learning Recurrent Perceptron Mappings
Authors: Bhaskar DasGupta, Eduardo D. Sontag
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Recurrent  perceptron  classifiers  generalize the classical  perceptron  model.  They take into account  those correlations and dependences  among input  coordinates  which  arise  from  linear  digital  filtering.  This paper provides tight bounds on sample complexity associated  to the fitting of such  models to experimental data.
Parallel analog VLSI architectures for computation of heading direction and time-to-contact
Authors: Giacomo Indiveri, Jörg Kramer, Christof Koch
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  describe  two  parallel analog VLSI  architectures  that integrate  optical flow  data obtained from  arrays of elementary velocity  sen(cid:173) sors to estimate heading direction and time-to-contact.  For heading  direction  computation,  we  performed  simulations to  evaluate  the  most important qualitative properties of the optical flow  field  and  determine the best  functional operators for  the  implementation of  the  architecture.  For  time-to-contact we  exploited  the  divergence  theorem  to integrate  data from  all  velocity  sensors  present  in the  architecture  and average out  possible errors.
A Dynamical Systems Approach for a Learnable Autonomous Robot
Authors: Jun Tani, Naohiro Fukumura
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: This  paper discusses  how  a  robot  can  learn  goal-directed  naviga(cid:173) tion  tasks  using  local  sensory  inputs.  The  emphasis  is  that  such  learning  tasks  could  be  formulated  as  an  embedding  problem  of  dynamical  systems:  desired  trajectories  in  a  task space  should  be  embedded  into an  adequate  sensory-based internal  state  space  so  that an unique mapping from the internal state space to the motor  command could  be established.  The  paper shows that a  recurrent  neural network suffices in self-organizing such an adequate internal  state space  from  the temporal  sensory input.  In  our experiments,  using  a  real  robot  with  a  laser  range  sensor,  the  robot  navigated  robustly  by achieving dynamical coherence with  the environment.  It was  also  shown  that  such  coherence  becomes  structurally  sta(cid:173) ble  as  the  global  attractor is  self-organized  in the coupling of the  internal and the environmental dynamics. 
1
A Framework for Non-rigid Matching and Correspondence
Authors: Suguna Pappu, Steven Gold, Anand Rangarajan
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Matching feature point sets lies at the core of many approaches to  object recognition. We present a framework for non-rigid match(cid:173) ing that begins with a skeleton module, affine point matching,  and then integrates multiple features to improve correspondence  and develops an object representation based on spatial regions to  model local transformations. The algorithm for feature matching  iteratively updates the transformation parameters and the corre(cid:173) spondence solution, each in turn. The affine mapping is solved in  closed form, which permits its use for data of any dimension. The  correspondence is set via a method for two-way constraint satisfac(cid:173) tion, called softassign, which has recently emerged from the neural  network/statistical physics realm. The complexity of the non-rigid  matching algorithm with multiple features is the same as that of  the affine point matching algorithm. Results for synthetic and real  world data are provided for point sets in 2D and 3D, and for 2D  data with multiple types of features and parts.
Hierarchical Recurrent Neural Networks for Long-Term Dependencies
Authors: Salah El Hihi, Yoshua Bengio
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  have already shown  that extracting long-term dependencies  from  se(cid:173) quential  data is  difficult,  both for  determimstic dynamical systems  such  as  recurrent  networks,  and  probabilistic  models  such  as  hidden  Markov  models  (HMMs)  or input/output hidden  Markov  models  (IOHMMs).  In  practice,  to  avoid  this  problem,  researchers  have  used  domain  specific  a-priori  knowledge  to give  meaning  to  the hidden or  state variables rep(cid:173) resenting  past context.  In  this  paper,  we  propose  to use  a  more general  type  of a-priori  knowledge,  namely  that  the  temporal  dependencIes  are  structured  hierarchically.  This implies that  long-term  dependencies  are  represented  by  variables with  a long time scale.  This principle is  applied  to a recurrent network which includes delays and multiple time scales.  Ex(cid:173) periments confirm the advantages of such  structures.  A similar approach  is  proposed for  HMMs  and IOHMMs.
Memory-based Stochastic Optimization
Authors: Andrew W. Moore, Jeff G. Schneider
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In  this  paper  we  introduce  new  algorithms for  optimizing  noisy  plants in which each experiment is  very expensive.  The algorithms  build a global non-linear model of the expected output at the same  time as using Bayesian linear regression analysis of locally weighted  polynomial models.  The local  model answers  queries  about  confi(cid:173) dence,  noise,  gradient  and  Hessians,  and  use  them  to  make auto(cid:173) mated decisions similar to those made by a practitioner of Response  Surface  Methodology.  The global  and  local  models are  combined  naturally  as  a  locally  weighted  regression.  We  examine the  ques(cid:173) tion of whether the global model can really  help optimization, and  we  extend  it  to  the  case  of time-varying functions.  We  compare  the new algorithms with a  highly tuned higher-order stochastic op(cid:173) timization algorithm on  randomly-generated functions  and  a  sim(cid:173) ulated  manufacturing  task.  We  note  significant  improvements in  total regret , time to converge, and final  solution quality.
Handwritten Word Recognition using Contextual Hybrid Radial Basis Function Network/Hidden Markov Models
Authors: Bernard Lemarié, Michel Gilloux, Manuel Leroux
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A hybrid and contextual radial basis function networklhidden Markov  model  off-line  handwritten  word  recognition  system  is  presented.  The  task  assigned  to  the radial  basis  function  networks  is  the  estimation of  emission probabilities associated to Markov states. The model is contex(cid:173) tual because the estimation of emission probabilities takes  into account  the left context of the current image segment as represented by its pred(cid:173) ecessor in the sequence. The new system does not outperform the previ(cid:173) ous system without context but acts differently.
Universal Approximation and Learning of Trajectories Using Oscillators
Authors: Pierre Baldi, Kurt Hornik
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Natural  and  artificial  neural  circuits  must  be  capable  of travers(cid:173) ing  specific  state  space  trajectories.  A  natural  approach  to  this  problem  is  to  learn  the  relevant  trajectories  from  examples.  Un(cid:173) fortunately,  gradient  descent  learning  of complex  trajectories  in  amorphous  networks  is  unsuccessful.  We  suggest  a  possible  ap(cid:173) proach  where  trajectories  are  realized  by  combining simple oscil(cid:173) lators,  in  various  modular ways.  We  contrast  two  regimes  of fast  and slow oscillations.  In all cases,  we show that banks of oscillators  with bounded frequencies have universal approximation properties.  Open  questions  are also discussed  briefly. 
1 
INTRODUCTION:  TRAJECTORY LEARNING 
The  design  of artificial  neural  systems,  in  robotics  applications  and  others,  often  leads to the problem of constructing a recurrent neural network capable of producing  a particular trajectory, in the state space of its visible units.  Throughout evolution,  biological neural systems,  such  as  central  pattern generators,  have  also  been faced  with  similar  challenges.  A  natural  approach  to  tackle  this  problem  is  to  try  to  "learn"  the  desired  trajectory,  for  instance  through  a  process  of trial  and  error  and  subsequent  optimization.  Unfortunately,  gradient descent  learning of complex  trajectories  in  amorphous  networks  is  unsuccessful.  Here,  we  suggest  a  possible  approach where  trajectories are realized,  in a  modular and hierarchical fashion,  by  combining simple oscillators.  In particular,  we  show  that  banks of oscillators have  universal approximation properties. 

Also  with the Jet Propulsion  Laboratory,  California  Institute of Technology. 

452 
P.  BALDI,  K.  HORNIK 
To  begin  with,  we  can restrict  ourselves  to the simple case  of a  network  with one!  visible  linear  unit  and consider  the  problem of adjusting  the  network  parameters  in  a  way  that  the output unit  activity  u(t)  is  equal  to a  target function  I(t),  over  an interval of time [0, T].  The hidden units of the network  may be non-linear and  satisfy,  for  instance, one of the usual  neural  network charging equations such  as 
dUi 
dt  = - Ti  + L..JjWij/jUj(t - Tij),
Correlated Neuronal Response: Time Scales and Mechanisms
Authors: Wyeth Bair, Ehud Zohary, Christof Koch
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  have  analyzed the relationship between  correlated spike  count  and the peak in the cross-correlation of spike trains for  pairs of si(cid:173) multaneously recorded neurons  from  a  previous  study of area MT  in  the  macaque  monkey  (Zohary  et  al.,  1994).  We  conclude  that  common  input,  responsible  for  creating peaks  on  the order of ten  milliseconds  wide  in  the  spike  train  cross-correlograms  (CCGs),  is  also  responsible  for  creating  the  correlation  in  spike  count  ob(cid:173) served  at  the  two  second  time  scale  of  the  trial.  We  argue  that  both common excitation and  inhibition  may  play significant  roles  in establishing this  correlation.
Worst-case Loss Bounds for Single Neurons
Authors: David P. Helmbold, Jyrki Kivinen, Manfred K. Warmuth
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  analyze  and  compare  the  well-known  Gradient  Descent  algo(cid:173) rithm  and  a  new  algorithm,  called  the  Exponentiated  Gradient  algorithm, for  training  a  single neuron  with  an  arbitrary  transfer  function .  Both  algorithms are  easily  generalized  to  larger  neural  networks,  and the  generalization  of Gradient  Descent  is  the stan(cid:173) dard  back-propagation  algorithm.  In  this  paper  we  prove  worst(cid:173) case  loss  bounds  for  both  algorithms  in  the  single  neuron  case.  Since  local  minima make  it  difficult  to  prove  worst-case  bounds  for  gradient-based  algorithms,  we  must  use  a  loss  function  that  prevents  the  formation of spurious  local  minima.  We  define  such  a  matching  loss  function  for  any  strictly  increasing  differentiable  transfer  function  and  prove  worst-case  loss  bound  for  any  such  transfer  function  and  its  corresponding  matching loss.  For  exam(cid:173) ple,  the  matching loss  for  the  identity function  is  the  square  loss  and  the matching loss  for  the logistic sigmoid is  the entropic loss.  The different  structure  of the bounds for  the  two  algorithms indi(cid:173) cates that the new  algorithm out-performs Gradient Descent  when  the inputs contain a  large number of irrelevant components. 
310 
D. P.  HELMBOLD, J.  KIVINEN, M.  K.  WARMUTH
The Geometry of Eye Rotations and Listing's Law
Authors: Amir A. Handzel, Tamar Flash
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  analyse  the  geometry  of  eye  rotations,  and  in  particular  saccades,  using  basic  Lie  group  theory  and  differential  geome(cid:173) try.  Various  parameterizations  of  rotations  are  related  through  a  unifying  mathematical treatment,  and  transformations  between  co-ordinate  systems  are  computed  using  the  Campbell-Baker(cid:173) Hausdorff  formula.  Next,  we  describe  Listing's  law  by  means  of  the  Lie  algebra  so(3).  This  enables  us  to  demonstrate  a  direct  connection  to  Donders'  law, by  showing that  eye  orientations  are  restricted  to the quotient space  80(3)/80(2).  The latter is  equiv(cid:173) alent to the sphere S2, which is exactly the space of gaze directions.  Our analysis provides  a  mathematical framework  for  studying the  oculomotor system  and  could  also  be  extended  to  investigate  the  geometry of mUlti-joint arm movements.
When is an Integrate-and-fire Neuron like a Poisson Neuron?
Authors: Charles F. Stevens, Anthony M. Zador
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In the  Poisson neuron  model, the output is  a  rate-modulated Pois(cid:173) son  process  (Snyder  and  Miller,  1991);  the  time  varying  rate  pa(cid:173) rameter  ret)  is  an  instantaneous  function  G[.]  of  the  stimulus,  ret)  =  G[s(t)].  In  a  Poisson  neuron,  then,  ret)  gives  the  instan(cid:173) taneous firing  rate-the instantaneous probability of firing  at  any  instant t-and the output is  a  stochastic function  of the input.  In  part because of its great simplicity, this model is  widely used  (usu(cid:173) ally with  the addition of a  refractory  period), especially  in  in  vivo  single unit electrophysiological studies,  where  set)  is usually taken  to be  the value of some sensory  stimulus.  In the  integrate-and-fire  neuron model, by  contrast, the output is a filtered and thresholded  function  of the input:  the input is  passed  through a  low-pass filter  (determined by the membrane time constant T)  and integrated un(cid:173) til the membrane potential vet)  reaches threshold 8,  at which  point  vet)  is reset  to its initial value.  By contrast with the Poisson model,  in the integrate-and-fire model the ouput is a deterministic function  of the input.  Although the integrate-and-fire model is  a caricature  of real  neural  dynamics,  it  captures  many of the  qualitative  fea(cid:173) tures,  and  is  often  used  as  a  starting point for  conceptualizing the  biophysical behavior of single neurons.  Here we show how a slightly  modified  Poisson model can  be  derived  from  the integrate-and-fire  model with noisy inputs  yet)  =  set) + net).  In  the modified model,  the  transfer  function  G[.] is  a  sigmoid  (erf)  whose  shape  is  deter(cid:173) mined  by  the  noise  variance  /T~.  Understanding  the  equivalence  between  the  dominant  in  vivo  and  in  vitro  simple neuron  models  may help  forge  links between  the  two levels.
Examples of learning curves from a modified VC-formalism
Authors: Adam Kowalczyk, Jacek Szymanski, Peter L. Bartlett, Robert C. Williamson
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  examine  the  issue  of evaluation of model  specific  parameters  in  a  modified VC-formalism.  Two examples are analyzed:  the 2-dimensional  homogeneous  perceptron  and  the  I-dimensional  higher  order  neuron.  Both models are solved theoretically, and their learning curves are com(cid:173) pared against  true learning  curves.  It is  shown  that the  formalism  has  the  potential  to  generate  a  variety  of learning  curves,  including  ones  displaying ''phase transitions."
Implementation Issues in the Fourier Transform Algorithm
Authors: Yishay Mansour, Sigal Sahar
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: The  Fourier  transform of boolean  functions  has  come  to  play  an  important role in proving many important learnability results.  We  aim to demonstrate that the  Fourier transform techniques  are also  a  useful  and  practical  algorithm in  addition  to  being  a  powerful  theoretical  tool.  We  describe  the more prominent changes  we  have  introduced  to  the  algorithm,  ones  that  were  crucial  and  without  which  the  performance  of  the  algorithm  would  severely  deterio(cid:173) rate.  One of the benefits we  present  is  the confidence  level for each  prediction  which  measures  the  likelihood the prediction is  correct.
Classifying Facial Action
Authors: Marian Stewart Bartlett, Paul A. Viola, Terrence J. Sejnowski, Beatrice A. Golomb, Jan Larsen, Joseph C. Hager, Paul Ekman
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: The Facial Action Coding System, (FACS),  devised by Ekman and  Friesen (1978), provides an objective meanS for measuring the facial  muscle  contractions  involved in  a  facial  expression.  In  this  paper,  we  approach automated facial expression  analysis by detecting and  classifying  facial  actions.  We  generated  a  database  of over  1100  image sequences  of 24  subjects  performing over  150 distinct facial  actions  or  action  combinations.  We  compare  three  different  ap(cid:173) proaches  to classifying the facial  actions  in  these  images:  Holistic  spatial analysis based on principal components of graylevel images;  explicit measurement of local image features  such  as wrinkles;  and  template matching with  motion flow  fields.  On  a  dataset contain(cid:173) ing six individual actions and 20 subjects, these methods had 89%,  57%, and 85% performances respectively for generalization to novel  subjects.  When combined,  performance improved to 92%.