mtybilly/Instruct2DS huggingface dataset:
Generating Long-term Trajectories Using Deep Hierarchical Networks
Authors: Yisong Yue, Jennifer Hobbs Stephan Zheng
Published in: Advances in Neural Information Processing Systems 29  (NIPS 2016) (Main Conference Track), 2016

Abstract: We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations. For instance, in sports, agents often choose action sequences with long-term goals in mind, such as achieving a certain strategic position. Conventional policy learning approaches, such as those based on Markov decision processes, generally fail at learning cohesive long-term behavior in such high-dimensional state spaces, and are only effective when fairly myopic decision-making yields the desired behavior. The key difficulty is that conventional models are ``single-scale'' and only learn a single state-action policy. We instead propose a hierarchical policy class that automatically reasons about both long-term and short-term goals, which we instantiate as a hierarchical neural network. We showcase our approach in a case study on learning to imitate demonstrated basketball trajectories, and show that it generates significantly more realistic trajectories compared to non-hierarchical baselines as judged by professional sports analysts.
Natural-Parameter Networks: A Class of Probabilistic Neural Networks
Authors: Xingjian SHI, Dit-Yan Yeung Hao Wang
Published in: Advances in Neural Information Processing Systems 29  (NIPS 2016) (Main Conference Track), 2016

Abstract: Neural networks (NN) have achieved state-of-the-art performance in various applications. Unfortunately in applications where training data is insufficient, they are often prone to overfitting. One effective way to alleviate this problem is to exploit the Bayesian approach by using Bayesian neural networks (BNN). Another shortcoming of NN is the lack of flexibility to customize different distributions for the weights and neurons according to the data, as is often done in probabilistic graphical models. To address these problems, we propose a class of probabilistic neural networks, dubbed natural-parameter networks (NPN), as a novel and lightweight Bayesian treatment of NN. NPN allows the usage of arbitrary exponential-family distributions to model the weights and neurons. Different from traditional NN and BNN, NPN takes distributions as input and goes through layers of transformation before producing distributions to match the target output distributions. As a Bayesian treatment, efficient backpropagation (BP) is performed to learn the natural parameters for the distributions over both the weights and neurons. The output distributions of each layer, as byproducts, may be used as second-order representations for the associated tasks such as link prediction. Experiments on real-world datasets show that NPN can achieve state-of-the-art performance.
Minimizing Quadratic Functions in Constant Time
Authors: Yuichi Yoshida Kohei Hayashi
Published in: Advances in Neural Information Processing Systems 29  (NIPS 2016) (Main Conference Track), 2016

Abstract: A sampling-based optimization method for quadratic functions is   proposed. Our method approximately solves the following   $n$-dimensional quadratic minimization problem in constant time,   which is independent of $n$:   $z^*=\min_{\bv \in \bbR^n}\bracket{\bv}{A \bv} +   n\bracket{\bv}{\diag(\bd)\bv} + n\bracket{\bb}{\bv}$,   where $A \in \bbR^{n \times n}$ is a matrix and $\bd,\bb \in \bbR^n$   are vectors. Our theoretical analysis specifies the number of   samples $k(\delta, \epsilon)$ such that the approximated solution   $z$ satisfies $|z - z^*| = O(\epsilon n^2)$ with probability   $1-\delta$. The empirical performance (accuracy and runtime) is   positively confirmed by numerical experiments.
SIMPLIFYING NEURAL NETS BY DISCOVERING FLAT MINIMA
Authors: Jürgen Schmidhuber Sepp Hochreiter
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We present a new algorithm for finding low complexity networks  with high generalization capability. The algorithm searches for  large connected regions of so-called ''fiat'' minima of the error func(cid:173) tion. In the weight-space environment of a "flat" minimum, the  error remains approximately constant. Using an MDL-based ar(cid:173) gument, flat minima can be shown to correspond to low expected  overfitting. Although our algorithm requires the computation of  second order derivatives, it has backprop's order of complexity.  Experiments with feedforward and recurrent nets are described. In  an application to stock market prediction, the method outperforms  conventional backprop, weight decay, and "optimal brain surgeon" .
A Mixture Model System for Medical and Machine Diagnosis
Authors: Terrence J. Sejnowski Magnus Stensmo
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Diagnosis of human disease or machine fault is a missing data problem  since many variables are initially unknown. Additional information needs  to be obtained. The j oint probability distribution of the data can be used to  solve this problem. We model this with mixture models whose parameters  are estimated by the EM algorithm.  This gives the benefit that missing  data in the database itself can also be handled correctly.  The request for  new information to refine the diagnosis is performed using the maximum  utility  principle.  Since  the  system  is  based  on  learning  it  is  domain  independent and less labor intensive than expert systems or probabilistic  networks.  An example using a heart disease database is presented.
Learning with Preknowledge: Clustering with Point and Graph Matching Distance Measures
Authors: Anand Rangarajan, Eric Mjolsness Steven Gold
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Prior constraints are imposed upon a learning problem in the form  of distance measures. Prototypical 2-D point sets and graphs are  learned by clustering with point matching and graph matching dis(cid:173) tance measures. The point matching distance measure is approx.  invariant under affine transformations - translation, rotation, scale  and shear - and permutations. It operates between noisy images  with missing and spurious points. The graph matching distance  measure operates on weighted graphs and is invariant under per(cid:173) mutations. Learning is formulated as an optimization problem .  Large objectives so formulated ('" million variables) are efficiently  minimized using a combination of optimization techniques - alge(cid:173) braic transformations, iterative projective scaling, clocked objec(cid:173) tives, and deterministic annealing.
Learning Local Error Bars for Nonlinear Regression
Authors: Andreas S. Weigend David A. Nix
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  present a  new  method for  obtaining  local  error bars for  nonlinear  regression,  i.e.,  estimates of the confidence in predicted values that de(cid:173) pend on the input.  We approach this problem by applying a maximum(cid:173) likelihood framework to an assumed distribution of errors.  We demon(cid:173) strate our method first on computer-generated data with locally varying,  normally distributed target noise.  We then apply it to laser data from the  Santa Fe  Time  Series Competition where the underlying system noise is  known quantization error and the error bars give local estimates of model  misspecification.  In  both  cases,  the method also provides a  weighted(cid:173) regression effect that improves generalization performance. 
1  Learning Local Error Bars Using a Maximum Likelihood 
Framework:  Motivation, Concept, and Mechanics 
Feed-forward artificial neural networks used for nonlinear regression can be interpreted as  predicting the  mean of the  target distribution as  a function  of (conditioned on)  the  input  pattern (e.g., Buntine & Weigend, 1991; Bishop, 1994), typically using one linear output unit  per output variable. If parameterized, this conditional target distribution (CID) may also be 
·http://www.cs.colorado.edu/~andreas/Home.html. 
This  paper  is  available  with  figures  in  colors  as  ftp://ftp.cs.colorado.edu/pub/  Time-Series/MyPapers/nix.weigenCLnips7.ps.Z  . 
490 
David A.  Nix,  Andreas S.  Weigend 
viewed as an error model (Rumelhart et al.,  1995).  Here, we present a simple method that  provides higher-order information about the cm than simply the mean.  Such additional  information could come from  attempting  to  estimate  the  entire cm with  connectionist  methods (e.g., "Mixture Density Networks," Bishop, 1994; "fractional binning, "Srivastava  & Weigend,  1994) or with non-connectionist methods such as  a Monte Carlo on a hidden  Markov model (Fraser & Dimitriadis, 1994). While non-parametric estimates of the shape  of a  C1D require large quantities of data,  our less data-hungry method (Weigend & Nix,  1994) assumes a specific parameterized form of the C1D (e.g., Gaussian) and gives us the  value of the error bar (e.g., the width of the Gaussian) by finding those parameters which  maximize the likelihood that the target data was generated by a particular network model.  In this paper we derive the specific update rules for the Gaussian case.  We  would like to  emphasize, however, that any parameterized unimodal distribution can be used for the em  in the method presented here. 
j------------,  I  I------T-------,  ,  I  A  I  A2,.  )  o y(x)  0  cr IX  /\  i  '.
Reinforcement Learning Methods for Continuous-Time Markov Decision Problems
Authors: Michael O. Duff Steven J. Bradtke
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Semi-Markov  Decision  Problems  are  continuous  time  generaliza(cid:173) tions  of discrete  time  Markov  Decision  Problems.  A  number  of  reinforcement  learning  algorithms  have  been  developed  recently  for  the  solution  of Markov  Decision  Problems,  based on  the ideas  of asynchronous dynamic programming and stochastic approxima(cid:173) tion.  Among these  are TD(,x), Q-Iearning,  and Real-time Dynamic  Programming.  After  reviewing  semi-Markov  Decision  Problems  and Bellman's optimality equation in  that context,  we  propose al(cid:173) gorithms similar  to those  named above,  adapted to the solution  of  semi-Markov Decision  Problems.  We demonstrate these algorithms  by  applying  them to the  problem of determining  the  optimal con(cid:173) trol for  a  simple  queueing  system.  We  conclude  with  a  discussion  of circumstances  under which these  algorithms may be usefully  ap(cid:173) plied.
Connectionist Speaker Normalization with Generalized Resource Allocating Networks
Authors: Diego Giuliani, Edmondo Trentin Cesare Furlanello
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The paper presents  a  rapid speaker-normalization technique based  on  neural  network  spectral  mapping.  The  neural  network  is  used  as a front-end  of a  continuous speech  recognition system  (speaker(cid:173) dependent,  HMM-based) to normalize the input acoustic data from  a  new  speaker.  The  spectral  difference  between  speakers  can  be  reduced  using  a  limited amount  of new  acoustic  data (40  phonet(cid:173) ically  rich  sentences).  Recognition  error  of phone  units  from  the  acoustic-phonetic  continuous  speech  corpus  APASCI  is  decreased  with an adaptability ratio of 25%.  We used  local basis networks of  elliptical  Gaussian  kernels,  with  recursive  allocation  of units  and  on-line  optimization of parameters  (GRAN model).  For  this  ap(cid:173) plication,  the  model included  a  linear  term.  The results  compare  favorably  with  multivariate linear  mapping based  on  constrained  orthonormal transformations.
A Novel Reinforcement Model of Birdsong Vocalization Learning
Authors: Terrence J. Sejnowski Kenji Doya
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Songbirds learn to imitate a tutor song through auditory and motor learn(cid:173) ing.  We  have developed a theoretical framework  for song learning that  accounts  for response properties of neurons that have been  observed in  many  of the nuclei  that are involved in  song learning.  Specifically,  we  suggest that the anteriorforebrain pathway, which is not needed for song  production  in  the  adult  but is  essential  for  song  acquisition,  provides  synaptic perturbations and adaptive evaluations for syllable vocalization  learning.  A computer model based on reinforcement learning was  con(cid:173) structed  that could replicate a real  zebra finch  song with 90% accuracy  based on a spectrographic measure.  The second generation of the bird(cid:173) song model replicated the tutor song with 96% accuracy.
Bias, Variance and the Combination of Least Squares Estimators
Authors: Ronny Meir
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We consider the effect of combining several least squares estimators  on  the expected  performance of a  regression  problem.  Computing  the exact bias and  variance curves  as  a function of the sample size  we  are able to quantitatively compare the effect of the combination  on the bias and variance separately, and thus on the expected  error  which  is  the sum of the  two.  Our exact  calculations,  demonstrate  that the combination of estimators is particularly useful  in the case  where the data set is small and noisy and the function to be learned  is  unrealizable.  For  large  data sets  the  single  estimator produces  superior  results.  Finally,  we  show  that  by  splitting  the  data set  into  several  independent  parts  and  training  each  estimator  on  a  different subset,  the performance can in some cases  be significantly  improved. 
Key  words:  Bias,  Variance,  Least  Squares,  Combination.
Hierarchical Mixtures of Experts Methodology Applied to Continuous Speech Recognition
Authors: Richard M. Schwartz, Jason J. Sroka, John Makhoul Ying Zhao
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: In  this  paper,  we  incorporate  the  Hierarchical  Mixtures  of Experts  (HME)  method  of probability  estimation,  developed  by  Jordan  [1],  into  an  HMM(cid:173) based  continuous  speech  recognition  system.  The  resulting  system  can  be  thought of as a continuous-density HMM system, but instead of using gaussian  mixtures,  the HME system employs a large set of hierarchically organized but  relatively  small  neural  networks  to  perform the probability density estimation.  The  hierarchical  structure  is  reminiscent  of  a  decision  tree  except  for  two  important differences:  each  "expert" or neural  net  performs a  "soft" decision  rather than a hard decision, and,  unlike ordinary decision trees,  the parameters  of all  the  neural  nets  in  the  HME  are  automatically  trainable  using  the  EM  algorithm.  We  report results on  the ARPA  5,OOO-word  and 4O,OOO-word Wall  Street Journal  corpus using  HME  models.
A Comparison of Discrete-Time Operator Models for Nonlinear System Identification
Authors: Ah Chung Tsoi Andrew D. Back
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We present a unifying view of discrete-time operator models used in the  context of finite word length linear signal processing.  Comparisons are  made between the recently presented gamma operator model, and the delta  and rho operator models for performing nonlinear system identification  and prediction using neural networks.  A new model based on an adaptive  bilinear  transformation  which  generalizes  all  of the  above  models  is  presented.
Learning Many Related Tasks at the Same Time with Backpropagation
Authors: Rich Caruana
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Hinton  [6]  proposed  that  generalization  in  artificial  neural  nets  should improve  if nets  learn  to represent the  domain's underlying  regularities.  Abu-Mustafa's  hints work  [1]  shows  that the  outputs  of a  backprop  net  can  be  used  as  inputs  through  which  domain(cid:173) specific  information can be given to the net.  We extend these ideas  by showing that a  backprop net learning many related tasks at the  same time can use these  tasks  as inductive  bias for  each other and  thus learn better.  We  identify five  mechanisms by which multitask  backprop improves generalization and give  empirical evidence that  multi task backprop generalizes  better  in real  domains.
Multidimensional Scaling and Data Clustering
Authors: Joachim Buhmann Thomas Hofmann
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Visualizing and structuring pairwise dissimilarity data are difficult combinatorial op(cid:173) timization problems known as multidimensional scaling or pairwise data clustering.  Algorithms for embedding dissimilarity data set in a Euclidian space, for clustering  these data and for actively selecting data to support the clustering process are discussed  in the maximum entropy framework. Active data selection provides a strategy to discover  structure in a data set efficiently with partially unknown data.
Predicting the Risk of Complications in Coronary Artery Bypass Operations using Neural Networks
Authors: Linda Kukolich, David Shahian Richard P Lippmann
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Experiments  demonstrated  that  sigmoid  multilayer  perceptron  (MLP)  networks  provide  slightly  better  risk  prediction  than  conventional  logistic  regression  when  used  to  predict  the  risk  of death,  stroke,  and  renal  failure  on  1257  patients  who  underwent  coronary  artery  bypass  operations at the Lahey Clinic. MLP networks with no hidden layer and  networks  with one hidden  layer were trained  using  stochastic  gradient  descent with early stopping. MLP networks and logistic regression used  the  same  input  features  and  were  evaluated  using  bootstrap  sampling  with  50  replications.  ROC  areas  for  predicting  mortality  using  preoperative  input  features  were  70.5%  for  logistic  regression  and  76.0%  for  MLP  networks.  Regularization  provided  by  early  stopping  was  an  important  component  of improved  perfonnance.  A  simplified  approach  to  generating  confidence  intervals  for  MLP risk  predictions  using  an  auxiliary  "confidence  MLP"  was  developed.  The  confidence  MLP  is  trained  to  reproduce  confidence  intervals  that  were  generated  during  training  using  the  outputs  of  50  MLP  networks  trained  with  different bootstrap samples.
Spatial Representations in the Parietal Cortex May Use Basis Functions
Authors: Terrence J. Sejnowski Alexandre Pouget
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The  parietal  cortex  is  thought  to  represent  the  egocentric  posi(cid:173) tions  of objects  in  particular  coordinate systems.  We  propose  an  alternative  approach  to  spatial  perception  of objects  in  the  pari(cid:173) etal  cortex from  the  perspective  of sensorimotor  transformations.  The responses of single parietal neurons can be modeled as  a gaus(cid:173) sian  function  of retinal  position  multiplied by  a  sigmoid function  of eye  position,  which  form  a  set of basis functions.  We  show  here  how  these  basis functions  can  be  used  to generate  receptive  fields  in  either  retinotopic or head-centered  coordinates by simple linear  transformations.  This raises the possibility that the parietal cortex  does  not  attempt to compute the positions of objects  in  a  partic(cid:173) ular  frame  of  reference  but  instead  computes  a  general  purpose  representation  of the retinal  location and eye  position from  which  any  transformation can  be  synthesized  by  direct  projection.  This  representation  predicts that hemineglect,  a  neurological syndrome  produced  by  parietal lesions,  should  not  be  confined  to egocentric  coordinates,  but should be observed in multiple frames of reference  in  single patients,  a  prediction supported  by several experiments. 
158 
Alexandre Pouget,  Terrence J.  Sejnowski
Reinforcement Learning Algorithm for Partially Observable Markov Decision Problems
Authors: Satinder P. Singh, Michael I. Jordan Tommi Jaakkola
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Increasing  attention has  been  paid to reinforcement  learning  algo(cid:173) rithms  in  recent  years,  partly  due  to  successes  in  the  theoretical  analysis  of their  behavior in  Markov  environments.  If the  Markov  assumption  is  removed,  however,  neither  generally  the  algorithms  nor  the  analyses  continue  to  be  usable.  We  propose  and  analyze  a  new  learning  algorithm  to  solve  a  certain  class  of  non-Markov  decision  problems.  Our  algorithm  applies  to  problems  in  which  the  environment  is  Markov,  but  the  learner  has  restricted  access  to  state  information.  The  algorithm involves  a  Monte-Carlo  pol(cid:173) icy evaluation combined with a  policy improvement method that is  similar to  that  of Markov  decision  problems  and  is  guaranteed  to  converge to a local maximum.  The algorithm operates in the space  of stochastic  policies,  a  space  which  can  yield  a  policy  that  per(cid:173) forms  considerably  better  than any deterministic policy.  Although  the  space  of stochastic  policies  is  continuous-even  for  a  discrete  action space-our algorithm is computationally tractable. 
346 
Tommi  Jaakkola,  Satinder P.  Singh,  Michaell.  Jordan
FINANCIAL APPLICATIONS OF LEARNING FROM HINTS
Authors: Yaser S. Abu-Mostafa
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The basic paradigm for learning in neural networks is 'learning from  examples' where a training set of input-output examples is used to  teach the network the target function. Learning from hints is a gen(cid:173) eralization of learning from examples where additional information  about the target function can be incorporated in the same learning  process. Such information can come from common sense rules or  special expertise. In financial market applications where the train(cid:173) ing data is very noisy, the use of such hints can have a decisive  advantage. We demonstrate the use of hints in foreign-exchange  trading of the U.S. Dollar versus the British Pound, the German  Mark, the Japanese Yen, and the Swiss Franc, over a period of 32  months. We explain the general method of learning from hints and  how it can be applied to other markets. The learning model for  this method is not restricted to neural networks.
An Auditory Localization and Coordinate Transform Chip
Authors: Timothy K. Horiuchi
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The  localization  and  orientation  to  various  novel  or  interesting  events  in  the  environment  is  a  critical  sensorimotor  ability in  all  animals,  predator  or  prey.  In  mammals,  the  superior  colliculus  (SC)  plays  a  major  role  in  this  behavior,  the  deeper  layers  ex(cid:173) hibiting topographically mapped responses to visual, auditory, and  somatosensory  stimuli.  Sensory  information arriving from  differ(cid:173) ent modalities should  then be  represented  in  the same coordinate  frame.  Auditory  cues,  in  particular,  are  thought to be  computed  in head-based coordinates which must then be transformed to reti(cid:173) nal coordinates.  In this paper, an analog VLSI implementation for  auditory localization in the azimuthal plane is described  which ex(cid:173) tends  the  architecture  proposed for  the  barn owl to a  primate eye  movement system  where  further  transformation  is  required.  This  transformation is intended to model the projection in primates from  auditory  cortical areas to the deeper layers of the primate superior  colliculus.  This  system  is  interfaced  with  an  analog  VLSI-based  saccadic eye movement system also being constructed  in our labo(cid:173) ratory.
Limits on Learning Machine Accuracy Imposed by Data Quality
Authors: L. D. Jackel, Wan-Ping Chiang Corinna Cortes
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Random  errors  and  insufficiencies  in  databases  limit  the  perfor(cid:173) mance  of any classifier  trained  from  and applied  to the  database.  In this paper we  propose a method to estimate the limiting perfor(cid:173) mance of classifiers imposed by the database.  We demonstrate this  technique  on  the  task  of predicting  failure  in  telecommunication  paths.
Interference in Learning Internal Models of Inverse Dynamics in Humans
Authors: Tom Brashers-Krug, Ferdinando A. Mussa-Ivaldi Reza Shadmehr
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Experiments were  performed  to  reveal  some of the  computational  properties  of the  human  motor  memory  system.  We  show  that  as  humans  practice  reaching  movements while  interacting  with  a  novel  mechanical environment, they learn an  internal model of the  inverse dynamics of that environment.  Subjects show recall of this  model  at testing  sessions  24  hours  after  the  initial practice.  The  representation  of the internal model in  memory is  such  that there  is  interference  when  there  is  an  attempt  to  learn  a  new  inverse  dynamics  map  immediately after  an  anticorrelated  mapping  was  learned.  We  suggest  that  this  interference  is  an  indication  that  the same computational elements  used  to encode  the first  inverse  dynamics  map  are  being  used  to  learn  the  second  mapping.  We  predict  that this leads to a forgetting  of the initially learned skill.
Optimal Movement Primitives
Authors: Terence D. Sanger
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The  theory  of Optimal  Unsupervised  Motor  Learning  shows  how  a  network  can  discover  a  reduced-order  controller for  an  unknown  nonlinear system by  representing only  the most significant  modes.  Here,  I extend the theory  to apply to command sequences,  so that  the most significant  components discovered  by  the  network  corre(cid:173) spond  to  motion  "primitives".  Combinations of these  primitives  can  be  used  to  produce  a  wide  variety  of  different  movements.  I  demonstrate  applications  to  human  handwriting  decomposition  and  synthesis,  as  well  as  to  the  analysis  of  electrophysiological  experiments  on  movements resulting  from  stimulation of the  frog  spinal cord.
Using a Saliency Map for Active Spatial Selective Attention: Implementation & Initial Results
Authors: Dean A. Pomerleau Shumeet Baluja
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: In many vision based tasks, the ability to focus attention on the important  portions of a scene is crucial for good performance on the tasks. In this paper  we present a simple method of achieving spatial selective attention through  the use of a saliency map. The saliency map indicates which regions of the  input retina are important for performing the task. The saliency map is cre(cid:173) ated through predictive auto-encoding. The performance of this method is  demonstrated on two simple tasks which have multiple very strong distract(cid:173) ing features in the input retina. Architectural extensions and application  directions for this model are presented.
Factorial Learning and the EM Algorithm
Authors: Zoubin Ghahramani
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Many real world learning problems are best characterized by an  interaction of multiple independent causes or factors. Discover(cid:173) ing such causal structure from the data is the focus of this paper.  Based on Zemel and Hinton's cooperative vector quantizer (CVQ)  architecture, an unsupervised learning algorithm is derived from  the Expectation-Maximization (EM) framework. Due to the com(cid:173) binatorial nature of the data generation process, the exact E-step  is computationally intractable. Two alternative methods for com(cid:173) puting the E-step are proposed: Gibbs sampling and mean-field  approximation, and some promising empirical results are presented.
Pairwise Neural Network Classifiers with Probabilistic Outputs
Authors: Stefan Knerr, Léon Personnaz, Gérard Dreyfus David Price
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Multi-class  classification  problems  can  be  efficiently  solved  by  partitioning the  original problem into  sub-problems  involving  only  two  classes:  for each pair of classes,  a (potentially small)  neural  network is  trained  using  only  the  data  of  these  two  classes.  We  show  how  to  combine the outputs of the  two-class  neural  networks in order to  obtain  posterior probabilities for  the class decisions.  The resulting probabilistic  pairwise classifier is part of a handwriting recognition  system which  is  currently applied to check reading. We present results on real world data  bases and show that, from a practical point of view, these results compare  favorably to other neural network approaches.
Real-Time Control of a Tokamak Plasma Using Neural Networks
Authors: Paul S. Haynes, Mike E U Smith, Tom N. Todd, David L. Trotman, Colin G. Windsor Chris M. Bishop
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: This  paper  presents  results  from  the  first  use  of neural  networks  for  the real-time feedback  control  of high  temperature plasmas in  a  tokamak fusion  experiment.  The tokamak is  currently  the  prin(cid:173) cipal  experimental  device  for  research  into  the  magnetic  confine(cid:173) ment  approach  to  controlled  fusion.  In  the  tokamak,  hydrogen  plasmas,  at  temperatures  of  up  to  100  Million  K,  are  confined  by  strong  magnetic  fields.  Accurate  control  of the  position  and  shape  of the plasma boundary  requires  real-time feedback  control  of the magnetic field  structure on  a  time-scale of a  few  tens of mi(cid:173) croseconds.  Software simulations have demonstrated that a  neural  network  approach  can  give  significantly  better  performance  than  the linear technique currently  used  on  most tokamak experiments.  The practical  application of the neural  network  approach  requires  high-speed  hardware,  for  which  a  fully  parallel  implementation of  the  multilayer perceptron,  using  a  hybrid  of digital and  analogue  technology,  has been  developed.
An Actor/Critic Algorithm that is Equivalent to Q-Learning
Authors: Andrew G. Barto Robert H. Crites
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We prove the convergence of an actor/critic algorithm that is equiv(cid:173) alent  to Q-Iearning by construction.  Its equivalence is  achieved by  encoding Q-values within  the  policy  and value function  of the  ac(cid:173) tor and critic.  The resultant actor/critic algorithm is novel in two  ways:  it  updates the critic  only  when  the  most  probable  action  is  executed from  any given state, and it  rewards  the actor using cri(cid:173) teria that depend on the relative probability of the action that was  executed.
Template-Based Algorithms for Connectionist Rule Extraction
Authors: Michael Mozer Jay A. Alexander
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Casting  neural  network  weights  in  symbolic  terms  is  crucial  for  interpreting  and explaining  the  behavior of a network.  Additionally,  in  some  domains,  a  symbolic  description  may  lead  to  more  robust  generalization.  We  present  a  principled  approach  to  symbolic  rule  extraction  based  on  the  notion  of  weight  templates,  parameterized  regions of weight space corresponding to specific symbolic expressions.  With  an  appropriate  choice  of representation,  we  show  how  template  parameters  may  be  efficiently  identified  and  instantiated  to  yield  the  optimal match to a unit's actual weights. Depending on the requirements  of  the  application  domain,  our  method  can  accommodate  arbitrary  disjunctions  and  conjunctions  with  O(k)  complexity,  simple  n-of-m  expressions with O( k!) complexity, or a more general class of recursive  n-of-m  expressions  with  O(k!)  complexity,  where  k  is  the  number  of  inputs  to  a  unit.  Our method  of rule  extraction  offers  several  benefits  over alternative approaches in the literature, and simulation results on  a  variety of problems demonstrate its effectiveness.
Reinforcement Learning with Soft State Aggregation
Authors: Tommi Jaakkola, Michael I. Jordan Satinder P. Singh
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: It is  widely  accepted  that the use  of more compact representations  than lookup tables is crucial to scaling reinforcement learning (RL)  algorithms to real-world problems.  Unfortunately almost all of the  theory  of reinforcement  learning assumes  lookup table representa(cid:173) tions.  In  this  paper  we  address  the  pressing  issue  of  combining  function  approximation and RL,  and present  1)  a function approx(cid:173) imator  based  on  a  simple extension  to  state  aggregation  (a  com(cid:173) monly  used  form  of  compact  representation),  namely  soft  state  aggregation,  2)  a  theory  of convergence for  RL  with arbitrary, but  fixed,  soft  state  aggregation,  3)  a  novel  intuitive  understanding of  the effect  of state aggregation on online RL, and 4)  a new heuristic  adaptive  state aggregation algorithm that finds  improved compact  representations  by  exploiting the  non-discrete  nature  of soft  state  aggregation.  Preliminary empirical results  are  also  presented.
A Connectionist Technique for Accelerated Textual Input: Letting a Network Do the Typing
Authors: Dean Pomerleau
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Each year people spend a huge amount of time typing. The text people type  typically contains a tremendous amount of redundancy due to predictable  word  usage  patterns  and  the  text's  structure.  This  paper  describes  a  neural network system call AutoTypist that monitors a person's typing and  predicts what will be entered  next.  AutoTypist displays the most likely  subsequent word to the typist, who can accept it with a single keystroke,  instead of typing it in its entirety.  The multi-layer perceptron at the heart  of Auto'JYpist adapts its predictions of likely subsequent text to the user's  word usage pattern,  and to the characteristics of the text currently being  typed.  Increases in typing speed of 2-3% when typing English prose and  10-20% when typing C code have been demonstrated using the system,  suggesting a potential time savings of more than 20 hours per user per year.  In addition to increasing typing speed, AutoTypist reduces the number of  keystrokes a user must type by a similar amount (2-3% for English,  10- 20% for computer programs).  This keystroke savings has the potential to  significantly reduce the frequency  and severity of repeated stress injuries  caused by typing, which are the most common injury suffered in today's  office environment.
Advantage Updating Applied to a Differential Game
Authors: Leemon C. Baird III, A. Harry Klopf Mance E. Harmon
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: An application of reinforcement learning to a linear-quadratic, differential  game  is  presented.  The reinforcement learning  system  uses  a  recently  developed  algorithm,  the residual gradient form  of advantage updating.  The  game  is a  Markov  Decision  Process  (MDP)  with continuous  time,  states, and actions, linear dynamics, and a quadratic cost function.  The  game consists of two players, a missile and a plane;  the missile pursues  the plane and  the plane evades the  missile.  The reinforcement learning  algorithm for optimal control is modified for differential games in order to  find the minimax point, rather than  the maximum.  Simulation results are  compared  to  the  optimal  solution,  demonstrating  that  the  simulated  reinforcement learning  system  converges  to  the optimal  answer.  The  performance of both the residual gradient and non-residual gradient forms  of advantage updating and Q-learning are compared.  The results show that  advantage  updating  converges faster  than  Q-learning in  all  simulations.  The results also show advantage updating converges regardless of the time  step duration; Q-learning is  unable to converge as the  time step duration  ~rows small. 
U.S .A.F.  Academy,  2354  Fairchild  Dr.  Suite 6K4l,  USAFA,  CO  80840-6234 
354 
Mance E.  Hannon,  Leemon C.  Baird ll/, A.  Harry Klopf 
1  ADVANTAGE  UPDATING 
The advantage updating algorithm (Baird, 1993) is a reinforcement learning algorithm in  which  two types  of information  are stored.  For each  state x, the  value V(x)  is  stored,  representing an estimate of the total discounted return expected when starting in  state x  and performing optimal actions.  For each state x and action u, the advantage, A(x,u), is  stored, representing  an  estimate of the  degree  to  which  the expected  total  discounted  reinforcement  is  increased  by  performing  action  u  rather  than  the  action  currently  considered best.  The optimal value function V* (x) represents the true value of each state.  The optimal advantage function A * (x,u)  will be zero if u is the optimal action (because u  confers no advantage relative to itself) and A * (x,u) will be negative for any suboptimal u  (because a suboptimal action has a negative advantage relative to  the best action).  The  optimal advantage function A * can be defined in terms of the optimal value function v*: 
A(x,u) = ~[RN(X,U)- V(x)+ rNV*(x')]
Pattern Playback in the 90s
Authors: Malcolm Slaney
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Deciding  the  appropriate  representation  to  use  for  modeling  human  auditory processing  is  a critical issue  in  auditory  science. While  engi(cid:173) neers have successfully performed many single-speaker tasks with LPC  and  spectrogram  methods,  more  difficult  problems  will  need  a  richer  representation. This paper describes a powerful auditory representation  known as the correlogram and shows how this non-linear representation  can be converted back into  sound, with no  loss  of perceptually  impor(cid:173) tant information.  The correlogram  is  interesting because  it is  a  neuro(cid:173) physiologically  plausible  representation  of  sound.  This  paper  shows  improved  methods  for  spectrogram  inversion  (conventional  pattern  playback), inversion of a cochlear model, and inversion of the  correlo(cid:173) gram representation.
An Analog Neural Network Inspired by Fractal Block Coding
Authors: Andreas G. Andreou Fernando J. Pineda
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We consider the problem of decoding block coded data, using a physical  dynamical system. We sketch out a decompression algorithm for fractal  block  codes  and  then  show  how  to  implement  a  recurrent  neural  network  using  physically  simple  but highly-nonlinear,  analog  circuit  models of neurons and synapses. The nonlinear system has many fixed  points, but we have at our disposal a procedure to choose the parameters  in  such a way  that only one solution, the desired solution, is stable. As  a  partial  proof of the  concept,  we  present experimental  data  from  a  small system a 16-neuron analog CMOS chip fabricated in a 2m analog  p-well process. This chip operates in the subthreshold regime and,  for  each choice of parameters, converges to a unique stable state. Each state  exhibits a qualitatively fractal shape.
Phase-Space Learning
Authors: Garrison W. Cottrell Fu-Sheng Tsung
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Existing recurrent  net learning algorithms are inadequate.  We  in(cid:173) troduce the conceptual framework of viewing recurrent  training as  matching vector fields of dynamical systems in phase space.  Phase(cid:173) space  reconstruction  techniques  make  the  hidden  states  explicit,  reducing  temporal  learning  to  a  feed-forward  problem.  In  short,  we  propose  viewing  iterated  prediction  [LF88]  as  the  best  way  of  training  recurrent  networks  on  deterministic  signals.  Using  this  framework,  we  can  train  multiple trajectories,  insure  their  stabil(cid:173) ity,  and  design  arbitrary dynamical systems.
An experimental comparison of recurrent neural networks
Authors: C. Lee Giles Bill G. Horne
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Many  different  discrete-time  recurrent  neural  network  architec(cid:173) tures  have  been  proposed.  However,  there  has  been  virtually  no  effort  to compare these arch:tectures experimentally.  In  this paper  we  review  and categorize many of these architectures and compare  how  they  perform on various classes  of simple problems including  grammatical inference  and nonlinear system identification.
Inferring Ground Truth from Subjective Labelling of Venus Images
Authors: Usama M. Fayyad, Michael C. Burl, Pietro Perona, Pierre Baldi Padhraic Smyth
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: In  remote sensing  applications  "ground-truth"  data is  often  used  as  the  basis for  training  pattern  recognition  algorithms  to  gener(cid:173) ate  thematic  maps  or  to  detect  objects  of  interest.  In  practical  situations, experts may visually examine the images and provide a  subjective noisy  estimate of the  truth.  Calibrating the  reliability  and bias  of expert labellers is  a  non-trivial problem.  In this paper  we  discuss  some  of  our  recent  work  on  this  topic  in  the  context  of  detecting  small  volcanoes  in  Magellan  SAR  images  of  Venus.  Empirical results (using the Expectation-Maximization procedure)  suggest  that  accounting  for  subjective  noise  can  be  quite  signifi(cid:173) cant in terms of quantifying both human and algorithm detection  performance.
Learning Prototype Models for Tangent Distance
Authors: Patrice Simard Trevor Hastie
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Simard,  LeCun  & Denker  (1993)  showed  that the performance of  nearest-neighbor  classification  schemes  for  handwritten  character  recognition  can  be  improved  by  incorporating  invariance  to  spe(cid:173) the  so  cific  transformations in  the  underlying  distance  metric  - called  tangent  distance.  The  resulting  classifier,  however,  can  be  prohibitively slow and memory intensive due to the large amount of  prototypes that need to be stored and used in the distance compar(cid:173) isons.  In this  paper we  develop  rich  models for  representing  large  subsets of the prototypes.  These models are either used singly per  class,  or  as  basic building blocks  in conjunction with the K-means  clustering  algorithm. 
*This work was performed while Trevor Hastie was a member of the Statistics and Data 
Analysis  Research  Group,  AT&T Bell  Laboratories,  Murray  Hill,  NJ  07974.
Diffusion of Credit in Markovian Models
Authors: Paolo Frasconi Yoshua Bengio
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: This  paper  studies the  problem of diffusion  in  Markovian  models,  such  as  hidden  Markov  models  (HMMs)  and  how  it  makes  very  difficult the task of learning of long-term dependencies in sequences.  Using results from Markov chain theory,  we show that the problem  of diffusion is reduced if the transition probabilities approach 0 or 1.  Under  this condition, standard HMMs have very  limited modeling  capabilities,  but input/output HMMs can still perform interesting  computations.