mtybilly/Instruct2DS huggingface dataset:
Hybrid NN/HMM-Based Speech Recognition with a Discriminant Neural Feature Extraction
Authors: Daniel Willett, Gerhard Rigoll
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: In this paper, we present a novel hybrid architecture for continuous speech  recognition systems.  It consists of a continuous HMM system extended  by an  arbitrary neural network that is  used as  a preprocessor that takes  several frames  of the feature vector as  input to produce more discrimin(cid:173) ative feature vectors with respect to the underlying HMM system.  This  hybrid system is an extension of a state-of-the-art continuous HMM sys(cid:173) tem, and in fact, it is the first hybrid system that really is capable of outper(cid:173) forming these standard systems with respect to the recognition accuracy.  Experimental results show an relative error reduction of about 10% that  we achieved on a remarkably good recognition system based on continu(cid:173) ous HMMs for the Resource Management 1 OOO-word continuous speech  recognition task.
2D Observers for Human 3D Object Recognition?
Authors: Zili Liu, Daniel Kersten
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Converging  evidence  has  shown  that  human  object  recognition  depends  on  familiarity  with  the  images  of  an  object.  Further,  the  greater  the  similarity  between  objects,  the  stronger  is  the  dependence  on  object  appearance,  and  the  more  important  two(cid:173) dimensional (2D) image information becomes.  These findings,  how(cid:173) ever, do not rule out the use of 3D structural information in recog(cid:173) nition,  and  the  degree  to  which  3D  information  is  used  in  visual  memory is an important issue.  Liu, Knill, & Kersten (1995) showed  that  any model  that  is  restricted  to  rotations  in  the  image  plane  of independent  2D  templates  could not  account for  human perfor(cid:173) mance in discriminating novel object views.  We now present results  from models of generalized radial basis functions  (GRBF), 2D near(cid:173) est  neighbor  matching that  allows  2D  affine  transformations,  and  a Bayesian statistical estimator that integrates over all possible 2D  affine  transformations.  The  performance  of the  human  observers  relative  to  each  of  the  models  is  better for  the  novel  views  than  for  the familiar  template views,  suggesting that humans generalize  better to novel  views  from  template views.  The  Bayesian estima(cid:173) tor yields the optimal performance with  2D  affine  transformations  and  independent  2D  templates.  Therefore,  models  of  2D  affine  matching  operations  with  independent  2D  templates  are unlikely  to account for  human recognition performance.
A Neural Network Based Head Tracking System
Authors: Daniel D Lee, H. S. Seung
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We have constructed an inexpensive video based motorized tracking system that learns to track a head. It uses real time graphical user inputs or an auxiliary infrared detector as supervisory signals to train a convolutional neural network. The inputs to the neural network consist of normalized luminance and chrominance images and motion information from frame differences. Subsampled images are also used to provide scale invariance. During the online training phases the neural network rapidly adjusts the input weights depending up on the reliability of the different channels in the surrounding environment. This quick adaptation allows the system to robustly track a head even when other objects are moving within a cluttered background.
Perturbative M-Sequences for Auditory Systems Identification
Authors: Mark Kvale, Christoph E. Schreiner
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: In  this  paper we  present a  new  method for  studying auditory sys(cid:173) tems  based  on  m-sequences.  The  method  allows  us  to  perturba(cid:173) tively  study  the  linear  response  of the  system  in  the  presence  of  various  other  stimuli,  such  as  speech  or  sinusoidal  modulations.  This allows one to construct linear kernels  (receptive fields)  at the  same time that other stimuli are being presented.  Using the method  we  calculate the modulation transfer function of single units in the  inferior colli cui us of the cat at different operating points and discuss  nonlinearities in the response.
Bach in a Box - Real-Time Harmony
Authors: Randall R. Spangler, Rodney M. Goodman, Jim Hawkins
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We describe a system for  learning J. S.  Bach's rules of musical har(cid:173) mony.  These  rules  are  learned  from  examples  and  are  expressed  as  rule-based  neural networks.  The rules  are  then applied  in  real(cid:173) time to generate new  accompanying harmony for  a  live  performer.  Real-time  functionality  imposes  constraints  on  the  learning  and  harmonizing processes,  including limitations on the types of infor(cid:173) mation the system can use  as  input and the amount of processing  the  system  can  perform.  We  demonstrate  algorithms  for  gener(cid:173) ating  and  refining  musical  rules  from  examples  which  meet  these  constraints.  We  describe  a  method  for  including  a  priori knowl(cid:173) edge  into the rules  which  yields significant performance gains.  We  then describe  techniques  for  applying  these  rules  to generate new  music  in  real-time.  We  conclude  the  paper  with  an  analysis  of  experimental results.
Use of a Multi-Layer Perceptron to Predict Malignancy in Ovarian Tumors
Authors: Herman Verrelst, Yves Moreau, Joos Vandewalle, Dirk Timmerman
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: This paper is concerned with the problem of Reinforcement Learn(cid:173) ing  (RL)  for  continuous  state  space  and  time  stocha.stic  control  problems.  We  state the Harnilton-Jacobi-Bellman  equation satis(cid:173) fied  by  the value  function  and use  a  Finite-Difference method for  designing a  convergent  approximation scheme.  Then we  propose a  RL  algorithm  based on this  scheme  and prove  its  convergence  to  the optimal solution. 
1 
Introduction to RL in the continuous, stochastic case 
The objective of RL is to find -thanks to a reinforcement signal- an optimal strategy  for  solving  a  dynamical  control problem.  Here we  sudy  the continuous  time,  con(cid:173) tinuous state-space stochastic case, which covers a wide variety of control problems  including target,  viability,  optimization  problems  (see  [FS93],  [KP95])}or  which  a  formalism  is  the following.  The evolution of the  current  state  x(t)  E  0  (the  state(cid:173) space, with 0  open subset of IRd ), depends on the control u(t)  E  U (compact subset)  by a  stochastic differential equation,  called the  state  dynamics: 
dx =  f(x(t), u(t))dt + a(x(t), u(t))dw 
(1)  where f  is the local drift  and a .dw  (with w  a brownian motion of dimension rand  (j  a d x r-matrix) the stochastic part (which appears for  several reasons such as lake  of precision,  noisy influence,  random fluctuations)  of the diffusion process.  For  initial state x  and control u(t),  (1)  leads  to  an infinity of possible  traj~tories  x(t).  For  some  trajectory  x(t)  (see  figure  I).,  let  T  be its  exit  time  from  0  (with  the convention  that  if x(t)  always  stays  in  0, then  T  =  00).  Then,  we  define  the  functional  J  of initial state x and control u(.) as the expectation for  all trajectories  of the discounted cumulative reinforcement  : 
J(x; u(.))  =  Ex,u( .) {loT '/r(x(t), u(t))dt +,,{ R(X(T))}
Receptive Field Formation in Natural Scene Environments: Comparison of Single Cell Learning Rules
Authors: Brian S. Blais, Nathan Intrator, Harel Z. Shouval, Leon N. Cooper
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  study  several  statistically and  biologically  motivated learning  rules  using  the same visual  environment,  one  made up  of natural  scenes,  and the same single cell  neuronal architecture.  This allows  us  to  concentrate  on  the  feature  extraction  and  neuronal  coding  properties  of these rules.  Included  in  these rules  are kurtosis  and  skewness  maximization,  the  quadratic form  of the  BCM  learning  rule,  and  single  cell  ICA.  Using  a  structure  removal  method,  we  demonstrate  that  receptive  fields  developed  using  these  rules  de(cid:173) pend  on  a  small  portion  of  the  distribution.  We  find  that  the  quadratic form  of the BCM rule behaves in  a manner similar to a  kurtosis maximization rule when the distribution contains kurtotic  directions,  although the  BCM  modification equations  are compu(cid:173) tationally simpler. 
424 
B.  S.  Blais, N.  Intrator, H.  Shouval and L  N.  Cooper
Reinforcement Learning for Call Admission Control and Routing in Integrated Service Networks
Authors: Peter Marbach, Oliver Mihatsch, Miriam Schulte, John N. Tsitsiklis
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We provide a model of the standard watermaze task, and of a more  challenging task involving novel platform locations, in which rats  exhibit one-trial learning after a few days of training.  The model  uses hippocampal place cells to support reinforcement learning,  and also,  in an integrated manner,  to build  and use  allocentric  coordinates.
The Error Coding and Substitution PaCTs
Authors: Gareth James, Trevor Hastie
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A new class of plug in  classification techniques have recently been de(cid:173) veloped in the statistics and machine learning literature.  A plug in clas(cid:173) sification  technique (PaCT) is  a  method that takes  a standard classifier  (such  as  LDA or TREES)  and plugs  it into an  algorithm to produce a  new classifier.  The  standard classifier  is  known as  the Plug  in  Classi(cid:173) fier  (PiC). These methods often produce large improvements over using  a single classifier.  In this paper we investigate one of these methods and  give some motivation for its success.
Modeling Complex Cells in an Awake Macaque during Natural Image Viewing
Authors: William E. Vinje, Jack L. Gallant
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  model  the  responses  of cells  in  visual  area  VI  during  natural  vision.  Our model consists  of a  classical energy  mechanism whose  output is divided by nonclassical gain control and texture  contrast  mechanisms.  We  apply  this  model  to  review  movies,  a  stimulus  sequence  that replicates  the stimulation a  cell  receives  during free  viewing  of natural  images.  Data  were  collected  from  three  cells  using five  different  review movies, and the model was fit  separately  to  the data from  each  movie.  For  the  energy  mechanism alone  we  find  modest  but  significant  correlations  (rE  =  0.41,  0.43,  0.59,  0.35)  between  model  and  data.  These  correlations  are  improved  somewhat when we  allow for suppressive surround effects  (rE+G  =  0.42,  0.56,  0.60,  0.37).  In  one  case  the  inclusion  of a  delayed  suppressive  surround dramatically improves the  fit  to the  data by  modifying the  time course  of the model's response.
Generalization in Decision Trees and DNF: Does Size Matter?
Authors: Mostefa Golea, Peter L. Bartlett, Wee Sun Lee, Llew Mason
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Recent  theoretical  results  for  pattern  classification  with  thresh(cid:173) olded real-valued  functions  (such  as  support  vector  machines,  sig(cid:173) moid  networks,  and  boosting)  give  bounds  on  misclassification  probability  that  do  not  depend  on  the size  of the  classifier,  and  hence can be considerably smaller than the bounds that follow from  the VC  theory.  In  this  paper,  we  show  that  these  techniques  can  be  more  widely  applied,  by  representing  other  boolean  functions  as  two-layer neural  networks  (thresholded convex  combinations of  boolean functions).  For example, we show that with high probabil(cid:173) ity any decision tree of depth no more than d that is consistent with  m training examples has misclassification probability no more than  o ( (~ (Neff VCdim(U)  log2 m log d)) 1/2),  where U is  the  class  of  node  decision  functions,  and  Neff  ::;  N  can  be  thought  of as  the  effective  number of leaves  (it  becomes small  as  the distribution on  the  leaves  induced  by  the  training  data gets  far  from  uniform).  This  bound  is  qualitatively different  from  the VC  bound  and  can  be considerably smaller.  We use the same technique to give similar results for DNF formulae. 
•  Author to whom correspondence should be addressed 
260 
M.  Golea,  P Bartlett,  W.  S. Lee and L  Mason
Local Dimensionality Reduction
Authors: Stefan Schaal, Sethu Vijayakumar, Christopher G. Atkeson
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: If globally high dimensional data has locally only low dimensional distribu(cid:173) tions,  it is  advantageous to perform a local dimensionality reduction before  further processing the data.  In this paper we examine several techniques for  local  dimensionality reduction  in the  context of locally weighted linear re(cid:173) gression.  As possible candidates, we derive local versions of factor analysis  regression, principle component regression, principle component regression  on joint distributions, and partial least squares regression. After outlining the  statistical bases of these  methods,  we perform Monte Carlo  simulations to  evaluate  their  robustness  with  respect  to  violations  of their  statistical  as(cid:173) sumptions.  One  surprising  outcome  is  that  locally  weighted  partial  least  squares  regression offers the best average results,  thus outperforming even  factor analysis, the theoretically most appealing of our candidate techniques.
Instabilities in Eye Movement Control: A Model of Periodic Alternating Nystagmus
Authors: Ernst R. Dow, Thomas J. Anastasio
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Nystagmus is  a pattern of eye movement characterized by  smooth rota(cid:173) tions  of the eye in one direction  and  rapid  rotations  in  the opposite di(cid:173) rection that reset eye position.  Periodic alternating nystagmus (PAN) is  a form  of uncontrollable  nystagmus  that  has  been  described  as  an  un(cid:173) stable but amplitude-limited oscillation.  PAN has been observed previ(cid:173) ously  only  in  subjects  with  vestibulo-cerebellar damage.  We describe  results in  which  PAN can be produced  in normal  subjects by prolonged  rotation in darkness.  We propose a new model  in  which the neural  cir(cid:173) cuits  that control eye movement are  inherently  unstable,  but  this  insta(cid:173) bility  is  kept  in  check  under  normal  circumstances  by  the  cerebellum.  Circumstances  which  alter  this  cerebellar  restraint,  such  as  vestibulo(cid:173) cerebellar damage or plasticity  due  to  rotation  in  darkness,  can  lead  to  PAN.
Neural Basis of Object-Centered Representations
Authors: Sophie Denève, Alexandre Pouget
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  present  a  neural  model  that  can  perform eye movements to a  particular side of an object regardless of the position and orienta(cid:173) tion  of the  object  in  space,  a  generalization  of a  task  which  has  been recently used by Olson and Gettner [4]  to investigate the neu(cid:173) ral structure of object-centered representations.  Our model uses an  intermediate representation in which units have oculocentric recep(cid:173) tive fields- just like collicular neurons- whose gain is modulated by  the side of the object to which the movement is directed, as  well as  the orientation of the object.  We show that these gain modulations  are consistent with Olson and Gettner's single cell recordings in the  supplementary eye  field.  This  demonstrates  that it  is  possible to  perform  an  object-centered  task  without  a  representation  involv(cid:173) ing an object-centered map, viz.,  without neurons whose receptive  fields are defined in object-centered coordinates.  We also show that  the same approach can account for  object-centered neglect, a  situ(cid:173) ation in which  patients with a  right parietal lesion neglect the left  side of objects regardless of the orientation of the objects. 
Several authors have argued that tasks such as object recognition [3]  and manipula(cid:173) tion  [4]  are easier to perform if the object is  represented in object-centered coordi(cid:173) nates, a representation in which the subparts of the object are encoded with respect  to a frame of reference centered on the object.  Compelling evidence for the existence  of such  representations in  the  cortex comes  from  experiments  on  hemineglect- a  neurological syndrome resulting from  unilateral lesions  of the parietal cortex such  that a  right lesion, for  example, leads  patients to ignore stimuli located on the left  side of their egocentric space.  Recently,  Driver et al.  (1994)  showed that the deficit  can  also  be  object-centered.  Hence,  hemineglect  patients can  detect  a  gap  in  the  upper edge of a triangle when this gap is associated with the right side of the object 
Neural Basis of Object-Centered Representations
The Storage Capacity of a Fully-Connected Committee Machine
Authors: Yuansheng Xiong, Chulan Kwon, Jong-Hoon Oh
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We study the storage capacity of a fully-connected  committee ma(cid:173) chine  with a  large number  K  of hidden  nodes.  The storage capac(cid:173) ity is obtained by analyzing the geometrical structure of the weight  space  related  to the internal representation .  By examining the as(cid:173) ymptotic behavior of order  parameters in the limit of large K, the  storage capacity Q c  is found to be proportional to ]{ Jln ]{ up to the  leading order.  This  result  satisfies  the  mathematical bound given  by  Mitchison  and  Durbin , whereas  the  replica-symmetric solution  in  a conventional Gardner's  approach  violates this bound.
Reinforcement Learning with Hierarchies of Machines
Authors: Ronald Parr, Stuart J. Russell
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We present a new approach to reinforcement learning in which the poli(cid:173) cies considered by the learning process are constrained by hierarchies of  partially specified machines.  This allows for the  use of prior knowledge  to reduce the search space and provides a framework in which knowledge  can  be  transferred  across  problems  and  in  which  component solutions  can be recombined to solve larger and more complicated problems.  Our  approach can be seen  as providing a link between reinforcement learn(cid:173) ing and "behavior-based" or "teleo-reactive" approaches to  control.  We  present provably  convergent algorithms  for  problem-solving and learn(cid:173) ing with hierarchical machines and demonstrate their effectiveness on a  problem with several thousand states.
On the Separation of Signals from Neighboring Cells in Tetrode Recordings
Authors: Maneesh Sahani, John S. Pezaris, Richard A. Andersen
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  discuss a solution to the problem of separating waveforms pro(cid:173) duced  by  multiple  cells  in  an  extracellular  neural  recording.  We  take an explicitly probabilistic approach, using latent-variable mod(cid:173) els  of varying  sophistication  to  describe  the distribution  of wave(cid:173) forms  produced  by  a  single  cell.  The  models  range from  a  single  Gaussian  distribution  of waveforms  for  each  cell  to  a  mixture  of  hidden  Markov models.  We  stress the  overall statistical structure  of the approach, allowing the details of the generative model chosen  to depend on the specific  neural preparation.
Task and Spatial Frequency Effects on Face Specialization
Authors: Matthew N. Dailey, Garrison W. Cottrell
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: There is  strong  evidence that face  processing is  localized in  the brain.  The  double  dissociation  between  prosopagnosia,  a  face  recognition  deficit occurring after brain damage, and visual object agnosia, difficulty  recognizing otber kinds of complex objects, indicates tbat face and non(cid:173) face  object recognition may be served by partially independent mecha(cid:173) nisms in the brain.  Is neural specialization innate or learned?  We sug(cid:173) gest  that this  specialization  could be tbe result  of a competitive learn(cid:173) ing mechanism that, during development, devotes neural resources to the  tasks they are best at performing. Furtber, we suggest that the specializa(cid:173) tion arises as an interaction between task requirements and developmen(cid:173) tal  constraints.  In  this paper,  we present a feed-forward computational  model  of visual processing, in which  two  modules compete to  classify  input stimuli.  When  one module receives low  spatial  frequency infor(cid:173) mation  and  the  other  receives high  spatial  frequency  information,  and  the task is to identify the faces while simply classifying the objects, the  low frequency network shows a strong specialization for faces.  No otber  combination  of tasks  and  inputs  shows  this  strong  specialization.  We  take these results as support for  the idea that an  innately-specified face  processing module is unnecessary.
Extended ICA Removes Artifacts from Electroencephalographic Recordings
Authors: Tzyy-Ping Jung, Colin Humphries, Te-Won Lee, Scott Makeig, Martin J. McKeown, Vicente Iragui, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Severe  contamination  of  electroencephalographic  (EEG)  activity  by  eye  movements, blinks, muscle,  heart  and line noise is  a  serious  problem for  EEG  interpretation  and  analysis.  Rejecting  contami(cid:173) nated  EEG  segments  results  in  a  considerable  loss  of information  and may be impractical for  clinical data.  Many methods have been  proposed  to  remove  eye  movement  and  blink  artifacts  from  EEG  recordings.  Often  regression  in  the  time  or  frequency  domain  is  performed  on  simultaneous  EEG  and  electrooculographic  (EOG)  recordings  to derive  parameters characterizing the appearance and  spread  of  EOG  artifacts  in  the  EEG  channels.  However,  EOG  records  also contain brain signals [1,  2],  so  regressing  out EOG ac(cid:173) tivity inevitably involves subtracting a portion of the relevant EEG  signal  from each  recording  as  well.  Regression  cannot  be  used  to  remove  muscle  noise  or  line  noise,  since  these  have  no  reference  channels.  Here , we  propose a new and generally applicable method  for  removing  a  wide  variety  of artifacts  from  EEG  records.  The  method  is  based  on  an  extended  version  of  a  previous  Indepen(cid:173) dent  Component  Analysis  (lCA)  algorithm  [3,  4]  for  performing  blind  source  separation  on  linear  mixtures of independent  source  signals  with  either  sub-Gaussian  or  super-Gaussian  distributions.  Our  results  show  that ICA  can  effectively  detect,  separate and  re(cid:173) move  activity  in  EEG  records  from  a  wide  variety  of artifactual  sources,  with  results  comparing favorably  to  those  obtained  using  regression-based  methods. 
Extended leA Removes Artifacts from EEG Recordings
Radial Basis Functions: A Bayesian Treatment
Authors: David Barber, Bernhard Schottky
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Bayesian methods have been successfully applied to regression and  classification  problems  in  multi-layer  perceptrons.  We  present  a  novel  application of Bayesian  techniques to Radial Basis Function  networks  by  developing a  Gaussian approximation to the posterior  distribution  which,  for  fixed  basis  function  widths,  is  analytic  in  the  parameters.  The setting of regularization  constants  by  cross(cid:173) validation is  wasteful  as  only  a  single optimal  parameter estimate  is  retained.  We  treat this  issue  by  assigning  prior distributions to  these constants, which are then adapted in light of the data under  a  simple re-estimation formula.
Independent Component Analysis for Identification of Artifacts in Magnetoencephalographic Recordings
Authors: Ricardo Vigário, Veikko Jousmäki, Matti Hämäläinen, Riitta Hari, Erkki Oja
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  have studied the application of an independent component analysis  (ICA)  approach  to  the  identification  and  possible  removal  of artifacts  from a magnetoencephalographic (MEG) recording. This statistical tech(cid:173) nique separates components according to the kurtosis of their amplitude  distributions  over  time,  thus  distinguishing  between  strictly  periodical  signals,  and regularly and irregularly occurring signals.  Many artifacts  belong to  the  last  category.  In order to  assess  the  effectiveness  of the  method, controlled artifacts were produced, which included saccadic eye  movements and blinks, increased muscular tension due to biting and the  presence of a digital watch inside  the  magnetically shielded room.  The  results demonstrate the  capability of the method to  identify and clearly  isolate the produced artifacts.
Classification by Pairwise Coupling
Authors: Trevor Hastie, Robert Tibshirani
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We discuss a strategy for  polychotomous classification that involves  estimating class probabilities for  each pair of classes,  and then cou(cid:173) pling the estimates together.  The coupling model is similar to  the  Bradley-Terry  method  for  paired  comparisons.  We  study  the  na(cid:173) ture  of the  class  probability estimates that  arise,  and examine  the  performance of the procedure in simulated datasets.  The classifiers  used  include  linear  discriminants  and  nearest  neighbors:  applica(cid:173) tion  to support vector  machines is  also briefly  described.
Hybrid Reinforcement Learning and Its Application to Biped Robot Control
Authors: Satoshi Yamada, Akira Watanabe, Michio Nakashima
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A  learning  system  composed of linear  control  modules,  reinforce(cid:173) ment  learning modules  and selection  modules  (a hybrid  reinforce(cid:173) ment learning system) is proposed for the fast learning of real-world  control  problems.  The  selection  modules  choose  one  appropriate  control module  dependent  on the  state.  This  hybrid learning sys(cid:173) tem was applied to the control of a stilt-type biped robot.  It learned  the control on a sloped floor  more quickly than the usual reinforce(cid:173) ment  learning  because  it  did  not  need  to learn  the  control  on  a  flat  floor,  where  the  linear  control  module  can  control the  robot.  When it was trained by a  2-step learning (during the first  learning  step, the selection module was trained by a training procedure con(cid:173) trolled  only  by  the  linear  controller),  it  learned  the  control  more  quickly.  The average  number of trials  (about  50)  is  so small  that  the learning system is  applicable to real robot control.
Synaptic Transmission: An Information-Theoretic Perspective
Authors: Amit Manwani, Christof Koch
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Here  we  analyze  synaptic  transmission  from  an  infonnation-theoretic  perspective. We derive c1osed-fonn expressions for the lower-bounds on  the capacity of a simple model of a cortical synapse under two explicit  coding paradigms.  Under the "signal estimation" paradigm, we assume  the signal to be encoded in the mean firing rate of a Poisson neuron.  The  perfonnance of an optimal linear estimator of the  signal  then  provides  a lower bound on the  capacity for signal estimation.  Under the  "signal  detection" paradigm, the presence or absence of the signal has to be de(cid:173) tected.  Perfonnance of the optimal spike detector allows us to compute  a lower bound on the capacity for signal detection.  We  find  that single  synapses (for empirically measured parameter values) transmit infonna(cid:173) tion poorly but  significant  improvement can be  achieved  with  a  small  amount of redundancy.
Just One View: Invariances in Inferotemporal Cell Tuning
Authors: Maximilian Riesenhuber, Tomaso Poggio
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: In  macaque  inferotemporal cortex  (IT),  neurons have  been found to re(cid:173) spond  selectively to complex  shapes  while  showing broad  tuning ("in(cid:173) variance")  with  respect  to stimulus transformations such as  translation  and  scale  changes  and  a  limited  tuning to rotation  in  depth.  Training  monkeys with novel,  paperclip-like objects, Logothetis et al. 9  could in(cid:173) vestigate whether these invariance properties are due to experience with  exhaustively many transformed instances of an object or if there are mech(cid:173) anisms that allow the cells to show response invariance also to previously  unseen instances of that object.  They found object-selective cells in an(cid:173) terior IT which exhibited limited invariance  to various transformations  after training with single object views.  While previous models accounted  for  the tuning of the  cells  for  rotations  in  depth and  for  their selectiv(cid:173) ity  to a  specific  object relative to a population of distractor objects,14,1  the model described here attempts to explain in a  biologically plausible  way  the additional properties of translation  and  size  invariance.  Using  the  same  stimuli  as  in  the  experiment,  we  find  that model  IT neurons  exhibit invariance properties which closely parallel those of real neurons.  Simulations show that the model  is capable of unsupervised learning of  view-tuned neurons. 
We thank Peter Dayan, Marcus Dill, Shimon Edelman, Nikos Logothetis, Jonathan Mumick and 
Randy O'Reilly for useful discussions and comments. 
216
Agnostic Classification of Markovian Sequences
Authors: Ran El-Yaniv, Shai Fine, Naftali Tishby
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Classification of finite sequences without explicit knowledge of their  statistical nature is  a  fundamental  problem  with  many  important  applications.  We  propose  a  new  information  theoretic  approach  to this  problem which is  based on the following  ingredients:  (i)  se(cid:173) quences are similar when they are likely to be generated by the same  source;  (ii) cross entropies can be estimated via "universal compres(cid:173) sion";  (iii)  Markovian  sequences  can  be  asymptotically-optimally  merged.  With these ingredients we  design a  method for  the classification of  discrete sequences whenever they can be compressed.  We introduce  the method and illustrate its application for hierarchical clustering  of languages and for  estimating similarities of protein sequences. 
1
Generalized Prioritized Sweeping
Authors: David Andre, Nir Friedman, Ronald Parr
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Prioritized sweeping  is  a  model-based  reinforcement learning  method  that  attempts  to  focus  an  agent's  limited  computational  resources  to  achieve a good estimate of the value of environment states.  To choose ef(cid:173) fectively where to spend a costly planning step, classic prioritized sweep(cid:173) ing  uses  a simple  heuristic  to  focus  computation on  the  states  that are  likely to have the largest errors.  In this paper, we  introduce generalized  prioritized sweeping, a principled method for generating such estimates  in a representation-specific manner.  This allows us to extend prioritized  sweeping beyond an explicit, state-based representation to deal with com(cid:173) pact representations that are necessary for dealing with large state spaces.  We  apply  this  method  for  generalized  model  approximators  (such  as  Bayesian networks), and describe preliminary experiments that compare  our approach with classical prioritized sweeping.
Multiresolution Tangent Distance for Affine-invariant Classification
Authors: Nuno Vasconcelos, Andrew Lippman
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: The ability  to  rely  on similarity  metrics  invariant to  image transforma(cid:173) tions is  an important issue for image classification tasks such as face or  character recognition. We analyze an invariant metric that has performed  well for the latter - the tangent distance - and study its limitations when  applied to regular images, showing that the most significant among these  (convergence to local minima) can be drastically reduced by computing  the distance in a multiresolution setting. This leads to the multi resolution  tangent distance,  which exhibits  significantly  higher invariance  to  im(cid:173) age transformations, and can be easily combined with robust estimation  procedures.
Detection of First and Second Order Motion
Authors: Alexander Grunewald, Heiko Neumann
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A  model  of  motion  detection  is  presented.  The  model  contains  three stages.  The first  stage is  unoriented and is  selective for  con(cid:173) trast  polarities.  The  next  two  stages  work  in  parallel.  A  phase  insensitive stage pools across  different  contrast polarities through  a  spatiotemporal filter  and thus can detect  first  and second order  motion.  A phase sensitive stage keeps  contrast polarities separate,  each of which is  filtered  through  a  spatiotemporal filter,  and thus  only first order motion can be detected.  Differential phase sensitiv(cid:173) ity can therefore account for  the detection of first  and second order  motion.  Phase insensitive detectors correspond to cortical complex  cells,  and phase sensitive detectors to simple  cells.
A Framework for Multiple-Instance Learning
Authors: Oded Maron, Tomás Lozano-Pérez
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Multiple-instance learning is a variation on supervised learning, where the  task  is  to  learn  a  concept  given  positive and  negative  bags  of instances.  Each  bag  may  contain many  instances,  but a  bag  is  labeled  positive even  if only one of the instances in it falls  within the concept.  A  bag is  labeled  negative  only  if all  the  instances  in  it are  negative.  We  describe  a  new  general  framework,  called Diverse  Density,  for  solving multiple-instance  learning problems.  We  apply this framework to learn a simple description  of a person from a series of images (bags) containing that person, to a stock  selection problem, and to the drug activity prediction problem.
Relative Loss Bounds for Multidimensional Regression Problems
Authors: Jyrki Kivinen, Manfred K. Warmuth
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  study  on-line generalized  linear  regression  with  multidimensional  outputs, i.e.,  neural  networks with multiple output nodes  but no  hidden  nodes.  We  allow  at the  final  layer transfer functions  such  as  the soft(cid:173) max function that need to consider the linear activations to all the output  neurons.  We use distance functions of a certain kind in two completely  independent roles in deriving and analyzing on-line learning algorithms  for  such tasks.  We use one distance function to define a matching loss  function for the (possibly multidimensional) transfer function, which al(cid:173) lows us to generalize earlier results from one-dimensional to multidimen(cid:173) sional outputs. We use another distance function as a tool for measuring  progress made by the on-line updates.  This shows how previously stud(cid:173) ied  algorithms  such  as  gradient  descent  and  exponentiated gradient fit  into a common  framework.  We  evaluate  the  performance of the  algo(cid:173) rithms  using relative  loss  bounds  that  compare  the  loss  of the  on-line  algoritm to the best off-line predictor from the relevant model class, thus  completely eliminating probabilistic assumptions about the data.
Multi-modular Associative Memory
Authors: Nir Levy, David Horn, Eytan Ruppin
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Motivated  by  the findings  of modular structure  in  the association  cortex,  we study a multi-modular model of associative memory that  can successfully  store  memory patterns  with different  levels  of ac(cid:173) tivity.  We show that the segregation of synaptic conductances into  intra-modular linear and inter-modular nonlinear ones considerably  enhances  the  network's  memory retrieval  performance.  Compared  with  the conventional, single-module associative  memory network,  the multi-modular network has two main advantages:  It is  less sus(cid:173) ceptible to damage to columnar input, and its response is consistent  with the cognitive data pertaining to category specific impairment.
Analysis of Drifting Dynamics with Neural Network Hidden Markov Models
Authors: Jens Kohlmorgen, Klaus-Robert Müller, Klaus Pawelzik
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  present  a  method  for  the  analysis  of  nonstationary  time  se(cid:173) ries  with multiple  operating modes.  In particular, it is possible to  detect and to model  both a  switching  of the dynamics  and a  less  abrupt,  time  consuming  drift  from  one  mode  to another.  This  is  achieved in two steps.  First, an unsupervised training method pro(cid:173) vides  prediction experts for  the inherent dynamical modes.  Then,  the trained experts are used in a hidden Markov model that allows  to  model  drifts.  An  application  to  physiological  wake/sleep  data  demonstrates that analysis and modeling of real-world time series  can be improved when the drift paradigm is  taken into account.
The Observer-Observation Dilemma in Neuro-Forecasting
Authors: Hans-Georg Zimmermann, Ralph Neuneier
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  explain how the training data can  be separated  into  clean informa(cid:173) tion and unexplainable noise.  Analogous to the data, the neural network  is  separated  into  a  time invariant  structure used  for  forecasting,  and  a  noisy part.  We propose a unified theory connecting the optimization al(cid:173) gorithms for cleaning and learning together with algorithms that control  the data noise and the parameter noise.  The combined algorithm allows  a data-driven local control of the liability of the network parameters and  therefore an  improvement  in generalization.  The approach is proven to  be very useful at the task of forecasting the German bond market. 
1  Introduction: The Observer-Observation Dilemma 
Human beings believe that they are able to solve a psychological version of the Observer(cid:173) Observation Dilemma. On the one hand, they use their observations to constitute an under(cid:173) standing of the laws of the world, on the other hand, they use this understanding to evaluate  the  correctness  of the incoming pieces  of information.  Of course,  as  everybody knows,  human beings are not free from making mistakes in this psychological dilemma.  We  en(cid:173) counter a similar situation when we try to build a mathematical model using data.  Learning  relationships from the data is only one part of the model building process.  Overrating this  part often leads to the phenomenon of overfitting in many applications (especially in eco(cid:173) nomic forecasting).  In practice, evaluation of the data is often done by external knowledge,  i. e.  by optimizing the model under constraints of smoothness and regularization [7].  If we  assume, that our model summerizes the best knowledge of the system to be identified, why  should we not use the model itself to evaluate the correctness of the data?  One approach to  do this is called Clearning [11].  In this paper, we present a unified approach of the interac(cid:173) tion between the data and a neural network (see also [8]).  It includes a new symmetric view  on the optimization algorithms, here learning and cleaning, and their control by parameter  and data noise. 
The Observer-Observation Dilemma in Neuro-Forecasting 
993
A Model of Early Visual Processing
Authors: Laurent Itti, Jochen Braun, Dale K. Lee, Christof Koch
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  propose  a  model for  early  visual  processing  in  primates.  The  model consists  of a  population of linear spatial filters  which  inter(cid:173) act  through non-linear excitatory  and inhibitory pooling.  Statisti(cid:173) cal estimation theory  is  then used  to derive human psychophysical  thresholds from the responses of the entire population of units.  The  model is  able  to reproduce  human thresholds for  contrast  and ori(cid:173) entation discrimination tasks,  and to predict contrast thresholds in  the presence  of masks of varying orientation and spatial frequency.
Data-Dependent Structural Risk Minimization for Perceptron Decision Trees
Authors: John Shawe-Taylor, Nello Cristianini
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A novel neural network model of pre-attention processing in visual(cid:173) search  tasks  is  presented.  Using displays of line orientations taken  from  Wolfe's experiments  [1992], we  study the  hypothesis  that the  distinction  between  parallel  versus  serial  processes  arises  from  the  availability of global information in  the internal representations  of  the  visual  scene.  The  model  operates  in  two  phases.  First,  the  visual  displays  are  compressed  via  principal-component-analysis.  Second,  the compressed data is processed by a target detector mod(cid:173) ule in order to identify the existence of a  target in the display.  Our  main  finding  is  that  targets  in  displays  which  were  found  exper(cid:173) imentally  to  be  processed  in  parallel can  be  detected  by  the  sys(cid:173) tem,  while  targets  in  experimentally-serial  displays  cannot .  This  fundamental  difference  is  explained  via  variance  analysis  of  the  compressed representations,  providing a  numerical criterion distin(cid:173) guishing parallel from serial displays.  Our model yields a  mapping  of response-time slopes that is similar to Duncan and Humphreys's  "search  surface"  [1989],  providing  an  explicit  formulation  of their  intuitive  notion  of feature  similarity.  It presents  a  neural  realiza(cid:173) tion of the  processing that may underlie the classical metaphorical  explanations of visual search. 
On Parallel versus Serial Processing: A  Computational Study a/Visual Search
Features as Sufficient Statistics
Authors: Davi Geiger, Archisman Rudra, Laurance T. Maloney
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: An image is often represented by a set of detected features.  We get  an enormous compression by representing images in this way.  Fur(cid:173) thermore,  we  get a  representation which  is  little affected by  small  amounts  of  noise  in  the  image.  However,  features  are  typically  chosen  in  an  ad  hoc  manner.  tures can be obtained using sufficient statistics.  The idea of sparse  data representation  naturally  arises.  We  treat  the  I-dimensional  and 2-dimensional signal reconstruction problem to make our ideas  concrete. 
\Ve  show  how  a  good  set  of fea(cid:173)
An Annealed Self-Organizing Map for Source Channel Coding
Authors: Matthias Burger, Thore Graepel, Klaus Obermayer
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  derive  and  analyse  robust  optimization  schemes  for  noisy  vector  quantization  on  the  basis  of deterministic  annealing.  Starting  from  a  cost  function  for  central  clustering  that  incorporates  distortions  from  channel  noise  we  develop  a  soft  topographic  vector  quantization  al(cid:173) gorithm  (STVQ)  which  is  based  on  the  maximum  entropy  principle  and which performs a maximum-likelihood estimate in  an expectation(cid:173) maximization (EM) fashion.  Annealing in the temperature parameter f3  leads to phase transitions in the existing code vector representation dur(cid:173) ing the cooling process for which we  calculate critical temperatures and  modes  as  a function of eigenvectors  and eigenvalues  of the covariance  matrix of the data and the transition matrix of the channel noise.  A whole  family  of vector quantization algorithms is derived from  STVQ, among  them  a  deterministic  annealing  scheme  for  Kohonen's  self-organizing  map  (SOM).  This  algorithm,  which  we  call  SSOM,  is  then  applied  to  vector quantization of image data to be sent via a noisy binary symmetric  channel.  The algorithm's performance is compared to those of LBG and  STVQ.  While it is  naturally superior to LBG, which does  not take into  account channel noise, its results compare very  well  to those of STVQ,  which is computationally much more demanding.