mtybilly/Instruct2DS huggingface dataset:
Convergence of the Wake-Sleep Algorithm
Authors: Shiro Ikeda, Shun-ichi Amari, Hiroyuki Nakahara
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The W-S (Wake-Sleep) algorithm is a simple learning rule for the models  with hidden variables.  It is  shown that this algorithm can be applied to  a factor  analysis model  which is  a linear version  of the  Helmholtz ma(cid:173) chine.  But even for a factor analysis model,  the general convergence is  not proved theoretically.  In this article, we describe the geometrical un(cid:173) derstanding of the W-S  algorithm in contrast with the EM (Expectation(cid:173) Maximization) algorithm and the em algorithm.  As the result, we prove  the convergence of the W-S  algorithm for the factor analysis model.  We  also show the condition for the convergence in general models.
Mean Field Methods for Classification with Gaussian Processes
Authors: Manfred Opper, Ole Winther
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We discuss the application of TAP mean field methods known from  the Statistical Mechanics of disordered systems to Bayesian classifi(cid:173) cation models with Gaussian processes.  In contrast to previous ap(cid:173) proaches,  no  knowledge about the distribution of inputs is  needed.  Simulation results for  the Sonar data set are given. 
1  Modeling with Gaussian Processes 
Bayesian models which are based on Gaussian prior distributions on function spaces  are promising non-parametric statistical tools.  They have been recently introduced  into the Neural  Computation community  (Neal  1996, Williams &  Rasmussen  1996,  Mackay  1997).  To  give their basic definition,  we  assume that the likelihood of the  output  or  target  variable  T  for  a  given  input  s  E  RN  can  be  written  in  the  form  p(Tlh(s))  where  h  : RN  --+  R  is  a  priori  assumed  to  be  a  Gaussian  random field.  If we  assume fields  with  zero  prior  mean,  the statistics  of h  is  entirely  defined  by  the second order correlations C(s, S')  ==  E[h(s)h(S')], where E  denotes expectations 
310 
M  Opper and 0.  Winther 
with respect  to the prior.  Interesting examples  are
Learning from Dyadic Data
Authors: Thomas Hofmann, Jan Puzicha, Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Dyadzc  data  refers  to  a  domain  with  two  finite  sets  of objects  in  which  observations are  made for  dyads , i.e., pairs with one element  from  either  set.  This  type  of data  arises  naturally  in  many  ap(cid:173) plication  ranging  from  computational  linguistics  and  information  retrieval to preference  analysis and computer vision.  In this paper,  we  present  a  systematic,  domain-independent framework  of learn(cid:173) ing  from  dyadic  data by  statistical  mixture  models.  Our  approach  covers different models with fiat  and hierarchical latent class struc(cid:173) tures.  We  propose  an  annealed  version  of the  standard  EM  algo(cid:173) rithm for  model fitting  which  is  empirically evaluated  on  a  variety  of data sets from  different  domains.
Call-Based Fraud Detection in Mobile Communication Networks Using a Hierarchical Regime-Switching Model
Authors: Jaakko Hollmén, Volker Tresp
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Fraud causes substantial losses to telecommunication carriers. Detec(cid:173) tion systems which automatically detect illegal use of the network can be  used to alleviate the problem. Previous approaches worked on features  derived from the call patterns of individual users. In this paper we present  a call-based detection system based on a hierarchical regime-switching  model. The detection problem is formulated as an inference problem on  the regime probabilities. Inference is implemented by applying the junc(cid:173) tion tree algorithm to the underlying graphical model. The dynamics are  learned from data using the EM algorithm and subsequent discriminative  training. The methods are assessed using fraud data from a real mobile  communication network.
Analyzing and Visualizing Single-Trial Event-Related Potentials
Authors: Tzyy-Ping Jung, Scott Makeig, Marissa Westerfield, Jeanne Townsend, Eric Courchesne, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Event-related  potentials (ERPs),  are portions  of electroencephalo(cid:173) graphic  (EEG)  recordings  that  are  both  time- and  phase-locked  to  experimental  events.  ERPs  are  usually  averaged  to  increase  their  signal/noise  ratio  relative  to  non-phase  locked  EEG  activ(cid:173) ity,  regardless  of the  fact  that  response  activity  in  single  epochs  may vary  widely  in  time course  and scalp distribution.  This study  applies a linear decomposition tool, Independent Component Anal(cid:173) ysis  (ICA)  [1],  to  multichannel single-trial  EEG  records  to  derive  spatial filters  that  decompose  single-trial  EEG epochs  into  a  sum  of temporally  independent  and spatially fixed  components  arising  from  distinct  or  overlapping  brain  or  extra-brain  networks.  Our  results  on  normal  and  autistic  subjects  show  that  ICA  can  sep(cid:173) arate  artifactual,  stimulus-locked,  response-locked,  and.  non-event  related  background  EEG  activities  into separate  components,  al(cid:173) lowing ( 1) removal of pervasive artifacts of all types from single-trial  EEG records,  and (2) identification of both stimulus- and response(cid:173) locked EEG components.  Second, this study proposes a new visual(cid:173) ization tool, the 'ERP image', for investigating variability in laten(cid:173) cies  and amplitudes of event-evoked responses  in spontaneous EEG  or  MEG  records.  We show  that sorting single-trial ERP epochs  in  order  of reaction  time  and  plotting  the  potentials  in  2-D  clearly  reveals  underlying  patterns  of response  variability  linked  to  per(cid:173) formance.  These  analysis  and  visualization  tools  appear  broadly  applicable to electrophyiological research  on both normal and clin(cid:173) ical  populations. 
Analyzing and Visualizing Single-Trial Event-Related Potentials
Orientation, Scale, and Discontinuity as Emergent Properties of Illusory Contour Shape
Authors: Karvel K. Thornber, Lance R. Williams
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  recent  neural  model  of  illusory  contour  formation  is  based  on  a  distribution  of natural  shapes  traced  by  particles  moving  with  constant speed in directions given by Brownian motions.  The input  to that model consists of pairs of position and direction constraints  and the  output  consists  of the  distribution  of contours joining all  such  pairs.  In general,  these contours will  not  be closed  and  their  distribution  will  not  be  scale-invariant.  In  this  paper,  we  show  how  to  compute  a  scale-invariant  distribution  of  closed  contours  given  position  constraints  alone  and  use  this  result  to  explain  a  well  known illusory contour effect.
Active Noise Canceling Using Analog Neuro-Chip with On-Chip Learning Capability
Authors: Jung-Wook Cho, Soo-Young Lee
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A modular analogue neuro-chip set with  on-chip learning capability  is  developed  for  active  noise  canceling.  The  analogue  neuro-chip  set  incorporates  the  error  backpropagation  learning  rule  for  practical  applications,  and  allows  pin-to-pin  interconnections  for  multi-chip  boards.  The  developed  neuro-board  demonstrated  active  noise  canceling  without  any  digital  signal  processor.  Multi-path  fading  of  acoustic  channels,  random  noise,  and  nonlinear distortion  of the  loud  speaker  are  compensated  by  the  adaptive  learning  circuits  of  the  neuro-chips. Experimental  results  are  reported  for  cancellation  of car  noise in  real time.
Coordinate Transformation Learning of Hand Position Feedback Controller by Using Change of Position Error Norm
Authors: Eimei Oyama, Susumu Tachi
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: In  order  to  grasp  an  object,  we  need  to  solve  the  inverse  kine(cid:173) matics problem, i.e., the coordinate transformation from  the visual  coordinates  to the joint  angle  vector  coordinates  of the arm.  Al(cid:173) though  several models  of coordinate transformation learning have  been proposed, they suffer from  a number of drawbacks.  In human  motion  control,  the  learning  of the  hand  position  error  feedback  controller in the inverse kinematics solver is important.  This paper  proposes  a  novel  model  of the coordinate transformation  learning  of  the  human  visual  feedback  controller  that  uses  the  change  of  the joint  angle vector and the corresponding change of the square  of the  hand  position  error  norm.  The  feasibility  of the  proposed  model  is  illustrated using numerical simulations.
Multi-Electrode Spike Sorting by Clustering Transfer Functions
Authors: Dmitry Rinberg, Hanan Davidowitz, Naftali Tishby
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  new  paradigm  is  proposed  for  sorting  spikes  in  multi-electrode  data using ratios of transfer functions  between cells and electrodes.  It  is  assumed  that  for  every  cell  and  electrode  there  is  a  stable  linear  relation.  These are  dictated  by  the properties of the tissue,  the electrodes  and  their relative  geometries.  The main  advantage  of the method is that it is  insensitive to variations in the shape and  amplitude of a  spike.  Spike  sorting  is  carried  out  in  two  separate  steps.  First,  templates  describing  the statistics of each spike type  are generated by clustering transfer function  ratios then spikes are  detected  in  the  data  using  the  spike  statistics.  These  techniques  were  applied  to  data generated  in  the  escape  response  system  of  the cockroach.
Fisher Scoring and a Mixture of Modes Approach for Approximate Inference and Learning in Nonlinear State Space Models
Authors: Thomas Briegel, Volker Tresp
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present Monte-Carlo generalized EM  equations for learning in  non(cid:173) linear state space models.  The difficulties lie in the Monte-Carlo E-step  which consists of sampling from  the posterior distribution of the hidden  variables given the observations. The new idea presented in this paper is  to generate samples from a Gaussian approximation to the true posterior  from  which it is easy to obtain independent samples.  The parameters  of  the Gaussian approximation are either derived from the extended Kalman  filter or the Fisher scoring algorithm. In case the posterior density is mul(cid:173) timodal  we propose to  approximate the posterior by  a sum of Gaussians  (mixture of modes approach).  We show that sampling from the approxi(cid:173) mate posterior densities obtained by the above algorithms leads to better  models  than  using  point estimates  for  the  hidden  states.  In  our exper(cid:173) iment,  the Fisher scoring algorithm obtained  a better approximation of  the posterior mode than the EKF. For a multimodal distribution, the mix(cid:173) ture of modes approach gave superior results.
Direct Optimization of Margins Improves Generalization in Combined Classifiers
Authors: Llew Mason, Peter L. Bartlett, Jonathan Baxter
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Cumulative training margin  dis(cid:173) tributions  for  AdaBoost  versus  our  "Direct  Optimization  Of  Margins"  (DOOM)  algorithm.  The dark curve is  AdaBoost, the  light  curve  is  DOOM.  DOOM  sacrifices  significant  training  er(cid:173) ror for  improved test error  (hori(cid:173) zontal marks on margin= 0 line)_ 
-1 
-0.8  -0.6  -0.4  -0.2  0 
0.2  0.4  0.6  0.8
A Neuromorphic Monaural Sound Localizer
Authors: John G. Harris, Chiang-Jung Pu, José Carlos Príncipe
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  describe the first  single microphone sound localization system  and its inspiration from theories of human monaural sound localiza(cid:173) tion.  Reflections and diffractions caused by the external ear (pinna)  allow  humans to estimate sound source elevations  using only one  ear.  Our single microphone localization model relies on a specially  shaped reflecting structure that serves the role  of the pinna.  Spe(cid:173) cially designed analog VLSI circuitry uses  echo-time processing to  localize the sound.  A CMOS integrated circuit has been designed,  fabricated,  and successfully demonstrated on actual sounds.
Boxlets: A Fast Convolution Algorithm for Signal Processing and Neural Networks
Authors: Patrice Simard, Léon Bottou, Patrick Haffner, Yann LeCun
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Signal  processing  and  pattern  recognition  algorithms make  exten(cid:173) sive  use  of convolution.  In  many cases, computational accuracy  is  not  as  important  as  computational  speed.  In  feature  extraction,  for  instance,  the  features  of interest  in  a  signal  are  usually  quite  distorted.  This form of noise justifies some level of quantization in  order  to  achieve  faster  feature  extraction .  Our  approach  consists  of approximating regions  of the  signal  with  low  degree  polynomi(cid:173) als,  and then differentiating the resulting signals in order to obtain  impulse functions  (or derivatives of impulse functions).  With this  representation,  convolution  becomes  extremely  simple and  can  be  implemented quite effectively.  The true  convolution can  be  recov(cid:173) ered  by  integrating  the  result  of  the  convolution.  This  method  yields  substantial speed  up  in feature  extraction  and is  applicable  to convolutional neural  networks.
Experimental Results on Learning Stochastic Memoryless Policies for Partially Observable Markov Decision Processes
Authors: John K. Williams, Satinder P. Singh
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Partially Observable Markov Decision  Processes  (pO "MOPs)  constitute  an  important  class  of reinforcement  learning problems  which  present  unique theoretical and computational difficulties.  In  the absence of the  Markov property,  popular reinforcement  learning  algorithms  such  as  Q-Iearning  may  no  longer  be  effective,  and  memory-based  methods  which remove partial observability via state-estimation  are notoriously  expensive.  An alternative approach is  to seek a stochastic memoryless  policy  which  for  each  observation  of  the  environment  prescribes  a  probability  distribution  over  available  actions  that  maximizes  the  average  reward  per  timestep.  A  reinforcement  learning  algorithm  which  learns  a locally optimal  stochastic memoryless  policy has  been  proposed by Jaakkola,  Singh and Jordan,  but not empirically verified.  We present  a variation  of this  algorithm,  discuss  its  implementation,  and demonstrate its viability using four test problems.
Attentional Modulation of Human Pattern Discrimination Psychophysics Reproduced by a Quantitative Model
Authors: Laurent Itti, Jochen Braun, Dale K. Lee, Christof Koch
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  previously  proposed  a  quantitative model  of early  visual  pro(cid:173) cessing  in  primates,  based  on  non-linearly interacting visual filters  and statistically efficient  decision.  We  now  use  this model to inter(cid:173) pret  the observed  modulation of a  range of human psychophysical  thresholds  with  and  without  focal  visual  attention.  Our  model  - calibrated  by  an  automatic fitting  procedure  - simultaneously  re(cid:173) produces  thresholds for  four  classical  pattern discrimination tasks,  performed while attention was engaged by another concurrent task.  Our model then predicts that the seemingly complex improvements  of certain  thresholds,  which  we  observed  when  attention  was  fully  available for  the  discrimination  tasks,  can  best  be  explained  by  a  strengthening  of competition among early  visual filters.
Signal Detection in Noisy Weakly-Active Dendrites
Authors: Amit Manwani, Christof Koch
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Here we derive measures quantifying the information loss of a synaptic  signal due to the presence of neuronal noise sources, as it electrotonically  propagates along a weakly-active dendrite. We model the dendrite as an  infinite linear cable, with noise sources distributed along its length.  The  noise sources we consider are thermal noise, channel noise arising from  the stochastic nature of voltage-dependent ionic channels (K+ and Na+)  and synaptic noise due to spontaneous background activity. We assess the  efficacy of information transfer using a signal detection paradigm where  the objective is to detect the presence/absence of a presynaptic spike from  the post-synaptic membrane voltage. This allows us to analytically assess  the role of each of these noise sources in  information transfer.  For our  choice  of parameters,  we find  that the  synaptic  noise  is  the  dominant  noise source which limits the maximum length over which information  be reliably transmitted.
Kernel PCA and De-Noising in Feature Spaces
Authors: Sebastian Mika, Bernhard Schölkopf, Alex J. Smola, Klaus-Robert Müller, Matthias Scholz, Gunnar Rätsch
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Kernel  PCA  as  a  nonlinear feature  extractor has  proven powerful  as  a  preprocessing step for classification algorithms.  But it can also be con(cid:173) sidered  as  a  natural  generalization of linear principal  component anal(cid:173) ysis.  This  gives  rise  to  the  question  how  to  use  nonlinear features  for  data compression, reconstruction, and de-noising, applications common  in  linear PCA.  This is  a nontrivial  task,  as the results provided by  ker(cid:173) nel PCA live in some high dimensional feature space and need not have  pre-images in  input space.  This work presents ideas for finding approxi(cid:173) mate pre-images, focusing on Gaussian kernels, and shows experimental  results  using  these pre-images in  data reconstruction and de-noising on  toy examples as well as on real world data. 
1  peA and Feature Spaces  Principal  Component Analysis  (PC A)  (e.g.  [3])  is  an  orthogonal  basis  transformation.  The  new  basis  is  found  by  diagonalizing  the  centered  covariance  matrix  of a  data  set  {Xk  E  RNlk  = 1, ... ,f}, defined  by  C  = ((Xi  - nates in the Eigenvector basis are called principal components.  The size of an Eigenvalue  >.  corresponding to  an  Eigenvector v  of C equals the amount of variance in  the direction  of v.  Furthermore, the directions of the  first  n Eigenvectors corresponding to the biggest  n Eigenvalues cover as much variance as possible by n orthogonal directions. In many ap(cid:173) plications they contain the most interesting information: for instance, in data compression,  where we project onto the directions with biggest variance to  retain  as  much information  as possible, or in de-noising, where we deliberately drop directions with small variance. 
(Xk))T).  The coordi(cid:173)
Multiple Paired Forward-Inverse Models for Human Motor Learning and Control
Authors: Masahiko Haruno, Daniel M. Wolpert, Mitsuo Kawato
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Humans  demonstrate  a  remarkable  ability  to  generate  accurate  and  appropriate motor behavior under many different and oftpn  uncprtain  environmental  conditions.  This  paper  describes  a  new  modular  ap(cid:173) proach to human motor learning and control, baspd on multiple pairs of  inverse  (controller)  and forward  (prpdictor)  models.  This architecture  simultaneously learns the multiple inverse models necessary for control  as well as how to select the inverse models appropriate for a given em'i(cid:173) ronm0nt.  Simulations of object manipulation demonstrates the ability  to learn  mUltiple  objects,  appropriate  generalization  to  novel  objects  and  the  inappropriate  activation  of motor  programs  based  on  visual  cues,  followed  by on-line  correction, seen  in  the  "size-weight  illusion".
SMEM Algorithm for Mixture Models
Authors: Naonori Ueda, Ryohei Nakano, Zoubin Ghahramani, Geoffrey E. Hinton
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present  a  split and  merge  EM  (SMEM)  algorithm to overcome the local  maximum problem in  parameter estimation of finite  mixture  models.  In the  case  of mixture  models,  non-global  maxima often  involve  having  too  many  components of a  mixture  model  in one part of the space and  too  few  in  an(cid:173) other, widely separated part of the space.  To escape from such configurations  we  repeatedly perform simultaneous split and merge operations using a  new  criterion  for  efficiently  selecting  the  split  and  merge  candidates.  We  apply  the proposed algorithm to the training of Gaussian mixtures and mixtures of  factor  analyzers  using synthetic  and  real  data and show  the effectiveness  of  using  the  split  and  merge  operations  to  improve the  likelihood  of both  the  training data and of held-out test data.
Learning Lie Groups for Invariant Visual Perception
Authors: Rajesh P. N. Rao, Daniel L. Ruderman
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: One  of the  most important problems  in  visual  perception is  that of visual  in(cid:173) variance:  how are objects perceived to be the same despite undergoing transfor(cid:173) mations such as translations, rotations or scaling?  In this paper, we describe a  Bayesian method for learning invariances based on Lie group theory.  We show  that previous approaches based on first-order Taylor series expansions of inputs  can be regarded as special cases of the Lie group approach, the latter being ca(cid:173) pable of handling in principle arbitrarily large transfonnations. Using a matrix(cid:173) exponential based generative model of images,  we derive an unsupervised al(cid:173) gorithm for learning Lie group operators from  input data containing infinites(cid:173) imal transfonnations.  The on-line unsupervised learning algorithm maximizes  the posterior probability of generating the training data.  We provide experimen(cid:173) tal results suggesting that the proposed method can learn Lie group operators for  handling reasonably large I-D translations and 2-D rotations.
Tractable Variational Structures for Approximating Graphical Models
Authors: David Barber, Wim Wiegerinck
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Graphical models provide a broad probabilistic framework with ap(cid:173) plications in speech recognition  (Hidden  Markov  Models),  medical  diagnosis  (Belief  networks)  and  artificial  intelligence  (Boltzmann  Machines).  However,  the  computing time is  typically  exponential  in the number of nodes in the graph.  Within the variational frame(cid:173) work for  approximating these models, we  present two classes of dis(cid:173) tributions,  decimatable  Boltzmann  Machines  and Tractable  Belief  Networks  that  go  beyond  the  standard  factorized  approach.  We  give  generalised  mean-field  equations  for  both  these  directed  and  undirected  approximations.  Simulation  results  on  a  small  bench(cid:173) mark problem suggest using these richer approximations compares  favorably  against others previously reported in the literature.
A Micropower CMOS Adaptive Amplitude and Shift Invariant Vector Quantiser
Authors: Richard Coggins, Raymond J. Wang, Marwan A. Jabri
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: In  this  paper we  describe the  architecture,  implementation and  experi(cid:173) mental results for an  Intracardiac Electrogram (ICEG) classification and  compression chip.  The chip processes  and  vector-quantises  30 dimen(cid:173) sional analogue vectors while consuming a maximum of 2.5  J-tW  power  for a heart rate of 60 beats per minute (1  vector per second) from a 3.3 V  supply.  This  represents  a significant advance on  previous  work  which  achieved ultra low power supervised morphology classification since the  template matching scheme used  in  this chip enables unsupervised blind  classification of abnonnal rhythms and the computational support for low  bit rate data compression.  The adaptive template matching scheme used  is tolerant to amplitude variations, and inter- and intra-sample time shifts.
Convergence Rates of Algorithms for Visual Search: Detecting Visual Contours
Authors: Alan L. Yuille, James M. Coughlan
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: This  paper  formulates  the  problem  of  visual  search  as  Bayesian  inference  and  defines  a  Bayesian  ensemble  of  problem  instances .  In  particular,  we  address  the  problem  of  the  detection  of  visual  contours  in  noise/clutter  by  optimizing  a  global  criterion  which  combines  local  intensity  and  geometry  information.  We  analyze  the  convergence  rates  of  A * search  algorithms  using  results  from  information theory to bound  the  probability of rare  events  within  the Bayesian ensemble.  This analysis determines characteristics of  the  domain ,  which  we  call  order  parameters,  that  determine  the  convergence  rates.  In  particular,  we  present  a  specific  admissible  A * algorithm with pruning which converges, with high probability,  with  expected  time  O(N)  in  the  size  of  the  problem.  In  addi(cid:173) tion,  we  briefly  summarize  extensions  of this  work  which  address  fundamental  limits  of target  contour  detectability  (Le.  algorithm  independent results)  and the use  of non-admissible heuristics.
Learning to Find Pictures of People
Authors: Sergey Ioffe, David A. Forsyth
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Finding articulated objects, like people, in pictures present.s a par(cid:173) ticularly difficult object. recognition problem. We show how t.o  find people by finding putative body segments, and then construct.(cid:173) ing assemblies of those segments that are consist.ent with the con(cid:173) straints on the appearance of a person that result from kinematic  properties. Since a reasonable model of a person requires at. least  nine segments, it is not possible to present every group to a classi(cid:173) fier. Instead, the search can be pruned by using projected versions  of a classifier that accepts groups corresponding to people. We  describe an efficient projection algorithm for one popular classi(cid:173) fier , and demonstrate that our approach can be used to determine  whether images of real scenes contain people.
Visualizing Group Structure
Authors: Marcus Held, Jan Puzicha, Joachim M. Buhmann
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Cluster  analysis  is  a  fundamental  principle  in  exploratory  data  analysis,  providing the user  with a  description  of the  group struc(cid:173) ture of given data.  A key  problem in  this context is the interpreta(cid:173) tion  and  visualization  of clustering  solutions  in  high- dimensional  or  abstract  data  spaces.  In  particular,  probabilistic  descriptions  of the  group structure,  essential  to  capture  inter-cluster  relation(cid:173) ships, are hardly assessable by simple inspection ofthe probabilistic  assignment  variables.  VVe  present  a  novel  approach  to  the  visual(cid:173) ization of group structure.  It is  based on  a statistical model of the  object  assignments  which  have  been  observed  or  estimated  by  a  probabilistic clustering  procedure.  The objects or  data points are  embedded  in  a  low  dimensional Euclidean space  by  approximating  the  observed  data statistics  with  a  Gaussian  mixture model.  The  algorithm provides a new approach to the visualization of the inher(cid:173) ent structure for a broad variety of data types,  e.g. histogram data,  proximity data and co-occurrence data.  To demonstrate the power  of the approach,  histograms of textured images are visualized as an  example of a  large-scale data mining application.
Modeling Surround Suppression in V1 Neurons with a Statistically Derived Normalization Model
Authors: Eero P. Simoncelli, Odelia Schwartz
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We examine the statistics of natural monochromatic images decomposed 
using a multi-scale wavelet basis.  Although the coefficients of this rep(cid:173)
resentation are  nearly decorrelated,  they  exhibit important higher-order 
statistical dependencies that cannot be eliminated with purely linear pro(cid:173)
c~ssing. In particular, rectified coefficients corresponding to basis func(cid:173)
tions at neighboring spatial positions, orientations and scales are highly 
correlated.  A method of removing these dependencies is  to divide each 
coefficient by  a  weighted  combination of its  rectified  neighbors.  Sev(cid:173)
eral successful models of the steady -state behavior of neurons in primary 
visual cortex are based on such "divisive normalization" computations, 
and thus our analysis provides a theoretical justification for these models. 
Perhaps more importantly, the statistical measurements explicitly specify 
the  weights that should be used in computing the normalization signal. 
We  demonstrate that this  weighting  is  qualitatively  consistent with  re(cid:173)
cent physiological experiments  that  characterize the  suppressive effect 
of stimuli presented outside of the classical receptive field.  Our obser(cid:173)
vations thus provide evidence for the hypothesis that early visual neural 
processing is well matched to these statistical properties of images.
Familiarity Discrimination of Radar Pulses
Authors: Eric Granger, Stephen Grossberg, Mark A. Rubin, William W. Streilein
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The  ARTMAP-FD  neural  network  performs  both  identification  (placing  test  patterns in  classes  encountered during  training)  and  familiarity  discrimination  (judging  whether a  test  pattern belongs  to  any  of  the  classes  encountered  during  training).  The  perfor(cid:173) mance of ARTMAP-FD  is  tested  on  radar pulse  data obtained in  the field,  and compared to that of the nearest-neighbor-based NEN  algorithm and to a  k  > 1 extension of NEN.
Graphical Models for Recognizing Human Interactions
Authors: Nuria Oliver, Barbara Rosario, Alex Pentland
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We describe  a real-time computer vision  and machine learning sys(cid:173) tem for  modeling and recognizing  human actions and interactions.  Two  different  domains  are  explored:  recognition  of  two-handed  motions in  the martial art  'Tai  Chi' , and multiple- person  interac(cid:173) tions  in a  visual  surveillance task.  Our system combines top-down  with  bottom-up information using  a  feedback  loop,  and  is  formu(cid:173) lated with  a  Bayesian  framework.  Two  different  graphical  models  (HMMs and Coupled HMMs) are used for modeling both individual  actions and multiple-agent interactions, and CHMMs are shown to  work  more efficiently  and  accurately  for  a  given  amount of train(cid:173) ing.  Finally,  to  overcome  the  limited  amounts  of training  data,  we  demonstrate  that  'synthetic  agents'  (Alife-style  agents)  can  be  used  to develop flexible  prior models of the person-to-person inter(cid:173) actions.
The Effect of Correlations on the Fisher Information of Population Codes
Authors: Hyoungsoo Yoon, Haim Sompolinsky
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  study  the  effect  of correlated  noise  on  the  accuracy  of popu(cid:173) lation  coding  using  a  model  of  a  population  of  neurons  that  are  broadly  tuned  to  an  angle  in  two-dimension.  The  fluctuations  in  the neuronal activity is  modeled  as  a  Gaussian noise with pairwise  correlations which decays exponentially with the difference between  the preferred orientations of the pair.  By calculating the Fisher in(cid:173) formation  of the  system,  we  show  that in  the biologically  relevant  regime of parameters positive correlations decrease the estimation  capability  of the  network  relative  to  the  uncorrelated  population.  Moreover strong positive  correlations  result  in  information  capac(cid:173) ity  which  saturates  to a  finite  value  as  the  number  of cells  in  the  population  grows.  In  contrast,  negative  correlations  substantially  increase the information capacity of the neuronal  population.
Spike-Based Compared to Rate-Based Hebbian Learning
Authors: Richard Kempter, Wulfram Gerstner, J. Leo van Hemmen
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  correlation-based learning  rule  at the  spike  level  is  formulated,  mathematically analyzed, and compared to learning in a firing-rate  description.  A  differential  equation  for  the  learning  dynamics  is  derived  under the assumption  that the time scales of learning and  spiking  can  be  separated.  For  a  linear  Poissonian  neuron  model  which receives time-dependent stochastic input we show that spike  correlations on  a  millisecond time scale play  indeed  a  role.  Corre(cid:173) lations between input and output spikes tend to stabilize structure  formation,  provided  that  the  form  of  the  learning  window  is  in  accordance with Hebb's  principle.  Conditions for  an intrinsic nor(cid:173) malization of the average synaptic weight are discussed.
Contrast Adaptation in Simple Cells by Changing the Transmitter Release Probability
Authors: Péter Adorján, Klaus Obermayer
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The contrast response function (CRF) of many neurons in the primary vi(cid:173) sual  cortex saturates and shifts towards higher contrast values following  prolonged presentation of high contrast visual stimuli.  Using a recurrent  neural  network of excitatory spiking neurons with adapting synapses we  show  that both effects could  be  explained  by  a fast  and  a slow compo(cid:173) nent in the synaptic adaptation.  (i) Fast synaptic depression leads to sat(cid:173) uration  of the CRF and  phase  advance  in  the cortical  response to  high  contrast stimuli.  (ii) Slow adaptation of the synaptic transmitter release  probability is derived such that the mutual information between the input  and  the output of a cortical neuron  is  maximal.  This component-given  by  infomax learning rule-explains contrast adaptation of the averaged  membrane  potential  (DC  component)  as  well  as  the surprising experi(cid:173) mental  result,  that  the  stimulus  modulated component  (Fl  component)  of a cortical cell's membrane potential adapts only weakly. Based on our  results,  we propose a new  experiment to estimate the strength of the ef(cid:173) fective  excitatory  feedback  to  a  cortical  neuron,  and  we  also  suggest a  relatively  simple experimental  test to justify our hypothesized synaptic  mechanism for contrast adaptation.
Optimizing Correlation Algorithms for Hardware-Based Transient Classification
Authors: R. Timothy Edwards, Gert Cauwenberghs, Fernando J. Pineda
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The perfonnance of dedicated VLSI neural processing hardware depends  critically  on  the  design  of the  implemented  algorithms.  We  have  pre(cid:173) viously  proposed an  algorithm  for  acoustic  transient classification  [1].  Having implemented and demonstrated this algorithm in a mixed-mode  architecture,  we  now  investigate  variants  on  the algorithm,  using  time  and frequency channel differencing, input and output nonnalization, and  schemes to binarize and train the template values, with the goal of achiev(cid:173) ing optimal classification perfonnance for the chosen hardware.
Information Maximization in Single Neurons
Authors: Martin Stemmler, Christof Koch
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Information from  the  senses must be compressed into the  limited range  of firing  rates  generated  by  spiking  nerve  cells.  Optimal  compression  uses all firing rates equally often, implying that the nerve cell's response  matches  the  statistics  of naturally  occurring  stimuli.  Since  changing  the  voltage-dependent ionic  conductances  in  the  cell  membrane  alters  the  flow  of information,  an  unsupervised,  non-Hebbian,  developmental  learning  rule  is  derived  to  adapt  the  conductances  in  Hodgkin-Huxley  model  neurons.  By  maximizing  the  rate  of information  transmission,  each firing rate within the model neuron's limited dynamic range is  used  equally often . 
An  efficient neuronal representation of incoming sensory  information should take  advan(cid:173) tage  of the  regularity  and  scale  invariance  of stimulus  features  in  the  natural  world.  In  the  case  of vision,  this  regularity  is  reflected  in  the  typical  probabilities of encountering  particular visual contrasts, spatial orientations, or colors [1].  Given these probabilities, an  optimized neural  code  would eliminate  any  redundancy, while  devoting  increased  repre(cid:173) sentation to commonly encountered features. 
At the level of a single spiking neuron, information about a potentially large range of stimuli  is compressed into a finite range of firing rates, since the maximum firing rate of a neuron is  limited.  Optimizing the information transmission through a single neuron in the  presence  of uniform, additive noise has an intuitive interpretation:  the most efficient representation  of the input uses every firing rate with equal probability.  An analogous principle for non(cid:173) spiking neurons has been tested experimentally by Laughlin [2], who matched the statistics 
Information Maximization in Single Neurons 
161
Mechanisms of Generalization in Perceptual Learning
Authors: Zili Liu, Daphna Weinshall
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The learning of many visual perceptual tasks has been shown to be  specific  to practiced stimuli,  while  new  stimuli  require re-Iearning  from  scratch.  Here  we  demonstrate  generalization  using  a  novel  paradigm in motion discrimination  where learning has been previ(cid:173) ously  shown  to  be  specific.  We  trained  subjects  to  discriminate  the  directions  of  moving  dots,  and  verified  the  previous  results  that learning does not transfer from the trained direction to a  new  one.  However,  by  tracking  the  subjects'  performance  across  time  in  the new  direction, we  found  that their rate of learning doubled.  Therefore, learning generalized in  a  task previously considered too  difficult  for  generalization.  We  also  replicated,  in  the  second  ex(cid:173) periment, transfer following  training with  "easy"  stimuli.  The specificity  of perceptual learning and  the  dichotomy  between  learning of "easy"  vs.  "difficult"  tasks were hypothesized to involve  different  learning  processes,  operating  at  different  visual  cortical  areas.  Here we show how to interpret these results in terms of signal  detection  theory.  With  the  assumption  of limited  computational  resources,  we  obtain  the  observed  phenomena  - direct  transfer  and change of learning rate - for increasing levels of task 'difficulty.  It  appears  that  human  generalization  concurs  with  the  expected  behavior of a  generic discrimination system.
Probabilistic Image Sensor Fusion
Authors: Ravi K. Sharma, Todd K. Leen, Misha Pavel
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present  a  probabilistic  method  for  fusion  of images produced  by multiple sensors.  The approach is  based on  an image formation  model in which  the sensor images are  noisy, locally linear functions  of an  underlying,  true  scene.  A  Bayesian framework  then  provides  for  maximum likelihood or  maximum a  posteriori  estimates of the  true  scene  from  the sensor  images.  Maximum likelihood estimates  of  the  parameters  of  the  image  formation  model  involve  (local)  second order image statistics, and thus are related to local principal  component  analysis.  We  demonstrate  the  efficacy  of the  method  on images from  visible-band  and infrared sensors.
On-Line Learning with Restricted Training Sets: Exact Solution as Benchmark for General Theories
Authors: H. C. Rae, Peter Sollich, Anthony C. C. Coolen
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: O(ws(s log d+log(dqh/ s))) and O(ws((h/ s) log q) +log(dqh/ s)) are  upper bounds for  the VC-dimension of a  set of neural networks of  units  with  piecewise  polynomial  activation  functions,  where  s  is  the  depth  of  the  network,  h  is  the  number  of  hidden  units,  w  is  the  number  of  adjustable  parameters,  q  is  the  maximum  of  the  number of polynomial segments of the activation function, and d is  the  maximum degree  of  the polynomials;  also  n(wslog(dqh/s))  is  a  lower  bound  for  the VC-dimension  of such  a  network set,  which  are tight for  the cases  s =  8(h)  and  s is  constant.  For the special  case q =  1,  the VC-dimension is  8(ws log d).
Phase Diagram and Storage Capacity of Sequence-Storing Neural Networks
Authors: A. Düring, Anthony C. C. Coolen, D. Sherrington
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We solve the dynamics of Hopfield-type neural networks which store se(cid:173) quences of patterns, close to saturation. The asymmetry of the interaction  matrix in such models leads to violation of detailed balance, ruling out an  equilibrium statistical mechanical analysis.  Using generating functional  methods we derive exact closed equations for dynamical order parame(cid:173) ters,  viz.  the  sequence overlap and  correlation and  response functions.  in  the  limit of an  infinite system size.  We  calculate the time  translation  invariant solutions of these equations. describing stationary limit-cycles.  which  leads  to  a phase diagram.  The  effective retarded  self-interaction  usually  appearing  in  symmetric  models  is  here  found  to  vanish,  which  causes  a  significantly  enlarged  storage  capacity  of eYe  ~ 0.269.  com(cid:173) pared to  eYe  ~ 0.139 for Hopfield networks s~oring static patterns.  Our  results  are  tested  against extensive  computer simulations and  excellent  agreement is found. 
212 
A.  Diiring,  A.  C.  C.  Coo/en  and D. Sherrington 
1  INTRODUCTION AND DEFINITIONS 
We  consider a system of N  neurons O'(t)  = {ai(t)  = ±1}, which can change their states  collectively  at  discrete  times  (parallel  dynamics).  Each  neuron  changes  its  state  with  a  probability Pi(t)  =  ~[l-tanh,Bai(t)[Lj Jijaj(t)+Oi(t)]], so that the transition matrix is  W[o'(s + l)IO'(s)]  = II e.BO',(s+l)[E;=l J, j O'} (s)+ o,( s)]-ln2cosh(i3[E; =1  J'JO'} ( s)+(J, (s )))
Shrinking the Tube: A New Support Vector Regression Algorithm
Authors: Bernhard Schölkopf, Peter L. Bartlett, Alex J. Smola, Robert C. Williamson
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A new algorithm for Support Vector regression is  described.  For a priori  chosen 1/,  it automatically adjusts a flexible tube of minimal radius to the  data such that  at most a fraction  1/  of the data points lie outside.  More(cid:173) over,  it  is  shown  how  to  use  parametric  tube  shapes  with  non-constant  radius. The algorithm is analysed theoretically and experimentally.
Lazy Learning Meets the Recursive Least Squares Algorithm
Authors: Mauro Birattari, Gianluca Bontempi, Hugues Bersini
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Lazy  learning  is  a  memory-based  technique  that,  once  a  query  is  re(cid:173) ceived, extracts a prediction interpolating locally the neighboring exam(cid:173) ples of the query which are considered relevant according to  a distance  measure.  In  this paper we propose a data-driven method to  select on a  query-by-query basis the optimal number of neighbors to be considered  for  each  prediction.  As  an  efficient way  to  identify  and  validate  local  models, the  recursive least  squares  algorithm  is  introduced  in  the  con(cid:173) text  of local  approximation and lazy learning.  Furthermore,  beside the  winner-takes-all strategy for model selection, a local combination of the  most promising models is explored.  The method  proposed  is  tested  on  six different datasets and compared with a state-of-the-art approach.