mtybilly/Instruct2DS huggingface dataset:
Convergence Rates of Algorithms for Visual Search: Detecting Visual Contours
Authors: James M. Coughlan Alan L. Yuille
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: This  paper  formulates  the  problem  of  visual  search  as  Bayesian  inference  and  defines  a  Bayesian  ensemble  of  problem  instances .  In  particular,  we  address  the  problem  of  the  detection  of  visual  contours  in  noise/clutter  by  optimizing  a  global  criterion  which  combines  local  intensity  and  geometry  information.  We  analyze  the  convergence  rates  of  A * search  algorithms  using  results  from  information theory to bound  the  probability of rare  events  within  the Bayesian ensemble.  This analysis determines characteristics of  the  domain ,  which  we  call  order  parameters,  that  determine  the  convergence  rates.  In  particular,  we  present  a  specific  admissible  A * algorithm with pruning which converges, with high probability,  with  expected  time  O(N)  in  the  size  of  the  problem.  In  addi(cid:173) tion,  we  briefly  summarize  extensions  of this  work  which  address  fundamental  limits  of target  contour  detectability  (Le.  algorithm  independent results)  and the use  of non-admissible heuristics.
Learning to Find Pictures of People
Authors: David A. Forsyth Sergey Ioffe
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Finding articulated objects, like people, in pictures present.s a par(cid:173) ticularly difficult object. recognition problem. We show how t.o  find people by finding putative body segments, and then construct.(cid:173) ing assemblies of those segments that are consist.ent with the con(cid:173) straints on the appearance of a person that result from kinematic  properties. Since a reasonable model of a person requires at. least  nine segments, it is not possible to present every group to a classi(cid:173) fier. Instead, the search can be pruned by using projected versions  of a classifier that accepts groups corresponding to people. We  describe an efficient projection algorithm for one popular classi(cid:173) fier , and demonstrate that our approach can be used to determine  whether images of real scenes contain people.
Visualizing Group Structure
Authors: Jan Puzicha, Joachim M. Buhmann Marcus Held
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Cluster  analysis  is  a  fundamental  principle  in  exploratory  data  analysis,  providing the user  with a  description  of the  group struc(cid:173) ture of given data.  A key  problem in  this context is the interpreta(cid:173) tion  and  visualization  of clustering  solutions  in  high- dimensional  or  abstract  data  spaces.  In  particular,  probabilistic  descriptions  of the  group structure,  essential  to  capture  inter-cluster  relation(cid:173) ships, are hardly assessable by simple inspection ofthe probabilistic  assignment  variables.  VVe  present  a  novel  approach  to  the  visual(cid:173) ization of group structure.  It is  based on  a statistical model of the  object  assignments  which  have  been  observed  or  estimated  by  a  probabilistic clustering  procedure.  The objects or  data points are  embedded  in  a  low  dimensional Euclidean space  by  approximating  the  observed  data statistics  with  a  Gaussian  mixture model.  The  algorithm provides a new approach to the visualization of the inher(cid:173) ent structure for a broad variety of data types,  e.g. histogram data,  proximity data and co-occurrence data.  To demonstrate the power  of the approach,  histograms of textured images are visualized as an  example of a  large-scale data mining application.
Modeling Surround Suppression in V1 Neurons with a Statistically Derived Normalization Model
Authors: Odelia Schwartz Eero P. Simoncelli
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We examine the statistics of natural monochromatic images decomposed 
using a multi-scale wavelet basis.  Although the coefficients of this rep(cid:173)
resentation are  nearly decorrelated,  they  exhibit important higher-order 
statistical dependencies that cannot be eliminated with purely linear pro(cid:173)
c~ssing. In particular, rectified coefficients corresponding to basis func(cid:173)
tions at neighboring spatial positions, orientations and scales are highly 
correlated.  A method of removing these dependencies is  to divide each 
coefficient by  a  weighted  combination of its  rectified  neighbors.  Sev(cid:173)
eral successful models of the steady -state behavior of neurons in primary 
visual cortex are based on such "divisive normalization" computations, 
and thus our analysis provides a theoretical justification for these models. 
Perhaps more importantly, the statistical measurements explicitly specify 
the  weights that should be used in computing the normalization signal. 
We  demonstrate that this  weighting  is  qualitatively  consistent with  re(cid:173)
cent physiological experiments  that  characterize the  suppressive effect 
of stimuli presented outside of the classical receptive field.  Our obser(cid:173)
vations thus provide evidence for the hypothesis that early visual neural 
processing is well matched to these statistical properties of images.
Familiarity Discrimination of Radar Pulses
Authors: Stephen Grossberg, Mark A. Rubin, William W. Streilein Eric Granger
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The  ARTMAP-FD  neural  network  performs  both  identification  (placing  test  patterns in  classes  encountered during  training)  and  familiarity  discrimination  (judging  whether a  test  pattern belongs  to  any  of  the  classes  encountered  during  training).  The  perfor(cid:173) mance of ARTMAP-FD  is  tested  on  radar pulse  data obtained in  the field,  and compared to that of the nearest-neighbor-based NEN  algorithm and to a  k  > 1 extension of NEN.
Graphical Models for Recognizing Human Interactions
Authors: Barbara Rosario, Alex Pentland Nuria Oliver
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We describe  a real-time computer vision  and machine learning sys(cid:173) tem for  modeling and recognizing  human actions and interactions.  Two  different  domains  are  explored:  recognition  of  two-handed  motions in  the martial art  'Tai  Chi' , and multiple- person  interac(cid:173) tions  in a  visual  surveillance task.  Our system combines top-down  with  bottom-up information using  a  feedback  loop,  and  is  formu(cid:173) lated with  a  Bayesian  framework.  Two  different  graphical  models  (HMMs and Coupled HMMs) are used for modeling both individual  actions and multiple-agent interactions, and CHMMs are shown to  work  more efficiently  and  accurately  for  a  given  amount of train(cid:173) ing.  Finally,  to  overcome  the  limited  amounts  of training  data,  we  demonstrate  that  'synthetic  agents'  (Alife-style  agents)  can  be  used  to develop flexible  prior models of the person-to-person inter(cid:173) actions.
An Entropic Estimator for Structure Discovery
Authors: Matthew Brand
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  introduce  a  novel  framework  for  simultaneous  structure  and  parameter  learning  in  hidden-variable conditional probability models,  based on an  en tropic  prior and  a solution  for its maximum a posteriori (MAP) estimator.  The MAP estimate minimizes uncertainty  in  all  respects:  cross-entropy  between  model  and  data;  entropy  of the  model ;  entropy  of  the  data's  descriptive  statistics.  Iterative  estimation  extinguishes  weakly  supported  parameters,  compressing and  sparsifying  the  model.  Trimming operators  accelerate  this  process  by  removing  excess  parameters  and,  unlike  most  pruning  schemes,  guarantee  an  increase  in  posterior  probability.  Entropic  estimation  takes  a  overcomplete  random  model  and simplifies  it,  inducing the  structure of relations  between  hidden  and  observed  variables.  Applied  to  hidden  Markov  models  (HMMs),  it  finds  a  concise  finite-state  machine  representing  the  hidden  structure  of a  signal.  We  entropically  model  music,  handwriting, and video time-series, and show that the resulting models are highly concise,  structured,  predictive,  and  interpretable:  Surviving  states  tend  to  be  highly  correlated  with meaningful partitions of the data, while surviving transitions provide a low-perplexity  model of the signal dynamics. 
1  . An entropic prior  In  entropic  estimation  we  seek to  maximize  the  information  content of parameters.  For  conditional  probabilities,  parameters  values  near  chance  add  virtually  no  information  to  the  model,  and  are  therefore  wasted  degrees  of  freedom.  In  contrast,  parameters  near  the  extrema  {O, I}  are  informative  because  they  impose  strong  constr·aints  on  the  class  of signals  accepted  by  the  model.  In  Bayesian  terms,  our  prior should  assert  that  parameters that do not reduce uncertainty are improbable.  We  can capture this intuition in  a surprisingly simple form:  For a model of N  conditional probabilities 9  = {(h , . . . , () N  }  we write 
(1) 
whence we can  see that the prior measures a model's freedom from ambiguity (H(9) is an  entropy measure).  Applying Pe (.)  to a multinomial yields the posterior 
p  (LlI)  P(wI9)Pe (9)  e  U  W
The Effect of Correlations on the Fisher Information of Population Codes
Authors: Haim Sompolinsky Hyoungsoo Yoon
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  study  the  effect  of correlated  noise  on  the  accuracy  of popu(cid:173) lation  coding  using  a  model  of  a  population  of  neurons  that  are  broadly  tuned  to  an  angle  in  two-dimension.  The  fluctuations  in  the neuronal activity is  modeled  as  a  Gaussian noise with pairwise  correlations which decays exponentially with the difference between  the preferred orientations of the pair.  By calculating the Fisher in(cid:173) formation  of the  system,  we  show  that in  the biologically  relevant  regime of parameters positive correlations decrease the estimation  capability  of the  network  relative  to  the  uncorrelated  population.  Moreover strong positive  correlations  result  in  information  capac(cid:173) ity  which  saturates  to a  finite  value  as  the  number  of cells  in  the  population  grows.  In  contrast,  negative  correlations  substantially  increase the information capacity of the neuronal  population.
Spike-Based Compared to Rate-Based Hebbian Learning
Authors: Wulfram Gerstner, J. Leo van Hemmen Richard Kempter
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  correlation-based learning  rule  at the  spike  level  is  formulated,  mathematically analyzed, and compared to learning in a firing-rate  description.  A  differential  equation  for  the  learning  dynamics  is  derived  under the assumption  that the time scales of learning and  spiking  can  be  separated.  For  a  linear  Poissonian  neuron  model  which receives time-dependent stochastic input we show that spike  correlations on  a  millisecond time scale play  indeed  a  role.  Corre(cid:173) lations between input and output spikes tend to stabilize structure  formation,  provided  that  the  form  of  the  learning  window  is  in  accordance with Hebb's  principle.  Conditions for  an intrinsic nor(cid:173) malization of the average synaptic weight are discussed.
Contrast Adaptation in Simple Cells by Changing the Transmitter Release Probability
Authors: Klaus Obermayer Péter Adorján
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The contrast response function (CRF) of many neurons in the primary vi(cid:173) sual  cortex saturates and shifts towards higher contrast values following  prolonged presentation of high contrast visual stimuli.  Using a recurrent  neural  network of excitatory spiking neurons with adapting synapses we  show  that both effects could  be  explained  by  a fast  and  a slow compo(cid:173) nent in the synaptic adaptation.  (i) Fast synaptic depression leads to sat(cid:173) uration  of the CRF and  phase  advance  in  the cortical  response to  high  contrast stimuli.  (ii) Slow adaptation of the synaptic transmitter release  probability is derived such that the mutual information between the input  and  the output of a cortical neuron  is  maximal.  This component-given  by  infomax learning rule-explains contrast adaptation of the averaged  membrane  potential  (DC  component)  as  well  as  the surprising experi(cid:173) mental  result,  that  the  stimulus  modulated component  (Fl  component)  of a cortical cell's membrane potential adapts only weakly. Based on our  results,  we propose a new  experiment to estimate the strength of the ef(cid:173) fective  excitatory  feedback  to  a  cortical  neuron,  and  we  also  suggest a  relatively  simple experimental  test to justify our hypothesized synaptic  mechanism for contrast adaptation.
Optimizing Correlation Algorithms for Hardware-Based Transient Classification
Authors: Gert Cauwenberghs, Fernando J. Pineda R. Timothy Edwards
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The perfonnance of dedicated VLSI neural processing hardware depends  critically  on  the  design  of the  implemented  algorithms.  We  have  pre(cid:173) viously  proposed an  algorithm  for  acoustic  transient classification  [1].  Having implemented and demonstrated this algorithm in a mixed-mode  architecture,  we  now  investigate  variants  on  the algorithm,  using  time  and frequency channel differencing, input and output nonnalization, and  schemes to binarize and train the template values, with the goal of achiev(cid:173) ing optimal classification perfonnance for the chosen hardware.
Information Maximization in Single Neurons
Authors: Christof Koch Martin Stemmler
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Information from  the  senses must be compressed into the  limited range  of firing  rates  generated  by  spiking  nerve  cells.  Optimal  compression  uses all firing rates equally often, implying that the nerve cell's response  matches  the  statistics  of naturally  occurring  stimuli.  Since  changing  the  voltage-dependent ionic  conductances  in  the  cell  membrane  alters  the  flow  of information,  an  unsupervised,  non-Hebbian,  developmental  learning  rule  is  derived  to  adapt  the  conductances  in  Hodgkin-Huxley  model  neurons.  By  maximizing  the  rate  of information  transmission,  each firing rate within the model neuron's limited dynamic range is  used  equally often . 
An  efficient neuronal representation of incoming sensory  information should take  advan(cid:173) tage  of the  regularity  and  scale  invariance  of stimulus  features  in  the  natural  world.  In  the  case  of vision,  this  regularity  is  reflected  in  the  typical  probabilities of encountering  particular visual contrasts, spatial orientations, or colors [1].  Given these probabilities, an  optimized neural  code  would eliminate  any  redundancy, while  devoting  increased  repre(cid:173) sentation to commonly encountered features. 
At the level of a single spiking neuron, information about a potentially large range of stimuli  is compressed into a finite range of firing rates, since the maximum firing rate of a neuron is  limited.  Optimizing the information transmission through a single neuron in the  presence  of uniform, additive noise has an intuitive interpretation:  the most efficient representation  of the input uses every firing rate with equal probability.  An analogous principle for non(cid:173) spiking neurons has been tested experimentally by Laughlin [2], who matched the statistics 
Information Maximization in Single Neurons 
161
Mechanisms of Generalization in Perceptual Learning
Authors: Daphna Weinshall Zili Liu
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The learning of many visual perceptual tasks has been shown to be  specific  to practiced stimuli,  while  new  stimuli  require re-Iearning  from  scratch.  Here  we  demonstrate  generalization  using  a  novel  paradigm in motion discrimination  where learning has been previ(cid:173) ously  shown  to  be  specific.  We  trained  subjects  to  discriminate  the  directions  of  moving  dots,  and  verified  the  previous  results  that learning does not transfer from the trained direction to a  new  one.  However,  by  tracking  the  subjects'  performance  across  time  in  the new  direction, we  found  that their rate of learning doubled.  Therefore, learning generalized in  a  task previously considered too  difficult  for  generalization.  We  also  replicated,  in  the  second  ex(cid:173) periment, transfer following  training with  "easy"  stimuli.  The specificity  of perceptual learning and  the  dichotomy  between  learning of "easy"  vs.  "difficult"  tasks were hypothesized to involve  different  learning  processes,  operating  at  different  visual  cortical  areas.  Here we show how to interpret these results in terms of signal  detection  theory.  With  the  assumption  of limited  computational  resources,  we  obtain  the  observed  phenomena  - direct  transfer  and change of learning rate - for increasing levels of task 'difficulty.  It  appears  that  human  generalization  concurs  with  the  expected  behavior of a  generic discrimination system.
Probabilistic Image Sensor Fusion
Authors: Todd K. Leen, Misha Pavel Ravi K. Sharma
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present  a  probabilistic  method  for  fusion  of images produced  by multiple sensors.  The approach is  based on  an image formation  model in which  the sensor images are  noisy, locally linear functions  of an  underlying,  true  scene.  A  Bayesian framework  then  provides  for  maximum likelihood or  maximum a  posteriori  estimates of the  true  scene  from  the sensor  images.  Maximum likelihood estimates  of  the  parameters  of  the  image  formation  model  involve  (local)  second order image statistics, and thus are related to local principal  component  analysis.  We  demonstrate  the  efficacy  of the  method  on images from  visible-band  and infrared sensors.
On-Line Learning with Restricted Training Sets: Exact Solution as Benchmark for General Theories
Authors: Peter Sollich, Anthony C. C. Coolen H. C. Rae
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: O(ws(s log d+log(dqh/ s))) and O(ws((h/ s) log q) +log(dqh/ s)) are  upper bounds for  the VC-dimension of a  set of neural networks of  units  with  piecewise  polynomial  activation  functions,  where  s  is  the  depth  of  the  network,  h  is  the  number  of  hidden  units,  w  is  the  number  of  adjustable  parameters,  q  is  the  maximum  of  the  number of polynomial segments of the activation function, and d is  the  maximum degree  of  the polynomials;  also  n(wslog(dqh/s))  is  a  lower  bound  for  the VC-dimension  of such  a  network set,  which  are tight for  the cases  s =  8(h)  and  s is  constant.  For the special  case q =  1,  the VC-dimension is  8(ws log d).
Phase Diagram and Storage Capacity of Sequence-Storing Neural Networks
Authors: Anthony C. C. Coolen, D. Sherrington A. Düring
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We solve the dynamics of Hopfield-type neural networks which store se(cid:173) quences of patterns, close to saturation. The asymmetry of the interaction  matrix in such models leads to violation of detailed balance, ruling out an  equilibrium statistical mechanical analysis.  Using generating functional  methods we derive exact closed equations for dynamical order parame(cid:173) ters,  viz.  the  sequence overlap and  correlation and  response functions.  in  the  limit of an  infinite system size.  We  calculate the time  translation  invariant solutions of these equations. describing stationary limit-cycles.  which  leads  to  a phase diagram.  The  effective retarded  self-interaction  usually  appearing  in  symmetric  models  is  here  found  to  vanish,  which  causes  a  significantly  enlarged  storage  capacity  of eYe  ~ 0.269.  com(cid:173) pared to  eYe  ~ 0.139 for Hopfield networks s~oring static patterns.  Our  results  are  tested  against extensive  computer simulations and  excellent  agreement is found. 
212 
A.  Diiring,  A.  C.  C.  Coo/en  and D. Sherrington 
1  INTRODUCTION AND DEFINITIONS 
We  consider a system of N  neurons O'(t)  = {ai(t)  = ±1}, which can change their states  collectively  at  discrete  times  (parallel  dynamics).  Each  neuron  changes  its  state  with  a  probability Pi(t)  =  ~[l-tanh,Bai(t)[Lj Jijaj(t)+Oi(t)]], so that the transition matrix is  W[o'(s + l)IO'(s)]  = II e.BO',(s+l)[E;=l J, j O'} (s)+ o,( s)]-ln2cosh(i3[E; =1  J'JO'} ( s)+(J, (s )))
Example-Based Image Synthesis of Articulated Figures
Authors: Trevor Darrell
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present a  method  for  learning complex appearance mappings.  such  as  occur  with  images  of articulated  objects.  Traditional  interpolation  networks  fail  on  this  case since appearance is  not necessarily  a smooth  function  nor a linear manifold for articulated objects.  We  define an  ap(cid:173) pearance mapping from examples by constructing a set of independently  smooth interpolation networks; these networks can cover overlapping re(cid:173) gions of parameter space.  A  set  growing  procedure is  used  to  find  ex(cid:173) ample  clusters  which  are  well-approximated  within  their  convex  hull;  interpolation then proceeds only within these sets of examples. With this  method physically valid  images are produced even in regions of param(cid:173) eter space where nearby examples have different appearances.  We show  results generating both simulated and real arm images.
Shrinking the Tube: A New Support Vector Regression Algorithm
Authors: Peter L. Bartlett, Alex J. Smola, Robert C. Williamson Bernhard Schölkopf
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A new algorithm for Support Vector regression is  described.  For a priori  chosen 1/,  it automatically adjusts a flexible tube of minimal radius to the  data such that  at most a fraction  1/  of the data points lie outside.  More(cid:173) over,  it  is  shown  how  to  use  parametric  tube  shapes  with  non-constant  radius. The algorithm is analysed theoretically and experimentally.
Lazy Learning Meets the Recursive Least Squares Algorithm
Authors: Gianluca Bontempi, Hugues Bersini Mauro Birattari
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Lazy  learning  is  a  memory-based  technique  that,  once  a  query  is  re(cid:173) ceived, extracts a prediction interpolating locally the neighboring exam(cid:173) ples of the query which are considered relevant according to  a distance  measure.  In  this paper we propose a data-driven method to  select on a  query-by-query basis the optimal number of neighbors to be considered  for  each  prediction.  As  an  efficient way  to  identify  and  validate  local  models, the  recursive least  squares  algorithm  is  introduced  in  the  con(cid:173) text  of local  approximation and lazy learning.  Furthermore,  beside the  winner-takes-all strategy for model selection, a local combination of the  most promising models is explored.  The method  proposed  is  tested  on  six different datasets and compared with a state-of-the-art approach.
Distributional Population Codes and Multiple Motion Models
Authors: Peter Dayan Richard S. Zemel
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Most theoretical and empirical studies of population codes make  the assumption that underlying neuronal activities is a unique and  unambiguous value of an encoded quantity. However, population  activities can contain additional information about such things as  multiple values of or uncertainty about the quantity. We have pre(cid:173) viously suggested a method to recover extra information by treat(cid:173) ing the activities  of the population of cells  as coding for  a com(cid:173) plete distribution over the coded quantity rather than just a single  value.  We  now  show how  this approach bears on psychophys(cid:173) ical and neurophysiological studies of population codes for mo(cid:173) tion direction in tasks involving transparent motion stimuli.  We  show that, unlike standard approaches, it is able to recover mul(cid:173) tiple motions from population responses, and also that its output  is consistent with both correct and erroneous human performance  on psychophysical tasks. 
A  population code can be defined  as a  set of units  whose  activities  collectively  encode some underlying variable (or variables).  The standard view is that popu(cid:173) lation codes are useful for accurately encoding the underlying variable when the  individual units are noisy.  Current statistical approaches to interpreting popula(cid:173) tion activity reflect this view, in that they determine the optimal single value that  explains the observed activity pattern given a particular model of the noise (and  possibly a loss function).  In our work, we have pursued an alternative hypothesis, that the population en(cid:173) codes  additional information  about  the  underlying  variable,  including  multiple  values and uncertainty.  The Distributional Population Coding (DPC) framework  finds the best probability distribution across values that fits the population activity  (Zemel, Dayan, & Pouget, 1998).  The DPC framework is appealing since it makes clear how extra information can  be conveyed in a population code.  In this paper, we use it to address a particu-
Distributional Population  Codes and Multiple Motion Models
Using Collective Intelligence to Route Internet Traffic
Authors: Kagan Tumer, Jeremy Frank David Wolpert
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  COllective INtelligence  (COIN)  is  a  set of interacting reinforce(cid:173) ment  learning  (RL)  algorithms  designed  in  an  automated fashion  so that their collective behavior optimizes a global utility function.  We summarize the theory of COINs, then present experiments us(cid:173) ing that theory to design COINs to control internet traffic routing.  These experiments  indicate that COINs  outperform all previously  investigated RL-based, shortest path routing algorithms.
Sparse Code Shrinkage: Denoising by Nonlinear Maximum Likelihood Estimation
Authors: Patrik O. Hoyer, Erkki Oja Aapo Hyvärinen
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Sparse  coding  is  a  method  for  finding  a  representation  of data in  which  each  of the  components of the representation is  only  rarely  significantly  active.  Such  a  representation is  closely  related  to re(cid:173) dundancy reduction and independent component analysis,  and has  some  neurophysiological  plausibility.  In  this  paper,  we  show  how  sparse coding can be used for denoising.  Using maximum likelihood  estimation of nongaussian variables corrupted by gaussian noise, we  show  how  to  apply a  shrinkage nonlinearity on  the components  of  sparse coding so  as  to reduce noise.  Furthermore,  we  show  how  to  choose the optimal sparse coding basis for  denoising.  Our method  is  closely  related  to  the  method  of wavelet  shrinkage,  but  has  the  important benefit over wavelet methods that both the features and  the shrinkage parameters are estimated directly from  the data.
Finite-Dimensional Approximation of Gaussian Processes
Authors: Christopher K. I. Williams, Manfred Opper Giancarlo Ferrari-Trecate
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Gaussian process (GP)  prediction suffers from  O(n3)  scaling with the  data set size n.  By using a finite-dimensional basis to approximate the  GP predictor,  the computational complexity  can be reduced.  We  de(cid:173) rive optimal finite-dimensional  predictors under a  number of assump(cid:173) tions,  and show  the superiority of these predictors over the  Projected  Bayes  Regression method  (which  is  asymptotically optimal).  We  also  show  how  to  calculate  the  minimal  model  size  for  a  given  n.  The  calculations are backed up  by numerical experiments.
Controlling the Complexity of HMM Systems by Regularization
Authors: Gerhard Rigoll Christoph Neukirchen
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: This paper introduces a method for regularization ofHMM systems that  avoids parameter overfitting caused by insufficient training data.  Regu(cid:173) larization is  done by augmenting the  EM  training method by a penalty  term  that  favors  simple  and  smooth HMM  systems.  The penalty  term  is  constructed as  a mixture model of negative exponential distributions  that is assumed to generate the state dependent emission probabilities of  the HMMs.  This new method is the successful transfer of a well known  regularization approach in neural networks to the HMM domain and can  be interpreted as a generalization of traditional state-tying for HMM sys(cid:173) tems.  The effect of regularization is demonstrated for continuous speech  recognition tasks by improving overfitted triphone models and by speaker  adaptation with limited training data.
Scheduling Straight-Line Code Using Reinforcement Learning and Rollouts
Authors: J. Eliot B. Moss Amy McGovern
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: In  1986, Tanner and Mead [1] implemented an interesting constraint sat(cid:173) isfaction  circuit  for  global  motion  sensing  in  a VLSI.  We  report  here  a  new  and  improved a VLSI implementation that provides smooth optical  flow as well as global motion in a two dimensional visual field.  The com(cid:173) putation of optical flow  is  an ill-posed problem, which expresses itself as  the aperture problem.  However, the optical flow  can be estimated by the  use of regularization methods, in  which additional constraints are intro(cid:173) duced in  terms of a global energy functional that must be minimized . We  show how the algorithmic constraints of Hom and Schunck [2]  on com(cid:173) puting smooth optical flow can be mapped onto the physical constraints  of an equivalent electronic network.
Learning to Estimate Scenes from Images
Authors: Egon C. Pasztor William T. Freeman
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  seek  the  scene  interpretation  that  best  explains  image  data.  For example,  we  may want to infer the projected velocities  (scene)  which  best  explain  two  consecutive  image  frames  (image).  From  synthetic data , we  model the relationship between image and scene  patches, and between a scene patch and neighboring scene patches.  Given' a  new  image,  we  propagate likelihoods in  a  Markov network  (ignoring  the  effect  of loops)  to  infer  the  underlying  scene.  This  yields  an  efficient  method  to form  low-level  scene  interpretations.  We  demonstrate the technique for  motion analysis  and estimating  high  resolution images from  low-resolution ones.
Facial Memory Is Kernel Density Estimation (Almost)
Authors: Garrison W. Cottrell, Thomas A. Busey Matthew N. Dailey
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  compare the  ability  of three exemplar-based memory models,  each  using  three  different  face  stimulus  representations,  to  account  for  the  probability a human subject responded "old" in an old/new facial mem(cid:173) ory experiment.  The models are  1) the  Generalized Context Model, 2)  SimSample,  a  probabilistic  sampling  model,  and  3)  MMOM,  a  novel  model related to kernel density estimation that explicitly encodes stim(cid:173) ulus  distinctiveness.  The representations  are  1)  positions of stimuli  in  MDS "face space," 2) projections of test faces onto the  "eigenfaces" of  the study set, and 3) a representation based on response to a grid of Gabor  filter jets.  Of the 9 model/representation combinations, only the distinc(cid:173) tiveness model in  MDS  space predicts the observed "morph familiarity  inversion" effect, in  which the subjects'  false alarm rate for morphs be(cid:173) tween similar faces  is higher than their hit rate for many of the  studied  faces.  This evidence is consistent with the hypothesis that human mem(cid:173) ory for faces is a kernel density estimation task, with the caveat that dis(cid:173) tinctive faces require larger kernels than do typical faces.
VLSI Implementation of Motion Centroid Localization for Autonomous Navigation
Authors: Viktor Gruev, Mohammed Abdel Ghani Ralph Etienne-Cummings
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  circuit  for  fast,  compact and  low-power focal-plane  motion  centroid  localization  is  presented.  This  chip,  which  uses  mixed  signal  CMOS  components  to  implement  photodetection,  edge  detection,  ON-set  detection  and  centroid  localization,  models  the  retina  and  superior  colliculus.  The  centroid  localization  circuit  uses  time-windowed  asynchronously  triggered  row  and  column  address  events  and  two  linear  resistive  grids  to  provide  the  analog  coordinates  of the  motion  centroid.  This  VLSI  chip  is  used  to  realize  fast  lightweight  autonavigating  vehicles.  The  obstacle  avoiding  line-following  algorithm is  discussed.
Discontinuous Recall Transitions Induced by Competition Between Short- and Long-Range Interactions in Recurrent Networks
Authors: C. F. Beckmann, Anthony C. C. Coolen N. S. Skantzos
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present exact analytical  equilibrium solutions  for  a  class  of recur(cid:173) rent neural network models,  with  both sequential and parallel neuronal  dynamics,  in  which  there  is  a  tunable  competition  between  nearest(cid:173) neighbour  and  long-range  synaptic  interactions.  This  competition  is  found  to induce novel coexistence phenomena as  well  as  discontinuous  transitions between pattern recall states, 2-cycles and non-recall states.
An Integrated Vision Sensor for the Computation of Optical Flow Singular Points
Authors: Christof Koch Charles M. Higgins
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A robust, integrative algorithm is presented for computing the position of  the focus  of expansion or axis of rotation (the singular point)  in optical  flow  fields  such  as  those  generated  by  self-motion.  Measurements  are  shown of a fully parallel CMOS analog VLSI motion sensor array which  computes the direction of local motion (sign of optical flow) at each pixel  and can directly implement this algorithm.  The flow  field  singular point  is  computed in real time  with a power consumption of less than 2 m W.  Computation of the singular point for  more general flow  fields  requires  measures of field  expansion and  rotation,  which  it  is  shown can also  be  computed in real-time hardware, again using only the sign of the optical  flow  field.  These measures, along with the location of the singular point,  provide robust real-time self-motion information for the visual guidance  of a moving platform such as a robot.
Robust, Efficient, Globally-Optimized Reinforcement Learning with the Parti-Game Algorithm
Authors: Ronald J. Williams Mohammad A. Al-Ansari
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Parti-game (Moore 1994a; Moore 1994b; Moore and Atkeson  1995) is a  reinforcement learning (RL) algorithm that has a lot of promise in over(cid:173) coming the curse of dimensionality that can plague RL algorithms when  applied to high-dimensional problems.  In  this paper we  introduce mod(cid:173) ifications to  the algorithm that further improve its performance and ro(cid:173) bustness. In addition, while parti-game solutions can be improved locally  by standard local path-improvement techniques, we introduce an add-on  algorithm in  the same spirit  as  parti-game that instead tries  to  improve  solutions in a non-local manner.
Learning a Hierarchical Belief Network of Independent Factor Analyzers
Authors: Hagai Attias
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Many  belief  networks  have  been  proposed  that  are  composed  of  binary units.  However,  for  tasks such  as  object  and speech  recog(cid:173) nition  which  produce real-valued  data,  binary network models  are  usually inadequate.  Independent component analysis  (ICA)  learns  a  model  from  real  data,  but  the  descriptive  power  of this  model  is  severly  limited.  We  begin  by  describing the  independent  factor  analysis  (IFA)  technique,  which  overcomes some of the limitations  of ICA.  We  then  create  a  multilayer  network  by  cascading single(cid:173) layer  IFA  models.  At  each  level,  the  IFA  network  extracts  real(cid:173) valued  latent  variables  that  are  non-linear  functions  of the  input  data  with  a  highly  adaptive  functional  form,  resulting  in  a  hier(cid:173) archical  distributed  representation  of  these  data.  Whereas  exact  maximum-likelihood  learning of the network  is  intractable,  we  de(cid:173) rive an algorithm that maximizes a  lower  bound on the likelihood,  based on a  variational approach.
A Reinforcement Learning Algorithm in Partially Observable Environments Using Short-Term Memory
Authors: Akira Hayashi Nobuo Suematsu
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  describe  a  Reinforcement Learning  algorithm  for  partially  observ(cid:173) able environments using short-term memory, which  we call BLHT. Since  BLHT learns a stochastic model  based on Bayesian Learning, the over(cid:173) fitting  problem  is  reasonably  solved.  Moreover,  BLHT has  an  efficient  implementation. This paper shows that the model learned by BLHT con(cid:173) verges  to  one  which  provides the  most accurate  predictions of percepts  and rewards, given short-term memory.
Maximum-Likelihood Continuity Mapping (MALCOM): An Alternative to HMMs
Authors: John E. Hogden David A. Nix
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We describe Maximum-Likelihood Continuity Mapping (MALCOM), an  alternative to  hidden Markov models (HMMs) for processing sequence  data such as speech.  While HMMs have a discrete "hidden" space con(cid:173) strained by  a fixed  finite-automaton architecture, MALCOM has a con(cid:173) tinuous hidden space-a continuity map-that is  constrained only  by  a  smoothness requirement on paths through the space.  MALCOM fits  into  the same probabilistic framework for speech recognition as  HMMs, but  it represents  a  more  realistic  model  of the  speech  production  process.  To  evaluate the extent to which MALCOM captures speech production  information, we  generated continuous speech continuity maps  for three  speakers  and  used  the  paths  through  them  to  predict measured  speech  articulator data.  The median  correlation  between  the MALCOM paths  obtained from  only the  speech  acoustics  and  articulator measurements  was 0.77 on an  independent test set not used  to train MALCOM or the  predictor.  This  unsupervised  model  achieved  correlations  over speak(cid:173) ers and articulators only 0.02 to 0.15 lower than those obtained using an  analogous supervised method which used articulatory measurements as  well as acoustics ..
Semiparametric Support Vector and Linear Programming Machines
Authors: Thilo-Thomas Frieß, Bernhard Schölkopf Alex J. Smola
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Semiparametric models  are  useful  tools  in the case  where  domain  knowledge exists about the function to be estimated or emphasis is  put onto understandability of the  model.  We  extend two learning  algorithms  - Support  Vector  machines  and  Linear  Programming  machines  to  this  case  and  give  experimental  results  for  SV  ma(cid:173) chines.
Regularizing AdaBoost
Authors: Takashi Onoda, Klaus R. Müller Gunnar Rätsch
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Boosting methods  maximize  a  hard  classification  margin  and  are  known as powerful techniques that do not exhibit overfitting for low  noise  cases.  Also for noisy data boosting will  try to enforce a  hard  margin and thereby give  too much  weight  to outliers,  which  then  leads to the dilemma of non-smooth fits  and overfitting.  Therefore  we  propose three algorithms to allow for  soft margin classification  by introducing regularization with slack variables into the boosting  concept:  (1)  AdaBoostreg  and  regularized  versions  of  (2)  linear  and  (3)  quadratic programming AdaBoost.  Experiments show  the  usefulness of the proposed algorithms in comparison to another soft  margin classifier:  the support vector machine. 
1
Graph Matching for Shape Retrieval
Authors: Andrew D. J. Cross, Edwin R. Hancock Benoit Huet
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  propose a new in-sample cross validation based method (randomized  GACV) for choosing smoothing or bandwidth parameters that govern the  bias-variance or fit-complexity tradeoff in  'soft' classification.  Soft clas(cid:173) sification refers to  a  learning procedure which  estimates the probability  that an example with a given attribute vector is  in  class  1 vs  class O.  The  target  for  optimizing  the  the  tradeoff is  the  Kullback-Liebler distance  between  the  estimated  probability  distribution  and  the  'true'  probabil(cid:173) ity  distribution,  representing  knowledge of an  infinite  population.  The  method uses a randomized estimate of the trace of a Hessian and mimics  cross validation at the cost of a single relearning with perturbed outcome  data.
DTs: Dynamic Trees
Authors: Nicholas J. Adams Christopher K. I. Williams
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: In  this  paper we  introduce a new  class of image models,  which  we  call  dynamic trees or  DTs.  A dynamic tree model specifies  a prior  over a  large number of trees, each one of which is  a tree-structured  belief  net  (TSBN).  Experiments  show  that  DTs  are  capable  of  generating images that are less blocky, and the models have better  translation  invariance properties  than  a  fixed,  "balanced"  TSBN.  We  also show that Simulated Annealing is effective at finding trees  which  have high  posterior probability.
Adding Constrained Discontinuities to Gaussian Process Models of Wind Fields
Authors: Ian T. Nabney, Christopher K. I. Williams Dan Cornford
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Gaussian Processes provide good prior models for  spatial data,  but can  be  too  smooth.  In  many  physical  situations  there  are  discontinuities  along bounding surfaces, for example fronts in near-surface wind fields.  We  describe  a  modelling  method  for  such  a  constrained  discontinuity  and demonstrate how to  infer the  model parameters in wind fields  with  MCMC sampling.
The Belief in TAP
Authors: David Saad Yoshiyuki Kabashima
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  show  the  similarity  between  belief propagation  and  TAP,  for  decoding  corrupted  messages  encoded  by  Sourlas's  method.  The  latter is  a  special  case of the Gallager  error-correcting code, where  the code word comprises products of J{ bits selected randomly from  the original message.  We examine the efficacy of solutions obtained  by the two methods for various values of J{ and show that solutions  for  J{  2':  3  may  be  sensitive  to  the  choice  of initial  conditions  in  the  case  of unbiased patterns.  Good  approximations  are obtained  generally  for  J{ = 2  and  for  biased  patterns  in  the  case  of  J{ 2':  3,  especially when  Nishimori's  temperature is  being used.
Synergy and Redundancy among Brain Cells of Behaving Monkeys
Authors: Naftali Tishby Itay Gat
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Determining the relationship between  the activity of a single nerve  cell  to that of an entire  population is  a  fundamental question  that  bears  on  the  basic  neural  computation  paradigms.  In  this  paper  we  apply  an  information theoretic  approach  to  quantify  the  level  of cooperative  activity  among cells  in  a  behavioral  context.  It is  possible to discriminate between  synergetic  activity of the cells  vs .  redundant  activity, depending on the difference  between  the infor(cid:173) mation  they  provide  when  measured  jointly  and  the  information  they  provide independently.  We define  a synergy  value that is  pos(cid:173) itive in the first  case and negative in  the second  and show  that the  synergy value can be measured by detecting the behavioral mode of  the animal from  simultaneously recorded  activity of the  cells.  We  observe  that  among  cortical  cells  positive  synergy  can  be  found,  while cells  from the  basal ganglia, active during the same task, do  not  exhibit similar synergetic  activity. 
titay,tishby}@cs.huji.ac.il  Permanent  address:  Institute  of Computer  Science  and  Center for  Neural  Computa(cid:173)
tion,  The  Hebrew  University,  Jerusalem  91904,  Israel.