mtybilly/Instruct2DS huggingface dataset:
Multiresolution Tangent Distance for Affine-invariant Classification
Authors: Andrew Lippman Nuno Vasconcelos
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: The ability  to  rely  on similarity  metrics  invariant to  image transforma(cid:173) tions is  an important issue for image classification tasks such as face or  character recognition. We analyze an invariant metric that has performed  well for the latter - the tangent distance - and study its limitations when  applied to regular images, showing that the most significant among these  (convergence to local minima) can be drastically reduced by computing  the distance in a multiresolution setting. This leads to the multi resolution  tangent distance,  which exhibits  significantly  higher invariance  to  im(cid:173) age transformations, and can be easily combined with robust estimation  procedures.
Effects of Spike Timing Underlying Binocular Integration and Rivalry in a Neural Model of Early Visual Cortex
Authors: Erik D. Lumer
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: In normal vision, the inputs from the two eyes are inte(cid:173) grated into a single percept. When dissimilar images are  presented to the two eyes, however, perceptual integra(cid:173) tion gives way to alternation between monocular inputs,  a phenomenon called binocular rivalry. Although recent  evidence indicates that binocular rivalry involves a mod(cid:173) ulation of neuronal responses in extrastriate cortex, the  basic mechanisms responsible for differential processing of  con:6.icting and congruent stimuli remain unclear. Using a  neural network that models the mammalian early visual  system, I demonstrate here that the desynchronized fir(cid:173) ing of cortical-like neurons that first receive inputs from  the two eyes results in rivalrous activity patterns at later  stages in the visual pathway. By contrast, synchronization  of firing among these cells prevents such competition. The  temporal coordination of cortical activity and its effects  on neural competition emerge naturally from the network  connectivity and from its dynamics. These results suggest  that input-related differences in relative spike timing at  an early stage of visual processing may give rise to the  phenomena both of perceptual integration and rivalry in  binocular vision.
Detection of First and Second Order Motion
Authors: Heiko Neumann Alexander Grunewald
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A  model  of  motion  detection  is  presented.  The  model  contains  three stages.  The first  stage is  unoriented and is  selective for  con(cid:173) trast  polarities.  The  next  two  stages  work  in  parallel.  A  phase  insensitive stage pools across  different  contrast polarities through  a  spatiotemporal filter  and thus can detect  first  and second order  motion.  A phase sensitive stage keeps  contrast polarities separate,  each of which is  filtered  through  a  spatiotemporal filter,  and thus  only first order motion can be detected.  Differential phase sensitiv(cid:173) ity can therefore account for  the detection of first  and second order  motion.  Phase insensitive detectors correspond to cortical complex  cells,  and phase sensitive detectors to simple  cells.
A Framework for Multiple-Instance Learning
Authors: Tomás Lozano-Pérez Oded Maron
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Multiple-instance learning is a variation on supervised learning, where the  task  is  to  learn  a  concept  given  positive and  negative  bags  of instances.  Each  bag  may  contain many  instances,  but a  bag  is  labeled  positive even  if only one of the instances in it falls  within the concept.  A  bag is  labeled  negative  only  if all  the  instances  in  it are  negative.  We  describe  a  new  general  framework,  called Diverse  Density,  for  solving multiple-instance  learning problems.  We  apply this framework to learn a simple description  of a person from a series of images (bags) containing that person, to a stock  selection problem, and to the drug activity prediction problem.
Monotonic Networks
Authors: Joseph Sill
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Monotonicity  is  a  constraint which  arises  in many application do(cid:173) mains.  We present  a  machine learning model,  the monotonic net(cid:173) work, for which monotonicity can be enforced exactly, i.e., by virtue  offunctional form .  A straightforward method for implementing and  training  a  monotonic  network  is  described.  Monotonic  networks  are  proven  to  be  universal  approximators of continuous,  differen(cid:173) tiable  monotonic functions.  We  apply  monotonic  networks  to  a  real-world  task  in  corporate  bond  rating  prediction  and  compare  them to other approaches.
Relative Loss Bounds for Multidimensional Regression Problems
Authors: Manfred K. Warmuth Jyrki Kivinen
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  study  on-line generalized  linear  regression  with  multidimensional  outputs, i.e.,  neural  networks with multiple output nodes  but no  hidden  nodes.  We  allow  at the  final  layer transfer functions  such  as  the soft(cid:173) max function that need to consider the linear activations to all the output  neurons.  We use distance functions of a certain kind in two completely  independent roles in deriving and analyzing on-line learning algorithms  for  such tasks.  We use one distance function to define a matching loss  function for the (possibly multidimensional) transfer function, which al(cid:173) lows us to generalize earlier results from one-dimensional to multidimen(cid:173) sional outputs. We use another distance function as a tool for measuring  progress made by the on-line updates.  This shows how previously stud(cid:173) ied  algorithms  such  as  gradient  descent  and  exponentiated gradient fit  into a common  framework.  We  evaluate  the  performance of the  algo(cid:173) rithms  using relative  loss  bounds  that  compare  the  loss  of the  on-line  algoritm to the best off-line predictor from the relevant model class, thus  completely eliminating probabilistic assumptions about the data.
Gradients for Retinotectal Mapping
Authors: Geoffrey J. Goodhill
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: The initial activity-independent formation of a  topographic map  in  the retinotectal  system has  long been thought  to  rely  on  the  matching of molecular cues expressed in gradients in the retina  and  the  tectum.  However,  direct experimental evidence for  the  existence of such gradients has only emerged since 1995. The new  data has provoked the discussion of a new set of models in the ex(cid:173) perimentalliterature. Here, the capabilities of these models are an(cid:173) alyzed, and the gradient shapes they predict in vivo are derived.
Multi-modular Associative Memory
Authors: David Horn, Eytan Ruppin Nir Levy
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Motivated  by  the findings  of modular structure  in  the association  cortex,  we study a multi-modular model of associative memory that  can successfully  store  memory patterns  with different  levels  of ac(cid:173) tivity.  We show that the segregation of synaptic conductances into  intra-modular linear and inter-modular nonlinear ones considerably  enhances  the  network's  memory retrieval  performance.  Compared  with  the conventional, single-module associative  memory network,  the multi-modular network has two main advantages:  It is  less sus(cid:173) ceptible to damage to columnar input, and its response is consistent  with the cognitive data pertaining to category specific impairment.
Analysis of Drifting Dynamics with Neural Network Hidden Markov Models
Authors: Klaus-Robert Müller, Klaus Pawelzik Jens Kohlmorgen
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  present  a  method  for  the  analysis  of  nonstationary  time  se(cid:173) ries  with multiple  operating modes.  In particular, it is possible to  detect and to model  both a  switching  of the dynamics  and a  less  abrupt,  time  consuming  drift  from  one  mode  to another.  This  is  achieved in two steps.  First, an unsupervised training method pro(cid:173) vides  prediction experts for  the inherent dynamical modes.  Then,  the trained experts are used in a hidden Markov model that allows  to  model  drifts.  An  application  to  physiological  wake/sleep  data  demonstrates that analysis and modeling of real-world time series  can be improved when the drift paradigm is  taken into account.
The Observer-Observation Dilemma in Neuro-Forecasting
Authors: Ralph Neuneier Hans-Georg Zimmermann
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  explain how the training data can  be separated  into  clean informa(cid:173) tion and unexplainable noise.  Analogous to the data, the neural network  is  separated  into  a  time invariant  structure used  for  forecasting,  and  a  noisy part.  We propose a unified theory connecting the optimization al(cid:173) gorithms for cleaning and learning together with algorithms that control  the data noise and the parameter noise.  The combined algorithm allows  a data-driven local control of the liability of the network parameters and  therefore an  improvement  in generalization.  The approach is proven to  be very useful at the task of forecasting the German bond market. 
1  Introduction: The Observer-Observation Dilemma 
Human beings believe that they are able to solve a psychological version of the Observer(cid:173) Observation Dilemma. On the one hand, they use their observations to constitute an under(cid:173) standing of the laws of the world, on the other hand, they use this understanding to evaluate  the  correctness  of the incoming pieces  of information.  Of course,  as  everybody knows,  human beings are not free from making mistakes in this psychological dilemma.  We  en(cid:173) counter a similar situation when we try to build a mathematical model using data.  Learning  relationships from the data is only one part of the model building process.  Overrating this  part often leads to the phenomenon of overfitting in many applications (especially in eco(cid:173) nomic forecasting).  In practice, evaluation of the data is often done by external knowledge,  i. e.  by optimizing the model under constraints of smoothness and regularization [7].  If we  assume, that our model summerizes the best knowledge of the system to be identified, why  should we not use the model itself to evaluate the correctness of the data?  One approach to  do this is called Clearning [11].  In this paper, we present a unified approach of the interac(cid:173) tion between the data and a neural network (see also [8]).  It includes a new symmetric view  on the optimization algorithms, here learning and cleaning, and their control by parameter  and data noise. 
The Observer-Observation Dilemma in Neuro-Forecasting 
993
A Model of Early Visual Processing
Authors: Jochen Braun, Dale K. Lee, Christof Koch Laurent Itti
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  propose  a  model for  early  visual  processing  in  primates.  The  model consists  of a  population of linear spatial filters  which  inter(cid:173) act  through non-linear excitatory  and inhibitory pooling.  Statisti(cid:173) cal estimation theory  is  then used  to derive human psychophysical  thresholds from the responses of the entire population of units.  The  model is  able  to reproduce  human thresholds for  contrast  and ori(cid:173) entation discrimination tasks,  and to predict contrast thresholds in  the presence  of masks of varying orientation and spatial frequency.
A Simple and Fast Neural Network Approach to Stereovision
Authors: Rolf D. Henkel
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A  neural  network  approach  to  stereovision  is  presented  based  on  aliasing effects of simple disparity estimators and a fast coherence(cid:173) detection  scheme.  Within  a  single network structure, a  dense  dis(cid:173) parity  map  with  an  associated  validation  map  and,  additionally,  the  fused  cyclopean  view  of the scene  are  available.  The network  operations  are  based  on  simple,  biological  plausible  circuitry;  the  algorithm is  fully  parallel and non-iterative.
Data-Dependent Structural Risk Minimization for Perceptron Decision Trees
Authors: Nello Cristianini John Shawe-Taylor
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A novel neural network model of pre-attention processing in visual(cid:173) search  tasks  is  presented.  Using displays of line orientations taken  from  Wolfe's experiments  [1992], we  study the  hypothesis  that the  distinction  between  parallel  versus  serial  processes  arises  from  the  availability of global information in  the internal representations  of  the  visual  scene.  The  model  operates  in  two  phases.  First,  the  visual  displays  are  compressed  via  principal-component-analysis.  Second,  the compressed data is processed by a target detector mod(cid:173) ule in order to identify the existence of a  target in the display.  Our  main  finding  is  that  targets  in  displays  which  were  found  exper(cid:173) imentally  to  be  processed  in  parallel can  be  detected  by  the  sys(cid:173) tem,  while  targets  in  experimentally-serial  displays  cannot .  This  fundamental  difference  is  explained  via  variance  analysis  of  the  compressed representations,  providing a  numerical criterion distin(cid:173) guishing parallel from serial displays.  Our model yields a  mapping  of response-time slopes that is similar to Duncan and Humphreys's  "search  surface"  [1989],  providing  an  explicit  formulation  of their  intuitive  notion  of feature  similarity.  It presents  a  neural  realiza(cid:173) tion of the  processing that may underlie the classical metaphorical  explanations of visual search. 
On Parallel versus Serial Processing: A  Computational Study a/Visual Search
Using Expectation to Guide Processing: A Study of Three Real-World Applications
Authors: Shumeet Baluja
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: In many real world tasks, only a small fraction of the available inputs are important  at any  particular time. This paper presents a method for ascertaining the relevance  of inputs  by  exploiting  temporal  coherence  and  predictability.  The  method  pro(cid:173) posed in this paper dynamically allocates relevance to inputs by using expectations  of their  future  values.  As  a  model  of the  task  is  learned,  the  model  is  simulta(cid:173) neously extended to create task-specific predictions of the future values of inputs.  Inputs which are either not relevant, and therefore not accounted for in the model,  or those which contain noise, will not be predicted accurately. These inputs can be  de-emphasized, and, in turn, a new, improved, model of the task created. The tech(cid:173) niques  presented  in  this  paper  have  yielded  significant  improvements  for  the  vision-based  autonomous control of a land vehicle,  vision-based hand tracking in  cluttered scenes, and the detection of faults in the etching of semiconductor wafers.
Features as Sufficient Statistics
Authors: Archisman Rudra, Laurance T. Maloney Davi Geiger
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: An image is often represented by a set of detected features.  We get  an enormous compression by representing images in this way.  Fur(cid:173) thermore,  we  get a  representation which  is  little affected by  small  amounts  of  noise  in  the  image.  However,  features  are  typically  chosen  in  an  ad  hoc  manner.  tures can be obtained using sufficient statistics.  The idea of sparse  data representation  naturally  arises.  We  treat  the  I-dimensional  and 2-dimensional signal reconstruction problem to make our ideas  concrete. 
\Ve  show  how  a  good  set  of fea(cid:173)
Factorizing Multivariate Function Classes
Authors: Juan K. Lin
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: The mathematical framework for  factorizing equivalence classes of  multivariate  functions  is  formulated  in  this  paper.  Independent  component analysis is  shown to be a  special case of this decompo(cid:173) sition.  Using  only  the  local  geometric  structure  of a  class  repre(cid:173) sentative,  we  derive an analytic solution for  the factorization.  We  demonstrate the factorization solution with numerical experiments  and present a  preliminary tie to decorrelation.
An Annealed Self-Organizing Map for Source Channel Coding
Authors: Thore Graepel, Klaus Obermayer Matthias Burger
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  derive  and  analyse  robust  optimization  schemes  for  noisy  vector  quantization  on  the  basis  of deterministic  annealing.  Starting  from  a  cost  function  for  central  clustering  that  incorporates  distortions  from  channel  noise  we  develop  a  soft  topographic  vector  quantization  al(cid:173) gorithm  (STVQ)  which  is  based  on  the  maximum  entropy  principle  and which performs a maximum-likelihood estimate in  an expectation(cid:173) maximization (EM) fashion.  Annealing in the temperature parameter f3  leads to phase transitions in the existing code vector representation dur(cid:173) ing the cooling process for which we  calculate critical temperatures and  modes  as  a function of eigenvectors  and eigenvalues  of the covariance  matrix of the data and the transition matrix of the channel noise.  A whole  family  of vector quantization algorithms is derived from  STVQ, among  them  a  deterministic  annealing  scheme  for  Kohonen's  self-organizing  map  (SOM).  This  algorithm,  which  we  call  SSOM,  is  then  applied  to  vector quantization of image data to be sent via a noisy binary symmetric  channel.  The algorithm's performance is compared to those of LBG and  STVQ.  While it is  naturally superior to LBG, which does  not take into  account channel noise, its results compare very  well  to those of STVQ,  which is computationally much more demanding.
How to Dynamically Merge Markov Decision Processes
Authors: David Cohn Satinder P. Singh
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We are frequently called upon to perform multiple tasks that com(cid:173) pete  for  our  attention  and  resource.  Often  we  know  the optimal  solution  to each  task in  isolation;  in  this  paper,  we  describe  how  this  knowledge  can  be exploited  to  efficiently  find  good  solutions  for doing the tasks in parallel.  We formulate this problem as that of  dynamically merging  multiple  Markov  decision  processes  (MDPs)  into a  composite MDP,  and present a  new  theoretically-sound dy(cid:173) namic programming algorithm for finding an optimal policy for the  composite MDP.  We  analyze various aspects of our algorithm and  illustrate its use on a  simple merging problem. 
Every day,  we  are faced  with the problem of doing mUltiple  tasks in parallel,  each  of which  competes  for  our  attention  and  resource.  If we  are  running  a  job  shop,  we  must  decide  which  machines  to allocate  to  which  jobs,  and  in  what  order,  so  that no jobs miss their deadlines.  If we  are a  mail delivery robot, we  must find  the  intended recipients of the mail while simultaneously avoiding fixed  obstacles  (such  as walls)  and mobile obstacles  (such as people),  and still manage to keep ourselves  sufficiently charged up.  Frequently we know how to perform each task in isolation; this paper considers how  we  can take the information we  have about the individual  tasks and combine it to  efficiently find an optimal solution for  doing the entire set of tasks in parallel.  More  importantly,  we  describe  a  theoretically-sound  algorithm  for  doing  this  merging  dynamically;  new tasks (such as a new job arrival at a job shop) can be assimilated  online into the solution being found for  the ongoing set of simultaneous tasks.
Refractoriness and Neural Precision
Authors: Markus Meister Michael J. Berry II
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: The relationship between a neuron's refractory period and the precision of  its response to identical stimuli was investigated. We constructed a model of  a spiking neuron that combines probabilistic firing with a refractory period.  For realistic refractoriness, the model closely reproduced both the average  firing rate and the response precision of a retinal ganglion cell. The model is  based on a "free" firing rate, which exists in the absence of refractoriness.  This function may be a better description of a spiking neuron's response  than the peri-stimulus time histogram.
Active Data Clustering
Authors: Joachim M. Buhmann Thomas Hofmann
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Active  data  clustering is a  novel technique for  clustering of proxim(cid:173) ity data which utilizes principles from sequential experiment design  in  order  to interleave data generation  and  data analysis.  The  pro(cid:173) posed  active data sampling strategy is  based on the  expected  value  of information, a concept rooting in statistical decision theory.  This  is considered to be an important step towards the analysis of large(cid:173) scale  data sets,  because  it  offers  a  way  to  overcome  the  inherent  data sparseness of proximity data.  '''Ie present applications to unsu(cid:173) pervised  texture segmentation in  computer vision and information  retrieval  in  document  databases.
Enhancing Q-Learning for Optimal Asset Allocation
Authors: Ralph Neuneier
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: This paper enhances  the Q-Iearning algorithm for optimal asset  alloca(cid:173) tion proposed in (Neuneier,  1996  [6]).  The new  formulation simplifies  the  approach by using only one value-function for  many  assets  and al(cid:173) lows  model-free  policy-iteration.  After  testing  the  new  algorithm  on  real  data,  the  possibility of risk  management  within the  framework  of  Markov  decision  problems is  analyzed.  The  proposed methods  allows  the  construction of a  multi-period portfolio management  system which  takes into account transaction costs, the risk preferences of the investor,  and several constraints on the allocation.
Nonparametric Model-Based Reinforcement Learning
Authors: Christopher G. Atkeson
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: This  paper  describes  some  of  the  interactions  of model  learning  algorithms  and  planning  algorithms  we  have  found  in  exploring  model-based reinforcement learning.  The paper focuses  on  how  lo(cid:173) cal  trajectory  optimizers can  be  used  effectively  with learned non(cid:173) parametric models.  We find  that trajectory  planners that are fully  consistent  with the learned model often  have difficulty finding  rea(cid:173) sonable  plans  in  the  early  stages  of learning.  Trajectory  planners  that  balance  obeying  the  learned  model  with  minimizing cost  (or  maximizing reward)  often  do  better,  even  if the  plan  is  not  fully  consistent  with  the  learned  model.
On Parallel versus Serial Processing: A Computational Study of Visual Search
Authors: Eytan Ruppin Eyal Cohen
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A novel neural network model of pre-attention processing in visual(cid:173) search  tasks  is  presented.  Using displays of line orientations taken  from  Wolfe's experiments  [1992], we  study the  hypothesis  that the  distinction  between  parallel  versus  serial  processes  arises  from  the  availability of global information in  the internal representations  of  the  visual  scene.  The  model  operates  in  two  phases.  First,  the  visual  displays  are  compressed  via  principal-component-analysis.  Second,  the compressed data is processed by a target detector mod(cid:173) ule in order to identify the existence of a  target in the display.  Our  main  finding  is  that  targets  in  displays  which  were  found  exper(cid:173) imentally  to  be  processed  in  parallel can  be  detected  by  the  sys(cid:173) tem,  while  targets  in  experimentally-serial  displays  cannot .  This  fundamental  difference  is  explained  via  variance  analysis  of  the  compressed representations,  providing a  numerical criterion distin(cid:173) guishing parallel from serial displays.  Our model yields a  mapping  of response-time slopes that is similar to Duncan and Humphreys's  "search  surface"  [1989],  providing  an  explicit  formulation  of their  intuitive  notion  of feature  similarity.  It presents  a  neural  realiza(cid:173) tion of the  processing that may underlie the classical metaphorical  explanations of visual search. 
On Parallel versus Serial Processing: A  Computational Study a/Visual Search
S-Map: A Network with a Simple Self-Organization Algorithm for Generative Topographic Mappings
Authors: Erkki Oja Kimmo Kiviluoto
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: The S-Map is a network with a simple learning algorithm that com(cid:173) bines  the  self-organization  capability  of the  Self-Organizing  Map  (SOM)  and the probabilistic interpretability of the Generative To(cid:173) pographic  Mapping  (GTM).  The  simulations  suggest  that  the  S(cid:173) Map algorithm has a  stronger tendency  to self-organize from  ran(cid:173) dom  initial  configuration  than  the  GTM.  The  S-Map  algorithm  can  be  further  simplified  to employ  pure  Hebbian  learning,  with(cid:173) out changing the qualitative behaviour of the network.
Bayesian Model of Surface Perception
Authors: Paul A. Viola William T. Freeman
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Image  intensity  variations  can  result  from  several  different  object  surface  effects,  including  shading from  3-dimensional  relief of the  object, or paint on the surface itself.  An essential problem in vision ,  which  people  solve  naturally,  is  to  attribute  the  proper  physical  cause,  e.g.  surface  relief or  paint,  to  an  observed  image.  We  ad(cid:173) dressed  this  problem  with  an  approach  combining  psychophysical  and  Bayesian  computational methods.  We  assessed  human performance on a set of test images, and found  that people made fairly consistent judgements of surface properties.  Our  computational  model  assigned  simple  prior  probabilities  to  different  relief or  paint  explanations for  an  image,  and  solved  for  the  most  probable  interpretation  in  a  Bayesian  framework.  The  ratings of the  test  images  by  our  algorithm compared surprisingly  well  with  the  mean  ratings of our subjects.
Training Methods for Adaptive Boosting of Neural Networks
Authors: Yoshua Bengio Holger Schwenk
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: "Boosting" is  a  general  method for  improving the  performance of any  learning algorithm that consistently generates classifiers  which  need to  perform only slightly better than random guessing.  A recently proposed  and very promising boosting algorithm is AdaBoost [5].  It has been ap(cid:173) plied with great success to several benchmark machine learning problems  using rather simple learning algorithms [4],  and decision trees [1,  2,  6].  In  this  paper we use AdaBoost to  improve the performances of neural  networks.  We compare training methods based on sampling the training  set and  weighting the cost function.  Our system  achieves  about  1.4%  error on  a  data base  of online handwritten  digits  from  more  than  200  writers.  Adaptive boosting of a multi-layer network achieved 1.5% error  on the UCI Letters and 8.1 % error on the UCI satellite data set.
An Analog VLSI Model of the Fly Elementary Motion Detector
Authors: Christof Koch Reid R. Harrison
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Flies are capable of rapidly detecting and integrating visual motion in(cid:173) formation in behaviorly-relevant ways.  The first  stage of visual motion  processing in flies is  a retinotopic array of functional units known as  el(cid:173) ementary motion detectors (EMDs).  Several decades ago, Reichardt and  colleagues developed a correlation-based model of motion detection that  described the behavior of these neural circuits.  We  have implemented a  variant of this model in a 2.0-JLm  analog CMOS VLSI process.  The re(cid:173) sult is a low-power, continuous-time analog circuit with integrated pho(cid:173) toreceptors that responds to motion in real  time.  The  responses of the  circuit to drifting sinusoidal gratings qualitatively resemble the temporal  frequency response, spatial frequency response, and direction selectivity  of motion-sensitive neurons observed in insects.  In addition to  its pos(cid:173) sible engineering applications, the circuit could potentially be used as a  building block for constructing hardware models of higher-level insect  motion integration.
Experiences with Bayesian Learning in a Real World Application
Authors: Georg Dorffner, Peter Rappelsberger, Josef Zeitlhofer Peter Sykacek
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: This  paper  reports  about  an  application  of Bayes'  inferred  neu(cid:173) ral  network  classifiers  in  the  field  of automatic sleep  staging.  The  reason  for  using Bayesian  learning for  this  task  is  two-fold.  First,  Bayesian  inference  is  known  to  embody  regularization  automati(cid:173) cally.  Second,  a  side  effect  of  Bayesian  learning  leads  to  larger  variance of network outputs in regions  without training data.  This  results  in  well  known  moderation  effects,  which  can  be  used  to  detect  outliers.  In  a  5  fold  cross-validation  experiment  the  full  Bayesian  solution  found  with  R.  Neals  hybrid  Monte  Carlo  algo(cid:173) rithm,  was  not  better than  a single maximum a-posteriori  (MAP)  solution found  with  D.J.  MacKay's  evidence  approximation.  In  a  second  experiment  we  studied  the  properties  of both  solutions  in  rejecting  classification of movement artefacts. 
Experiences with Bayesian Learning in a Real World Application
Analytical Study of the Interplay between Architecture and Predictability
Authors: Ido Kanter, David A. Kessler Avner Priel
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  study  model  feed  forward  networks  as  time  series  predictors  in  the stationary limit.  The focus  is  on  complex,  yet  non-chaotic,  behavior.  The main question we  address is whether the asymptotic  behavior is  governed  by  the architecture,  regardless  the details  of  the  weights .  We  find  hierarchies  among  classes  of  architectures  with  respect  to the attract or dimension  of the long term sequence  they are capable of generating;  larger number of hidden  units  can  generate higher dimensional attractors.  In the case of a perceptron,  we  develop  the  stationary  solution  for  general  weights,  and  show  that  the  flow  is  typically  one  dimensional.  The  relaxation  time  from  an  arbitrary  initial  condition  to  the  stationary  solution  is  found  to scale linearly with the size  of the network.  In multilayer  networks, the number of hidden units gives bounds on the number  and  dimension  of the  possible  attractors.  We  conclude that  long  term  prediction  (in  the  non-chaotic  regime)  with  such  models  is  governed by attractor dynamics related to the architecture. 
Neural networks provide an important tool as model free estimators for the solution  of problems when the real model is  unknown,  or  weakly known.  In the last decade  there has  been a growing interest in the application of such tools in the area of time  series  prediction  (see  Weigand  and Gershenfeld,  1994).  In this paper we  analyse a  typical class of architectures used in this field,  i.e.  a feed  forward network governed  by the following  dynamic rule:
A Hippocampal Model of Recognition Memory
Authors: Kenneth A. Norman, James L. McClelland Randall C. O'Reilly
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A  rich  body  of data  exists  showing that  recollection of specific  infor(cid:173) mation makes an important contribution to  recognition memory,  which  is distinct from the contribution of familiarity, and is not adequately cap(cid:173) tured by existing unitary memory models. Furthennore, neuropsycholog(cid:173) ical evidence indicates that recollection is sub served by the hippocampus.  We  present a  model,  based  largely  on  known features  of hippocampal  anatomy and physiology, that accounts for the following key character(cid:173) istics of recollection:  1) false recollection is rare (i.e., participants rarely  claim to recollect having studied nonstudied items), and 2) increasing in(cid:173) terference leads to less recollection but apparently does not compromise  the quality of recollection (i.e., the extent to which recollected infonna(cid:173) tion veridically reflects events that occurred at study).
Wavelet Models for Video Time-Series
Authors: Chuanyi Ji Sheng Ma
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: In this work, we tackle the problem of time-series modeling of video  traffic.  Different from the existing methods which model the time(cid:173) series  in the time domain, we  model the wavelet  coefficients  in the  wavelet  domain.  The strength  of the wavelet  model includes  (1)  a  unified  approach to model both the long-range and the short-range  dependence  in the video traffic simultaneously, (2)  a  computation(cid:173) ally efficient  method on  developing the model and generating high  quality video traffic,  and (3) feasibility of performance analysis us(cid:173) ing the model.
Multi-time Models for Temporally Abstract Planning
Authors: Richard S. Sutton Doina Precup
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Planning 
Doina Precup, Richard S. Sutton 
University of Massachusetts 
Amherst, MA 01003 
{dprecuplrich}@cs.umass.edu
Linear Concepts and Hidden Variables: An Empirical Study
Authors: Dan Roth Adam J. Grove
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Some learning techniques for classification tasks work indirectly, by first trying  to fit a full probabilistic model to the observed data.  Whether this is a good idea  or not depends on the robustness with respect to deviations from  the postulated  model.  We  study this question experimentally in a restricted, yet non-trivial and  interesting case:  we consider a conditionally independent attribute (CIA) model  which  postulates  a  single  binary-valued hidden variable z  on  which all other  attributes (i.e., the target and the observables) depend. In this model, finding the  most likely value of anyone variable (given known values for the others) reduces  to testing a linear function of the observed values. 
We  learn  CIA  with  two  techniques:  the  standard  EM  algorithm,  and  a  new  algorithm we develop based on covariances.  We  compare these, in a controlled  fashion, against an algorithm (a version of Winnow) that attempts to find a good  linear classifier directly.  Our conclusions help delimit the fragility  of using the  CIA model for classification: once the data departs from this model, performance  quickly degrades and drops below that of the directly-learned linear classifier.
Comparison of Human and Machine Word Recognition
Authors: Cyril Latimer, Marwan A. Jabri Markus Schenkel
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  present  a  study  which  is  concerned with  word  recognition  rates  for  heavily  degraded  documents.  We  compare  human  with  machine  read(cid:173) ing capabilities in a series of experiments, which  explores the interaction  of word/non-word recognition, word frequency  and legality of non-words  with  degradation level.  We  also study the influence  of character segmen(cid:173) tation, and compare human performance with that of our artificial neural  network model for reading.  We found that the proposed computer model  uses  word  context  as  efficiently  as  humans,  but performs  slightly  worse  on the pure character recognition task.
Characterizing Neurons in the Primary Auditory Cortex of the Awake Primate Using Reverse Correlation
Authors: Michael Merzenich R. Christopher DeCharms
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: While  the  understanding of the  functional  role  of different  classes  of neurons in the awake primary visual cortex has been extensively  studied since the time of Hubel and Wiesel (Hubel and Wiesel, 1962),  our  understanding  of the  feature  selectivity  and  functional  role  of  neurons in  the primary  auditory  cortex  is  much farther  from  com(cid:173) plete.  Moving bars have long been recognized as an optimal stimulus  for  many visual cortical neurons,  and this finding  has recently been  confirmed  and extended in detail using reverse correlation methods  (Jones and Palmer,  1987;  Reid  and Alonso,  1995; Reid  et al.,  1991;  llingach et al.,  1997).  In this study, we recorded from neurons in the  primary auditory cortex of the awake  primate,  and used  a  novel  re(cid:173) verse correlation technique to compute receptive fields  (or preferred  stimuli), encompassing both multiple frequency components and on(cid:173) going time.  These spectrotemporal receptive fields  make clear that  neurons in the primary auditory cortex, as in the primary visual cor(cid:173) tex, typically show considerable structure in their feature processing  properties, often including multiple excitatory and inhibitory regions  in their receptive fields.  These neurons can be sensitive to stimulus  edges in frequency  composition or in time,  and sensitive to stimulus  transitions  such as  changes in  frequency.  These  neurons  also  show  strong responses and selectivity to continuous frequency  modulated  stimuli analogous to visual drifting gratings.
Phase Transitions and the Perceptual Organization of Video Sequences
Authors: Yair Weiss
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Estimating  motion  in  scenes  containing  multiple  moving  objects  remains  a  difficult  problem  in  computer  vision.  A  promising  ap(cid:173) proach  to this  problem  involves  using  mixture  models,  where  the  motion of each object is a component in the mixture.  However, ex(cid:173) isting methods  typically  require specifying in advance the number  of components  in  the  mixture,  i.e.  the  number  of objects  in  the  scene. 
Here  we  show  that  the  number  of objects can  be  estimated auto(cid:173) matically in a maximum likelihood framework, given an assumption  about the level of noise in the video sequence.  We derive analytical  results  showing  the  number  of models  which  maximize  the  likeli(cid:173) hood for a given noise level in a given sequence.  We illustrate these  results on a real video sequence, showing how the phase transitions  correspond to different  perceptual organizations of the scene. 
• 
Figure  la depicts  a  scene  where motion  estimation is  difficult  for  many  computer  vision  systems.  A  semi-transparent  surface  partially  occludes  a  second  surface,  and  the  camera is  translating  horizontally.  Figure  1 b  shows  a  slice  through  the  horizontal  component  of  the  motion  generated  by  the  camera  - points  that  are  closer  to  the  camera move  faster  than  those  further  away.  In  practice,  the  local  motion information would be noisy as shown in figure lc and this imposes conflicting  demands on a  motion analysis system - reliable estimates require pooling together  many measurements while avoiding mixing together measurements derived from the  two different  surfaces. 
Phase Transitions and the Perceptual Organization of Video Sequences 
851 
. , .. . ,. "....
A Generic Approach for Identification of Event Related Brain Potentials via a Competitive Neural Network Structure
Authors: Hava T. Siegelmann, Hillel Pratt, Gideon F. Inbar Daniel H. Lange
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We  present  a  novel generic  approach to the problem of Event  Related  Potential identification and classification,  based on a competitive N eu(cid:173) ral  Net  architecture.  The network  weights  converge  to the embedded  signal  patterns,  resulting  in  the  formation  of a  matched  filter  bank.  The network performance is analyzed via a simulation study, exploring  identification  robustness  under  low  SNR conditions  and compared  to  the  expected  performance  from  an information  theoretic  perspective.  The  classifier  is  applied  to  real  event-related  potential  data  recorded  during  a  classic  odd-ball  type  paradigm;  for  the  first  time,  within(cid:173) session  variable  signal  patterns  are  automatically  identified,  dismiss(cid:173) ing  the  strong  and  limiting  requirement  of  a-priori  stimulus-related  selective  grouping of the recorded  data. 
902 
D.  H.  Lange, H.  T.  Siegelmann, H.  Pratt and G.  F.  Inbar
On the Infeasibility of Training Neural Networks with Small Squared Errors
Authors: Van H. Vu
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We demonstrate that the problem of training neural networks with  small (average)  squared error is computationally intractable.  Con(cid:173) sider a data set of M  points (Xi, Yi),  i  =  1,2, ... , M, where  Xi  are  input  vectors  from  Rd,  Yi  are  real  outputs  (Yi  E  R).  For  a  net- work 10  in some class F of neural networks,  (11M) L~l (fO(Xi)(cid:173) Yi)2)1/2 - inlfEF(l/ M) "2:f!1 (f(Xi) - YJ2)1/2  is the (avarage) rel(cid:173) ative error  occurs  when one  tries to fit  the  data set  by 10.  We will  prove for several classes F  of neural networks that achieving a  rela(cid:173) tive  error  smaller than some fixed  positive threshold  (independent  from  the size  of the data set)  is  NP-hard.
Structure Driven Image Database Retrieval
Authors: Paul A. Viola Jeremy S. De Bonet
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: A  new  algorithm  is  presented  which  approximates  the  perceived  visual  similarity  between  images.  The images  are  initially  trans(cid:173) formed  into  a  feature  space  which  captures  visual  structure,  tex(cid:173) ture  and  color  using  a  tree  of filters.  Similarity  is  the  inverse  of  the distance in this  perceptual feature  space.  Using this  algorithm  we  have constructed an image database system which can perform  example based retrieval on large image databases.  Using carefully  constructed target sets, which limit variation to only a single visual  characteristic, retrieval rates are quantitatively compared to those  of standard methods.
Bayesian Robustification for Audio Visual Fusion
Authors: Paul Mineiro Javier R. Movellan
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: We discuss the problem of catastrophic fusion in multimodal recog(cid:173) nition  systems.  This  problem  arises  in systems  that  need  to  fuse  different  channels in  non-stationary environments.  Practice shows  that when  recognition modules  within each modality are tested in  contexts inconsistent with their assumptions, their influence on the  fused  product tends to increase,  with catastrophic results.  We  ex(cid:173) plore  a  principled  solution  to  this  problem  based  upon  Bayesian  ideas  of  competitive  models  and  inference  robustification:  each  sensory channel is  provided  with  simple  white-noise context mod(cid:173) els,  and  the  perceptual  hypothesis  and  context  are  jointly  esti(cid:173) mated.  Consequently, context deviations are interpreted as changes  in white noise contamination strength, automatically adjusting the  influence of the module.  The approach is  tested on  a fixed  lexicon  automatic audiovisual speech  recognition problem  with very good  results.
A Neural Network Model of Naive Preference and Filial Imprinting in the Domestic Chick
Authors: Lucy E. Hadden
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Filial imprinting in domestic chicks is of interest in psychology, biology,  and  computational  modeling because  it  exemplifies  simple,  rapid,  in(cid:173) nately programmed learning which is biased toward learning about some  objects.  Hom et  al.  have  recently discovered a  naive visual preference  for  heads  and  necks  which develops  over the  course  of the  first  three  days of life.  The  neurological basis of this predisposition is  almost en(cid:173) tirely unknown;  that  of imprinting-related learning  is  fairly  clear.  This  project is  the  first  model  of the  predisposition consistent with  what  is  known about learning in imprinting. The model develops the predisposi(cid:173) tion appropriately, learns to "approach" a training object, and replicates  one  interaction between the  two  processes.  Future work  will  replicate  more interactions between imprinting  and the  predisposition in chicks,  and analyze why the system works.