mtybilly/Instruct2DS huggingface dataset:
One sketch for all: Theory and Application of Conditional Random Sampling
Authors: Kenneth W. Church, Trevor J. Hastie Ping Li
Published in: Advances in Neural Information Processing Systems 21  (NIPS 2008) (Main Conference Track), 2008

Abstract: Conditional Random Sampling (CRS) was originally proposed for efficiently computing pairwise ($l_2$, $l_1$) distances, in static, large-scale, and sparse data sets such as text and Web data. It was previously presented using a heuristic argument. This study extends CRS to handle dynamic or streaming data, which much better reflect the real-world situation than assuming static data. Compared with other known sketching algorithms for dimension reductions such as stable random projections, CRS exhibits a significant advantage in that it is ``one-sketch-for-all.'' In particular, we demonstrate that CRS can be applied to efficiently compute the $l_p$ distance and the Hilbertian metrics, both are popular in machine learning. Although a fully rigorous analysis of CRS is difficult, we prove that, with a simple modification, CRS is rigorous at least for an important application of computing Hamming norms. A generic estimator and an approximate variance formula are provided and tested on various applications, for computing Hamming norms, Hamming distances, and $\chi^2$ distances.
The Mondrian Process
Authors: Yee W. Teh Daniel M. Roy
Published in: Advances in Neural Information Processing Systems 21  (NIPS 2008) (Main Conference Track), 2008

Abstract: We describe a novel stochastic process that can be used to construct a multidimensional generalization of the stick-breaking process and which is related to the classic stick breaking process described by Sethuraman1994 in one dimension. We describe how the process can be applied to relational data modeling using the de Finetti representation for infinitely and partially exchangeable arrays.
Scalable Algorithms for String Kernels with Inexact Matching
Authors: Pai-hsi Huang, Vladimir Pavlovic Pavel P. Kuksa
Published in: Advances in Neural Information Processing Systems 21  (NIPS 2008) (Main Conference Track), 2008

Abstract: We present a new family of linear time algorithms based on sufficient statistics for string comparison with mismatches under the string kernels framework. Our algorithms improve theoretical complexity bounds of existing approaches while scaling well with respect to the sequence alphabet size, the number of allowed mismatches and the size of the dataset. In particular, on large alphabets with loose mismatch constraints our algorithms are several orders of magnitude faster than the existing algorithms for string comparison under the mismatch similarity measure. We evaluate our algorithms on synthetic data and real applications in music genre classification, protein remote homology detection and protein fold prediction. The scalability of the algorithms allows us to consider complex sequence transformations, modeled using longer string features and larger numbers of mismatches, leading to a state-of-the-art performance with significantly reduced running times.
Onset-based Sound Segmentation
Authors: Leslie S. Smith
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A technique for  segmenting sounds using processing based on mam(cid:173) malian  early  auditory  processing  is  presented.  The  technique  is  based on features in sound which neuron spike recording suggests  are  detected in  the  cochlear  nucleus.  The  sound  signal  is  band(cid:173) passed  and  each  signal  processed  to  enhance  onsets  and  offsets.  The onset and offset signals are compressed, then clustered both in  time and  across  frequency  channels  using  a  network  of integrate(cid:173) and-fire  neurons.  Onsets  and  offsets  are  signalled  by  spikes,  and  the timing of these spikes used to segment the sound.
Beating a Defender in Robotic Soccer: Memory-Based Learning of a Continuous Function
Authors: Manuela M. Veloso Peter Stone
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Learning how to adjust to an opponent's position is critical to  the success of having intelligent agents collaborating towards the  achievement of specific tasks in unfriendly environments. This pa(cid:173) per describes our work on a Memory-based technique for to choose  an action based on a continuous-valued state attribute indicating  the position of an opponent. We investigate the question of how an  agent performs in nondeterministic variations of the training situ(cid:173) ations. Our experiments indicate that when the random variations  fall within some bound of the initial training, the agent performs  better with some initial training rather than from a tabula-rasa.
A Neural Network Autoassociator for Induction Motor Failure Prediction
Authors: Angelo Marcantonio, Christian Darken, Stephen Jose Hanson, Gary M. Kuhn, N. Iwan Santoso Thomas Petsche
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  present results  on  the  use  of neural  network based  autoassociators  which  act  as  novelty  or anomaly  detectors  to  detect  imminent  motor  failures.  The autoassociator is  trained  to  reconstruct  spectra obtained  from the healthy motor.  In laboratory tests, we have demonstrated that the  trained autoassociator has a small reconstruction error on measurements  recorded from healthy motors but a larger error on those recorded from a  motor with a fault.  We have designed and built a motor monitoring system  using an  autoassociator for anomaly detection and are in the process of  testing the system at three industrial and commercial sites.
The Gamma MLP for Speech Phoneme Recognition
Authors: Ah Chung Tsoi, Andrew D. Back Steve Lawrence
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  define  a  Gamma  multi-layer  perceptron  (MLP)  as  an  MLP  with the usual synaptic weights replaced by gamma filters  (as pro(cid:173) posed by de Vries and Principe  (de Vries and Principe,  1992)) and  associated  gain  terms  throughout  all  layers.  We  derive  gradient  descent  update equations and apply  the  model  to  the recognition  of speech  phonemes.  We  find  that  both  the  inclusion  of gamma  filters  in  all  layers,  and  the  inclusion  of synaptic  gains,  improves  the  performance  of the  Gamma  MLP.  We  compare  the  Gamma  MLP  with  TDNN,  Back-Tsoi FIR MLP,  and Back-Tsoi I1R  MLP  architectures, and a  local approximation scheme.  We find  that the  Gamma MLP results in an substantial reduction  in error rates.
Laterally Interconnected Self-Organizing Maps in Hand-Written Digit Recognition
Authors: Joseph Sirosh, Risto Miikkulainen Yoonsuck Choe
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: An  application  of  laterally  interconnected  self-organizing  maps  (LISSOM)  to handwritten  digit recognition  is  presented.  The lat(cid:173) eral connections  learn the correlations of activity between units on  the  map.  The  resulting  excitatory  connections  focus  the  activity  into local patches and the inhibitory connections decorrelate redun(cid:173) dant activity on the map.  The map thus forms internal representa(cid:173) tions that are easy to recognize with e.g.  a perceptron network.  The  recognition rate on a subset of NIST database 3 is 4.0% higher with  LISSOM  than  with  a  regular  Self-Organizing  Map  (SOM)  as  the  front  end,  and 15.8% higher than recognition of raw input bitmaps  directly.  These results form a promising starting point for  building  pattern  recognition systems with a  LISSOM  map as  a  front  end.
Improved Silicon Cochlea using Compatible Lateral Bipolar Transistors
Authors: Eric Fragnière, Eric A. Vittoz André van Schaik
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Analog  electronic  cochlear  models  need  exponentially  scaled  filters.  CMOS  Compatible  Lateral  Bipolar  Transistors  (CLBTs)  can  create  exponentially scaled currents when biased using a resistive line with a  voltage  difference  between both  ends  of the  line.  Since these  CLBTs  are  independent  of  the  CMOS  threshold  voltage,  current  sources  implemented  with  CLBTs  are  much  better  matched  than  current  sources  created  with  MOS  transistors  operated  in  weak  inversion.  Measurements  from  integrated  test  chips  are  shown  to  verify  the  improved matching.
On the Computational Power of Noisy Spiking Neurons
Authors: Wolfgang Maass
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: It  has  remained  unknown  whether  one  can  in  principle  carry out  reliable digital  computations with networks of biologically realistic  models  for  neurons.  This  article  presents  rigorous  constructions  for  simulating in  real-time arbitrary given  boolean circuits  and fi(cid:173) nite automata with arbitrarily high reliability by networks of noisy  spiking neurons.  In  addition  we  show  that  with  the  help  of  "shunting  inhibition"  even  networks  of very  unreliable  spiking  neurons  can  simulate  in  real-time  any  McCulloch-Pitts  neuron  (or  "threshold  gate"),  and  therefore  any  multilayer  perceptron  (or  "threshold  circuit")  in  a  reliable  manner.  These  constructions  provide  a  possible  explana(cid:173) tion for  the fact  that biological  neural systems can carry out quite  complex  computations within  100  msec.  It  turns  out that the assumption  that these  constructions require  about the shape of the  EPSP's and  the behaviour of the  noise  are  surprisingly weak.
Parallel Optimization of Motion Controllers via Policy Iteration
Authors: R. Sitaraman, Roderic A. Grupen Jefferson A. Coelho Jr.
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: This paper describes a  policy iteration algorithm for optimizing the  performance  of a  harmonic function-based  controller  with  respect  to a  user-defined  index.  Value functions  are represented  as poten(cid:173) tial  distributions  over  the  problem  domain,  being  control  policies  represented  as  gradient  fields  over  the  same domain.  All  interme(cid:173) diate  policies  are intrinsically  safe,  i.e.  collisions  are not  promoted  during  the adaptation process.  The algorithm  has  efficient  imple(cid:173) mentation in parallel  SIMD  architectures.  One  potential  applica(cid:173) tion - travel distance  minimization - illustrates  its usefulness.
Model Matching and SFMD Computation
Authors: Dan W. Hammerstrom Steven Rehfuss
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In  systems  that  process sensory  data there is  frequently  a  model  matching stage where class hypotheses are combined to recognize a  complex entity.  We introduce a new model of parallelism, the Single  Function  Multiple  Data  (SFMD) model, appropriate to this stage.  SFMD functionality can be added with small hardware expense to  certain existing SIMD architectures, and as an incremental addition  to the programming model.  Adding  SFMD  to an SIMD  machine  will  not  only  allow  faster  model  matching,  but  also  increase  its  flexibility as a  general purpose machine and its scope in performing  the initial stages of sensory processing.
Learning with ensembles: How overfitting can be useful
Authors: Anders Krogh Peter Sollich
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  study  the  characteristics  of learning  with ensembles.  Solving  exactly  the  simple  model  of an  ensemble  of linear  students,  we  find  surprisingly  rich  behaviour.  For  learning  in  large  ensembles,  it  is  advantageous  to  use  under-regularized  students,  which  actu(cid:173) ally  over-fit  the  training data.  Globally optimal performance  can  be  obtained by  choosing  the training set  sizes  of the students ap(cid:173) propriately.  For  smaller ensembles,  optimization of the  ensemble  weights  can yield significant improvements in ensemble generaliza(cid:173) tion  performance,  in particular if the  individual students  are  sub(cid:173) ject to noise in the training process.  Choosing students with a wide  range of regularization parameters makes this improvement robust  against changes in the unknown level of noise  in the training data.
Analog VLSI Processor Implementing the Continuous Wavelet Transform
Authors: Gert Cauwenberghs R. Timothy Edwards
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We present an integrated analog processor for real-time wavelet decom(cid:173) position and reconstruction of continuous temporal signals covering the  audio frequency range. The processor performs complex harmonic modu(cid:173) lation and Gaussian lowpass filtering in 16 parallel channels, each clocked  at a different rate, producing a multiresolution mapping on a logarithmic  frequency scale. Our implementation uses mixed-mode analog and dig(cid:173) ital circuits, oversampling techniques, and switched-capacitor filters to  achieve a wide linear dynamic range while maintaining compact circuit  size and low power consumption. We include experimental results on the  processor and characterize its components separately from measurements  on a single-channel test chip.
A Novel Channel Selection System in Cochlear Implants Using Artificial Neural Network
Authors: Raymond J. Wang Marwan A. Jabri
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: State-of-the-art  speech  processors  in  cochlear  implants  perform  channel selection  using  a spectral maxima strategy.  This strategy  can  lead  to  confusions  when  high  frequency  features  are  needed  to discriminate between  sounds.  We  present  in this paper  a  novel  channel selection strategy based upon pattern recognition which al(cid:173) lows  "smart" channel selections to be made.  The proposed strategy  is  implemented using  multi-layer  perceptrons  trained  on  a  multi(cid:173) speaker labelled speech database.  The input to the network are the  energy coefficients of N  energy channels.  The output of the system  are  the indices  of the M  selected  channels.  We  compare  the  performance  of our  proposed  system  to  that  of  spectral maxima strategy, and show that our strategy can produce  significantly better results.
Rapid Quality Estimation of Neural Network Input Representations
Authors: Jude W. Shavlik Kevin J. Cherkauer
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: The choice of an input representation for a neural network can have  a  profound  impact  on  its  accuracy  in  classifying  novel  instances.  However,  neural networks are typically  computationally expensive  to  train,  making  it  difficult  to  test  large  numbers  of alternative  representations.  This  paper  introduces  fast  quality  measures  for  neural  network  representations,  allowing  one  to  quickly  and  ac(cid:173) curately estimate which  of a  collection  of possible  representations  for  a  problem is  the best.  We show  that our measures for  ranking  representations are more accurate than a previously published mea(cid:173) sure,  based on experiments with  three difficult,  real-world pattern  recognition  problems.
Generating Accurate and Diverse Members of a Neural-Network Ensemble
Authors: Jude W. Shavlik David W. Opitz
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Neural-network  ensembles  have  been  shown  to  be  very  accurate  classification  techniques.  Previous  work  has  shown  that  an effec(cid:173) tive ensemble  should  consist  of networks  that are not  only  highly  correct,  but  ones  that  make  their  errors on  different  parts  of the  input  space  as  well.  Most  existing  techniques,  however,  only  in(cid:173) directly  address  the  problem  of  creating  such  a  set  of  networks.  In  this  paper  we  present  a  technique  called  ADDEMUP  that  uses  genetic  algorithms  to  directly  search  for  an  accurate  and  diverse  set  of trained  networks.  ADDEMUP  works  by  first  creating an ini(cid:173) tial  population,  then  uses  genetic  operators  to  continually  create  new  networks,  keeping the set of networks that are as  accurate as  possible while disagreeing with each other as much as possible.  Ex(cid:173) periments on three DNA problems show that  ADDEMUP  is able to  generate a  set of trained  networks that is  more accurate than sev(cid:173) eral existing  approaches.  Experiments  also  show  that  ADDEMUP  is  able  to  effectively  incorporate  prior  knowledge,  if  available,  to  improve the quality of its ensemble.
A Model of Spatial Representations in Parietal Cortex Explains Hemineglect
Authors: Terrence J. Sejnowski Alexandre Pouget
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We have recently developed a theory of spatial representations in  which the position of an object is not encoded in a particular frame  of reference but, instead, involves neurons computing basis func(cid:173) tions of their sensory inputs. This type of representation is able  to perform nonlinear sensorimotor transformations and is consis(cid:173) tent with the response properties of parietal neurons. We now ask  whether the same theory could account for the behavior of human  patients with parietal lesions. These lesions induce a deficit known  as hemineglect that is characterized by a lack of reaction to stimuli  located in the hemispace contralateral to the lesion. A simulated  lesion in a basis function representation was found to replicate three  of the most important aspects of hemineglect: i) The models failed  to cross the leftmost lines in line cancellation experiments, ii) the  deficit affected multiple frames of reference and, iii) it could be  object centered. These results strongly support the basis function  hypothesis for spatial representations and provide a computational  theory of hemineglect at the single cell level.
Temporal Difference Learning in Continuous Time and Space
Authors: Kenji Doya
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A continuous-time, continuous-state version of the temporal differ(cid:173) ence (TD) algorithm is derived in order to facilitate the application  of reinforcement  learning to real-world  control tasks and  neurobi(cid:173) ological modeling.  An optimal nonlinear feedback control law  was  also  derived  using the derivatives  of the value  function.  The  per(cid:173) formance  of the algorithms  was  tested in a  task of swinging  up  a  pendulum with limited torque.  Both the  "critic"  that specifies the  paths to the upright position and the "actor"  that works as a non(cid:173) linear feedback  controller  were  successfully implemented by radial  basis function  (RBF)  networks.
Tempering Backpropagation Networks: Not All Weights are Created Equal
Authors: Terrence J. Sejnowski Nicol N. Schraudolph
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Terrence J. Sejnowski 
Computational Neurobiology Lab  The Salk Institute for BioI. Studies  San Diego, CA 92186-5800, USA
Improving Policies without Measuring Merits
Authors: Satinder P. Singh Peter Dayan
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Performing policy  iteration  in  dynamic  programming should  only  require knowledge of relative rather than absolute measures of the  utility of actions (Werbos,  1991) - what Baird  (1993)  calls the  ad(cid:173) vantages  of actions at states.  Nevertheless, most existing methods  in dynamic programming (including Baird's) compute some form of  absolute utility function .  For smooth problems,  advantages satisfy  two  differential  consistency  conditions  (including  the  requirement  that they be free of curl), and we show that enforcing these can lead  to appropriate policy improvement solely in terms of advantages. 
1
Selective Attention for Handwritten Digit Recognition
Authors: Ethem Alpaydin
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Completely  parallel object  recognition  is  NP-complete.  Achieving  a  recognizer  with  feasible  complexity  requires  a  compromise  be(cid:173) tween parallel and sequential processing  where a  system selectively  focuses  on  parts  of a  given  image,  one  after  another.  Successive  fixations  are  generated  to sample the  image and these  samples are  processed  and abstracted  to  generate  a  temporal context  in which  results are integrated over time.  A computational model based on a  partially recurrent feedforward network is proposed and made cred(cid:173) ible  by  testing  on  the  real-world  problem  of recognition  of hand(cid:173) written  digits with encouraging results.
Learning Model Bias
Authors: Jonathan Baxter
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In  this  paper  the  problem  of learning appropriate  domain-specific  bias is  addressed.  It is shown that this can be achieved by learning  many related  tasks from  the  same domain,  and a  theorem  is  given  bounding the number  tasks that must be learnt.  A  corollary of the  theorem  is  that if the  tasks  are  known to  possess  a  common  inter(cid:173) nal  representation  or  preprocessing  then  the  number  of examples  required  per  task for  good generalisation  when learning n  tasks  si(cid:173) multaneously  scales  like  O(a + ~),  where  O(a)  is  a  bound  on  the  minimum  number  of examples  requred  to  learn  a  single  task,  and  O( a + b)  is  a  bound  on the  number  of examples  required  to  learn  each  task independently.  An  experiment  providing  strong qualita(cid:173) tive  support for  the  theoretical  results  is  reported.
Estimating the Bayes Risk from Sample Data
Authors: Tong Xu Robert R. Snapp
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A new nearest-neighbor method is described for estimating the Bayes risk  of a multiclass pattern claSSification  problem  from  sample data  (e.g.,  a  classified training set). Although it is assumed that the classification prob(cid:173) lem can be accurately described by sufficiently smooth class-conditional  distributions, neither these distributions, nor the corresponding prior prob(cid:173) abilities of the classes are required.  Thus  this  method can  be applied to  practical problems where the underlying probabilities are not known.  This  method is illustrated using two different pattern recognition problems.
A Model of Auditory Streaming
Authors: Michael J. Denham Susan L. McCabe
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: An essential feature of intelligent sensory processing is the ability to  focus on the part of the signal of interest against a background of  distracting signals, and to be able to direct this focus at will. In this  paper the problem of auditory scene segmentation is considered and a  model of the early stages of the process is proposed. The behaviour of  the model is shown to be in agreement with a number of well known  psychophysical results. The principal contribution of this model lies in  demonstrating how streaming might result from interactions between  the tonotopic patterns of activity of input signals and traces of previous  activity which feedback and influence the way in which subsequent  signals are processed.
Context-Dependent Classes in a Hybrid Recurrent Network-HMM Speech Recognition System
Authors: Anthony J. Robinson, Mike Hochberg Dan J. Kershaw
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A  method  for  incorporating  context-dependent  phone  classes  in  a  connectionist-HMM  hybrid  speech  recognition  system  is  intro(cid:173) duced.  A modular approach is adopted, where single-layer networks  discriminate between different context classes given the phone class  and  the  acoustic data.  The context networks  are combined with a  context-independent  (CI)  network  to  generate  context-dependent  (CD)  phone  probability estimates.  Experiments show  an  average  reduction  in  word  error  rate  of 16%  and  13%  from  the  CI  system  on  ARPA 5,000  word  and  SQALE  20,000  word  tasks respectively.  Due  to improved modelling, the decoding speed  of the  CD system  is  more than twice  as fast  as  the CI system.
Exploiting Tractable Substructures in Intractable Networks
Authors: Michael I. Jordan Lawrence K. Saul
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  develop  a  refined  mean field  approximation for  inference  and  learning  in  probabilistic neural  networks.  Our  mean field  theory,  unlike most, does not assume that the units behave as independent  degrees  of freedom;  instead,  it  exploits  in  a  principled  way  the  existence of large substructures that are computationally tractable.  To  illustrate  the  advantages  of this  framework,  we  show  how  to  incorporate weak higher order interactions into a first-order  hidden  Markov  model,  treating  the  corrections  (but  not  the  first  order  structure)  within mean field  theory.
Statistical Theory of Overtraining - Is Cross-Validation Asymptotically Effective?
Authors: Noboru Murata, Klaus-Robert Müller, Michael Finke, Howard Hua Yang Shun-ichi Amari
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A  statistical  theory  for  overtraining  is  proposed.  The  analysis  treats realizable stochastic neural networks, trained with Kullback(cid:173) Leibler loss in the  asymptotic case.  It is shown that the asymptotic  gain  in  the  generalization error  is  small if we  perform early  stop(cid:173) ping, even if we have access to the optimal stopping time.  Consider(cid:173) ing cross-validation stopping we answer the question:  In what ratio  the examples should be divided into training and testing sets in or(cid:173) der  to  obtain  the  optimum  performance.  In  the  non-asymptotic  region  cross-validated early stopping always  decreases  the general(cid:173) ization  error.  Our  large  scale  simulations  done  on  a  CM5  are  in  nice  agreement with our analytical findings.
Primitive Manipulation Learning with Connectionism
Authors: Yoky Matsuoka
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Infants' manipulative exploratory behavior within the environment  is a vehicle of cognitive stimulation[McCall 1974].  During this time,  infants practice and perfect sensorimotor patterns that become be(cid:173) havioral modules which will be seriated and imbedded in more com(cid:173) plex actions.  This paper explores the development of such primitive  learning systems  using  an  embodied  light-weight  hand  which  will  be  used for  a  humanoid being  developed  at the  MIT Artificial  In(cid:173) telligence  Laboratory[Brooks and Stein  1993].  Primitive grasping  procedures  are  learned  from  sensory  inputs  using  a  connectionist  reinforcement  algorithm while two submodules preprocess  sensory  data to  recognize  the  hardness  of objects  and  detect  shear  using  competitive  learning  and  back-propagation  algorithm  strategies,  respectively.  This  system  is  not  only  consistent  and  quick  dur(cid:173) ing the initial learning stage,  but also adaptable to new  situations  after  training is completed.
Silicon Models for Auditory Scene Analysis
Authors: John Wawrzynek John Lazzaro
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  are  developing  special-purpose,  low-power  analog-to-digital  converters  for  speech  and  music  applications,  that  feature  analog  circuit  models  of biological  audition  to  process  the  audio  signal  before  conversion.  This  paper describes  our most recent  converter  design,  and a working system that uses several copies ofthe chip to  compute  multiple representations  of sound  from  an  analog input.  This  multi-representation  system  demonstrates  the  plausibility of  inexpensively implementing an auditory scene  analysis approach to  sound  processing.
Human Face Detection in Visual Scenes
Authors: Shumeet Baluja, Takeo Kanade Henry A. Rowley
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  present  a  neural  network-based face  detection  system.  A retinally  connected  neural  network  examines  small  windows  of an  image,  and  decides  whether each  window  contains  a  face.  The  system  arbitrates  between multiple networks to improve performance over a single network.  We  use  a  bootstrap  algorithm for training, which  adds  false  detections  into the training set as  training progresses.  This eliminates the difficult  task  of manually  selecting  non-face  training examples,  which  must  be  chosen  to span the entire space of non-face images.  Comparisons with  another state-of-the-art face  detection system are presented;  our system  has better performance in terms of detection and false-positive rates.
SPERT-II: A Vector Microprocessor System and its Application to Large Problems in Backpropagation Training
Authors: Krste Asanovic, Brian Kingsbury, James Beck, David Johnson, Nelson Morgan John Wawrzynek
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We  report  on  our  development  of a  high-performance  system  for  neural  network  and other signal  processing  applications.  We  have  designed  and  implemented  a  vector  microprocessor  and  pack(cid:173) aged  it  as  an  attached  processor  for  a  conventional  workstation.  We  present  performance  comparisons  with  commercial  worksta(cid:173) tions on neural  network  backpropagation training.  The SPERT-II  system  demonstrates  significant  speedups  over  extensively  hand(cid:173) optimization code running on  the workstations.
Stochastic Hillclimbing as a Baseline Method for Evaluating Genetic Algorithms
Authors: Martin Wattenberg Ari Juels
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We investigate the effectiveness of stochastic hillclimbing as a baseline for  evaluating the  performance  of genetic  algorithms  (GAs)  as combinato(cid:173) rial function optimizers.  In particular, we address two problems to which  GAs have been applied in the literature:  Koza's ll-multiplexer problem  and the jobshop  problem.  We  demonstrate that simple  stochastic  hill(cid:173) climbing methods are  able  to achieve results  comparable or superior  to  those obtained by the GAs designed to address these two problems.  We  further  illustrate,  in the case  of the jobshop  problem,  how  insights  ob(cid:173) tained in the formulation of a stochastic hillclimbing algorithm can lead  to improvements in the encoding used by a  GA.
Using the Future to "Sort Out" the Present: Rankprop and Multitask Learning for Medical Risk Evaluation
Authors: Shumeet Baluja, Tom Mitchell Rich Caruana
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A patient visits the doctor; the doctor reviews  the patient's history,  asks questions, makes basic measurements (blood pressure,  .. . ), and  prescribes  tests  or  treatment .  The  prescribed  course  of action  is  based  on  an assessment of patient risk-patients at higher risk  are  given  more  and  faster  attention.  It is  also  sequential- it  is  too  expensive  to  immediately order  all  tests  which  might  later  be  of  value.  This  paper  presents  two  methods  that  together  improve  the  accuracy  of  backprop  nets  on  a  pneumonia  risk  assessment  problem  by  10-50%.  Rankprop improves on backpropagation  with  sum of squares error in ranking patients by risk.  Multitask learning  takes advantage of future lab tests  available in the training set,  but  not  available  in  practice  when  predictions  must  be  made.  Both  methods are broadly  applicable.
Exponentially many local minima for single neurons
Authors: Mark Herbster, Manfred K. Warmuth Peter Auer
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We show that for a single neuron with the logistic function as the transfer  function the number of local  minima of the error function based on the  square loss can grow exponentially in the dimension.
An Information-theoretic Learning Algorithm for Neural Network Classification
Authors: Ajit V. Rao, Kenneth Rose, Allen Gersho David J. Miller
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: A  new  learning  algorithm is  developed for  the design  of statistical  classifiers  minimizing  the  rate  of misclassification.  The  method,  which  is  based on  ideas  from  information  theory  and  analogies  to  statistical  physics,  assigns  data to  classes  in  probability.  The  dis(cid:173) tributions  are  chosen  to minimize the  expected  classification  error  while  simultaneously enforcing  the classifier's structure and a level  of "randomness"  measured  by  Shannon's entropy.  Achievement of  the classifier structure is  quantified by an  associated cost.  The con(cid:173) strained optimization  problem is  equivalent to the minimization of  a  Helmholtz  free  energy,  and  the  resulting  optimization  method  is  a  basic  extension  of the  deterministic  annealing  algorithm  that  explicitly  enforces  structural  constraints on  assignments  while  re(cid:173) ducing  the  entropy  and  expected  cost  with  temperature.  In  the  limit of low temperature, the error rate is minimized directly and a  hard classifier with the requisite structure is  obtained.  This learn(cid:173) ing algorithm can be used to design a variety of classifier structures.  The approach is  compared with standard methods for  radial basis  function  design  and  is  demonstrated  to  substantially  outperform  other  design  methods  on  several  benchmark  examples,  while  of(cid:173) ten retaining design complexity comparable to,  or only moderately  greater than that of strict descent-based methods. 
592 
D.  NnLLER.A.RAO.K. ROSE.A. GERSHO
Improving Elevator Performance Using Reinforcement Learning
Authors: Andrew G. Barto Robert H. Crites
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: This paper describes the application of reinforcement learning (RL)  to the difficult  real  world problem of elevator dispatching.  The el(cid:173) evator domain poses a  combination of challenges not seen in  most  RL  research to date.  Elevator systems operate in continuous state  spaces  and in continuous time  as discrete  event  dynamic systems.  Their  states  are  not  fully  observable  and  they  are  nonstationary  due to changing passenger arrival rates.  In addition, we  use a team  of RL  agents,  each  of which is  responsible  for  controlling one  ele(cid:173) vator car.  The team receives  a  global reinforcement  signal  which  appears noisy to each agent due to the effects of the actions of the  other agents, the random nature of the arrivals and the incomplete  observation of the state.  In  spite  of these complications,  we  show  results that in simulation surpass the best of the heuristic elevator  control  algorithms  of which  we  are  aware.  These  results  demon(cid:173) strate  the  power  of RL  on  a  very  large  scale  stochastic  dynamic  optimization problem of practical utility.
Optimal Asset Allocation using Adaptive Dynamic Programming
Authors: Ralph Neuneier
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In  recent  years,  the  interest  of investors  has  shifted  to  computer(cid:173) ized asset allocation (portfolio management) to exploit the growing  dynamics of the capital markets.  In  this paper,  asset  allocation is  formalized  as  a  Markovian  Decision  Problem  which  can  be  opti(cid:173) mized  by applying dynamic programming or reinforcement learning  based  algorithms.  Using an artificial exchange rate,  the asset  allo(cid:173) cation strategy optimized with reinforcement learning (Q-Learning)  is  shown  to  be  equivalent  to  a  policy  computed  by  dynamic  pro(cid:173) gramming.  The approach is  then tested on the task to invest liquid  capital  in  the  German  stock  market.  Here,  neural  networks  are  used  as  value function  approximators.  The  resulting  asset  alloca(cid:173) tion  strategy  is  superior  to  a  heuristic  benchmark  policy.  This  is  a  further  example  which  demonstrates  the  applicability  of neural  network  based  reinforcement  learning  to  a  problem setting with  a  high  dimensional state space.
Recursive Estimation of Dynamic Modular RBF Networks
Authors: Maha Kadirkamanathan Visakan Kadirkamanathan
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: In this paper, recursive estimation algorithms for dynamic modular  networks  are  developed.  The  models  are  based  on Gaussian  RBF  networks  and  the  gating network  is  considered  in two  stages:  At  first,  it  is  simply  a  time-varying  scalar  and  in  the  second,  it  is  based  on the state, as in the mixture of local experts scheme.  The  resulting  algorithm  uses  Kalman  filter  estimation  for  the  model  estimation and the gating probability estimation.  Both, 'hard' and  'soft' competition based estimation schemes are developed where in  the former, the most probable network is  adapted and in the latter  all networks  are  adapted by  appropriate weighting of the data.
A Neural Network Classifier for the I100 OCR Chip
Authors: Timothy P. Allen John C. Platt
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: This paper describes a neural network classifier for  the 11000  chip, which  optically  reads  the  E13B  font  characters  at  the  bottom of checks.  The  first  layer  of  the  neural  network  is  a  hardware  linear  classifier  which  recognizes  the  characters  in  this  font .  A  second  software  neural  layer  is  implemented  on  an  inexpensive  microprocessor  to  clean  up  the  re(cid:173) sults of the first  layer.  The hardware linear  classifier  is  mathematically  specified  using  constraints  and  an  optimization  principle.  The  weights  of the  classifier  are  found  using  the  active  set  method,  similar  to  Vap(cid:173) nik's separating hyperplane algorithm.  In 7.5 minutes ofSPARC 2 time,  the  method solves for  1523  Lagrange mUltipliers,  which  is  equivalent  to  training  on  a  data set  of approximately  128,000  examples.  The  result(cid:173) ing network  performs quite  well:  when  tested  on  a  test set  of 1500  real  checks,  it has  a  99.995%  character accuracy  rate. 
1  A  BRIEF OVERVIEW OF THE 11000  CHIP 
At Synaptics, we have created the 11000,  an analog VLSI chip that, when combined  with associated software, optically  reads  the E13B font  from  the bottom of checks.  This  E13B  font  is  shown  in  figure  1.  The  overall  architecture  of the  11000  chip  is  shown  in  figure  2.  The 11000  recognizes  checks  hand-swiped  through  a  slot.  A  lens  focuses  the image  of the bottom of the  check  onto the  retina.  The  retina has  circuitry  which  locates  the  vertical  position  of the  characters  on  the  check .  The  retina then  sends  an  image  vertically  centered  around  a  possible  character  to  the  classifier.  The classifier  in  the nooo  has  a  tough job.  It  must  be  very accurate  and immune  to  noise  and ink scribbles  in  the input .  Therefore , we  decided  to use  an  integrated  segmentation  and  recognition  approach  (Martin  &  Pittman,  1992)(Platt,  et  al.,  1992).  When the classifier produces a  strong response,  we  know  that a  character is  horizontally  centered in  the retina. 
A Neural Network Classifier for the 11000 OCR Chip 
939 
Figure  1: The  E13B font,  as  seen  by  the 11000  chip
Modeling Saccadic Targeting in Visual Search
Authors: Gregory J. Zelinsky, Mary M. Hayhoe, Dana H. Ballard Rajesh P. N. Rao
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Visual cognition depends criticalIy on the ability to make rapid eye movements  known as saccades that orient the fovea over targets of interest in a visual  scene. Saccades are known to be ballistic: the pattern of muscle activation  for foveating a prespecified target location is computed prior to the movement  and visual feedback is precluded. Despite these distinctive properties, there  has been no general model of the saccadic targeting strategy employed by  the human visual system during visual search in natural scenes. This paper  proposes a model for saccadic targeting that uses iconic scene representations  derived from oriented spatial filters at multiple scales. Visual search proceeds  in a coarse-to-fine fashion with the largest scale filter responses being compared  first. The model was empirically tested by comparing its perfonnance with  actual eye movement data from human subjects in a natural visual search task;  preliminary results indicate substantial agreement between eye movements  predicted by the model and those recorded from human subjects.
Plasticity of Center-Surround Opponent Receptive Fields in Real and Artificial Neural Systems of Vision
Authors: T. Furukawa, M. Yamada, T. Saito S. Yasui
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Despite  the  phylogenic  and  structural  differences,  the  visual  sys(cid:173) tems of different species, whether vertebrate or invertebrate, share  certain functional properties.  The center-surround opponent recep(cid:173) tive field  (CSRF)  mechanism represents  one  such example.  Here,  analogous  CSRFs  are  shown  to  be  formed  in  an  artificial  neural  network which learns to localize contours (edges)  of the luminance  difference.  Furthermore,  when  the  input  pattern  is  corrupted  by  a  background noise,  the CSRFs of the hidden  units  becomes shal(cid:173) lower and broader with decrease of the signal-to-noise ratio (SNR).  The same kind of SNR-dependent plasticity is present in the CSRF  of real visual neurons; in bipolar cells of the carp retina as is shown  here  experimentally,  as  well  as  in  large  monopolar  cells  of the  fly  compound eye  as  was  described  by others.  Also,  analogous  SNR(cid:173) dependent  plasticity  is  shown  to  be  present  in  the  biphasic  flash  responses  (BPFR) of these  artificial  and  biological visual systems .  Thus,  the spatial  (CSRF)  and  temporal  (BPFR) filtering  proper(cid:173) ties with  which a  wide variety of creatures see the world appear to  be optimized for  detectability of changes in space  and time.
Active Gesture Recognition using Learned Visual Attention
Authors: Alex Pentland Trevor Darrell
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We have developed a foveated  gesture recognition system that runs  in an unconstrained office environment with an active camera.  Us(cid:173) ing vision routines previously implemented for  an interactive envi(cid:173) ronment,  we  determine  the  spatial  location  of salient  body  parts  of a  user  and guide  an active  camera to obtain images of gestures  or expressions.  A hidden-state reinforcement  learning paradigm is  used  to implement visual  attention.  The  attention  module selects  targets  to foveate  based on the goal of successful  recognition,  and  uses  a  new  multiple-model  Q-Iearning  formulation.  Given  a  set  of target  and  distractor  gestures,  our  system  can  learn  where  to  foveate  to maximally discriminate a  particular gesture.
On Neural Networks with Minimal Weights
Authors: Jehoshua Bruck Vasken Bohossian
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: Linear threshold elements are the basic building blocks of artificial  neural  networks.  A  linear  threshold element computes  a  function  that is  a sign of a weighted sum of the input variables.  The weights  are  arbitrary  integers;  actually,  they  can  be  very  big  integers-(cid:173) exponential  in  the  number  of  the  input  variables.  However,  in  practice,  it  is  difficult  to  implement  big  weights.  In  the  present  literature  a  distinction  is  made  between  the  two  extreme  cases:  linear threshold functions  with polynomial-size weights as opposed  to those  with exponential-size weights.  The  main  contribution  of  this paper is  to fill  up  the gap by further  refining  that separation.  Namely,  we  prove that the class of linear threshold functions  with  polynomial-size  weights  can  be  divided  into  subclasses  according  to the degree of the polynomial.  In fact,  we  prove  a  more general  that there exists a minimal weight linear threshold function  result- for  any arbitrary number of inputs and any  weight  size.  To prove  those results we  have developed a  novel technique for  constructing  linear threshold functions  with minimal weights.
Neural Networks with Quadratic VC Dimension
Authors: Eduardo D. Sontag Pascal Koiran
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: This paper shows that neural  networks  which  use  continuous acti(cid:173) vation functions have VC  dimension at least as large as the square  of  the  number  of weights  w.  This  result  settles  a  long-standing  open  question,  namely whether  the well-known  O( w log w)  bound,  known for hard-threshold nets, also held for  more general sigmoidal  nets.  Implications for  the number of samples needed for  valid gen(cid:173) eralization are  discussed.
Factorial Hidden Markov Models
Authors: Michael I. Jordan Zoubin Ghahramani
Published in: Advances in Neural Information Processing Systems 8  (NIPS 1995) (Main Conference Track), 1995

Abstract: We present a framework for learning in hidden Markov models with  distributed  state  representations.  Within  this framework ,  we  de(cid:173) rive  a  learning algorithm based  on  the  Expectation-Maximization  (EM)  procedure for  maximum likelihood estimation.  Analogous to  the  standard  Baum-Welch  update  rules,  the  M-step  of our  algo(cid:173) rithm is  exact and can be solved analytically.  However,  due  to the  combinatorial nature of the  hidden state representation,  the exact  E-step is intractable.  A simple and tractable mean field approxima(cid:173) tion is  derived.  Empirical results on a set of problems suggest that  both  the mean field  approximation and Gibbs sampling are viable  alternatives to  the computationally expensive exact algorithm.