mtybilly/Instruct2DS huggingface dataset:
Segmental Neural Net Optimization for Continuous Speech Recognition
Authors: Richard Schwartz, John Makhoul, George Zavaliagkos Ying Zhao
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Previously, we had developed the concept of a Segmental Neural Net (SNN) for  phonetic modeling in continuous speech recognition (CSR). This kind of neu(cid:173) ral network technology advanced the state-of-the-art of large-vocabulary CSR,  which employs Hidden Marlcov Models (HMM), for the ARPA 1oo0-word Re(cid:173) source Management corpus. More Recently, we started porting the neural net  system to a larger, more challenging corpus - the ARPA 20,Ooo-word Wall Street  Journal (WSJ) corpus. During the porting, we explored the following research  directions to refine the system: i) training context-dependent models with a reg(cid:173) ularization method; ii) training SNN with projection pursuit; and ii) combining  different models into a hybrid system. When tested on both a development set  and an independent test set, the resulting neural net system alone yielded a per(cid:173) fonnance at the level of the HMM system, and the hybrid SNN/HMM system  achieved a consistent 10-15% word error reduction over the HMM system. This  paper describes our hybrid system, with emphasis on the optimization methods  employed.
Adaptive knot Placement for Nonparametric Regression
Authors: Vladimir Cherkassky Hossein L. Najafi
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We show how an "Elman" network architecture, constructed from  recurrently connected oscillatory associative memory network mod(cid:173) ules, can employ selective "attentional" control of synchronization  to direct the flow of communication and computation within the  architecture to solve a grammatical inference problem.  Previously we have shown how the discrete time "Elman" network  algorithm can be implemented in a network completely described  by continuous ordinary differential equations. The time steps (ma(cid:173) chine cycles) of the system are implemented by rhythmic variation  (clocking) of a bifurcation parameter. In this architecture, oscilla(cid:173) tion amplitude codes the information content or activity of a mod(cid:173) ule (unit), whereas phase and frequency are used to "softwire" the  network. Only synchronized modules communicate by exchang(cid:173) ing amplitude information; the activity of non-resonating modules  contributes incoherent crosstalk noise.  Attentional control is modeled as a special subset of the hidden  modules with ouputs which affect the resonant frequencies of other  hidden modules. They control synchrony among the other mod(cid:173) ules and direct the flow of computation (attention) to effect transi(cid:173) tions between two subgraphs of a thirteen state automaton which  the system emulates to generate a Reber grammar. The internal  crosstalk noise is used to drive the required random transitions of  the automaton.
Bayesian Modeling and Classification of Neural Signals
Authors: Michael S. Lewicki
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Signal  processing  and  classification  algorithms often  have limited  applicability resulting from an inaccurate model of the signal's un(cid:173) derlying  structure.  We  present  here  an  efficient,  Bayesian  algo(cid:173) rithm for modeling a signal composed of the superposition of brief,  Poisson-distributed functions.  This methodology is  applied  to  the  specific  problem  of modeling  and  classifying  extracellular  neural  waveforms which  are  composed  of a  superposition of an  unknown  number of action potentials CAPs).  Previous  approaches have had  limited success due largely to the problems of determining the spike  shapes,  deciding  how  many  are  shapes  distinct,  and  decomposing  overlapping APs.  A  Bayesian solution to each of these problems is  obtained  by inferring  a  probabilistic model of the  waveform.  This  approach quantifies the uncertainty of the form and number of the  inferred  AP  shapes  and  is  used  to  obtain  an  efficient  method  for  decomposing  complex overlaps.  This algorithm can extract  many  times  more  information than  previous methods  and facilitates  the  extracellular  investigation of neuronal  classes  and  of interactions  within neuronal  circuits.
Supervised learning from incomplete data via an EM approach
Authors: Michael I. Jordan Zoubin Ghahramani
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Real-world  learning  tasks  may involve  high-dimensional  data sets  with  arbitrary  patterns of missing  data.  In  this  paper  we  present  a  framework  based  on  maximum likelihood  density  estimation for  learning from such  data set.s.  VVe  use  mixture models for  the  den(cid:173) sity  estimates and  make two  distinct  appeals  to  the  Expectation(cid:173) Maximization  (EM)  principle  (Dempster  et  al.,  1977)  in  deriving  a  learning  algorithm-EM is  used  both  for  the  estimation of mix(cid:173) ture  components  and  for  coping  wit.h  missing  dat.a.  The  result(cid:173) ing  algorithm  is  applicable  t.o  a  wide  range  of supervised  as  well  as  unsupervised  learning  problems.  Result.s  from  a  classification  benchmark-t.he iris  data set-are presented.
Correlation Functions in a Large Stochastic Neural Network
Authors: Haim Sompolinsky Iris Ginzburg
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Most theoretical investigations of large recurrent networks focus on  the properties of the macroscopic order parameters such as popu(cid:173) lation averaged activities or average overlaps with memories. How(cid:173) ever, the statistics of the fluctuations in the local activities may  be an important testing ground for comparison between models  and observed cortical dynamics. We evaluated the neuronal cor(cid:173) relation functions in a stochastic network comprising of excitatory  and inhibitory populations. We show that when the network is in  a stationary state, the cross-correlations are relatively weak, i.e.,  their amplitude relative to that of the auto-correlations are of or(cid:173) der of 1/ N, N being the size of the interacting population. This  holds except in the neighborhoods of bifurcations to nonstationary  states. As a bifurcation point is approached the amplitude of the  cross-correlations grows and becomes of order 1 and the decay time(cid:173) constant diverges. This behavior is analogous to the phenomenon  of critical slowing down in systems at thermal equilibrium near a  critical point. Near a Hopf bifurcation the cross-correlations ex(cid:173) hibit damped oscillations.
Clustering with a Domain-Specific Distance Measure
Authors: Eric Mjolsness, Anand Rangarajan Steven Gold
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: With  a  point matching distance  measure which  is  invariant under  translation,  rotation  and  permutation,  we  learn  2-D  point-set  ob(cid:173) jects,  by clustering noisy point-set images.  Unlike  traditional clus(cid:173) tering methods which use distance measures that operate on feature  vectors - a representation common to most problem domains - this  object-based  clustering technique employs a  distance measure spe(cid:173) cific  to  a  type  of object  within  a  problem  domain.  Formulating  the clustering problem as  two  nested objective functions,  we derive  optimization  dynamics  similar  to  the  Expectation-Maximization  algorithm used  in mixture models.
Speaker Recognition Using Neural Tree Networks
Authors: Richard J. Mammone Kevin R. Farrell
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: A new classifier is presented for text-independent speaker recognition. The  new classifier is called the modified neural tree network (MNTN). The  NTN is a hierarchical classifier that combines the properties of decision  trees and feed-forward neural networks. The MNTN differs from the stan(cid:173) dard NTN in that a new learning rule based on discriminant learning  is used, which minimizes the classification error as opposed to a norm  of the approximation error. The MNTN also uses leaf probability mea(cid:173) sures in addition to the class labels. The MNTN is evaluated for several  speaker identification experiments and is compared to multilayer percep(cid:173) trons (MLPs) , decision trees, and vector quantization (VQ) classifiers. The  VQ classifier and MNTN demonstrate comparable performance and per(cid:173) form significantly better than the other classifiers for this task. Addition(cid:173) ally, the MNTN provides a logarithmic saving in retrieval time over that  of the VQ classifier. The MNTN and VQ classifiers are also compared  for several speaker verification experiments where the MNTN is found to  outperform the VQ classifier.
Tonal Music as a Componential Code: Learning Temporal Relationships Between and Within Pitch and Timing Components
Authors: Janet Wiles Catherine Stevens
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: This study explores the extent to which a network that learns the  temporal relationships within and between the component features of  Western tonal music can account for music theoretic and psychological  phenomena such as the tonal hierarchy and rhythmic expectancies.  Predicted and generated sequences were recorded as the representation of  a 153-note waltz melody was learnt by a predictive, recurrent network.  The network learned transitions and relations between and within pitch  and timing components: accent and duration values interacted in the  development of rhythmic and metric structures and, with training, the  network developed chordal expectancies in response to the activation of  individual tones. Analysis of the hidden unit representation revealed  that musical sequences are represented as transitions between states in  hidden unit space.
Connectionist Models for Auditory Scene Analysis
Authors: Richard O. Duda
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Although  the  visual  and  auditory  systems  share  the  same  basic  tasks  of informing  an  organism  about  its  environment,  most  con(cid:173) nectionist  work  on  hearing  to  date  has  been  devoted  to  the  very  different  problem of speech  recognition .  VVe  believe that the  most  fundamental task of the auditory system is  the analysis of acoustic  signals into components corresponding to individual sound sources,  which  Bregman has called auditory scene  analysis .  Computational  and connectionist work on auditory scene analysis is  reviewed,  and  the  outline  of  a  general  model  that  includes  these  approaches  is  described.
Learning in Compositional Hierarchies: Inducing the Structure of Objects from Data
Authors: Joachim Utans
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: I  propose a learning algorithm for  learning hierarchical  models  for ob(cid:173) ject recognition.  The  model  architecture  is  a  compositional  hierarchy  that  represents  part-whole relationships:  parts  are  described  in  the lo(cid:173) cal  context of substructures  of the  object.  The  focus  of this  report  is  inducing  the  structure  of  learning  hierarchical  models  from  data,  i.e.  model  prototypes from  observed exemplars of an  object.  At each  node  in the hierarchy, a probability distribution governing its parameters must  be learned.  The connections between  nodes reflects  the structure of the  object.  The  formulation of substructures  is  encouraged  such  that  their  parts  become  conditionally  independent.  The  resulting  model  can  be  interpreted  as  a  Bayesian  Belief Network  and  also  is  in  many  respects  similar to the stochastic visual grammar described by Mjolsness.
VLSI Phase Locking Architectures for Feature Linking in Multiple Target Tracking Systems
Authors: Thomas G. Edwards Andreas G. Andreou
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Recent  physiological  research  has  shown  that  synchronization  of  oscillatory  responses  in  striate  cortex  may  code  for  relationships  between  visual  features  of objects.  A  VLSI  circuit  has  been  de(cid:173) signed  to  provide  rapid  phase-locking  synchronization of multiple  oscillators to allow for further exploration of this neural mechanism.  By  exploiting the  intrinsic  random transistor  mismatch of devices  operated  in  subthreshold,  large  groups  of phase-locked  oscillators  can  be  readily  partitioned  into  smaller  phase-locked  groups.  A  mUltiple target  tracker for  binary images is described  utilizing this  phase-locking  architecture.  A  VLSI  chip  has  been  fabricated  and  tested  to  verify  the  architecture.  The  chip  employs  Pulse  Ampli(cid:173) tude  Modulation  (PAM)  to encode  the output at  the  periphery  of  the system.
A Network Mechanism for the Determination of Shape-From-Texture
Authors: Leif H. Finkel Kô Sakai
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We propose a computational model for how the cortex discriminates  shape and depth from texture. The model consists of four stages: (1)  extraction of local spatial frequency, (2) frequency characterization, (3)  detection of texture compression by normalization, and (4) integration  of the normalized frequency over space. The model accounts for a  number of psychophysical observations including experiments based on  novel random textures. These textures are generated from white noise  and manipulated in Fourier domain in order to produce specific  frequency spectra. Simulations with a range of stimuli, including real  images, show qualitative and quantitative agreement with human  perception.
Encoding Labeled Graphs by Labeling RAAM
Authors: Alessandro Sperduti
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: In  this  paper  we  propose  an  extension  to  the  RAAM  by  Pollack.  This  extension,  the  Labeling  RAAM  (LRAAM),  can  encode  la(cid:173) beled  graphs  with  cycles  by  representing pointers explicitly.  Data  encoded  in  an  LRAAM  can  be  accessed  by  pointer  as  well  as  by  content.  Direct  access  by  content  can  be  achieved  by  transform(cid:173) ing  the  encoder  network  of  the  LRAAM  into  an  analog  Hopfield  network  with  hidden  units.  Different  access  procedures  can  be  defined  depending  on  the  access  key.  Sufficient  conditions  on  the  asymptotical stability of the associated Hopfield network are briefly  introduced.
Learning Mackey-Glass from 25 examples, Plus or Minus 2
Authors: Garrison Cottrell, Halbert White Mark Plutowski
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We apply active exemplar selection (Plutowski &. White, 1991;  1993) to predicting a chaotic time series. Given a fixed set of ex(cid:173) amples, the method chooses a concise subset for training. Fitting  these exemplars results in the entire set being fit as well as de(cid:173) sired. The algorithm incorporates a method for regulating network  complexity, automatically adding exempla.rs and hidden units as  needed. Fitting examples generated from the Mackey-Glass equa(cid:173) tion with fractal dimension 2.1 to an rmse of 0.01 required about 25  exemplars and 3 to 6 hidden units. The method requires an order  of magnitude fewer floating point operations than training on the  entire set of examples, is significantly cheaper than two contend(cid:173) ing exemplar selection techniques, and suggests a simpler active  selection technique that performs comparably.
Structured Machine Learning for 'Soft' Classification with Smoothing Spline ANOVA and Stacked Tuning, Testing and Evaluation
Authors: Yuedong Wang, Chong Gu, Ronald Klein MD, Barbara Klein MD Grace Wahba
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We describe the use of smoothing spline analysis of variance (SS(cid:173) ANOVA) in the penalized log likelihood context, for learning  (estimating) the probability p of a '1' outcome, given a train(cid:173) ing set with attribute vectors and outcomes. p is of the form  pet) = eJ(t) /(1 + eJ(t)), where, if t is a vector of attributes, f  is learned as a sum of smooth functions of one attribute plus a  sum of smooth functions of two attributes, etc. The smoothing  parameters governing f are obtained by an iterative unbiased risk  or iterative GCV method. Confidence intervals for these estimates  are available. 

Introduction to 'soft' classification and the bias-variance tradeoff. 

In medical risk factor analysis records of attribute vectors and outcomes (0 or 1)  for each example (patient) for n examples are available as training data. Based on  the training data, it is desired to estimate the probability p of the 1 outcome for any
Computational Differences between Asymmetrical and Symmetrical Networks
Authors: Peter Dayan Zhaoping Li
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Symmetrically connected  recurrent networks have recently been  used as models of a host of neural computations.  However, be(cid:173) cause of the separation between excitation and inhibition, biolog(cid:173) ical  neural networks  are  asymmetrical.  We  study characteristic  differences between asymmetrical networks and their symmetri(cid:173) cal  counterparts,  showing  that  they  have  dramatically  different  dynamical behavior and also how the differences can be exploited  for computational ends.  We  illustrate our results in the case of a  network that is a selective amplifier.
A Randomized Algorithm for Pairwise Clustering
Authors: Daphna Weinshall, Michael Werman Yoram Gdalyahu
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We present a stochastic clustering algorithm based on pairwise sim(cid:173) ilarity  of datapoints.  Our  method  extends  existing  deterministic  methods, including agglomerative algorithms, min-cut graph  algo(cid:173) rithms,  and  connected  components.  Thus  it  provides  a  common  framework for  all  these  methods.  Our  graph-based  method  differs  from  existing  stochastic  methods  which  are  based  on  analogy  to  physical  systems.  The  stochastic  nature  of our  method  makes  it  more  robust  against  noise,  including  accidental  edges  and  small  spurious clusters.  We demonstrate the superiority of our algorithm  using  an example with  3 spiraling bands and  a  lot of noise.
Risk Sensitive Reinforcement Learning
Authors: Oliver Mihatsch Ralph Neuneier
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A directed generative model for binary data using a  small number  of hidden  continuous  units  is  investigated.  A  clipping  nonlinear(cid:173) ity distinguishes the model from conventional principal components  analysis.  The relationships between the correlations of the underly(cid:173) ing continuous Gaussian variables  and the binary output variables  are utilized  to learn  the  appropriate weights  of the network.  The  advantages of this  approach are illustrated on a  translationally in(cid:173) variant binary distribution and on handwritten digit  images.
Convergence of the Wake-Sleep Algorithm
Authors: Shun-ichi Amari, Hiroyuki Nakahara Shiro Ikeda
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: The W-S (Wake-Sleep) algorithm is a simple learning rule for the models  with hidden variables.  It is  shown that this algorithm can be applied to  a factor  analysis model  which is  a linear version  of the  Helmholtz ma(cid:173) chine.  But even for a factor analysis model,  the general convergence is  not proved theoretically.  In this article, we describe the geometrical un(cid:173) derstanding of the W-S  algorithm in contrast with the EM (Expectation(cid:173) Maximization) algorithm and the em algorithm.  As the result, we prove  the convergence of the W-S  algorithm for the factor analysis model.  We  also show the condition for the convergence in general models.
Mean Field Methods for Classification with Gaussian Processes
Authors: Ole Winther Manfred Opper
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We discuss the application of TAP mean field methods known from  the Statistical Mechanics of disordered systems to Bayesian classifi(cid:173) cation models with Gaussian processes.  In contrast to previous ap(cid:173) proaches,  no  knowledge about the distribution of inputs is  needed.  Simulation results for  the Sonar data set are given. 
1  Modeling with Gaussian Processes 
Bayesian models which are based on Gaussian prior distributions on function spaces  are promising non-parametric statistical tools.  They have been recently introduced  into the Neural  Computation community  (Neal  1996, Williams &  Rasmussen  1996,  Mackay  1997).  To  give their basic definition,  we  assume that the likelihood of the  output  or  target  variable  T  for  a  given  input  s  E  RN  can  be  written  in  the  form  p(Tlh(s))  where  h  : RN  --+  R  is  a  priori  assumed  to  be  a  Gaussian  random field.  If we  assume fields  with  zero  prior  mean,  the statistics  of h  is  entirely  defined  by  the second order correlations C(s, S')  ==  E[h(s)h(S')], where E  denotes expectations 
310 
M  Opper and 0.  Winther 
with respect  to the prior.  Interesting examples  are
Learning from Dyadic Data
Authors: Jan Puzicha, Michael I. Jordan Thomas Hofmann
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Dyadzc  data  refers  to  a  domain  with  two  finite  sets  of objects  in  which  observations are  made for  dyads , i.e., pairs with one element  from  either  set.  This  type  of data  arises  naturally  in  many  ap(cid:173) plication  ranging  from  computational  linguistics  and  information  retrieval to preference  analysis and computer vision.  In this paper,  we  present  a  systematic,  domain-independent framework  of learn(cid:173) ing  from  dyadic  data by  statistical  mixture  models.  Our  approach  covers different models with fiat  and hierarchical latent class struc(cid:173) tures.  We  propose  an  annealed  version  of the  standard  EM  algo(cid:173) rithm for  model fitting  which  is  empirically evaluated  on  a  variety  of data sets from  different  domains.
Call-Based Fraud Detection in Mobile Communication Networks Using a Hierarchical Regime-Switching Model
Authors: Volker Tresp Jaakko Hollmén
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Fraud causes substantial losses to telecommunication carriers. Detec(cid:173) tion systems which automatically detect illegal use of the network can be  used to alleviate the problem. Previous approaches worked on features  derived from the call patterns of individual users. In this paper we present  a call-based detection system based on a hierarchical regime-switching  model. The detection problem is formulated as an inference problem on  the regime probabilities. Inference is implemented by applying the junc(cid:173) tion tree algorithm to the underlying graphical model. The dynamics are  learned from data using the EM algorithm and subsequent discriminative  training. The methods are assessed using fraud data from a real mobile  communication network.
Analyzing and Visualizing Single-Trial Event-Related Potentials
Authors: Scott Makeig, Marissa Westerfield, Jeanne Townsend, Eric Courchesne, Terrence J. Sejnowski Tzyy-Ping Jung
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Event-related  potentials (ERPs),  are portions  of electroencephalo(cid:173) graphic  (EEG)  recordings  that  are  both  time- and  phase-locked  to  experimental  events.  ERPs  are  usually  averaged  to  increase  their  signal/noise  ratio  relative  to  non-phase  locked  EEG  activ(cid:173) ity,  regardless  of the  fact  that  response  activity  in  single  epochs  may vary  widely  in  time course  and scalp distribution.  This study  applies a linear decomposition tool, Independent Component Anal(cid:173) ysis  (ICA)  [1],  to  multichannel single-trial  EEG  records  to  derive  spatial filters  that  decompose  single-trial  EEG epochs  into  a  sum  of temporally  independent  and spatially fixed  components  arising  from  distinct  or  overlapping  brain  or  extra-brain  networks.  Our  results  on  normal  and  autistic  subjects  show  that  ICA  can  sep(cid:173) arate  artifactual,  stimulus-locked,  response-locked,  and.  non-event  related  background  EEG  activities  into separate  components,  al(cid:173) lowing ( 1) removal of pervasive artifacts of all types from single-trial  EEG records,  and (2) identification of both stimulus- and response(cid:173) locked EEG components.  Second, this study proposes a new visual(cid:173) ization tool, the 'ERP image', for investigating variability in laten(cid:173) cies  and amplitudes of event-evoked responses  in spontaneous EEG  or  MEG  records.  We show  that sorting single-trial ERP epochs  in  order  of reaction  time  and  plotting  the  potentials  in  2-D  clearly  reveals  underlying  patterns  of response  variability  linked  to  per(cid:173) formance.  These  analysis  and  visualization  tools  appear  broadly  applicable to electrophyiological research  on both normal and clin(cid:173) ical  populations. 
Analyzing and Visualizing Single-Trial Event-Related Potentials
Replicator Equations, Maximal Cliques, and Graph Isomorphism
Authors: Marcello Pelillo
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present  a  new  energy-minimization  framework  for  the  graph  isomorphism  problem  which  is  based  on  an  equivalent  maximum  clique formulation.  The approach is centered around a fundamental  result proved by  Motzkin and Straus in the mid-1960s, and recently  expanded in  various ways,  which  allows  us  to formulate the maxi(cid:173) mum clique problem in  terms of a standard quadratic program.  To  solve  the  program we  use  "replicator"  equations,  a  class  of simple  continuous- and discrete-time dynamical systems developed in var(cid:173) ious  branches  of theoretical  biology.  We  show  how,  despite  their  inability  to escape from  local  solutions,  they  nevertheless  provide  experimental results which are competitive with those obtained us(cid:173) ing more elaborate mean-field  annealing heuristics.
Orientation, Scale, and Discontinuity as Emergent Properties of Illusory Contour Shape
Authors: Lance R. Williams Karvel K. Thornber
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  recent  neural  model  of  illusory  contour  formation  is  based  on  a  distribution  of natural  shapes  traced  by  particles  moving  with  constant speed in directions given by Brownian motions.  The input  to that model consists of pairs of position and direction constraints  and the  output  consists  of the  distribution  of contours joining all  such  pairs.  In general,  these contours will  not  be closed  and  their  distribution  will  not  be  scale-invariant.  In  this  paper,  we  show  how  to  compute  a  scale-invariant  distribution  of  closed  contours  given  position  constraints  alone  and  use  this  result  to  explain  a  well  known illusory contour effect.
Active Noise Canceling Using Analog Neuro-Chip with On-Chip Learning Capability
Authors: Soo-Young Lee Jung-Wook Cho
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A modular analogue neuro-chip set with  on-chip learning capability  is  developed  for  active  noise  canceling.  The  analogue  neuro-chip  set  incorporates  the  error  backpropagation  learning  rule  for  practical  applications,  and  allows  pin-to-pin  interconnections  for  multi-chip  boards.  The  developed  neuro-board  demonstrated  active  noise  canceling  without  any  digital  signal  processor.  Multi-path  fading  of  acoustic  channels,  random  noise,  and  nonlinear distortion  of the  loud  speaker  are  compensated  by  the  adaptive  learning  circuits  of  the  neuro-chips. Experimental  results  are  reported  for  cancellation  of car  noise in  real time.
Coordinate Transformation Learning of Hand Position Feedback Controller by Using Change of Position Error Norm
Authors: Susumu Tachi Eimei Oyama
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: In  order  to  grasp  an  object,  we  need  to  solve  the  inverse  kine(cid:173) matics problem, i.e., the coordinate transformation from  the visual  coordinates  to the joint  angle  vector  coordinates  of the arm.  Al(cid:173) though  several models  of coordinate transformation learning have  been proposed, they suffer from  a number of drawbacks.  In human  motion  control,  the  learning  of the  hand  position  error  feedback  controller in the inverse kinematics solver is important.  This paper  proposes  a  novel  model  of the coordinate transformation  learning  of  the  human  visual  feedback  controller  that  uses  the  change  of  the joint  angle vector and the corresponding change of the square  of the  hand  position  error  norm.  The  feasibility  of the  proposed  model  is  illustrated using numerical simulations.
Multi-Electrode Spike Sorting by Clustering Transfer Functions
Authors: Hanan Davidowitz, Naftali Tishby Dmitry Rinberg
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A  new  paradigm  is  proposed  for  sorting  spikes  in  multi-electrode  data using ratios of transfer functions  between cells and electrodes.  It  is  assumed  that  for  every  cell  and  electrode  there  is  a  stable  linear  relation.  These are  dictated  by  the properties of the tissue,  the electrodes  and  their relative  geometries.  The main  advantage  of the method is that it is  insensitive to variations in the shape and  amplitude of a  spike.  Spike  sorting  is  carried  out  in  two  separate  steps.  First,  templates  describing  the statistics of each spike type  are generated by clustering transfer function  ratios then spikes are  detected  in  the  data  using  the  spike  statistics.  These  techniques  were  applied  to  data generated  in  the  escape  response  system  of  the cockroach.
Fisher Scoring and a Mixture of Modes Approach for Approximate Inference and Learning in Nonlinear State Space Models
Authors: Volker Tresp Thomas Briegel
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present Monte-Carlo generalized EM  equations for learning in  non(cid:173) linear state space models.  The difficulties lie in the Monte-Carlo E-step  which consists of sampling from  the posterior distribution of the hidden  variables given the observations. The new idea presented in this paper is  to generate samples from a Gaussian approximation to the true posterior  from  which it is easy to obtain independent samples.  The parameters  of  the Gaussian approximation are either derived from the extended Kalman  filter or the Fisher scoring algorithm. In case the posterior density is mul(cid:173) timodal  we propose to  approximate the posterior by  a sum of Gaussians  (mixture of modes approach).  We show that sampling from the approxi(cid:173) mate posterior densities obtained by the above algorithms leads to better  models  than  using  point estimates  for  the  hidden  states.  In  our exper(cid:173) iment,  the Fisher scoring algorithm obtained  a better approximation of  the posterior mode than the EKF. For a multimodal distribution, the mix(cid:173) ture of modes approach gave superior results.
Probabilistic Modeling for Face Orientation Discrimination: Learning from Labeled and Unlabeled Data
Authors: Shumeet Baluja
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: This paper presents probabilistic modeling methods to solve the problem of dis(cid:173) criminating between five facial  orientations with  very  little labeled data.  Three  models are explored. The first model maintains no inter-pixel dependencies, the  second model is capable of modeling a set of arbitrary pair-wise dependencies,  and the last model allows  dependencies  only  between  neighboring pixels. We  show that for all three of these models, the accuracy of the learned models can  be greatly improved by  augmenting a small number of labeled training images  with  a  large  set of unlabeled  images using  Expectation-Maximization.  This  is  important because it is often difficult to obtain image labels, while many unla(cid:173) beled images  are  readily  available.  Through  a  large  set  of empirical  tests,  we  examine the benefits  of unlabeled data  for  each  of the  models.  By  using only  two randomly selected labeled examples per class, we can discriminate between  the five  facial orientations with an accuracy of 94%; with six labeled examples,  we achieve an accuracy of 98%.
Direct Optimization of Margins Improves Generalization in Combined Classifiers
Authors: Peter L. Bartlett, Jonathan Baxter Llew Mason
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Cumulative training margin  dis(cid:173) tributions  for  AdaBoost  versus  our  "Direct  Optimization  Of  Margins"  (DOOM)  algorithm.  The dark curve is  AdaBoost, the  light  curve  is  DOOM.  DOOM  sacrifices  significant  training  er(cid:173) ror for  improved test error  (hori(cid:173) zontal marks on margin= 0 line)_ 
-1 
-0.8  -0.6  -0.4  -0.2  0 
0.2  0.4  0.6  0.8
A Neuromorphic Monaural Sound Localizer
Authors: Chiang-Jung Pu, José Carlos Príncipe John G. Harris
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  describe the first  single microphone sound localization system  and its inspiration from theories of human monaural sound localiza(cid:173) tion.  Reflections and diffractions caused by the external ear (pinna)  allow  humans to estimate sound source elevations  using only one  ear.  Our single microphone localization model relies on a specially  shaped reflecting structure that serves the role  of the pinna.  Spe(cid:173) cially designed analog VLSI circuitry uses  echo-time processing to  localize the sound.  A CMOS integrated circuit has been designed,  fabricated,  and successfully demonstrated on actual sounds.
Boxlets: A Fast Convolution Algorithm for Signal Processing and Neural Networks
Authors: Léon Bottou, Patrick Haffner, Yann LeCun Patrice Simard
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Signal  processing  and  pattern  recognition  algorithms make  exten(cid:173) sive  use  of convolution.  In  many cases, computational accuracy  is  not  as  important  as  computational  speed.  In  feature  extraction,  for  instance,  the  features  of interest  in  a  signal  are  usually  quite  distorted.  This form of noise justifies some level of quantization in  order  to  achieve  faster  feature  extraction .  Our  approach  consists  of approximating regions  of the  signal  with  low  degree  polynomi(cid:173) als,  and then differentiating the resulting signals in order to obtain  impulse functions  (or derivatives of impulse functions).  With this  representation,  convolution  becomes  extremely  simple and  can  be  implemented quite effectively.  The true  convolution can  be  recov(cid:173) ered  by  integrating  the  result  of  the  convolution.  This  method  yields  substantial speed  up  in feature  extraction  and is  applicable  to convolutional neural  networks.
Experimental Results on Learning Stochastic Memoryless Policies for Partially Observable Markov Decision Processes
Authors: Satinder P. Singh John K. Williams
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Partially Observable Markov Decision  Processes  (pO "MOPs)  constitute  an  important  class  of reinforcement  learning problems  which  present  unique theoretical and computational difficulties.  In  the absence of the  Markov property,  popular reinforcement  learning  algorithms  such  as  Q-Iearning  may  no  longer  be  effective,  and  memory-based  methods  which remove partial observability via state-estimation  are notoriously  expensive.  An alternative approach is  to seek a stochastic memoryless  policy  which  for  each  observation  of  the  environment  prescribes  a  probability  distribution  over  available  actions  that  maximizes  the  average  reward  per  timestep.  A  reinforcement  learning  algorithm  which  learns  a locally optimal  stochastic memoryless  policy has  been  proposed by Jaakkola,  Singh and Jordan,  but not empirically verified.  We present  a variation  of this  algorithm,  discuss  its  implementation,  and demonstrate its viability using four test problems.
Attentional Modulation of Human Pattern Discrimination Psychophysics Reproduced by a Quantitative Model
Authors: Jochen Braun, Dale K. Lee, Christof Koch Laurent Itti
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  previously  proposed  a  quantitative model  of early  visual  pro(cid:173) cessing  in  primates,  based  on  non-linearly interacting visual filters  and statistically efficient  decision.  We  now  use  this model to inter(cid:173) pret  the observed  modulation of a  range of human psychophysical  thresholds  with  and  without  focal  visual  attention.  Our  model  - calibrated  by  an  automatic fitting  procedure  - simultaneously  re(cid:173) produces  thresholds for  four  classical  pattern discrimination tasks,  performed while attention was engaged by another concurrent task.  Our model then predicts that the seemingly complex improvements  of certain  thresholds,  which  we  observed  when  attention  was  fully  available for  the  discrimination  tasks,  can  best  be  explained  by  a  strengthening  of competition among early  visual filters.
Signal Detection in Noisy Weakly-Active Dendrites
Authors: Christof Koch Amit Manwani
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Here we derive measures quantifying the information loss of a synaptic  signal due to the presence of neuronal noise sources, as it electrotonically  propagates along a weakly-active dendrite. We model the dendrite as an  infinite linear cable, with noise sources distributed along its length.  The  noise sources we consider are thermal noise, channel noise arising from  the stochastic nature of voltage-dependent ionic channels (K+ and Na+)  and synaptic noise due to spontaneous background activity. We assess the  efficacy of information transfer using a signal detection paradigm where  the objective is to detect the presence/absence of a presynaptic spike from  the post-synaptic membrane voltage. This allows us to analytically assess  the role of each of these noise sources in  information transfer.  For our  choice  of parameters,  we find  that the  synaptic  noise  is  the  dominant  noise source which limits the maximum length over which information  be reliably transmitted.
Kernel PCA and De-Noising in Feature Spaces
Authors: Bernhard Schölkopf, Alex J. Smola, Klaus-Robert Müller, Matthias Scholz, Gunnar Rätsch Sebastian Mika
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Kernel  PCA  as  a  nonlinear feature  extractor has  proven powerful  as  a  preprocessing step for classification algorithms.  But it can also be con(cid:173) sidered  as  a  natural  generalization of linear principal  component anal(cid:173) ysis.  This  gives  rise  to  the  question  how  to  use  nonlinear features  for  data compression, reconstruction, and de-noising, applications common  in  linear PCA.  This is  a nontrivial  task,  as the results provided by  ker(cid:173) nel PCA live in some high dimensional feature space and need not have  pre-images in  input space.  This work presents ideas for finding approxi(cid:173) mate pre-images, focusing on Gaussian kernels, and shows experimental  results  using  these pre-images in  data reconstruction and de-noising on  toy examples as well as on real world data. 
1  peA and Feature Spaces  Principal  Component Analysis  (PC A)  (e.g.  [3])  is  an  orthogonal  basis  transformation.  The  new  basis  is  found  by  diagonalizing  the  centered  covariance  matrix  of a  data  set  {Xk  E  RNlk  = 1, ... ,f}, defined  by  C  = ((Xi  - nates in the Eigenvector basis are called principal components.  The size of an Eigenvalue  >.  corresponding to  an  Eigenvector v  of C equals the amount of variance in  the direction  of v.  Furthermore, the directions of the  first  n Eigenvectors corresponding to the biggest  n Eigenvalues cover as much variance as possible by n orthogonal directions. In many ap(cid:173) plications they contain the most interesting information: for instance, in data compression,  where we project onto the directions with biggest variance to  retain  as  much information  as possible, or in de-noising, where we deliberately drop directions with small variance. 
(Xk))T).  The coordi(cid:173)
Multiple Paired Forward-Inverse Models for Human Motor Learning and Control
Authors: Daniel M. Wolpert, Mitsuo Kawato Masahiko Haruno
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Humans  demonstrate  a  remarkable  ability  to  generate  accurate  and  appropriate motor behavior under many different and oftpn  uncprtain  environmental  conditions.  This  paper  describes  a  new  modular  ap(cid:173) proach to human motor learning and control, baspd on multiple pairs of  inverse  (controller)  and forward  (prpdictor)  models.  This architecture  simultaneously learns the multiple inverse models necessary for control  as well as how to select the inverse models appropriate for a given em'i(cid:173) ronm0nt.  Simulations of object manipulation demonstrates the ability  to learn  mUltiple  objects,  appropriate  generalization  to  novel  objects  and  the  inappropriate  activation  of motor  programs  based  on  visual  cues,  followed  by on-line  correction, seen  in  the  "size-weight  illusion".
SMEM Algorithm for Mixture Models
Authors: Ryohei Nakano, Zoubin Ghahramani, Geoffrey E. Hinton Naonori Ueda
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We  present  a  split and  merge  EM  (SMEM)  algorithm to overcome the local  maximum problem in  parameter estimation of finite  mixture  models.  In the  case  of mixture  models,  non-global  maxima often  involve  having  too  many  components of a  mixture  model  in one part of the space and  too  few  in  an(cid:173) other, widely separated part of the space.  To escape from such configurations  we  repeatedly perform simultaneous split and merge operations using a  new  criterion  for  efficiently  selecting  the  split  and  merge  candidates.  We  apply  the proposed algorithm to the training of Gaussian mixtures and mixtures of  factor  analyzers  using synthetic  and  real  data and show  the effectiveness  of  using  the  split  and  merge  operations  to  improve the  likelihood  of both  the  training data and of held-out test data.
Learning Lie Groups for Invariant Visual Perception
Authors: Daniel L. Ruderman Rajesh P. N. Rao
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: One  of the  most important problems  in  visual  perception is  that of visual  in(cid:173) variance:  how are objects perceived to be the same despite undergoing transfor(cid:173) mations such as translations, rotations or scaling?  In this paper, we describe a  Bayesian method for learning invariances based on Lie group theory.  We show  that previous approaches based on first-order Taylor series expansions of inputs  can be regarded as special cases of the Lie group approach, the latter being ca(cid:173) pable of handling in principle arbitrarily large transfonnations. Using a matrix(cid:173) exponential based generative model of images,  we derive an unsupervised al(cid:173) gorithm for learning Lie group operators from  input data containing infinites(cid:173) imal transfonnations.  The on-line unsupervised learning algorithm maximizes  the posterior probability of generating the training data.  We provide experimen(cid:173) tal results suggesting that the proposed method can learn Lie group operators for  handling reasonably large I-D translations and 2-D rotations.
Tractable Variational Structures for Approximating Graphical Models
Authors: Wim Wiegerinck David Barber
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Graphical models provide a broad probabilistic framework with ap(cid:173) plications in speech recognition  (Hidden  Markov  Models),  medical  diagnosis  (Belief  networks)  and  artificial  intelligence  (Boltzmann  Machines).  However,  the  computing time is  typically  exponential  in the number of nodes in the graph.  Within the variational frame(cid:173) work for  approximating these models, we  present two classes of dis(cid:173) tributions,  decimatable  Boltzmann  Machines  and Tractable  Belief  Networks  that  go  beyond  the  standard  factorized  approach.  We  give  generalised  mean-field  equations  for  both  these  directed  and  undirected  approximations.  Simulation  results  on  a  small  bench(cid:173) mark problem suggest using these richer approximations compares  favorably  against others previously reported in the literature.
A Micropower CMOS Adaptive Amplitude and Shift Invariant Vector Quantiser
Authors: Raymond J. Wang, Marwan A. Jabri Richard Coggins
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: In  this  paper we  describe the  architecture,  implementation and  experi(cid:173) mental results for an  Intracardiac Electrogram (ICEG) classification and  compression chip.  The chip processes  and  vector-quantises  30 dimen(cid:173) sional analogue vectors while consuming a maximum of 2.5  J-tW  power  for a heart rate of 60 beats per minute (1  vector per second) from a 3.3 V  supply.  This  represents  a significant advance on  previous  work  which  achieved ultra low power supervised morphology classification since the  template matching scheme used  in  this chip enables unsupervised blind  classification of abnonnal rhythms and the computational support for low  bit rate data compression.  The adaptive template matching scheme used  is tolerant to amplitude variations, and inter- and intra-sample time shifts.