mtybilly/Instruct2DS huggingface dataset:
A Non-linear Information Maximisation Algorithm that Performs Blind Separation
Authors: Terrence J. Sejnowski Anthony J. Bell
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: A new learning algorithm is derived which performs online stochas(cid:173) tic gradient ascent in the mutual information between outputs and  inputs  of a  network.  In  the  absence  of a  priori knowledge  about  the  'signal'  and  'noise'  components  of the  input,  propagation  of  information depends  on  calibrating network  non-linearities to  the  detailed  higher-order  moments of the  input  density functions.  By  incidentally  minimising  mutual  information  between  outputs,  as  well  as  maximising  their  individual  entropies,  the  network  'fac(cid:173) torises'  the  input  into  independent  components.  As  an  example  application,  we  have  achieved  near-perfect  separation of ten  digi(cid:173) tally mixed speech signals.  Our simulations lead us to believe  that  our network performs better at blind separation than the Herault(cid:173) J utten network,  reflecting the fact that it is derived rigorously from  the  mutual information objective. 
468 
Anthony J.  Bell,  Terrence  J.  Sejnowski
Pulsestream Synapses with Non-Volatile Analogue Amorphous-Silicon Memories
Authors: Alan F. Murray, Stephen Churcher, J. Hajto, M. J. Rose A. J. Holmes
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: This  paper  presents  results  from  the  first  use  of neural  networks  for  the real-time feedback  control  of high  temperature plasmas in  a  tokamak fusion  experiment.  The tokamak is  currently  the  prin(cid:173) cipal  experimental  device  for  research  into  the  magnetic  confine(cid:173) ment  approach  to  controlled  fusion.  In  the  tokamak,  hydrogen  plasmas,  at  temperatures  of  up  to  100  Million  K,  are  confined  by  strong  magnetic  fields.  Accurate  control  of the  position  and  shape  of the plasma boundary  requires  real-time feedback  control  of the magnetic field  structure on  a  time-scale of a  few  tens of mi(cid:173) croseconds.  Software simulations have demonstrated that a  neural  network  approach  can  give  significantly  better  performance  than  the linear technique currently  used  on  most tokamak experiments.  The practical  application of the neural  network  approach  requires  high-speed  hardware,  for  which  a  fully  parallel  implementation of  the  multilayer perceptron,  using  a  hybrid  of digital and  analogue  technology,  has been  developed.
Dynamic Cell Structures
Authors: Gerald Sommer Jörg Bruske
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Dynamic Cell Structures (DCS) represent a family  of artificial  neural  architectures  suited  both  for  unsupervised  and  supervised  learning.  They belong to the recently [Martinetz94] introduced class of Topology  Representing  Networks  (TRN)  which  build  perlectly topology  pre(cid:173) serving feature maps. DCS empI'oy a modified Kohonen learning rule  in conjunction with competitive Hebbian learning. The Kohonen type  learning rule serves to adjust the synaptic weight vectors while Hebbian  learning  establishes  a  dynamic  lateral  connection  structure  between  the units reflecting the topology of the feature manifold. In case of super(cid:173) vised learning, i.e. function approximation, each neural unit implements  a Radial Basis Function, and an  additional layer of linear output units  adjusts according to a delta-rule. DCS is the first RBF-based approxima(cid:173) tion  scheme attempting to concurrently learn and  utilize a perfectly to(cid:173) pology preserving map for improved performance.  Simulations on a selection of CMU-Benchmarks indicate that the DCS  idea applied to the Growing Cell Structure algorithm [Fritzke93] leads  to an  efficient and elegant algorithm  that can  beat conventional  models  on similar tasks.
Single Transistor Learning Synapses
Authors: Chris Diorio, Bradley A. Minch, Carver Mead Paul E. Hasler
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  describe single-transistor silicon synapses that compute, learn,  and  provide  non-volatile  memory retention.  The single  transistor  synapses  simultaneously  perform long  term  weight  storage,  com(cid:173) pute the product of the input and the weight value, and update the  weight value according to a Hebbian or a backpropagation learning  rule.  Memory  is  accomplished  via  charge  storage  on  polysilicon  floating  gates, providing long-term  retention without refresh.  The  synapses efficiently use the physics of silicon to perform weight  up(cid:173) dates; the weight value is  increased using tunneling and the weight  value  decreases  using  hot  electron  injection.  The  small  size  and  low  power operation of single transistor synapses allows  the devel(cid:173) opment  of  dense  synaptic  arrays.  We  describe  the  design,  fabri(cid:173) cation,  characterization,  and  modeling  of an  array  of single  tran(cid:173) sistor  synapses.  When  the steady state source  current  is  used  as  the representation of the weight  value,  both the incrementing and  decrementing functions  are  proportional to a  power of the source  current.  The synaptic  array  was  fabricated  in  the standard  21'm  double - poly,  analog process available from  MOSIS.
Comparing the prediction accuracy of artificial neural networks and other statistical models for breast cancer survival
Authors: David B. Rosen, Philip H. Goodman Harry B. Burke
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The  TNM  staging  system  has  been  used  since  the  early  1960's  to  predict  breast  cancer  patient  outcome.  In  an  attempt  to  in(cid:173) crease  prognostic  accuracy,  many putative prognostic factors  have  been  identified.  Because  the  TNM  stage  model  can  not  accom(cid:173) modate  these  new  factors,  the  proliferation  of factors  in  breast  cancer  has  lead  to  clinical  confusion.  What  is  required  is  a  new  computerized  prognostic system  that  can test  putative prognostic  factors  and  integrate  the  predictive  factors  with  the  TNM  vari(cid:173) ables  in order to increase  prognostic  accuracy.  Using  the area un(cid:173) der  the curve of the receiver  operating characteristic,  we  compare  the  accuracy  of the  following  predictive  models  in  terms  of five  year  breast cancer-specific  survival:  pTNM staging system,  princi(cid:173) pal component analysis,  classification and regression  trees,  logistic  regression,  cascade  correlation neural network,  conjugate gradient  descent  neural,  probabilistic neural network,  and backpropagation  neural network.  Several statistical models are significantly more ac-
Learning direction in global motion: two classes of psychophysically-motivated models
Authors: Lucia M. Vaina V. Sundareswaran
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Perceptual learning is  defined  as  fast  improvement  in  performance and  retention  of the  learned  ability  over  a  period  of time.  In a  set  of psy(cid:173) chophysical experiments  we  demonstrated  that  perceptual learning oc(cid:173) curs for the discrimination of direction in stochastic motion stimuli.  Here  we  model  this  learning  using  two  approaches:  a clustering  model  that  learns  to  accommodate  the motion  noise,  and an averaging  model  that  learns to  ignore the noise.  Simulations of the models show  performance  similar to the psychophysical results.
On-line Learning of Dichotomies
Authors: H. S. Seung, H. Sompolinsky N. Barkai
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The performance of on-line algorithms for learning dichotomies is studied. In on-line learn(cid:173) ing, the number of examples P is equivalent to the learning time, since each example is  presented only once. The learning curve, or generalization error as a function of P, depends  on the schedule at which the learning rate is lowered. For a target that is a perceptron rule,  the learning curve of the perceptron algorithm can decrease as fast as p- 1 , if the sched(cid:173) ule is optimized. If the target is not realizable by a perceptron, the perceptron algorithm  does not generally converge to the solution with lowest generalization error. For the case  of unrealizability due to a simple output noise, we propose a new on-line algorithm for a  perceptron yielding a learning curve that can approach the optimal generalization error as  fast as p-l/2. We then generalize the perceptron algorithm to any class of thresholded  smooth functions learning a target from that class. For "well-behaved" input distributions,  if this algorithm converges to the optimal solution, its learning curve can decrease as fast  as p-l.
Asymptotics of Gradient-based Neural Network Training Algorithms
Authors: Terrence L. Fine Sayandev Mukherjee
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We study the asymptotic properties of the sequence of iterates of  weight-vector estimates obtained by training a multilayer feed for(cid:173) ward neural network with a basic gradient-descent method using  a fixed learning constant and no batch-processing. In the one(cid:173) dimensional case, an exact analysis establishes the existence of a  limiting distribution that is not Gaussian in general. For the gen(cid:173) eral case and small learning constant, a linearization approximation  permits the application of results from the theory of random ma(cid:173) trices to again establish the existence of a limiting distribution.  We study the first few moments of this distribution to compare  and contrast the results of our analysis with those of techniques of  stochastic approximation.
Convergence Properties of the K-Means Algorithms
Authors: Yoshua Bengio Léon Bottou
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: This paper studies the convergence properties of the well known  K-Means clustering algorithm. The K-Means algorithm can be de(cid:173) scribed either as a gradient descent algorithm or by slightly extend(cid:173) ing the mathematics of the EM algorithm to this hard threshold  case. We show that the K-Means algorithm actually minimizes the  quantization error using the very fast Newton algorithm.
Using Voice Transformations to Create Additional Training Talkers for Word Spotting
Authors: Richard P Lippmann Eric I. Chang
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Speech  recognizers  provide  good  performance  for  most  users  but  the  error rate often  increases dramatically  for a small  percentage of talkers  who are "different" from  those talkers used for training.  One expensive  solution to  this problem is  to  gather more training data in an attempt to  sample these outlier users. A second solution, explored in this paper,  is  to artificially enlarge the number of training talkers by transforming the  speech of existing training talkers. This approach is similar to enlarging  the training set for  OCR digit recognition  by  warping the training digit  images,  but  is  more  difficult  because  continuous  speech  has  a  much  larger number of dimensions  (e.g.  linguistic,  phonetic,  style,  temporal,  spectral) that differ across talkers. We explored the use of simple linear  spectral warping to enlarge a 48-talker training data base used for  word  spotting.  The  average  detection  rate  overall  was  increased  by  2.9  percentage  points  (from  68.3%  to  71.2%)  for  male  speakers  and  2.5  percentage  points  (from  64.8%  to  67.3%)  for  female  speakers.  This  increase is  small but similar to  that obtained by  doubling the amount of  training data.
Forward dynamic models in human motor control: Psychophysical evidence
Authors: Zoubin Ghahramani, Michael I. Jordan Daniel M. Wolpert
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Based  on  computational  principles,  with  as  yet  no  direct  experi(cid:173) mental  validation,  it  has  been  proposed  that  the  central  nervous  system  (CNS)  uses  an internal model to simulate the  dynamic be(cid:173) havior of the motor system in planning, control and learning (Sut(cid:173) ton  and  Barto,  1981;  Ito,  1984;  Kawato  et  aI.,  1987;  Jordan  and  Rumelhart,  1992;  Miall et aI.,  1993).  We  present  experimental re(cid:173) sults  and simulations based on a  novel  approach  that investigates  the temporal propagation of errors  in the sensorimotor integration  process.  Our results  provide  direct  support for  the  existence  of an  internal model.
Direction Selectivity In Primary Visual Cortex Using Massive Intracortical Connections
Authors: Christof Koch, Rodney Douglas Humbert Suarez
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Almost all models of orientation and direction selectivity in visual  cortex are based on feedforward connection schemes, where genicu(cid:173) late input provides all excitation to both pyramidal and inhibitory  neurons. The latter neurons then suppress the response of the for(cid:173) mer for non-optimal stimuli. However, anatomical studies show  that up to 90 % of the excitatory synaptic input onto any corti(cid:173) cal cell is provided by other cortical cells. The massive excitatory  feedback nature of cortical circuits is embedded in the canonical  microcircuit of Douglas &. Martin (1991). We here investigate ana(cid:173) lytically and through biologically realistic simulations the function(cid:173) ing of a detailed model of this circuitry, operating in a hysteretic  mode. In the model, weak geniculate input is dramatically ampli(cid:173) fied by intracortical excitation, while inhibition has a dual role: (i)  to prevent the early geniculate-induced excitation in the null di(cid:173) rection and (ii) to restrain excitation and ensure that the neurons  fire only when the stimulus is in their receptive-field. Among the 
4 
Humbert Suarez, Christo! Koch, Rodney Douglas 
insights gained are the possibility that hysteresis underlies visual  cortical function, paralleling proposals for short-term memory, and  strong limitations on linearity tests that use gratings. Properties  of visual cortical neurons are compared in detail to this model and  to a classical model of direction selectivity that does not include  excitatory corti co-cortical connections. The model explain a num(cid:173) ber of puzzling features of direction-selective simple cells, includ(cid:173) ing the small somatic input conductance changes that have been  measured experimentally during stimulation in the null direction.  The model also allows us to understand why the velocity-response  curve of area 17 neurons is different from that of their LG N affer(cid:173) ents, and the origin of expansive and compressive nonlinearities in  the contrast-response curve of striate cortical neurons.
Bayesian Query Construction for Neural Network Models
Authors: Jörg Kindermann Gerhard Paass
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: If data collection is costly, there is much to be gained by actively se(cid:173) lecting particularly informative data points in a sequential  way.  In  a  Bayesian decision-theoretic framework we  develop  a  query selec(cid:173) tion  criterion  which  explicitly takes  into account  the intended  use  of the model predictions.  By  Markov Chain Monte Carlo methods  the  necessary  quantities  can  be  approximated to  a  desired  preci(cid:173) sion.  As  the  number  of data  points  grows,  the  model  complexity  is  modified  by  a  Bayesian  model  selection  strategy.  The  proper(cid:173) ties of two  versions  of the criterion  ate demonstrated in numerical  experiments.
A Silicon Axon
Authors: Paul E. Hasler, Chris Diorio, Carver Mead Bradley A. Minch
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  present  a  silicon  model of an  axon  which  shows  promise  as  a  building  block for  pulse-based  neural  computations involving cor(cid:173) relations of pulses  across  both space  and time.  The  circuit shares  a  number  of features  with  its  biological  counterpart  including  an  excitation  threshold,  a  brief refractory  period  after  pulse  comple(cid:173) tion, pulse amplitude restoration, and pulse width restoration.  We  provide  a simple explanation of circuit operation and present  data  from  a  chip fabricated  in  a standard  2Jlm  CMOS  process  through  the MOS  Implementation Service  (MOSIS).  We  emphasize the ne(cid:173) cessity of the restoration of the width of the pulse in time for stable  propagation in  axons.
Plasticity-Mediated Competitive Learning
Authors: Terrence J. Sejnowski Nicol N. Schraudolph
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Differentiation between the nodes of a competitive learning net(cid:173) work is conventionally achieved through competition on the ba(cid:173) sis of neural activity.  Simple inhibitory mechanisms are  limited  to  sparse  representations,  while  decorrelation  and  factorization  schemes that support distributed representations are computation(cid:173) ally unattractive.  By letting neural plasticity mediate the compet(cid:173) itive interaction instead, we obtain diffuse, nonadaptive alterna(cid:173) tives for fully distributed representations.  We  use this technique  to  Simplify and improve our binary information gain optimiza(cid:173) tion algorithm for feature extraction (Schraudolph and Sejnowski,  1993); the same approach could be used to improve other learning  algorithms.
Patterns of damage in neural networks: The effects of lesion area, shape and number
Authors: James A. Reggia Eytan Ruppin
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Current understanding of the effects of damage on neural networks  is  rudimentary, even  though such  understanding  could lead to im(cid:173) portant insights concerning neurological and psychiatric disorders.  Motivated  by  this  consideration,  we  present  a  simple  analytical  framework for  estimating the functional  damage resulting from fo(cid:173) cal  structural  lesions  to  a  neural  network.  The  effects  of focal  le(cid:173) sions of varying area,  shape and number on the retrieval capacities  of a spatially-organized associative memory.  Although our analyti(cid:173) cal results are based on some approximations, they correspond well  with simulation results.  This study sheds light on  some important  features  characterizing  the  clinical  manifestations of multi-infarct  dementia, including the strong  association between  the number of  infarcts and the prevalence of dementia after stroke,  and the 'mul(cid:173) tiplicative' interaction that has been  postulated  to occur  between  Alzheimer's disease  and multi-infarct dementia. 
*Dr.  Reggia  is  also  with  the  Department  of Neurology  and  the Institute  of Advanced 
Computer Studies  at the University  of Maryland. 
36 
Eytan  Ruppin,  James A.  Reggia
A Study of Parallel Perturbative Gradient Descent
Authors: Joshua Alspector D. Lippe
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  have  continued  our  study  of a  parallel  perturbative  learning  method  [Alspector  et al.,  1993]  and implications for  its implemen(cid:173) tation in analog VLSI. Our new results indicate that, in most cases,  a single parallel perturbation (per pattern presentation) of the func(cid:173) tion  parameters  (weights in a  neural network)  is  theoretically  the  best  course.  This  is  not  true,  however,  for  certain  problems  and  may  not  generally  be  true  when  faced  with  issues  of implemen(cid:173) tation  such  as limited  precision.  In  these  cases,  multiple  parallel  perturbations may be best as indicated in our previous results.
A Neural Model of Delusions and Hallucinations in Schizophrenia
Authors: James A. Reggia, David Horn Eytan Ruppin
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  implement and study a computational model of Stevens'  [19921  theory  of the  pathogenesis  of schizophrenia.  This  theory  hypoth(cid:173) esizes  that  the  onset  of schizophrenia  is  associated  with  reactive  synaptic regeneration occurring in brain regions receiving degener(cid:173) ating temporal lobe  projections.  Concentrating on  one  such  area,  the  frontal  cortex,  we  model  a  frontal  module  as  an  associative  memory neural  network  whose  input synapses  represent  incoming  temporal  projections.  We  analyze  how,  in  the  face  of weakened  external input projections, compensatory strengthening of internal  synaptic connections and increased  noise levels can maintain mem(cid:173) ory  capacities  (which  are  generally  preserved  in  schizophrenia) .  However,  These  compensatory  changes  adversely  lead  to  sponta(cid:173) neous,  biased  retrieval  of stored  memories,  which  corresponds  to  the occurrence  of schizophrenic  delusions  and hallucinations with(cid:173) out  any  apparent  external  trigger,  and  for  their  tendency  to  con(cid:173) centrate on just few  central themes.  Our  results explain why  these  symptoms tend  to wane  as  schizophrenia progresses,  and why  de(cid:173) layed  therapeutical intervention leads to a  much slower  response. 
150 
Eytan Ruppin,  James  A.  Reggia,  David Hom
Correlation and Interpolation Networks for Real-time Expression Analysis/Synthesis
Authors: Irfan A. Essa, Alex Pentland Trevor Darrell
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  describe  a  framework  for  real-time  tracking  of facial  expressions  that  uses  neurally-inspired  correlation  and  interpolation  methods.  A  distributed view-based representation is used to characterize facial state,  and is  computed using a  replicated correlation network.  The ensemble  response of the set of view correlation scores is input to a network based  interpolation method, which maps perceptual state to motor control states  for  a  simulated  3-D  face  model.  Activation levels  of the  motor  state  correspond to muscle activations in an  anatomically derived model.  By  integrating fast and robust 2-D processing with 3-D models, we obtain a  system that is able to quickly track and interpret complex facial motions  in real-time.
Neural Network Ensembles, Cross Validation, and Active Learning
Authors: Jesper Vedelsby Anders Krogh
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Learning  of continuous  valued  functions  using  neural  network  en(cid:173) sembles  (committees) can give  improved accuracy,  reliable estima(cid:173) tion of the generalization error,  and active learning.  The  ambiguity  is  defined as the variation of the output of ensemble members aver(cid:173) aged  over  unlabeled  data, so  it quantifies the  disagreement  among  the networks.  It is  discussed  how  to use the ambiguity in combina(cid:173) tion with cross-validation to give a reliable estimate of the ensemble  generalization error, and how this type of ensemble cross-validation  can sometimes improve performance.  It is  shown  how  to estimate  the optimal weights of the ensemble members using unlabeled  data.  By a generalization of query  by  committee, it is finally shown how  the ambiguity can be used to select new training data to be labeled  in an active learning scheme.
Extracting Rules from Artificial Neural Networks with Distributed Representations
Authors: Sebastian Thrun
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Although artificial neural networks have been applied in a variety of real-world scenarios  with remarkable success,  they  have often been criticized for exhibiting a low degree of  human comprehensibility.  Techniques that compile compact sets of symbolic rules out  of artificial  neural  networks  offer  a  promising perspective  to  overcome  this  obvious  deficiency of neural network representations.  This paper  presents  an  approach  to  the  extraction of if-then rules  from  artificial  neu(cid:173) Its  key  mechanism  is  validity  interval  analysis,  which  is  a  generic  ral  networks.  tool  for  extracting  symbolic  knowledge  by  propagating rule-like knowledge through  Backpropagation-style neural  networks.  Empirical studies in a robot arm domain illus(cid:173) trate the appropriateness of the proposed method for extracting rules from networks with  real-valued and distributed representations.
A model of the hippocampus combining self-organization and associative memory function
Authors: Eric Schnell, Joshua Berke, Edi Barkai Michael E. Hasselmo
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: A model of the hippocampus is presented which forms rapid self -orga(cid:173) nized representations of input arriving via the perforant path, performs  recall of previous associations in region CA3, and performs comparison  of this recall with afferent input in region CA 1.  This comparison drives  feedback regulation of cholinergic modulation to set appropriate  dynamics for learning of new representations in region CA3 and CA 1.  The network responds to novel patterns with increased cholinergic mod(cid:173) ulation, allowing storage of new self-organized representations, but  responds to familiar patterns with a decrease in acetylcholine,  allowing  recall based on previous representations.  This requires selectivity of the  cholinergic suppression of synaptic transmission in stratum radiatum of  regions CA3 and CAl, which has been demonstrated experimentally.
Glove-TalkII: Mapping Hand Gestures to Speech Using Neural Networks
Authors: Geoffrey E. Hinton Sidney Fels
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Glove-TaikII is  a  system which  translates  hand gestures  to speech  through  an  adaptive interface.  Hand  gestures  are  mapped  contin(cid:173) uously  to  10  control  parameters  of a  parallel formant  speech  syn(cid:173) thesizer.  The mapping allows the hand to act  as  an  artificial  vocal  tract  that  produces  speech  in  real  time.  This  gives  an  unlimited  vocabulary  in addition to direct  control of fundamental  frequency  and  volume.  Currently,  the  best  version  of Glove-TalkII  uses  sev(cid:173) eral  input  devices  (including  a  CyberGlove,  a  ContactGlove,  a  3- space  tracker,  and a foot-pedal),  a  parallel formant speech  synthe(cid:173) sizer  and  3 neural  networks.  The gesture-to-speech  task  is  divided  into  vowel  and  consonant  production  by  using  a  gating  network  to weight  the outputs of a  vowel  and  a  consonant  neural  network.  The  gating  network  and  the  consonant  network  are  trained  with  examples  from  the  user.  The  vowel  network  implements  a  fixed,  user-defined  relationship  between  hand-position  and  vowel  sound  and does  not require any training examples from the user.  Volume,  fundamental  frequency  and  stop  consonants  are  produced  with  a  fixed  mapping from  the  input devices.  One subject  has  trained  to  speak  intelligibly with Glove-TalkII. He  speaks slowly with speech  quality  similar  to a  text-to-speech  synthesizer  but  with  far  more  natural-sounding pitch  variations. 
844 
S.  Sidney  Fe Is,  Geoffrey  Hinton
Learning in large linear perceptrons and why the thermodynamic limit is relevant to the real world
Authors: Peter Sollich
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  present  a  new  method  for  obtaining  the  response  function  9  and  its  average  G  from  which  most  of the  properties  of learning  and  generalization  in  linear  perceptrons  can  be  derived.  We  first  rederive the  known results for  the 'thermodynamic limit' of infinite  perceptron  size  N  and  show  explicitly  that  9  is  self-averaging  in  this limit.  We  then discuss extensions of our  method to  more gen(cid:173) eral  learning scenarios with  anisotropic teacher  space priors,  input  distributions, and weight  decay  terms.  Finally, we  use our method  to calculate the  finite  N  corrections of order  1/ N  to G  and discuss  the corresponding  finite  size  effects  on generalization  and learning  dynamics.  An  important  spin-off  is  the  observation  that  results  obtained  in  the  thermodynamic limit are often  directly  relevant  to  systems of fairly  modest, 'real-world' sizes.
Learning Saccadic Eye Movements Using Multiscale Spatial Filters
Authors: Dana H. Ballard Rajesh P. N. Rao
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  describe  a  framework  for  learning  saccadic  eye  movements  using  a  photometric representation of target points  in natural scenes.  The rep(cid:173) resentation takes the form of a high-dimensional vector comprised of the  responses  of spatial filters  at different  orientations and  scales.  We  first  demonstrate the use  of this  response  vector in  the task of locating pre(cid:173) viously foveated  points in a  scene and subsequently use  this property in  a  multisaccade strategy to derive  an adaptive motor map for  delivering  accurate saccades.
A Charge-Based CMOS Parallel Analog Vector Quantizer
Authors: Volnei Pedroni Gert Cauwenberghs
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We present an analog VLSI chip for parallel analog vector quantiza(cid:173) tion. The MOSIS 2.0 J..Lm double-poly CMOS Tiny chip contains an  array of 16 x 16 charge-based distance estimation cells, implementing a  mean absolute difference (MAD) metric operating on a 16-input analog  vector field and 16 analog template vectors. The distance cell includ(cid:173) ing dynamic template storage measures 60 x 78 J..Lm2 • Additionally,  the chip features a winner-take-all (WTA) output circuit of linear com(cid:173) plexity, with global positive feedback for fast and decisive settling of a  single winner output. Experimental results on the complete 16 x 16 VQ  system demonstrate correct operation with 34 dB analog input dynamic  range and 3 J..Lsec cycle time at 0.7 mW power dissipation.
Boosting the Performance of RBF Networks with Dynamic Decay Adjustment
Authors: Jay Diamond Michael R. Berthold
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Radial  Basis  Function  (RBF)  Networks,  also  known  as  networks  of locally-tuned processing units  (see  [6])  are well  known for  their  ease  of use.  Most  algorithms  used  to  train  these  types  of  net(cid:173) works,  however,  require a  fixed  architecture, in  which  the number  of units  in  the  hidden  layer  must  be  determined  before  training  starts.  The RCE training algorithm, introduced by  Reilly,  Cooper  and Elbaum  (see  [8]),  and its  probabilistic extension,  the P-RCE  algorithm,  take advantage of a  growing structure in  which  hidden  units are only introduced when  necessary.  The nature of these al(cid:173) gorithms allows training to reach stability much faster  than is  the  case  for  gradient-descent  based  methods.  Unfortunately  P-RCE  networks do not adjust the standard deviation of their prototypes  individually, using only one global value for  this parameter.  This  paper introduces the Dynamic Decay Adjustment  (DDA)  al(cid:173) gorithm  which  utilizes  the  constructive  nature  of the  P-RCE  al(cid:173) gorithm together with independent adaptation of each prototype's  decay factor.  In addition, this radial adjustment is  class dependent  and  distinguishes  between  different  neighbours.  It is  shown  that  networks  trained  with  the  presented  algorithm  perform  substan(cid:173) tially better than common RBF networks. 
522 
Michael  R.  Berthold,  Jay  Diamond
An Alternative Model for Mixtures of Experts
Authors: Michael I. Jordan, Geoffrey E. Hinton Lei Xu
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We propose an alternative model for mixtures of experts which uses  a  different  parametric form  for  the gating network.  The  modified  model is  trained by the EM  algorithm.  In comparison with earlier  models-trained by either EM  or gradient ascent-there is no need  to  select  a  learning  stepsize.  We  report  simulation experiments  which  show  that  the  new  architecture  yields  faster  convergence.  We  also  apply the new  model to two  problem domains:  piecewise  nonlinear function approximation and the combination of multiple  previously  trained classifiers.
Catastrophic Interference in Human Motor Learning
Authors: Reza Shadmehr, Emanuel Todorov Tom Brashers-Krug
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: Biological sensorimotor systems are not static maps that transform  input  (sensory  information)  into  output  (motor  behavior).  Evi(cid:173) dence  from  many  lines  of research  suggests  that  their  representa(cid:173) tions are plastic, experience-dependent entities.  While this plastic(cid:173) ity is  essential for  flexible  behavior, it presents  the nervous system  with difficult organizational challenges.  If the sensorimotor system  adapts itself to perform well  under one set of circumstances,  will it  then perform poorly when placed  in an environment with different  demands  (negative  transfer)?  Will  a  later  experience-dependent  change  undo  the benefits  of previous  learning  (catastrophic  inter(cid:173) ference)?  We  explore  the first  question  in  a separate paper in  this  volume  (Shadmehr et  al.  1995).  Here  we  present  psychophysical  and computational results that explore the question of catastrophic  interference  in the context of a  dynamic motor learning task.  Un(cid:173) der  some conditions, subjects show  evidence  of catastrophic  inter(cid:173) ference.  Under  other  conditions,  however,  subjects  appear  to  be  immune to  its  effects.  These  results  suggest  that  motor  learning  can  undergo  a  process  of consolidation.  Modular neural  networks  are  well  suited  for  the  demands of learning multiple input/output  mappings.  By  incorporating the notion of fast- and slow-changing  connections  into  a  modular architecture,  we  were  able  to  account  for  the psychophysical results. 
20 
Tom  Brashers-Krug,  Reza Shadmelzr,  Emanuel Todorov
On the Computational Complexity of Networks of Spiking Neurons
Authors: Wolfgang Maass
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We investigate the computational power of a formal model for net(cid:173) works of spiking neurons,  both for  the assumption of an unlimited  timing precision, and for  the case of a limited timing precision.  We  also prove upper and lower bounds for the number of examples that  are  needed  to train such networks. 
1 
Introduction and  Basic Definitions 
There  exists substantial evidence  that timing phenomena such  as  temporal  differ(cid:173) ences  between  spikes  and  frequencies  of oscillating  subsystems  are  integral  parts  of various  information processing  mechanisms  in  biological  neural  systems  (for  a  survey  and references  see  e.g.  Abeles,  1991; Churchland and Sejnowski,  1992; Aert(cid:173) sen,  1993).  Furthermore simulations of a  variety  of specific  mathematical models  for  networks of spiking neurons have shown  that temporal coding offers interesting  possibilities for  solving  classical  benchmark-problems such  as  associative  memory,  binding, and pattern segmentation (for an overview see Gerstner et al., 1992).  Some  aspects  of these  models have  also  been studied  analytically, but  almost nothing is  known about their computational complexity (see Judd and Aihara, 1993, for some  first  results  in  this  direction).  In  this article  we  introduce  a  simple formal model  SNN  for  networks  of spiking neurons  that allows us  to model the  most  important  timing phenomena of neural nets (including synaptic modulation), and we prove up(cid:173) per and lower bounds for its computational power and learning complexity.  Further 
184
New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence
Authors: Chien-Ping Lu, Anand Rangarajan, Suguna Pappu, Eric Mjolsness Steven Gold
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: A  fundamental  open  problem  in  computer  vision-determining  pose  and  correspondence  between  two  sets  of  points  in  space(cid:173) is  solved with a novel, robust  and easily implementable algorithm.  The  technique  works  on  noisy  point sets  that  may be  of unequal  sizes  and  may  differ  by  non-rigid  transformations.  A  2D  varia(cid:173) tion  calculates  the  pose  between  point  sets  related  by  an  affine  transformation-translation, rotation, scale and shear.  A 3D to 3D  variation calculates translation and rotation.  An objective describ(cid:173) ing  the  problem is  derived  from  Mean field  theory.  The objective  is  minimized with clocked  (EM-like)  dynamics.  Experiments with  both  handwritten  and  synthetic  data provide  empirical  evidence  for  the method.
PCA-Pyramids for Image Compression
Authors: Kurt Hornik Horst Bischof
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: This paper presents a new  method for image compression by neural  networks.  First, we  show  that we  can use neural networks in  a  py(cid:173) ramidal framework, yielding the so-called PCA pyramids.  Then we  present an image compression method based on the PCA pyramid,  which  is  similar  to  the  Laplace  pyramid  and  wavelet  transform.  Some experimental results  with  real images are reported.  Finally,  we  present  a  method  to  combine  the  quantization  step  with  the  learning of the PCA pyramid.
Morphogenesis of the Lateral Geniculate Nucleus: How Singularities Affect Global Structure
Authors: Klaus Schulten, Joseph G. Malpeli Svilen Tzonev
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The macaque lateral geniculate nucleus (LGN) exhibits an intricate  lamination pattern, which changes midway through the nucleus at a  point coincident with small gaps due to the blind spot in the retina.  We  present a  three-dimensional model of morphogenesis in  which  local cell interactions cause a  wave  of development of neuronal re(cid:173) ceptive fields  to propagate through the nucleus  and establish two  distinct lamination patterns.  We  examine the interactions between  the wave  and the localized singularities due  to the gaps,  and find  that the gaps induce the change in lamination pattern.  We  explore  critical factors which  determine general LGN organization.
Reinforcement Learning Predicts the Site of Plasticity for Auditory Remapping in the Barn Owl
Authors: Cedric Deffayet, Terrence J. Sejnowski Alexandre Pouget
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The auditory system of the barn owl  contains several spatial maps.  In young barn owls raised with optical prisms over their eyes,  these  auditory  maps are  shifted  to stay  in  register  with the  visual  map,  suggesting  that  the  visual  input  imposes  a  frame  of reference  on  the  auditory  maps.  However,  the  optic  tectum,  the  first  site  of  convergence  of visual  with  auditory information, is  not the  site  of  plasticity for  the  shift  of the  auditory  maps;  the  plasticity occurs  instead  in  the  inferior  colliculus, which  contains  an  auditory  map  and projects into the optic tectum.  We explored a model of the owl  remapping in which  a global reinforcement signal whose  delivery  is  controlled  by  visual foveation.  A hebb  learning rule gated by  rein(cid:173) forcement  learned  to appropriately adjust auditory  maps.  In addi(cid:173) tion,  reinforcement  learning  preferentially  adjusted  the  weights  in  the inferior colliculus,  as in the owl brain, even  though the weights  were  allowed  to  change  throughout  the auditory system.  This ob(cid:173) servation  raises  the  possibility  that  the  site  of learning  does  not  have  to  be  genetically specified,  but  could  be  determined  by  how  the learning procedure  interacts with the network  architecture. 
126 
Alexandre Pouget,  Cedric Deffayet,  Te"ence J.  Sejnowski 
c:::======:::::»  •
A Lagrangian Formulation For Optical Backpropagation Training In Kerr-Type Optical Networks
Authors: Steven R. Skinner, Alvaro A. Cruz-Cabrara, Elizabeth C. Behrman James Edward Steck
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: A training method based on a form of continuous spatially distributed  optical error back-propagation is presented for an all optical network  composed of nondiscrete neurons and weighted interconnections. The all  optical network is feed-forward and is composed of thin layers of a Kerr(cid:173) type self focusing/defocusing nonlinear optical material. The training  method is derived from a Lagrangian formulation of the constrained  minimization of the network error at the output. This leads to a  formulation that describes training as a calculation of the distributed error  of the optical signal at the output which is then reflected back through the  device to assign a spatially distributed error to the internal layers. This  error is then used to modify the internal weighting values. Results from  several computer simulations of the training are presented, and a simple  optical table demonstration of the network is discussed. 
772 
Elizabeth C. Behrman 
1 KERR TYPE MATERIALS 
Kerr-type optical networks utilize thin layers of Kerr-type nonlinear materials, in which the  index of refraction can vary within the material and depends on the amount of light striking  the material at a given location. The material index of refraction can be described by:  n(x)=no+nzI(x), where 110 is the linear index of refraction, ~ is the nonlinear coefficient, and  I(x) is the irradiance of a applied optical field as a function of position x across the material  layer (Armstrong, 1962). This means that a beam of light (a signal beam carrying  information perhaps) passing through a layer of Kerr-type material can be steered or  controlled by another beam of light which applies a spatially varying pattern of intensity  onto the Kerr-type material. Steering of light with a glass lens (having constance index of  refraction) is done by varying the thickness of the lens (the amount of material present) as  a function of position. Thus the Kerr effect can be loosely thought of as a glass lens whose  geometry and therefore focusing ability could be dynamically controlled as a function of  position across the lens. Steering in the Kerr material is accomplished by a gradient or  change in the material index of refraction which is created by a gradient in applied light  intensity. This is illustrated by the simple experiment in Figure 1 where a small weak probe  beam is steered away from a straight path by the intensity gradient of a more powerful pump  beam.
Nonlinear Image Interpolation using Manifold Learning
Authors: Stephen M. Omohundro Christoph Bregler
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: The problem of interpolating between specified images in an image  sequence  is  a  simple,  but  important  task  in  model-based  vision.  We  describe  an  approach based  on  the abstract  task of  "manifold  learning"  and present  results  on  both synthetic and real image se(cid:173) quences.  This  problem  arose  in  the  development  of a  combined  lip-reading and speech  recognition system.
A Growing Neural Gas Network Learns Topologies
Authors: Bernd Fritzke
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: An incremental network model is  introduced which is  able to learn  the important topological relations in a given set of input vectors by  means of a  simple Hebb-like learning rule.  In contrast to previous  approaches like the "neural gas"  method of Martinetz and Schulten  (1991,  1994), this model has no parameters which change over time  and is able to continue learning, adding units and connections, until  a  performance criterion has been met.  Applications of the model  include vector quantization, clustering, and interpolation.
Learning to Play the Game of Chess
Authors: Sebastian Thrun
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: This paper presents NeuroChess,  a program which learns to play chess from the final  outcome of games.  NeuroChess learns chess  board  evaluation functions,  represented  by artificial neural  networks.  It integrates inductive neural  network learning, temporal  differencing, and a variant of explanation-based learning.  Performance results illustrate  some of the strengths and weaknesses of this approach.
Interior Point Implementations of Alternating Minimization Training
Authors: Peter T. Szymanski Michael Lemmon
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: This paper presents an alternating minimization (AM) algorithm  used in the training of radial basis function and linear regressor  networks. The algorithm is a modification of a small-step interior  point method used in solving primal linear programs. The algo(cid:173) rithm has a convergence rate of O( fo,L) iterations where n is a  measure of the network size and L is a measure of the resulting  solution's accuracy. Two results are presented that specify how  aggressively the two steps of the AM may be pursued to ensure  convergence of each step of the alternating minimization.
A Convolutional Neural Network Hand Tracker
Authors: John C. Platt Steven J. Nowlan
Published in: Advances in Neural Information Processing Systems 7  (NIPS 1994) (Main Conference Track), 1994

Abstract: We  describe  a system that can track  a hand in a sequence of video  frames  and recognize hand gestures  in  a user-independent  manner.  The system  locates  the  hand in  each  video  frame  and  determines  if the hand is open or closed.  The tracking system is  able to track  the  hand  to  within  ±10  pixels  of its  correct  location  in  99.7%  of  the frames  from a  test set  containing video sequences  from  18  dif(cid:173) ferent  individuals captured in 18  different  room environments.  The  gesture  recognition network correctly  determines if the hand being  tracked  is  open  or  closed  in  99.1 % of the  frames  in this  test  set .  The system has been designed to operate in real time with existing  hardware.