mtybilly/Instruct2DS huggingface dataset:
Dual Mechanisms for Neural Binding and Segmentation
Authors: Paul Sajda, Leif H. Finkel
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We  propose  that  the  binding  and  segmentation  of visual  features  is  mediated  by  two  complementary  mechanisms;  a  low  resolu(cid:173) tion,  spatial-based,  resource-free  process  and  a  high  resolution,  temporal-based,  resource-limited  process.  In the visual  cortex,  the  former  depends  upon  the orderly  topographic organization in  stri(cid:173) ate  and  extrastriate  areas  while  the  latter  may  be  related  to  ob(cid:173) served  temporal  relationships  between  neuronal  activities .  Com(cid:173) puter  simulations  illustrate  the  role  the  two  mechanisms  play  in  figure/ ground discrimination,  depth-from-occlusion,  and the vivid(cid:173) ness  of perceptual completion. 
1  COMPLEMENTARY BINDING  MECHANISMS 
The  "binding  problem"  is  a  classic  problem  in  computational  neuroscience  which  considers how  neuronal activities are grouped to create mental representations.  For  the  case  of visual  processing,  the  binding  of neuronal  activities  requires  a  mecha(cid:173) nism  for  selectively  grouping fragmented  visual  features  in  order  to  construct  the  coherent representations (i.e.  objects)  which we  perceive.  In this paper we  argue for  the  existence  of two  complementary  mechanisms  for  neural  binding,  and  we  show  how  such mechanisms  may operate  in  the  constructiO:l  of intermediate-level visual  representations.
The Statistical Mechanics of k-Satisfaction
Authors: Scott Kirkpatrick, Géza Györgyi, Naftali Tishby, Lidror Troyansky
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: The  satisfiability of random  CNF  formulae  with  precisely  k  vari(cid:173) ables per clause ("k-SAT") is a popular testbed for the performance  of search  algorithms.  Formulae have  M  clauses  from  N  variables,  randomly  negated,  keeping  the  ratio  a  =  M / N  fixed .  For  k  =  2,  this  model  has  been  proven  to  have  a  sharp  threshold  at  a  = 1  between  formulae which  are almost aways satisfiable and formulae  which  are  almost never  satisfiable  as  N  --jo  00 .  Computer  experi(cid:173) ments for  k  = 2,  3,  4,  5  and  6,  (carried  out  in  collaboration  with  B.  Selman  of ATT Bell  Labs). show  similar threshold  behavior for  each  value  of k.  Finite-size  scaling,  a  theory  of the  critical  point  phenomena used  in statistical physics, is shown  to characterize the  size  dependence  near  the  threshold.  Annealed  and  replica-based  mean field  theories  give a  good account  of the results. 
"Permanent  address:  IBM  TJ Watson  Research  Center,  Yorktown  Heights,  NY 10598  USA.  (kirk@watson.ibm.com)  Portions  of  this  work  were  done  while  visiting  the  Salk  Institute,  with support from  the McDonnell-Pew  Foundation.
Lipreading by neural networks: Visual preprocessing, learning, and sensory integration
Authors: Gregory J. Wolff, K. Venkatesh Prasad, David G. Stork, Marcus Hennecke
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We have developed visual preprocessing algorithms for extracting  phonologically relevant features from the grayscale video image of  a speaker, to provide speaker-independent inputs for an automat(cid:173) ic lipreading ("speechreading") system. Visual features such as  mouth open/closed, tongue visible/not-visible, teeth visible/not(cid:173) visible, and several shape descriptors of the mouth and its motion  are all rapidly computable in a manner quite insensitive to lighting  conditions. We formed a hybrid speechreading system consisting  of two time delay neural networks (video and acoustic) and inte(cid:173) grated their responses by means of independent opinion pooling  - the Bayesian optimal method given conditional independence,  which seems to hold for our data. This hybrid system had an er(cid:173) ror rate 25% lower than that of the acoustic subsystem alone on a  five-utterance speaker-independent task, indicating that video can  be used to improve speech recognition.
Training Neural Networks with Deficient Data
Authors: Volker Tresp, Subutai Ahmad, Ralph Neuneier
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We  analyze how  data with uncertain or missing input features  can  be  incorporated  into  the  training  of a  neural  network.  The  gen(cid:173) eral  solution requires  a  weighted  integration over  the unknown  or  uncertain  input although  computationally cheaper  closed-form so(cid:173) lutions  can  be  found  for  certain  Gaussian  Basis  Function  (GBF)  networks.  We also discuss  cases  in which heuristical solutions such  as substituting the mean of an unknown input can  be harmful.
When will a Genetic Algorithm Outperform Hill Climbing
Authors: Melanie Mitchell, John H. Holland, Stephanie Forrest
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We analyze a simple hill-climbing algorithm (RMHC) that was pre(cid:173) viously shown to outperform a genetic algorithm (GA) on a simple  "Royal  Road"  function.  We  then  analyze  an  "idealized"  genetic  algorithm (IGA)  that is  significantly faster  than RMHC  and that  gives  a lower  bound for  GA speed.  We  identify the features  of the  IGA that give rise  to this speedup,  and discuss  how  these features  can be incorporated into a real GA.
Unsupervised Parallel Feature Extraction from First Principles
Authors: Mats Österberg, Reiner Lenz
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We describe a number of learning rules that can be used to train un(cid:173) supervised  parallel feature  extraction systems.  The learning  rules  are  derived  using  gradient  ascent  of a  quality  function.  We  con(cid:173) sider  a  number of quality functions  that are  rational functions  of  higher  order  moments of the  extracted  feature  values.  We  show  that  one  system  learns  the  principle  components  of  the  correla(cid:173) tion matrix.  Principal component analysis systems are usually not  optimal feature  extractors  for  classification.  Therefore  we  design  quality functions which produce feature vectors that support unsu(cid:173) pervised  classification.  The properties of the  different systems are  compared  with  the  help  of different  artificially  designed  datasets  and a  database consisting of all Munsell color spectra.
Classification of Electroencephalogram using Artificial Neural Networks
Authors: A C Tsoi, D S C So, A Sergejew
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: In this paper, we will consider the problem of classifying electroencephalo(cid:173) gram (EEG) signals of normal subjects, and subjects suffering from psychi(cid:173) atric disorder, e.g., obsessive compulsive disorder, schizophrenia, using a  class of artificial neural networks, viz., multi-layer perceptron. It is shown  that the multilayer perceptron is capable of classifying unseen test EEG  signals to a high degree of accuracy.
Comparison Training for a Rescheduling Problem in Neural Networks
Authors: Didier Keymeulen, Martine de Gerlache
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Airline  companies  usually  schedule  their flights  and  crews  well  in  advance  to optimize  their  crew  pools  activities.  Many  events such  as  flight  delays or  the  absence  of a  member require  the  crew  pool  rescheduling team to change the initial schedule  (rescheduling).  In  this paper, we  show  that the neural network comparison paradigm  applied  to  the  backgammon  game  by  Tesauro  (Tesauro  and  Se(cid:173) jnowski,  1989)  can  also  be applied  to the rescheduling  problem of  an  aircrew  pool.  Indeed  both  problems  correspond  to  choosing  the best solut.ion  from a set of possible ones without ranking them  (called here best choice problem).  The paper explains from a math(cid:173) ematical point of view  the architecture and the learning strategy of  the backpropagation neural network used for  the best  choice  prob(cid:173) lem.  We  also  show  how  the learning  phase  of the  network  can  be  accelerated.  Finally  we  apply  the  neural  network  model  to some  real rescheduling  problems for  the  Belgian  Airline  (Sabena).
Bounds on the complexity of recurrent neural network implementations of finite state machines
Authors: Bill G. Horne, Don R. Hush
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: In this paper the efficiency of recurrent neural network implementa(cid:173) tions of m-state finite state machines will be explored. Specifically,  it will be shown that the node complexity for the unrestricted case  can be bounded above by 0 ( fo) . It will also be shown that the  node complexity is 0 (y'm log m) when the weights and thresholds  are restricted to the set {-I, I}, and 0 (m) when the fan-in is re(cid:173) stricted to two. Matching lower bounds will be provided for each  of these upper bounds assuming that the state of the FSM can be  encoded in a subset of the nodes of size rlog m 1.
Optimal Brain Surgeon: Extensions and performance comparisons
Authors: Babak Hassibi, David G. Stork, Gregory Wolff
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: a  second-order 
We  extend  Optimal  Brain  Surgeon  (OBS)  - to  allow  for  general  error  mea(cid:173) method  for  pruning  networks  - sures, and explore a reduced computational and storage implemen(cid:173) tation  via  a  dominant  eigenspace  decomposition.  Simulations  on  nonlinear,  noisy  pattern  classification  problems  reveal  that  OBS  does  lead  to  improved  generalization,  and  performs  favorably  in  comparison with Optimal Brain Damage (OBD).  We  find  that the  required  retraining  steps  in  OBD  may  lead  to inferior  generaliza(cid:173) tion, a result that can be interpreted as due to injecting noise back  into the system.  A common technique is to stop training of a large  network at the minimum validation error.  We  found  that the test  error  could  be  reduced  even  further  by  means  of  OBS  (but  not  OBD)  pruning.  Our  results justify the t  ~ 0  approximation  used  in  OBS  and  indicate  why  retraining  in  a  highly  pruned  network  may lead to inferior  performance.
Mixtures of Controllers for Jump Linear and Non-Linear Plants
Authors: Timothy W. Cacciatore, Steven J. Nowlan
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We describe an extension to the Mixture of Experts architecture for  modelling and  controlling dynamical systems which  exhibit  multi(cid:173) ple modes of behavior.  This extension is based on a Markov process  model,  and suggests  a  recurrent  network for  gating a  set  of linear  or  non-linear controllers.  The new  architecture  is  demonstrated  to  be  capable  of learning  effective  control  strategies  for  jump  linear  and  non-linear plants with multiple modes of behavior.
A Hybrid Radial Basis Function Neurocomputer and Its Applications
Authors: Steven S. Watkins, Paul M. Chau, Raoul Tawel, Bjorn Lambrigtsen, Mark Plutowski
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: A neurocomputer was  implemented using radial basis functions  and a  combination  of  analog  and  digital  VLSI  circuits.  The  hybrid  system  uses  custom  analog  circuits  for  the  input  layer  and  a  digital  signal  processing board for the hidden and output layers. The system combines  the  advantages of both analog and digital circuits. featuring low power  consumption while minimizing overall system error. The analog circuits  have been fabricated and tested, the  system has been built,  and several  applications  have  been  executed  on  the  system.  One  application  provides significantly better results for  a remote sensing problem than  have been previously obtained using conventional methods.
Resolving motion ambiguities
Authors: K. I. Diamantaras, D. Geiger
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We address the problem of optical flow reconstruction and in par(cid:173) ticular the problem of resolving ambiguities near edges. They oc(cid:173) cur due to (i) the aperture problem and (ii) the occlusion problem,  where pixels on both sides of an intensity edge are assigned the same  velocity estimates (and confidence). However, these measurements  are correct for just one side of the edge (the non occluded one).  Our approach is to introduce an uncertamty field with respect to  the estimates and confidence measures. We note that the confi(cid:173) dence measures are large at intensity edges and larger at the con(cid:173) vex sides of the edges, i.e. inside corners, than at the concave side.  We resolve the ambiguities through local interactions via coupled  Markov random fields (MRF) . The result is the detection of motion  for regions of images with large global convexity.
Developing Population Codes by Minimizing Description Length
Authors: Richard S. Zemel, Geoffrey E. Hinton
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: The Minimum Description Length principle (MDL) can be used to  train the hidden units of a neural network to extract a representa(cid:173) tion that is cheap to describe but nonetheless allows the input to  be reconstructed accurately. We show how MDL can be used to  develop highly redundant population codes. Each hidden unit has  a location in a low-dimensional implicit space. If the hidden unit  activities form a bump of a standard shape in this space, they can  be cheaply encoded by the center ofthis bump. So the weights from  the input units to the hidden units in an autoencoder are trained  to make the activities form a standard bump. The coordinates of  the hidden units in the implicit space are also learned, thus allow(cid:173) ing flexibility, as the network develops a discontinuous topography  when presented with different input classes. Population-coding in  a space other than the input enables a network to extract nonlinear  higher-order properties of the inputs. 
Most existing unsupervised learning algorithms can be understood using the Min(cid:173) imum Description Length (MDL) principle (Rissanen, 1989). Given an ensemble  of input vectors, the aim of the learning algorithm is to find a method of coding  each input vector that minimizes the total cost, in bits, of communicating the input  vectors to a receiver. There are three terms in the total description length: 
• The code-cost is the number of bits required to communicate the code 
that the algorithm assigns to each input vector.
Neural Network Definitions of Highly Predictable Protein Secondary Structure Classes
Authors: Alan Lapedes, Evan Steeg, Robert Farber
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We use two co-evolving neural networks to determine new classes  of protein secondary structure which are significantly more pre(cid:173) dictable from local amino sequence than the conventional secondary  structure classification. Accurate prediction of the conventional  secondary structure classes: alpha helix, beta strand, and coil, from  primary sequence has long been an important problem in compu(cid:173) tational molecular biology. Neural networks have been a popular  method to attempt to predict these conventional secondary struc(cid:173) ture classes. Accuracy has been disappointingly low. The algo(cid:173) rithm presented here uses neural networks to similtaneously exam(cid:173) ine both sequence and structure data, and to evolve new classes  of secondary structure that can be predicted from sequence with  significantly higher accuracy than the conventional classes. These  new classes have both similarities to, and differences with the con(cid:173) ventional alpha helix, beta strand and coil.
Coupled Dynamics of Fast Neurons and Slow Interactions
Authors: A.C.C. Coolen, R. W. Penney, D. Sherrington
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: A simple model of coupled dynamics of fast neurons and slow inter(cid:173) actions, modelling self-organization in recurrent neural networks,  leads naturally to an effective statistical mechanics characterized  by a partition function which is an average over a replicated system.  This is reminiscent of the replica trick used to study spin-glasses,  but with the difference that the number of replicas has a physi(cid:173) cal meaning as the ratio of two temperatures and can be varied  throughout the whole range of real values. The model has inter(cid:173) esting phase consequences as a function of varying this ratio and  external stimuli, and can be extended to a range of other models. 
1 A SIMPLE MODEL WITH FAST DYNAMIC 
NEURONS AND SLOW DYNAMIC INTERACTIONS 
As the basic archetypal model we consider a system of Ising spin neurons (J'i E  {-I, I}, i E {I, ... , N}, interacting via continuous-valued symmetric interactions,  Iij, which themselves evolve in response to the states of the neurons. The neurons  are taken to have a stochastic field-alignment dynamics which is fast compared with  the evolution rate of the interactions hj, such that on the time-scale of Iii-dynamics  the neurons are effectively in equilibrium according to a Boltzmann distribution,
Connectionist Modeling and Parallel Architectures
Authors: Joachim Diederich, Ah Chung Tsoi
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: The  introduction  of  specialized  hardware  platforms  for  connectionist  modeling 
("connectionist  supercomputer")  has  created a number  of research  topics.  Some of 
these issues are controversial, e.g.  the efficient implementation of incremental learn(cid:173)
ing techniques, the need for the dynamic reconfiguration of networks and possible 
programming environments for these machines.
Structural and Behavioral Evolution of Recurrent Networks
Authors: Gregory M. Saunders, Peter J. Angeline, Jordan B. Pollack
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: This paper introduces GNARL, an evolutionary program which induces  recurrent neural networks that are structurally unconstrained. In contrast  to constructive and destructive algorithms, GNARL employs a popula(cid:173) tion of networks and uses a fitness  function's  unsupervised feedback to  guide  search  through  network  space.  Annealing  is  used  in  generating  both  gaussian  weight  changes  and  structural  modifications.  Applying  GNARL  to  a complex  search and collection task demonstrates  that the  system is capable of inducing networks with complex internal dynamics.
Two-Dimensional Object Localization by Coarse-to-Fine Correlation Matching
Authors: Chien-Ping Lu, Eric Mjolsness
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We  present  a  Mean  Field  Theory  method  for  locating  two(cid:173) dimensional  objects  that  have  undergone  rigid  transformations.  The  resulting  algorithm  is  a  form  of  coarse-to-fine  correlation  matching.  We first  consider  problems of matching synthetic point  data,  and  derive  a  point matching objective function.  A  tractable  line segment matching objective function is  derived by considering  each  line segment  as  a  dense  collection of points,  and approximat(cid:173) ing it by a sum of Gaussians.  The algorithm is tested on real images  from  which line segments are extracted  and  matched.
WATTLE: A Trainable Gain Analogue VLSI Neural Network
Authors: Richard Coggins, Marwan Jabri
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: This paper describes a low power analogue VLSI neural network  called Wattle. Wattle is a 10:6:4 three layer perceptron with multi(cid:173) plying DAC synapses and on chip switched capacitor neurons fabri(cid:173) cated in 1.2um CMOS. The on chip neurons facillitate variable gain  per neuron and lower energy/connection than for previous designs.  The intended application of this chip is Intra Cardiac Electrogram  classification as part of an implantable pacemaker / defibrillator sys(cid:173) tem. Measurements of t.he chip indicate that 10pJ per connection  is achievable as part of an integrated system. Wattle has been suc(cid:173) cessfully trained in loop on parity 4 and ICEG morphology classi(cid:173) fication problems.
Fast Non-Linear Dimension Reduction
Authors: Nanda Kambhatla, Todd K. Leen
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We  present  a  fast  algorithm  for  non-linear  dimension  reduction.  The  algorithm  builds  a  local  linear model  of the data by  merging  PCA  with  clustering  based  on  a  new  distortion  measure.  Exper(cid:173) iments  with  speech  and  image  data indicate  that  the  local  linear  algorithm produces encodings with lower distortion than those built  by five  layer auto-associative networks.  The local linear algorithm  is  also more than an order of magnitude faster  to train.
Observability of Neural Network Behavior
Authors: Max Garzon, Fernanda Botelho
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We prove that except possibly for small exceptional sets, discrete(cid:173) time analog neural nets are globally observable, i.e. all their cor(cid:173) rupted pseudo-orbits on computer simulations actually reflect the  true dynamical behavior of the network. Locally finite discrete  (boolean) neural networks are observable without exception.
Optimal Signalling in Attractor Neural Networks
Authors: Isaac Meilijson, Eytan Ruppin
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: In  [Meilijson  and  Ruppin,  1993]  we  presented  a  methodological  framework  describing  the  two-iteration  performance  of Hopfield(cid:173) like  attractor  neural  networks  with  history-dependent,  Bayesian  dynamics.  We  now  extend  this  analysis in  a number of directions:  input  patterns  applied  to  small subsets  of neurons,  general  con(cid:173) nectivity  architectures  and  more efficient  use  of history.  We  show  that the optimal signal (activation) function has a slanted sigmQidal  shape, and provide an intuitive account of activation functions with  a non-monotone shape.  This function endows the model with some  properties  characteristic of cortical  neurons' firing.
Directional Hearing by the Mauthner System
Authors: Audrey L. Guzik, Robert C. Eaton
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We provide a computational description of the function of the Mau(cid:173) thner  system.  This  is  the  brainstem  circuit  which  initiates  fast(cid:173) start  escapes  in  teleost  fish  in  response  to  sounds.  Our  simula(cid:173) tions,  using  back propagation in  a  realistically  constrained  feedfor(cid:173) ward  network,  have generated hypotheses  which  are directly inter(cid:173) pretable  in  terms  of the  activity  of the  auditory  nerve  fibers,  the  principle  cells  of the  system  and  their  associated  inhibitory  neu(cid:173) rons.
Asynchronous Dynamics of Continuous Time Neural Networks
Authors: Xin Wang, Qingnan Li, Edward K. Blum
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Motivated by mathematical modeling, analog implementation and  distributed simulation of neural networks, we present a definition of  asynchronous  dynamics  of general  CT dynamical systems  defined  by  ordinary  differential equations,  based  on  notions of local  times  and  communication  times.  We  provide  some  preliminary  results  on  globally  asymptotical  convergence  of  asynchronous  dynamics  for  contractive  and  monotone CT  dynamical systems.  When  ap(cid:173) plying  the  results  to  neural  networks,  we  obtain  some  conditions  that ensure  additive-type neural  networks  to be  asynchronizable.
Development of Orientation and Ocular Dominance Columns in Infant Macaques
Authors: Klaus Obermayer, Lynne Kiorpes, Gary G. Blasdel
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Maps of orientation preference and ocular dominance were recorded  optically from the cortices of 5 infant macaque monkeys, ranging in  age from 3.5 to 14 weeks.  In agreement with previous observations,  we  found  that basic features  of orientation and ocular  dominance  maps, as well  as correlations between them, are present  and robust  by  3.5  weeks  of age.  We  did  observe  changes  in  the  strength  of  ocular dominance signals, as  well  as  in the spacing of ocular dom(cid:173) inance bands, both of which  increased steadily between  3.5 and  14  weeks  of age.  The  latter  finding  suggests  that  the  adult  spacing  of ocular dominance bands depends on cortical growth in  neonatal  animals.  Since  we  found  no  corresponding  increase  in  the spacing  of orientation  preferences,  however,  there  is  a  possibility  that  the  orientation preferences  of some cells  change as  the cortical surface  expands.  Since  correlations  between  the  patterns  of orientation  selectivity  and ocular dominance  are  present  at  an  age,  when  the  visual system is still immature, it seems more likely that  their  de(cid:173) velopment may be an innate process and may not require extensive  visual experience.
Temporal Difference Learning of Position Evaluation in the Game of Go
Authors: Nicol N. Schraudolph, Peter Dayan, Terrence J. Sejnowski
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: The  game of Go has a high branching factor that defeats the tree  search  approach  used  in  computer  chess,  and  long-range  spa(cid:173) tiotemporal interactions that make position evaluation extremely  difficult.  Development of conventional Go programs is hampered  by their  knowledge-intensive  nature.  We  demonstrate a  viable  alternative by training networks to evaluate Go positions via tem(cid:173) poral difference (TD)  learning.  Our approach  is  based  on network architectures  that reflect  the  spatial organization of both  input and  reinforcement  signals on  the  Go  board,  and  training  protocols  that  provide  exposure  to  competent (though unlabelled)  play.  These  techniques yield  far  better performance than undifferentiated networks trained by self(cid:173) play alone.  A network with less than 500 weights learned within  3,000 games of 9x9 Go a position evaluation function that enables  a primitive one-ply search to defeat a commercial Go program at  a low playing level.
Recovering a Feed-Forward Net From Its Output
Authors: Charles Fefferman, Scott Markel
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We study feed-forward  nets with arbitrarily many layers,  using the stan(cid:173) dard  sigmoid,  tanh  x.  Aside  from  technicalities,  our  theorems  are:  1.  Complete knowledge of the output of a neural net for  arbitrary inputs  uniquely specifies  the architecture,  weights and thresholds;  and 2.  There  are  only  finitely  many  critical  points  on  the  error  surface  for  a  generic  training problem. 
Neural  nets  were  originally  introduced  as  highly  simplified  models  of the  nervous  system.  Today  they  are  widely  used  in  technology  and  studied  theoretically  by  scientists  from  several  disciplines.  However,  they  remain little understood.  Mathematically, a  (feed-forward)  neural  net  consists  of: 
(1)  A  finite  sequence  of positive integers  (Do, D 1 ,  ... ,  D£);  (2)  A family of real numbers (wJ d  defined for 1 :5 e 5:  L,  1 5:  j  5:  D l ,  1 5:  k :5  Dl-l ; 
and 
(3)  A family of real  numbers  (OJ)  defined  for  15: f 5:  L,  15: j  5:  Dl. 
The sequence  (Do,  D 1 ,  .. "  DL )  is  called the  architecture of the neural net, while the  W]k  are called  weights and  the  OJ  thresholds.  Neural  nets  are used  to compute non-linear  maps from  }R.N  to  }R.M  by  the following  construction.  vVe  begin by fixing a nonlinear function 0-( x)  of one  variable.  Analogy  with  the  nervous  system  suggests  that  we  take  o-(x)  asymptotic  to  constants  as  x  tends  to ±oo;  a  standard  choice,  which  we  adopt  throughout  this  paper,  is  o-(.r)  = 

Alternate address:  Dept.  of Mathematics. Princeton University, Princeton, NJ  08544-1000.
Optimal Stochastic Search and Adaptive Momentum
Authors: Todd K. Leen, Genevieve B. Orr
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Stochastic  optimization  algorithms  typically  use  learning  rate  schedules  that  behave asymptotically  as  J.t(t)  = J.to/t.  The ensem(cid:173) ble dynamics  (Leen and Moody,  1993) for such algorithms provides  an easy path to results  on  mean squared weight  error and asymp(cid:173) totic  normality.  We  apply  this  approach  to  stochastic  gradient  algorithms with momentum.  We  show that  at late times,  learning  is  governed  by  an  effective learning rate  J.tejJ  = J.to/(l  - f3)  where  f3  is  the  momentum  parameter.  We  describe  the  behavior  of the  asymptotic  weight  error  and  give  conditions  on  J.tejJ  that  insure  optimal  convergence speed.  Finally,  we  use  the  results to develop  an adaptive form of momentum that achieves optimal convergence  speed  independent of J.to.
Stability and Observability
Authors: Max H. Garzon, Fernanda Botelho
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: he theme was the effect of perturbations of the defining parameters of a neural net(cid:173)
work due to:  1)  mea"urement" (particularly with analog networks); 2)  di"cretization 
due to a)  digital implementation of analog nets; b)  bounded-precision implementa(cid:173)
tion  of digital networks;  or c)  inaccurate evaluation of the  transfer function(s};  3) 
noise in or incomplete input and/or output of the net or individual cells  (particu(cid:173)
larly with analog networks).
The Role of MT Neuron Receptive Field Surrounds in Computing Object Shape from Velocity Fields
Authors: G. T. Buracas, T. D. Albright
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: The goal of this work was to investigate the role of primate  MT  neurons  in  solving  the  structure  from  motion  (SFM)  problem.  Three  types  of  receptive  field  (RF)  surrounds  found in area MT neurons (K.Tanaka et al.,1986; Allman et  al.,1985) correspond, as our analysis suggests, to the oth,  pt  and 2nd  order fuzzy  space-differential operators.  The large  surround/center  both  differentiation  of  smooth  velocity  fields  and  discontinuity  detection  at  boundaries  of  objects.  The  model  is  in  agreement  with  recent  psychophysical  data  on  surface  interpolation involvement  in  SFM.  We  suggest  that  area  MT  partially  segregates  information  about  object  shape  from  information  about  spatial  relations  necessary  for  navigation and manipulation.
Segmental Neural Net Optimization for Continuous Speech Recognition
Authors: Ying Zhao, Richard Schwartz, John Makhoul, George Zavaliagkos
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Previously, we had developed the concept of a Segmental Neural Net (SNN) for  phonetic modeling in continuous speech recognition (CSR). This kind of neu(cid:173) ral network technology advanced the state-of-the-art of large-vocabulary CSR,  which employs Hidden Marlcov Models (HMM), for the ARPA 1oo0-word Re(cid:173) source Management corpus. More Recently, we started porting the neural net  system to a larger, more challenging corpus - the ARPA 20,Ooo-word Wall Street  Journal (WSJ) corpus. During the porting, we explored the following research  directions to refine the system: i) training context-dependent models with a reg(cid:173) ularization method; ii) training SNN with projection pursuit; and ii) combining  different models into a hybrid system. When tested on both a development set  and an independent test set, the resulting neural net system alone yielded a per(cid:173) fonnance at the level of the HMM system, and the hybrid SNN/HMM system  achieved a consistent 10-15% word error reduction over the HMM system. This  paper describes our hybrid system, with emphasis on the optimization methods  employed.
Adaptive knot Placement for Nonparametric Regression
Authors: Hossein L. Najafi, Vladimir Cherkassky
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We show how an "Elman" network architecture, constructed from  recurrently connected oscillatory associative memory network mod(cid:173) ules, can employ selective "attentional" control of synchronization  to direct the flow of communication and computation within the  architecture to solve a grammatical inference problem.  Previously we have shown how the discrete time "Elman" network  algorithm can be implemented in a network completely described  by continuous ordinary differential equations. The time steps (ma(cid:173) chine cycles) of the system are implemented by rhythmic variation  (clocking) of a bifurcation parameter. In this architecture, oscilla(cid:173) tion amplitude codes the information content or activity of a mod(cid:173) ule (unit), whereas phase and frequency are used to "softwire" the  network. Only synchronized modules communicate by exchang(cid:173) ing amplitude information; the activity of non-resonating modules  contributes incoherent crosstalk noise.  Attentional control is modeled as a special subset of the hidden  modules with ouputs which affect the resonant frequencies of other  hidden modules. They control synchrony among the other mod(cid:173) ules and direct the flow of computation (attention) to effect transi(cid:173) tions between two subgraphs of a thirteen state automaton which  the system emulates to generate a Reber grammar. The internal  crosstalk noise is used to drive the required random transitions of  the automaton.
Supervised learning from incomplete data via an EM approach
Authors: Zoubin Ghahramani, Michael I. Jordan
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Real-world  learning  tasks  may involve  high-dimensional  data sets  with  arbitrary  patterns of missing  data.  In  this  paper  we  present  a  framework  based  on  maximum likelihood  density  estimation for  learning from such  data set.s.  VVe  use  mixture models for  the  den(cid:173) sity  estimates and  make two  distinct  appeals  to  the  Expectation(cid:173) Maximization  (EM)  principle  (Dempster  et  al.,  1977)  in  deriving  a  learning  algorithm-EM is  used  both  for  the  estimation of mix(cid:173) ture  components  and  for  coping  wit.h  missing  dat.a.  The  result(cid:173) ing  algorithm  is  applicable  t.o  a  wide  range  of supervised  as  well  as  unsupervised  learning  problems.  Result.s  from  a  classification  benchmark-t.he iris  data set-are presented.
Correlation Functions in a Large Stochastic Neural Network
Authors: Iris Ginzburg, Haim Sompolinsky
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Most theoretical investigations of large recurrent networks focus on  the properties of the macroscopic order parameters such as popu(cid:173) lation averaged activities or average overlaps with memories. How(cid:173) ever, the statistics of the fluctuations in the local activities may  be an important testing ground for comparison between models  and observed cortical dynamics. We evaluated the neuronal cor(cid:173) relation functions in a stochastic network comprising of excitatory  and inhibitory populations. We show that when the network is in  a stationary state, the cross-correlations are relatively weak, i.e.,  their amplitude relative to that of the auto-correlations are of or(cid:173) der of 1/ N, N being the size of the interacting population. This  holds except in the neighborhoods of bifurcations to nonstationary  states. As a bifurcation point is approached the amplitude of the  cross-correlations grows and becomes of order 1 and the decay time(cid:173) constant diverges. This behavior is analogous to the phenomenon  of critical slowing down in systems at thermal equilibrium near a  critical point. Near a Hopf bifurcation the cross-correlations ex(cid:173) hibit damped oscillations.
Clustering with a Domain-Specific Distance Measure
Authors: Steven Gold, Eric Mjolsness, Anand Rangarajan
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: With  a  point matching distance  measure which  is  invariant under  translation,  rotation  and  permutation,  we  learn  2-D  point-set  ob(cid:173) jects,  by clustering noisy point-set images.  Unlike  traditional clus(cid:173) tering methods which use distance measures that operate on feature  vectors - a representation common to most problem domains - this  object-based  clustering technique employs a  distance measure spe(cid:173) cific  to  a  type  of object  within  a  problem  domain.  Formulating  the clustering problem as  two  nested objective functions,  we derive  optimization  dynamics  similar  to  the  Expectation-Maximization  algorithm used  in mixture models.
Speaker Recognition Using Neural Tree Networks
Authors: Kevin R. Farrell, Richard J. Mammone
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: A new classifier is presented for text-independent speaker recognition. The  new classifier is called the modified neural tree network (MNTN). The  NTN is a hierarchical classifier that combines the properties of decision  trees and feed-forward neural networks. The MNTN differs from the stan(cid:173) dard NTN in that a new learning rule based on discriminant learning  is used, which minimizes the classification error as opposed to a norm  of the approximation error. The MNTN also uses leaf probability mea(cid:173) sures in addition to the class labels. The MNTN is evaluated for several  speaker identification experiments and is compared to multilayer percep(cid:173) trons (MLPs) , decision trees, and vector quantization (VQ) classifiers. The  VQ classifier and MNTN demonstrate comparable performance and per(cid:173) form significantly better than the other classifiers for this task. Addition(cid:173) ally, the MNTN provides a logarithmic saving in retrieval time over that  of the VQ classifier. The MNTN and VQ classifiers are also compared  for several speaker verification experiments where the MNTN is found to  outperform the VQ classifier.
Tonal Music as a Componential Code: Learning Temporal Relationships Between and Within Pitch and Timing Components
Authors: Catherine Stevens, Janet Wiles
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: This study explores the extent to which a network that learns the  temporal relationships within and between the component features of  Western tonal music can account for music theoretic and psychological  phenomena such as the tonal hierarchy and rhythmic expectancies.  Predicted and generated sequences were recorded as the representation of  a 153-note waltz melody was learnt by a predictive, recurrent network.  The network learned transitions and relations between and within pitch  and timing components: accent and duration values interacted in the  development of rhythmic and metric structures and, with training, the  network developed chordal expectancies in response to the activation of  individual tones. Analysis of the hidden unit representation revealed  that musical sequences are represented as transitions between states in  hidden unit space.
VLSI Phase Locking Architectures for Feature Linking in Multiple Target Tracking Systems
Authors: Andreas G. Andreou, Thomas G. Edwards
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: Recent  physiological  research  has  shown  that  synchronization  of  oscillatory  responses  in  striate  cortex  may  code  for  relationships  between  visual  features  of objects.  A  VLSI  circuit  has  been  de(cid:173) signed  to  provide  rapid  phase-locking  synchronization of multiple  oscillators to allow for further exploration of this neural mechanism.  By  exploiting the  intrinsic  random transistor  mismatch of devices  operated  in  subthreshold,  large  groups  of phase-locked  oscillators  can  be  readily  partitioned  into  smaller  phase-locked  groups.  A  mUltiple target  tracker for  binary images is described  utilizing this  phase-locking  architecture.  A  VLSI  chip  has  been  fabricated  and  tested  to  verify  the  architecture.  The  chip  employs  Pulse  Ampli(cid:173) tude  Modulation  (PAM)  to encode  the output at  the  periphery  of  the system.
A Network Mechanism for the Determination of Shape-From-Texture
Authors: Kô Sakai, Leif H. Finkel
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We propose a computational model for how the cortex discriminates  shape and depth from texture. The model consists of four stages: (1)  extraction of local spatial frequency, (2) frequency characterization, (3)  detection of texture compression by normalization, and (4) integration  of the normalized frequency over space. The model accounts for a  number of psychophysical observations including experiments based on  novel random textures. These textures are generated from white noise  and manipulated in Fourier domain in order to produce specific  frequency spectra. Simulations with a range of stimuli, including real  images, show qualitative and quantitative agreement with human  perception.
Learning Mackey-Glass from 25 examples, Plus or Minus 2
Authors: Mark Plutowski, Garrison Cottrell, Halbert White
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We apply active exemplar selection (Plutowski &. White, 1991;  1993) to predicting a chaotic time series. Given a fixed set of ex(cid:173) amples, the method chooses a concise subset for training. Fitting  these exemplars results in the entire set being fit as well as de(cid:173) sired. The algorithm incorporates a method for regulating network  complexity, automatically adding exempla.rs and hidden units as  needed. Fitting examples generated from the Mackey-Glass equa(cid:173) tion with fractal dimension 2.1 to an rmse of 0.01 required about 25  exemplars and 3 to 6 hidden units. The method requires an order  of magnitude fewer floating point operations than training on the  entire set of examples, is significantly cheaper than two contend(cid:173) ing exemplar selection techniques, and suggests a simpler active  selection technique that performs comparably.
Structured Machine Learning for 'Soft' Classification with Smoothing Spline ANOVA and Stacked Tuning, Testing and Evaluation
Authors: Grace Wahba, Yuedong Wang, Chong Gu, Ronald Klein MD, Barbara Klein MD
Published in: Advances in Neural Information Processing Systems 6  (NIPS 1993) (Main Conference Track), 1993

Abstract: We describe the use of smoothing spline analysis of variance (SS(cid:173) ANOVA) in the penalized log likelihood context, for learning  (estimating) the probability p of a '1' outcome, given a train(cid:173) ing set with attribute vectors and outcomes. p is of the form  pet) = eJ(t) /(1 + eJ(t)), where, if t is a vector of attributes, f  is learned as a sum of smooth functions of one attribute plus a  sum of smooth functions of two attributes, etc. The smoothing  parameters governing f are obtained by an iterative unbiased risk  or iterative GCV method. Confidence intervals for these estimates  are available. 

Introduction to 'soft' classification and the bias-variance tradeoff. 

In medical risk factor analysis records of attribute vectors and outcomes (0 or 1)  for each example (patient) for n examples are available as training data. Based on  the training data, it is desired to estimate the probability p of the 1 outcome for any
Computational Differences between Asymmetrical and Symmetrical Networks
Authors: Zhaoping Li, Peter Dayan
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: Symmetrically connected  recurrent networks have recently been  used as models of a host of neural computations.  However, be(cid:173) cause of the separation between excitation and inhibition, biolog(cid:173) ical  neural networks  are  asymmetrical.  We  study characteristic  differences between asymmetrical networks and their symmetri(cid:173) cal  counterparts,  showing  that  they  have  dramatically  different  dynamical behavior and also how the differences can be exploited  for computational ends.  We  illustrate our results in the case of a  network that is a selective amplifier.
A Randomized Algorithm for Pairwise Clustering
Authors: Yoram Gdalyahu, Daphna Weinshall, Michael Werman
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: We present a stochastic clustering algorithm based on pairwise sim(cid:173) ilarity  of datapoints.  Our  method  extends  existing  deterministic  methods, including agglomerative algorithms, min-cut graph  algo(cid:173) rithms,  and  connected  components.  Thus  it  provides  a  common  framework for  all  these  methods.  Our  graph-based  method  differs  from  existing  stochastic  methods  which  are  based  on  analogy  to  physical  systems.  The  stochastic  nature  of our  method  makes  it  more  robust  against  noise,  including  accidental  edges  and  small  spurious clusters.  We demonstrate the superiority of our algorithm  using  an example with  3 spiraling bands and  a  lot of noise.
Risk Sensitive Reinforcement Learning
Authors: Ralph Neuneier, Oliver Mihatsch
Published in: Advances in Neural Information Processing Systems 11  (NIPS 1998) (Main Conference Track), 1998

Abstract: A directed generative model for binary data using a  small number  of hidden  continuous  units  is  investigated.  A  clipping  nonlinear(cid:173) ity distinguishes the model from conventional principal components  analysis.  The relationships between the correlations of the underly(cid:173) ing continuous Gaussian variables  and the binary output variables  are utilized  to learn  the  appropriate weights  of the network.  The  advantages of this  approach are illustrated on a  translationally in(cid:173) variant binary distribution and on handwritten digit  images.