Computing with Action Potentials
Authors: John J. Hopfield, Carlos D. Brody, Sam Roweis
Published in: Advances in Neural Information Processing Systems 10  (NIPS 1997) (Main Conference Track), 1997

Abstract: Most computational engineering based loosely on biology uses  contin(cid:173) uous variables to represent neural activity.  Yet most neurons communi(cid:173) cate with action potentials.  The engineering view is equivalent to using  a rate-code for representing information and for computing. An increas(cid:173) ing number of examples are being discovered in which biology may not  be using rate codes.  Information can be represented using the timing of  action  potentials,  and  efficiently computed with  in this  representation.  The "analog match" problem of odour identification is a simple problem  which can be efficiently solved using action potential timing and an un(cid:173) derlying rhythm. By using adapting units to effect a fundamental change  of representation of a problem, we map the recognition of words (hav(cid:173) ing uniform time-warp) in connected speech into the same analog match  problem.  We describe the architecture and preliminary results of such a  recognition system.  Using the fast events of biology in conjunction with  an  underlying rhythm  is  one  way  to  overcome the  limits  of an  event(cid:173) driven view of computation. When the intrinsic hardware is much faster  than the time scale of change of inputs, this approach can greatly increase  the effective computation per unit time on a given quantity of hardware. 
1  Spike timing  Most neurons communicate using action potentials - stereotyped pulses of activity that are  propagated along axons without change of shape over long distances by active regenerative  processes.  They  provide  a  pulse-coded way of sending  information.  Individual action  potentials last about 2 ros.  Typical active nerve cells generate 5-100 action potentials/sec. 
Most biologically inspired engineering of neural networks represent the activity of a nerve  cell by a  continuous variable which  can be interpreted as  the short-time average rate  of  generating action potentials.  Most traditional discussions  by neurobiologists concerning  how information is represented and processed in the brain have similarly relied on using  "short term mean firing rate" as the carrier of information and the basis for computation.  But this is often an ineffective way to compute and represent information in neurobiology. 
*Dept.  of Molecular Biology, Princeton University.  jhopfield@watson.princeton. edu 
t  Computation &  Neural Systems, California Institute of Technology. 
Computing with Action Potentials 
167 
To  define "short term mean firing rate" with reasonable accuracy, it is necessary to  either  wait for several action potentials to arrive from a single neuron, or to average over many  roughly equivalent cells.  One  of these  necessitates  slow processing;  the  other requires  redundant "wetware". 
Since  action potentials are  short events with  sharp  rise  times,  action potential timing  is  another way that information can be represented and  computed with ([Hopfield,  1995]).  Action potential timing seems to  be the basis for some neural computations, such as the  determination of a sharp response time to an ultrasonic pulse generated by the moustache  bat.  In this system,  the bat generates a  10 ms pulse during which the frequency changes  monotonically with  time  (a "chirp").  In the  cochlea and cochlear nucleus,  cells  which  are  responsive to  different frequencies  will  be  sequentially driven,  each producing zero  or one action potentials during the  time when the frequency is  in their responsive band.  These action potentials converge onto a target cell.  However, while the times of initiation  of the  action potentials from  the  different frequency bands are  different,  the  length and  propagation speed of the various  axons have been coordinated to result in all the  action  potentials arriving at the target cell at the same time, thus recognizing the "chirped" pulse  as a whole, while discriminating against random sounds of the same overall duration.  Taking this hint from biology, we next investigate the use of action potential timing to rep(cid:173) resent information and compute with in one of the  fundamental computational problems  relevant to olfaction, noting why the elementary "neural net" engineering solution is poor,  and showing why computing with action potentials lacks the deficiencies of the  conven(cid:173) tional elementary solution. 
2  Analog match  The simplest computational problem of odors is merely to identify a known odor when a  single odor dominates the olfactory scene.  Most natural odors consist of mixtures of sev(cid:173) eral molecular species.  At some particular strength a complex odor b can be described by  the concentrations Nt  of its constitutive molecular of species i.  If the stimulus intensity  changes, each component increases (or decreases) by the same multiplicative factor.  It is  convenient to describe the stimulus as a product of two factors, an intensity .A  and normal(cid:173) ized components n~ as: