The nervous system represents time dependent signals in sequences of discrete action potentials or spikes; all spikes are identical so that information is carried only in the spike arrival times. We show how to quantify this information, in bits, free from any assumptions about which features of the spike train or input signal are most important, and we apply this approach to the analysis of experiments on a motion sensitive neuron in the fly visual system. This neuron transmits information about the visual stimulus, at rates of up to 90 bits/s, within a factor of two of the physical limit set by the entropy of the spike train itself. As you read this text, optical signals reaching your retina are encoded into sequences of identical pulses, termed action potentials or spikes, that propagate along the ∼ 10 6 fibers of the optic nerve from eye to brain. This spike encoding appears almost universal, occurring in animals as diverse as worms and humans, and spanning all the sensory modalities [1] . The molecular mechanisms for the generation and propagation of action potentials are well understood [2] , as are the mathematical reasons for the selection of stereotyped pulses by the dynamics of the nerve cell membrane [3] . Less well understood is the function of these spikes as a code [4] : How do the sequences of spikes represent the sensory world, and how much information is conveyed in this representation? The temporal sequence of spikes provides a large capacity for transmitting information, as emphasized by MacKay and McCulloch 45 years ago [5] . One central question in studies of the nervous system is whether the brain takes advantage of this large capacity, or whether variations in spike timing represent noise which must be averaged away [4, 6] . In response to a long sample of time varying stimuli, the spike train of a single neuron varies, and we can quantify this variability by the entropy per unit time of the spike train, S(∆τ ) [7] , which depends on the time resolution ∆τ with which we record the spike arrival times. If we repeat the same time dependent stimulus, we see a similar, but not precisely identical, sequence of spikes (Fig. 1) . This variability at fixed input can also be quantified by an entropy, which we call the conditional or noise entropy per unit time N (∆τ ). The information that the spike train provides about the stimulus is the difference between the total spike train entropy and this conditional entropy, R info = S − N [7] . Because the noise entropy is positive (semi)definite, the entropy rate sets the capacity for transmitting information, and we can define an efficiency ǫ(∆τ ) = R info (∆τ )/S(∆τ ) with which this capacity is used [8] . The question of whether spike timing is important is really the question of whether this efficiency is high at small ∆τ [4] . For some neurons, we understand enough about what the spike train represents that direct "decoding" of the spike train is possible; the information extracted by these decoding methods can be more than half of the total spike train entropy with ∆τ ∼ 1 ms [8] . The idea that sensory neurons provide a maximally efficient representation of the outside world has also been suggested as an optimization principle from which many features of these cells' responses can be derived [9] . But, particularly in the central nervous system [6] , assumptions about what is being encoded should be viewed with caution. The goal of this paper is, therefore, to give a completely model independent estimate of entropy and information in neural spike trains as they encode dynamic signals. We begin by discretizing the spike train into time bins of size ∆τ , and examining segments of the spike train in windows of length T , so that each possible neural response is a "word" with T /∆τ symbols. Let us call the normalized count of the i th wordp i , and then the "naive estimate" of the entropy is where the notation reminds us that our estimate of the entropy depends on the size of the data set we use in accumulating the histogram. The true entropy is and we are interested in the entropy rate The difficutly is that, especially at large T , very large data sets are required to ensure convergence of S naive to the true entropy S. Imagine that we have a spike train with mean spike rater ∼ 40 spikes/s and we sample with a time resolution ∆τ = 3 ms. In a window of T = 100 ms, the maximum entropy consistent with this mean rate [4, 5] is S ∼ 17.8 bits, and we will see that the entropy of real spike trains is not far from this bound. But then there 1
Prior work investigating the mutual information in neural communications presented in REF has expressed the entropy of a source through the time-binning of the spike train, and defining the set of possible symbols as the possible neural response within the period.
Entropy and Information in Neural Spike Trains
arXiv: Condensed Matter
Physics Biology