next up previous
Next: Model assumptions Up: Methods for estimating spike Previous: Reconstruction method

Direct method

In this paper we will use a direct method [DeWeese, 1995, DeWeese, 1996, Stevens and Zador, 1996, de Ruyter van Steveninck et al., 1997] to estimate the mutual information. Direct methods use another form of the expression eq. (5) for mutual information,
 equation478
The first term tex2html_wrap_inline1245 is the entropy of the output spike train itself, while the second tex2html_wrap_inline1303 is the conditional entropy of the output given the inputs. The first term measures of the variability of the spike train in response to the ensemble of different inputs, while the second measures the reliability of the response to repeated presentations of the same inputs. The second term depends on the reliability of the synapses and spike generating mechanism: to the extent the same inputs produce the same outputs, this term approaches zero.

The direct method has two advantages over the reconstruction method in the present context. First, it does not require the construction of a ``reconstructor'' for estimating the input from the output. Although the optimal linear reconstructor is straightforward to estimate, the construction of more sophisticated (i.e. nonlinear) reconstructors can be a delicate art. Second, it provides an estimate of information that is limited only by the errors in the estimation of tex2html_wrap_inline1245 and tex2html_wrap_inline1303; the reconstruction method by contrast provides only a lower bound on the mutual information that is limited by the quality of the reconstructor.

As noted above, the estimation of tex2html_wrap_inline1245 and tex2html_wrap_inline1303 can require vast amounts of data. If, however, interspike intervals (ISIs) in the output spike train were independent, then the entropies could be simply expressed in terms of the entropy of the associated ISI distributions. The information per spike tex2html_wrap_inline1313 is then given simply by
 equation491
where H(T) are tex2html_wrap_inline1317 are total and conditional entropies, respectively, of the ISI distribution. The information rate (units: bits/second) is then just the information per spike (units: bits/spike) times the firing rate R (units: spikes/second),
 equation496

The representation of the output spike train as a sequence of firing times tex2html_wrap_inline1321 is entirely equivalent (except for edge effects) to the representation as a sequence of ISIs tex2html_wrap_inline1323, where tex2html_wrap_inline1325. The advantage of using ISIs rather than spike times is that H(T) depends only on the ISI distribution p(T), which is a univariate distribution. This dramatically reduces the amount of data required.

In the sequel we assume that spike times are discretized at a finite time resolution tex2html_wrap_inline1331. The assumption of finite precision keeps the potential information finite. If this assumption is not made, each spike has potentially infinite information capacity; for example, a message of arbitrary length could be encoded in the decimal expansion of a single ISI.

Eq. 8 represents the information per spike as the difference between two entropies. The first term is the total entropy per spike,
 equation501
where tex2html_wrap_inline1333 is the probability that the length of the ISI was between tex2html_wrap_inline1335 and tex2html_wrap_inline1337. The distribution of ISIs can be obtained from a single long (ideally, infinite) sequence of spike times.

The second term is the conditional entropy per spike. The conditional entropy is just the entropy of the ISI distribution in response to a particular set m of input spikes tex2html_wrap_inline1341, averaged over all possible sets of inputs spikes
 equation504
where tex2html_wrap_inline1343 represents average. Here tex2html_wrap_inline1345 is the probability of obtaining an ISI of length tex2html_wrap_inline1347 in response to a particular particular set of input spikes tex2html_wrap_inline1341.

We used the following algorithm for estimating the conditional entropy:

  1. Generate ensemble of input spikes. Some particular ensemble of input spikes (corresponding, for example, to m=17) is generated, tex2html_wrap_inline1353, where tex2html_wrap_inline1355 are independent homogenous Poisson processes (for convenience we assume they have the same rate, but this is not essential).
  2. Compute conditional ISI distribution. The conditional distribution tex2html_wrap_inline1357 of ISIs of the model neuron is obtained by measuring the ISIs on a large (ideally, an infinite) number of trials in which a synaptic current is generated from tex2html_wrap_inline1359 using the synaptic noise equations eq. 2 or eq. 3. If the noise is nonzero, then each realization of the synaptic current tex2html_wrap_inline1361 is slightly different, leading to variability in the output ISI.
  3. Compute conditional entropy for this input ensemble. From the conditional distribution, the conditional entropy in response to this particular input ensemble is computed as tex2html_wrap_inline1363. This ISI distribution depends on the amount of synaptic noise assumed; if there is no noise, the output distribution assumes only a single value and the conditional entropy is zero.
  4. Repeat, and average over conditional entropies for other ensemble. The average conditional entropy per spike is calculated by repeating this procedure for a large (ideally, infinite) number of input patterns tex2html_wrap_inline1365 and averaging over the resulting conditional entropies.

In summary, we have described the three steps required to compute the information rate in our model. First, the total entropy per spike is computed from Eq. 10, and the conditional entropy per spike is computed from Eq. 11. Next, the information per spike is computed from Eq. 8. Finally, the information rate (information per time) is computed from Eq. 9.


next up previous
Next: Model assumptions Up: Methods for estimating spike Previous: Reconstruction method

Tony Zador
Fri Nov 28 10:17:14 PST 1997