SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We analyzed a previously published dataset ([63], see also SM3.1 for details) of source-level MEG data recorded while human participants performed a visual decision-making task. At the beginning of each trial, a reference stimulus was presented (contrast  $50\%$ ), followed by a test stimulus that consisted of a sequence of 10 visual samples with variable contrasts (Fig. 3A). After the test stimulus sequence, participants reported their choice of whether the average contrast of the samples was greater or smaller than the reference contrast. The previous study on these data ([63]) analyzed the encoding of stimulus and choice signals in individual areas, but did not study information transfer. We focused on gamma-band activity (defined as the instantaneous power of the  $40 - 75\mathrm{Hz}$  frequency band), because it is the most prominent band for visual contrast information encoding [24, 45, 17] and information propagation [7, 3] in the visual system. Previous work has demonstrated that gamma-band transmission is stronger in the feedforward (from lower to higher in the visual cortical hierarchy) than in the feedback (from higher to lower in the visual cortical hierarchy) direction [55, 3, 36], suggesting that gamma is a privileged frequency channel for transmitting feedforward information. However, these previous studies did not determine the content of the information being transmitted.
To address this question, we quantified FIT in a network of three visual cortical areas (Fig. 3B) that we selected because they encoded high amounts of stimulus information and because they were sufficiently far apart  $(\geq 2.8\mathrm{cm})$  to minimize leakage in source reconstruction (see SM3.1.1 Fig. S9) [63, 23]. The areas, listed in order of position, from lower to higher, in the cortical hierarchy were: primary visual cortex (V1), area V3A, and area LO3. Because participants made errors (behavioral performance was  $75\%$  correct), in each trial the stimulus presented could differ from the participant's choice. We thus assessed the content of the information flow by computing FIT about either the sensory stimulus  $(FIT_{S}$ ; using as feature the mean contrast across all 10 visual samples) or the reported choice  $(FIT_{C})$ , in each instant of time in the  $[-100, 500]\mathrm{ms}$  peri-stimulus time window (because stimulus information was higher in the first 500ms post-stimulus, see SM3.1.4 and Fig. S9) and across a range of putative inter-area delays  $\delta$ . In Fig. 3C we show the resulting  $FIT_{S}$
time-delay information maps for the example pair of regions V1 and V3A. A cluster-permutation analysis [34, 14] revealed significant feedforward stimulus-specific information transmission from V1 to V3A (but no significant feedback from V3A to V1) localized 200-400ms after the stimulus onset, with an inter-area communication delay between 65 and 250ms (see SM3.1.4 and Fig. S9).
Figure 3: Information flow across the human visual hierarchy with MEG. (A) Sketch of task. (B) Cortical surface map of the location of the three considered visual regions. (C) Temporal profiles of stimulus information and time-delay stimulus FIT maps for an example regions pair (V1, V3A). Top to bottom: stimulus information in V1; time-delay FIT map in the feedforward  $(\mathrm{V}1\rightarrow \mathrm{V}3\mathrm{A})$  direction; time-delay FIT map in the feedback  $(\mathrm{V}3\mathrm{A}\rightarrow \mathrm{V}1)$  direction; stimulus information in V3A. (D) Comparison between FIT about stimulus  $(FIT_{S})$  and FIT about choice  $(FIT_{C})$  in the visual network in the feedforward (left) and feedback (right) directions. (E) Comparison between feedforward and feedback transmission in the network for TE (left) and stimulus FIT (right). (F) Same as E but for feedforward transmission on correct vs error trials. In all panels, lines and image plots show averages and errorbars SEM across participants, experimental sessions and regions pairs (in case of FIT and TE) or regions (in case of mutual information). \*: p<0.05, \*\*: p<0.01, \*\*\*: p<0.001. Information-theoretic quantities were computed from the gamma band ([40-75]Hz) power of source-level MEG, first computed separately for the left and right hemisphere and then averaged.
We compared properties of overall information propagation (computed with TE) and feature-specific information flow (computed with FIT) across all pairs of areas within the considered visual cortical network. To determine the prevalent content of information flow in the network, we compared  $FIT_{S}$  and  $FIT_{C}$  transmitted in the feedforward and in the feedback directions (Fig. 3D). Gamma-band transmitted more information about the stimulus than about choice (i.e.  $FIT_{S} > FIT_{C}$ ) in both the feedforward ( $p < 10^{-3}$  two-tailed paired t-test) and in the feedback ( $p < 0.01$  two-tailed paired t-test) direction, with a larger difference for the feedforward direction. This result is supported by simulations where we show that, in presence of multiple simultaneously transmitted features, FIT ranks correctly the features about which most information is transmitted (see SM2.3 and Fig. S4). Thus, we focused on stimulus-specific information flow in the following FIT analyses. We then studied the leading direction of information flow. Both the total amount of information propagation (TE) and the stimulus-specific information flow ( $FIT_{S}$ ) were larger in the feedforward than in the feedback direction (Fig. 3E), but with a larger effect for  $FIT_{S}$  ( $p < 10^{-6}$  two-tailed paired t-test) compared to TE ( $p < 0.05$  two-tailed paired t-test). Together, these results show that gamma-band activity in the visual system carries principally information about the stimulus (rather than choice) and propagates it more feedforward than feedback.
We next assessed the behavioral relevance of the feedforward stimulus information transmitted by the gamma band. A previous study showed that the overall (feature-unspecific) strength of feedforward gamma band information propagation negatively correlates with reaction times, indicating that stronger feedforward gamma activity propagation favors faster decisions [46]. However, no study has addressed whether stimulus information transmitted forward in the gamma band promotes decision accuracy. We addressed this question by comparing how  $FIT_{S}$  varied between trials when
participants made a correct or incorrect choice (Fig. 3F). We matched the number of correct and error trials to avoid data size confounds [40].  $FIT_{S}$  in the feedforward direction was significantly lower in error than in correct trials (Fig. 3F, right;  $p = 0.001$  two-tailed paired t-test), while TE did not vary between correct and error trials (Fig. 3F, left;  $p = 0.38$  two-tailed paired t-test). Feedback information transmission (both in terms of overall information propagation, TE, and stimulus specific information flow,  $FIT_{S}$ ), did not vary between correct and incorrect trials. This indicates that the feedforward flow of stimulus information, rather than the overall information propagation, is key for forming correct choices based on sensory evidence.
These results provide the new discovery that the gamma band transmits feedforward stimulus information of behavioral relevance, and highlight the power of FIT in revealing the content and direction of information flow between brain areas.
4.2 Eye-specific interhemispheric information flow during face detection
We next tested the ability of FIT to detect feature-specific information flow between brain hemispheres. We analyzed a published EEG dataset recorded from human participants detecting the presence of either a face or a random texture from an image covered by a bubble mask randomly generated in each trial ([47]; see SM3.2.1 for details). Previous analysis of these data [26] showed that eye visibility in the image (defined as the proportion of image pixels in the eye region visible through the mask) is the most relevant image feature for successful face discrimination. It then showed that eye-specific information appears first at  $\sim 120\mathrm{ms}$  post-image presentation in the Occipito-Temporal (OT) region of the hemisphere contra-lateral with respect to the position of the eye, and then appears  $\sim 20 - 40\mathrm{ms}$  later in the ipsi-lateral OT region (Fig. 4A). However, this study did not determine if the eye information in the ipsi-later hemisphere is received from the contra-lateral hemisphere. To address this issue, we computed FIT transmission of eye-specific information between the Left OT (LOT) and Right OT (ROT) regions (using the electrodes within these regions that had most information as in [26], see SM3.2.2). Left Eye (LE) FIT from the contra- to the ipsi-lateral OT (ROT to LOT) peaked between 150 to  $190\mathrm{ms}$  after image onset with transfer delays of  $20 - 80\mathrm{ms}$  (Fig. 4B). Right eye (RE) FIT from the contra- to the ipsi-lateral LE and RE had statistically significant FIT peaks in the time-delay maps (cluster-permutation analysis,  $p < 0.01$ ; see SM3.2.2 and Fig. S10). Thus, FIT determined the communication window for contra-lateral flow of eye-specific information with high precision.
Figure 4: Inter-hemispheric eye-specific information flow during face detection using human EEG. (A) Schematic of the putative information flow. LOT (ROT) denote Left (Right) occipito-temporal regions. LE (RE) denote the Left (Right) Eye visibility feature. (B) Information (lines) carried by the EEG in each region, and FIT (image plot) about LE contra-lateral transfer. (C) Same as B for RE. (D) Contra- to ipsi-lateral vs ipsi- to contra-lateral transfers for TE and FIT for both LE and RE. Dots and images plot averages and errorbars plot SEM across participants  $(N = 15)$ .
To gain further insight about the directionality and feature-specificity of the information flow across hemispheres, we compared FIT and TE across transfer directions and/or eye-specific visibility
conditions (Fig. 4D, middle and right). Right-to-left LE FIT was significantly larger than left-to-right LE FIT  $(p < 0.001$  two-tailed paired t-test) or right-to-left RE FIT  $(p < 0.01$  two-tailed paired t-test). Left-to-right RE FIT was significantly larger than right-to-left RE FIT  $(p < 0.05$  two-tailed paired t-test) or left-to-right LE FIT  $(p < 0.05$  two-tailed paired t-test). In contrast, we found no significant difference between directions for the overall propagated information (TE), Fig. 4D, left). Thus, the use of FIT revealed a temporally localized flow of eye information across hemispheres that was feature-selective (i.e. about mainly the contra-lateral eye) and direction-specific (contra-to-ipsi-lateral), without direction specificity in the overall information propagation (TE) across hemispheres.
Finally, to more tightly localize the origin of eye-specific contra-lateral information flow, we asked whether the contra-lateral OT electrodes selected in our analyses were the sole senders of interhemispheric eye-specific information. We used the conditioned version of FIT, cFIT, to compute the amount of transfer of eye information from the contra- to the ipsi-lateral OT after removing the effect of eye-specific information possibly routed through alternative sending locations (see SM3.4). We found (Fig. S12A) that the effect we measured with FIT was robust even when conservatively removing with cFIT the information that could have been routed through other locations.
4.3 Stimulus-specific information flow in a thalamocortical network
We finally used FIT to measure the feature- and direction-specificity of information flow in the thalamocortical somatosensory and visual pathways. We analyzed a published dataset in which multi-unit spiking activity was simultaneously recorded in anaesthetized rats from the primary visual and somatosensory cortices, and from first-order visual and somatosensory thalamic nuclei ([8], see SM3.3.1 for details), during either unimodal visual, unimodal tactile, or bimodal (visual and tactile) stimulation. This analysis tests FIT on another major type of brain recordings (spiking activity). Moreover, due to the wealth of knowledge about the thalamocortical network [53, 16], we can validate FIT against the highly-credible predictions that information about basic sensory features flows from thalamus to cortex, and that somatosensory and visual pathways primarily transmit tactile and visual information, respectively. Using FIT, we found (see SM3.3.3 and Fig. S11) that sensory information flowed primarily from thalamus to cortex, rather than from cortex to thalamus. We also found that the feedforward somatosensory pathway transmits more information about tactile- than about visually-discriminative features, and that the feedforward visual pathway transmits more information about visually- than tactile-discriminative features. Importantly, TE was similarly strong in both directions, and when considering tactile- or visually-discriminative features. This confirms the power of FIT for uncovering stimulus-specific information transfer, and indicates a partial dissociation between overall information propagation and neural transfer of specific information.
5 Comparison with previously published measures
We finally examine how FIT differs from alternative methods for identifying components of the flow of information about specific features. We focus on measures that implement the Wiener-Granger discounting of the information present in the past activity of the sender. Other methods, that do not implement this (and thus just correlate past information of the sender with present information of the receiver), erroneously identify information already encoded in the past activity of the receiver as information transmitted from a sender (see SM4.3).
A possible simple proxy for identifying feature-specific information flow is quantifying how the total amount of transmitted information (TE) is modulated by the feature [7]. For the case of two feature values, this amounts to the difference of TE computed for each individual value. We show in SM4.1, using simulations, that this measure can fail in capturing feature-related information flow even in simple scenarios of feature information transmission. Additionally, when tested on MEG data, it could not assess the directionality of information transmission within brain networks (see SM4.1).
A previous study [25] defined a measure, Directed Feature Information (DFI), which computes feature-specific information redundant between the present activity of the receiver and the past activity of the sender, conditioned on the past activity of the receiver. However, DFI used a measure of redundancy that conflated the effects of redundancy and synergy (see SM1.5). Because of this, DFI is, both on real and on simulated data, often negative and thus not interpretable as measure of information flow (see SM4.2). In contrast, FIT is non-negative and uses PID to consider only redundant information between sender and receiver, as appropriate to identify transmission of information. Moreover,
because DFI discounts only past activity of the sender rather than its feature-specific information, it was less precise, conservative and sensitive in localizing direction and timing of feature-specific information flow (as shown in SM4.2 and Fig. S15 with simulated and real data).
Finally, a study defined feature-specific information using PID in the space of four variables  $S$ ,  $X_{past}$ ,  $Y_{past}$ , and  $Y_{pres}$  [4]. However, this measure was not upper bounded by either feature information encoded in the past of the sending region or the total information flowing between regions.
We developed and validated FIT, an information theoretic measure of the feature-specific information transfer between a sender  $X$  and a receiver  $Y$ . FIT combines the PID concepts of redundancy and uniqueness of information [62] with the Wiener-Granger causality principle [11] to isolate, within the overall transmitted information (TE), the flow of information specifically related to a feature  $S$ .
The strengths and limitations of FIT as a neural data analysis tool stem from those of information theory for studying neural information processing. Information theory has led to major advances to neural coding because of its ability to capture linear and non-linear interactions at all orders making little assumptions [43, 15]. This is important because deviations from linearity and order of interactions vary in often unknown ways between brain areas, stimulus types and recording modalities [13, 28, 20]. Using such a general formalism avoids potentially biasing results with wrong assumptions. However, the price to pay for the fact that information theory includes full probability distributions is that it is data hungry. While our definitions of FIT and cFIT are straightforwardly valid for multivariate analyses including conditioning on the information of multiple regions [37] (as in cFIT) or obtaining more conservative estimates of information transmission on which information in the receiver  $Y$  is requested to be unique with respect to the information of the sender and receiver at multiple past time points [58], for data sampling reasons in practice in real data these analyses are confined to conditioning to one region or a single past time point [7, 39, 38]. In future work, we aim to make FIT applicable to analyses of multiple regions or time points coupling it with advanced non-parametric [49] methods to robustly estimate its multivariate probability distributions.
The generality of our approach lends itself to further developments. Importantly, we defined FIT directly at the level of PID atoms. This means that, although here we implemented FIT using the original definition of redundancy in PID [62] because it has the advantage of being non-negative for all information atoms, FIT can be easily implemented also using other PID redundancy measures [22, 5, 2, 18, 30] with complementary advantages and disadvantages (see SM1.2).
We individuate two directions for improvement. First, even though the surrogate permutation test we developed to assess FIT significance provided reasonable results on real data and worked well well also with artefacts due to instantaneous mixing of sources (see SM2.7), further research is needed to generate more refined surrogate data generation techniques to rule out more conservatively false feature-specific communication scenarios [52]. Second, how best to select the time of past activity to compute FIT, an issue of significance in Wiener-Granger measures [60], remains to be determined.
To demonstrate the properties of FIT, we performed numerical simulations in different communication scenarios and compared FIT against TE (Fig. 2). These simulations confirmed that TE effectively detected the overall propagation of information, but it did not detect the flow of feature-specific information. FIT, in contrast, reliably detected feature- and direction-specific information flow with high temporal sensitivity. We confirmed the utility of FIT in applications to neural data. In three brain datasets spanning the range of electrophysiological recordings (spiking activity, MEG and EEG), FIT credibly determined the directionality and feature specificity of information flow. Importantly, in most of these datasets this happened in the absence of variations in the overall information propagation between the same brain regions (measured with TE). The partial dissociation between overall activity flow and feature-specific flow found consistently in simulations and data has important implications. First, it highlights the need of introducing a specific measure of feature information transfer such as FIT, as it resolves question unaddressed by content-unspecific measures. Second, it establishes that measuring feature-specific components of information flow between brain regions is critical to go beyond the measurement of overall neural activity propagation and uncover aspects of cross-area communication relevant for ongoing behavior. Thus, as methods to record from multiple brain regions rapidly advance, FIT is well suited to uncover fundamental principles in how brain regions communicate.
This research has received funding from the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3), from the NIH Brain Initiative (grants U19 NS107464, R01 NS109961, R01 NS108410), and from the Simons Foundation (SFARI Human Cognitive and Behavioral Science grant 982347). We thank G. Bondanelli and G.M. Lorenz for useful discussion and feedback on the manuscript.
[60] M. Wibral, N. Pampu, V. Priesemann, F. Siebenhuhner, H. Seiwert, M. Lindner, J. T. Lizier, and R. Vicente. Measuring information-transfer delays. PLoS ONE, 8(2):e55809, 2013.
[61] N. Wiener. The theory of prediction. In Modern Mathematics for Engineers. E. F. Beckenbach, New York: McGraw-Hill, 1956.
[62] P. L. Williams and R. D. Beer. Nonnegative decomposition of multivariate information. arXiv:1004.2515, 2010.
[63] N. Wilming, P. R. Murphy, F. Meyniel, and T. H. Donner. Large-scale dynamics of perceptual decision information across human cortex. Nature Communications, 11(1):5109, 2020.
Supplementary Material for An information-theoretic quantification of the content of communication between brain regions
SM1 Definitions, derivations, and properties of the information theoretic quantities
In this section, we first define the basic information theoretic quantities that we use in the paper. We next introduce basic concepts of the PID theory needed for our derivations. We then use these concepts to derive a mathematical definition of FIT and then prove some of its key mathematical properties.
SM1.1 Definition of the Shannon information quantities used in this study
In the following we describe and provide analytical expression for the quantities of Shannon's information theory that we used in this paper. These are the mutual information between the stimulus and the neural activity, and the Transfer Entropy (TE), which are estimated based on the probabilities of activities of neural signals  $X$  and  $Y$  and of stimulus features  $S$ .
The mutual information  $I(S; X_{pres})$  between the stimulus feature  $S$  and the neural activity  $X_{pres}$  of  $X$  at the present time is a non-parametric measure that quantifies the full single-trial statistical relationship between  $S$  and  $X_{pres}$ . It captures the effect of all linear and nonlinear interactions between these variables. It is defined as follows [35]:
$$
I (S; X _ {p r e s}) = \sum_ {s, x _ {p r e s}} p (s, x _ {p r e s}) \log \frac {p (s , x _ {p r e s})}{p (s) p \left(x _ {p r e s}\right)} \tag {S1}
$$
where  $p(s, x_{pres})$  is the joint probability, sampled across experimental trials, of observing in a given trial the joint occurrence of stimulus feature value  $s \in S$ , and activity  $x_{pres} \in X_{pres}$ . The sum spans all possible events.  $I(S; X_{pres})$  is non-negative, and it is zero if and only if  $S$  and  $X_{pres}$  are independent. Similar expressions and properties hold for the information  $I(S; X_{past}), I(S; Y_{past})$  between the stimulus and the past activity  $X_{past}, Y_{past}$  of  $X$  and  $Y$ , respectively.
TE [34] is an information theoretic measure that utilizes the Wiener-Granger principle to quantify the overall propagation of information by neural activity from a putative sender  $X$  to a putative receiver  $Y$  as the mutual information between the present neural activity of the receiver  $Y_{pres}$  and the past activity of the sender  $X_{past}$ , conditioned upon the past activity of the receiver  $Y_{past}$ . The expression of TE as a function of the joint probability distribution  $P(X_{past}, Y_{pres}, Y_{past})$  is as follows: