SlowGuess/NeurIPS_2023_OCR huggingface dataset:
with  $E_{x}$ ,  $E_{y}$  being independent Gaussian noise. The stimulus feature  $S$  was a binary value extracted independently in each trial. The source  $Z$  encoded the feature with additive Gaussian noise. The amount of stimulus-feature information encoded in  $Z$  increased linearly over time. We controlled the SNR of  $X$  and  $Y$  by changing A (which sets the relative level of stimulus-feature signal in  $X, Y$ ) and fixing noise standard deviation to 1. On this model FIT and TE had spurious positive values (Fig. S8B). We used the permutation test introduced in Section SM1.7, testing for spurious values induced by  $X, Y$  covariations due to feature-signal sharing. We found that this test correctly ruled out as non-significant FIT and TE values generated only by source sharing with no real transmission (Fig. S8B). Importantly, analysis of this model also showed that with instantaneous source mixing (and notably under the assumption that recording noise is constant over the time of the trial) the ratio between stimulus-feature info in  $X$  and  $Y$  is constant in time (Fig. S8A). This gives a useful heuristic: while the finding that the feature information time courses of two individual regions that overlap in time cannot be used to rule in or out communication of information about the feature between the two areas (see SM2.2 and Fig 2D-E), different timecourses of stimulus-feature info in  $X$  vs  $Y$  cannot be easily explained by instantaneous source mixing. We measured all real-data FIT in cases with a delay in stimulus-feature info latencies between  $X$  and  $Y$  ( $X$  to  $Y$  info latencies: MEG: 17-35ms between V1 and higher areas, Fig. S9B. EEG: 25ms across hemispheres (Fig 4B). Spike data: 20ms from thalamus to cortex, Fig. S11). Overall, these findings speak against dominant mixing of a feature-informative source in our analyses.
Finally, we simulated the case with real FIT between two "pure" signals  $Z_{1}$  and  $Z_{2}$  that are unevenly mixed in the measured  $X,Y$ :
A Definition of cFIT
B Parallel information transfer scenario
Figure S7: Simulation tests of the significance of FIT and cFIT. (A) Schematic of the cFIT definition. Left: intuitive definition with set-theoretic diagrams. Right: breakdown of  $FIT_X$  (in red) and  $FIT_Z$  (in light blue) into finer information atoms considered in the definition of  $cFIT_{X|Z}$  (atom that can be part of  $cFIT_{X|Z}$  are indicated in yellow). Only atoms of  $FIT_X$ ,  $FIT_Z$  and  $cFIT_{X|Z}$  belonging to the PID having  $S$  as target variable are shown.  $FIT_X$  is the feature information transmitted from  $X$  to  $Y$ ,  $FIT_Z$  is the one transmitted from  $Z$  to  $Y$ ,  $cFIT_{X|Z}$  is the cFIT from  $X$  to  $Y$  conditioned on the stimulus-feature information of  $Z$ . (B) Schematic of the scenario implemented in the simulations: both  $X$  and  $Z$  transmit feature information to  $Y$ . (C) FIT dependence on the amount of stimulus-feature information transmitted from  $Z$  to  $Y$  ( $W_{ZY}$ ) even when simulating a case ( $W_{XY} = 0$ ) in which there was no within-trial transmission from  $X$  to  $Y$ . FIT grows with  $W_{ZY}$ , but its value is always non-significant using the permutation null hypothesis described in Section SM1.7. The dashed green line shows the 99th percentile of the FIT null hypothesis distribution described in Section SM1.7. For comparison, the dashed gray line shows the 99th percentile of the null-hypothesis distribution that would have been obtained simply shuffling  $S$  across all trials. The fact that the latter remains so low across all values of  $W_{zy}$  highlights the need of using a shuffling procedure that preserves the stimulus-feature information in the individual nodes, as we did in this paper and described in Section SM1.7 (D) FIT and cFIT as function of feature-related transmission from  $X$  to  $Y$  ( $W_{XY}$ ) and from  $Z$  to  $Y$  ( $W_{ZY}$ ). * indicate significant values ( $p < 0.01$ , permutation test) for the considered parameter set. In Panels C,D, results plot mean across 50 simulations (2000 trials each). (E) Example of shuffled distributions in a case in which there is not stimulus-feature information in  $Y$ . While the null hypothesis values of permuting  $X$  at fixed  $S$  give much more conservative and effective null hypothesis values when the analyzed network has stimulus-feature information across the nodes (see panel C), in specific cases of no feature information in parts of the network it may be safer to consider also the permutation of  $S$  across all trials, as this has more available independent permutations form the data and thus gives wider distributions. The example is with the simulations performed in Fig. 2A, for the set of parameters ( $W_{stim} = 0$ ,  $W_{noise} = 0.6$ ).
$$
X = Z _ {1} + A Z _ {2} + E _ {x}
$$
$$
Y = Z _ {2} + B Z _ {1} + E _ {y} \tag {S44}
$$
Since adding a new feature-informative channel ( $Z_{1}$  to  $Y$  and  $Z_{2}$  to  $X$ ) increases the stimulus-feature information in  $X, Y$ , we set the standard deviation of independent Gaussian noise  $E_{x}, E_{y}$  to equalize SNR of  $X$  and  $Y$  across the simulated parameters space. We found (Fig. S8C) that mixing  $(A, B > 0)$  reduced FIT and TE compared to the pure case ( $A = B = 0$ ). However, the correct direction of information transfer was always detected for all mixtures. Thus, FIT is reasonably conservative and robust to this mixing.
One shared source
Figure S8: Simulated tests of FIT and TE in the presence of source mixing. A-B: results of the null model with unequal sharing of one feature-informative source  $Z$ . (A) Proportion of stimulus-feature info with only the null model (blue line, described in SM2.7 and eq. S43) and with real transmission of feature information  $X \rightarrow Y$  added to it (red line). (B) FIT and TE of null model (solid lines; see eq. S43) fall below the permutation's test 99th percentile (dashed lines) for any proportion of sharing A. (C) FIT and TE computed from a model with two stimulus-feature informative sources  $Z_{1}$ ,  $Z_{2}$  communicating  $Z_{1} \rightarrow Z_{2}$  and mixed into  $X$  and  $Y$ , vs mixing proportions  $A$ ,  $B$  in  $X$  and  $Y$  (see eq. S44). All panels plot results averaged over 50 simulations.
SM3 Details and further analyses of experimental data
SM3.1 MEG data
SM3.1.1 Behavioral task and MEG recordings
presented in this paper focused on three visual cortical ROIs that were all more than  $2.8\mathrm{cm}$  apart from one another (i.e., minimizing spatial filter leakage) and encoded large stimulus information: primary visual cortex (V1), area V3A (which carried maximal stimulus information in the "Dorsal Stream Visual Cortex" group [13]), and area LO3 (which carried high stimulus information in the "MT+ and Neighboring Visual Areas" group [13]).
SM3.1.2 Parameters and details of the Information theoretic analyses
For the analysis of FIT and TE we used gamma-band instantaneous power obtained by computing the time-frequency representations of single-trial data via the multi-taper method and then averaging the obtained powers in the  $[40 - 75]\mathrm{Hz}$  band, exactly as described in the original publication [39]. To estimate the joint probability distributions of the neural activity and the stimulus (or the choice) used to compute the information theoretic quantities, we binned the MEG gamma power from each ROI into 2 equally populated bins and then computed empirically the frequency of occurrence of each response bin across all available trials. The stimulus features used for the information analyses was the average contrast of the 10 samples presented on each trial, discretized into two values. (We coded 0 the average contrast if it was below the reference contrast and 1 the average contrast if it was above the reference contrast). The choice feature used for the information analyses was the binary choice reported by the participant in each trial, with the choice "average contrast stronger than reference" coded as 1 and the choice "average contrast weaker than reference" coded as 0. We computed the information quantities for both the feedforward and the feedback direction for the left and the right hemisphere separately and then averaged the two.
Unless otherwise stated, we computed information quantities using all available trials (correct and error trials). For the specific set of information analyses comparing correct and error trials (Fig. 3G,H), we randomly subsampled correct trials so that the number of correct and error trials used to compute the information quantities was the same for each session. In this way, the information values for correct and error trials can be compared fairly because their difference cannot reflect possible differences in limited-sampling biases due to different data numerosity [26].
SM3.1.3 Statistical analyses
We established significance of the information measures in the time-delay space using a cluster-based nonparametric statistical test described in Section SM1.7, see [24; 8].
To provide a quantification values of TE and FIT across participants, sessions and network links (Fig. 3D,F,H), we selected a rectangular region in the time-delay domain to select the TE and FIT values for the across sessions statistics, centered around the FIT significant cluster (FIT-specific region). We computed the average over delays and then picked the maximum over time within this region. This gave us one single TE and one single FIT value for each hemisphere in each session. The comparisons of values across participants, subjects and links was performed using two-tailed paired t-tests.
SM3.1.4 Additional results
Here we list the results of a number of additional analyses that could not be inserted in the main text due to lack of space but that are helpful to better understand and support the conclusions presented in the main text.
The first set of results regards the encoding of information in individual regions of the visual network, rather than the transmission across regions of information about the stimulus. We reported temporal profiles of stimulus information in the three selected ROIs in Fig. S9B. Instantaneous information profiles showed a clear lag in the onset of stimulus information that could not be explained by instantaneous source mixing (see Section SM2.7). The amount of mutual information about the stimulus carried by the power of the gamma band in the visual cortical network is larger in the first half of the presentation of the stimulus ([0-500]ms peri-stimulus, denoted as 'early') than in second half of the presentation of the stimulus ([500-1000]ms peri-stimulus, denoted as 'late') within the trial (Fig. S9C, left). This is why we concentrated the FIT analyses on the first part of the trial (the early window). In the early part of the trial, the gamma band activity in the visual cortical network carries more stimulus than choice information (Fig. S9C, middle) and this information is higher in correct compared to error trials (Fig. S9C, right). These results are useful to confirm that stimulus
information coding is of more prominent importance in the visual network and that the presence of this information is key to perform accurate perceptual discriminations.
The second set of results regards additional findings about the transmission across regions of information about the stimulus. We produced network representation showing the relative strength of individual TE and stimulus FIT links contributing to the observed differences in directionality (Fig. S9D, compare with Fig.3D) and feedforward behavioral relevance (Fig. S9E, compare with Fig.3E) of information transmission. We found no difference of FIT stimulus nor TE in the feedback direction between correct and error trials (Fig. S9F). For the example pair V1-V3A we identified a significant cluster of stimulus FIT feedforward (V1 to V3A) but not feedback (V3A to V1) in the time-delay domain (Fig. S9G; cluster statistics,  $p < 0.01$ ). These results suggest that feedforward propagation of stimulus information, but not feedback propagation, is specifically key for correct behavior.
In the main text we indicated that the time delay region in which FIT about the stimulus was significant was in the interval 200 to  $400\mathrm{ms}$  after the stimulus onset, with an inter-area communication delay between 65 and  $250\mathrm{ms}$ . This statement is supported by the plot in Fig. S9G of the time-delay map points that are significant according to the cluster permutation test.
Finally, we performed a control analysis to quantify TE in a time-delay region around which TE was maximal. This is of interest because in the main text analyses (Figure3E-H) we compared FIT and TE using a time-delay region around the peak of FIT. The TE panel (Fig. S9H) shows that TE peaks in a different time-delay region with respect to stimulus FIT (Fig. 3C). Taking a DI-specific box centered around the TE peak in time-delay to select information values we could not assess the direction nor the behavioral relevance of information transmission (Fig. S9I).
SM3.2 EEG data
SM3.2.1 Behavioral task and EEG recordings
We next analyzed a publicly available EEG dataset [31]. Data are available at [https://datadryad.org/stash/dataset/doi:10.5061/dryad.8m2g3]. Full details (including details of approvals from Ethical Committees) are reported in the original publication. Here we summarize them briefly. The EEG data were recorded while participants  $(N = 16)$  performed a face detection task. Participants were presented with an image hidden behind a bubble mask that was randomly generated in each trial. The presented image was a image of a face in half of the trials and a random texture in the other half of the trials. Participants were instructed to report whether a face was present or not. In our analyses, we only considered correct trials where the face was correctly detected by the participants (approximately 1000 trials per subject). Following the recommendations of the original publications analysing these data [31; 18], we excluded one participant from the analysis due to a poor EEG signal that did not contain significant eye visibility information in any of the electrodes. All analyses in our paper are based on the  $N = 15$  selected participants. EEGs were recorded by fitting participants with a Biosemi head cap comprising 128 EEG electrodes. EEG data were re-referenced offline to an average reference, band-pass filtered between  $1\mathrm{Hz}$  and  $30\mathrm{Hz}$  using a fourth order Butterworth filter, down-sampled to  $500\mathrm{Hz}$  sampling rate and baseline corrected using the average activity between 300ms pre-stimulus and stimulus presentation. ICA was performed to reduce blink and eye- movement artifacts, as implemented in the infomax algorithm from EEGLAB [10]. Components representing blinks and eye movements were identified by visual inspection of their topographies, time courses, and amplitude spectra.
SM3.2.2 Details of the information theoretic analyses and additional results
For the analyses of TE and FIT, we selected the EEG electrodes in the left and the right Occipito-Temporal regions that had the highest mutual information about the visibility of the contra-lateral eye, exactly as done in previous papers [18]. (Specifically, in Ref. [18], the authors used the following criteria to select one electrode in LOT and one in ROT for each participant, see Fig. S13A. For LOT they selected the electrode with maximum right eye MI from electrodes on the radial axes of P07, P7, and TP7, excluding midline Oz and neighboring O1 radial axes. On the right hemisphere, for ROT the author selected the EEG electrode with maximum left eye information from sensors on the radial axes of PO8, P8, TP8, excluding midline Oz and neighboring O2 radial axes). We computed the first derivatives of the EEG signal for both Occipito-Temporal sensors and used both its absolute values
Figure S9: Additional analyses of the MEG dataset. (A) Correlation between LCMV spatial filters of neighboring sources as a function of their distance. (B) Time course of stimulus information in each ROI. (C) Properties of mutual information between stimulus and MEG activity. Left: Peak values of stimulus information over time in the first (early time window, from 0 to 500ms post-stimulus onset) and second half (late time window, 500 to 1000ms after stimulus onset) of the stimulus presentation window. Middle: Peak of stimulus and choice information carried by MEG gamma activity in the early window. Right: Peak of stimulus information values encoded by MEG gamma activity in the early window in correct and incorrect trials, respectively. (D) Graphs representing the strength of feedforward (yellow) and feedback (orange) information transmission in the network for TE (top) and stimulus FIT (bottom). Links are weighted proportionally to the communication strength between each pair. The arrows on the bottom points toward the dominant direction of overall transmission, and are weighted proportionally to the difference between feedforward and feedback transmission. (E) Same as D but for feedforward transmission in correct (green) vs error (gray) trials. (F) Values of TE (left) and of FIT about the stimulus (right) computed in the feedback direction separately in correct and in error trials. (G) Plot of the points with significant values of the stimulus FIT between V1 and V3A (top) and V3A and V1 (bottom) according to a cluster permutation test. Only points that are significant are colored. Color scale is the same as the Fig. 3C. (H) TE time-delay map in the V1 to V3A direction. (I) Values of TE using a time-delay box centered around the TE peak in the time-delay map. In all panels, lines and image plots show averages and errorbars SEM across participants, experimental sessions and regions pairs (in case of FIT and TE) or regions (in case of mutual information).  $*$ : p<0.05,  $\*\*:$  p<0.01,  $\*\*\*:$  p<0.001. All information-theoretic quantities were computed from power time courses of source-level MEG signals in the gamma-band ([40-75]Hz), first computed separately for left and right hemisphere and then averaged.
and first derivatives to compute the information quantities, for consistency with the information-encoding analyses performed in a previous study [18]. As stimulus feature for the computation of mutual information and FIT, we used the visibility of an eye (defined as the fraction of pixels within the eye region that were not hidden by the bubble mask). This feature was discretized using 2 equipopulated bins. We computed the information quantities for all combinations of directionality of flow across hemispheres (left to right, right to left) and eye identity (left or right eye). We computed significance of FIT in the time-delay using the cluster-based permutation test described in Section SM1.7. This analysis revealed a significant cluster of FIT about the left eye in the right-to-left
Figure S10: (A) Plot of the points with significant values of the left-eye (LE) visibility FIT from the EEG of Right Occipito-Temporal (ROT) to Left Occipito-Temporal (LOT) electrodes, according to a cluster permutation test. Only points that are significant are colored. (B) Same as A but for the Right-Eye (RE) visibility FIT from the EEG of Left Occipito-Temporal (LOT) to Right Occipito-Temporal (ROT) electrodes. The color scale of panel A and B is the same as the corresponding plots in Figure 3B and C, respectively.
direction (Fig. S10A) and a about the right eye in the left-to-right direction (Fig. S10B). To provide a quantification values of TE and FIT across participants (Fig. 4D), we selected a rectangular region in the time-delay domain to select the TE and FIT values for the across participants statistics, centered around the contra-lateral FIT significant cluster (same for both eyes, as they were significant in very similar time-delay regions). We computed the average over delays and then picked the maximum over time within this region. This gave us one single TE and one single FIT value for each subject. The comparisons of values across participants was performed using two-tailed paired t-tests.
SM3.3 Analysis of spiking activity in a thalamocortical network
SM3.3.1 Electrophysiological experiments
We analysed previously published [5] recordings of multi-unit spiking activity simultaneously obtained from electrodes placed in the primary visual cortex (V1), primary somatosensory cortex (S1), first-order visual thalamus (the lateral geniculate nucleus, LGN), and the first-order somatosensory thalamus (the ventral posteromedial nucleus, VPM) of anaesthetized rats (Fig. 3A). Data are made available with this NeurIPS submission as Supplemental Material. Full details (including details of approvals from Ethical Committees and Local Authorities) are reported in the original publication. Here we summarize them briefly.
These data were recorded from N=6 rats (using one-shank Silicon Michigan probes, Neuronexus Technologies;  $100 - \mu \mathrm{m}$  intersite spacing) in three stimulation conditions: visual stimulation, whiskers tactile stimulation and bimodal stimulation (simultaneous visual and tactile). All experiments were conducted under urethane anesthesia. The visual stimuli consisted of a light flash (50-ms-long LED light flashes at 300 lux). The unimodal somatosensory stimulus consisted of a whisker deflection. For bimodal stimulation, whisker deflection and light flashes were applied in the same hemifield. Stimuli were randomly presented across trials. In our analysis, we considered only stimulation contra-lateral to the recorded brain areas. Each type of stimulus was presented 100 times. The non-stimulated eye was covered with an aluminum foil patch. Neural activity was recorded at a sampling rate of  $32\mathrm{kHz}$ , bandpass filtered  $(0.1\mathrm{Hz}$  and  $5\mathrm{kHz})$  then down-sampled to  $8\mathrm{kHz}$ . In the current work, we used the recordings from infragranular layers of S1 and V1 and from VPM and LGN. Multi-unit spike times were first detected from the band-passed  $(400 - 3,000\mathrm{Hz}$ , fourth-order IIR Butterworth Filter) extracellular potential in each electrode by threshold crossing ( $>3$  SD). A spikes train was obtained for all channels using a temporal binning of  $0.125\mathrm{ms}$ $(1/8\mathrm{kHz})$ . For each brain region, spiking activity was then pooled together using all recorded spikes from all electrodes related to that region.
SM3.3.2 Parameters and details of the information analyses
To compute mutual information and FIT, we defined two different stimulus set of interest. To measure information related to tactile discrimination, we used a "tactile-discriminative set" made of the unimodal visual and the bimodal visual-tactile stimulus (the two stimuli in the set are discriminated by the presence or absence of a tactile stimulus). Similarly, to measure information related to visual discrimination, we used a "visual-discriminative set" made of unimodal tactile and the bimodal
visual-tactile stimulus (the two stimuli in the set are discriminated by the presence or absence of a visual stimulus).
As stimulus feature for the computation of mutual information and FIT for the tactile (visual) discriminative set, we used a binary value indicating either the delivery of a visual (tactile) unimodal or a bimodal stimulation. We computed the information quantities for all combinations of directionality of flow across the visual (LGN to V1 and V1 to LGN) and somatosensory (VPM to S1 and S1 to VPM) thalamo-cortical pathways and stimulus-discriminative sets (visual or tactile). To provide a quantification values of TE and FIT across animals (Fig. S11E,F,H,I), we selected a rectangular region in the time-delay domain to select the TE and FIT values for the across animals statistics, centered around the FIT peaks about the tactile stimulus in the VPM to S1 direction (for the somatosensory pathway) and about the visual stimulus in the LGN to V1 direction (for the visual pathway). We computed the average over delays and then picked the maximum over time within this region. This gave us one single TE and one single FIT value for each animal. The comparisons of values across animals was performed using two-tailed paired t-tests.