# The rhythms of predictive coding? Pre-stimulus phase modulates the influence of shape perception on luminance judgments

Biao Han, Rufin Vanrullen

► To cite this version:

Biao Han, Rufin Vanrullen. The rhythms of predictive coding? Pre-stimulus phase modulates the influence of shape perception on luminance judgments. Scientific Reports, 2017, 7 (1), 10.1038/srep43573. hal-02342020

HAL Id: hal-02342020

https://hal.science/hal-02342020v1

Submitted on 6 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
SCIENTIFIC REPORTS

<figure><img src="image_2.png" /></figure>

OPEN

# The rhythms of predictive coding?
## Pre-stimulus phase modulates the influence of shape perception on luminance judgments

Received: 19 October 2016

Accepted: 25 January 2017

Published: 06 March 2017

Biao Han<fnref n="1" />,² & Rufin VanRullen<fnref n="1" />,²

Predictive coding is an influential model emphasizing interactions between feedforward and feedback signals. Here, we investigated the temporal dynamics of these interactions. Two gray disks with different versions of the same stimulus, one enabling predictive feedback (a 3D-shape) and one impeding it (random-lines), were simultaneously presented on the left and right of fixation. Human subjects judged the luminance of the two disks while EEG was recorded. The choice of 3D-shape or random-lines as the brighter disk was used to assess the influence of feedback signals on sensory processing in each trial (i.e., as a measure of post-stimulus predictive coding efficiency). Independently of the spatial response (left/right), we found that this choice fluctuated along with the pre-stimulus phase of two spontaneous oscillations: a ~5 Hz oscillation in contralateral frontal electrodes and a ~16 Hz oscillation in contralateral occipital electrodes. This pattern of results demonstrates that predictive coding is a rhythmic process, and suggests that it could take advantage of faster oscillations in low-level areas and slower oscillations in high-level areas.

The outside world provides us only the light, but our visual system is capable of extracting the basic features in low-level areas and understanding them as meaningful concepts in high-level areas. Interactions between these different areas involve both feedforward and feedback processes. However, the underlying mechanisms in these dynamical processes, especially in the temporal domain, are not fully understood.

Predictive coding is a model emphasizing interactions between feedforward and feedback signals. Specifically, it suggests that the brain employs an efficient coding strategy for these interactions by generating predictions in higher-level areas and comparing them with the incoming sensory signals in the lower-level areas<fnref n="1" />. Previous neuroimaging evidence revealed the existence of such two-way communication²; however, different computational models of predictive coding have suggested very different details of implementation<fnref n="1" />,³⁻⁵.

It has been proposed that the feed-forward and feedback in predictive coding could take advantage of oscillations for information processing⁶,⁷. On the one hand, recent neurophysiological evidence on laminar-specific oscillations and the functional roles of different layers suggested a faster oscillation for the feed-forward pathway and a slower oscillation for the feedback pathway⁶,⁸⁻¹¹. On the other hand, recent studies showed a link between behavioral performance and cortical oscillations in perception¹²,¹³ and reaction time¹⁴,¹⁵. Since neural oscillations can reflect the cyclic fluctuations of excitability in a network, investigation of the relationship between trial-to-trial variability and the phase of ongoing oscillations could link specific oscillations to cognitive functions (e.g. attention).

Here, we used this approach to investigate the specific influence of ongoing oscillations on the feed-forward/ feedback interactions in predictive coding, by measuring their effect on perception for different pre-stimulus oscillatory phases. A minimal definition of “predictive coding” encompasses any stimulus-driven higher-level activity that modulates, in a feedback manner, the lower-level sensory processing of the same stimulus. In a typical predictive coding experiment, therefore, two conditions must be compared, which differ in the amount or the

<footnote n="1">¹Centre National de la Recherche Scientifique, Unité Mixte de Recherche 5549, Faculté de Médecine de Purpan, 31052, Toulouse Cedex, France. ²Université de Toulouse, Centre de Recherche Cerveau et Cognition, Université Paul Sabatier, 31062, Toulouse, France. Correspondence and requests for materials should be addressed to R.V.R. (email: rufin.vanrullen@cnrs.fr)</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Illustration of the experimental paradigm. In each run of trials, a blank screen with only a central fixation point was presented for 1000 to 1500 ms randomly. Then, two circular gray disks, one with a 3D-shape stimulus in the center and the other with random-lines, were presented randomly on either side (left or right) of the fixation point for 150 ms. Subsequently, a question mark appeared in the center of the screen. Subjects were instructed to fixate the fixation point all the time, and report the side of the brighter disk with the corresponding arrow key after the question mark appeared. In the main experimental trials, luminance values of the disks were adjusted to obtain a 50% selection probability of 3D-shape/random-lines disk. In addition, there were 15% catch trials intermixed with the main experimental trials to monitor the subjects' ability to judge the luminance difference throughout the experiment. In these catch trials, one disk was 20% brighter/darker than in the main experimental trials.</figcaption></figure>

type of predictive feedback signals. Here, we chose one of the first experimental manipulations used in predictive coding to generate different amounts of predictive feedback¹⁶: shape perception.

Specifically, 3D-shape outlines and random-lines versions of the same stimuli, similar to the stimuli used in a previous influential study¹⁶, were used in this experiment. It has been shown that 3D-shape outlines can be easily recognized (presumably activating higher-level areas) and thus produce more (or at least, different) predictive feedback than the random-lines versions¹⁶. To measure the effect of different amounts of predictive feedback, we asked the subjects to judge the luminance (report the side of the brighter disk) of two gray disks simultaneously displayed on the left and right of fixation on a black background, one containing the 3D-shape outlines and the other containing the random-lines version. Our previous study showed that the 3D-shape (which generates more predictive feedback¹⁶) increases the subjective luminance of the disk relative to the random-lines version¹⁷. Thus, perceived luminance can be used as a marker for the efficiency of predictive feedback on every trial. Here, the luminance of the disks was adjusted to achieve about 50% choice rate for 3D-shape vs. random-lines disks for all trials (this was achieved by slightly increasing the luminance of the random-lines disk). With our knowledge of the predictive feedback's effect on luminance judgements¹⁷, we can predict that in trials where predictive feedback efficiency is higher than average, subjects would be more likely to report the 3D-shape disk as brighter; and *vice versa*, when predictive feedback efficiency is lower than average, subjects would more likely report the random-lines disk as brighter. We recorded EEG signals and analyzed the relationship between pre-stimulus oscillation phase and the post-stimulus judgment. If predictive feedback mechanisms during the post-stimulus stage involved one or more periodic processes, then the oscillatory state of the system just before stimulus onset (as reflected in pre-stimulus oscillatory phase) should have an influence on the outcome of predictive feedback, and thus on the perceptual judgment. We found that, independent from the spatial choice (left/right side), the phase of 5 Hz contralateral frontal and 16 Hz contralateral occipital pre-stimulus oscillations modulated the subject's choice of a brighter 3D-shape disk (more effective predictive feedback) or a brighter random-lines disk (less effective predictive feedback). Since in the predictive coding framework, higher hierarchical level areas are assumed to send predictive feedback and lower hierarchical level areas to send prediction error signals, our results could imply that in the post-stimulus stage, the brain sends predictive feedback periodically at a preferred phase of a theta frequency oscillation in the frontal region, and sends prediction errors periodically at a preferred phase of a beta frequency oscillation in the occipital region.

## Results

Human observers judged the luminance of two disks that were presented for 150 ms on the left and right of a central fixation point. The disks contained different versions of the same stimulus, one with a 3D-shape enabling predictive feedback, and the other with a random-lines version of the same shape which impeded predictive feedback (Fig. 1). Before the stimulus onset there was a random period of time (1000–1500 ms) with only the fixation point on the screen. After the stimulus offset a question mark appeared at the center and the subjects were instructed to report the side with the brighter disk. In the main experimental trials, the luminance of the
disks was adjusted, such that observers reported the 3D-shape disk as brighter in half of the trials. 15% of trials were catch trials: extreme luminance values were assigned to one disk to monitor the subject's ability to judge the luminance difference.

**Behavioral Results.** On average, subjects judged the 3D-shape disk as brighter in half of the trials (49.24% ± 1.57%, mean ± standard error of the mean, SEM) in the main experimental condition, as expected. The luminance judgment correct rate in the catch trials (subjects judged the disk with higher luminance value as brighter or judged the disk with lower luminance value as darker) was high (93.98% ± 1.83%, mean ± SEM), indicating that subjects were adequately engaged in the luminance judgment task.

**Electrophysiological Results.** We focused on the relationship between the oscillatory phase and the trial-by-trial variations in the efficiency of post-stimulus predictive coding. EEG was recorded during the experiment. Regarding the expected time window of such a relationship, the post-stimulus phase information is driven to a large extent by stimulus-locked activity (e.g. evoked potentials) and is thus further removed from spontaneous activity. On the other hand, since a spontaneous oscillation is a continuous process, pre-stimulus phase information is likely to be related to the oscillatory phase at the (post-stimulus) time point at which predictive coding actually happens. Therefore, the pre-stimulus phase can be a better measurement of the relevant state of a spontaneous oscillation than the post-stimulus phase. Thus, we expected the relation between oscillatory phase and behavior to be most visible in the pre-stimulus time window, where phase information reflects spontaneous fluctuations in neuronal excitability. We used classical stimuli¹⁶ for inducing different amounts of predictive feedback on the left and right of the screen: 3D-shape and random-lines versions of the same stimuli (Fig. 1; the 3D-shape version enabling predictive feedback, the random-lines simultaneously impeding it, as described in Murray *et al*.¹⁶). To measure the effective amount of post-stimulus predictive feedback on each trial, we probed the perceived luminance of the disks under the stimuli. We have previously demonstrated that the 3D-shape (producing more predictive feedback, as described by ref. 16) results in a relatively higher perceived luminance than the random-lines¹⁷. Here, this net effect was compensated on each trial by slightly lowering the luminance of that disk so that the average likelihood of perceiving either disk brighter was about 50% (see Methods). Therefore, residual fluctuations of luminance perception on every trial can be thought to arise from trial-by-trial fluctuations in the efficiency of post-stimulus predictive coding: the 3D-shape disk may still be perceived brighter on trials where predictive coding was more efficient than average, and darker on trials where it was less efficient than average. Of course, spatial bias and/or trial-by-trial fluctuations in the direction of spatial attention may well also contribute to the choice of which disk appears brighter on a given trial. Thus, for each subject we divided all trials into two datasets based on their spatial choice (left-side choice vs. right-side choice), and we only investigated the relation between pre-stimulus EEG phase and 3D-shape/random-lines choice within each dataset. As the correlates of choosing the prediction-consistent stimulus (3D-shape) were expected to be strongest on electrodes contralateral to that stimulus, which would map onto opposite hemispheres for the two datasets, before plotting any scalp topographies we permuted the electrode positions (symmetrically across the midline axis) of all right-side choice trials. This procedure resulted in a mapping of ipsilateral effects to the spatial choice onto left electrodes, and contralateral effects onto right electrodes.

We estimated the relation between EEG phase and predictive coding via the phase opposition product (POP, see Methods). This measure should be maximal when 3D-shape choice trials (prediction-consistent) and random-lines choice trials (prediction-inconsistent) tend to have opposite phase values. For each subject, dataset, electrode, time point and oscillatory frequency, we obtained surrogate POP values (80,000,000 surrogates) by randomly permuting the trial outcomes, keeping the number of trials constant. Both real and surrogate POP values were averaged across datasets and subjects. The significance was determined as the proportion of surrogate POP values that were more extreme than the observed value. P-values were corrected for multiple comparisons across time points, frequencies and electrodes (100 × 30 × 64) using the FDR method (FDR α = 0.05, corresponding to a P value threshold of 9.53 × 10⁻⁶). To show the overall POP in the time-frequency domain, a z-score was computed by comparing the real POP values (combined across all subjects, datasets, and electrodes) to the mean and standard deviation of a null-hypothesis distribution with 10,000 surrogate POP values (generated using the same procedure described before, and also combined across all subjects, datasets, and electrodes). This analysis revealed a significant phase-behavior relation between the post-stimulus 3D-shape/random-lines choice and two pre-stimulus oscillations (Fig. 2A): one theta-frequency oscillation (~3.1 Hz to 7.6 Hz) in the time window from −545 ms to −268 ms, and one beta-frequency oscillation (~13.2 Hz to 25.7 Hz) in the time window from −107 ms to −25 ms. Green outlines mark the significant time-frequency regions (at least one significant electrode) after FDR correction.

Scalp topographies of the z-score show that the two oscillatory effects involve distinct electrode groups and presumably distinct brain regions (Fig. 2B and C): the theta-frequency effect is maximal over frontal regions and the beta-frequency effect over occipital regions. In both cases, these effects are contralateral to the side that subjects chose as “brighter” (i.e., the right side of the topographies, due to our electrode permutation procedure). Electrodes with at least one significant time-frequency point (after FDR correction) inside the corresponding time-frequency window are highlighted in green.

To quantify the influence of pre-stimulus oscillations on post-stimulus choice, we binned single trials according to the phase at the optimal time-frequency point (for the theta oscillation: −397 ms, 5.4 Hz; for the beta oscillation: −68 ms, 16.5 Hz). Single trials were thus sorted in 13 phase bins based on the average phase of the significant electrodes for each oscillation (four frontal electrodes for the theta oscillation, three occipital electrodes for the beta oscillation). For each phase bin we then computed the post-stimulus choice probability of the 3D-shape disk. These choice probabilities were normalized by dividing them by the overall 3D-shape choice probability across all phase bins. For each experimental dataset (left- vs. right-side choice), phase bins were rotated
<figure><img src="image_4.png" /><figcaption>**Figure 2.** Pre-stimulus EEG phase predicts luminance judgment of 3D-shape disk vs. random-lines disk.</figcaption></figure>

(A) The relation between pre-stimulus phase and luminance judgment (our measure of predictive coding) is assessed using POP values (phase opposition product; see details in Methods). The time-frequency map is the z-score of observed POP values (combined across all subjects, datasets and electrodes), each value compared with a null-hypothesis distribution of 10,000 surrogate POP values (also combined across all subjects, datasets and electrodes) characterized by its mean and SD. Time 0 indicates stimulus onset. P-values were derived from a comparison of POP values against 80,000,000 surrogates, and corrected for multiple comparisons across all time points, frequencies and electrodes using the FDR method (FDR α = 0.05, corresponding to a P value threshold of $9.53 \times 10^{-6}$). The green outlines mark the significant time-frequency regions (at least one significant electrode) after FDR correction. A significant relation is apparent between the effect of shape perception on luminance judgments and the EEG phase of ~5 Hz and ~16 Hz pre-stimulus oscillations. (B) Scalp topography of (z-scored) POP values around 16 Hz (frequency range 13.2 to 25.7 Hz; time range -107 to -25 ms). Electrodes highlighted in green have at least one significant time-frequency point (after FDR correction) inside the corresponding red box. Due to our electrode-permutation procedure, in this topography the electrodes ipsilateral to the spatial choice are displayed on the left and those contralateral to the spatial choice are displayed on the right. Thus, the topography shows a contralateral occipital effect for the 16 Hz oscillation. (C) Same as B, but for the ~5 Hz oscillations (frequency range 3.1 to 7.6 Hz; time range -545 to -268 ms). The topography shows a contralateral frontal effect for the 5 Hz oscillation.

such that the phase at which 3D-shape disk choice probability was largest was aligned to a phase angle of zero. As a result of this alignment, the 3D-shape choice probability is necessarily maximal at a phase angle of zero; therefore, the zero-phase bin was discarded from further analyses. For both frequencies, the 3D-shape disk choice probability monotonically decreased to a minimum at the opposite phase angle, confirming that pre-stimulus phase affected post-stimulus judgment (Fig. 3). A one-way ANOVA showed that both pre-stimulus theta phase and pre-stimulus beta phase significantly modulated the 3D-shape disk choice probability (for theta oscillation, $F_{(11, 27)} = 3.95, p = 2.23 \times 10^{-5}$; for beta oscillation $F_{(11, 27)} = 6.17, p = 3.86 \times 10^{-9}$). The magnitude of each effect was determined as the difference between the maximum and minimum 3D-shape disk choice probabilities across all phase bins. The frontal theta oscillation accounted for a difference of ~14% of the 3D-shape disk choice probability between phase bins, and the occipital beta oscillation accounted for a difference of ~19%.

Because the time-frequency analysis relies on signal convolution with wavelet filters whose duration is non-negligible, one might wonder whether the observed pre-stimulus phase differences could actually be driven by stimulus-evoked activity. For example, at 16 Hz the above time-frequency analysis used a 250 ms time window (4 cycles, 125 ms from the past and 125 ms into the future); thus, significant phase effects observed at -67 ms pre-stimulus may be contaminated by post-stimulus activity. To rule out such contamination, we repeated the POP time-frequency analysis with one-cycle wavelets at all frequencies, and compared the timing of pre-stimulus phase effects with the time-frequency region of possible post-stimulus contamination, determined using the wavelet window length at each frequency (Fig. 4). Both theta- and beta-frequency phase effects were replicated in this analysis, and were found to lie outside of the possible post-stimulus contamination zone.

We also ascertained that phase effects were not caused by any eye movement artifacts that may have survived our artifact rejection procedure. For example, the observed pre-stimulus phase differences could be thought to reflect different patterns of eye blink or saccades for different perceptual outcomes. Therefore, we applied our POP time-frequency analysis to the horizontal and vertical EOG (HEOG and VEOG) signals. P-value maps (obtained
<figure><img src="image_5.png" /><figcaption>Figure 3. Normalized choice probability of 3D-shape disks as a function of pre-stimulus phase.</figcaption></figure>

(A) Relationship between frontal pre-stimulus theta phase and choice of 3D-shape disk as the brighter disk. Single trials were binned into 13 bins, centered on the maximal phase bin for each subject (the central value is not displayed because it is not interpretable due to the realignment). The curve indicates that the oscillatory phase of frontal electrodes (4 significant electrodes shown in inset topography) at 5.4 Hz and –397 ms (relative to stimulus onset) modulates the luminance judgment by ~14%. Error bars represent SEM across subjects.

(B) Same as A, but for the occipital beta pre-stimulus phase. The oscillatory phase of occipital electrodes (3 significant electrodes shown in inset topography) at 16.5 Hz and –68 ms (relative to stimulus onset) modulates the luminance judgment by ~19%.

<figure><img src="image_6.png" /><figcaption>Figure 4. Significance of POP values in a one-cycle wavelet analysis. (A) P-value map of the POP values combined across previously identified frontal electrodes (green points in inset topography) for the theta-frequency phase effect. The P-values were calculated by comparing the observed POP values with 80,000,000 surrogates. The semi-transparent red area on the time-frequency map indicates the zone of possible contamination by post-stimulus activity (based on the wavelet window length at each frequency, centered on the time of stimulus onset, 0 ms). The previously observed theta-frequency phase effect lies outside of the contamination zone. The P-values here are uncorrected for multiple comparison. (B) Same as A, but for the beta-frequency phase effect. The beta-frequency phase effect also lies outside the contamination zone. Altogether, these findings indicate that pre-stimulus phase differences are not caused by post-stimulus evoked activity.</figcaption></figure>

by comparison of POP values against 80,000,000 surrogates) did not reveal any signs of systematic eye movements in either the theta- or the beta-frequency bands (Fig. 5), ruling out an explanation of our pre-stimulus phase effects in terms of ocular artifacts.
<figure><img src="image_7.png" /><figcaption>**Figure 5.** Significance of the POP values of VEOG and HEOG signals. (A) P-value map of the POP values for the VEOG. The P-values were calculated using a similar procedure as in the main analysis: comparing the observed POP values with 80,000,000 surrogate POP values. (B) Same as A, but for the HEOG. There was no significant pre-stimulus time-frequency window with significant POP values in either VEOG or HEOG, indicating that the observed phase effects were not due to ocular artifacts.</figcaption></figure>

Discussion

We investigated the temporal dynamics of predictive coding by exploring the relation between pre-stimulus oscil-
latory phase and the presumed trial-by-trial variations in post-stimulus predictive feedback. We used 3D-shape
outlines and random-lines versions of the same stimuli (as in one of the seminal predictive coding studies¹⁶)
to induce different amounts of predictive feedback (Fig. 1), and measured the corresponding effects on lumi-
nance judgment as trial-by-trial markers of the efficiency of predictive coding. (To reiterate our experimental
logic: Feedforward processing of the 3D shape likely induced predictive feedback, which increased the per-
ceived brightness of the 3D-shape disk. The more efficient this predictive coding, the larger the effect of pre-
dictive feedback, and the brighter the 3D-shape disk should be perceived.) Using a similar analysis method as
in well-established studies of the relationship between pre-stimulus phase and post-stimulus behavior¹⁸,¹⁹, we
found that two pre-stimulus ongoing oscillations from different regions and frequencies could strongly influence
the post-stimulus luminance judgment, and thus the post-stimulus predictive coding efficiency: a contralateral
frontal theta oscillation and a contralateral occipital beta oscillation (Fig. 2). The phase of the theta oscillation
before stimulus onset could explain 14% of the luminance judgment difference while the phase of the beta oscil-
lation could explain 19% (Fig. 3). Control analyses ruled out contamination of the phase-behavior relationship by
post-stimulus activity (Fig. 4) or ocular artifacts (Fig. 5). These results not only imply that the feedforward/feed-
back interactions in predictive coding involve periodic processes, but also reveal two periodicities with different
sources. Since the occipital and frontal signals likely reflect activity from hierarchically lower and higher areas,
respectively, and since predictive coding theory suggests that the brain sends back predictions from higher areas
and sends prediction errors from lower areas, our results allow us to speculate on a possible temporal dynamic for
predictive coding: predictions sent periodically at a theta frequency, prediction errors sent periodically at a beta
frequency. Of course, more direct connectivity studies would be needed in the future to confirm these postulated
roles for occipital and frontal signals.

The experimental paradigm used in this study takes advantage of the relationship between shape percep-
tion and predictive coding: 3D-shape outlines are assumed to generate more predictive feedback than the
random-lines version of the same stimulus. Murray *et al*.¹⁶ used similar stimuli to provide one of the first evidence
of predictive coding: compared to the random-lines, 3D-shape outlines increased activity in the lateral occipital
complex (LOC), but decreased it in primary visual cortex (V1), suggesting an increase of predictive feedback
accompanied by a decrease in prediction errors¹⁶,²⁰. Here, we used the same paired stimuli as in the original
study, placed them on two gray disks and asked subjects to judge the luminance of the disks. This luminance
judgment, associated with perceived contrast, is likely to have a positive and monotonic relationship with neural
activity in early visual cortex²¹⁻²⁴. Thus, the variability in luminance judgment associated with the 3D-shape vs.
random-lines stimuli could reflect trial-by-trial changes in the effect of predictive feedback on neural activity in
early visual cortex.

Previous fMRI studies showed that shape perception could not only reduce¹⁶, but also up-regulate neural activity in V1²⁵. Our own previous study found that 3D-shape disks were generally perceived brighter than random-lines disks, and that this effect could be attributed to predictive coding rather than attentional biases¹⁷. In the present study, we compensated for this net effect by adjusting the disks' luminance to obtain a 50% selection probability of 3D-shape/random-lines disks. (This operation suggested that predictive feedback has an excitatory effect on lower level activity, which is in contrast to many predictive coding models. This apparent discrepancy was already extensively discussed in our previous paper¹⁷. For the present purposes, it is sufficient to note that, if predictive coding is a rhythmic process, both an excitatory and a suppressive main effect of predictive coding would produce the same outcome: a periodic modulation of contrast judgments.) We then focused on the remaining variability in luminance judgement as a trial-by-trial marker of the efficiency of predictive coding. On the other hand, systematic spatial biases (e.g. a general tendency to respond to the left or right stimulus)
and/or trial-by-trial fluctuations in the direction of spatial attention can also be expected to affect the luminance judgement²⁶. As a matter of fact, spatial attention itself appears to involve a periodic process¹²,¹⁸,²⁷,²⁸ which could potentially influence the luminance judgement. We carefully avoided these potential confounding factors by dividing the trials into two datasets based on the post-stimulus spatial response (left/right) and performing the analysis within each dataset. If spatial attention biases, for example, were the only cause of the perceived luminance changes, the left-response dataset would pool all trials with a left-side attention bias (and similarly for the right-response dataset), and within each dataset pre-stimulus oscillatory phases would not bear any relation to post-stimulus luminance judgments. Of course, this analysis controls for spatial biases and changes in spatial attention, but cannot rule out object-based attention related to the shape of the object; however, it could be argued that this type of object-based attention is indistinguishable from (or even, identical to) the predictive feedback arising from the predictive coding framework. To sum up, the existence of significant phase-behavior relationships in our analysis can therefore be safely attributed to predictive coding mechanisms, rather than spatial attention or other spatial biases.

Neurophysiological recordings have shown that feedforward and feedback may take advantage of oscillations in different frequencies. Laminar recordings showed that high-frequency oscillations are prominently generated in superficial layers and low-frequency oscillations in deep layers⁹,¹⁰,²⁹. Since superficial and deep layers correspond respectively to the main sources of forward and feedback projections³⁰⁻³², it follows that feedforward communication takes advantage of high-frequency oscillations and feedback takes advantage of low-frequency oscillations. A recent study with simultaneous recordings and micro-stimulation in different layers in V1 and V4 confirmed this notion⁸. Several authors have independently proposed that low-frequency oscillations send predictions via feedback, while high-frequency oscillations send prediction errors via feedforward⁶,¹¹,³³⁻³⁶. Even though the exact frequency bands (theta, beta) are not entirely consistent with those described in the previous studies (e.g. beta, gamma in ref. 11), our results still provide support for distinct functional roles of low and high frequency oscillations in predictive coding at the EEG and behavioral level.

Our results also provide supportive evidence for the hypothesized functions of frontal theta-band and occipital beta-band oscillations. Our conclusions are in line with the notion that 5–10 Hz oscillations could contribute to “top-down” control³⁷,³⁸, which has already been suggested based on attentional phase effects on perception¹²,¹⁸, reaction time¹⁴,¹⁵,³⁹ and perceptual variability in TMS-induced effects¹³,⁴⁰. We found the origin of such theta periodicity in contralateral frontal electrodes, compatible with the involvement of frontal areas in the top-down controlling process²,⁴¹,⁴² and with the involvement of 5–10 Hz oscillations in this region⁴³. On the other hand, local field potential (LFP) recordings showed that, in mammalian visual cortex, beta frequency oscillations are also prominent during the deployment of top-down control⁴⁴⁻⁴⁸. Our findings of beta frequency phase effects on predictive feedback in the occipital area are concordant with such LFP results and suggest a valuable role for the beta frequency oscillations in predictive coding.

In summary, we measured the relation between pre-stimulus oscillations and a predictive feedback-induced effect to investigate the neural oscillations involved in predictive coding. We found that the pre-stimulus phases of frontal theta-frequency oscillations and occipital beta-frequency oscillations jointly determine post-stimulus subjective judgments. These results shed light on the temporal dynamics of predictive coding, and suggest a periodic predictive coding process with faster oscillations in lower areas and slower oscillations in higher areas.

## Materials and Methods

**Subjects.** Fifteen volunteers participated in the experiment. One participant was excluded from the analysis due to the poor behavioral performance in catch trials (<60% trials were correctly reported, with a chance level of 50%, see below). Fourteen participants remained in the sample (8 females, mean age 28.01 ± 4.81 years, four left-handed, four with left eye dominance). All subjects had normal or corrected to normal vision. The study was approved by the local ethics committee “Sud-Ouest et Outre-Mer I” and followed the Code of Ethics of the World Medical Association (Declaration of Helsinki). All subjects provided signed informed consent before starting the experiments.

**Apparatus.** Stimuli were presented at 57 cm distance using a desktop computer (2.09 GHz Intel processor, Windows XP) with a cathode ray monitor (resolution: 800 × 600 pixels; refresh rate: 140 Hz, Gamma corrected luminance function). Stimuli were designed and presented via the Psychophysics Toolbox (Brainard, 1997) running in MATLAB (MathWorks).

**Stimuli and tasks.** Stimuli consisted of a central white fixation point (diameter: 0.2 degrees of visual angle) and two circular gray disks (diameter: 4 degrees each) presented randomly to the left and right of fixation (3 degrees eccentricity). One 3D-shape stimulus was in the center of one disk (3D-shape disk) and one random-lines version of the same stimulus in the other (random-lines disk). The 3D-shape and random-lines stimulus pair was randomly chosen from twenty pairs of stimuli generated beforehand using a method similar to Murray *et al*.¹⁶: 3D-shapes were generated by randomly selecting 4–6 vertices, connecting the vertices and adding small extensions to render perceived depth; random-lines stimuli were created by breaking the 3D-shape at its intersections and randomly shifting the lines within the display¹⁶. The diameter of both 3D-shape and random-lines stimuli was 3 degrees. The stimulus outlines were black.

Before stimulus onset, there was a blank screen with only the fixation point that lasted from 1000 to 1500 ms (random uniform distribution). Then the two disks and the fixation point appeared for 150 ms. After that, a question mark appeared in the center of the screen. There were two kinds of randomly mixed experimental trials: the main experimental trials and the catch trials. In main experimental trials, the luminance of the disks was adjusted (i.e. the random-line disks were set 1.45% brighter than the 3D-shape disks) based on a previous study¹⁷ to obtain an average 50% selection rate of 3D-shape/random-lines disks. In catch trials, one of the disks had its luminance
value changed up or down by 20% compared to the luminance used in the main experimental trials, while the other disk kept the same luminance as in the main experimental trials. Subjects were presented with 4 or 8 blocks of 200 trials with 85% main experimental trials and 15% catch trials (the first 6 of the 14 subjects performed only 4 blocks of the present experiment, together with 4 blocks of another experiment that was eventually canceled and whose data were not analyzed). Subjects were instructed to fixate the fixation point all the time, judge the luminance of the disks and respond using the arrow keys (left arrow to indicate that left disk is brighter, right arrow for right disk brighter) on a standard 105 key keyboard when the question mark appeared. There was no feedback after the response.

**EEG data acquisition and analysis.** EEG was recorded at 1024 Hz using a Biosemi system (64 active electrodes). Horizontal and vertical electro-oculograms (EOG) were recorded by three additional electrodes around the subjects' eyes. For data pre-processing, the EEG and EOG data were downsampled offline to 256 Hz, re-referenced to average reference and epoched around the stimulus onset in each trial for data analysis via the MATLAB (MathWorks) and EEGLAB toolbox⁴⁹. Individual electrode data were visually inspected, and channel data containing artifacts were interpolated by the mean of adjacent electrodes (three subjects had one electrode containing artifacts, one subject had two; the positions of the interpolated electrodes were different across subjects).

As the post-stimulus spatial choice was lateralized on each trial to the left or right side, the pre-stimulus oscillatory correlates of the post-stimulus luminance judgment may not only reflect the oscillation's influence on shape perception and predictive coding, but also its influence on spatial choice (i.e., pre-stimulus oscillations may bias the left/right spatial choice independently of the 3D-shape/random-lines content inside of the disk). To avoid any contribution from the spatial choice, we first divided the trials for each subject into two trial datasets based on the post-stimulus spatial choice, and performed the time-frequency analysis (described below) within each dataset. We reasoned that this analysis would lead to shape perception correlates not on a given fixed set of electrodes, but rather on different electrode groups depending on the side of choice (i.e., electrodes "contralateral" or "ipsilateral" to the spatial choice). Therefore, we arbitrarily chose to permute the electrode locations for the dataset corresponding to a right-side choice: we replaced the left-hemisphere electrodes by the symmetric ones from the right and vice versa (midline electrodes were unaffected). With this new electrode assignment, left-hemisphere electrodes would thus always correspond to those ipsilateral to the spatial choice, and right-hemisphere electrodes to contralateral ones.

For the time-frequency analysis, time-frequency transformations were first generated over all channels using EEGLAB with a function akin to a wavelet transform, starting with 3 cycles at 2 Hz and increasing to 5 cycles at 50 Hz in the multiple-cycle analysis, and with 1 cycle from 2 Hz to 50 Hz in the one-cycle analysis. This yields a complex representation of the amplitude, A, and the phase, φ, for trial j at time t and frequency f:

$$tf_j(t, f) = A_j(t, f)e^{i\varphi_j(t, f)} \quad (1)$$

The phase of this representation can be extracted by normalizing the complex vector to the unit length:

$$\Phi_j(t, f) = \frac{tf_j(t, f)}{|tf_j(t, f)|} \quad (2)$$

Inter-trial phase coherence (ITPC) measures the phase consistency across trials. We calculated the ITPC using the method described previously⁵⁰:

$$ITPC(t, f) = \frac{1}{N} \left| \sum_{j=1}^{N} \Phi_j(t, f) \right| \quad (3)$$

where N is the number of trials in one group of trials. Note that the ITPC is a scalar rather than a vector and it is always non-negative.

Here, we wanted to evaluate the relation between the pre-stimulus oscillatory phase and the influence of shape perception on luminance judgment (our measure of the efficiency of predictive coding). Would a particular pre-stimulus phase occur more frequently for trials with post-stimulus 3D-shape disk choice, and the opposite phase for trials with post-stimulus random-lines disk choice? In the pre-stimulus period, because intertrial intervals are randomized and unpredictable, the phase of the spontaneous EEG signal at a given pre-stimulus time should follow a uniform distribution over all trials. However, if (and only if) there is a systematic relation between EEG phase and behavioral outcome, higher-than-chance inter-trial phase coherence should be observed in each of the trial subgroups. Indeed, if EEG phases of two subgroups corresponding to different behavioral outcomes tend to oppose each other, the phases of one subgroup will concentrate around one phase angle and the phases of the other subgroup will concentrate around the other phase angle; thus, high ITPC values will be observed for both subgroups. Vice versa, if EEG phases of two subgroups do not oppose each other, the phases of the two subgroups will distribute uniformly across different phase angles; thus, low ITPC values will be observed. Therefore, the ITPC values of the two subgroups are reliable indicators of phase opposition. The product or the sum of the two subgroup inter-trial phase coherence values can therefore summarize, in a single variable, the strength of the phase-behavior relation¹⁸,¹⁹,⁵¹. Here we chose to use the product of the inter-trial phase coherence values, a measure we call *Phase Opposition Product (POP)*:

$$POP = ITPC_a \cdot ITPC_b \quad (4)$$
Again, high ITPC values for the two subgroups (suggesting different phase angles for the different trial subgroups) will result in a high POP value; and two low ITPC values (suggesting random/uniformly distributed phases for the different trial subgroups) will result in a low POP value. Thus, the measure of POP will be maximal when two subgroups show strong inter-trial phase coherence, in other words, strong phase opposition between two subgroups.

To accurately assess the significance of the phase-behavior relation without any assumption about the probability distribution of the POP values, we performed a nonparametric permutation test: We first computed the POP values for each point in the time-frequency plane from -650 to 150 ms, from 2 to 50 Hz for each electrode, dataset, and subject and then averaged across all datasets and all subjects. Surrogate POP values were obtained by randomly assigning the trials to one or the other condition for each subject (keeping the number of trials in each condition constant) and recalculating the grand-average POP values. We computed the P value by simply and directly counting the number of surrogate POP values that were more extreme than the observed value. Here, we used 80,000,000 surrogates and thus assigned the P value of $1.25 \times 10^{-8}$ to the points without any more extreme POP values in the surrogates. The P values were corrected for multiple comparisons over time points, frequencies and electrodes using the FDR method ($FDR \alpha = 0.05$, corresponding to a P value threshold of $9.53 \times 10^{-6}$). To show the overall POP in the time-frequency domain, we computed a z-score by combining the observed POPs across all datasets, subjects, and electrodes and comparing the value with the mean and SD of a null-hypothesis distribution with 10,000 surrogate POP values (generated using the procedure described before, and also combined across all electrodes, subjects, and datasets).

## References

1. Rao, R. P. & Ballard, D. H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. *Nat. Neurosci.* **2**, 79–87 (1999).

2. Summerfield, C. et al. Predictive codes for forthcoming perception in the frontal cortex. *Science* (80–)., **314**, 1311–1314 (2006).

3. Friston, K. A theory of cortical responses. *Philos. Trans. R. Soc. Lond. B. Biol. Sci.* **360**, 815–836 (2005).

4. Spratling, M. W. Reconciling predictive coding and biased competition models of cortical function. *Front. Comput. Neurosci.* **2**, 4 (2008).

5. Spratling, M. W. Predictive Coding as a Model of Response Properties in Cortical Area V1. *J. Neurosci.* **30**, 3531–3543 (2010).

6. Fontolan, L., Morillon, B., Liegeois-Chauvel, C. & Giraud, A.-L. The contribution of frequency-specific activity to hierarchical information processing in the human auditory cortex. *Nat. Commun.* **5**, 4694 (2014).

7. Bastos, A. M. et al. A DCM study of spectral asymmetries in feedforward and feedback connections between visual areas V1 and V4 in the monkey. *Neuroimage* **108**, 460–475 (2015).

8. van Kerkoerle, T. et al. Alpha and gamma oscillations characterize feedback and feedforward processing in monkey visual cortex. *Proc. Natl. Acad. Sci. USA* **111**, 14332–14341 (2014).

9. Buffalo, E. a, Fries, P., Landman, R., Buschman, T. J. & Desimone, R. Laminar differences in gamma and alpha coherence in the ventral stream. *Proc. Natl. Acad. Sci. USA* **108**, 11262–11267 (2011).

10. Maier, A., Adams, G. K., Aura, C. & Leopold, D. A. Distinct superficial and deep laminar domains of activity in the visual cortex during rest and stimulation. *Front. Syst. Neurosci.* **4**, 1–11 (2010).

11. Bastos, A. M. et al. Canonical microcircuits for predictive coding. *Neuron* **76**, 695–711 (2012).

12. Busch, N. A. & VanRullen, R. Spontaneous EEG oscillations reveal periodic sampling of visual attention. *Proc. Natl. Acad. Sci. USA* **107**, 16048–16053 (2010).

13. Dugué, L., Marque, P. & VanRullen, R. The Phase of Ongoing Oscillations Mediates the Causal Relation between Brain Excitation and Visual Perception. *J. Neurosci.* **31**, 11889–11893 (2011).

14. Drewes, J. & VanRullen, R. This Is the Rhythm of Your Eyes: The Phase of Ongoing Electroencephalogram Oscillations Modulates Saccadic Reaction Time. *J Neurosci* **31**, 4698–4708 (2011).

15. Song, K., Meng, M., Chen, L., Zhou, K. & Luo, H. Behavioral Oscillations in Attention: Rhythmic Alpha Pulses Mediated through Theta Band. *J Neurosci* **34**, 4837–4844 (2014).

16. Murray, S. O., Kersten, D., Olshausen, B. A., Schrater, P. & Woods, D. L. Shape perception reduces activity in human primary visual cortex. *Proc. Natl. Acad. Sci. USA* **99**, 15164–15169 (2002).

17. Han, B. & VanRullen, R. Shape perception enhances perceived contrast: evidence for excitatory predictive feedback? *Sci. Rep.* **6**, 22944 (2016).

18. Busch, N. a, Dubois, J. & VanRullen, R. The phase of ongoing EEG oscillations predicts visual perception. *J. Neurosci.* **29**, 7869–7876 (2009).

19. VanRullen, R., Busch, N. A., Drewes, J. & Dubois, J. Ongoing EEG Phase as a Trial-by-Trial Predictor of Perceptual and Attentional Variability. *Front. Psychol.* **2**, 60 (2011).

20. Clark, A. Whatever next? Predictive brains, situated agents, and the future of cognitive science. *Behav. Brain Sci.* **36**, 181–204 (2013).

21. Dean, A. F. The relationship between response amplitude and contrast for cat striate cortical neurones. *J. Physiol.* **318**, 413–427 (1981).

22. Albrecht, D. G. & Hamilton, D. B. Striate cortex of monkey and cat: contrast response function. *J. Neurophysiol.* **48**, 217–237 (1982).

23. Goodyear, B. G. & Menon, R. S. Effect of luminance contrast on BOLD fMRI response in human primary visual areas. *J. Neurophysiol.* **79**, 2204–2207 (1998).

24. Boynton, G. M., Demb, J. B., Glover, G. H. & Heeger, D. J. Neuronal basis of contrast discrimination. *Vision Res.* **39**, 257–269 (1999).

25. Kok, P. & de Lange, F. P. Shape Perception Simultaneously Up- and Downregulates Neural Activity in the Primary Visual Cortex. *Curr. Biol.* **24**, 1531–1535 (2014).

26. Carrasco, M. M. et al. Attention alters appearance. *Nat. Neurosci.* **7**, 308–313 (2004).

27. Landau, A. N. & Fries, P. Attention samples stimuli rhythmically. *Curr. Biol.* **22**, 1000–1004 (2012).

28. Fiebelkorn, I. C., Saalmann, Y. B. & Kastner, S. Rhythmic sampling within and between objects despite sustained attention at a cued location. *Curr. Biol.* **23**, 2553–8 (2013).

29. Roopun, A. K. et al. A beta2-frequency (20–30 Hz) oscillation in nonsynaptic networks of somatosensory cortex. *Proc. Natl. Acad. Sci. USA* **103**, 15646–15650 (2006).

30. Douglas, R. J. & Martin, K. A. C. Neuronal circuits of the neocortex. *Annu. Rev. Neurosci.* **27**, 419–451 (2004).

31. Barbas, H. & Rempel-Clower, N. Cortical structure predicts the pattern of corticocortical connections. *Cereb. Cortex* **7**, 635–646 (1997).

32. Wang, X.-I. Neurophysiological and computational principles of cortical rhythms in cognition. *Physiol. Rev.* **90**, 1195–1268 (2010).

33. Yordanova, J., Kolev, V. & Kirov, R. Brain oscillations and predictive processing. *Front. Psychol.* **3**, 1–2 (2012).

34. Arnal, L. H. & Giraud, A. L. Cortical oscillations and sensory predictions. *Trends Cogn. Sci.* **16**, 390–398 (2012).
35. Bauer, M., Stenner, M. P., Friston, K. J. & Dolan, R. J. Attentional Modulation of Alpha/Beta and Gamma Oscillations Reflect Functionally Distinct Processes. *J Neurosci* **34**, 16117–16125 (2014).

36. Todorovic, A., van Ede, F., Maris, E. & de Lange, F. P. Prior Expectation Mediates Neural Adaptation to Repeated Sounds in the Auditory Cortex: An MEG Study. *J. Neurosci.* **31**, 9118–9123 (2011).

37. Jensen, O., Bonnefond, M. & VanRullen, R. An oscillatory mechanism for prioritizing salient unattended stimuli. *Trends Cogn. Sci.* **16**, 200–206 (2012).

38. VanRullen, R. Visual attention: a rhythmic process? *Curr. Biol.* **23**, R1110–2 (2013).

39. Huang, Y., Chen, L. & Luo, H. Behavioral Oscillation in Priming: Competing Perceptual Predictions Conveyed in Alternating Theta-Band Rhythms. *J Neurosci* **35**, 2830–2837 (2015).

40. Dugué, L., Marque, P. & VanRullen, R. Theta Oscillations Modulate Attentional Search Performance Periodically. *J. Cogn. Neurosci.* **945–958**, doi: 10.1162/jocn (2015).

41. Summerfield, C. & Egner, T. Expectation (and attention) in visual cognition. *Trends Cogn. Sci.* **13**, 403–409 (2009).

42. Summerfield, C. & de Lange, F. P. Expectation in perceptual decision making: neural and computational mechanisms. *Nat. Rev. Neurosci* (2014).

43. Phillips, J. M., Vinck, M., Everling, S. & Womelsdorf, T. A long-range fronto-parietal 5- to 10-Hz network predicts 'top-down' controlled guidance in a task-switch paradigm. *Cereb. Cortex* **24**, 1996–2008 (2014).

44. Lopes da Silva, F. H., van Rotterdam, A., Storm van Leeuwen, W. & Tielen, A. M. Dynamic characteristics of visual evoked potentials in the dog. *II. Beta frequency selectivity in evoked potentials and background activity*. *Electroencephalogr. Clin. Neurophysiol.* **29**, 260–268 (1970).

45. Bekisz, M. & Wróbel, A. Attention-dependent coupling between beta activities recorded in the cat's thalamic and cortical representations of the central visual field. *Eur. J. Neurosci.* **17**, 421–426 (2003).

46. Buschman, T. J. & Miller, E. K. Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices. *Science* (80–). **315**, 1860–1862 (2007).

47. Bosman, C. A. *et al*. Attentional Stimulus Selection through Selective Synchronization between Monkey Visual Areas. *Neuron* **75**, 875–888 (2012).

48. Grothe, I., Neitzel, S. D., Mandon, S. & Kreiter, A. K. Switching neuronal inputs by differential modulations of gamma-band phase-coherence. *J. Neurosci.* **32**, 16172–80 (2012).

49. Delorme, A. & Makeig, S. EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. *J. Neurosci. Methods* **134**, 9–21 (2004).

50. Lachaux, J. P., Rodriguez, E., Martinerie, J. & Varela, F. J. Measuring phase synchrony in brain signals. *Hum. Brain Mapp.* **208**, 194–208 (1999).

51. VanRullen, R. How to Evaluate Phase Differences between Trial Groups in Ongoing Electrophysiological Signals. *Front. Neurosci.* **10**, 426 (2016).

## Acknowledgements

Biao Han is supported by China Scholarship Council; Rufin VanRullen is supported by an ERC Consolidator Grant P-CYCLES number 614244.

## Author Contributions

B.H. and R.V. conceived the experiments, B.H. conducted the experiments, B.H. analysed the results under supervision of R.V. B.H. and R.V. wrote the manuscript.

## Additional Information

**Competing financial interests:** The authors declare no competing financial interests.

**How to cite this article:** Han, B. and VanRullen, R. The rhythms of predictive coding? Pre-stimulus phase modulates the influence of shape perception on luminance judgments. *Sci. Rep.* **7**, 43573; doi: 10.1038/srep43573 (2017).

**Publisher's note:** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_8.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

© The Author(s) 2017