# Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills

Pierre Vassiliadis, Elena Beanato, Traian Popa, Fabienne Windel, Takuya Morishita, Esra Neufeld, Julie Duque, Gerard Derosiere, Maximilian Wessel, Friedhelm Hummel

► To cite this version:

Pierre Vassiliadis, Elena Beanato, Traian Popa, Fabienne Windel, Takuya Morishita, et al. Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills. Nature Human Behaviour, 2024, 8 (8), pp.1581-1598. 10.1038/s41562-024-01901-z. hal-04784976

HAL Id: hal-04784976

https://hal.science/hal-04784976v1

Submitted on 18 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

# Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills

Received: 7 November 2022

Accepted: 23 April 2024

Published online: 29 May 2024

<figure><img src="image_3.png" /></figure>

Check for updates

Pierre Vassiliadis Ⓔ<fnref n="1" />,²,³, Elena Beanato Ⓔ<fnref n="1" />,², Traian Popa Ⓔ<fnref n="1" />,², Fabienne Windel Ⓔ<fnref n="1" />,², Takuya Morishita Ⓔ<fnref n="1" />,², Esra Neufeld Ⓔ⁴, Julie Duque Ⓔ³, Gerard Derosiere Ⓔ³,⁵, Maximilian J. Wessel Ⓔ<fnref n="1" />,²,⁶ & Friedhelm C. Hummel Ⓔ<fnref n="1" />,²,⁷✉

Reinforcement feedback can improve motor learning, but the underlying brain mechanisms remain underexplored. In particular, the causal contribution of specific patterns of oscillatory activity within the human striatum is unknown. To address this question, we exploited a recently developed non-invasive deep brain stimulation technique called transcranial temporal interference stimulation (tTIS) during reinforcement motor learning with concurrent neuroimaging, in a randomized, sham-controlled, double-blind study. Striatal tTIS applied at 80 Hz, but not at 20 Hz, abolished the benefits of reinforcement on motor learning. This effect was related to a selective modulation of neural activity within the striatum. Moreover, 80 Hz, but not 20 Hz, tTIS increased the neuromodulatory influence of the striatum on frontal areas involved in reinforcement motor learning. These results show that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.

The ability to learn from past outcomes, often referred to as reinforcement learning, is fundamental for complex biological systems<fnref n="1" />. Reinforcement learning has been classically studied in the context of decision-making, when agents have to decide among a discrete number of potential options². There is increasing recognition that reinforcement learning processes are also at play in other contexts, including during practice of a new motor skill³⁻⁵. For instance, the addition of reinforcement feedback during motor training can improve motor learning, presumably by boosting the retention of newly acquired motor memories⁶,⁷. Interestingly, reinforcement feedback also appears to be relevant for the rehabilitation of patients suffering from motor impairments⁸⁻¹⁰. Yet, despite these promising results, there is currently

a limited understanding of the brain mechanisms that are critical to implement this behaviour.

A prominent hypothesis in the field is that the striatum, a structure that is particularly active during both reinforcement¹¹ and motor learning¹², may be causally involved in the beneficial effects of reinforcement on motor learning. The striatum shares dense connections with dopaminergic structures of the midbrain as well as with prefrontal and motor cortical regions¹³ and is therefore well positioned to mediate reinforcement motor learning¹⁴⁻¹⁶. This idea is supported by neuroimaging studies showing reward-related activation of the striatum during motor learning¹⁷,¹⁸. More specifically, within the striatum, oscillatory activity in specific frequency bands is suggested to be involved in aspects of

<footnote n="1">¹Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, École Polytechnique Fédérale de Lausanne (EPFL), Geneva, Switzerland. ²Defitech Chair of Clinical Neuroengineering, Neuro-X Institute, EPFL Valais, Clinique Romande de Réadaptation, Sion, Switzerland. ³Institute of Neuroscience, Université Catholique de Louvain, Brussels, Belgium. ⁴Foundation for Research on Information Technologies in Society, Zurich, Switzerland. ⁵Lyon Neuroscience Research Center, Impact Team, Inserm U1028, CNRS UMR5292, Lyon 1 University, Bron, France. ⁶Department of Neurology, University Hospital Würzburg, Würzburg, Germany. ⁷Clinical Neuroscience, University of Geneva Medical School, Geneva, Switzerland. ✉e-mail: friedhelm.hummel@epfl.ch</footnote>
reinforcement processing. Previous rodent studies have shown that striatal high gamma oscillations (-80 Hz) transiently increase following reward delivery¹⁹⁻²³, but not when reward is withheld¹⁹. Hence, dynamic changes of high gamma activity in the striatum¹⁹,²⁴,²⁵ and in other parts of the basal ganglia²⁶,²⁷ may encode the outcome of previous movements (that is, success or failure) and support learning. Consistent with a role of such oscillatory activity in reinforcement learning, high gamma activity in the striatum shows coherence with frontal cortex oscillations and is upregulated by dopaminergic agonists¹⁹. This body of work thus suggests that reinforcement-related modulation of striatal oscillatory activity, especially in the gamma range, may be crucial for reinforcement learning of motor skills. Conversely, striatal beta oscillations (-20 Hz) have been largely associated with sensorimotor functions²⁸. For instance, beta oscillations in the striatum are exacerbated in Parkinson's disease and associated with the severity of motor symptoms²⁹⁻³¹. Consistently, excessive beta connectivity is reduced by anti-Parkinsonian treatment in proportion to the related motor improvement³². Taken together, these elements suggest that striatal high gamma and beta activity may have different functional roles preferentially associated with reinforcement and sensorimotor functions, respectively.

The studies mentioned above provide associative evidence linking the presence of reinforcement with changes of neural activity in the striatum determined through neuroimaging¹⁷,¹⁸, but they do not allow us to draw conclusions regarding its causal role in reinforcement motor learning in humans. The only causal evidence available to date comes from animal work showing modulation of reinforcement-based decision-making with striatal stimulation³³,³⁴. A reason for the current absence of investigations of the causal role of the striatum in human behaviour is related to its deep localization in the brain. Current non-invasive brain stimulation techniques, such as transcranial magnetic stimulation and classical transcranial electric stimulation, do not allow the selective targeting of deep brain regions, because these techniques exhibit a steep depth-focality trade-off³⁵,³⁶. Studies of patients with striatal lesions³⁷,³⁸ or invasive deep brain stimulation of connected nuclei³⁹,⁴⁰ have provided insights into the role of the basal ganglia in reinforcement learning. However, their conclusions are partially limited by the fact that the studied patients also exhibit altered network properties resulting from the underlying pathology (for example, neurodegeneration or lesions) or from the respective compensatory mechanisms. Here we address these challenges by exploiting transcranial temporal interference stimulation (tTIS), a recently introduced non-invasive electric brain stimulation approach allowing us to target deep brain regions in a frequency-specific and focal manner in the physiological state⁴¹,⁴².

The concept of tTIS was initially proposed and validated on the hippocampus of rodents⁴¹ and was then further tested through computational modelling⁴³⁻⁴⁷ and in first applications on cortical areas in humans⁴⁸,⁴⁹. tTIS requires two pairs of electrodes to be placed on the head, each pair delivering a high-frequency alternating current. One key element is that this frequency has to be high enough (that is, in the kHz range) to avoid direct neuronal entrainment, owing to the low-pass-filtering properties of neuronal membranes⁵⁰. The second key element is the application of a small difference of frequency between the two alternating currents. The superposition of the electric fields creates an envelope oscillating at this low-frequency difference, which can be steered towards individual deep brain structures (for example, by optimizing electrodes' placement) and is in a range able to influence neuronal activity⁴¹,⁵¹⁻⁵³. An interesting feature of tTIS is the ability to stimulate at a particular frequency of interest to preferentially interact with specific neuronal processes⁴¹,⁴². Despite these exciting opportunities, current evidence for tTIS-related neuromodulation of deep brain structures, such as the striatum, remains sparse in humans⁵²,⁵³.

Here we combine tTIS with electric field modelling for target localization, behavioural data and functional magnetic resonance

imaging (fMRI) to evaluate the causal role of specific patterns of striatal activity in reinforcement learning of motor skills. On the basis of the studies mentioned above, we hypothesized that striatal tTIS at high gamma frequency (tTIS<sub>80Hz</sub>) would disturb the fine-tuning of high gamma oscillatory activity in the striatum and thereby would perturb reinforcement motor learning, in contrast to beta (tTIS<sub>20Hz</sub>) or sham (tTIS<sub>Sham</sub>) stimulation. More specifically, we reasoned that applying a constant high gamma rhythm in the striatum would disturb the temporally precise and reinforcement-specific modulation of high gamma activity. Moreover, given that the stimulation protocol was not individualized to endogenous high gamma activity and not synchronized to ongoing activity in other hubs of the reinforcement learning network (for example, the frontal cortex), we anticipated disruptive rather than beneficial effects of tTIS<sub>80Hz</sub>.

In line with our prediction, we report that tTIS<sub>80Hz</sub> disrupted motor learning compared with the controls, but only in the presence of reinforcement. To evaluate the potential neural correlates of these behavioural effects, we measured blood-oxygen-level-dependent (BOLD) activity in the striatum and effective connectivity between the striatum and frontal cortical areas involved in reinforcement motor learning. We found that the disruptive effect of tTIS<sub>80Hz</sub> on reinforcement learning was associated with a specific modulation of BOLD activity in the putamen and caudate, but not in the cortex, supporting the ability of tTIS to selectively modulate striatal activity without affecting overlying cortical areas. Moreover, tTIS<sub>80Hz</sub> also increased the neuromodulatory influence of the striatum on frontal cortical areas involved in reinforcement motor learning. Overall, the present study shows that tTIS can non-invasively and selectively modulate a striatal mechanism involved in reinforcement learning.

## Results

A total of 24 healthy participants (15 women, 25.3 ± 0.1 years old (mean ± s.e.)) performed a force-tracking task in the MRI scanner with concurrent tTIS of the striatum. The task required the participants to modulate the force applied on a hand-grip force sensor to track a moving target with a cursor with the right, dominant hand⁵⁴,⁵⁵ (Fig. 1a). In each block, the participants had to learn a new pattern of motion of the target (Supplementary Fig. 1a and Methods). In Reinf<sub>ON</sub> blocks, the participants were provided with online reinforcement feedback during training, giving them real-time information about success or failure throughout the trial, indicated as a green or red target, respectively (please see Supplementary Video 1 for the task). The reinforcement feedback was delivered according to a closed-loop schedule⁸, in which the success criterion to consider a force sample as successful was updated on the basis of the median performance over the four previous trials (see Methods for more details). In Reinf<sub>OFF</sub> blocks, the participants practised with visually matched random feedback (cyan/magenta). Importantly, in both types of blocks, training was performed with partial visual feedback of the cursor, a condition that has been shown to maximize reinforcement effects in various motor learning paradigms⁵,⁵⁶⁻⁵⁸ and that yielded significant effects of reinforcement on motor learning, as also demonstrated in an additional behavioural study testing another group of healthy participants on the same task (n = 24; Supplementary Fig. 1b–e). Before and after training, the participants performed pre- and post-training assessments with full visual feedback, no reinforcement and no tTIS, allowing us to evaluate motor learning. To assess the effect of tTIS on reinforcement-related benefits in motor learning and the associated neural changes, the participants performed six blocks of 36 trials in the MRI machine, with concurrent tTIS during training, delivered with a Δf of 20 Hz (tTIS<sub>20Hz</sub>) or 80 Hz (tTIS<sub>80Hz</sub>) or as a sham (tTIS<sub>Sham</sub>: 3 tTIS<sub>TYPE</sub> × 2 Reinf<sub>TYPE</sub> conditions; Fig. 1b,c). The order of the conditions was balanced among the 24 participants, ensuring that any potential carry-over effect would have the same impact on each experimental condition. To determine the best electrode montage to stimulate the human striatum (putamen, caudate and nucleus accumbens (NAc))
a

<figure><img src="image_4.png" /></figure>

**Trial types**
* Test (pre/post)
* Full visual feedback
* No reinforcement

**Reinforcement**
* Partial visual feedback
* Reinforcement
* Partial visual feedback
* No reinforcement

b

<figure><img src="image_5.png" /></figure>

c

<figure><img src="image_6.png" /></figure>

**Fig. 1 | Striatal tTIS during reinforcement learning of motor skills in the MRI machine.** a, Motor learning task. The participants were required to squeeze a hand-grip force sensor (depicted in the upper right corner of the figure) to track a moving target (the larger circle with a cross in the centre) with a cursor (the smaller black circle)⁵⁴,⁵⁵. Pre- and post-training assessments were performed with full visual feedback of the cursor and no reinforcement. In $Reinf_{ON}$ and $Reinf_{OFF}$ trials, the participants practised the task with or without reinforcement feedback, respectively. In $Reinf_{ON}$ trials, the colour of the target varied in real time as a function of the participants' tracking performance. b, Experimental procedure. The participants performed the task in the MRI machine with concomitant tTIS. Blocks of training were composed of 36 trials (4 pre-training, 24 training and 8 post-training trials) interspersed with short resting periods (represented as plus signs in the figure). The six training types resulted from the combination of three $tTIS_{TYPES}$ and two $Reinf_{TYPES}$. c, Concept of tTIS. On the left, two pairs of electrodes are shown on a head model, and currents $I_1$ and $I_2$ are

d

<figure><img src="image_7.png" /></figure>

e

<figure><img src="image_8.png" /></figure>

applied with frequencies $f_1$ and $f_2 + \Delta f$. On the right, the interference of the two electric fields within the brain is represented for two different locations with high and low envelope modulation. $E_1(t)$ and $E_2(t)$ represent the modulation of the fields' magnitude over time. tTIS was delivered with a $\Delta f$ of 20 or 80 Hz or as a sham (a ramp-up and immediate ramp-down of high-frequency currents with a flat envelope). **d**, Electric field modelling with the striatal montage. The colours show the temporal interference exposure (electric field modulation magnitude). **e**, Temporal interference exposure in the striatum and in the overlying cortex. The violin plots show the tTIS exposure distribution over the voxels in the striatum and cortex underneath the stimulation electrodes. The magnitude of the field in the cortex was extracted from the BNA⁶⁴ regions underneath the stimulation electrodes (F3–F4 and TP7–TP8). The black bar represents the mean. Voxels with outlying tTIS exposure (±5 s.d. around the mean) were removed from the plot (21 values from a total of 46,479 considered voxels).

bilaterally), computational modelling with a realistic head model was conducted with Sim4Life⁵⁹ (Methods). The selected montage (F3–F4 and TP7–TP8) generated a theoretical temporal interference electric field that was -30–40% stronger in the striatum than in the overlying cortex, reaching magnitudes of 0.5 to 0.6 V m⁻¹ (Fig. 1d,e).

## tTIS<sub>80Hz</sub> disrupts reinforcement learning of motor skills

Task performance was evaluated by means of the Error, which was defined as the absolute difference between the applied and target

force averaged across samples for each trial, as done previously⁵,⁵⁵,⁵⁸ (Fig. 2a). Across conditions, the post-training Error was lower than the pre-training Error (single-sample two-sided *t*-test on the normalized post-training data: $t_{24} = -2.69; P = 0.013$; Cohen's *d* = -0.53; 95% confidence interval (CI), (-0.99, -0.09)), indicating significant motor learning during the task (Fig. 2b). Such improvement was greater when participants had trained with reinforcement (Reinf<sub>TYPE</sub> effect in the linear mixed model (LMM): $F_{1,1062.2} = 5.17; P = 0.023$; partial eta-squared ($\eta_p^2$), 0.005; 95% CI, (0.00, 0.02)), confirming the beneficial effect of
<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

**Fig. 2 | Behavioural results.** a, Motor performance across training. The raw Error data (expressed in percentage of maximum voluntary contraction (MVC)) from the 24 participants are presented in the left panel for the different experimental conditions in bins of four trials. The increase in Error during training is related to the visual uncertainty (that is, the intermittent disappearance of the cursor) that was applied to enhance reinforcement effects. The three plots on the right represent the pre-training normalized Error in the tTIS<sub>Sham</sub>, tTIS<sub>20Hz</sub> and tTIS<sub>80Hz</sub> blocks. Reinforcement-related benefits represent the improvement in the Error measured in the Reinf<sub>ON</sub> and Reinf<sub>OFF</sub> blocks during training (reflecting benefits in motor performance) or at post-training (reflecting benefits in learning).

b, Averaged learning across conditions. The violin plot shows the Error distribution at post-training (expressed in percentage of pre-training) averaged across conditions, as well as individual participant data. A single-sample two-sided *t*-test showed that the post-training Error was lower than the pre-training level, indicating significant learning in the task (*P* = 0.013; *n* = 24 participants). c, Motor learning. The averaged Error at post-training (normalized to pre-training) and the corresponding individual data points in the different experimental conditions are shown in the left and right panels, respectively, for the participants included in the analysis (that is, after outlier detection; remaining *n* = 23). The reduction of Error at post-training reflects

<figure><img src="image_12.png" /></figure>

true improvement at tracking the target in test conditions (in the absence of reinforcement, visual uncertainty or tTIS). The LMM run on these data revealed a specific effect of tTIS<sub>80Hz</sub> on reinforcement-related benefits in learning (analysis of variance (ANOVA) with Satterthwaite approximation followed by two-sided pairwise comparisons via estimated marginal means with Tukey adjustment). Learning was disrupted with Reinf<sub>ON</sub> in the tTIS<sub>80Hz</sub> condition compared with the tTIS<sub>20Hz</sub> (*P* = 0.039) and tTIS<sub>Sham</sub> (*P* < 0.001) conditions. d, Motor performance. The averaged Error during training (normalized to pre-training) and the corresponding individual data points in the different experimental conditions are shown in the left and right panels, respectively, for the participants included in the analysis (that is, after outlier detection; *n* = 23). The Error change during training reflects the joint contribution of the experimental manipulations (visual uncertainty, potential reinforcement and tTIS) to motor performance. The LMM run on these data showed a frequency-dependent effect of tTIS on motor performance, irrespective of reinforcement (ANOVA with Satterthwaite approximation followed by two-sided pairwise comparisons via estimated marginal means with Tukey adjustment). Motor performance was disrupted irrespective of reinforcement in the tTIS<sub>20Hz</sub> (versus tTIS<sub>Sham</sub>: *P* < 0.001) and tTIS<sub>80Hz</sub> (versus tTIS<sub>Sham</sub>: *P* < 0.001; versus tTIS<sub>20Hz</sub>: *P* = 0.031) conditions. The data are represented as mean ± s.e.

reinforcement on motor learning<sup>7,57</sup>. Crucially, though, this effect depended on the type of stimulation applied during training (Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub> interaction: *F*<sub>2,1063.5</sub> = 2.11; *P* = 0.034; *η*<sub>p</sub><sup>2</sup> = 0.006; 95% CI, (0.00, 0.02); Fig. 2c). While reinforcement significantly improved learning when training was performed with tTIS<sub>Sham</sub> (two-sided Tukey-corrected pairwise comparison: *P* = 0.036; *d* = −0.22; 95% CI, (−0.46, 0.01)) and tTIS<sub>20Hz</sub> (*P* = 0.0089; *d* = −0.27; 95% CI, (−0.51, −0.04)), this was not the case with tTIS<sub>80Hz</sub> (*P* = 0.43; *d* = 0.083; 95% CI, (−0.14, 0.31)). Consistently, direct between-condition comparisons showed that in the Reinf<sub>ON</sub> condition, learning was reduced with tTIS<sub>80Hz</sub> compared with tTIS<sub>20Hz</sub> (*P* = 0.039; *d* = 0.26; 95% CI, (0.02, 0.49)) and tTIS<sub>Sham</sub>

(*P* < 0.001; *d* = 0.45; 95% CI, (0.19, 0.72)), while there was no evidence for a difference between tTIS<sub>20Hz</sub> and tTIS<sub>Sham</sub> (*P* = 0.15; *d* = 0.20; 95% CI, (−0.04, 0.43)). This disruption of motor learning with tTIS<sub>80Hz</sub> was not observed in the absence of reinforcement (tTIS<sub>80Hz</sub> versus tTIS<sub>20Hz</sub>: *P* = 0.59; *d* = −0.10; 95% CI, (−0.33, 0.12); tTIS<sub>80Hz</sub> versus tTIS<sub>Sham</sub>: *P* = 0.34; *d* = 0.15; 95% CI, (−0.08, 0.38)). These results point to the fact that high gamma striatal tTIS specifically disrupts the benefits of reinforcement for motor learning and not motor learning in general.

Although training with tTIS<sub>20Hz</sub> did not alter the benefits of reinforcement for motor learning, we found that learning without reinforcement was significantly impaired in this condition (tTIS<sub>20Hz</sub> versus
tTIS<sub>Sham</sub>: *P* = 0.046; *d* = 0.25; 95% CI, (0.01, 0.49); Fig. 2c). This suggests that tTIS<sub>20Hz</sub> may disrupt a qualitatively different mechanism involved in motor learning from sensory feedback<sup>60</sup>, in line with the role of striatal beta oscillations in sensorimotor function<sup>28</sup>.

Next, we evaluated the effect of tTIS on motor performance during training itself. As shown in Fig. 2a, the Error was generally higher during training than in test trials due to the presence of visual uncertainty during this phase. The extent of this disruption was reduced in the presence of reinforcement (Reinf<sub>TYPE</sub>: $F_{1,3262.4} = 339.89; P < 0.001; \eta_p^2 = 0.09; 95\%$ CI, (0.08, 0.11)), demonstrating the ability of participants to exploit real-time reinforcement information to improve tracking (Fig. 2d). Notably, this effect was not modulated by tTIS<sub>TYPE</sub> (Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub>: $F_{2,3265.8} = 0.91; P = 0.40; \eta_p^2 = 6 \times 10^{-4}$), indicating that tTIS did not directly influence reinforcement gains during tracking. However, striatal stimulation did impact general tracking performance independently of reinforcement, as indicated by a significant tTIS<sub>TYPE</sub> effect (tTIS<sub>TYPE</sub>: $F_{2,3262.4} = 42.85; P < 0.001; \eta_p^2 = 0.03; 95\%$ CI, (0.02, 0.04)). This effect was due to an increase in the Error when tTIS<sub>20Hz</sub> was applied (*P* < 0.001; *d* = 0.28; 95% CI, (0.16, 0.39) when compared with tTIS<sub>Sham</sub>), which was even larger during tTIS<sub>80Hz</sub> (*P* < 0.001; *d* = 0.38; 95% CI, (0.25, 0.52) and *P* = 0.031; *d* = 0.11; 95% CI, (0.02, 0.20) when compared with tTIS<sub>Sham</sub> and tTIS<sub>20Hz</sub>, respectively). An additional analysis showed that the detrimental effect of tTIS on motor performance was actually due to an impaired ability to improve performance during training (LMM with continuous fixed effect Trial: tTIS<sub>TYPE</sub> × Trial interaction: $F_{2,3399} = 4.46; P = 0.012; \eta_p^2 = 0.003; 95\%$ CI, (0.00, 0.01); post hoc tests: tTIS<sub>Sham</sub> versus tTIS<sub>20Hz</sub>: $P = 0.013; d = -0.02; 95\%$ CI, (-0.03, 0.00); tTIS<sub>Sham</sub> versus tTIS<sub>80Hz</sub>: $P = 0.068; d = -0.01; 95\%$ CI, (-0.03, 0.00); tTIS<sub>20Hz</sub> versus tTIS<sub>80Hz</sub>: $P = 0.81; d = 0.004; 95\%$ CI, (-0.01, 0.02); Supplementary Fig. 1f). However, again, this effect did not depend on the presence of reinforcement (Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub> × Trial: $F_{2,3399} = 0.51; P = 0.60; \eta_p^2 = 3 \times 10^{-4}$). We also found that the detrimental effect of striatal tTIS did not depend on the availability of visual information on the cursor, but rather that tTIS had a general effect on motor performance irrespective of visual and reinforcement feedback (Supplementary Information). This analysis also confirmed that reinforcement gains in motor performance were stronger when visual information was not available (Supplementary Fig. 1g), in line with the behavioural data mentioned above (Supplementary Fig. 1b) and previous studies<sup>56,61</sup>. Overall, these results suggest that striatal tTIS altered motor performance in a frequency-dependent manner but did not influence the ability to rapidly adjust motor commands on the basis of reinforcement feedback during training. Hence, tTIS<sub>80Hz</sub> may not disrupt real-time processing of reinforcement feedback but may instead impair the beneficial effect of reinforcements on the retention of motor memories<sup>6,7</sup>.

To further understand this dissociation, we ran additional analyses exploring the relationship between reinforcement gains in the training (performed with partial visual feedback and Reinf<sub>ON</sub> or Reinf<sub>OFF</sub>) and post-training phases (performed with full visual feedback and no reinforcement). We found consistent positive associations between individual reinforcement gains at the end of training (T6) and at the beginning of post-training (Post1) in the tTIS<sub>Sham</sub> (robust linear regression: $R^2 = 0.45, P < 0.001$) and tTIS<sub>20Hz</sub> ($R^2 = 0.36, P = 0.003$) conditions and in the additional behavioural dataset ($R^2 = 0.31, P = 0.009$, Supplementary Fig. 2). This association was abolished specifically in the tTIS<sub>80Hz</sub> condition ($R^2 = 0.028, P = 0.39$): participants who benefited from reinforcement during training did not exhibit gains in learning at post-training (see Supplementary Information for more details on this analysis). This suggests that the disruption of reinforcement motor learning with tTIS<sub>80Hz</sub> did not concern all participants (in this case, we would still have found a correlation but an upward shift in the intercept) but primarily affected participants who actually benefited from reinforcement during training, further supporting the idea of a specific disruption of reinforcement motor learning with tTIS<sub>80Hz</sub>.

These effects could not be explained by potential differences in initial performance between conditions (Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub>: $F_{2,519.99} = 1.08; P = 0.34; \eta_p^2 = 0.004; 95\%$ CI, (0.00, 0.02)), by changes in the flashing properties of the reinforcement feedback (that is, the frequency of colour change during tracking; Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub>: $F_{2,3283} = 0.19; P = 0.82; \eta_p^2 = 1 \times 10^{-4}$) or by differences in success rate in the Reinf<sub>ON</sub> blocks (that is, the proportion of success feedback during tracking; tTIS<sub>TYPE</sub>: $F_{2,1702} = 0.17; P = 0.84; \eta_p^2 = 2 \times 10^{-4}$). There was also no evidence that the Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub> effect on learning was influenced by the order of the reinforcement conditions (analysis on sub-groups based on whether participants experienced Reinf<sub>ON</sub> or Reinf<sub>OFF</sub> first; no Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub> × Group<sub>TYPE</sub> interaction: $F_{2,1105.06} = 1.75; P = 0.17; \eta_p^2 = 0.003; 95\%$ CI, (0.00, 0.01); see Supplementary Information for more details on these analyses).

Finally, we confirmed that these results were not a consequence of inefficient blinding. During debriefing after the experiment, only 6/24 participants were able to successfully identify the order of the stimulation applied (for example, real–real–placebo; chance level, 4/24; Fisher exact test on proportions, *P* = 0.74). Consistently, the magnitude (Supplementary Fig. 3a) and type (Supplementary Fig. 3b) of tTIS-evoked sensations evaluated before the experiment were qualitatively similar across conditions, and tTIS was generally well tolerated in all participants (no adverse events reported). This suggests that blinding was successful and is unlikely to explain our findings. More generally, this indicates that tTIS evokes very limited sensations (for example, only 2/24 and 1/24 participants rated sensations evoked at 2 mA as “strong” for tTIS<sub>20Hz</sub> and tTIS<sub>80Hz</sub>, respectively; Supplementary Fig. 3a) that are compatible with efficient blinding.

## Behavioural effect of tTIS<sub>80Hz</sub> is linked to striatal modulation

As mentioned above, task-based fMRI was acquired during training with concomitant tTIS. This allowed us to evaluate the neural effects of tTIS and their potential relationship to the behavioural effects reported above. As a first qualitative evaluation of the data, we performed a whole-brain analysis in the tTIS<sub>Sham</sub> condition to assess the network activated during reinforcement motor learning (Reinf<sub>ON</sub> condition). Consistent with previous neuroimaging studies employing similar tasks<sup>62,63</sup>, we found prominent BOLD activations in a motor network including the putamen, thalamus, cerebellum and sensorimotor cortex, particularly in the left hemisphere, contralateral to the trained hand (Supplementary Fig. 4 and Supplementary Table 1). However, contrasting Reinf<sub>ON</sub> and Reinf<sub>OFF</sub> conditions did not reveal any significant cluster at the whole-brain level. This first analysis thus did not reveal any region specifically activated in the presence of reinforcement, but rather confirms the involvement of a motor network engaged in this type of task irrespective of the reinforcement feedback.

As a second step, we evaluated the effect of tTIS on striatal activity, as a function of the type of reinforcement feedback and focusing on the same regions of interest (ROIs) that were used to optimize tTIS exposure in the modelling. We extracted averaged BOLD activity within the bilateral putamen, caudate and NAc based on the Brainnetome Atlas (BNA)<sup>64</sup> in the different experimental conditions and considered these six striatal ROIs (ROI<sub>STR</sub>) as fixed effects in the LMM. This model revealed a significant enhancement of striatal activity with Reinf<sub>ON</sub> with respect to Reinf<sub>OFF</sub> ($F_{1,800.01} = 13.23; P < 0.001; \eta_p^2 = 0.02; 95\%$ CI, (0.00, 0.04)), consistent with previous literature<sup>11</sup>, but no tTIS<sub>TYPE</sub> effect ($F_{2,800.01} = 0.46; P = 0.63; \eta_p^2 = 0.001; 95\%$ CI, (0.00, 0.01)) and no interaction (all *P* > 0.65; Fig. 3a). Despite the absence of effects of tTIS on averaged striatal activity, we then asked whether the behavioural effects of tTIS<sub>80Hz</sub> on reinforcement motor learning (that is, tTIS<sub>80Hz</sub> versus tTIS<sub>20Hz</sub> and tTIS<sub>Sham</sub> with Reinf<sub>ON</sub>) could be linked to the modulation of activity in core brain regions. To do so, we ran a whole-brain analysis focusing on the main behavioural effects mentioned above. The results revealed that the effect of tTIS<sub>80Hz</sub> (with respect to tTIS<sub>20Hz</sub>) on motor learning in the Reinf<sub>ON</sub> condition was specifically related to
a

<figure><img src="image_13.png" /></figure>

b

<figure><img src="image_14.png" /></figure>

**Fig. 3** | Striatal activity. **a**, Striatal BOLD responses. A 3D reconstruction of the striatal masks used in the current experiment is surrounded by plots showing averaged BOLD activity for each mask in the different experimental conditions. An LMM run on these data showed higher striatal responses in the Reinf<sub>ON</sub> than in the Reinf<sub>OFF</sub> condition, but no effect of tTIS<sub>TYPE</sub> and no interaction (n = 24 participants). The data are represented as mean ± s.e. **b**, Whole-brain activity associated with the behavioural effect of tTIS<sub>80Hz</sub> on reinforcement motor

learning. The correlation between tTIS-related modulation of striatal activity (tTIS<sub>80Hz</sub>–tTIS<sub>20Hz</sub>) and learning abilities in the Reinf<sub>ON</sub> condition (n = 24) is shown. Significant clusters of correlation were found in the left putamen and bilateral caudate (t-contrast; uncorrected P = 0.001 at the voxel level; corrected cluster-based false discovery rate, P = 0.05). The lower panel shows individual robust linear regressions for the three significant regions highlighted in the whole-brain analysis.

the modulation of activity in two clusters encompassing the left putamen and bilateral caudate (Fig. 3b and Supplementary Table 2). The presence of the high-frequency carrier (kHz) in both stimulation conditions rules out the possibility that the correlation was due to putative

neuromodulatory effects of high-frequency stimulation. No significant clusters were found for the tTIS<sub>80Hz</sub>–tTIS<sub>Sham</sub> contrast or for the control tTIS<sub>20Hz</sub>–tTIS<sub>Sham</sub> contrast, indicating that the reported correlation is not due to a general link between striatal activity and reinforcement motor
a

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /><figcaption>**Fig. 4 | Striatum-to-frontal-cortex effective connectivity.** a, Motor network.</figcaption></figure>

A 3D reconstruction of the masks used for the motor network (that is, dorso-lateral putamen (dlPu), dorsal caudate (dCa), M1 and SMA) is shown on the left.

The plot on the right shows the effective connectivity from motor striatum to motor cortex in the different experimental conditions (n = 24 participants).

Note the increase of connectivity with tTIS<sub>80Hz</sub> in the presence of reinforcement (tTIS<sub>80Hz</sub> - Reinf<sub>ON</sub>; *P* = 0.001 (versus tTIS<sub>80Hz</sub> - Reinf<sub>OFF</sub>) and *P* < 0.001 (versus tTIS<sub>Sham</sub> - Reinf<sub>ON</sub>). b, Reward network. A 3D reconstruction of the masks used for

b

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

the reward network (that is, ventro-medial putamen (vmPu), NAc, vmPFC and ACC) is shown on the left. The plot on the right shows the effective connectivity from motor striatum to motor cortex in the different experimental conditions (n = 24). ROIs were defined on the basis of the BNA¹². In **a** and **b**, the outputs of LMMs were analysed using ANOVA with Satterthwaite approximation followed by two-sided pairwise comparisons via estimated marginal means with Tukey adjustment. The data are represented as mean ± s.e.

learning. Overall, these results provide evidence that the detrimental effect of tTIS<sub>80Hz</sub> on reinforcement learning of motor skills is related to the modulation of neural activity specifically in the striatum.

## tTIS<sub>80Hz</sub> enhances striatum-to-frontal-cortex connectivity

Interactions between the striatum and frontal cortex are crucial for a variety of behaviours, including motor and reinforcement learning¹³. In particular, reinforcement motor learning requires the use of information about task success to guide future motor commands⁵, a process in which the striatum may play an integrative role at the interface between fronto-striatal loops involved in reward processing and motor control¹³,⁶⁵. In a subsequent analysis, we asked whether striatal tTIS modulates striatum-to-frontal-cortex communication during reinforcement motor learning. More specifically, we computed effective connectivity (using the generalized psychophysiological interactions (gPPI) method⁶⁶) between striatal and frontal regions classically associated with motor and reward-related functions, and thought to be involved in reinforcement motor learning⁶⁷,⁶⁸. For the motor network, we evaluated effective connectivity between motor parts of the striatum (that is, dorso-lateral putamen and dorsal caudate) and two regions strongly implicated in motor learning: the medial part of the supplementary motor area (SMA) and the part of the primary motor cortex (M1) associated with upper limb functions (Fig. 4a). For the reward network, we assessed connectivity between parts of the striatum classically associated with limbic functions (that is, the NAc, the ventro-medial putamen and two frontal areas involved in reward processing: the anterior cingulate cortex (ACC) and the ventro-medial prefrontal cortex (vmPFC); Fig. 4b)¹¹. The LMM run with the fixed effects Reinf<sub>TYPE</sub>, tTIS<sub>TYPE</sub> and Network<sub>TYPE</sub> showed a significant effect of tTIS<sub>TYPE</sub> (*F*<sub>2,2264.0</sub> = 5.42; *P* = 0.005; *η*<sub>p</sub><sup>2</sup> = 0.005; 95% CI, (0.00, 0.01)) that was due to higher connectivity in the tTIS<sub>80Hz</sub> condition than in tTIS<sub>Sham</sub> (Tukey-corrected *P* = 0.004; *d* = 0.16; 95% CI, (0.05, 0.28)). There was no significant difference in connectivity between tTIS<sub>80Hz</sub> and tTIS<sub>20Hz</sub> (*P* = 0.069; *d* = 0.11; 95% CI, (0.00, 0.22)) or between tTIS<sub>20Hz</sub> and tTIS<sub>Sham</sub> (*P* = 0.58; *d* = 0.051; 95% CI, (-0.05, 0.16)). Hence, tTIS<sub>80Hz</sub>, but not tTIS<sub>20Hz</sub>, enhanced effective connectivity between the striatum and frontal cortex during motor training. This increase in effective connectivity with tTIS<sub>80Hz</sub> actually led to a connectivity closer to the resting state (values closer to 0; Methods). Put differently, while the task

induced a reduction in effective connectivity between striatum and frontal cortex, tTIS<sub>80Hz</sub> disrupted this modulation by bringing connectivity back to the resting state.

The LMM did not reveal any effect of Reinf<sub>TYPE</sub> (*F*<sub>1,2264.0</sub> = 0.010; *P* = 0.92; *η*<sub>p</sub><sup>2</sup> = 5 × 10<sup>-6</sup>), Network<sub>TYPE</sub> (*F*<sub>1,2264.0</sub> = 3.16; *P* = 0.076; *η*<sub>p</sub><sup>2</sup> = 0.001; 95% CI, (0.00, 0.01)) or a double interaction (Reinf<sub>TYPE</sub> × Network<sub>TYPE</sub>: *F*<sub>1,2264.0</sub> = 3.52; *P* = 0.061; *η*<sub>p</sub><sup>2</sup> = 0.002; 95% CI, (0.00, 0.01)). Yet, we did find a significant Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub> × Network<sub>TYPE</sub> interaction (*F*<sub>2,2264.0</sub> = 4.87; *P* = 0.008; *η*<sub>p</sub><sup>2</sup> = 0.004; 95% CI, (0.00, 0.01)). This triple interaction was related to the fact that tTIS<sub>80Hz</sub> increased connectivity in the Reinf<sub>ON</sub> condition in the motor network (Reinf<sub>ON</sub> versus Reinf<sub>OFF</sub>: *P* = 0.001; *d* = 0.33; 95% CI, (0.11, 0.55); Fig. 4a), while this effect was not observed in the reward network (*P* = 0.063; *d* = -0.19; 95% CI, (-0.40, 0.02); Fig. 4b). There was no evidence for such an increase in either of the two networks when either tTIS<sub>Sham</sub> or tTIS<sub>20Hz</sub> was applied (all *P* > 0.40, all *d* [-0.09, -0.02]). Moreover, in the motor network, connectivity in the Reinf<sub>ON</sub> condition was higher with tTIS<sub>80Hz</sub> than with tTIS<sub>Sham</sub> (*P* < 0.001; *d* = 0.42; 95% CI, (0.19, 0.65)). This effect did not reach significance when contrasting tTIS<sub>80Hz</sub> with tTIS<sub>20Hz</sub> (*P* = 0.059; *d* = 0.23; 95% CI, (0.02, 0.44); Fig. 4a). These data suggest that tTIS<sub>80Hz</sub> enhanced the neuromodulatory influence of the striatum on motor cortex during task performance, but only in the presence of reinforcement. In the reward network, post hoc tests revealed that connectivity in the Reinf<sub>OFF</sub> condition was significantly higher with tTIS<sub>80Hz</sub> than with tTIS<sub>20Hz</sub> (*P* = 0.045; *d* = 0.24; 95% CI, (0.03, 0.46); Fig. 4b), in line with the general effect of tTIS<sub>TYPE</sub> on connectivity reported above. This pattern of results suggests that the increase of connectivity from striatum to frontal cortex observed with tTIS<sub>80Hz</sub> depends on the presence of reinforcement, in particular in the motor network. This reinforcement-dependent increase of connectivity may reflect the preferential effect of tTIS<sub>80Hz</sub> on striatal gamma oscillations⁶⁹ in a situation where these oscillations are already boosted by the presence of reinforcement¹⁰ (Discussion).

In a subsequent analysis, we verified that these results did not depend on the specific frontal ROIs considered in the analysis (ROI<sub>TYPE</sub>: M1 and SMA in the motor network and ACC and vmPFC in the reward network). Importantly, we did not find a tTIS<sub>TYPE</sub> × Reinf<sub>TYPE</sub> × ROI<sub>TYPE</sub> interaction in the motor network (*F*<sub>2,1112</sub> = 0.83; *P* = 0.44; *η*<sub>p</sub><sup>2</sup> = 0.001; 95% CI, (0.00, 0.01)) or in the reward network (*F*<sub>2,1112</sub> = 0.61; *P* = 0.54; *η*<sub>p</sub><sup>2</sup> = 0.001; 95% CI, (0.00, 0.01)), suggesting that the main connectivity
results were consistent within a network and were not influenced by the specific frontal ROI included in the analysis (see Supplementary Information for more details on this analysis). As an additional control, we verified that the effects of tTIS<sub>TYPE</sub> on connectivity could not be observed in a control network associated with language (as defined by ref. 70), which was unlikely to be involved in the present task and did not include the striatum (Methods). As expected, there was no evidence for a modulation of effective connectivity within the language network by Reinf<sub>TYPE</sub> (F<sub>1,547</sub> = 0.81; *P* = 0.37; *η*<sub>p</sub><sup>2</sup> = 0.001; 95% CI, (0.00, 0.01)) or by tTIS<sub>TYPE</sub> (F<sub>2,547</sub> = 0.58; *P* = 0.56; *η*<sub>p</sub><sup>2</sup> = 0.002; 95% CI, (0.00, 0.01)) or by Reinf<sub>TYPE</sub> × tTIS<sub>TYPE</sub> (F<sub>2,547</sub> = 0.45; *P* = 0.64; *η*<sub>p</sub><sup>2</sup> = 0.002; 95% CI, (0.00, 0.01)). Hence, tTIS-related and reinforcement-related changes in connectivity were consistent within the considered fronto-striatal networks and not observed in a control network unrelated to the task.

Contrary to the BOLD results presented above, we did not find any correlations between the effects of tTIS<sub>80Hz</sub> on connectivity and motor learning, in either the motor network (robust linear regression: tTIS<sub>80Hz</sub> - tTIS<sub>Sham</sub>, R<sup>2</sup> = 0.019, *P* = 0.48; tTIS<sub>80Hz</sub> - tTIS<sub>20Hz</sub>, R<sup>2</sup> = 0.034, *P* = 0.54) or the reward network (tTIS<sub>80Hz</sub> - tTIS<sub>Sham</sub>, R<sup>2</sup> = 0.037, *P* = 0.46; tTIS<sub>80Hz</sub> - tTIS<sub>20Hz</sub>, R<sup>2</sup> < 0.001, *P* = 0.75), suggesting some degree of independence between the effect of tTIS<sub>80Hz</sub> on reinforcement motor learning and that on effective connectivity.

Overall, these results highlight the ability of tTIS<sub>80Hz</sub>, but not tTIS<sub>20Hz</sub>, to modulate striatum-to-frontal-cortex connectivity, depending on the presence of reinforcement. However, the absence of a correlation with behaviour suggests that this effect may not be directly associated with the detrimental effect of tTIS<sub>80Hz</sub> on reinforcement motor learning or that tTIS<sub>80Hz</sub>-related changes in striato-frontal communication were linked to other aspects of reinforcement learning not captured by our task.

## Neural effects of tTIS<sub>80Hz</sub> depend on impulsivity

Determining individual factors that shape responsiveness to non-invasive brain stimulation approaches is a crucial step towards better understanding the mechanisms of action as well as envisioning the stratification of patients in future clinical interventions<sup>71</sup>. A potential factor that could explain inter-individual differences in responsiveness to tTIS<sub>80Hz</sub> is the level of impulsivity. Impulsivity has been associated with changes of gamma oscillatory activity in the striatum of rats<sup>72</sup> and with the activity of fast-spiking interneurons in the striatum<sup>73,74</sup>, a neuronal population that is strongly entrained to gamma rhythms<sup>19,21</sup> and may therefore be particularly sensitive to tTIS<sub>80Hz</sub>. In a subsequent exploratory analysis, we asked whether the neural effects of tTIS<sub>80Hz</sub> were associated with impulsivity levels, as evaluated by a well-established independent delay-discounting questionnaire performed at the beginning of the experiment<sup>75,76</sup>. A whole-brain analysis revealed that impulsivity was associated with the effect of tTIS<sub>80Hz</sub> on BOLD activity (with respect to tTIS<sub>20Hz</sub>) specifically in the left caudate nucleus (Supplementary Fig. 5a,b and Supplementary Table 3). Moreover, the effect of tTIS<sub>80Hz</sub> on striatum-to-motor-cortex connectivity reported above was negatively correlated with impulsivity when contrasting tTIS<sub>80Hz</sub> with both tTIS<sub>Sham</sub> (Supplementary Fig. 5c, left) and tTIS<sub>20Hz</sub> (Supplementary Fig. 5c, middle). Such correlations were absent when contrasting tTIS<sub>20Hz</sub> with tTIS<sub>Sham</sub> (Supplementary Fig. 5c, right), as well as when considering the same contrasts in the reward instead of the motor network (see Supplementary Information for more details). Taken together, these results suggest that inter-individual variability in impulsivity might influence the neural responses to striatal tTIS<sub>80Hz</sub>.

## Discussion

In this study, we combined striatal tTIS with electric field modelling, behavioural and fMRI analyses to evaluate the causal role of the striatum in reinforcement learning of motor skills in healthy humans. tTIS<sub>80Hz</sub>, but not tTIS<sub>20Hz</sub>, disrupted the ability to learn from reinforcement feedback. This behavioural effect was associated with modulation of neural

activity specifically in the striatum. We also show that tTIS<sub>80Hz</sub>, but not tTIS<sub>20Hz</sub>, increased the neuromodulatory influence of the striatum on connected frontal cortical areas involved in reinforcement motor learning. Finally, inter-individual variability in the neural effects of tTIS<sub>80Hz</sub> could be partially explained by impulsivity, suggesting that this trait may constitute a determinant of responsiveness to high gamma striatal tTIS. Overall, the present study shows that striatal tTIS can non-invasively modulate a striatal mechanism involved in reinforcement learning, expanding our tools for the study of causal relationships between deep brain structures and human behaviour.

We investigated the causal role of the human striatum in reinforcement learning of motor skills in healthy humans, a question that cannot be addressed with conventional non-invasive brain stimulation techniques. In particular, by stimulating at different frequencies, we aimed to dissociate striatal mechanisms involved in reinforcement and sensorimotor learning. In line with our main hypothesis, we found that striatal tTIS<sub>80Hz</sub> altered reinforcement learning of a motor skill. Such disruption was frequency- and reinforcement-specific: learning was not altered with striatal tTIS<sub>20Hz</sub> in the presence of reinforcement, or when striatal tTIS<sub>80Hz</sub> was delivered in the absence of reinforcement. The rationale to stimulate at high gamma frequency was based on previous work showing reinforcement-related modulation of gamma oscillations in the striatum<sup>19–21,24,26,72,77</sup> and in the frontal cortex<sup>77–80</sup>. Several neuronal mechanisms may contribute to the detrimental effect of tTIS<sub>80Hz</sub> on reinforcement motor learning. First, as tTIS<sub>80Hz</sub> consisted of a constant high gamma oscillating field applied on the striatum, it may have perturbed the encoding of reinforcement information into high gamma oscillations<sup>19–21,25–27</sup>, preventing participants from learning the motor skill on the basis of different outcomes. Put differently, tTIS<sub>80Hz</sub> may specifically saturate high gamma activity in the striatum, preventing reinforcement-related modulations<sup>81</sup>. Moreover, because reinforcement motor learning probably engages synchronized activity in a network of regions including fronto-striatal loops, neuromodulation of a single node of the circuit may alter the synchronization of activity in the network<sup>81</sup> and the temporal coordination with interacting rhythms<sup>25</sup>. Finally, because we did not have access to electrophysiological recordings of oscillatory activity in the striatum, the applied stimulation was not personalized, as it did not take into account the individual high gamma frequency peak associated with reward processing and the potential heterogeneity of gamma activity within the striatum<sup>24</sup>. Hence, tTIS<sub>80Hz</sub> may have resulted in a frequency mismatch between the endogenous high gamma activity and the externally imposed rhythm, which could paradoxically result in a reduction of neuronal entrainment, in particular when the frequency mismatch is relatively low<sup>82</sup>. Importantly, in contrast to striatal tTIS<sub>80Hz</sub>, we found that tTIS<sub>20Hz</sub> reduced learning, but only in the absence of reinforcement. This result fits well with the literature linking striatal beta oscillations to sensorimotor functions<sup>28,29,31,83–85</sup>. Taken together, an interpretation of these results is that different oscillations in the striatum support qualitatively distinct motor learning mechanisms, with beta activity contributing mostly to sensory-based learning and high gamma activity being particularly important for reinforcement learning. This being said, it is important to note that because we do not have concurrent electrophysiological recordings in the striatum, we cannot be sure whether the effects of tTIS<sub>20Hz</sub> and tTIS<sub>80Hz</sub> were related to frequency-specific interactions with beta or high gamma rhythms, respectively, or rather resulted from different broadband responses when stimulating at these frequencies. Yet, these results still suggest that sensory- and reinforcement-based motor learning rely on partially different neural mechanisms, in line with previous literature<sup>8,9,60,68,86,87</sup>.

Striatal tTIS also impaired tracking performance during training, irrespective of the presence of reinforcement. This frequency-dependent reduction of motor performance may be due to altered neuronal processing in the sensorimotor striatum that may lead to less fine-tuned motor control abilities<sup>88</sup>. Importantly, though, tTIS
did not modulate the ability of participants to benefit from real-time reinforcement feedback during motor performance. This suggests that striatal tTIS<sub>80Hz</sub> altered the beneficial effects of reinforcement on learning (as evaluated in test conditions at post-training), but not on motor performance (as evaluated during training). This dissociation between the effects of striatal tTIS<sub>80Hz</sub> on reinforcement-related gains in motor performance and in learning may be explained by the fact that these two phases of the protocol probe different processes<sup>7,55,58,89–91</sup>. While the improvement of motor performance with reinforcement relies on rapid feedback corrections based on expected outcomes<sup>67,92–96</sup>, reinforcement gains in learning (that is, probed in test conditions without reinforcement) may rather reflect the beneficial effect of reinforcement on the retention of motor memories<sup>4,7,55,90</sup>. This idea that the mechanisms underlying performance changes in training and retention phases are partially different is well supported by previous motor learning literature<sup>6,8,97</sup>. For instance, in sensorimotor adaptation paradigms, the presence of reward boosts motor memory retention but not the adaptation process itself<sup>7,86,90,91,98,99</sup>, and M1 transcranial direct current stimulation modulates the effect of reward on retention but has no effect on the training phase<sup>90</sup>. Such dissociation also appears to generalize to other motor learning tasks<sup>18,100</sup>, including force-tracking paradigms<sup>55</sup> (see also Supplementary Fig. 1b). Importantly, while reinforcement gains in motor performance and learning seem to reflect the operation of partially dissociable mechanisms, it is no surprise that these processes are correlated at the group level (Supplementary Fig. 2), as they may be influenced by common individual factors (for example, sensitivity to reward)<sup>101</sup>. In contrast, the absence of correlation in the striatal tTIS<sub>80Hz</sub> condition suggests that the stimulation particularly impaired reinforcement gains in learning in the participants who initially benefited from reinforcement during training (Supplementary Fig. 2a). Hence, a potential explanation for the present results is that striatal tTIS<sub>80Hz</sub> did not disrupt rapid motor corrections based on recent outcomes during training, but may rather have altered the strengthening of the memory trace based on reinforcements<sup>6,7</sup>. Overall, these results are compatible with the view that specific patterns of oscillatory activity in the striatum are involved in motor control and learning processes<sup>31</sup> and can be modulated with electrical stimulation<sup>69,102,103</sup>.

To better understand the neural effects and frequency specificity of tTIS, we coupled striatal tTIS and task performance with simultaneous fMRI acquisition. The imaging results support the view that the effect of tTIS<sub>80Hz</sub> on reinforcement learning of motor skills was indeed related to neuromodulation of the striatum. When considering averaged BOLD activity, we found a general increase of striatal activity when reinforcement was provided<sup>11</sup>, but no effect of tTIS. Crucially, though, the detrimental effect of tTIS<sub>80Hz</sub> on reinforcement learning was related to a specific modulation of activity in the caudate and putamen, providing evidence that the present behavioural effects were indeed driven by focal neuromodulation of the striatum (Fig. 3). Interestingly, participants with stronger disruption of reinforcement learning at the behavioural level were also the ones exhibiting stronger suppression of striatal activity with tTIS<sub>80Hz</sub> (than with tTIS<sub>20Hz</sub>), suggesting that tTIS-induced reduction of striatal activity is detrimental for reinforcement motor learning. Further analyses showed that tTIS<sub>80Hz</sub>, but not tTIS<sub>20Hz</sub>, increased the neuromodulatory influence of the striatum on frontal areas known to be important for motor learning and reinforcement processing<sup>97,104</sup>. More specifically, tTIS<sub>80Hz</sub> disrupted the task-related decrease in connectivity observed with tTIS<sub>sham</sub> and tTIS<sub>20Hz</sub>, bringing connectivity closer to resting-state values. This effect depended on the type of network considered (reward versus motor) and on the presence of reinforcement. Striatal tTIS<sub>80Hz</sub> coupled with reinforcement increased connectivity between the motor striatum and the motor cortex, while this effect was not observed when considering the connectivity between limbic parts of the striatum and prefrontal areas involved in reward processing (Fig. 4). This result may reflect the differential influence of striatal tTIS on distinct subparts of

the striatum, depending on their pattern of activity during the task<sup>53</sup>. A recent study in non-human primates showed that transcranial alternating current stimulation can have opposite effects on neuronal activity depending on the initial entrainment of neurons to the target frequency<sup>82</sup>. Hence, the present differential effects of tTIS<sub>80Hz</sub> on motor and reward striato-frontal pathways may be due to different initial patterns of activity in these networks in the presence of reinforcement. Electrophysiological recordings with higher temporal resolution than fMRI are required to confirm or infirm this hypothesis. Overall, the present neuroimaging results support the idea that the behavioural effects of striatal tTIS<sub>80Hz</sub> on reinforcement learning are associated with a selective modulation of striatal activity that influences striato-frontal communication.

The fact that we observed increased connectivity with tTIS<sub>80Hz</sub> and at the same time a disruption of behaviour may appear contradictory at first glance. Yet, multiple lines of evidence indicate that increases in connectivity are not necessarily beneficial for behaviour. For instance, the severity of motor symptoms in Parkinson's disease is associated with excessive connectivity in the beta band, and the reduction of such connectivity with treatment is associated with clinical improvement<sup>29,32</sup>. Moreover, there is evidence that excessive functional<sup>105,106</sup> as well as structural<sup>107,108</sup> connectivity in fronto-striatal circuits is associated with impulsivity. Hence, the increase in connectivity observed with tTIS<sub>80Hz</sub> appears to be compatible with the behavioural findings. This being said, contrary to the BOLD results, we did not find any correlation between the effects of tTIS<sub>80Hz</sub> on connectivity and on reinforcement motor learning, suggesting some degree of independence between these two effects. Future studies could aim at determining whether tTIS<sub>80Hz</sub>-related changes in striato-frontal communication are linked to other aspects of reward processing that are not captured by our reinforcement motor learning task.

From a methodological point of view, the present results provide experimental support for the idea that the effects of tTIS are related to amplitude modulation of electric fields deep in the brain and not to the high-frequency fields themselves, in line with recent work<sup>41,42,52,53</sup>. The different behavioural and neural effects of striatal tTIS<sub>80Hz</sub> and tTIS<sub>20Hz</sub> despite comparable carrier frequencies (centred on 2 kHz) indicate that temporal interference was indeed the driving force of the present effects. Moreover, the disruption of reinforcement motor learning with tTIS<sub>80Hz</sub> (relative to tTIS<sub>20Hz</sub>) was specifically related to neuromodulation of the striatum, where the amplitude of the tTIS field was highest according to our simulations (see refs. 51,52 for recent validations of comparable simulations in cadaver experiments). Hence, we believe that the frequency- and reinforcement-dependent tTIS effects reported here cannot be explained by direct modulation of neural activity by the high-frequency fields. Yet, disentangling the neural effects of the low-frequency envelope and the high-frequency carrier appears to be an important next step to better characterize the mechanisms underlying tTIS<sup>47</sup>. We also note that the tTIS field strengths achieved according to our simulations (in the range of 0.5–0.6 V m<sup>-1</sup>) were sufficient to induce behavioural and neural effects, in line with recent data<sup>52,53</sup> (see also ref. 48). Determining the minimum effective dose for tTIS is an important line of future research given recent simulation results suggesting that stimulation via amplitude modulation with high-frequency carrier signals (such as those arising during tTIS) may require higher dosages than conventional electrical stimulation with low frequencies (such as during transcranial alternating current stimulation), probably due to the low-pass-filtering properties of neurons<sup>43,109</sup>.

Finally, the strength of the behavioural effects of tTIS can be considered small to medium<sup>110</sup> (d = 0.2–0.5). We note that these effect sizes are consistent with studies applying other types of non-invasive brain stimulation in healthy young adults, in the context of both motor learning (see ref. 111 for a meta-analysis) and reward tasks (for example, refs. 112,113), despite the much longer stimulation time used in these studies (between 3 and 20 times longer). Moreover, when expressed
relative to the plateau of performance in the task (Supplementary Fig. 1d), the effect of $tTIS_{80Hz}$ represents a complete disruption of a -24% reinforcement-related learning gain (Supplementary Fig. 1e). Overall, although they are moderate, we believe that the present effect sizes are relevant and consistent with what can be expected from the non-invasive brain stimulation literature.

## Limitations

The present study includes some limitations that we would like to acknowledge. First, at the imaging level, we did not find a significant effect of reinforcement at the whole-brain level. This might be due to the short duration of the task (6 × 40 s), combined with the fact that we did not couple reinforcement to monetary incentives, a manipulation known to boost striatal activity in the context of motor learning¹⁸. Yet, when considering BOLD activity in the striatal ROIs, we did find a significant effect of reinforcement, suggesting that our experimental manipulation did increase striatal activity but that the strength of the effect was insufficient to survive at the whole-brain level. Second, we did not find any effect of tTIS when considering averaged BOLD activity. Again, the short duration of the blocks may contribute to this non-significant effect. Another possible interpretation is that the effect of tTIS on BOLD activity is not uniform across participants, as it probably depends on individual anatomy and function of the targeted brain region, as observed for other non-invasive brain stimulation techniques¹¹⁴. Consistently, we found a correlation between levels of impulsivity and the neural effects of $tTIS_{80Hz}$ (both BOLD and connectivity; Supplementary Fig. 5). Importantly, though, when including learning as a behavioural regressor, we did find significant clusters of correlation specifically in the striatum (Fig. 3), suggesting that the behavioural effects were indeed related to modulation of activity in the target region. This result was significant when contrasting $tTIS_{80Hz}$ to the active control ($tTIS_{20Hz}$), but not to $tTIS_{Sham}$. Overall, we believe that the fMRI data do support the idea that the behavioural effects of the stimulation were indeed related to modulation of neural activity in the striatum, also in line with the present simulations on realistic head models (Fig. 1) and the connectivity results (Fig. 4). This idea is also in agreement with another recent study investigating the effects of tTIS on motor sequence learning⁵³. However, a limitation of the present dataset is the very short duration of stimulation and imaging for each experimental condition, which may explain some inconsistencies in the results. Hence, following this proof-of-concept study showing robust behavioural effects and related neural changes, future studies including longer fMRI and stimulation sessions are required to further confirm these results.

Finally, in the present study, the computational modelling was performed on a realistic, detailed head model (that is, the MIDA model⁵⁹; Methods). One limitation of this approach is that the electric field simulations do not take individual structural information into account. Such individual modelling would require information on brain anisotropy, an aspect that is likely to significantly influence tTIS exposure⁴⁴,¹¹⁵. However, in the present study, diffusion MRI to evaluate fractional anisotropy was not acquired. Future studies including diffusion MRI data will allow for personalized modelling, paving the way for individualized tTIS informed by brain structure³².

## Conclusion

The present findings show the ability of non-invasive striatal tTIS to interfere with reinforcement learning in humans through selective modulation of striatal activity and support the causal functional role of the human striatum in reinforcement motor learning. This deep brain stimulation was well tolerated and compatible with efficient blinding, suggesting that tTIS provides the option to circumvent the steep depth-focality trade-off of current non-invasive brain stimulation approaches in a safe and effective way. Overall, tTIS opens possibilities for the study of causal brain-behaviour relationships and for the

treatment of neuropsychiatric disorders associated with alterations of deep brain structures.

## Methods

### Participants

All participants gave their written informed consent in accordance with the Declaration of Helsinki and with the approval of the Cantonal Ethics Committee Vaud, Switzerland (project number 2020-00127). A total of 48 right-handed healthy volunteers participated in the study. Of these, 24 participants were enrolled for the main tTIS study (15 women, 25.3 ± 0.7 years old (mean ± s.e.)). Another group of 24 volunteers participated in the behavioural control experiment (Supplementary Fig. 1b-d; 14 women, 24.2 ± 0.5 years old). Handedness was determined via a shortened version of the Edinburgh Handedness Inventory¹¹⁶ (laterality index, 89.3 ± 2.14% for the main study and 86.4 ± 2.51% for the control experiment). None of the participants had any neurological or psychiatric disorder or were taking any centrally acting medication (see Supplementary Information for a complete list of exclusion criteria). Finally, all participants were asked to fill out a delay-discounting monetary choice questionnaire¹¹⁷, which evaluates the propensity of participants to choose smaller, sooner rewards over larger, later rewards, a preference commonly associated with choice impulsivity⁷⁵,¹¹⁸. The participants were financially compensated at a standard rate of 20 CHF per hour.

## Experimental procedures

The study employed a randomized, double-blind, sham-controlled design. Following screening and inclusion, the participants were invited to a single experimental session including the performance of a motor learning task with concurrent tTIS of the striatum and fMRI. Overall, the participants practised six blocks of trials, which resulted from the combination of two reinforcement feedback conditions (Reinf<sub>TYPE</sub>: Reinf<sub>ON</sub> or Reinf<sub>OFF</sub>) with three types of striatal stimulation (tTIS<sub>TYPE</sub>: tTIS<sub>Sham</sub>, tTIS<sub>20Hz</sub> or tTIS<sub>80Hz</sub>).

**Motor learning task.** *General aspects.* The participants practised an adaptation of a widely used force-tracking motor task⁵⁴,⁵⁵ with an fMRI-compatible fibre-optic grip-force sensor (Current Designs) positioned in their right hand. This task has the advantage of evaluating learning in a context in which movements have to be dynamically adjusted in response to constantly evolving sensory information. Such careful, continuous force control represents a situation that is relevant in many daily-life activities, including situations involving limited visual feedback such as when learning to drive or to manipulate fragile objects¹¹⁹. In these situations, the learner has to use somatosensory information in combination with information about task success to improve future motor commands, a process that might be particularly relevant in tasks where visual information is limited as well as in early stages of motor learning, when the desired sensory state is unknown⁵,⁵⁸,⁶⁸,¹²⁰. In addition to these elements, force-modulation tasks are relevant for rehabilitation as they can be used to evaluate motor function in clinical populations¹²¹,¹²².

The task was developed in MATLAB 2018 (MathWorks) exploiting the Psychophysics Toolbox extensions¹²³,¹²⁴ and was displayed on a computer screen with a refresh rate of 60 Hz. The task required the participants to squeeze the force sensor to control a cursor displayed on the screen. Increasing the exerted force resulted in the cursor moving vertically and upward in a linear way. Each trial started with a preparatory period in which a sidebar appeared at the bottom of the screen (Fig. 1a). After a variable time interval (0.9 to 1.1 s), a cursor (a black circle) popped up in the sidebar, and simultaneously a target (a grey larger circle with a cross in the middle) appeared, indicating the start of the movement period. The participants were asked to modulate the force applied on the transducer to keep the cursor as close as possible to the centre of the target. The target moved in a sequential way along a
single vertical axis for 7 s. The maximum force required (that is, the force required to reach the target when it was in the uppermost part of the screen; Max Target<sub>Force</sub>) was set at 4% of MVC evaluated at the beginning of the experiment. This low force level was chosen on the basis of pilot experiments to limit muscular fatigue. Finally, each trial ended with a blank screen displayed for 2 s before the beginning of the next trial.

*Trial types and reinforcement manipulation.* During the experiment, the participants were exposed to different types of trials (Fig. 1a and Supplementary Video 1). In test trials, the cursor remained on the screen, and the target was consistently displayed in grey for the whole duration of the trial. These trials served to evaluate pre- and post-training performance for each block, without any disturbance. In Reinf<sub>ON</sub> and Reinf<sub>OFF</sub> trials (used during training only), we provided only partial visual feedback to the participants to increase the impact of reinforcement on learning<sup>5,56-58</sup>. The cursor was only intermittently displayed during the trial: it was always displayed in the first second of the trial and then disappeared for a total of 4.5 s randomly split in the remaining time by bits of 0.5 s. The cursor was therefore displayed 35.7% of the time during these trials (2.5 s over the 7 s trial). Importantly, unlike the cursor, the target always remained on the screen for the whole trial, and the participants were instructed to continue to track the target even when the cursor was away.

In addition to this visual manipulation, in Reinf<sub>ON</sub> trials, the participants also trained with reinforcement feedback indicating success or failure of the tracking in real time. The participants were informed that, during these trials, the colour of the target would vary as a function of their performance: the target was displayed in green when tracking was considered successful and in red when it was considered a failure. Online success on the task was determined on the basis of the Error, defined as the absolute force difference between the force required to be in the centre of the target and the exerted force<sup>5,54,55,58</sup>. The Error, expressed as a percentage of MVC, was computed for each frame refresh and allowed to classify a sample as successful or not on the basis of a closed-loop reinforcement schedule<sup>8</sup>. More specifically, for each training trial, a force sample (recorded at 60 Hz, corresponding to the refresh rate of the monitor) was considered successful if the computed Error was below the median Error over the four previous trials at this specific sample. Put differently, to be successful, the participants had to constantly beat their previous performance. This closed-loop reinforcement schedule allowed us to deliver consistent reinforcement feedback across individuals and conditions (see the control analysis on success rates in the Supplementary Information), while maximizing uncertainty on the presence of reinforcement, an aspect that is crucial for efficient reinforcement motor learning<sup>125</sup>. In addition to this closed-loop design, samples were also considered successful if the cursor was very close to the centre of the target (that is, within one radius around the centre, corresponding to an Error below 0.2% of MVC). This was done to prevent any conflict between visual information (provided by the position of the cursor relative to the target) and reinforcement feedback (provided by the colour of the target), which could occur in situations of extremely good performance (when the closed-loop Error cut-off is below 0.2% of MVC).

As a control, Reinf<sub>OFF</sub> trials were similar to Reinf<sub>ON</sub> trials, with the only difference being that the displayed colours were either cyan or magenta and were generated randomly. The participants were explicitly told that, in this condition, the colours were displayed randomly and could be ignored. The visual properties of the target in the Reinf<sub>OFF</sub> condition were designed to match the Reinf<sub>ON</sub> condition in terms of relative luminance (cyan (RGB, 127.5, 242.1, 255)) matched to green (127.5, 255, 127.5) and magenta (211.7, 127.5, 255) to red (255, 127.5, 127.5)) and average frequency of change in colours (that is, the average number of changes in colours divided by the total duration of a trial; Supplementary Information).

In this task, training trials differed from test trials regarding not only the colour of the target (red/green or cyan/magenta in training

trials and grey in test trials) but also the visual feedback experienced (partial and full visual feedback in training and test trials, respectively). This choice was motivated by several reasons. First, we wanted to evaluate learning in the classical, unperturbed, version of the force-tracking task<sup>34,55</sup>, which is compatible with clinical translation. Second, on the basis of additional behavioural data on another group of participants (n = 24; Supplementary Fig. 1b-d), we found that significant effects of reinforcement on learning were observed only when training was performed with partial visual feedback (displayed for 35.7% of the trial time, as in the present study), in line with previous results<sup>36,61</sup>. However, this additional study also revealed very limited improvement of performance during training with partial visual feedback, potentially due to ceiling effects on performance in this condition. Yet, the improvement of performance when comparing the pre- and post-training assessments suggested that practising the task with partial visual feedback still induced significant learning of the skill. Finally, the change in visual feedback between training and post-training was the same in all experimental conditions; this aspect of the task is therefore unlikely to explain the reinforcement as well as the stimulation effects reported here.

Even though our study focused on reinforcement motor learning, it is worth mentioning that other learning mechanisms such as error-based or strategic processes are likely to be also engaged during the force-tracking task and may have recruited other brain regions beyond the striatum<sup>4</sup>. Notably, though, our protocol was specifically designed to compare learning in the Reinf<sub>ON</sub> and Reinf<sub>OFF</sub> conditions in the same individuals while keeping the other parameters of the task constant, to specifically isolate the contribution of reinforcement processes in motor learning.

*Motor learning protocol.* After receiving standardized instructions about the force-tracking task, the participants practised five blocks of familiarization (total of 75 trials) without tTIS. The first block of familiarization included 20 trials with the target moving in a regular fashion (0.5 Hz sinusoid). Then, in a second block of familiarization, the participants performed 35 trials of practice with an irregular pattern, with the same properties as the training patterns (see below). Finally, we introduced the reinforcement manipulation and let the participants perform two short blocks (eight trials each) including Reinf<sub>ON</sub> and Reinf<sub>OFF</sub> trials. These first four blocks of familiarization were performed outside the MRI environment. A final familiarization block (four trials) was performed after installation in the scanner, to allow the participants to get used to performing the task in the MRI machine. This long familiarization allowed the participants to get acquainted with the use of the force sensor before the beginning of the experiment.

During the main part of the experiment, the participants performed six blocks of trials in the MRI machine with concurrent striatal tTIS (Fig. 1b). Each block was composed of 4 pre-training trials followed by 24 training and 8 post-training trials. Pre- and post-training trials were performed in test conditions, without tTIS, and were used to evaluate motor learning. Training trials were performed with or without reinforcement feedback and with concomitant striatal tTIS and were used as a proxy of motor performance. During training, trials were interspersed with 25 s resting periods every four trials (used for fMRI contrasts; see below). The order of the six experimental conditions was pseudo-randomized across participants: the six blocks were divided into three pairs of blocks with the same tTIS condition, and each pair was then composed of one Reinf<sub>ON</sub> and one Reinf<sub>OFF</sub> block. Within this structure, the order of the tTIS<sub>TYPE</sub> and Reinf<sub>TYPE</sub> conditions were balanced among the 24 participants. Hence, this randomization allowed us to ensure that any order effect that may arise from the repetition of the learning blocks would have the same impact on each experimental condition (for example, four participants experienced tTIS<sub>80Hz</sub>-Reinf<sub>ON</sub> in the first block, four other participants in the second block, four in the third block and so on).

As mentioned above, the protocol involved multiple evaluations of motor learning within the same experimental session. To limit
carry-over effects from one block to the next, each experimental block was associated with a different pattern of movement of the target (Supplementary Fig. 1a). Put differently, in each block, the participants had to generate a new pattern of force to successfully track the target. To balance the patterns' difficulty, they all consisted of the summation of five sinusoids of variable frequency (range, 0.1–1.5 Hz) that presented the following properties: the average force was between 45% and 55% of MaxTarget<sub>Force</sub>, the absolute average derivative was between 54% and 66% of MaxTarget<sub>Force</sub> per second and the number of peaks was 14 (defined as an absolute change of force of at least 1% of MaxTarget<sub>Force</sub>). These parameters were determined on the basis of pilot experiments to obtain a relevant level of difficulty for young healthy adults and consistent learning across the different patterns.

**Striatal tTIS.** *General concept.* tTIS is an innovative non-invasive brain stimulation approach, in which two or more independent stimulation channels deliver high-frequency currents in the kHz range (oscillating at *f*<sub>1</sub> and *f*<sub>1</sub> + *Δf*; Fig. 1c). These high-frequency currents are assumed to be too high to effectively modulate neuronal activity<sup>41,50,126</sup>. Still, by applying a small shift in frequency, they result in a modulated electric field with the envelope oscillating at the low-frequency *Δf* (target frequency) where the two currents overlap. The peak of the modulated envelope amplitude can be steered towards specific areas located deep in the brain, by tuning the positions of the electrodes and the current ratio across stimulation channels<sup>41</sup> (Fig. 1c,d). On the basis of these properties, tTIS has been shown to be able to focally target the activity of deep structures in rodents, without engaging overlying tissues<sup>41</sup>. Here we applied tTIS via surface electrodes applying a low-intensity, sub-threshold protocol following the currently accepted cut-offs and safety guidelines for low-intensity transcranial electric stimulation in humans<sup>127</sup>.

**Stimulators.** The currents for tTIS were delivered by two independent DS5 isolated bipolar constant current stimulators (Digitimer Ltd). The stimulation patterns were generated using a custom-based MATLAB graphical user interface and transmitted to the current sources using a standard digital-analogue converter (DAQ USB-6216, National Instruments). Finally, an audio transformer was added between stimulators and participants to avoid possible direct current accumulation.

**Stimulation protocols.** During the six training blocks, we applied three different types of striatal tTIS (two blocks each): a stimulation with a tTIS envelope modulated at 20 Hz (tTIS<sub>20Hz</sub>), a stimulation with a tTIS envelope modulated at 80 Hz (tTIS<sub>80Hz</sub>) and a sham stimulation (tTIS<sub>Sham</sub>). For tTIS<sub>20Hz</sub>, the posterior stimulation channel (TP7-TP8; see below) delivered a 1.99 kHz stimulation, while the anterior one delivered a 2.01 kHz stimulation (*Δf* = 20 Hz). For tTIS<sub>80Hz</sub>, the posterior and anterior channels delivered 1.96 kHz and 2.04 kHz, respectively (*Δf* = 80 Hz). Hence, in both conditions, the high-frequency component was comparable, and the only difference was *Δf*. During each block, tTIS was applied for five minutes (6 × 50 s) during training. Each stimulation period started and ended with currents ramping up and down, respectively, for 5 s. tTIS was applied only while the participants were performing the motor task and not during resting periods or pre- and post-training assessments. Finally, tTIS<sub>Sham</sub> consisted of a ramping-up (5 s) immediately followed by a ramping-down (5 s) of 2 kHz currents delivered without any shift in frequency. This condition allowed us to mimic the sensations experienced during the active conditions tTIS<sub>20Hz</sub> and tTIS<sub>80Hz</sub>, while delivering minimal brain stimulation (Supplementary Fig. 3). A trigger was sent 5 s before the beginning of each trial to align the beginning of the task and the beginning of the frequency shift after the ramp-up. Other tTIS parameters were set as follows: current intensity per stimulation channel, 2 mA (baseline-to-peak); electrode type, round, conductive rubber with conductive cream/paste; electrode size, 3 cm<sup>2</sup> (see the ContES checklist in the Supplementary Information for more details).

The stimulation was applied in the MRI environment (Siemens 3T MAGNETOM Prisma; Siemens Healthcare) using a standard RF filter module and MRI-compatible cables (neuroConn GmbH). The technological, safety and noise tests and the methodological factors can be found in the Supplementary Information (Supplementary Table 4) and are based on the ContES checklist<sup>128</sup>.

**Modelling.** Electromagnetic simulations were carried out to identify optimized electrode placement and current steering parameters. The simulations were performed using the MIDA head model<sup>59</sup>, a detailed anatomical head model featuring >100 distinguished tissues and regions that was derived from multi-modal image data of a healthy female volunteer. Importantly, for brain stimulation modelling, the model differentiates different scalp layers, skull layers, grey and white matter, cerebrospinal fluid, and the dura and accounts for electrical conductivity anisotropy and neural orientation on the basis of diffusion tensor imaging data. Circular electrodes (radius, 0.7 cm) were positioned on the skin according to the 10–10 system, and the electromagnetic exposure was computed using the ohmic-current-dominated electro-quasistatic solver from Sim4Life v.5.0 (ZMT Zurich MedTech AG), which is suitable due to the dominance of ohmic currents over displacement currents and the long wavelength compared with the simulation domain<sup>129</sup>. Dielectric properties were assigned on the basis of the ITIS Tissue Properties Database v.4.0 (ref. 130). Rectilinear discretization was performed, and grid convergence as well as solver convergence analyses were used to ensure negligible numerical uncertainty, resulting in a grid that included more than 54 million voxels. Dirichlet voltage boundary conditions and then current normalization were applied. The electrode-head interface contact was treated as ideal. tTIS exposure was quantified according to the maximum modulation envelope magnitude formula from Grossman et al.<sup>41</sup>. A sweep over 960 permutations of the four electrode positions was then performed, considering symmetric and asymmetric montages with parallel (sagittal and coronal) or crossing current paths, while quantifying bilateral striatum (putamen (BNA regions 225, 226, 229 and 230), caudate (BNA regions 219, 220, 227 and 228) and NAc (BNA regions 223 and 224)) exposure performance according to three metrics: (1) target exposure strength, (2) focality ratio (the ratio of target tissue volume above the threshold compared to the whole-brain tissue volume above the threshold, a measure of stimulation selectivity) and (3) activation ratio (the percentage of target volume above the threshold with respect to the total target volume, a measure of target coverage). We defined the threshold as the 98th volumetric iso-percentile level of the tTIS. From the resulting Pareto-optimal front, two configurations stood out particularly: one that maximized focality and activation (AF3-AF4 and P7-P8) and one that accepted a reduction of these two metrics by a quarter, while increasing the target exposure strength by more than 50% (F3-F4 and TP7-TP8). This last montage was selected to ensure sufficient tTIS exposure in the striatum<sup>53</sup> (Fig. 1c,d).

**Electrode positioning and stimulation-related sensations.** On the basis of the modelling approach described above, we defined the stimulation electrode positions in the framework of the EEG 10–10 system<sup>131</sup>. The optimal montage leading in terms of target (that is, bilateral striatum) exposure strength and selectivity was composed of the following electrodes: F3, F4, TP7 and TP8. Their locations were marked with a pen on the scalp, and, after skin preparation (cleaning with alcohol), round conductive rubber electrodes of 3 cm<sup>2</sup> were placed, adding a conductive paste (Ten20, Weaver; or Abralyt HiCl, Easycap GmbH) as an interface to the skin. The electrodes were held in position with tape, and the cables were oriented towards the top to allow good positioning inside the scanner. Impedances were checked and optimized until they were below 20 kΩ (ref. 48). Once good contact was obtained, we tested different intensities of stimulation for each stimulation protocol to familiarize the participants with the perceived sensations and
to systematically document them. tTIS<sub>Sham</sub>, tTIS<sub>20Hz</sub> and tTIS<sub>80Hz</sub> were applied for 20 seconds with the following increasing current amplitudes per channel: 0.5 mA, 1 mA, 1.5 mA and 2 mA. The participants were asked to report any kind of sensation, and, if a sensation was felt, they were asked to grade the intensity from 1 to 3 (light to strong) as well as give at least one adjective to describe it (Supplementary Fig. 3). Following this step, the cables were removed and replaced by MRI-compatible cables, and a bandage was added to apply pressure on the electrodes and keep them in place. An impedance check was repeated in the MRI machine right before the training and then again at the end of all recordings.

**MRI data acquisition.** Structural and functional images were acquired using a 3T MAGNETOM PRISMA scanner (Siemens). T1-weighted images were acquired via the 3D MPRAGE sequence with the following parameters: repetition time (TR), 2.3 s; echo time (TE), 2.96 ms; flip angle, 9°; slices, 192; voxel size, $1 \times 1 \times 1$ mm$^3$; field-of-view (FOV), 256 mm. Anatomical T2 images were also acquired with the following parameters: TR, 3 s; TE, 409 ms; flip angle, 120°; slices, 208; voxel size, $0.8 \times 0.8 \times 0.8$ mm$^3$; FOV, 320 mm. Finally, functional images were recorded using echo-planar imaging sequences with the following parameters: TR, 1.25 s; TE, 32 ms; flip angle, 58°; slices, 75; voxel size, $2 \times 2 \times 2$ mm$^3$; FOV, 112 mm.

**Data and statistical analyses**

Data and statistical analyses were carried out with MATLAB 2018a (MathWorks) and the R software environment (version 2021) for statistical computing and graphics¹³². Robust linear regressions were fitted with the MATLAB function robustfit. LMMs were fitted using the lmer function of the lme4 package in R¹³³. As random effects, we added intercepts for participants and block. The normality of residuals and the homoscedasticity of the data were systematically checked, and logarithmic transformations were applied when necessary (that is, when the skewness of the residuals' distribution was not between -2 and 2 (ref. 134) or when homoscedasticity was violated on the basis of visual inspection). To mitigate the impact of isolated influential data points on the outcome of the final model, we used tools of the influence.ME package to detect and remove influential cases on the basis of the following criterion: distance > 4 × mean distance¹³⁵. Statistical significance was determined using the anova function with Satterthwaite's approximations of the lmerTest package¹³⁶. For specific post hoc comparisons, we conducted two-sided pairwise tests by computing estimated marginal means with the emmeans package with Tukey adjustment of *P* values to correct for multiple comparisons¹³⁷. Standardized effect size measures were obtained using the *eff_size* function of the emmeans package¹³⁸ and the *eta_squared* function of the effectsize package¹³⁹. The level of significance was set at *P* < 0.05.

**Behavioural data.** *Evaluation of motor learning.* The main goal of the present study was to evaluate the influence of striatal tTIS on reinforcement motor learning. To do so, we first removed trials in which participants did not react within 1 s after the appearance of the cursor and target, considering that these extremely long preparation times may reflect substantial fluctuations in attention¹⁴⁰. This occurred extremely rarely (0.52% of the whole dataset). For each participant and each trial, we then quantified the tracking Error as the absolute force difference between the applied and required force, as done previously⁵,⁵⁵,⁵⁸. Tracking performance during training and post-training trials was then normalized according to each participant's initial level by expressing the Error data as a percentage of the average pre-training Error for each block. To test our main hypothesis predicting specific effects of striatal tTIS on reinforcement motor learning, we performed an LMM on the post-training data with tTIS<sub>TYPE</sub> and Reinf<sub>TYPE</sub> as fixed effects. We then ran the same analysis on the training data, to evaluate whether striatal tTIS also impacted motor performance while stimulation was being delivered.

As a control, we checked that initial performance at pre-training was not different between conditions with an LMM on the Error data obtained at pre-training. Again, tTIS<sub>TYPE</sub> and Reinf<sub>TYPE</sub> were considered as fixed effects. Finally, another LMM was fitted with the fixed effect tTIS<sub>TYPE</sub> to verify that the amount of positive reinforcement (as indicated by a green target) in the Reinf<sub>ON</sub> blocks was similar across tTIS<sub>TYPES</sub>.

**fMRI data.** *Imaging preprocessing.* We analysed the functional imaging data using Statistical Parametric Mapping v.12 (Wellcome Department of Cognitive Neurology) implemented in MATLAB R2018a (MathWorks). All functional images underwent a common preprocessing procedure including the following steps: slice time correction, spatial realignment to the first image, normalization to the standard MNI space and smoothing with a 6 mm full-width half-maximal Gaussian kernel. T1 anatomical images were then co-registered to the mean functional image and segmented. This allowed us to obtain bias-corrected grey and white matter images by normalizing the functional images via the forward deformation field. To select participants with acceptable levels of head movement, framewise displacement was calculated for each run. A visual check of both non-normalized and normalized images was performed to ensure good preprocessing quality. Finally, possible tTIS-related artefacts were investigated on the basis of signal-to-noise ratio maps (see below).

**Signal-to-noise ratio.** Total signal-to-noise ratio maps were computed to check the presence of possible artefacts induced by the electrical stimulation. The values were calculated for each voxel by dividing the mean of the voxel time series by its standard deviation. Spherical ROIs were then defined both underneath the tTIS electrodes and at four different locations distant from the electrodes as a control. The centre of each spherical ROI was obtained by projecting the standard MNI coordinates of each electrode on the scalp¹⁴¹ towards the centre of the brain. After visual inspection of the ROIs, average total signal-to-noise ratio maps were extracted within each sphere. An LMM was used to compare the average signal-to-noise ratio underneath the electrodes versus the control regions and between stimulation protocols. The results of this analysis are presented in the Supplementary Information (Supplementary Fig. 6).

**Task-based BOLD activity analysis.** A general linear model was implemented at the single-participant level to estimate signal amplitude. Eight regressors were included in the model: six head motion parameters (displacement and rotation) and normalized time series within the white matter and the cerebrospinal fluid. Linear contrasts were then computed to estimate specific activity during the motor task with respect to resting periods. Functional activation was also extracted within specific ROIs individually defined on the basis of structural images. More specifically, the Freesurfer recon-all function was run on the basis of the structural T1w and T2w images (Freesurfer v.7.1.1, https://surfer.nmr.mgh.harvard.edu/; coded in Bash (v.4.4.20(1)-release) and Python (v.3.8.3)). The BNA parcellation was derived on the individual participant space, and the selected ROIs were then co-registered to the functional images and normalized to the MNI space. BOLD activity within the individual striatal masks was averaged and compared between different striatal nuclei—namely, the putamen (BNA regions 225, 226, 229 and 230), caudate (BNA regions 219, 220, 227 and 228) and NAc (BNA regions 223 and 224). Comparisons between conditions were presented for uncorrected *P* = 0.001 at the voxel level and multiple comparison corrected at the cluster level to reduce the false discovery rate, *P* = 0.05.

**Effective connectivity analyses.** As an additional investigation, we computed task-modulated effective functional connectivity by means of the CONN toolbox 2021a (www.nitrc.org/projects/conn, RRID:SCR_009550) running in MATLAB R2018a (MathWorks). An
additional denoising step was added by applying band-pass filtering from 0.01 to 0.1 Hz and by regressing potential confounders (white matter, cerebrospinal fluid and realignment parameters). After that, gPPI connectivity was extracted within specific pre-defined customized sub-networks: a reward network and a motor network. gPPI evaluates condition-specific changes in effective connectivity, defined as the directed effect that one brain region has on another under some model of neuronal coupling¹⁴². In particular, gPPI considers a series of equations in which activity in a ROI (pre-defined frontal areas in our case) depends on a specific condition (the 'psychological' factor) and on activity in the seed region (striatum here, the 'physiological' factor). By solving these equations, it is possible to determine a coefficient that represents task modulation of effective connectivity¹⁴³. Importantly, task-related changes in effective connectivity are expressed relative to rest, and therefore values closer to 0 reflect a connectivity similar to the resting state.

The reward network was defined as follows: two regions within the striatum, the NAc (BNA regions 223 and 224) and the ventro-medial putamen (BNA regions 225 for left and 226 for right), and two frontal areas, the ACC (BNA regions 177, 179 and 183 for left and 178, 180 and 184 for right) and the orbitofrontal cortex within the vmPFC (BNA regions 41, 45, 47, 49 and 187 for left and 42, 46, 48, 50 and 188 for right). The motor network included the following areas: the dorso-lateral putamen (BNA 229 for left and 230 for right), the dorsal caudate (BNA regions 227 for left and 228 for right), the medial part of the SMA (BNA regions 9 for left and 10 for right) and the part of the M1 associated with upper limb function (BNA regions 57 for left and 58 for right). We considered connectivity in the left and right motor and reward networks regardless of laterality. These ROIs were selected on the basis of the following rationale. First, they are consistent with previous literature on reinforcement learning of motor skills⁶⁸,⁹⁰,¹⁴⁴,¹⁴⁵. Second, there is structural and functional evidence for these fronto-striatal connections¹⁴⁶,¹⁴⁷. Third, the frontal areas included in the analyses are well-established hubs of the motor learning (M1 and SMA; see ref. 12 for a meta-analysis) and reward networks (vmPFC and ACC; see ref. 11 for a meta-analysis). Finally, gPPI was also extracted within a control language network, defined on the basis of the functional atlas described by Shirer et al.⁷⁰.

## Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

## Data availability

All data necessary to generate the main results and figures are available in the Zenodo repository (https://doi.org/10.5281/zenodo.10458885)¹⁴⁸. The BNA was used and can be downloaded from http://atlas.brain-netome.org/. The tissue properties used for the modelling of electric fields were based on the IT'STissue Properties Database v.4.0 and can be downloaded here: https://itis.swiss/virtual-population/tissue-properties/overview/.

## Code availability

The scripts necessary to generate the main results are available in the Zenodo repository (https://doi.org/10.5281/zenodo.10458885)¹⁴⁸.

## References

1. Neftci, E. O. & Averbeck, B. B. Reinforcement learning in artificial and biological systems. *Nat. Mach. Intell.* **1**, 133–143 (2019).

2. Schultz, W. Neuronal reward and decision signals: from theories to data. *Physiol. Rev.* **95**, 853–951 (2015).

3. Dhawale, A. K., Smith, M. A. & Ölveczky, B. P. The role of variability in motor learning. *Annu. Rev. Neurosci.* **40**, 479–498 (2017).

4. Spampinato, D. & Celnik, P. Multiple motor learning processes in humans: defining their neurophysiological bases. *Neuroscientist* **27**, 246–267 (2021).

5. Vassiliadis, P. et al. Reward boosts reinforcement-based motor learning. *iScience* **24**, 102821 (2021).

6. Huang, V. S., Haith, A., Mazzoni, P. & Krakauer, J. W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. *Neuron* **70**, 787–801 (2011).

7. Galea, J. M., Mallia, E., Rothwell, J. & Diedrichsen, J. The dissociable effects of punishment and reward on motor learning. *Nat. Neurosci.* **18**, 597–602 (2015).

8. Therrien, A. S., Wolpert, D. M. & Bastian, A. J. Effective reinforcement learning following cerebellar damage requires a balance between exploration and motor noise. *Brain* **139**, 101–114 (2016).

9. Vassiliadis, P., Derosiere, G. & Duque, J. Beyond motor noise: considering other causes of impaired reinforcement learning in cerebellar patients. *eNeuro* **6**, ENEURO.0458-18.2019 (2019).

10. Widmer, M. et al. Reward during arm training improves impairment and activity after stroke: a randomized controlled trial. *Neurorehabil. Neural Repair* **36**, 140–150 (2022).

11. Bartra, O., McGuire, J. T. & Kable, J. W. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. *NeurolImage* **76**, 412–427 (2013).

12. Hardwick, R. M., Rottschy, C., Miall, R. C. & Eickhoff, S. B. A quantitative meta-analysis and review of motor learning in the human brain. *NeurolImage* **67**, 283–297 (2013).

13. Haber, S. N. Corticostriatal circuitry. *Dialogues Clin. Neurosci.* **18**, 7–21 (2016).

14. Balleine, B. W., Delgado, M. R. & Hikosaka, O. The role of the dorsal striatum in reward and decision-making. *J. Neurosci.* **27**, 8161–8165 (2007).

15. Piray, P., den Ouden, H. E. M., van der Schaaf, M. E., Toni, I. & Cools, R. Dopaminergic modulation of the functional ventrodorsal architecture of the human striatum. *Cereb. Cortex* **27**, 485–495 (2017).

16. Hori, Y. et al. Ventral striatum links motivational and motor networks during operant-conditioned movement in rats. *NeurolImage* **184**, 943–953 (2019).

17. Wachter, T., Lungu, O. V., Liu, T., Willingham, D. T. & Ashe, J. Differential effect of reward and punishment on procedural learning. *J. Neurosci.* **29**, 436–443 (2009).

18. Widmer, M., Ziegler, N., Held, J., Luft, A. & Lutz, K. Rewarding feedback promotes motor skill consolidation via striatal activity. *Prog. Brain Res.* **229**, 303–323 (2016).

19. Berke, J. D. Fast oscillations in cortical–striatal networks switch frequency following rewarding events and stimulant drugs. *Eur. J. Neurosci.* **30**, 848–859 (2009).

20. van der Meer, M. A. A. et al. Integrating early results on ventral striatal gamma oscillations in the rat. *Front. Neurosci.* **4**, 300 (2010).

21. van der Meer, M. A. A. & Redish, A. D. Low and high gamma oscillations in rat ventral striatum have distinct relationships to behavior, reward, and spiking activity on a learned spatial decision task. *Front. Integr. Neurosci.* **3**, 9 (2009).

22. Dwiel, L. L., Khokhar, J. Y., Connerney, M. A., Green, A. I. & Doucette, W. T. Finding the balance between model complexity and performance: using ventral striatal oscillations to classify feeding behavior in rats. *PLoS Comput. Biol.* **15**, 4 (2019).

23. Matsumoto, J. et al. Neuronal responses in the nucleus accumbens shell during sexual behavior in male rats. *J. Neurosci.* **32**, 1672–1686 (2012).

24. Kalenscher, T., Lansink, C. S., Lankelma, J. V. & Pennartz, C. M. A. Reward-associated gamma oscillations in ventral striatum are regionally differentiated and modulate local firing activity. *J. Neurophysiol.* **103**, 1658–1672 (2010).
25. Cohen, M. X. et al. Good vibrations: cross-frequency coupling in the human nucleus accumbens during reward processing. *J. Cogn. Neurosci.* **21**, 875–889 (2009).

26. Sepe-Forrest, L., Carver, F. W., Quentin, R., Holroyd, T. & Nugent, A. C. Basal ganglia activation localized in MEG using a reward task. *NeuroImage Rep.* **1**, 100034 (2021).

27. Herrojo-Ruiz, M. et al. Involvement of human internal globus pallidus in the early modulation of cortical error-related activity. *Cereb. Cortex* **24**, 1502–1517 (2014).

28. Jenkinson, N. & Brown, P. New insights into the relationship between dopamine, beta oscillations and motor function. *Trends Neurosci.* **34**, 611–618 (2011).

29. Brown, P. Abnormal oscillatory synchronisation in the motor system leads to impaired movement. *Curr. Opin. Neurobiol.* **17**, 656–664 (2007).

30. McCarthy, M. M. et al. Striatal origin of the pathologic beta oscillations in Parkinson's disease. *Proc. Natl Acad. Sci. USA* **108**, 11620–11625 (2011).

31. Kondabolu, K. et al. Striatal cholinergic interneurons generate beta and gamma oscillations in the corticostriatal circuit and produce motor deficits. *Proc. Natl Acad. Sci. USA* **113**, 3159–3168 (2016).

32. Silberstein, P. et al. Cortico-cortical coupling in Parkinson's disease and its modulation by therapy. *Brain* **128**, 1277–1291 (2005).

33. Williams, Z. M. & Eskandar, E. N. Selective enhancement of associative learning by microstimulation of the anterior caudate. *Nat. Neurosci.* **9**, 562–568 (2006).

34. Nakamura, K. & Hikosaka, O. Facilitation of saccadic eye movements by postsaccadic electrical stimulation in the primate caudate. *J. Neurosci.* **26**, 12885–12895 (2006).

35. Deng, Z. D., Lisanby, S. H. & Peterchev, A. V. Electric field depth-focality tradeoff in transcranial magnetic stimulation: simulation comparison of 50 coil designs. *Brain Stimul.* **6**, 1–13 (2013).

36. Wagner, T. et al. Transcranial direct current stimulation: a computer-based human model study. *NeuroImage* **35**, 1113–1124 (2007).

37. Nickchen, K. et al. Reversal learning reveals cognitive deficits and altered prediction error encoding in the ventral striatum in Huntington's disease. *Brain Imaging Behav.* **11**, 1862–1872 (2017).

38. Schmidt, L. et al. Disconnecting force from money: effects of basal ganglia damage on incentive motivation. *Brain* **131**, 1303–1310 (2008).

39. Seymour, B. et al. Deep brain stimulation of the subthalamic nucleus modulates sensitivity to decision outcome value in Parkinson's disease. *Sci. Rep.* **6**, 32509 (2016).

40. Atkinson-Clement, C. et al. Effects of subthalamic nucleus stimulation and levodopa on decision-making in Parkinson's disease. *Mov. Disord.* **34**, 377–385 (2019).

41. Grossman, N. et al. Noninvasive deep brain stimulation via temporally interfering electric fields. *Cell* **169**, 1029–1041.e16 (2017).

42. Song, S., Zhang, J., Tian, Y., Wang, L. & Wei, P. Temporal interference stimulation regulates eye movements and neural activity in the mice superior colliculus. *Proc. Annual International Conference of the IEEE Engineering in Medicine and Biology Society* 6231–6234 (IEEE, 2021); https://doi.org/10.1109/EMBC46164.2021.9629968

43. Esmaeilpour, Z., Kronberg, G., Reato, D., Parra, L. C. & Bikson, M. Temporal interference stimulation targets deep brain regions by modulating neural oscillations. *Brain Stimul.* **14**, 55–65 (2021).

44. Rampersad, S. et al. Prospects for transcranial temporal interference stimulation in humans: a computational study. *NeuroImage* **202**, 116124 (2019).

45. von Conta, J. et al. Interindividual variability of electric fields during transcranial temporal interference stimulation (tTIS). *Sci. Rep.* **11**, 20357 (2021).

46. Cao, J., Doiron, B., Goswami, C. & Grover, P. The mechanics of temporal interference stimulation. Preprint at bioRxiv https://doi.org/10.1101/2020.04.23.051870 (2020).

47. Mirzakhaliili, E., Barra, B., Capogrosso, M. & Lempka, S. F. Biophysics of temporal interference stimulation. *Cell Syst.* **11**, 557–572.e5 (2020).

48. von Conta, J. et al. Benchmarking the effects of transcranial temporal interference stimulation (tTIS) in humans. *Cortex* **154**, 299–310 (2022).

49. Ma, R. et al. High gamma and beta temporal interference stimulation in the human motor cortex improves motor functions. *Front. Neurosci.* **15**, 800436 (2022).

50. Hutcheon, B. & Yarom, Y. Resonance, oscillation and the intrinsic frequency preferences of neurons. *Trends Neurosci.* **23**, 216–222 (2000).

51. Acerbo, E. et al. Focal non-invasive deep-brain stimulation with temporal interference for the suppression of epileptic biomarkers. *Front. Neurosci.* **16**, 1–12 (2022).

52. Violante, I. R. et al. Non-invasive temporal interference electrical stimulation of the human hippocampus. *Nat. Neurosci.* **26**, 1994–2004 (2023).

53. Wessel, M. J. et al. Noninvasive theta-burst stimulation of the human striatum enhances striatal activity and motor skill learning. *Nat. Neurosci.* **26**, 2005–2016 (2023).

54. Steel, A., Silson, E. H., Stagg, C. J. & Baker, C. I. The impact of reward and punishment on skill learning depends on task demands. *Sci. Rep.* **6**, 36056 (2016).

55. Abe, M. et al. Reward improves long-term retention of a motor memory through induction of offline memory gains. *Curr. Biol.* **21**, 557–562 (2011).

56. Izawa, J. & Shadmehr, R. Learning from sensory and reward prediction errors during motor adaptation. *PLoS Comput. Biol.* **7**, 3 (2011).

57. Mawase, F., Uehara, S., Bastian, A. J. & Celnik, P. Motor learning enhances use-dependent plasticity. *J. Neurosci.* **37**, 2673–2685 (2017).

58. Vassiliadis, P., Lete, A., Duque, J. & Derosiere, G. Reward timing matters in motor learning. *iScience* **25**, 104290 (2022).

59. Iacono, M. I. et al. MIDA: a multimodal imaging-based detailed anatomical model of the human head and neck. *PLoS ONE* **10**, e0124126 (2015).

60. Areshenkoff, C. N., de Brouwer, A. J., Gale, D. J., Nashed, J. Y. & Gallivan, J. P. Separate and shared low-dimensional neural architectures for error-based and reinforcement motor learning. Preprint at biorXiv https://doi.org/10.1101/2022.08.16.504134 (2022).

61. Cashaback, J. G. A., McGregor, H. R., Mohatarem, A. & Gribble, P. L. Dissociating error-based and reinforcement-based loss functions during sensorimotor learning. *PLoS Comput. Biol.* **13**, 7 (2017).

62. Floyer-Lea, A. & Matthews, P. M. Changing brain networks for visuomotor control with increased movement automaticity. *J. Neurophysiol.* **92**, 2405–2412 (2004).

63. Floyer-Lea, A. & Matthews, P. M. Distinguishable brain activation networks for short- and long-term motor skill learning. *J. Neurophysiol.* **94**, 512–518 (2005).

64. Jiang, T. Brainnetome: a new -ome to understand the brain and its disorders. *NeuroImage* **80**, 263–272 (2013).

65. Graybiel, A. M. & Grafton, S. T. The striatum: where skills and habits meet. *Cold Spring Harb. Perspect. Biol.* **7**, 3 (2015).

66. McLaren, D. G., Ries, M. L., Xu, G. & Johnson, S. C. A generalized form of context-dependent psychophysiological interactions
(gPPI): a comparison to standard approaches. *NeurolImage* **61**, 1277-1286 (2012).

67. Codol, O., Holland, P. J., Manohar, S. G. & Galea, J. M. Reward-based improvements in motor control are driven by multiple error-reducing mechanisms. *J. Neurosci.* **40**, 3604-3620 (2020).

68. Sidarta, A., Vahdat, S., Bernardi, N. F. & Ostry, D. J. Somatic and reinforcement-based plasticity in the initial stages of human motor learning. *J. Neurosci.* **36**, 11682-11692 (2016).

69. Krause, M. R., Vieira, P. G., Csorba, B. A., Pilly, P. K. & Pack, C. C. Transcranial alternating current stimulation entrains single-neuron activity in the primate brain. *Proc. Natl Acad. Sci. USA* **116**, 5747-5755 (2019).

70. Shirer, W. R., Ryali, S., Rykhlevskaia, E., Menon, V. & Greicius, M. D. Decoding subject-driven cognitive states with whole-brain connectivity patterns. *Cereb. Cortex* **22**, 158-165 (2012).

71. Morishita, T. & Hummel, F. C. Non-invasive Brain Stimulation (NIBS) in motor recovery after stroke: concepts to increase efficacy. *Curr. Behav. Neurosci. Rep.* **4**, 280-289 (2017).

72. Donnelly, N. A. et al. Oscillatory activity in the medial prefrontal cortex and nucleus accumbens correlates with impulsivity and reward outcome. *PLoS ONE* **9**, 14-17 (2014).

73. Schall, T. A., Wright, W. J. & Dong, Y. Nucleus accumbens fast-spiking interneurons in motivational and addictive behaviors. *Mol. Psychiatry* **26**, 234-246 (2021).

74. Pisansky, M. T. et al. Nucleus accumbens fast-spiking interneurons constrain impulsive action. *Biol. Psychiatry* **86**, 836-847 (2019).

75. Kirby, K. N., Petry, N. M. & Bickel, W. K. Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. *J. Exp. Psychol. Gen.* **128**, 78-87 (1999).

76. Mitchell, J. M., Fields, H. L., D'Esposito, M. & Boettiger, C. A. Impulsive responding in alcoholics. *Alcohol. Clin. Exp. Res.* **29**, 2158-2169 (2005).

77. Catanese, J., Carmichael, J. E. & van der Meer, M. A. A. Low- and high-gamma oscillations deviate in opposite directions from zero-phase synchrony in the limbic corticostriatal loop. *J. Neurophysiol.* **116**, 5-17 (2016).

78. Rothé, M., Quilodran, R., Sallet, J. & Procyk, E. Coordination of high gamma activity in anterior cingulate and lateral prefrontal cortical areas during adaptation. *J. Neurosci.* **31**, 11110-11117 (2011).

79. Del Arco, A., Park, J., Wood, J., Kim, Y. & Moghaddam, B. Adaptive encoding of outcome prediction by prefrontal cortex ensembles supports behavioral flexibility. *J. Neurosci.* **37**, 8363-8373 (2017).

80. Yoshimoto, A., Shibata, Y., Kudara, M., Ikegaya, Y. & Matsumoto, N. Enhancement of motor cortical gamma oscillations and sniffing activity by medial forebrain bundle stimulation precedes locomotion. *eNeuro* **9**, ENEURO.0521-21.2022 (2022).

81. Grover, S., Nguyen, J. A., Viswanathan, V. & Reinhart, R. M. G. High-frequency neuromodulation improves obsessive-compulsive behavior. *Nat. Med.* **27**, 232-238 (2021).

82. Krause, M. R., Vieira, P. G., Thivierge, J. P. & Pack, C. C. Brain stimulation competes with ongoing oscillations for control of spike timing in the primate brain. *PLoS Biol.* **20**, e5 (2022).

83. Courtemanche, R., Fujii, N. & Graybiel, A. M. Synchronous, focally modulated β-band oscillations characterize local field potential activity in the striatum of awake behaving monkeys. *J. Neurosci.* **23**, 11741-11752 (2003).

84. Costa, R. M. et al. Rapid alterations in corticostriatal ensemble coordination during acute dopamine-dependent motor dysfunction. *Neuron* **52**, 359-369 (2006).

85. Engel, A. K. & Fries, P. Beta-band oscillations—signalling the status quo? *Curr. Opin. Neurobiol.* **20**, 156-165 (2010).

86. Uehara, S., Mawase, F. & Celnik, P. Learning similar actions by reinforcement or sensory-prediction errors rely on distinct physiological mechanisms. *Cereb. Cortex* **28**, 3478-3490 (2018).

87. Mathis, M. W., Mathis, A. & Uchida, N. Somatosensory cortex plays an essential role in forelimb motor adaptation in mice. *Neuron* **93**, 1493-1503.e6 (2017).

88. Brücke, C. et al. Scaling of movement is related to pallidal y oscillations in patients with dystonia. *J. Neurosci.* **32**, 1008-1019 (2012).

89. Soderstrom, N. C. & Bjork, R. A. Learning versus performance: an integrative review. *Perspect. Psychol. Sci.* **10**, 176-199 (2015).

90. Spampinato, D. A., Satar, Z. & Rothwell, J. C. Combining reward and M1 transcranial direct current stimulation enhances the retention of newly learnt sensorimotor mappings. *Brain Stimul.* https://doi.org/10.1016/j.brs.2019.05.015 (2019).

91. Shmuelof, L. et al. Overcoming motor 'forgetting' through reinforcement of learned actions. *J. Neurosci.* **32**, 14617-14621a (2012).

92. Dhawale, A. K., Miyamoto, Y. R., Smith, M. A. & Ölveczky, B. P. Adaptive regulation of motor variability. *Curr. Biol.* **29**, 3551-3562. e7 (2019).

93. Carroll, T. J., McNamee, D., Ingram, J. N. & Wolpert, D. M. Rapid visuomotor responses reflect value-based decisions. *J. Neurosci.* **39**, 3906-3920 (2019).

94. De Comite, A., Crevecoeur, F. & Lefèvre, P. Reward-dependent selection of feedback gains impacts rapid motor decisions. *eNeuro* **9**, ENEURO.0439-21.2022 (2022).

95. Codol, O. et al. Sensorimotor feedback loops are selectively sensitive to reward. *eLife* **12**, e81325 (2023).

96. Vassiliadis, P. & Derosiere, G. Selecting and executing actions for rewards. *J. Neurosci.* **40**, 6474-6476 (2020).

97. Krakauer, J. W., Hadjiosif, A. M., Xu, J., Wong, A. L. & Haith, A. M. Motor learning. *Compr. Physiol.* **9**, 613-663 (2019).

98. Quattrocchi, G. et al. Pharmacological dopamine manipulation does not alter reward-based improvements in memory retention during a visuomotor adaptation task. *eNeuro* **5**, ENEURO.0453-17.2018 (2018).

99. Codol, O., Holland, P. J. & Galea, J. M. The relationship between reinforcement and explicit control during visuomotor adaptation. *Sci. Rep.* **8**, 9121 (2018).

100. Truong, C. et al. Error-based and reinforcement learning in basketball free throw shooting. *Sci. Rep.* **13**, 499 (2023).

101. Dayan, E., Hamann, J. M., Averbeck, B. B. & Cohen, L. G. Brain structural substrates of reward dependence during behavioral performance. *J. Neurosci.* **34**, 16433-16441 (2014).

102. Johnson, L. et al. Dose-dependent effects of transcranial alternating current stimulation on spike timing in awake nonhuman primates. *Sci. Adv.* **6**, e36 (2020).

103. Beliaeva, V., Savvateev, I., Zerbi, V. & Polania, R. Toward integrative approaches to study the causal role of neural oscillations via transcranial electrical stimulation. *Nat. Commun.* **12**, 2243 (2021).

104. Averbeck, B. & O'Doherty, J. P. Reinforcement-learning in fronto-striatal circuits. *Neuropsychopharmacology* **47**, 147-162 (2022).

105. Ma, I. et al. Ventral striatal hyperconnectivity during rewarded interference control in adolescents with ADHD. *Cortex* **82**, 225-236 (2016).

106. Wang, Q. et al. Dissociated neural substrates underlying impulsive choice and impulsive action. *NeurolImage* **134**, 540-549 (2016).

107. Mosley, P. E. et al. The structural connectivity of discrete networks underlies impulsivity and gambling in Parkinson's disease. *Brain* **142**, 3917-3935 (2019).

108. Hampton, W. H., Alm, K. H., Venkatraman, V., Nugiel, T. & Olson, I. R. Dissociable frontostriatal white matter connectivity underlies reward and motor impulsivity. *NeuroImage* **150**, 336-343 (2017).

109. Negahbani, E., Kasten, F. H., Herrmann, C. S. & Fröhlich, F. Targeting alpha-band oscillations in a cortical model with
amplitude-modulated high-frequency transcranial electric stimulation. *NeuroImage* **173**, 3–12 (2018).

110. Cohen, J. *Statistical Power Analysis for the Behavioral Sciences* 13 (L. Erlbaum Associates, 1988).

111. Hashemirad, F., Zoghi, M., Fitzgerald, P. B. & Jaberzadeh, S. The effect of anodal transcranial direct current stimulation on motor sequence learning in healthy individuals: a systematic review and meta-analysis. *Brain Cogn.* **102**, 1–12 (2016).

112. Soutschek, A., Kang, P., Ruff, C. C., Hare, T. A. & Tobler, P. N. Brain stimulation over the frontopolar cortex enhances motivation to exert effort for reward. *Biol. Psychiatry* **84**, 38–45 (2018).

113. Wischnewski, M., Zerr, P. & Schutter, D. J. L. G. Effects of theta transcranial alternating current stimulation over the frontal cortex on reversal learning. *Brain Stimul.* **9**, 705–711 (2016).

114. Guerra, A., López-Alonso, V., Cheeran, B. & Suppa, A. Variability in non-invasive brain stimulation studies: reasons and results. *Neurosci. Lett.* **719**, 133330 (2020).

115. Cassarà, A. M. et al. Safety recommendations for temporal interference stimulation in the brain. Preprint at *bioRxiv* https://doi.org/10.1101/2022.12.15.520077 (2022).

116. Oldfield, R. C. The assessment and analysis of handedness: the Edinburgh inventory. *Neuropsychologia* **9**, 97–113 (1971).

117. Kaplan, B. A. et al. Automating scoring of delay discounting for the 21- and 27-item Monetary Choice Questionnaires. *Behav. Anal.* **39**, 293–304 (2016).

118. Mitchell, M. R. & Potenza, M. N. Recent insights into the neurobiology of impulsivity. *Curr. Addict. Rep.* **1**, 309–319 (2014).

119. Clemente, F., D'Alonzo, M., Controzzi, M., Edin, B. B. & Cipriani, C. Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses. *IEEE Trans. Neural Syst. Rehabil. Eng.* **24**, 1314–1322 (2016).

120. Bernardi, N. F., Darainy, M. & Ostry, D. J. Somatosensory contribution to the initial stages of human motor learning. *J. Neurosci.* **35**, 14316–14326 (2015).

121. Hardwick, R. M., Rajan, V. A., Bastian, A. J., Krakauer, J. W. & Celnik, P. A. Motor learning in stroke. *Neurorehabil. Neural Repair* **31**, 178–189 (2017).

122. Raspopovic, S. et al. Restoring natural sensory feedback in real-time bidirectional hand prostheses. *Sci. Transl. Med.* **6**, 222ra19 (2014).

123. Brainard, D. H. The Psychophysics Toolbox. *Spat. Vis.* **10**, 433–436 (1997).

124. Pelli, D. G. The Video Toolbox software for visual psychophysics: transforming numbers into movies. *Spat. Vis.* **10**, 437–442 (1997).

125. Dayan, E., Averbeck, B. B., Richmond, B. J. & Cohen, L. G. Stochastic reinforcement benefits skill acquisition. *Learn. Mem.* **21**, 140–142 (2014).

126. Grossman, N. Modulation without surgical intervention. *Science* **361**, 461–462 (2018).

127. Antal, A. et al. Low intensity transcranial electric stimulation: safety, ethical, legal regulatory and application guidelines. *Clin. Neurophysiol.* **128**, 1774–1809 (2017).

128. Ekhtiari, H. et al. A checklist for assessing the methodological quality of concurrent tES-fMRI studies (ContES checklist): a consensus study and statement. *Nat. Protoc.* **17**, 596–617 (2022).

129. Bossetti, C. A., Birdno, M. J. & Grill, W. M. Analysis of the quasi-static approximation for calculating potentials generated by neural stimulation. *J. Neural Eng.* **5**, 44–53 (2008).

130. Hasgall, P. et al. *IT'IS Database for Thermal and Electromagnetic Parameters of Biological Tissues* v.4.1 (IT'IS Foundation, 2022); https://doi.org/10.13099/VIP21000-04-1

131. Seeck, M. et al. The standardized EEG electrode array of the IFCN. *Clin. Neurophysiol.* **128**, 2070–2077 (2017).

132. R Core Team (2021). *R: A language and environment for statistical computing*. (R Foundation for Statistical Computing, 2021). https://www.R-project.org/

133. Bates, D., Mächler, M., Bolker, B. M. & Walker, S. C. Fitting linear mixed-effects models using lme4. *J. Stat. Softw.* **67**, 1–48 (2015).

134. Ryu, E. Effects of skewness and kurtosis on normal-theory based maximum likelihood test statistic in multilevel structural equation modeling. *Behav. Res. Methods* **43**, 1066–1074 (2011).

135. Nieuwenhuis, R., te Grotenhuis, M. & Pelzer, B. Influence.ME: tools for detecting influential data in mixed effects models. *R J.* **4**, 38–47 (2012).

136. Luke, S. G. Evaluating significance in linear mixed-effects models in *R. Behav. Res. Methods* **49**, 1494–1502 (2017).

137. Searle, S. R., Speed, F. M. & Milliken, G. A. Population marginal means in the linear model: an alternative to least squares means. *Am. Stat.* **34**, 216–221 (1980).

138. Lenth, R. emmeans: Estimated marginal means, aka least-squares means. R package version 1.10.1 https://CRAN.R-project.org/package=emmeans (2023).

139. Ben-Shachar, M., Lüdecke, D. & Makowski, D. effectsize: estimation of effect size indices and standardized parameters. *J. Open Source Softw.* **5**, 2815 (2020).

140. Derosière, G., Billot, M., Ward, E. T. & Perrey, S. Adaptations of motor neural structures' activity to lapses in attention. *Cereb. Cortex* **25**, 66–74 (2015).

141. Okamoto, M. et al. Three-dimensional probabilistic anatomical craniocerebral correlation via the international 10-20 system oriented for transcranial functional brain mapping. *NeuroImage* **21**, 99–111 (2004).

142. Friston, K. J. Functional and effective connectivity in neuroimaging: A synthesis. *Hum. Brain. Mapp.* **2**, 56–78 (1994).

143. Di, X., Zhang, Z. & Biswal, B. B. Understanding psychophysiological interaction and its relations to beta series correlation. *Brain Imaging Behav.* **15**, 958–973 (2021).

144. Bowles, S. et al. Vagus nerve stimulation drives selective circuit modulation through cholinergic reinforcement. *Neuron* **110**, 2867–2868 (2022).

145. Codol, O., Galea, J. M., Jalali, R. & Holland, P. J. Reward-driven enhancements in motor control are robust to TMS manipulation. *Exp. Brain Res.* **238**, 1781–1793 (2020).

146. Draganski, B. et al. Evidence for segregated and integrative connectivity patterns in the human basal ganglia. *J. Neurosci.* **28**, 7143–7152 (2008).

147. Morris, L. S. et al. Fronto-striatal organization: defining functional and microstructural substrates of behavioural flexibility. *Cortex* **74**, 118–133 (2016).

148. Vassiliadis, P. et al. Non-invasive stimulation of the human striatum disrupts reinforcement learning of motor skills—UPHUMMEL—EPFL. *Zenodo* https://doi.org/10.5281/zenodo.10458885 (2024).

## Acknowledgements

P.V. was a PhD student supported by the Fund for Research training in Industry and Agriculture (FRIA/FNRS; FC29690) and grants by the Platform for Education and Talent (Gustave Boël—Sofina Fellowships) and Wallonie-Bruxelles International. J.D. was supported by the Belgian FNRS (TO08219F). G.D. was a post-doctoral fellow supported by the Belgian FNRS (1B134.18). The research was partially funded by grants from the Novartis Research Foundation—FreeNovation (Basel, Switzerland) to M.J.W. and E.N., the Bertarelli Foundation (Catalyst ‘Deep-MCI-T’, Gstaad, Switzerland) to F.C.H., the SNSF Lead Agency (NiBS-iCog 320030L_197899) to F.C.H., the ‘Personalized Health and Related Technologies’ (PHRT#2017-205) of the ETH Domain to F.C.H. and the Defitech Foundation (Morges, Switzerland) to F.C.H. This work has also been supported by the Social and hUman ceNtered XR (SUN)
project, which has received funding from the Horizon Europe Research and Innovation Programme under grant agreement no. 101092612. The views and opinions expressed in this paper are those of the authors only and do not necessarily reflect those of the European Union. Neither the European Union nor the European Commission can be held responsible for them. We acknowledge access to and expertise of the Neuromodulation and the MRI facilities of the Human Neuroscience Platform of the Campus Biotech Geneva and of the MRI Platform of the HVS (Sion). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. We also thank L. Cohen for insightful feedback on a previous version of this manuscript.

## Author contributions

**Conceptualization:** P.V., E.B., T.P., J.D., G.D., M.J.W. and F.C.H.
**Methodology:** P.V., E.B., T.P., F.W., E.N., M.J.W. and F.C.H. Formal
**analysis:** P.V. and E.B. **Investigation:** P.V., E.B., F.W. and T.M. **Resources:**
F.C.H. **Data curation:** P.V., E.B. and F.W. **Writing original draft/
visualization:** P.V. and E.B. **Review and editing:** all authors. **Supervision:**
F.C.H. **Funding:** P.V., T.P., E.N., J.D., G.D., M.J.W. and F.C.H.

## Competing interests

E.N. is co-founder of TI Solutions AG, a company committed to producing hardware and software solutions to support tTIS research. The remaining authors declare no competing interests.

## Additional information

**Supplementary information** The online version contains supplementary material available at https://doi.org/10.1038/s41562-024-01901-z.

**Correspondence and requests for materials** should be addressed to Friedhelm C. Hummel.

**Peer review information** *Nature Human Behaviour* thanks Pablo Celnik, Florian Kasten and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

**Reprints and permissions information** is available at www.nature.com/reprints.

**Publisher's note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

© The Author(s) 2024
## Reporting Summary

Nature Portfolio wishes to improve the reproducibility of the work that we publish. This form provides structure for consistency and transparency in reporting. For further information on Nature Portfolio policies, see our [Editorial Policies](#) and the [Editorial Policy Checklist](#).

## Statistics

For all statistical analyses, confirm that the following items are present in the figure legend, table legend, main text, or Methods section.

n/a | Confirmed

☐ ☑ The exact sample size (n) for each experimental group/condition, given as a discrete number and unit of measurement

☐ ☑ A statement on whether measurements were taken from distinct samples or whether the same sample was measured repeatedly

☐ ☑ The statistical test(s) used AND whether they are one- or two-sided
*Only common tests should be described solely by name; describe more complex techniques in the Methods section.*

☐ ☑ A description of all covariates tested

☐ ☑ A description of any assumptions or corrections, such as tests of normality and adjustment for multiple comparisons

☐ ☑ A full description of the statistical parameters including central tendency (e.g. means) or other basic estimates (e.g. regression coefficient) AND variation (e.g. standard deviation) or associated estimates of uncertainty (e.g. confidence intervals)

☐ ☑ For null hypothesis testing, the test statistic (e.g. *F*, *t*, *r*) with confidence intervals, effect sizes, degrees of freedom and *P* value noted
*Give *P* values as exact values whenever suitable.*

☑ ☐ For Bayesian analysis, information on the choice of priors and Markov chain Monte Carlo settings

☑ ☐ For hierarchical and complex designs, identification of the appropriate level for tests and full reporting of outcomes

☐ ☑ Estimates of effect sizes (e.g. Cohen's *d*, Pearson's *r*), indicating how they were calculated

*Our web collection on statistics for biologists contains articles on many of the points above.*

## Software and code

Policy information about [availability of computer code](#)

**Data collection** We used custom-made (Matlab 2018a) scripts to collect the behavioural data

**Data analysis**

**Behaviour:** Matlab 2018a
MRI data: Statistical Parametric Mapping 12 (SPM12; https://www.fil.ion.ucl.ac.uk/spm/) implemented in MATLAB R2018a (Mathworks, Sherborn, MA, USA), Freesurfer (coded in Bash, 4.4.20(1)-release, and Python (version 3.8.3), https://surfer.nmr.mgh.harvard.edu/); CONN toolbox 2021a (www.nitrc.org/projects/conn, RRID:SCR_009550) running in Matlab 2018a.
**Both (statistics):**
R (R Core Team 2021, Vienna, Austria) for linear mixed models: lme4 package (lmer function), influence.ME package, emmeans package and effectsize packages
Matlab R2018a function robustfit for robust linear regressions

For manuscripts utilizing custom algorithms or software that are central to the research but not yet described in published literature, software must be made available to editors and reviewers. We strongly encourage code deposition in a community repository (e.g. GitHub). See the Nature Portfolio [guidelines for submitting code & software](#) for further information.
# Data

Policy information about availability of data

All manuscripts must include a data availability statement. This statement should provide the following information, where applicable:

- Accession codes, unique identifiers, or web links for publicly available datasets

- A description of any restrictions on data availability

- For clinical datasets or third party data, please ensure that the statement adheres to our policy

**Data availability:**

All data necessary to generate the main results and figures are available in the Zenodo repository (https://doi.org/10.5281/zenodo.10458885). The Brainnetome atlas was used and can be downloaded from: http://atlas.brainnetome.org/. Tissue properties used for modelling of electric fields were based on the IT'ST Tissue Properties Database v4.0 and can be downloaded here: https://itis.swiss/virtual-population/tissue-properties/overview/

# Human research participants

Policy information about studies involving human research participants and Sex and Gender in Research.

**Reporting on sex and gender**

We recruited a total of 29 women and 19 men. Sex was determined based on self-reporting. We did not consider the factor sex in the analyses.

**Population characteristics**

We recruited 24 healthy adults (15 women, 25.3 ± 0.1 years old) for the main experiment and 24 other healthy adults (14 women, 24.2 ± 0.5 years old) for the additional experiment

**Recruitment**

We recruited participants based on platforms for recruitment of subjects and based on previous lists of participants who had performed previous experiments at EPFL. We also distributed flyers at Campus Biotech in Geneva and other public places.

Selection bias: Healthy young subjects were recruited to a significant part within the university community through verbal or written advertisements. This entails that a disproportionately high number of subjects with a high level of education were recruited. To minimize the impact of this selection bias, the study employed a randomized within-subject design and we distributed the advertisement also at other public places.

**Ethics oversight**

All participants gave their written informed consent in accordance with the Declaration of Helsinki and the approval of the Cantonal Ethics Committee Vaud, Switzerland (project number 2020-00127).

Note that full information on the approval of the study protocol must also be provided in the manuscript.

# Field-specific reporting

Please select the one below that is the best fit for your research. If you are not sure, read the appropriate sections before making your selection.

<figure><img src="image_19.png" /></figure>

Life sciences

<figure><img src="image_20.png" /></figure>

Behavioural & social sciences

<figure><img src="image_21.png" /></figure>

Ecological, evolutionary & environmental sciences

For a reference copy of the document with all sections, see [nature.com/documents/nr-reporting-summary-flat.pdf](https://nature.com/documents/nr-reporting-summary-flat.pdf)

# Life sciences study design

All studies must disclose on these points even when the disclosure is negative.

**Sample size**

A priori sample size calculation was performed based on effect sizes obtained in our previous behavioral study on reinforcement motor learning (Vassiliadis et al., 2021, iScience) employing a similar paradigm, with a level of significance p < 0.05 (two-sided) and power (1-β) of 0.90, which resulted in a total sample size of 23 subjects. In order to balance the order of conditions for each subject, we decided to recruit 24 subjects.

**Data exclusions**

We removed trials from the behavioural analysis in which participants did not react within 1 s after the appearance of the cursor and target, considering that these extremely long preparation times may reflect significant fluctuations in attention. Moreover, to mitigate the impact of isolated influential data points on the outcome of the linear mixed model analyses, we used tools of the influence.ME package (R) to detect and remove influential cases based on Cook's distance: distance > 4 * mean distance. This never removed more than one participant per analysis.

**Replication**

The main behavioral effects of reinforcement on motor learning (observed in the tTISSham and tTIS20Hz conditions) were replicated in an additional, independent cohort of 24 participants (See Figure S1b, c, e).

**Randomization**

All participants performed all experimental conditions. The order of the 6 experimental conditions was pseudo-randomised across participants: the 6 blocks were divided into 3 pairs of blocks with the same tTIS condition and each pair was then composed of one ReinfON and one ReinfOFF block. Within this structure, the order of the tTISTYPE and ReinfTYPE conditions were balanced among the 24 participants.
# Reporting for specific materials, systems and methods

We require information from authors about some types of materials, experimental systems and methods used in many studies. Here, indicate whether each material, system or method listed is relevant to your study. If you are not sure if a list item applies to your research, read the appropriate section before selecting a response.

## Materials & experimental systems

<table><thead><tr><th>n/a</th><th>Involved in the study</th></tr></thead><tbody><tr><td>☑</td><td>Antibodies</td></tr><tr><td>☑</td><td>Eukaryotic cell lines</td></tr><tr><td>☑</td><td>Palaeontology and archaeology</td></tr><tr><td>☑</td><td>Animals and other organisms</td></tr><tr><td>☑</td><td>Clinical data</td></tr><tr><td>☑</td><td>Dual use research of concern</td></tr></tbody></table>

## Methods

<table><thead><tr><th>n/a</th><th>Involved in the study</th></tr></thead><tbody><tr><td>☑</td><td>ChIP-seq</td></tr><tr><td>☑</td><td>Flow cytometry</td></tr><tr><td>☐</td><td>MRI-based neuroimaging</td></tr></tbody></table>

## Magnetic resonance imaging

### Experimental design

**Design type** Task-based fMRI, block design

**Design specifications** Each subject performed 6 series of 36 trials in the MRI. One trial lasted 10 seconds. During Training, trials were grouped by bins of 4 and separated by resting periods of 25s.

**Behavioral performance measures** We measured the force applied on the MRI-compatible force sensor allowing us to compute the Error relative to the moving target. As explained above, we verified that participants reacted within 1s of the beginning of the trial to make sure that our data was not corrupted by significant fluctuations in attention. Moreover, eye-tracking was also checked during the experiment to verify that participants did not fall asleep in the MRI.

### Acquisition

**Imaging type(s)** Functional and structural

**Field strength** 3T

**Sequence & imaging parameters** Structural and functional images were acquired using a 3T MAGNETOM PRISMA scanner (Siemens, Erlangen, Germany). T1-weighted images were acquired via the 3D MPRAGE sequence with the following parameters: TR = 2.3 s; TE = 2.96 ms; flip angle = 9°; slices = 192; voxel size = 1 × 1 × 1 mm, FOV = 256 mm; matrix size = 192 × 240 × 256; orientation = sagittal, phase encoding dir = A >> P. Anatomical T2 images were also acquired with the following parameters: TR = 3 s; TE = 409 ms; flip angle = 120°; slices = 208; voxel size = 0.8 × 0.8 × 0.8 mm, FOV = 320 mm; matrix size = 208 × 320 × 320; orientation = sagittal, phase encoding dir = A >> P. Finally, functional images were recorded using Echo-Planar Imaging (EPI) sequences with the following parameters: TR = 1.25 s; TE = 32 ms; flip angle = 58°; slices = 75; voxel size = 2 × 2 × 2 mm; FOV = 112 mm; matrix size = 192 × 240 × 256; orientation = transversal, phase encoding dir = A >> P.

**Area of acquisition** Whole-brain

**Diffusion MRI** ☐ Used ☑ Not used

### Preprocessing

**Preprocessing software** Structural: the Freesurfer recon-all function was run based on the structural T1w and T2w images (https://surfer.nmr.mgh.harvard.edu/). The BNA parcellation was derived on the individual subject space and the selected ROIs were then co-registered to the functional images and normalised to the MNI space.

Functional: we analyzed functional imaging data using Statistical Parametric Mapping 12 (SPM12; The Wellcome Department of Cognitive Neurology, London, UK) implemented in MATLAB R2018a (Mathworks, Sherborn, MA). All functional images underwent a common preprocessing including the following steps: slice time correction, spatial realignment to the first image, normalization to the standard MNI space and smoothing with a 6 mm full-width half-maximal Gaussian kernel. T1 anatomical images were then co-registered to the mean functional image and segmented. This allowed to obtain bias-corrected gray and white matter images, by normalizing the functional images via the forward deformation field.

**Normalization** SPM normalization to MNI space, linear and non-linear transformation based on deformation fields obtained from segmentation in SPM.
Normalization template MNI152 T1, 1mm

Noise and artifact removal Visual check for co-registration and normalisation. Framewise displacement (FD) was computed and subjects showing more than 40% of time points of FD larger than 0.5mm. No subjects were excluded based on this criterion.

Volume censoring We did not apply volume censoring

## Statistical modeling & inference

Model type and settings A general linear model was implemented at the single-subject level in order to estimate signal amplitude. Eight regressors were included in the model: 6 head motion parameters (displacement and rotation) and normalised time series within the white matter and the corticospinal fluid.

Effect(s) tested

Whole brain, 1 sample t-test on the first-level contrasts:
- Sham, ReinfON
- Sham, ReinfON - ReinfOFF

Correlation with behavior - whole brain, multiple regression with individual behavioral results:
- tTIS80Hz vs. tTIS20Hz, ReinfON
- tTIS80Hz vs. tTISSham, ReinfON
- tTIS20Hz vs. tTISSham, ReinfON

Specify type of analysis: ☐ Whole brain ☐ ROI-based ☒ Both

Number of regions below are based on the Brainnectome atlas (even numbers: right side, odd numbers, left side).

BOLD analysis:
Putamen (225, 226, 229, 230), caudate (219, 220, 227, 228) and NAc (223 and 224)

Anatomical location(s)

Effective connectivity:
Motor network: dlPu (229, 230), dCa (227,228), M1 (57 and 58), SMA (9 and 10)
Reward network: NAc (223 and 224), vmPu (225 and 226), vmPFC (41, 45, 47, 49, 187, 42, 46, 48, 50, 188), ACC (77, 179, 183, 178, 180, 184)

Control: Language network as defined by Shirer et al., 2012

Statistic type for inference
(See Eklund et al. 2016)

Voxel-wise uncorrected p=0.001 and cluster FDR corrected p=0.05
For activation map during simple condition (Sham, ReinfON): voxel FWE corrected p=0.05 and cluster FDR corrected p=0.05

Correction

Voxel-wise uncorrected p=0.001 and cluster FDR corrected p=0.05
For activation map during simple condition (Sham, ReinfON): voxel FWE corrected p=0.05 and cluster FDR corrected p=0.05

## Models & analysis

n/a | Involved in the study

☐ ☒ Functional and/or effective connectivity

☒ ☐ Graph analysis

☒ ☐ Multivariate modeling or predictive analysis

Functional and/or effective connectivity

A generalized Psycho-Physiological Interactions (gPPI) connectivity method (from the CONN toolbox) was used to evaluate effective connectivity.

All the following conditions were included in the model:

- Sham, ReinfON

- Sham, ReinfOFF

- tTIS20Hz, ReinfON

- tTIS20Hz, ReinfOFF

- tTIS80Hz, ReinfON

- tTIS80Hz, ReinfOFF