# Dopamine and learning: Does the midbrain teach with surprise or worth?

Robin Piquet, Yacine Tensaouti, Shauna L. Parkes

<figure><img src="image_1.png" /></figure>

## To cite this version:

Robin Piquet, Yacine Tensaouti, Shauna L. Parkes. Dopamine and learning: Does the midbrain teach with surprise or worth?. Current Biology, 2025, 35 (16), pp.R806-R808. 10.1016/j.cub.2025.06.075. hal-05487165

HAL Id: hal-05487165

https://hal.science/hal-05487165v1

Submitted on 31 Jan 2026

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
1

Re-evaluating dopamine: Does the midbrain teach with surprise or worth?

2

Robin Piquet¹,*, Yacine Tensaouti²,*, Shauna L. Parkes²,#

3

¹Department of Experimental Psychology, University of Oxford, Oxford, UK

4

²Univ. Bordeaux, CNRS, INCIA, UMR 5287, F-33000 Bordeaux, France

5

*These authors contributed equally

6

#Corresponding author: shauna.parkes@u-bordeaux.fr

7

8

Summary

9

10 it drives learning by generating a reward prediction error - indicating that something
11 unexpected has occurred - and not simply by changing the perceived value of the
12 reward.

13

14 You visit your favourite *boulangerie* each morning. Every time you order a
15 croissant, it is handed to you in a paper bag with a red label. The red label thus
16 becomes associated with the tasty croissant inside. One day, in addition to the red
17 label, there is a new yellow label on the bag. When you look inside, as expected,
18 there is your croissant. You thus decide that this yellow label adds no information and
19 your brain comes to ignore it. This is an example of the well-established blocking
20 phenomenon¹, which illustrates that learning about new cues fails when outcomes
21 are already fully predicted (Figure 1). Now imagine that when you're handed the bag
22 with the red label and the new yellow label, surprise - there are two croissants inside!
23 This mismatch between the actual outcome (two croissants) and what we expected
24 (one croissant) is referred to as a prediction error. Formal theories of associative
25 learning stipulate that this error will restore, or *unblock*, learning about the yellow
26 label¹⁻⁵. In the future, when you see the yellow label you will likely expect to receive
27 two tasty croissants.

28 The behavioural phenomena of blocking and unblocking have been crucial in
29 establishing that prediction error (i.e., “surprise”) is a necessary and sufficient
30 circumstance for learning to occur⁶. There is also clear evidence that dopamine
31 neurons respond in ways consistent with prediction error signals⁷. For instance,
32 midbrain dopamine neurons increase their firing to a surprising reward but not to a
33 predicted reward⁸,⁹ (Figure 1). Artificial stimulation of these neurons is also able to
restore or unblock learning in situations where there is no prediction error¹⁰⁻¹². What then does midbrain dopamine activity represent? In this issue of *Current Biology*, Usypchuk et al.¹³ tackle this question.

<figure><img src="image_3.png" /><figcaption>**Figure 1. Unblocking the expected – how surprise drives learning. (Left)** Your local *boulangerie* always serves their croissants in a paper bag with a red label. Over time, you come to associate that red label with your tasty croissant. (Middle) One day, there's a red label and a yellow label on the bag, but your usual croissant is inside. You therefore don't learn anything about the yellow label as the outcome (one croissant) was already fully predicted by the red label (*Blocking*, top panel). Now, imagine instead that when you receive the bag with both the red and the gold labels, to your surprise, you discover two tasty croissants! This unexpected outcome allows you to learn about the yellow label (*Unblocking*, bottom panel). Such errors in prediction are reflected in midbrain dopamine neurons, which increase their firing to unexpected outcomes (e.g., two croissants) but not to expected outcomes (one croissant). (Right) New findings by Usypchuk et al. examine how dopamine firing drives learning. They show that midbrain dopamine transients signal surprise (the mismatch between prediction and reality), rather than simply a change in reward value. *Image created using icons from the open resource Flatiron.*</figcaption></figure>

One possibility is that dopamine transients reflect the reward prediction error itself, signalling when outcomes differ from expectations. Alternatively, they may represent a scalar quantity of how good or desirable an outcome is, serving as a running estimate of reward value. That is, do we learn about the yellow tag because dopamine neurons value two croissants more than one, or because they weren't expecting it? These two accounts have been notoriously difficult to disentangle¹⁴, as they both rely on a prediction error; an unexpected increase (or decrease) in reward value necessarily gives rise to an error in prediction. Usypchuk et al.¹³ manage to
62 dissociate these competing hypotheses and test whether midbrain dopamine 63 transients support learning by signalling a reward prediction error or a change in 64 value.

65 To do this, the authors return to the classic associative learning paradigms 66 described above, blocking and unblocking. In their version, rats first learn that cue A 67 predicts one food pellet reward. Then, during the blocking phase, a compound cue 68 (AX) is also followed by this same reward. Because A already predicts the food 69 pellet, little to no learning occurs about X (i.e., blocking to X). However, if something 70 unexpected happens, such as an increase in reward (two pellets instead of one, for 71 example), learning about X is “unblocked”.

72 In their first experiment, the authors use optogenetic manipulation to replicate 73 the previously mentioned finding that learning about an uninformative environmental 74 event or cue (X) can be restored (“unblocked”) if midbrain dopamine neurons are 75 stimulated at the time of reward during blocking; even though, as expected, only one 76 food pellet is delivered<sup>10-12</sup>. This is an important finding, but one that is consistent 77 with the predictions of the reward prediction error *and* scalar value models: dopamine 78 stimulation could either directly generate this error in prediction (reward prediction 79 error), or indirectly by increasing the perceived value of the outcome relative to prior 80 experience (scalar value). Both mechanisms would result in learning about X.

81 Usypchuk et al.<sup>13</sup> then adopt computational modelling to reveal an important, 82 and untested, dissociation in the predictions of the reward prediction error and scalar 83 value accounts. Although both accounts predict that dopamine stimulation will 84 unblock learning under standard conditions (as described above), they sharply 85 diverge in how they handle *expectation*. The reward prediction error model proposes 86 that dopamine stimulation produces an artificial surprise or error signal, which can 87 drive theoretically infinite learning regardless of expectation. By contrast, the scalar 88 value model predicts that dopamine increases the perceived value of the reward, 89 raising the asymptote of learning, but only when the added value is unexpected.

90 In a second experiment, the authors validate this core prediction of the scalar 91 value account: by increasing the reward magnitude at the time of blocking (i.e., by 92 introducing an unexpected increase in value) they successfully unblock learning 93 about cue X. Crucially, when the same reward increase is introduced earlier in 94 training, such that it becomes expected by the time of blocking, learning remains 95 blocked. This confirms a key principle of the scalar value hypothesis: only
unexpected changes in reward value trigger learning. This logic sets a critical trap for the value model and a decisive test for the reward prediction error account. If dopamine transients merely encode reward value, then stimulation of dopamine neurons that becomes predictable (just like the reward increase) should lose its capacity to unblock learning. But if dopamine encodes prediction errors, then even fully expected dopamine stimulation at the time of blocking should still act as a teaching signal, resulting in unblocking of X.

Using a clever twist on the classic blocking paradigm, the authors show that animals keep learning even when dopamine stimulation is expected, strongly supporting the reward prediction error account. They introduce optogenetic stimulation of dopamine neurons halfway through the initial learning with cue A, making it fully expected at the time of blocking, and then deliver the same stimulation during the blocking phase with compound cue AX. Despite its predictability, the stimulation still unblocked learning about X. The authors conclude that dopamine transients do not merely signal scalar value, but instead act as ongoing prediction error signals that drive learning, even in the absence of surprise in the conventional sense. In addition, they show that high frequency (50Hz), but not low frequency (4Hz - typical baseline tonic dopamine activity¹⁵), stimulation unblocks learning about X, even when it's expected. This final result supports a key prediction of their reward prediction error model: that high-frequency dopamine stimulation can override the influence of natural, low-frequency activity and act as the primary driver of learning.

While these results add strong support to the reward prediction error hypothesis, it is nonetheless increasingly recognized that dopamine neurons, including those studied here in the ventral tegmental area (VTA), are functionally heterogeneous¹⁶⁻²⁰. Distinct subpopulations of VTA dopamine neurons vary in their molecular profiles and anatomical projections, and may play distinct roles in signalling reward value, salience, and prediction error. Studies that target these subpopulations, based on genetic markers and connectivity (inputs and outputs), will be essential to understand the specific computational functions linked to each subpopulation of dopamine neurons and determine whether they support distinct motivational or learning processes. For now, the results from Usypchuk et al.¹³, indicate that the reason why we learn about the yellow label on the paper bag is not because we value two croissants more than one, but rather because the bonus croissant was just not what we expected.
132 1. Kamin, L.J (1968). Attention-like processes in classical conditioning. In M.R. Jones (Ed.), Miami symposium on the prediction of behavior: Aversive stimuli (University of Miami Press), pp. 9–32.

135 2. Rescorla, R.A., and Wagner, A.R. (1972). A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II: Current Research and Theory, A. H. Black and W. F. Prokasy, eds. (Appleton-Century-Crofts), pp. 64–99.

139 3. Dickinson, A., Hall, G., and Mackintosh, N.J. (1976). Surprise and the attenuation of blocking. J. Exp. Psychol. Anim. Behav. Process. 2, 313–322.
140 https://doi.org/10.1037/0097-7403.2.4.313.

142 4. Mackintosh, N.J., Bygrave, D.J., and Picton, B.M. (1977). Locus of the effect of a surprising reinforcer in the attenuation of blocking. Q. J. Exp. Psychol. 29, 327–336. https://doi.org/10.1080/14640747708400608.

145 5. Mackintosh, N.J. (1975). A theory of attention: variations in the associability of stimuli with reinforcement. Psychol. Rev. 82, 276–298.

147 6. Rescorla, R.A. (1988). Pavlovian conditioning: It's not what you think it is. Am. Psychol. 43, 151–160. https://doi.org/10.1037/0003-066X.43.3.151.

149 7. Schultz, W., and Dickinson, A. (2000). Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23, 473–500. https://doi.org/10.1146/annurev.neuro.23.1.473.

151 8. Waelti, P., Dickinson, A., and Schultz, W. (2001). Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48.
152 https://doi.org/10.1038/35083500.

154 9. Mirenowicz, J., and Schultz, W. (1994). Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024–1027.
155 https://doi.org/10.1152/jn.1994.72.2.1024.

157 10. Steinberg, E.E., Keiflin, R., Boivin, J.R., Witten, I.B., Deisseroth, K., and Janak, P.H. (2013). A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 16, 966–973. https://doi.org/10.1038/nn.3413.

160 11. Sharpe, M.J., Chang, C.Y., Liu, M.A., Batchelor, H.M., Mueller, L.E., Jones, J.L., Niv, Y., and Schoenbaum, G. (2017). Dopamine transients are sufficient and necessary for acquisition of model-based associations. Nat. Neurosci. 20, 735–742. https://doi.org/10.1038/nn.4538.

164 12. Keiflin, R., Pribut, H.J., Shah, N.B., and Janak, P.H. (2019). Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions. Curr. Biol. CB 29, 93-103.e3. https://doi.org/10.1016/j.cub.2018.11.050.
167

13. Usypchuk, A., Maes, E., Lozzi, M., Avramidis, D., Geoffrey Schoenbaum, Esber, G., Gardner, M., and Iordanova, M. (2025). Disentangling Prediction Error and Value: A Formal Test of Dopamine's Role in Reinforcement Learning.

168

169

170

14. Sharpe, M.J., Batchelor, H.M., Mueller, L.E., Yun Chang, C., Maes, E.J.P., Niv, Y., and Schoenbaum, G. (2020). Dopamine transients do not act as model-free prediction errors during associative learning. Nat. Commun. 11, 106. https://doi.org/10.1038/s41467-019-13953-1.

171

172

173

174

15. Marinelli, M., and McCutcheon, J.E. (2014). Heterogeneity of dopamine neuron activity across traits and states. Neuroscience 282, 176-197. https://doi.org/10.1016/j.neuroscience.2014.07.034.

175

176

177

16. Lerner, T.N., Holloway, A.L., and Seiler, J.L. (2021). Dopamine, Updated: Reward Prediction Error and Beyond. Curr. Opin. Neurobiol. 67, 123-130. https://doi.org/10.1016/j.conb.2020.10.012.

178

179

180

17. Matsumoto, M., and Hikosaka, O. (2009). Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459, 837-841. https://doi.org/10.1038/nature08028.

181

182

183

18. Lammel, S., Ion, D.I., Roeper, J., and Malenka, R.C. (2011). Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 70, 855-862. https://doi.org/10.1016/j.neuron.2011.03.025.

184

185

186

19. Engelhard, B., Finkelstein, J., Cox, J., Fleming, W., Jang, H.J., Ornelas, S., Koay, S.A., Thiberge, S.Y., Daw, N.D., Tank, D.W., et al. (2019). Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons. Nature 570, 509-513. https://doi.org/10.1038/s41586-019-1261-9.

187

188

189

190

20. Millard, S.J., Hoang, I.B., Sherwood, S., Taira, M., Reyes, V., Greer, Z., O'Connor, S.L., Wassum, K.M., James, M.H., Barker, D.J., et al. (2024). Cognitive representations of intracranial self-stimulation of midbrain dopamine neurons depend on stimulation frequency. Nat. Neurosci. 27, 1253-1259. https://doi.org/10.1038/s41593-024-01643-1.

191

192

193

194

195

196