Linking reward-learning and affect in health and depression

Relatively little is known about the mechanism underlying major depressive disorder (MDD), necessitating the exploration of novel investigative frameworks. In recent years reward processing has emerged as a promising theoretical framework for investigating depressive symptoms. In parallel, computational modelling has emerged as a promising analysis framework for leveraging the richness of (all manner of) psychiatric data. In this thesis I will use a reward processing framework and computational modelling to investigate a number of areas that are relevant to MDD.
My findings are from 2 studies. In the first study, I apply reinforcement learning models to behavioural and neuroimaging (functional magnetic resonance imaging; fMRI) data to investigate the effect of Pramipexole, a promising antidepressant, on behavioural and neural reward learning. The results of this study are reported in chapters 2 and 3. In chapter 2, I report on the behavioural findings from this study: Pramipexole specifically increases choice accuracy in the reward condition of a probabilistic instrumental learning task, with no effect in the loss condition. Behavioural modelling (alone) does not clearly arbitrate between potential underlying mechanisms. In Chapter 3, I report on the neuroimaging findings from this study: Pramipexole decreases the BOLD response to reward prediction errors in the ventromedial prefrontal cortex. Combined with the behavioural modelling, this finding indicates that Pramipexole enhances choice accuracy by reducing the decay of estimated values during reward learning.
In the second study, I investigate the mechanisms underlying affective instability, an emerging area of interest in depression research. I record participants’ affective reports during an online reward learning task, and ‘in real life’ using experience sampling method (ESM). The results of this study are reported in chapters 4 and 5. In chapter 4 I separately characterise participants’ task based and real-world affective profiles by applying a Bayesian filter to each dataset to calculate the parameter values that underlie participants’ mean affect, the extent to which their affect fluctuates around this mean and the extent to which this mean changes over time. I then compare parameters from the two datasets and find that participants’ affective profiles ‘within-task’ reflect their affective profiles ‘in real life’. In Chapter 5, I explore a link between the (broad) reinforcement learning approach used in chapters 2/3 and the topic of chapter 4: affective instability. Specifically, I test a previously proposed model that links reinforcement learning and affective instability. I find that the model is able to replicate participant reported (within-task) affect without fitting to participant affect.
In sum, this thesis reports on a number of analyses that utilize computational modelling and a reward processing framework to link different types of data. If applied to clinical datasets, this approach may help to unpick the mechanisms underlying depression and its treatment.

Keywords: computational modelling
neuroscience
psychiatry
pramipexole
depression
Advisor: Browning, M
Harmer, C
Geddes, J