# How does apathy impact exploration-exploitation decision-making in older patients with neurocognitive disorders?

Lyne Daumas, Raphaël Zory, Isabel Junquera-Badilla, Marion Ferrandez, Eric Ettore, Philippe Robert, Guillaume Sacco, Valeria Manera, Stephen Ramanoël

<figure><img src="image_1.png" /></figure>

## To cite this version:

Lyne Daumas, Raphaël Zory, Isabel Junquera-Badilla, Marion Ferrandez, Eric Ettore, et al., How does apathy impact exploration-exploitation decision-making in older patients with neurocognitive disorders?. npj Aging, 2023, 9 (1), 10.1038/s41514-023-00121-5, http://dx.doi.org/10.1038/s41514-023-00121-5

HAL Id: hal-04692435

https://hal.science/hal-04692435v1

Submitted on 9 Sep 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

# How does apathy impact exploration-exploitation decision-making in older patients with neurocognitive disorders?

Lyne Daumas<fnref n="1" />,²✉, Raphaël Zory<fnref n="1" />,³, Isabel Junquera-Badilla ID⁴, Marion Ferrandez²,⁵, Eric Ettore²,⁵,⁶, Philippe Robert²,⁶, Guillaume Sacco ID²,⁵,⁶,⁷, Valeria Manera²,⁶,⁹ and Stephen Ramanoël ID<fnref n="1" />,⁸,⁹

Apathy is a pervasive clinical syndrome in neurocognitive disorders, characterized by a quantitative reduction in goal-directed behaviors. The brain structures involved in the physiopathology of apathy have also been connected to the brain structures involved in probabilistic reward learning in the exploration-exploitation dilemma. This dilemma in question involves the challenge of selecting between a familiar option with a more predictable outcome, and another option whose outcome is uncertain and may yield potentially greater rewards compared to the known option. The aim of this study was to combine experimental procedures and computational modeling to examine whether, in older adults with mild neurocognitive disorders, apathy affects performance in the exploration-exploitation dilemma. Through using a four-armed bandit reinforcement-learning task, we showed that apathetic older adults explored more and performed worse than non-apathetic subjects. Moreover, the mental flexibility assessed by the Trail-making test-B was negatively associated with the percentage of exploration. These results suggest that apathy is characterized by an increased explorative behavior and inefficient decision-making, possibly due to weak mental flexibility to switch toward the exploitation of the more rewarding options. Apathetic participants also took longer to make a choice and failed more often to respond in the allotted time, which could reflect the difficulties in action initiation and selection. In conclusion, the present results suggest that apathy in participants with neurocognitive disorders is associated with specific disturbances in the exploration-exploitation trade-off and sheds light on the disturbances in reward processing in patients with apathy.

npj Aging (2023)9:25; https://doi.org/10.1038/s41514-023-00121-5

## INTRODUCTION

Neurocognitive disorders are characterized by a decline in one or more cognitive domains that go beyond what is expected from normal aging. According to the DSM-5¹, Mild neurocognitive disorders are characterized by a level of cognitive decline that requires compensatory strategies and accommodations to help maintain independence and perform activities of daily living, whereas Major neurocognitive disorders are characterized by cognitive impairment which affects autonomy in activities of daily living. In both Mild and Major neurocognitive disorders, neuropsychiatric symptoms are highly prevalent², and among them, apathy is a multidimensional syndrome³ particularly pervasive, associated with numerous adverse outcomes⁴,⁵. Apathy is defined as a quantitative reduction in goal-directed behaviors and can manifest across several domains, including cognition, behavior, and emotion⁶. Clinically, apathy is manifested by difficulties in starting or maintaining the usual or desired activities, and it may be therefore complicated to identify it as some other symptoms can manifest similarly, such as fatigue⁷. From an etiological point of view, apathy may arise from dysfunction occurring at any step of the process of achieving goal-directed behaviors, including at the level of initiation, execution, and control of voluntary actions⁸.

Goal-directed behavior (i.e., voluntary actions toward outcome) relies on several interrelated processes, starting with an internal valuation step mainly supported by the ventral striatum (VS), the orbitofrontal cortex (OFC), the ventromedial prefrontal cortex (vmPFC) and the anterior cingulate cortex (ACC). The ACC

attributes to stimuli of the environment, subjective values based on their hedonic or aversive potential, as well as the potential cost of action⁹⁻¹¹. Goal-directed behavior relies on the capacity to represent and anticipate the action outcomes¹², based on the previously learned associations between the actions and their consequences, which is mediated by the dopaminergic system¹³. The outcome value and cost of action are in this way accounted in the action selection phase, and the subsequent generation of behavior toward or away from stimuli. The decision-making processes of whether to engage or not in behavior are thus underpinned by weighting the prospective value of rewards (i.e., objects or events associated with positive outcomes that individuals pursue) against the amount of effort required to reach them¹⁴.

In apathetic patients, behavioral and neuroimaging studies suggest that these mechanisms of reward-based decision-making are disrupted, explaining at least in part their reduced willingness to engage in behavior¹⁵⁻¹⁷. More specifically, several lines of evidence have related apathy to lesions or dysfunctions in the brain circuit linking the frontal territories to the basal ganglia, notably in the vmPFC, the OFC, the ACC, and the VS¹⁸⁻²⁰. Additionally, apathy has been related to dysfunction in the dopaminergic system²¹; and in line with this idea, some studies found improvement of apathy and reward sensitivity through dopaminergic medication²². For instance, Adam et al., described in their study, in apathetic patients exhibiting focal lesions in the basal ganglia connecting to the OFC and vmPFC, a reward

<footnote n="1">¹Université Côte d'Azur, LAMHESS, Nice, France. ²Université Côte d'Azur, CoBTeK, Nice, France. ³Institut Universitaire de France, Paris, France. ⁴Université Côte d'Azur, NeuroMod Institute, Nice, France. ⁵Université Côte d'Azur, Centre Hospitalier Universitaire de Nice, service Clinique Gériatrique de Soins Ambulatoires, Centre Mémoire de Ressources et de Recherche, Nice, France. ⁶Association Innovation Alzheimer, Nice, France. ⁷Univ Angers, Université de Nantes, LPPL, SFR CONFLUENCES, 49000 Angers, France. ⁸Sorbonne Université, INSERM, CNRS, Institut de la Vision, 17 rue Moreau, 75012 Paris, France. ⁹These authors contributed equally: Valeria Manera, Stephen Ramanoël.</footnote>

<footnote>✉email: daumas.l@chu-nice.fr</footnote>
insensitivity; and then its improvement following dopamine receptor agonist take²³. Regarding experimental studies using choice tasks manipulating the level of effort and reward, it has been shown that apathetic individuals exhibited greater effort discounting (i.e., devaluation of subjective reward value by the amount of effort), so apathetic individuals needed larger rewards to engage in action¹⁷,²⁴.

Regarding the exploration-exploitation decision-making, this dilemma is daily met by people and consists of choosing between a familiar option with a more predictable outcome and a less certain but potentially more rewarding option²⁵⁻²⁷. In other words, in an uncertain and changing environment where the values of potential options are unknown and can change over time, people must decide whether to utilize ("exploit") the knowledge that has been accumulated or to explore new options. The neural evidence showed that exploration-exploitation is supported by the vmPFC and the rostral PFC, as well as the OFC, the ACC, and the VS²⁸⁻³¹, with more particularly the involvement of the vmPFC in exploitation, and the rostral PFC in exploration³². Behaviorally, excessive exploitation restricts the possibility of obtaining new information, whereas too much exploration is associated with an inefficient use of acquired knowledge³³. An optimal balance between exploration and exploitation is therefore required to maximize benefits, and it is also necessary to correctly learn and utilize the gained information. To this end, there is strong evidence that dopaminergic neurons play a central role in this type of learning by encoding the reward predictor errors (i.e., the differences between the experienced and expected outcome), which acts as a learning signal to update value predictions and then guide the action selection and future behaviors³⁴⁻³⁶.

In older adults, studies showed different exploration-exploitation patterns compared with younger adults³⁷,³⁸, and it has been suggested that it could be due to age-related modifications in the dopaminergic system, as well as to the decline in cognitive functions, notably in executive functions³⁹,⁴⁰. Indeed, executive functions are involved in decision-making, Their dysfunction is, by the way, also closely linked to apathy, and refers more particularly to impairment in the capacity to elaborate a plan of action (cognitive apathy)⁸,⁴¹. In Parkinson's disease and schizophrenia, two pathologies with a high prevalence of apathy, some studies reported changes in solving the exploration-exploitation dilemma⁴²,⁴³. However, while it has been suggested that emotional apathy in dementia is underpinned by altered reward learning⁴⁴, very few studies investigated direct links between apathy and exploration-exploitation decision-making⁴⁵, notably in older adults with neurocognitive disorders. Yet, several pieces of evidence (e.g., common brain areas such as vmPFC, ACC, VS; dopaminergic system involvement, executive functions) raise the question of whether apathy in neurocognitive disorders affects exploration-exploitation decision-making. This investigation could be particularly relevant because this type of decision-making is performed daily by individuals and relies on socio-emotional and executive processing that is analogous to that of real-life goal-directed behavior (e.g., affective value attribution to environmental stimuli, integration of feedback, behavioral adjustments). This type of task could therefore be relevant to capture the complex nature of the multidimensional syndrome of apathy which includes emotional, cognitive, and behavioral components. Furthermore, in some cases, the standard tests of executive functions fail and/or are dissociated from real-life dysexecutive problems, which can be explained as the socio-emotional aspects are not considered. For instance, it has been observed that patients with OFC damage can be not cognitively impaired but exhibit severe daily life decision-making problems, with often behavior disorders. This functional decision-making task could provide new insight into apathy-related deficits which can have a direct impact on daily life, as well as regarding the underlying mechanisms of apathy, notably in neurocognitive disorders where

apathy is highly prevalent and associated with faster clinical decline. Thus, the aim of this study was to examine the impact of apathy on exploration-exploitation decision-making in the population of older adults exhibiting a mild neurocognitive disorder, by comparing apathetic older adults with mild neurocognitive disorders vs. non-apathetic older adults with mild neurocognitive disorders.

## RESULTS

Forty-six subjects from 60 to 89 years old were included (mean = 75.91 ± 6.54). According to the apathy diagnostic criteria, 19 were apathetic (age = 78.05 ± 5.05) and 27 were non-apathetic (age = 74.41 ± 7.12). Clinical and demographic data are presented in Table 1.

## Exploration and performance on the bandit task

The percentage of explorative choices was higher in apathetic (49.79 ± 16.16, 95%CI [42.00, 57.58]) compared to non-apathetic participants (37.56 ± 14.7, 95%CI [31.74, 43.37]), however, their performance, as measured by the total points earned on the bandit task, was lower (80.44 ± 8.29, 95%CI [76.44, 84.44]) vs. 89.60 ± 5.18, 95%CI [87.56, 91.65], Fig. 1). The MANCOVA revealed that these differences between groups were statistically significant after controlling for age, MFI, and TMT-B scores (F(2,39) = 5.62, p = 0.007, η² = 0.22). No effect of gender was found (F(2,39) = 0.75, p = 0.47, η² = 0.04) but the analysis revealed an effect of the TMT-B (F(2,39) = 4.37, p = 0.019, η² = 0.18) such that worse performance was associated with more exploration (Fig. 2).

Behavioral data were fitted with the four reinforcement-learning models. The model comparison revealed that the winning model was those with the "Kalman filter" learning rule and the "SoftMax" choice rule (Fig. 3), which considers that the bandit with the largest expected payoff had a higher probability of being chosen and the decision to explore others option had different probabilities, based on their previous payoff. Thus, the choice parameter β adjusted for each participant, and giving information about the way they behaved on the task was used to examine whether apathetic patients used a different strategy. Consistent with the results above, the β was lower among apathetic participants, indicating more explorative choices and the

<table><caption>**Table 1.** Sample characteristics.</caption><thead><tr><th></th><th>All sample<br>(n = 46)</th><th>Apathetic<br>participants<br>(n = 19)</th><th>Non-apathetic<br>participants<br>(n = 27)</th></tr></thead><tbody><tr><td>Age (years)</td><td>75.91 ± 6.54</td><td>78.05 ± 5.05</td><td>74.41 ± 7.12</td></tr><tr><td>Sex (male)</td><td>17</td><td>9</td><td>8</td></tr><tr><td>Education<br>level</td><td>1/8/10/27</td><td>1/4/4/10</td><td>0/4/6/17</td></tr><tr><td>Apathy<br>Inventory</td><td>3.80 ± 4.52</td><td>8.74 ± 2.56</td><td>0.33 ± 0.73*</td></tr><tr><td>MFI</td><td>56.43 ± 19.42</td><td>71.05 ± 13.72</td><td>46.15 ± 16.03*</td></tr><tr><td>MoCA</td><td>24.78 ± 3.36</td><td>23.72 ± 3.55</td><td>25.48 ± 3.09</td></tr><tr><td>Stroop test</td><td>-0.27 ± 1.29</td><td>-0.40 ± 1.47</td><td>-0.18 ± 1.18</td></tr><tr><td>Corsi block-<br>tapping test</td><td>14.02 ± 4.04</td><td>12.83 ± 3.62</td><td>14.88 ± 4.19</td></tr><tr><td>TMT-B</td><td>60.11 ± 31.97</td><td>46.67 ± 32.40</td><td>69.07 ± 38.89*</td></tr></tbody></table>

Education levels correspond, respectively, to primary, secondary middle school, secondary high school, and higher education.

MFI Multidimensional Fatigue Inventory, MoCA Montreal cognitive assessment, TMT-B Trail-making test-part B.

*p* < 0.05; *p*-value from independent sample *t*-tests or Chi-squared tests.
MANCOVA confirmed a main effect of apathy on the combined $\beta$ (apathetic group $0.118 \pm 0.087$, 95%CI [0.076, 0.160] vs. non-apathetic group $0.129 \pm 0.087$, 95%CI [0.101, 0.158]) and total point (apathetic group $80.44 \pm 8.29$, 95%CI [76.44, 84.44] vs. non-apathetic group $89.60 \pm 5.18$, 95%CI [87.56, 91.65]) variables after controlling for age, MFI and TMT-B scores ($F(2,39) = 5.23$, $p = 0.009$, $\eta^2 = 0.21$), as well as the effect of the TMT-B ($F(2,39) = 5.88$, $p = 0.006$, $\eta^2 = 0.23$). In addition, according to the t-tests, apathetic participants had significantly more missing responses compared to the non-apathetic participants ($t(44) = 4.78$, $p < 0.001$, mean difference = 9.01, 95%CI [5.26, 12.91], $d = 1.43$, 95%CI [0.76, 2.08]), and significantly higher

<figure><img src="image_4.png" /><figcaption>**Fig. 1** Percentage of exploration and total points earned on the bandit task in apathetic vs. non-apathetic participants. The horizontal dashed lines represent the mean of percentage exploration for the apathetic (blue) and non-apathetic participants (green). The vertical dashed lines represent the means of the total number of points earned, as measured by the percentage of theoretical maximum gain. The bottom bounds, the central lines, and the top bounds of the boxplots correspond, respectively, to the first quartiles (Q1), the medians, and the third quartiles (Q3). The minimums and maximums are represented by the ends of the whiskers. \*p < 0.05 (MANCOVA and t-test).</figcaption></figure>

latency of response ($t(44) = 3.52$, $p = 0.001$, mean difference = 185.56, 95%CI [79.24, 291.88], $d = 1.05$, 95%CI [0.42, 1.68]).

## Exploration-exploitation decision-making on the bandit task and cognitive performances

Among all participants, according to linear regressions, the TMT-B performance was significantly associated with the percentage of exploration after adjusting for age ($\beta = 0.25$, 95%CI [-0.40, -0.10], $\beta = -0.50$, $p = 0.002$) such that better performance led to lower exploration. On the other hand, the MoCA and the Corsi scores were negatively associated with the percentage exploration (MoCA, $\beta = -1.63$, 95%CI [-3.04, -0.23], $\beta = -0.34$, $p = 0.023$; Corsi, $\beta = -1.48$, 95%CI [-2.65, -0.32], $\beta = -0.37$, $p = 0.013$) but no longer after adjusting for age (MoCA, $\beta = -1.47$, 95%CI [-3.02, 0.07], $\beta = -0.31$, $p = 0.06$; Corsi, $\beta = -1.40$, 95%CI [-2.85, 0.04], $\beta = -0.35$, $p = 0.056$). The Stroop performance was not associated with the percentage of exploration ($\beta = -1.49$, 95%CI [-5.50, 2.52], $\beta = -0.121$, $p = 0.46$).

## DISCUSSION

Whether goal-directed behavior is worth pursuing or not is contingent upon the balance between effort and reward. Also, decision-making can be required to choose between exploiting a familiar option and exploring a lesser-known but potentially more rewarding option. The weighting effort-reward process has been proposed to be disrupted by apathy. However, very little is known regarding explorative-exploitative decision-making despite some evidence suggesting that it could also potentially be disrupted. The aim of the present study was to examine the explore-exploit decision-making in the apathy syndrome among older adults with mild neurocognitive disorders. As hypothesized, apathetic patients behaved differently than non-apathetic patients. They took a longer time to respond and explored more compared to the non-apathetic during the reinforcement-learning task. This pattern was associated with worse performance, as evidenced by lesser total point gain, thereby suggesting an excessive explorative and inefficient decision-making in apathetic individuals³³.

The percentage of exploration was not significantly associated with the inhibition performance, nor with the memory performance after controlling for age, using the Stroop and Corsi tests. However, the TMT-B performance, assessing mental flexibility, was associated with explorative decision-making, such that lower performance was associated with higher exploration. This result suggests that reduced flexibility could be one explanation for high explorative choices found in apathetic patients with mild neurocognitive disorders compared with non-apathetic patients. Indeed, at the start, exploration is needed in order to learn about the values of available options, but then to be optimal, with the experience, using gained information, the more rewarding option

<figure><img src="image_5.png" /><figcaption>**Fig. 2** Percentage of exploration and total points earned on the bandit task according to the TMT-B score. Higher values of TMT-B (warmer colors) indicate better performance.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Fig. 3** Comparison of models' fits. Lower value of Negative log-likelihood, BIC, and AIC indicates better fitting.</figcaption></figure>

must be exploited, thereby reducing the exploration. In an environment where options can change over time, to be efficient, people need to be able to switch their responses according to changes in the environment and, therefore, to adapt. Thus, a lack of flexibility to switch from the default explorative mode toward the exploitative mode can be one of the hypotheses to explain the prolonged exploration and worse performance in apathetic patients. These findings and this hypothesis of reduced flexibility are consistent with the apathy-related executive dysfunction⁴¹, and converge with results of the recent study of Scholl et al.⁴⁵, where apathy has been linked to increased persisting in exploration, what they termed "decision inertia". Participants with apathetic traits kept searching for new offers without considering that the appropriate strategy would be to stop the search. To describe this phenomenon of persistence in a sequence of searches for longer than appropriate, they employed the well-imaged expression to be "stuck in a rut". But besides that, some studies have also reported higher exploration in individuals suffering from attention deficit hyperactivity disorders⁴⁶,⁴⁷. This suggests that the excessive exploration found in apathetic participants could also result from reduced attention, leading to reduced focus on the exploitation of the most relevant option. In line with this idea, a study conducted by Chau et al., used a visual scanning task and an eye-tracker to assess the attentional bias in apathy, in Alzheimer disease and found that apathetic participants had reduced duration of fixation on social images⁴⁸. Thus, future studies should consider both mental flexibility and attention in the solving of exploration-exploitation decision-making among apathetic individuals.

One other complementary hypothesis that could explain the higher exploration and suboptimal decision-making on the reinforcement-learning task among apathetic participants compared with non-apathetic is a deficit in reward processing, including in the use of reward cues (which guide the behavior) and learning steps⁴⁹,⁵⁰, notably, as apathy has been associated with dysfunction in dopaminergic system²¹ which play a key role in learning¹³. Indeed, for efficiently solving the exploration-exploitation dilemma, it is necessary to correctly use the feedback, and learn and update the payoffs in order to identify the most rewarding option, thereby avoiding spending too much time in inefficient exploration. This hypothesis is consistent as apathy has been related several times to impaired reward processing¹⁶,⁵¹,⁵². For instance, Wong et al.⁴⁴ demonstrated empirically, using a reward learning task, in older patients with neurocognitive disorders that emotional apathy was associated with difficulties in using socio-emotional rewards to guide behavior and impaired learning. Furthermore, several studies using gambling tasks, such as the Iowa gambling task (IGT), showed less efficient reward-based decision-making in apathy (i.e., lesser gain), which is

consistent with present findings. For instance, a negative association between the level of apathy on the action initiation dimension and performance on the IGT was found in a study by Bayard et al. in patients with Alzheimer's disease and mild cognitive impairment⁵³. Similarly, in the study by Njomboro et al., apathetic patients with brain damage performed worse on the IGT by choosing more often options that led to a lesser gain in the long term⁵⁴. Taken together, this evidence supports the idea that apathetic individuals may exhibit difficulties in using outcomes to identify and then exploit the most rewarding options. In other words, apathetic individuals may lose themselves in a multitude of ineffective explorations. In addition, it can be assumed that abnormalities in other steps of reward processing have contributed to this pattern of decision-making, such as reward insensitivity¹²,⁵⁵. It could be, therefore, interesting in future studies to add tools allowing, objectively to measure the sensitivity to reward in probabilistic reward learning tasks. Last, current findings could be explained by the relying of exploitation on the structures involved in anticipation, valuation, and experience of rewards comprising notably the mesocorticolimbic dopaminergic pathway and the vmPFC³², which are also found disrupted in apathy¹⁵,¹⁸.

In addition, to be more explorative, to perform the four-armed bandit task, apathetic participants took longer time to make their choice and more often failed to respond in the allotted time. These results may reflect the difficulties in action initiation³ and difficulties in making a choice⁵⁶ which are both clinical features of apathy. Longer times to perform decisional tasks or initiate motor actions have been previously reported in apathetic individuals. For instance, using a grasping task involving manual motricity, Manera et al.⁵⁷ recorded longer times to start the action, which has been suggested as an index of deficit in action initiation. In a study of Martinez-Horta et al., using a gambling task, reduced reaction times were also recorded⁵². Here, using the bandit reinforcement-learning task, these difficulties in making choices seem to have been expressed by both longer response times and inertial and suboptimal decisions. These results are, therefore, in line with clinical aspects regarding this feature that is explicitly formulated in clinical diagnostic criteria as follows "takes longer to make choices when different alternatives exist (e.g., selecting TV programs, preparing meals, choosing from a menu, etc.)"⁵⁶.

To summarize, in the bandit task, as in more ecological situations, optimal decision-making involves the capacity to adapt one's actions according to environmental changes. This needs mental flexibility involved to be able to update new information and to manage attention efficiently. According to the "ambidexterity hypothesis"⁵⁸, the higher the ability to balance exploration-exploitation, the better the performance. Not enough exploration deprives people of information about their environment, whereas too much exploration may lead to inefficient decision-making³³. In the present study, this exploration-exploitation decision-making has been examined among older apathetic patients. For that, we notably used reinforcement-learning models, as they are widely used to capture the computational process of based-feedback learning and to describe how individuals behave and make decisions⁵⁹,⁶⁰. Like in numerous studies, the SoftMax choice rule from the computational model had the best fit with the behavioral data of participants, and therefore the $\beta$ free parameter was used to identify the individual specificities regarding the exploration⁶⁰. Results showed higher exploration choices, suboptimal decision-making, and longer time of response in apathetic individuals compared with non-apathetic. Taken together, results fit well with clinical characteristics such as retardation, limited interest, and difficulties in making choices and suggest that the exploration-exploitation dilemma represents an interesting framework in apathy research, especially in neurocognitive disorders where apathy is highly prevalent. Indeed, it allowed us to identify a specific behavioral phenotype associated with apathy in mild neurocognitive disorders, thereby providing a new approach for
capturing a specific way to behave (behavioral level). It could also help to provide insight into potential disrupted mechanisms (explanatory process level)⁶¹. However, here, some factors that might influence findings, such as attention, have not been measured, and no objective measure to assess the underlying mechanisms has been used. Thus, it could be interesting to add eye tracking to further investigate the role of attention and reward sensitivity. In addition, further behavioral studies should specifically examine these potentially disrupted mechanisms, using, for instance, neuroimaging or pharmacologic interventions (e.g., examine the association between brain activation in orbital-ventromedial prefrontal cortex and exploration-exploitation decision-making in apathetic individuals, assess the effects of the dopaminergic drug in explorative behavior among apathetic individuals), and this in various population (e.g., young adults, healthy older adults, older adults with other neuropsychiatric diseases such as schizophrenia). Furthermore, considering the previous studies showing disruption of effort-reward-based decision-making in apathy and the present results suggesting an imbalance in exploration-exploitation trade-off, it could be relevant to examine whether other factors of decision-making are affected by apathy, such as the temporal proximity to obtain a reward that also influences the subjective value attributed to options and the subsequent action selection. This could be especially relevant as it has been suggested that delay, effort, and probability decision-making are closely related and underpinned by some common underlying brain area⁶²,⁶³. Present results provide new insight. These research works on apathy and decision-making could in perspective be used to develop tasks for clinical practice, for aiding apathy detection. Nevertheless, it must be noticed that one of the limitations of the study is the absence of a healthy aged matched control group, so future studies should include one in order to assess the role of neurocognitive disorders.

# METHODS

## Participants

Forty-six patients aged 65 years and older with mild neurocognitive disorders (NCD) were recruited in a Research Memory Center. Indeed, in order to examine the effect of apathy on exploration-exploitation decision-making in the NCD, we included and compared older adults with mild NCD exhibiting apathy vs. older adults with NCD without apathy. Here, healthy older adults without NCD were not included (but this is the subject of another ongoing study that includes the general population, including older adults, where apathy is assessed using self-reported scales). The mild NCD were identified by physicians according to the Diagnosis and Statistical Manual of Mental Disorders, 5th edition criterion (DSM-5¹) based on clinical interviews and using the neuropsychological testing performed by neuropsychologists. Participants had to be able to speak and understand French. Non-inclusion criteria were (1) the presence of current uncontrolled major psychiatric disorders (e.g., major depressive disorder), (2) visual, auditive, and motor impairments not allowing to perform the task, (3) incapacity to provide the consent to participate. In the sample, 19 patients were identified as apathetic using the Apathy Diagnostic Criteria³. The study was performed in accordance with the declaration of Helsinki and approved by the ethical committee “Comité de Protection des Personnes (CPP) Ile de France IV”, N° IDRCB: 2021-A03126-35. Written informed consent to participate in the study was obtained from all participants. An a-priori power analysis was performed using the G*Power software, in order to compare the exploration-exploitation decision-making in apathetic vs. non-apathetic participants. The sample size was estimated to be 42 subjects (subjects ($α = 0.05$, Power = 0.80, effect size $d = 0.8$).

## Measures

**Apathy and fatigue assessment.** Apathy diagnostic criteria³ was used by clinicians to identify apathetic patients. In addition, the clinician version of Apathy Inventory (AI)⁶⁴ was administered. The AI assesses emotional blunting, loss of initiative, and loss of interest. Each domain is scored from 0 ("no problem") to 4 ("major problem"). The total score ranges from 0 to 12, with a higher score indicating greater apathy.

Fatigue was assessed using the Multidimensional Fatigue Inventory (MFI)⁶⁵, a 20-item self-report questionnaire. Items are scored on a five-point Likert scale, and total scores range from 20 to 100 such that a higher score indicates greater fatigue.

**Cognitive assessment.** The Montreal Cognitive Assessment⁶⁶ (MoCA) was administered to assess global cognitive functioning. Scores ranged from 0 to 30 with the higher score indicating better performance. In addition, as executive functions play a crucial role in problem-solving and decision-making as well as in apathy, and they have been related to the resolving of such task⁶⁷,⁶⁸, the several aspects of executive functions that include the working memory (also called "updating"), the cognitive flexibility and the inhibitory control⁶⁹ were assessed using specific tests. More specifically, the Corsi block-tapping test⁷⁰ was used to assess visuo-spatial short-term memory and working memory (i.e., capacity to retain information while performing mental operations on that information). A higher score indicated a higher performance. Cognitive flexibility (i.e., capacity to switch a course of thought or action in order to adapt to changes of situation) was assessed using part B of Trail making test⁷¹ (TMT-B), and participants were classified in percentile using normative data. The Stroop test⁷² assessed the inhibitory control (i.e., capacity to inhibit dominant responses to stimuli in order to select one more appropriate), and the z-score was used.

**Exploration-exploitation decision-making task.** To assess the exploration-exploitation decision-making, a modified version of the "four-armed bandit task" from Daw et al.⁵⁹ was performed on a touch-screen computer (Fig. 4) using the Inquisit software (Millisecond, United States). The task included 200 trials divided into two blocks of 100 trials and separated by a 60-s break. Each trial started with a presentation of four different colored squares representing the four choice options. The participants had a maximum of 1.5 s to choose one of the four squares by finger touching the screen. Then, the number of points earned was displayed in the chosen colored square. However, the payoff of the non-chosen was not displayed. The payoff for each trial ranged from 1 to 100 points, and the objective for the participant was to accumulate as many points as possible across trials. If no choice was made within the given time, a red cross was displayed in the center of the screen, meaning a missed trial with no point earned. Participants were informed during the instruction phase that the number of points paid off by the four squares gradually and independently changed from trial to trial.

The number of points paid off by the slot machine i on trial t was generated in a Gaussian distribution with a standard deviation of 4 and an approximate mean $μ_{i,t}$, which changed from one trial to the next as below:

$$μ_i, t + 1 = λμ_i, t + (1 - λ)θ + v \quad (1)$$

The decay parameter $λ$ was 0.9836, and the decay center $θ$ was 50. The diffusion noise $v$ was zero-mean Gaussian with a standard deviation of 2.8. The values of rewards were displayed only by selecting the slot; thus, by selecting different alternatives, the participant can learn about the options and use this information to modify their decisions in order to improve gains. Each choice was classified as explorative if participants selected a slot that did not have the highest known payoff at this point and as exploitative if
<figure><img src="image_7.png" /><figcaption>**Fig. 4 Four-armed bandit task.** Experimental task used to assess the exploration-exploitation decision-making trade-off. Illustration of the timeline within a trial, and evolution of gain as a function of trial for each option (red, green, yellow, blue). At the bottom, for each trial, the best option (with the best payoff on the current trial) is represented (red, green, yellow or blue dot/option).</figcaption></figure>

the selected slot was the one with the maximum payoff known at this point. The percentage of explorative choices was computed as well as the total score, as measured by the total number of points earned to the theoretical maximum gain ratio. The number of missing responses and time to choose was also recorded.

### Reinforcement-learning model

Behavioral data was modeled using four reinforcement-learning models by combining two different learning rules (k constant vs. Kalman filter) with two different choice rules (SoftMax vs. ε-greedy). The learning rules computed and updated the expected payoffs (μ̂ᵢ,ₜ) of each option (i) on each trial (t) based on previous choices and obtained rewards and then, based on these estimations, the choice rules computed the probability of each bandit to getting selected (Pᵢ,ₜ). These approaches were used as the ε-greedy and k constant are considered as a basic rule in the reinforcement-learning model, and because the SoftMax is a major one it is largely used in studies, including in those involving older adults⁶⁰.

With the first learning rule (k constant) tested, the estimated payoff of the chosen c bandit on trial t was updated by the obtained reward r as follows:

$$ \hat{\mu}_{c,t}^{\text{post}} = \hat{\mu}_{c,t}^{\text{pre}} + \hat{\kappa}\delta_t \quad (2) $$

with δt the prediction error on trial t corresponding to rt − μ̂c,t and k the learning rate parameter. For the unchosen bandit, the estimated payoffs do not change.

For the second learning rule tested, the Kalman filter as implemented by Daw et al. (2006)⁵⁹ was run. In this Bayesian model, in addition to tracking the mean payoff (μ̂ᵢ,ₜ), the model computed the uncertainty (σ²ᵢ,ₜ), which corresponded to the

variance of the distribution of the expected reward, and determined the learning rate from trial by trial (k̂t). When the participant chose a bandit c on trial t, the posterior estimated distribution with mean (μ̂postc,t) and variance (σ²postc,t) were updated as follows:

$$ \hat{\mu}_{c,t}^{\text{post}} = \hat{\mu}_{c,t}^{\text{pre}} + \hat{\kappa}\delta_t \quad (3) $$

with δt the prediction error on trial t corresponding to rt − μ̂c,t

$$ \hat{\sigma}_{c,t}^{2\text{post}} = (1 - \hat{k}_t)\hat{\sigma}_{c,t}^{2\text{pre}} \quad (4) $$

with k̂t the learning rate computed as σ²postc,t / (σ²postc,t + σ²o) where σ²o is the estimated observation variance.

For all bandits, the distribution of the expected payoffs (prior mean and variance) on the subsequent trial were updated as follows:

$$ \hat{\mu}_{i,t+1}^{\text{pre}} = \hat{\lambda}\hat{\mu}_{i,t}^{\text{post}} + (1 - \hat{\lambda})\hat{\theta} \quad (5) $$

$$ \hat{\sigma}_{i,t+1}^{2\text{pre}} = \hat{\lambda}^2 \hat{\sigma}_{i,t}^{2\text{post}} + \hat{\sigma}_d^2 \quad (6) $$

Then, using the SoftMax choice rule, the resulting probabilities (Pᵢ,ₜ) of choosing option (i) on trial (t) were computed according to:

$$ P_{i,t} = \frac{\exp(\beta \hat{\mu}_{i,t}^{\text{pre}})}{\sum_j \exp(\beta \hat{\mu}_{j,t}^{\text{pre}})} \quad (7) $$

with β the parameter giving information about the explorative behavior of participant such that lower β indicate more exploration.

The second choice rule was the ε-greedy, as used by Daw et al. (2006). With this model, the probabilities (Pᵢ,ₜ) of choosing option
(i) on trial (t) were computed according to:

$$P_{i,t} = \varepsilon + (1 - 4\varepsilon) \frac{\exp\left(\left(\frac{100}{\sqrt{\sum k_{\mu_{i,k}}}}\right) \mu_{i,t}\right)}{\sum_j \exp\left(\left(\frac{100}{\sqrt{\sum k_{\mu_{j,k}}}}\right) \mu_{j,t}\right)} \quad (8)$$

with ε giving information about the exploitative behavior of participant such that lower ε indicate more exploitation.

Thus, the ε-greedy rule considered that the greedy choice, the one with the largest anticipated pay-off, had a higher probability to be chosen, and the other, non-greedy, options were uniformly explored whereas, with the SoftMax rule, the decision to explore other options had different probability and were based on their previous payoff. While the percentage of exploration was based on the seen payoff (such that the choice of the bandit with the highest payoff seen at this time corresponded to exploitation), in the models, the classification was based on the estimated pay-off, as computed in the learning rule, (such that the choice of the bandit with the highest estimated pay-off correspond to exploitation). The free parameters of models (β or ε) were computed to fit with the data of each participant in order to give information about the way each of them behaved.

In order to identify the best model, models' fits were compared using the negative log-likelihood, the Akaike Information Criterion (AIC), and the Bayesian Information Criterion (BIC).

## Statistical analysis

To examine behavioral performance on the bandit task and the associated efficiency in apathetic vs. non-apathetic participants with mild neurocognitive disorders, multivariate analyses of covariance (MANCOVA) with apathy status as a predictor factor were conducted. One was run with the percentage of exploration and the total number of points as dependent variables. The other one was conducted with the total number of points and the estimated parameter from the best reinforcement model (β, ε). All were adjusted by age, cognitive performance, and fatigue level. Effect sizes were computed using the partial eta squared (η²) where a value of 0.01, 0.06, and 0.14 corresponded, respectively, to a small, medium, and large effect size. For the number of missing responses and the chosen time, one-sided Student t-tests were performed, and Cohen's d was used to calculate the effect size. To examine the associations between cognitive performance and exploration behavior, linear regressions were conducted. For all tests, the alpha level was set at 0.05.

## Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

## DATA AVAILABILITY

The datasets generated and analyzed during the current study are available from the corresponding authors upon reasonable request.

Received: 18 April 2023; Accepted: 14 September 2023; Published online: 30 October 2023

## REFERENCES

1. American Psychiatric Association. *Diagnostic and Statistical Manual of Mental Disorders*. Fifth Edition. American Psychiatric Association; 2013. https://doi.org/10.1176/appi books.9780890425596.

2. Ismail, Z. et al. Neuropsychiatric symptoms as early manifestations of emergent dementia: Provisional diagnostic criteria for mild behavioral impairment. *Alzheimers Dement*. **12**, 195–202 (2016).

3. Miller, D. S. et al. Diagnostic criteria for apathy in neurocognitive disorders. *Alzheimers Dement*. **17**, 1892–1904 (2021).

4. Starkstein, S. E., Jorge, R. & Mizrahi, R. The prevalence, clinical correlates and treatment of apathy in Alzheimer's disease. *Eur. J. Psychiatry*. **20**, 96–106 (2006).

5. van Reekum, R., Stuss, D. T. & Ostrander, L. Apathy: why care. *J. Neuropsychiatry Clin. Neurosci*. **17**, 7–19 (2005).

6. Chong, T. T. J. Definition: Apathy. *Cortex* **128**, 326–327 (2020).

7. Daumas, L. et al. Associations, overlaps and dissociations between apathy and fatigue. *Sci. Rep*. **12**, 7387 (2022).

8. Levy, R. Apathy: a pathology of goal-directed behaviour: a new concept of the clinic and pathophysiology of apathy. *Rev. Neurol. (Paris)* **168**, 585–597 (2012).

9. Haber, S. N. & Behrens, T. E. J. The neural network underlying incentive-based learning: implications for interpreting circuit disruptions in psychiatric disorders. *Neuron*. **83**, 1019–1039 (2014).

10. Padoa-Schioppa, C. Neurobiology of economic choice: a good-based model. *Annu. Rev. Neurosci*. **34**, 333–359 (2011).

11. Levy, D. J. & Glimcher, P. W. The root of all value: a neural common currency for choice. *Curr. Opin. Neurobiol*. **22**, 1027–1038 (2012).

12. Zwosta, K., Ruge, H. & Wolfensteller, U. Neural mechanisms of goal-directed behavior: outcome-based response selection is associated with increased functional coupling of the angular gyrus. *Front. Hum. Neurosci*. **9**, 180 (2023).

13. Schultz, W. Dopamine reward prediction-error signalling: a two-component response. *Nat. Rev. Neurosci*. **17**, 183–195 (2016).

14. Bijleveld, E., Custers, R. & Aarts, H. Adaptive reward pursuit: How effort requirements affect unconscious reward responses and conscious reward decisions. *J. Exp. Psychol. Gen*. **141**, 728–742 (2012).

15. Le Heron, C., Apps, M. A. J. & Husain, M. The anatomy of apathy: A neurocognitive framework for amotivated behaviour. *Neuropsychologia* **118**, 54–67 (2018).

16. Le Heron, C., Holroyd, C. B., Salamone, J. & Husain, M. Brain mechanisms underlying apathy. *J. Neurol. Neurosurg. Psychiatry* **90**, 302–312 (2019).

17. Bonnelle, V., Veromann, K. R., Burnett Heyes, S., Lo Sterzo, E., Manohar, S. & Husain, M. Characterization of reward and effort mechanisms in apathy. *J. Physiol.-Paris* **109**, 16–26 (2015).

18. Raimo, S., Santangelo, G., D'lorio, A., Trojano, L. & Grossi, D. Neural correlates of apathy in patients with neurodegenerative disorders: an activation likelihood estimation (ALE) meta-analysis. *Brain Imag. Behav*. **13**, 1815–1834 (2019).

19. Kos, C., van Tol, M. J., Marsman, J. B. C., Knegtering, H. & Aleman, A. Neural correlates of apathy in patients with neurodegenerative disorders, acquired brain injury, and psychiatric disorders. *Neurosci. Biobehav. Rev*. **69**, 381–401 (2016).

20. Starkstein, S. E. & Brockman, S. The neuroimaging basis of apathy: Empirical findings and conceptual challenges. *Neuropsychologia* **118**, 48–53 (2018).

21. Chong, T. T. J. & Husain, M. The role of dopamine in the pathophysiology and treatment of apathy. *Prog. Brain Res*. **229**, 389–426 (2016).

22. Le Bouc, R. et al. Computational Dissection of Dopamine Motor and Motivational Functions in Humans. *J. Neurosci*. **36**, 6623–6633 (2016).

23. Adam, R. et al. Dopamine reverses reward insensitivity in apathy following globus pallidus lesions. *Cortex* **49**, 1292–1303 (2013).

24. Jurgelis, M., Chong, W. B., Atkins, K. J., Cooper, P. S., Coxon, J. P. & Chong, T. T. J. Heightened effort discounting is a common feature of both apathy and fatigue. *Sci. Rep*. **11**, 22283 (2021).

25. Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. *Philos. Trans. R Soc. Lond. B Biol. Sci*. **362**, 933–942 (2007).

26. Mehlhorn, K. et al. Unpacking the exploration-exploitation tradeoff: A synthesis of human and animal literatures. *Decision* **2**, 191–215 (2015).

27. March, J. G. Exploration and Exploitation in Organizational Learning. *Organ. Sci*. **2**, 71–87 (1991).

28. Domenech, P., Rheims, S. & Koechlin, E. Neural mechanisms resolving exploitation-exploration dilemmas in the medial prefrontal cortex. *Science* **369**, eabb0184 (2020).

29. de Boer, L. et al. Attenuation of dopamine-modulated prefrontal value signals underlies probabilistic reward learning deficits in old age. *eLife* **6**, e26424 (2017).

30. Badre, D., Doll, B. B., Long, N. M. & Frank, M. J. Rostrolateral Prefrontal Cortex and Individual Differences in Uncertainty-Driven Exploration. *Neuron* **73**, 595–607 (2012).

31. Tomov, M. S., Truong, V. Q., Hundia, R. A. & Gershman, S. J. Dissociable neural correlates of uncertainty underlie different exploration strategies. *Nat. Commun*. **11**, 2371 (2020).

32. Laureiro-Martinez, D., Brusoni, S. & Zollo, M. The neuroscientific foundations of the exploration-exploitation dilemma. *J. Neurosci. Psychol. Econ*. **3**, 95–115 (2010).

33. Levinthal, D. A. & March, J. G. The myopia of learning. *Strat. Mgmt. J*. **14**, 95–112 (1993).
34. Cui, G. et al. Concurrent activation of striatal direct and indirect pathways during action initiation. *Nature* **494**, 238–242 (2013).

35. Hamid, A. A. et al. Mesolimbic dopamine signals the value of work. *Nat. Neurosci.* **19**, 117–126 (2016).

36. Humphries, M. Dopaminergic control of the exploration-exploitation trade-off via the basal ganglia. *Front. Neurosci.* **6**, 9. https://doi.org/10.3389/fnins.2012.00009 (2012).

37. Mizell, J. M. et al. *Differential Impacts of Healthy Cognitive Aging on Directed and Random Exploration*. *PsyArXiv*. https://doi.org/10.31234/osf.io/64jnk (2022).

38. Mata, R., Wilke, A. & Czienskowski, U. Foraging across the life span: is there a reduction in exploration with aging? *Front. Neurosci.* **7**, 53 (2013).

39. Chowdhury, R. et al. Dopamine restores reward prediction errors in old age. *Nat. Neurosci.* **16**, 648–653 (2013).

40. Spreng, R. N. & Turner, G. R. From exploration to exploitation: a shifting mental mode in late life development. *Trend. Cogn. Sci.* **25**, 1058–1071 (2021).

41. McPherson, S., Fairbanks, L., Tiken, S., Cummings, J. L. & Back-Madruga, C. Apathy and executive function in Alzheimer's disease. *J. Int. Neuropsychol. Sci.* **8**, 373–381 (2002).

42. Cathomas, F. et al. Increased random exploration in schizophrenia is associated with inflammation. *npj Schizophr.* **7**, 6 (2021).

43. Peterson, D. A., Elliott, C., Song, D. D., Makeig, S., Sejnowski, T. J. & Poizner, H. Probabilistic reversal learning is impaired in Parkinson's disease. *Neuroscience.* **163**, 1092–1101 (2009).

44. Wong, S. et al. Altered reward processing underpins emotional apathy in dementia. *Cogn Affect Behav Neurosci.* Published online November 23 (2022). https://doi.org/10.3758/s13415-022-01048-2.

45. Scholl, J., Trier, H. A., Rushworth, M. F. S. & Kolling, N. The effect of apathy and compulsivity on planning and stopping in sequential decision-making. *Chambers CD, ed.* *PLoS Biol.* **20**, e3001566 (2022).

46. Addicott, M. A., Pearson, J. M., Schechter, J. C., Sapyta, J. J., Weiss, M. D. & Kollins, S. H. Attention-deficit/hyperactivity disorder and the explore/exploit trade-off. *Neuropsychopharmacol.* **46**, 614–621 (2021).

47. Van den Driessche, C., Chevrier, F., Cleeremans, A. & Sackur, J. Lower Attentional Skills predict increased exploratory foraging patterns. *Sci. Rep.* **9**, 10948 (2019).

48. Chau, S. A., Chung, J., Herrmann, N., Eizenman, M. & Lanctôt, K. L. Apathy and Attentional Biases in Alzheimer's Disease. *J. Alzheimers Dis.* **51**, 837–846 (2016).

49. Chakroun, K., Mathar, D., Wiehler, A., Ganzer, F. & Peters, J. Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making. *eLife.* **9**, e51260 (2020).

50. Wilson, R. C., Bonawitz, E., Costa, V. D. & Ebitz, R. B. Balancing exploration and exploitation with information and randomization. *Curr. Opin. Behav. Sci.* **38**, 49–56 (2021).

51. Manohar, S. G. & Husain, M. Human ventromedial prefrontal lesions alter incentivisation by reward. *Cortex* **76**, 104–120 (2016).

52. Martinez-Horta, S. et al. Apathy in Parkinson's Disease: Neurophysiological Evidence of Impaired Incentive Processing. *J. Neurosci.* **34**, 5918–5926 (2014).

53. Bayard, S., Jacus, J. P., Raffard, S. & Gely-Nargeot, M. C. Apathy and Emotion-Based Decision-Making in Amnesic Mild Cognitive Impairment and Alzheimer's Disease. *Behav. Neurol.* **2014**, 1–7 (2014).

54. Njomboro, P., Deb, S. & Humphreys, G. W. Apathy and executive functions: insights from brain damage involving the anterior cingulate cortex. *Case Rep.* **2012**, bcr0220125934 (2012).

55. Rangel, A., Camerer, C. & Montague, P. R. A framework for studying the neurobiology of value-based decision making. *Nat. Rev. Neurosci.* **9**, 545–56 (2008).

56. Robert, P. et al. Is it time to revise the diagnostic criteria for apathy in brain disorders? The 2018 international consensus group. *Eur. psychiatr.* **54**, 71–76 (2018).

57. Manera, V., Galperti, G. & Rovini, E. et al. Grasping Social Apathy: The Role of Reach-To-Grasp Action Kinematics for the Assessment of Social Apathy in Mild Neurocognitive Disorders. *J Alzheimers Dis.* **81**, 569–582 (2021).

58. Tushman, M. L. & O'Reilly, C. A. Ambidextrous Organizations: Managing Evolutionary and Revolutionary Change. *Calif. Manag. Rev.* **38**, 8–29 (1996).

59. Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. *Nature* **441**, 876–879 (2006).

60. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction. *IEEE Trans. Neural Netw.* **9**, 1054 (1998).

61. Pessiglione, M., Vinckier, F., Bouret, S., Daunizeau, J. & Le Bouc, R. Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases. *Brain* **141**, 629–650 (2018).

62. Massar, S. A. A., Libedinsky, C., Weiyan, C., Huettel, S. A. & Chee, M. W. L. Separate and overlapping brain areas encode subjective value during delay and effort discounting. *Neuroimage* **120**, 104–113 (2015).

63. Bailey, M. R., Simpson, E. H. & Balsam, P. D. Neural substrates underlying effort, time, and risk-based decision making in motivated behavior. *Neurobiol. Learn. Mem.* **133**, 233–256 (2016).

64. Robert, P. H. et al. The Apathy Inventory: assessment of apathy and awareness in Alzheimer's disease, Parkinson's disease and mild cognitive impairment. *Int. J. Geriat. Psychiatry* **17**, 1099–1105 (2002).

65. Smets, E. M. A., Garssen, B., Bonke, B. & De Haes, J. C. J. M. The multidimensional Fatigue Inventory (MFI) psychometric qualities of an instrument to assess fatigue. *J. Psychosomat. Res.* **39**, 315–325 (1995).

66. Nasreddine, Z. S. et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. *J. Am. Geriatr. Soc.* **53**, 695–699 (2005).

67. Lehto, J. E. & Elorinne, E. Gambling as an executive function task. *Appl. Neuropsychol.* **10**, 234–238 (2003).

68. Rmus, M., McDougle, S. D. & Collins, A. G. E. The Role of Executive Function in Shaping Reinforcement Learning. *Curr. Opin. Behav. Sci.* **38**, 66–73 (2021).

69. Miyake, A., Friedman, N. P., Witzki, A. H., Howerter, A. & Wager, T. D. The unity and diversity of executive functions and their contributions to complex "Frontal Lobe" tasks: a latent variable analysis. *Cogn. Psychol.* **41**, 49–100 (2000).

70. Berch, D. B., Krikorian, R. & Huha, E. M. The Corsi Block-Tapping Task: Methodological and Theoretical Considerations. *Brain and Cognition.* **38**, 317–338 (1998).

71. Reitan, R. M. Trail making test. Manual for administration, scoring, and interpretation. *Indianapolis, IN: Indiana University Press*. Published online 1956.

72. Bench, C. et al. Investigations of the functional anatomy of attention using the Stroop test. *Neuropsychologia* **31**, 907–922 (1993).

## ACKNOWLEDGEMENTS

This study received funding from Neuromod, Université Côte d'Azur to acquire the Inquisit software, and a grant from the Graduate School of Health Science Ecosystems —HEALTHY of Université Côte d'azur, funded by the Agence National de la Recherche (ANR).

## AUTHOR CONTRIBUTIONS

L.D., R.Z., S.R. and V.M. designed the study. E.E., G.S., L.D. and M.F. included patients and carried out the protocol. I.J., L.D., R.Z., S.R. and V.M. analyzed the data. L.D., P.R., R.Z., S.R. and V.M. wrote the manuscript. S.R. and V.M. are co-last authors.

## COMPETING INTERESTS

The authors declare no competing interests.

## ADDITIONAL INFORMATION

**Supplementary information** The online version contains supplementary material available at https://doi.org/10.1038/s41514-023-00121-5.

**Correspondence** and requests for materials should be addressed to Lyne Daumas.

**Reprints and permission information** is available at http://www.nature.com/reprints

**Publisher's note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_8.png" /></figure>

**Open Access** This article is licensed under a Creative Commons

Attribution 4.0 International License, which permits use, sharing,

adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

© The Author(s) 2023