Computational Psychiatry of ADHD: Neural Gain Impairments across Marrian Levels of Analysis
Authors: Tobias U. Hauser, Vincenzo G. Fiore, Michael Moutoussis, Raymond J. Dolan
Journal: Trends in Neurosciences (2016)

ADHD is one of the most common psychiatric disorders during childhood, but the neurocognitive mechanisms behind it remain elusive. Behaviourally, ADHD is best characterized by increased variability across multiple cognitive domains and timescales. By using Marr's three levels of analysis, we show how impairments in neural gain can explain ADHD abnormalities, spanning from behaviour to neural activity. On an algorithmic and implementation level, we show how increased variability can be caused by neural gain impairments, and how it can be modelled using reinforcement learning and corticostriatal network models. We furthermore show how these levels can be linked to impairments in catecholamine systems (dopamine and noradrenaline). Attention-deficit hyperactivity disorder (ADHD), one of the most common psychiatric disorders, is characterised by unstable response patterns across multiple cognitive domains. However, the neural mechanisms that explain these characteristic features remain unclear. Using a computational multilevel approach, we propose that ADHD is caused by impaired gain modulation in systems that generate this phenotypic increased behavioural variability. Using Marr's three levels of analysis as a heuristic framework, we focus on this variable behaviour, detail how it can be explained algorithmically, and how it might be implemented at a neural level through catecholamine influences on corticostriatal loops. This computational, multilevel, approach to ADHD provides a framework for bridging gaps between descriptions of neuronal activity and behaviour, and provides testable predictions about impaired mechanisms. Attention-deficit hyperactivity disorder (ADHD), one of the most common psychiatric disorders, is characterised by unstable response patterns across multiple cognitive domains. However, the neural mechanisms that explain these characteristic features remain unclear. Using a computational multilevel approach, we propose that ADHD is caused by impaired gain modulation in systems that generate this phenotypic increased behavioural variability. Using Marr's three levels of analysis as a heuristic framework, we focus on this variable behaviour, detail how it can be explained algorithmically, and how it might be implemented at a neural level through catecholamine influences on corticostriatal loops. This computational, multilevel, approach to ADHD provides a framework for bridging gaps between descriptions of neuronal activity and behaviour, and provides testable predictions about impaired mechanisms. Maintaining one's mental focus is hard, especially when reading a dry and complicated paper. Suddenly you would rather clean the kitchen or surf the Internet. Nevertheless, most people maintain focus and persist with the task at hand. Neurobiologically, we propose that the catecholaminergic brain systems (see Glossary) modulate attention [1Clark C.R. et al.Catecholamines and attention. I: Animal and clinical studies.Neurosci. Biobehav. Rev. 1987; 11: 341-352Crossref PubMed Scopus (117) Google Scholar] by increasing the neural gain and, thus, suppressing cognitive switching [2Fiore V.G. et al.Keep focussing: striatal dopamine multiple functions resolved in a single mechanism tested in a simulated humanoid robot.Front. Psychol. 2014; 5: 124Crossref PubMed Scopus (31) Google Scholar] (Box 1).Box 1Neural Gain: Catecholamines Regulating Stability of Neuronal SystemsThe brain can be thought of as a signal-processing machine that selects relevant information to act. Overburdening with information means that it needs to decide which aspects of its inputs to treat as important by boosting these relevant signals, and which aspects to treat as unimportant and attenuate. The brain cannot just rely on amplifying the strongest signal and filtering out everything else, but must keep a balance between competing signals according to environmental and internal demands. The degree to which neural signals are amplified or suppressed has been termed 'neural gain' and this effect can be mimicked by a sigmoidal function (Equation I):fGx=11+e−Gx+B,[I] where an input signal x is amplified by the neural gain factor G [70Servan-Schreiber D. et al.A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior.Science. 1990; 249: 892-895Crossref PubMed Scopus (541) Google Scholar] (Figure IA).In high neural gain states (Figure IA, orange), neural populations strengthen strong and attenuate weak incoming signals. This leads to neural representations that are less susceptible to noise [71Cohen J.D. et al.Computational perspectives on dopamine function in prefrontal cortex.Curr. Opin. Neurobiol. 2002; 12: 223-229Crossref PubMed Scopus (301) Google Scholar]. Such states are most beneficial in conditions where the brain needs to avoid distraction, such as fleeing from a predator.By contrast, in low neural gain states (Figure IB, blue), the system is not dominated by the most prevalent signals and, thus, it is more likely to detect weaker signals that may carry important information [29Eldar E. et al.The effects of neural gain on attention and learning.Nat. Neurosci. 2013; 16: 1146-1153Crossref PubMed Scopus (255) Google Scholar]. Such states can be helpful because weak, but important, information might be carried in a nondominant channel. For example, seeing the silhouette of a predator in the grass or in the periphery of vision.Neural gain can also be related to neural attractor states: high gain leads to stable behaviours and attractor states where neural networks quickly converge to stable firing patterns (Figure IC, pink starting states quickly and consistently result in the same end states; cf supplemental information online). However, low gain is characterised by variable attractor states and behaviours (Figure ID, pink starting states end up in multiple unstable states).Neural gain should affect widespread neural populations. Thus, it is not surprising that the catecholaminergic neurotransmitter systems [i.e., dopamine (DA) and noradrenaline (NA)] have been found to function as neural gain modulators [31Cohen J.D. Aston-Jones G. Cognitive neuroscience: decision amid uncertainty.Nature. 2005; 436: 471-472Crossref PubMed Scopus (29) Google Scholar, 70Servan-Schreiber D. et al.A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior.Science. 1990; 249: 892-895Crossref PubMed Scopus (541) Google Scholar, 71Cohen J.D. et al.Computational perspectives on dopamine function in prefrontal cortex.Curr. Opin. Neurobiol. 2002; 12: 223-229Crossref PubMed Scopus (301) Google Scholar, 72Durstewitz D. A few important points about dopamine's role in neural network dynamics.Pharmacopsychiatry. 2006; 39: S72-S75Crossref PubMed Scopus (27) Google Scholar]. Both systems innervate many cortical and subcortical areas (Figure IB). Moreover, these systems modulate ongoing neural activity, rather than sending their own excitatory or inhibitory signals [29Eldar E. et al.The effects of neural gain on attention and learning.Nat. Neurosci. 2013; 16: 1146-1153Crossref PubMed Scopus (255) Google Scholar, 70Servan-Schreiber D. et al.A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior.Science. 1990; 249: 892-895Crossref PubMed Scopus (541) Google Scholar, 73Aston-Jones G. Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.Annu. Rev. Neurosci. 2005; 28: 403-450Crossref PubMed Scopus (2668) Google Scholar].Although catecholaminergic systems have many similarities, they serve different functions: DA has strong projections to prefrontal and striatal areas and has mainly been associated with learning and reward-related information processing [40Schultz W. et al.A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599Crossref PubMed Scopus (6003) Google Scholar]. By contrast, NA mainly innervates prefrontal areas and, to a lesser extent, striatal areas [73Aston-Jones G. Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.Annu. Rev. Neurosci. 2005; 28: 403-450Crossref PubMed Scopus (2668) Google Scholar]. It also subserves a general focussing on relevant information, irrespective of the cognitive domain [74Eldar, E. et al. (in revision) Do you see the forest or the tree? Neural gain and integration during perceptual processingGoogle Scholar]. However, clearer distinctions are yet to be drawn that might eventually help to diagnose impairments of either system. The brain can be thought of as a signal-processing machine that selects relevant information to act. Overburdening with information means that it needs to decide which aspects of its inputs to treat as important by boosting these relevant signals, and which aspects to treat as unimportant and attenuate. The brain cannot just rely on amplifying the strongest signal and filtering out everything else, but must keep a balance between competing signals according to environmental and internal demands. The degree to which neural signals are amplified or suppressed has been termed 'neural gain' and this effect can be mimicked by a sigmoidal function (Equation I):fGx=11+e−Gx+B,[I] where an input signal x is amplified by the neural gain factor G [70Servan-Schreiber D. et al.A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior.Science. 1990; 249: 892-895Crossref PubMed Scopus (541) Google Scholar] (Figure IA). In high neural gain states (Figure IA, orange), neural populations strengthen strong and attenuate weak incoming signals. This leads to neural representations that are less susceptible to noise [71Cohen J.D. et al.Computational perspectives on dopamine function in prefrontal cortex.Curr. Opin. Neurobiol. 2002; 12: 223-229Crossref PubMed Scopus (301) Google Scholar]. Such states are most beneficial in conditions where the brain needs to avoid distraction, such as fleeing from a predator. By contrast, in low neural gain states (Figure IB, blue), the system is not dominated by the most prevalent signals and, thus, it is more likely to detect weaker signals that may carry important information [29Eldar E. et al.The effects of neural gain on attention and learning.Nat. Neurosci. 2013; 16: 1146-1153Crossref PubMed Scopus (255) Google Scholar]. Such states can be helpful because weak, but important, information might be carried in a nondominant channel. For example, seeing the silhouette of a predator in the grass or in the periphery of vision. Neural gain can also be related to neural attractor states: high gain leads to stable behaviours and attractor states where neural networks quickly converge to stable firing patterns (Figure IC, pink starting states quickly and consistently result in the same end states; cf supplemental information online). However, low gain is characterised by variable attractor states and behaviours (Figure ID, pink starting states end up in multiple unstable states). Neural gain should affect widespread neural populations. Thus, it is not surprising that the catecholaminergic neurotransmitter systems [i.e., dopamine (DA) and noradrenaline (NA)] have been found to function as neural gain modulators [31Cohen J.D. Aston-Jones G. Cognitive neuroscience: decision amid uncertainty.Nature. 2005; 436: 471-472Crossref PubMed Scopus (29) Google Scholar, 70Servan-Schreiber D. et al.A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior.Science. 1990; 249: 892-895Crossref PubMed Scopus (541) Google Scholar, 71Cohen J.D. et al.Computational perspectives on dopamine function in prefrontal cortex.Curr. Opin. Neurobiol. 2002; 12: 223-229Crossref PubMed Scopus (301) Google Scholar, 72Durstewitz D. A few important points about dopamine's role in neural network dynamics.Pharmacopsychiatry. 2006; 39: S72-S75Crossref PubMed Scopus (27) Google Scholar]. Both systems innervate many cortical and subcortical areas (Figure IB). Moreover, these systems modulate ongoing neural activity, rather than sending their own excitatory or inhibitory signals [29Eldar E. et al.The effects of neural gain on attention and learning.Nat. Neurosci. 2013; 16: 1146-1153Crossref PubMed Scopus (255) Google Scholar, 70Servan-Schreiber D. et al.A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior.Science. 1990; 249: 892-895Crossref PubMed Scopus (541) Google Scholar, 73Aston-Jones G. Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.Annu. Rev. Neurosci. 2005; 28: 403-450Crossref PubMed Scopus (2668) Google Scholar]. Although catecholaminergic systems have many similarities, they serve different functions: DA has strong projections to prefrontal and striatal areas and has mainly been associated with learning and reward-related information processing [40Schultz W. et al.A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599Crossref PubMed Scopus (6003) Google Scholar]. By contrast, NA mainly innervates prefrontal areas and, to a lesser extent, striatal areas [73Aston-Jones G. Cohen J.D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance.Annu. Rev. Neurosci. 2005; 28: 403-450Crossref PubMed Scopus (2668) Google Scholar]. It also subserves a general focussing on relevant information, irrespective of the cognitive domain [74Eldar, E. et al. (in revision) Do you see the forest or the tree? Neural gain and integration during perceptual processingGoogle Scholar]. However, clearer distinctions are yet to be drawn that might eventually help to diagnose impairments of either system. For 5% of the population, the ability to focus is disturbed to an extent that strongly affects their daily functioning. Many are diagnosed with ADHD [3Polanczyk G. et al.The worldwide prevalence of ADHD: a systematic review and metaregression analysis.Am. J. Psychiatry. 2007; 164: 942-948Crossref PubMed Scopus (2241) Google Scholar], a developmental psychiatric disorder thought to arise, in part, out of a genetic vulnerability [4Burt S.A. Rethinking environmental contributions to child and adolescent psychopathology: a meta-analysis of shared environmental influences.Psychol. Bull. 2009; 135: 608-637Crossref PubMed Scopus (340) Google Scholar]. ADHD is characterised by inattention, hyperactivity, and/or impulsivity [5APA Diagnostic and Statistical Manual of Mental Disorders: DSM-5. American Psychiatric Association, 2013Google Scholar] and its negative effects on a person's occupational success, wellbeing, and health risks (e.g., for substance abuse [6Faraone S.V. et al.Attention-deficit/hyperactivity disorder.Nat. Rev. Dis. Primer. 2015; (Published online August 6, 2015)https://doi.org/10.1038/nrdp.2015.20Crossref PubMed Scopus (781) Google Scholar]) make it important to understand this disorder. Research on ADHD has intensified since the early 1990s [6Faraone S.V. et al.Attention-deficit/hyperactivity disorder.Nat. Rev. Dis. Primer. 2015; (Published online August 6, 2015)https://doi.org/10.1038/nrdp.2015.20Crossref PubMed Scopus (781) Google Scholar] without clear candidate genes or brain response patterns predicting the disorder having been identified. There is no unifying theory explaining the pathophysiology of ADHD. Indeed, current classification criteria are likely to subsume multiple brain disorders with a similar behavioural expression within the label 'ADHD'. Here, we use a multilevel approach to propose that ADHD crucially involves an impairment of neural gain modulation leading to inappropriately variable behaviour. By using Marr's three levels of analysis [7Marr D. Vision: A Computational Investigation Into the Human Representation and Processing of Visual Information. MIT Press, 1982Google Scholar] (Box 2), we show how it is possible to translate behavioural findings into mathematical algorithms and neural circuit impairments (and vice versa). This approach also provides fruitful hypotheses about potential neurobiological subgroups, which could be the object of future investigation.Box 2Mechanisms of Psychiatric Disorders: From Behaviour to Neurons and BackPsychiatric disorders are classically diagnosed based on symptom reports and clinical observations. These clinical features are rarely diagnostic of specific underlying pathological mechanisms. Here, we propose a multilevel approach to understand psychiatric disorders and their neural underpinnings. To generate hypotheses about malfunctioning brain systems, a fine-grained dissection of a patient's behaviour is important. Once consistent behavioural signatures have been found (e.g., increased response variability), we have to bridge the gap between behaviour and the neural processes that give rise to this behaviour. At the most abstract level, we formulate the key computational issue, that is, establish what problem the brain tries to solve (e.g., an optimal balance between exploiting a good foraging ground and exploring new grounds). Here, we try to answer this question from a normative perspective. Subsequently, we have to formulate how the problem is solved. At this 'algorithmic level', reinforcement learning has been shown to be useful [65Sutton R.S. Barto A.G. Reinforcement Learning: An Introduction. MIT Press, 1998Google Scholar]. Models should fulfil several requirements: (i) a good match of model predictions with the actual behaviour of an agent; (ii) model must outperform other (more simple and more complex) models in terms of model evidence; (iii) the model should have high biological plausibility (e.g., phasic DA studies lend support to RPE reinforcement learning models). Model and parameter comparison in health versus disease can then elucidate processes that underpin impairments (e.g., decision temperature parameter driving variability in ADHD; [16Hauser T.U. et al.Role of the medial prefrontal cortex in impaired decision making in juvenile attention-deficit/hyperactivity disorder.JAMA Psychiatry. 2014; 71: 1165-1173Crossref PubMed Scopus (103) Google Scholar]). Model predictions from the algorithmic level can be used to inform data such as neuroimaging, which seeks to identify neural correlates and dynamics. By using model-derived predictions (e.g., RPEs), we can look for regions (e.g., medial prefrontal cortex) whose activity to model on the level below, thus connecting algorithmic with implementation levels. At the latter level, we can then simulate complex dynamics of neuronal systems to understand impairments. Here, we can test how problematic catecholamine systems can affect behaviour and neural activity. Thus, we can formulate new theories about neural mechanisms and potential subgroups, such as low striatal DA versus decreased frontal NA subgroups in ADHD.Using such multilevel approaches in computational psychiatry [75Montague P.R. et al.Computational psychiatry.Trends Cogn. Sci. 2012; 16: 72-80Abstract Full Text Full Text PDF PubMed Scopus (480) Google Scholar, 76Dayan P. et al.Taming the shrewdness of neural function: methodological challenges in computational psychiatry.Curr. Opin. Behav. Sci. 2015; 5: 128-132Crossref Scopus (7) Google Scholar] helps link several levels of symptom analysis (behaviour, algorithmic, and neuronal). By finding new diagnostic subgroups, we can in principle refine therapies, based on more specific predictions about the efficacy of medication (e.g., stimulant versus nonstimulant medication) or of therapies engaging specific learning mechanisms (cognitive-behavioral therapy, neurofeedback) (Figure I). Psychiatric disorders are classically diagnosed based on symptom reports and clinical observations. These clinical features are rarely diagnostic of specific underlying pathological mechanisms. Here, we propose a multilevel approach to understand psychiatric disorders and their neural underpinnings. To generate hypotheses about malfunctioning brain systems, a fine-grained dissection of a patient's behaviour is important. Once consistent behavioural signatures have been found (e.g., increased response variability), we have to bridge the gap between behaviour and the neural processes that give rise to this behaviour. At the most abstract level, we formulate the key computational issue, that is, establish what problem the brain tries to solve (e.g., an optimal balance between exploiting a good foraging ground and exploring new grounds). Here, we try to answer this question from a normative perspective. Subsequently, we have to formulate how the problem is solved. At this 'algorithmic level', reinforcement learning has been shown to be useful [65Sutton R.S. Barto A.G. Reinforcement Learning: An Introduction. MIT Press, 1998Google Scholar]. Models should fulfil several requirements: (i) a good match of model predictions with the actual behaviour of an agent; (ii) model must outperform other (more simple and more complex) models in terms of model evidence; (iii) the model should have high biological plausibility (e.g., phasic DA studies lend support to RPE reinforcement learning models). Model and parameter comparison in health versus disease can then elucidate processes that underpin impairments (e.g., decision temperature parameter driving variability in ADHD; [16Hauser T.U. et al.Role of the medial prefrontal cortex in impaired decision making in juvenile attention-deficit/hyperactivity disorder.JAMA Psychiatry. 2014; 71: 1165-1173Crossref PubMed Scopus (103) Google Scholar]). Model predictions from the algorithmic level can be used to inform data such as neuroimaging, which seeks to identify neural correlates and dynamics. By using model-derived predictions (e.g., RPEs), we can look for regions (e.g., medial prefrontal cortex) whose activity to model on the level below, thus connecting algorithmic with implementation levels. At the latter level, we can then simulate complex dynamics of neuronal systems to understand impairments. Here, we can test how problematic catecholamine systems can affect behaviour and neural activity. Thus, we can formulate new theories about neural mechanisms and potential subgroups, such as low striatal DA versus decreased frontal NA subgroups in ADHD. Using such multilevel approaches in computational psychiatry [75Montague P.R. et al.Computational psychiatry.Trends Cogn. Sci. 2012; 16: 72-80Abstract Full Text Full Text PDF PubMed Scopus (480) Google Scholar, 76Dayan P. et al.Taming the shrewdness of neural function: methodological challenges in computational psychiatry.Curr. Opin. Behav. Sci. 2015; 5: 128-132Crossref Scopus (7) Google Scholar] helps link several levels of symptom analysis (behaviour, algorithmic, and neuronal). By finding new diagnostic subgroups, we can in principle refine therapies, based on more specific predictions about the efficacy of medication (e.g., stimulant versus nonstimulant medication) or of therapies engaging specific learning mechanisms (cognitive-behavioral therapy, neurofeedback) (Figure I). To understand a psychiatric disorder, it is important to unite several levels of impairments spanning symptoms, behaviour, neural, and neurochemical markers. Here, we selectively review the most consistent neurocognitive impairments and go on to argue that these can all be explained by impaired neural gain. Behavioural findings in ADHD are numerous, and here we confine ourselves to a general pattern of ADHD-related impairments consistently present across domains and tasks. One of the most consistent findings in subjects with ADHD is an increase in reaction time (RT) variability (such as RT standard deviations). This is reliably found across many tasks, laboratories, and countries [8Castellanos F.X. Tannock R. Neuroscience of attention-deficit/hyperactivity disorder: the search for endophenotypes.Nat. Rev. Neurosci. 2002; 3: 617-628Crossref PubMed Scopus (1420) Google Scholar] and is one of the best behavioural classifiers for ADHD [9Leth-Steensen C. et al.Mean response times, variability, and skew in the responding of ADHD children: a response time distributional approach.Acta Psychol. (Amst.). 2000; 104: 167-190Crossref PubMed Scopus (456) Google Scholar]. Simple response tasks, such as the continuous performance task (CPT, Box 3), require a participant to respond to prelearned target stimuli while withholding an action for nontarget stimuli. This simple response-to-target, nonresponse-to-nontarget pattern is used in a variety of task settings that investigate different cognitive domains, such as attention (alertness, vigilance, and sustained attention tasks), response inhibition (Go/NoGo and Flanker tasks), or working memory (n-back tasks). Across all these tasks, patients with ADHD generally make less target-related responses (errors of omission) and more nontarget responses (errors of commission) [10Losier B.J. et al.Error patterns on the continuous performance test in non-medicated and medicated samples of children with and without ADHD: a meta-analytic review.J. Child Psychol. Psychiatry. 1996; 37: 971-987Crossref PubMed Scopus (354) Google Scholar]. Subsequently, we use the CPT as an example of these response biases and to illustrate how these impairments can be caused by decreased neural gain (Box 3, Box 4).Box 3How Neural Gain Affects Action Selection: The Algorithmic LevelMathematical accounts of decision making and learning allow underlying mechanisms to be formalised in precise terms. Such formulations were first introduced during the early 20th century by Hull, Thorndike, and others, and have experienced a renaissance in recent years. Models based on reinforcement learning (RL) theory [65Sutton R.S. Barto A.G. Reinforcement Learning: An Introduction. MIT Press, 1998Google Scholar] have proved to be particularly useful to describe neural processes, such as phasic DA [40Schultz W. et al.A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599Crossref PubMed Scopus (6003) Google Scholar].Reinforcement learning models often invoke two complementary modules: a valuation module that describes how values are learned or inferred from environmental cues, and a second module that describes an action selection process that explains how an agent selects between multiple choice options. It does this by taking the observation into account that humans and animals do not always choose the best option exploitatively, but select the option with a frequency proportional to its value (Herrnstein's matching law [77Herrnstein R.J. Relative and absolute strength of response as a function of frequency of reinforcement.J. Exp. Anal. Behav. 1961; 4: 267-272Crossref PubMed Scopus (1488) Google Scholar]). This is usually formulated as a softmax decision function (Equation I):pai=eQai/τ∑k=1NeQak/τ,[I] where the probability of choosing option ai is relative to the value of the alternative options Qa1…N. Importantly, the decision arbitration is modulated by a decision temperature parameter τ. This parameter moderates how deterministically the selection process follows the goodness of the choice options. In other words, the temperature τ dictates whether an agent strictly exploits the best option or whether it shows a more variable behaviour that allows selection of options with lower values. A low temperature parameter τ (Figure IA, orange) determines a high exploitative behaviour, whereas a high temperature parameter τ stands for an exploratory, variable behaviour (Figure IA, blue).The neural implementation of a decision temperature τ (or its inverse formulation: precision) has only recently started to be studied. In decision-making and planning, τ is proposed to be encoded by DA [78Friston K. et al.The anatomy of choice: dopamine and decision-making.Philos. Trans. R. Soc. Lond. B: Biol. Sci. 2014; 369: 20130481Crossref PubMed Scopus (164) Google Scholar, 79Schwartenbeck P. et al.The dopaminergic midbrain encodes the expected certainty about desired outcomes.Cereb. Cortex. 2014; 25: 3434-3445Crossref PubMed Scopus (115) Google Scholar]. More recent accounts of noradrenergic neural gain also render a likely modulator of a decision temperature [29Eldar E. et al.The effects of neural gain on attention and learning.Nat. Neurosci. 2013; 16: 1146-1153Crossref PubMed Scopus (255) Google Scholar, 74Eldar, E. et al. (in revision) Do you see the forest or the tree? Neural gain and integration during perceptual processingGoogle Scholar, 80Jepma M. Nieuwenhuis S. Pupil diameter predicts changes in the exploration-exploitation trade-off: evidence for the adaptive gain theory.J. Cogn. Neurosci. 2011; 23: 1587-1596Crossref PubMed Scopus (275) Google Scholar]. This is reasonable because high neural gain more strongly suppresses low-valued options and boosts high-valued options, rendering action selection more deterministic, whereas low neural gain dissociates less strongly between these options and facilitates selection of nonoptimal options.Here, we illustrate how an increased decision temperature can mimic ADHD variability in the CPT (Figure IB, cf supplemental information online), where subjects have to respond when an A-X-sequence appears and an increased temperature causes ADHD-like error patterns (Figure IC). Mathematical accounts of decision making and learning allow underlying mechanisms to be formalised in precise terms. Such formulations were first introduced during the early 20th century by Hull, Thorndike, and others, and have experienced a renaissance in recent years. Models based on reinforcement learning (RL) theory [65Sutton R.S. Barto A.G. Reinforcement Learning: An Introduction. MIT Press, 1998Google Scholar] have proved to be particularly useful to describe neural processes, such as phasic DA [40Schultz W. et al.A neural substrate of prediction and reward.Science. 1997; 275: 1593-1599Crossref PubMed Scopus (6003) Google Scholar]. Reinforc