Bidirectional Modulation of Goal-Directed Actions by Prefrontal Cortical Dopamine

Paul K. Hitchcott, Jennifer J. Quinn and Jane R. TaylorDepartment of Psychiatry, Division of Molecular Psychiatry,Yale University School of Medicine, New Haven, CT 06508,USAInstrumental actions are a vital cognitive asset that endows anorganism with sensitivity to the consequences of its behavior.Response--outcome feedback allows responding to be shaped inorder to maximize beneﬁcial, and minimize detrimental, outcomes.Lesionsofthemedialprefrontalcortex(mPFC)resultinbehaviorthatisinsensitivetochangesinoutcomevalueinanimalsandcompulsivebehavior in several human psychopathologies. Such insensitivity tochangesinoutcomevalueisadeﬁningcharacteristicofinstrumentalhabits: responses that are controlled by antecedent stimuli ratherthan goal expectancy. Little is known regarding the neurochemicalsubstrates mediating this sensitivity. The present experiments usedsensitivity to posttraining outcome devaluation to index the action--habit status of instrumental responding. Infusions of dopamine intothe ventral mPFC (vmPFC), but not dorsal mPFC, restored outcomesensitivity bidirectionally—decreasing responding following out-come devaluation and increasing responding when the outcomewas not devalued. This bidirectionality makes the possibility thatthese infusions nonspeciﬁcally dysregulated vmPFC dopaminetransmission unlikely. VmPFC dopamine promoted instrumentalresponding appropriate to outcome value. Reinforcer consumptiondataindicated that thiswasnotaconsequence of alteredsensitivitytothereinforceritself.WesuggestthatvmPFCdopaminereengagesattentional processes underlying goal-directed behavior.Keywords: action, attention, dopamine, habit, instrumental, prefrontalIntroductionAcquiring knowledge about the relationship between behaviorand its consequences is a vital cognitive asset that permitsresponses to be acquired and/or modiﬁed in the pursuit ofspeciﬁc goals (Balleine and Dickinson 1998). This ‘‘response--outcome association’’ is one of many encoded during instru-mental conditioning (Colwill 1994). Clearly, dysfunction of theneural substrates that encode or process this information mayresult in maladaptive behavior (Toates 1998).Instrumental responses controlled by knowledge of theirspeciﬁc consequences are termed ‘‘actions.’’ The goal-directednature of actions is reﬂected by their sensitivity to posttrainingchanges in outcome value. Thus, the performance of an actionspontaneously tracks changes in the value of its speciﬁcoutcome (Dickinson 1985). Continuous monitoring of theresponse--outcomerelationshiprequiresallocationofprefrontalcortex (PFC)-dependent, limited-capacity cognitive processes(Gehring and Knight 2000; Miller and Cohen 2001). This canbecome redundant once responding has been shaped to pro-duce the optimal behavioral outcome. Thus, extended, stableresponse--outcome conditions promote a shift in the control ofinstrumental responding from being goal directed to beingstimulus elicited. Instrumental responses triggered by environ-mental stimuli are termed ‘‘habits.’’ The stimulus--response na-ture of habits is reﬂected by their insensitivity to posttrainingchanges in outcome value (Dickinson 1985). This action--habitshift allows efﬁcient responding to be maintained while permit-ting the reallocation of cognitive resources to other demands(Gehring and Knight 2000). Conversely, the inability to shiftfrom habit to action leaves an organism unable to readily adaptto changing environmental conditions. This deﬁcit is apparentin various human psychiatric populations in whom generalcognitive functions are intact; yet, they compulsively performmaladaptive behaviors that are primarily stimulus elicited(Lhermitte 1986; Toates 1998; Montague and Berns 2002).Studies in animals indicate that medial PFC (mPFC) integrityis necessary both for the initial acquisition of task-relevantresponses and the suppression of task-irrelevant, especiallyprepotent, responses (Balleine and Dickinson 1998; Coutureauand Killcross 2003; Killcross and Coutureau 2003; Clark et al.2004; Ostlund and Balleine 2005). Whereas these studies helpdelineate the anatomical substrates mediating the balancebetween actions and habits, the neurochemical substrates thatspeciﬁcally inﬂuence this balance are not well understood.Dopamine is one likely candidate because activation of dopa-mine neurons increases during response acquisition but de-clines following extended training under stable (i.e., habitpromoting) conditions (Ljungberg et al. 1992). The preferentialcortical targets of these dopaminergic neurons are preciselythose areas implicated in outcome sensitivity, namely, theventralmPFC(vmPFC)(VanEdenetal.1987;Smileyetal.1992).Whereas striatal dopamine has been implicated in habitualinstrumental responding (Faure et al. 2005), there have been noinvestigations of mPFC dopamine modulation of actions and/orhabits. Here, we tested the hypothesis that infusions ofdopamine into the vmPFC would promote goal-directed behav-ior by increasing the sensitivity of instrumental responding tothe value of its outcome. Accordingly, we examined whether,following habit formation, vmPFC infusions of dopamine wouldpromote responding appropriate to outcome value, which wemanipulated using a devaluation procedure that we and othershave previously shown to reduce the performance of aninstrumental action (e.g., Dickinson et al. 2002; Miles et al.2003; Hitchcott et al. 2005; Quinn et al. 2006). Because thevmPFC is implicated in the production of adaptive behavior(e.g., Dalley et al. 2004), and dopamine has been suggested toexertapermissiveeffectonthisfunction(Hollermanetal.2000;Sullivan 2004; Fletcher et al. 2005; Floresco et al. 2006), wehypothesized that dopamine infusions would promote adaptivebehavioral adjustments appropriate to current outcome value