Revealing abrupt transitions from goal-directed to habitual behavior

Domain: biology
Published: 2026-03-31
Editorial outcome: minor_revision

Abstract
The speed of goal-directed to habit transitions has been debated since Clark Hull asked in 1943: is habit formation slow or sudden? To address this, male mice were given home-cage access to citric-acid water that reduced—without eliminating—reward-seeking for plain water in an auditory go/no-go task. Animals learned to discriminate quickly but exhibited ongoing state-like fluctuations in engagement. Strikingly, these fluctuations abruptly ceased (transition) long after discrimination stabilized, with HMM-GLM modeling pinpointing a ~3 trial transition. We confirmed this as a goal-directed to habit transition using sensory-specific outcome devaluation, DLS lesions, motor stereotypy, and pupillary responses. Dual-site fiber photometry showed equally abrupt DLS dynamics at the transition: outcome-related activity dropped and stimulus-response activity sharpened, suggesting a switch-like mechanism that recruits a readily available habit circuit rather than gradual changes across a threshold. Thus, habits can emerge suddenly, mediated by an abrupt dorsostriatal shift from outcome- to stimulus-driven processing.

Introduction
Humans and other animals are often thought to be creatures of habit. When driving, for example, we are initially told that the color of a traffic light should guide our actions: green to go and red to stop. Through practice, we learn purposefully and are driven by the conscious goal to follow the rules of the road. Over time, these rules become automatic; without deliberation, we will push the gas pedal on a green light and the brake pedal for a red light. More specifically, an initial goal-directed action (R) in response to a cue (S) yields a desired outcome (O) which then slowly evolves into a habit where the cue elicits the action (S-R) without necessarily having the goal in mind1,2. The automatization of decisions can be thought of as an efficient way to offload well-learned contingencies to free up resources for more flexible, goal-directed learning3. The expression and perseverance of habits, however, can also be maladaptive with neural circuits being co-opted in substance use disorders or compulsive behaviors4–6. Understanding the exact time course of habit development is critical to disentangling its neural basis and could help inform future interventional strategies for combating habit-related disorders.

An assumption of a slow, gradual shift from goal-directed to habitual control underpins most models of learning and informs relevant approaches to understanding the neurobiological basis of habitual behavior7–14. To date, however, the nature, timing, and properties of the transition in decision control has been challenging to pinpoint due to methodological constraints. In rodents, studies of habits often use distinct reinforcement schedules to bias goal-directed, or habitual behavior15. The gold standard, however, for assessing whether a behavior is under goal-directed or habitual control at a specific time point exploits the observation that goal-directed actions are sensitive to the outcome16,17 (e.g., animals will only perform an action when the reward is desired) while habitual behavior is less sensitive to the outcome (e.g., animals will continue to perform said action even if the reward is not explicitly desired). This behavioral characterization relies on defining habitual behavior as the loss or absence of goal-directed control18,19. Sensitivity to the outcome has been successfully operationalized in laboratory testing with outcome devaluation4,20 procedures in which a specific reward is devalued (through satiety or taste aversion). Outcome devaluation, or a related alternative called contingency degradation, is typically implemented at set time points outside of the normal training regimen (e.g., the middle and end of a multi-day training period). To date, no approach exists to disambiguate between goal-directed and habitual control in real-time, during training21,22

Although powerful, methods such as outcome devaluation and contingency degradation inherently limit assessing the nature, timing, and properties of the transition between goal-directed and habitual control in individual animals due to the discrete test sessions and cohort-level comparisons. Can we behaviorally identify habitual transitions in real-time and during training? Is the transition slow or sudden? What are the characteristics of these transitions in individual animals? What are the neural mechanisms that relate to the transition in real-time? Addressing these questions requires a new behavioral approach that assesses the decision mode en passant, without discrete test sessions, without explicitly biasing behavior to one or the other process, and within individual animals.

Here, we present such an approach. We reasoned we could reduce—but not eliminate—reward-seeking for plain water droplets in an auditory cued go/no-go task by allowing animals free access to a citric acid (CA) water in the home-cage. CA water largely fulfills hydration needs, but mice still show a strong preference for plain water when it becomes available. Thus, we reasoned that we could infer behavior as ‘goal-directed’ if engagement were to be weaker or fluctuate during a task that provides plain water (based on the underlying desire for plain water); in contrast, under habitual control (in which animals are less sensitive to the outcome), the S-R nature of the behavior would drive high and stable engagement despite ongoing changes in the underlying desire for the outcome.

Results
Free access to CA water reduces reward-seeking for plain water
We gave mice ad libitum access in the home cage to water laced with citric acid (CA) (Fig. 1a, CA yellow) which makes it slightly acidic to the taste. Mice continue to drink to provide ongoing hydration but gain less weight than mice given ad libitum access to plain water23,24. We refer to these mice as self-restricted as they individually balance their hydration needs against their mild distaste for CA water. Before instrumental training, self-restricted mice lost significantly less weight than mice on a standard water restriction (WR) protocol (Fig. 1b; WR85 = 17.8%±2.3%, CA = 8.9%±1.9% average and std weight loss respectively, p=0.000055, Wilcoxon rank sum test). This difference was maintained throughout training (Supplementary Fig. 1a) (Wilcoxon rank sum test, p=1.0x10-8), while no differences were observed in the animals’ initial weight (Supplementary Fig. 1b; Wilcoxon rank sum test, p=0.98). Before mice begin discriminative auditory training (Fig. 1a), they first experience 2 days of instrumental training in which they learn to make an un-cued, self-paced instrumental lick to subsequently receive a small reward (lick training, 3 µl plain water droplet) (Fig. 1a; lavender). Self-restricted CA mice obtained significantly fewer rewards compared to WR mice during these sessions (Fig. 1c; Wilcoxon rank sum test, p=0.00079). We also assessed licking patterns as a proxy of reward-seeking. Self-restricted mice executed fewer licks per session (Supplementary Fig. 1c-1e; yellow; Wilcoxon rank sum test, p=0.021), and exhibited different lick patterns (Supplementary Fig. 1c-d), while maintaining the same lick frequency when engaged in licking (Supplementary Fig. 1f) (Wilcoxon rank sum test, p=0.67). This suggests that self-restricted CA mice exhibit reduced reward-seeking for plain water despite similar consummatory behavior during active licking.

Abrupt transition in engagement may reflect a change in decision control
Mice were then trained on a discriminative auditory go/no-go task in which they learned to lick to one tone (S+, rewarded) for a water reward (hit) and withhold licking to another tone (S-, non-rewarded) to avoid a timeout (Fig.1d; correct reject). Self-restricted CA mice exhibited lower response rates to the S+, consistent with reduced levels of reward-seeking, but surprisingly only minor differences in discrimination performance throughout learning. Specifically, when restricting performance assessment to blocks of high engagement (>50 trials with >90% hit rate), task performance during learning was similar to WR mice (Fig. 1e and 1f). In addition, self-restricted and WR mice exhibited similar, high discrimination performance (75%) within 1,500 trials (WR=1132 ± 87 and CA=1422 ± 234 trials, Wilcoxon rank-sum test p=0.93).

We next sought to explore in detail the impact of reduced reward-seeking on task engagement. To do that, we assessed changes in response-rate to the S+. These changes could be driven by a continuously lower or, alternatively, a fluctuating response rate. In WR mice, animals initially increase their action rates to both tones, followed by a slow reduction in response to the S- (Fig. 2a; left example animal). The S+ response (hit rate) stayed consistently high with minimal variability. We observed a striking contrast in self-restricted CA mice (Fig. 2a; right), where for thousands of trials, they exhibit a fluctuating hit rate, regularly shifting from epochs of high engagement (high hit rate) to low engagement (low hit rate), suggesting that mice are intermittently engaging in the task. We thus focused on the hit rate as the response rate of interest (Fig. 2b). Self-restricted CA mice showed significantly fewer blocks of high engagement (hit rate > 90%) compared to WR mice (Fig. 2c; Wilcoxon rank sum test, p=0.0028).

Surprisingly, after this high-variability phase, most self-restricted CA mice abruptly transitioned to a low-variability phase (Fig. 2d; red line=transition), an effect rarely seen in WR85 mice (Supplementary Fig. 2a). We observed that 10 out of 12 (83.3%) of the CA mice exhibited high hit rate variability, of which 8 out of 10 (80%) transitioned to a low-variability phase (Supplementary Fig. 2b). Transitions occurred more than 1,000 trials after animals exhibited expert discrimination performance, suggesting this transition was not due to ongoing contingency learning (d’>2 expert=1436±454 trials vs. transition=3832±385 trials) (Supplementary Fig. 2c). To confirm that this change in hit rate was not due to a sudden change in underlying motivation, we measured the weights of the animals daily. Importantly, we observed (1) no differences in discrimination performance around the transition (paired t-test, p=0.69) (Supplementary Fig. 2d) (2) no evidence of changes in weight pre- versus post-transition (Supplementary Fig. 2e) (paired t-test, p=0.20), and (3) no changes in consumption in the home cage based on measurements of post-task and next-day weights (Supplementary Fig. 2f) (paired t-test, p=0.087). This suggests that self-restricted CA mice do not exhibit increased baseline motivation post-transition and, instead, points to a shift in decision control, possibly one from goal-directed to habitual.

Our analysis thus far required categorization of behavior based on pre-defined criteria (low versus high variability, pre- versus post-transition) and experimenter-defined parameters. We sought to test whether a bottom-up, model-based approach could identify behavioral states in an unbiased, and trial-by-trial, manner. To do this, we applied a generalized linear model that incorporates a hidden Markov process (HMM-GLM)25 on trial-by-trial behavioral data after animals reached expert discrimination performance (Fig. 2e). The HMM-GLM identified two states that best described the behavior in expert animals, as defined by the lowest cross-validated Bayesian Information Criterion value (BIC) (Fig. 2f-g). Both states were sensitive to the stimulus but with distinct action biases. State 1 (pink) exhibited high engagement, evident by a positive bias and high hit rate (i.e. action rate on target trials), while State 2 (dark green) exhibited strong disengagement, evidenced by a strongly negative bias and low hit rate (Fig. 2g-2j). The HMM-GLM accurately recapitulated the behavioral data (Fig. 2i and Supplementary Fig. 3e; solid line) but also allowed us to resolve state-like transitions in behavior that are obscured when using temporally-binned behavioral data (Supplementary Figure 3a-d). Interestingly, before the transition, self-restricted CA mice regularly switched between the two states both within and across sessions. After the transition, however, State 1 (engaged) dominated behavioral performance (Fig. 2i -2l and Supplementary Fig. 3e). We then used the HMM-GLM model to predict the transition in a bottom-up manner which we found to be similar to the behaviorally predicted one while providing greater temporal specificity (Supplementary Fig. 3f) (Wilcoxon rank sum test, p=0.82). The model-defined shifts from engaged to disengaged states before the transition were strikingly abrupt (Fig. 2k). Moreover, the last transition, reflecting the putative transition between goal-directed and habitual behavior, was as abrupt as the earlier ones, occurring in ~3 trials (Fig. 2l; last). Consistent with this interpretation, hit rate declined across a session immediately pre-transition, but remained stable post-transition (Supplementary Fig. 4a), suggesting that post-transition responding is less sensitive to within-session factors such as satiety. Together, both quantification of behavioral data and model-based approaches using the HMM-GLM converge on the abruptness of the identified transition thereby motivating a more direct test of decision control.

Sensory-specific outcome devaluation validates transition to habit
To directly test whether the observed transitions in response stability relate to the underlying psychological constructs of goal-directed and habit, we employed sensory-specific outcome devaluation—a gold standard for differentiating goal-directed and habitual control of behavior. A behavior is considered habitual when animals continue to perform an action even when the outcome is devalued via pre-task satiation15,16,26,27. This sensitivity to devaluation should be specific to the task-associated reward so as to differentiate general motivation (for water or food) from outcome-specific value3,28–30. We conducted sensory-specific outcome devaluation tests in the same animals at two discrete time points during behavioral training, one pre-transition and the other post-transition (Fig. 3a). Vanilla-sucralose flavored water was used as the valued condition while plain water was used as the devalued condition. Animals received free access to either reward before a given test session and were put into the auditory go/no-go task under extinction.

Pre-transition, animals had lower hit rates (normalized, see Methods) in the devalued condition compared to the valued condition, indicating that behavior is under goal-directed control (Fig. 3b). Post-transition, this devaluation effect was abolished, with animals now having similar hit rates in both the valued and devalued conditions. Moreover, the devaluation index (see Methods) of the animals decreased significantly across the transition (Fig. 3c). These results demonstrate that the transitions in response reliability in our task directly relate to the nature of decision control, namely a transition from goal-directed to habit.

Bilateral lesions to the DLS prevent transitions to habit
The two decision processes (goal-directed and habitual) are thought to be sub-served by distinct neural circuits in the dorsal striatum31. The dorsomedial striatum (DMS) and dorsolateral striatum (DLS) are thought to enable goal-directed and habitual behavior, respectively32,33. Using standard outcome devaluation procedures, rodents with lesions to the DLS persist in goal-directed mode (i.e. when they should be sensitive to reward devaluation) even after significant amounts of overtraining34. Thus, DLS lesions provide a powerful and orthogonal approach to further test the validity of this paradigm in assessing the underlying decision mode. To do this, we bilaterally lesioned the DLS in self-restricted CA mice (NMDA 20µg/µl, 100nl/site) before head post implantation and behavioral training (Fig. 3d). All DLS-lesioned self-restricted CA mice had visible, localized, and overlapping lesions (Fig. 3e-f, and Supplementary Fig. 4d), while shams did not (Supplementary Fig. 4e-f). DLS-lesioned self-restricted CA mice exhibited high variability in hit rates that persisted for much longer than in sham CA mice and rarely transitioned to low variability (Fig. 3g-3i and Supplementary Fig. 4g) (60% sham vs 20% lesioned).

Importantly, self-restricted CA mice with DLS lesions exhibited no significant deficits in the learning of the task contingencies or in lick-related motor behavior (Supplementary Fig. 4h-k), ruling out the possibility that the observed effect was driven by motor refinement or differences in task efficiency, two functions also ascribed to the DLS. These data offer independent evidence, through the manipulation of habit-relevant striatal circuits, that the transitions we observe are indeed genuine transitions from goal-directed to habitual behavior.

Licking microstructures demonstrate motor automaticity post-transition
In skill-based learning, motor patterns of ‘automaticity’ can be used as evidence for the expression of habits35. We analyzed lick microstructures in detail to determine the extent to which transitions in hit-rate variability were concomitant with changes in motor automaticity. Before transitions, self-restricted CA mice exhibited highly variable lick microstructures (Fig. 4a; top). Post-transition, however, three aspects of their licking behavior abruptly appear: a uniform lick stereotypy (Fig. 4a; Post), an increase in consummatory licks (Fig. 4b-c), and a reduction in reaction time (Fig. 4d). These patterns were consistent across trials and sessions after habit transitions demonstrating the simultaneous appearance of motor automaticity.

Pupillometry as biomarker for decision control
Pupillary response dynamics are thought to reflect multiple different features, including sensory cues36,37, arousal/task engagement38,39, motor activity40,41, and motivation42,43. The extent to which pupil dynamics can provide a biomarker for decision control, however, remains unexplored. We hypothesized that pupil dynamics may track outcome sensitivity and thus performed a detailed inspection of trial-level, phasic pupillary dynamics across behavioral transitions.

We focused our analysis on hit trials in which the animals consumed the reward (rewarded) and those in which they made the instrumental lick to the S+ correctly but did not consume the reward (no-reward). Pre-transition, no-reward trials elicited lower pupil response than the rewarded trials (Fig. 4e-f; Pre-NR vs Pre-R), suggesting a differential coding of the absence/presence of reward. This difference was significantly reduced post-transition, and more strikingly, the post-transition no-reward trials elicited significantly higher pupillary responses than pre-transition (Fig. 4e-f; Post-NR vs Pre-NR) with noticeable effects occurring within 10 trials of the transition (Fig. 4g).

The observed pupillary dynamics could reflect a loss of outcome sensitivity after transitions to habit or could be driven by transition-related differences in task engagement33,34, motor activity, or motivation (but not sensory cues, since all analysis is done on S+ trials). First, we excluded all disengaged trials from our analysis, given that pupil responses were lower on disengaged versus engaged trials (see Methods, Supplementary Fig. 5b). Second, we focused our analysis on no-reward trials when licking ceased after the first instrumental lick since animals exhibit higher lick numbers post-transition (Supplementary Fig. 5d and Fig. 4c). In addition, we observed that the pupil response had little correlation to lick number per trial (Supplementary Fig. 5e). Third, within-session reduction in motivation due to satiety led to fewer licks per trial (Supplementary Fig. 5f), but had no effect on the phasic pupillary response, both pre- and post-transition, for both rewarded and no-reward trials (Supplementary Fig. 5f and 5g). Given that these key factors were ruled out, we then sought to further test whether the pupil response we observed was related to outcome sensitivity by exploiting experimenter-triggered reward omission trials (versus no-reward trials when animals occasionally did not consume the reward). On these trials, animals correctly respond to the S+, continued to lick, but a reward was not delivered. The pupil response on omission trials shared the same feature as no-reward trials: the signal increased across the transition and became closer to that of rewarded trials (Supplementary Fig. 5c). These combined data demonstrate that pupillary dynamics are less outcome sensitive immediately after habit transitions, suggesting that phasic pupillary responses can provide a useful biomarker of decision control.

Abrupt reductions in outcome-related signaling in the dorsal striatum mark the moment of a transition
The behavioral identification of abrupt transitions to habit, and the critical involvement of the DLS, led us to next focus on how neural activity in the dorsal striatum evolves across transitions. In prior work, it has been challenging to observe neural activity changes in real-time as animals initially learn in a goal-directed manner and then transition to habit. Our behavioral findings could be explained by a gradual model – whereby neural activity related to habitual behavior slowly ramps up and takes control of behavior only after reaching a threshold. This could lead to the behavioral appearance of an abrupt shift, despite the gradual change in the underlying circuit activity, and would suggest that habits arise through the gradual strengthening of a habitual controller. Alternatively, neural activity could experience a switch-like change in activity at the moment of the transition. This would suggest that a habit controller is readily available but only comes online when recruited at a precise moment in time. To test between these two plausible models, we performed dual fiber photometry in the DLS and DMS (Fig. 5a) as animals transitioned from goal-directed to habitual behavior.

We first assessed the within-trial dynamics of the DLS and DMS by focusing on one hundred trials around the behaviorally-identified transition (Fig. 5b). To minimize effects driven purely by changes in general motivation in engaged versus disengaged blocks of trials (Supplementary Fig. 6a; see Methods for classification), we selected only engaged trials for appropriate comparison across the transition. As has been described previously44–46, we observed both early-in-trial (cue-response related) and late-in-trial (outcome-related) activity patterns (Supplementary Fig. 6b-c). When aligning neural data to the transition, we observed a striking reduction in the late-in-trial activity in both the DLS and DMS post-transition (Fig. 5c), with a more pronounced reduction in the DLS (Fig. 5; top, and Supplementary Fig. 6l-m). Accompanying this reduction was a sharpening of the early-in-trial signal in the DLS (Fig. 5e and 5f). These data suggest that post-transition, the DLS, and to a lesser extent the DMS, shows reduced outcome-related activity and more stereotyped responses to the cue itself, consistent with a stimulus-response mode of control. Importantly, these differences were locked to the transition and were not present when average activity was computed on random trials from the transition day (Supplementary Fig. 6n), nor when activity was averaged around the same within-session transition trial, but 3 days before the true transition day (Supplementary Fig. 6o).

These findings suggest that both the DLS and DMS exhibit stable patterns of activity prior to the behavioral shift, indicating that neural signatures of habitual control may emerge prior to their behavioral manifestation. To assess that possibility, we sought to understand when the DMS and DLS encoded the different trial types (hit, miss, false alarm, correct reject) versus the individual task events (cue, action, outcome). At the outset of learning (i.e. the first 20 trials), we observed no evidence of contingency-specific activity, with activity in the DLS and DMS being dominated by individual task events (e.g. cues, licking vs no licking) (Supplementary Fig. 6b, and Supplementary Fig. 6d; Pre-Acq.). When animals became expert at the discriminative task, but hundreds to thousands of trials before the transition (pre-transition), we observed robust contingency-specific activity that then remained stable until the moment of the transition (Supplementary Fig. 6c-d; Pre-transition, and Supplementary Fig. 6e; Pre-transition early vs. late).

Finally, we sought to determine how precise the within-trial dynamics in the DLS and DMS tracked the behaviorally-identified transition. We performed a trial-by-trial analysis after aligning all animals to the transition. The DLS exhibited an abrupt drop in late-in-trial activity at the moment of a behaviorally-identified transition (Fig. 5g, top, five trials pre- and post-transition and Fig. 5h, bottom, black, 50 trials pre- and post-transition), with a similar drop in DMS late-in-trial activity occurring over the next several trials (Fig. 5g; bottom, and Fig. 5h; bottom, gray). One possibility is that this change could be driven by changes in lick structure (observed post-transition, Fig. 4a-d, and Supplementary Fig. 6f-g), and not a change in decision control. We found no relationship between consummatory licks and late-in-trial signal changes (Supplementary Fig. 6i), highlighting the independence of these signals from motor automaticity linked to habit. Even when restricting our analysis to trials with the same response latency pre- and post-transition, we continued to observe this robust drop in outcome-related activity (Supplementary Fig. 6m). To further test whether these signals reflected outcome-related computations, we compared the amplitude of the late-in-trial signal in no-reward pre-transition trials (i.e. trials where the animal executes an operant lick but no consummatory licks, Pre-NR, gray), with rewarded post-transition trials (Post-R, red) in both DLS and DMS. Despite the completely distinct lick structures, these two signals were indistinguishable (Supplementary Fig. 6j-k). Moreover, this similarity suggests that the dorsal striatum becomes less sensitive to the presence of the outcome when behavior is habitual. Taken together, these neural data show that transitions to habit are indeed abrupt, occur at trial-level resolution, and are governed by a switch-like change in outcome-related signaling in the dorsal striatum.

Methods
Animals
All mice were housed in standard plastic cages with 1-4 littermates and kept in a 12-h/12-h light/dark cycle (10:30 am / 10:30 pm) with controlled temperature (19.5-22°C) and humidity (35-38%). All the mice used in this study were male C57BL/6J from Jackson Labs (strain# 000664) and were between 11 to 15 weeks of age at the start of training. All the experimental and surgical procedures were approved and performed in accordance with the Johns Hopkins University IACUC protocol (license # MO20A272). Sample sizes were determined based on standard cohort sizes from relevant literature. Mouse allocation to specific groups was randomized but the experimenters were not blinded to group types.

Surgical procedures
Mice were anesthetized with isoflurane (5.0% at induction, 1.5-2.5% during surgery) and placed on a stereotactic apparatus (Kopf). The hair over their skull was removed with hair removal cream and the area disinfected with betadine. The skin over the skull was removed and the area was cleaned of connective tissue with 3% H2O2. A custom-made stainless-steel head-post was fixed onto the exposed skull with C&B Metabond dental cement (Parkell). The animals were given 1-3 days to recover.

Striatal lesions
Mice that underwent bilateral DLS excitotoxic lesions or sham injections, received N-Methyl-D-aspartic acid (NMDA; Sigma Aldrich, 20µg/µl NMDA in PBS1x with 10% glycerol) or vehicle (PBS1x with 10% glycerol) respectively (with a Hamilton syringe and a Harvard Apparatus Pump 11 Elite, 100nL/injection-site at a 70nL/min). The injections were made at AP+1.0mm, DV±2.6mm, ML-2.8mm via burr holes which were sealed with Jet Denture Repair Acrylic (Lang Dental) prior to headpost implantation.

Fiber photometry
The animals were injected with AAV-syn-jGCaMP8m-WPRE, (≥1×10¹³ vg/mL, 1:10 diluted in saline), Addgene Cat #162375-AAV9) in one hemisphere targeting DLS and DMS in the other. The injections were made at AP+1.0mm, DV±2.6mm, ML-2.8mm for DLS and AP+1.0mm, DV±1.5mm, ML-2.8mm for DMS via burr holes, which were sealed with Jet Denture Repair Acrylic (Lang Dental) prior to skin closure with suture (5-0 coated vicryl plus undyed 1X27" RB-2, Ethicon #VCP433H). After 2 weeks of expression, animals underwent fiber implantation surgeries . The surgical site was reopened, and optic fibers (Φ 1.25 mm Ceramic Ferrule, 200um Core, 0.39NA, length = 3mm, RWD # R-FOC-BL200C-39NA) were implanted in the burr holes and secured with dental cement, followed by headpost implantation. Habituation began 7-10 days after the animals regained their original weights.

Habituation and water restriction paradigms
After recovery from surgery, animals were handled and habituated prior to the start of training for at least 10 days based on previous studies82. Head-fixed experimental CA mice and their littermate controls (WR85) underwent the same surgical, habituation and testing procedure. Animals were handled by the experimenter/s at increasing times every day, exposed, and habituated to the head fixation station. The different water restriction paradigms started after at least 3 days of handling. The standard water restriction (WR85) protocol prevented the animals from accessing water in their home cage. The mice were weighed daily, and a limited amount of water (~1.0 mL) was given individually to maintain 80-85% of their original weight. For naturalistic water restriction with citric acid (CA), animals had ad libitum access in their home cage to a bottle of tap water with citric acid dissolved following reported protocols23,24. The mice were slowly introduced to the taste of CA, increasing its concentration daily from 0.5% CA to 1-3%, and adjusted accordingly within this range to keep the animals at ~95% of their original weights.

Behavioral training
All behavioral training was done using Bpod State Machines (r1 or r2, Sanworks). After habituation, lick training was performed for two days in which animals had to lick in order to receive a small water droplet (un-cued). Immediately after water delivery, licks would not be rewarded until 2 seconds had passed. The lick training session ended either when 1 ml of water was consumed, or the session had reached 30 minutes. On a subsequent session, mice began training on a go/no-go auditory task. Behavioral events (trial structure, stimulus and reward delivery, lick detection) were controlled and stored using a custom-written MATLAB program (2018b, The MathWorks) interfacing with the Bpod, an electrostatic speaker driver (E1, TDT) and an infrared beam for lick detection. In a subset of animals, facial movements and pupil size were measured with either: 1) a Raspberry Pi (3B) and a Raspberry Pi camera module (NoIR v2) coupled with a Bright-Pi infrared LED array (PiSupply); or 2) an Arducam 100fps Global Shutter USB Camera and a custom-made infrared LED array. Mice were head-fixed inside a Plexiglass tube facing a lick-tube. A free field electrostatic speaker (ES1, TDT) was located ~5 cm from the animal’s left ear and each sound (either 4757 Hz or 8000 Hz, as target or foil stimuli) was calibrated to an intensity of 60-62 dB (SPL). The pupil camera and IR LED array were positioned ~6 cm away from the animal’s face in a 60-degree angle. Everything was enclosed in a custom-made sound-attenuated box. Target and foil tones were pseudo-randomly ordered (equilibrated every 20 trials). Each trial consisted of a pre-stimulus no-lick period (2 s), stimulus presentation (100 ms), delay (100 ms), response period (2 s) and variable inter-trial interval (ITI). Typically, mice were trained for ~300-320 trials per with a short block of 20 non-reinforced trials interleaved in the middle of the session83,84. Training lasted for a maximum of 30 days.

Sensory-specific Reward Devaluation
A cohort of 8 animals underwent the same citric acid protocol as previously described. Individual-animal action rates were measured in blocks of 50 trials to obtain hit and false-alarm rates in small blocks. A transition was defined by a criterion of >1600 trials of stable action rate and low action variability. We adapted existing protocols for food-driven tasks29,85,86 in mice to suit our task structure with thirst as the motivator87. Tap water (task reward) was the devalued condition, whereas a vanilla-sucralose solution (2.5% vanilla, 0.4% sucralose) was used as the valued condition. Sucralose does not provide caloric intake but is detectable by the animals as a distinguishable flavor88. Vanilla was added to further increase the salience of flavors between the two tastants. Animals were pre-exposed to the vanilla-sucralose solution for 1h during habituation to avoid neophobia. Sensory-specific reward devaluation tests were performed on days 7 (pre-transition) and 23 (post-transition) for 6 animals. One animal reached transition criteria on day 32; thus, it received the post-transition reward devaluation (RD) test on day 33. On valued and devalued testing days, animals received ad libitum access to either solution in each day: water (devalued condition) or vanilla-sucralose solution (valued condition) for 10 minutes prior to a non-rewarded devaluation session (sometimes referred to in the literature as devaluation under extinction). The two devaluation sessions were performed on consecutive days, and the order in which the animals received valued and devalued conditions was counterbalanced. Access to the solution was provided in a separate cage for each animal, which was immediately transferred to the behavioral setup for testing. The weight difference before and after access to the reward was used to calculate the consumption amount (mL). During devaluation tests, animals received 40 trials in total (20 S+ and 20 S-), and no reward was provided for correct licking to the S+. The raw hit rate was calculated as the percentage of responded S+ trials over total S+ trials in a given devaluation session. The trial structure was otherwise identical to training trials. Animals (n = 1) that did not show a habitual transition by day 32 were excluded from the analysis. All other animals (n = 7) were used for the final analysis.

Fiber photometry recordings
Mice underwent the same citric acid training protocol as previously described. Additionally, during each training session, fluorescence signals were recorded in both DLS and DMS for the same animals, using a commercial fiber photometry setup (Neurophotometrics, FP3002), coupled to the Bpod (Sanworks) and controlled via customized Bonsai workflow89. Pairs of animals were recorded simultaneously, using a 4x branching patch cord (length = 3m, NA = 0.37, 200 µm/1.25 mm black ceramic FOC, FC connection, mbf Bioscience #NPM-BPC-4). LEDs delivered two excitation wavelengths (470 nm for GCaMP6s, 415 nm for isosbestic) interleaved at 50 Hz. LED power was calibrated to an intensity of 50 µW at the tip of the patch cord for each channel. Fluorescence emission was collected with a CMOS sensor, with a region of interest defined around the end of each patch cord for every recording session. The patch cord was bleached using maximum intensity of both LEDs, for ~8-12 h before the start of every cohort. Photometry data was acquired with Bonsai and subsequently exported to MATLAB for analysis. All signals were aligned to analog TTL pulses generated by the Bpod, marking behavioral events.

Histology
After every experiment, the brains of all animals were obtained via transcardiac perfusion90 and stored in 4% paraformaldehyde solution in PBS1x overnight. After further dehydration in 30% sucrose (Sigma-Aldrich), the brains were frozen in OCT gel (Tissue-Tek®) and sliced using a cryostat (Leica) into 50um slices.

Striatal lesions
The slices were mounted on gelatin-coated slides (FD Neuro) and left at room temperature to dry overnight. The following day, the slides were stained using 1% Cresyl Violet acetate (Sigma-Aldrich #C5042) solution and cover glasses were placed and fixated with Permount™ Mounting Medium (Fisher Chemical™ #SP15). The slides were imaged under Brightfield settings in a Zeiss upright microscope (Axio Zoom.V16).

Fiber photometry
The slices were washed with 1XPBS at RT on shaker (30 rpm) for 3 times, 5min each. Permeabilization was done with 0.2% PBS-Triton (TritonTM X-100, Sigma-Aldrich #X100-500ML) at RT on shaker for 1h. Blocking was done with 5% Normal Donkey Serum (Sigma-Aldrich # D9663) in 0.2% PBST at RT on shaker for 1h. The slides were stored in primary antibody (Goat Anti-GFP IgG, 1:1000 dilution in blocking solution, Novus Biologicals #NB100-1770SS) at 4°C overnight. The next day, the slices were washed with 0.2% PBST at RT on shaker for 3 times, 5min each, then stained with secondary antibody (Alexa Fluor® 488 AffiniPure™ Donkey Anti-Goat IgG, 1:500 dilution in 0.2% PBST, Jackson Immuno #705-545-003, ex488nm/em496nm) at RT on shaker for 2h. The slices were washed with 1XPBS at RT on a shaker for 3 times, 5min each, before mounted on microscope slides (Premium Superfrost® Plus Microscope Slides, VWR # 48311-703) with DAPI Fluoromount-G® (SouthernBiotech #0100-20, ex405nm/em465nm), and imaged with Zeiss LSM 700 confocal microscope under 10X magnification.

Statistical analysis
All analyses were performed using custom-written MATLAB code (The MathWorks, 2019b, 2021a, or 2022a) or R environment. All datasets were tested for normality using a one-sample Shapiro-Wilk test; then, parametric or non-parametric statistical tests were applied accordingly. Two-sample (paired or unpaired) t-tests were used for parametric data, and Wilcoxon rank sum tests were used for non-parametric data. Where required, paired comparisons were made. For multiple comparison analyses, (1-way, 2-way, repeated measure) ANOVA or ANCOVA was performed for parametric data, and permutation-based ANOVA for non-parametric data. Post-hoc pairwise comparison was conducted with Bonferroni corrections or LSD test. To build a Receiver Operating Characteristic (ROC curve) (Fig. 3i) we used the transition probability of lesioned and sham animals to obtain the area under the curve (AUC) and generate a shuffled probability distribution to statistically test our experimental animals' distribution difference. Significance was determined as the difference in AUC value between lesioned and sham animals when it fell beyond the 95th percentile confidence interval (Fig. 3j). Except for one-sided paired t-test which uses two-sided 90% confidence interval, all confidence intervals correspond to α=0.05. Significance is represented as n.s. p>0.05, * p≤0.05, ** p≤0.01, *** p≤0.001, and **** p≤0.0001. Effect size was calculated as ηp2 for F-based ANOVA, partial R2 for Χ2-based ANOVA, Cohen’s d for unpaired t-test and Wilcoxon rank-sum test, Cohen’s d for paired t-test. In most cases of Wilcoxon signed-rank z and rank-sum tests, z values were reported when possible (otherwise U or W was reported).

Behavioral analysis
The first instrumental lick to the target tone to trigger target delivery was used to determine whether the trial is a hit or miss. The subsequent licks after reward delivery were quantified as consummatory licks. The action rate was calculated as the percentage of hit trials among all target trials. Individual-animal action rates were measured in blocks of 50 trials to obtain hit and false-alarm rates in discrete but small blocks that allowed us to observe behavioral variability. Behavioral discriminability was calculated using the z-scored hit rate minus the z-scored false-alarm rate (d’). To avoid infinite values when rates of 1 or 0 are present, the values were corrected by 1-1/2N or 1/2N respectively, where N corresponds to the number of trials. For all the data presented in this paper, we considered animals to have effectively learned the task by calculating d’ during non-reinforced trial blocks, which has previously been demonstrated as an accurate measure of task acquisition83. Only mice with a d’ > 2 during these non-reinforced trial blocks were included in the analysis (corresponding to 45 out of 48 mice tested, 1 WR85, 1 CA-sham and 1 CA-lesioned mice did not learn the task and thus were excluded).

HMM-GLM model implementation
We fit a GLM-HMM model to trial-by-trial choices of each mouse from 4 days before putative habitual transitions to 4 days after the putative transition (9 days in total). Each state in HMM contains a Bernoulli GLM defined by a weight vector specifying how stimulus inputs and bias are integrated in that state. The model was fit using a previously published expectation-maximization (EM) algorithm25. To identify the optimal number of states, we evaluated the cross-validated BIC by fitting choice data from the 5th day before and after habitual transition. A 2-state model was sufficient to explain the choice behavior of six animals, capturing an engaged state and a disengaged state, whereas a 3-state model was needed for two animals, capturing an additional low-discrimination state. For these two animals, we focused only on the engaged and the disengaged state in subsequent analysis. To compute state occupancy, we first inferred the behaviorally dominant state as the state with the highest probability in each trial, and then calculated the percentage of trials that a state is dominant in a 50-trial bin. We inferred the habitual transition by identifying the last trial bin where the occupancy of disengaged state was above a threshold of 30%. The number of trials needed for transitions between engaged and disengaged state is calculated by the number of trials needed for the dominant state to reach 75% probability after the transition. To compare the model-inferred transitions with behavioral data, we quantified the slope of inferred state probability by the GLM (z-scored) at the trial of state transition, compared to the slope of hit rate changes during state transitions (z-scored), quantified using various bin sizes around the transition.

Preprocessing of pupillometry data
Pupillometry videos of 30-40 minutes long (n=8) were taken as the training dataset for a DeepLabCut91,92 (DLC) pre-trained model (resnet_50). Manual labeling of pupil contour consisted of 8 points (up, down, left, right, up-left, up-right, down-left, down-right) across 180 randomly selected frames. The network was trained for >200,000 iterations until the loss rate plateaued. The final network was used to analyze the pupillometry videos from the experiment. Custom MATLAB code (The MathWorks, 2021a) was then used to remove blink artifact, reconstruct pupil diameter, and apply a low pass filter (3 Hz) to the data. Individual trials for individual animals were normalized to the first frame of stimulus onset.

For behavioral data, 10-trial windows were taken around each hit trial (5 before + 5 after), and only those with window hit rate higher than 0.75 were selected as engaged trials. Trials in which animals make only 1 lick to the target stimulus with no subsequent consummatory licks were classified as non-consumed hit trials. Non-rewarded trials were introduced in 20 trial blocks (10 targets and 10 foils) on random days, and hits to target tones served as omission trials. All trials are within 3 days pre-and post-transition. Numbers of data points were matched across groups via taking averages of randomly selected trials in the larger groups.

Fiber photometry recording analysis
Fiber photometry signals were corrected for photobleaching by customized function. The baseline was calculated using a moving mean function, and the ΔF/F was computed as: dff = (raw signal / baseline) - 1. Both the signal and isosbestic reference are converted into z-scores by subtracting the median and dividing by the standard deviation. The reference was fit to the signal using robust linear regression, and the final ΔF/F was calculated as the difference between the z-scored signal and the aligned reference. The ΔF/F was smoothed using a moving mean with a window of 10 to reduce noise.

After photobleaching correction, signals were cut, taking a 4s window 10 frames before tone onset. Each trial trace was aligned and normalized to tone onset. All data shown is aligned to tone onset (time 0 s). For a trial-aligned transition selection, first, engagement blocks were defined as consecutive periods with a hit rate over 85% after task acquisition. The last transition was selected per animal back-tracing to the last block of Misses before a high-engagement block. Engagement for individual trials was defined around Hit trials, using a window of 5 trials before and 5 trials after each hit, and measuring miss rates. If the miss rate was smaller than 75%, that hit and the block of different trials on that 11-trial window were identified as ‘engaged’. All analysis was restricted to engaged trials unless otherwise stated. Pre-transition epochs comprised from after task acquisition to the trial before the transition for individual animals. All AUC calculations were done using the trapz MATLAB function for selected windows. 50 hits were selected pre- and post-transition for each animal in Figure 5c-f and pooled together. For operant latency distributions, a histogram that contained the average of 50 trials per animal with latencies larger than 20 ms from tone onset were composed in 12 bins. The operant latency jitter was obtained using Levene tests based on absolute deviations from the group mean. ‘Cue’ window comprised a 1 s window after tone onset. ‘Outcome’ window comprised the following time after the cue (~2.8s). Logistic fits were applied to the average of 50 trials per animal for the cue + outcome, cue alone or outcome alone AUC windows. Logistic fits were obtained using a moving window over the averaged trials with a 5-trial window length.

Task acquisition was defined as the day in which animals show consistent d’>1.5 (trial # at the end of that day). Average acquisition trial was 1200, with std=424 trials, n=7 mice for DLS, n=6 mice for DMS (Supplementary Fig. 6d). The average transition trial was 10292, with std=3318 trials, n=7 mice for DLS, n=6 mice for DMS (Supplementary Fig. 6e). For ‘early’ and ‘late’ windows of pre-transition data, ‘early’ trials comprised the first 1171 trials after acquisition, and the ‘late’ were the last 1171 trials before the transition, to select a strong comparison. Consummatory licks were counted as any licks happening after initial the operant lick. In Supplementary Fig. 6j, for the DLS, n=7 animals pooling together all trials, with a total of 17,369 pre-transition trials with reward, n=3,553 pre-transition trials without water consumption, n=6,340 trials with reward post-transition, and n=477 post-transition without water consumption. For the DMS, n=6 animals pooling together all trials, with a total of n=13,671 pre-transition trials with water consumption, n=3,383 pre-transition trials without water consumption, n=5,972 trials with water consumption post-transition, and n=417 trials without water consumption post-transition. A sub-selection of pre- and post-transition trials was applied to assess the effect of short latency trials in the late-in-trial DLS activity (Supplementary Fig. 6m). From a total of 350 trials shown in Fig. 5c, only 91 trials pre-transition complied with the rule of belonging to trials with 1st lick latencies >0.328 s. These were compared with 29 trials post-transition under the same rule. As a control, we selected 20 DLS activity trials per animal on the transition day and shuffled trial identity into two groups mimicking pre (black) and post (red) transition (Supplementary Fig. 6n). A second control was made by maintaining the transition structure but selecting 20 trials 3 days before the transition, as a ‘pseudo-transition’. We grouped pre (black) and post (red) pseudo-transition trials (Supplementary Fig. 6o).

Sensory-specific reward devaluation test analysis
We determined that the differential consumption of vanilla-sucralose and tap water during the devaluation sessions was a covariate for the hit rate21,29,93,94.

We performed a two-way repeated-measures ANOVA on consumption amount across conditions (Supplementary Fig. 4b). The ANOVA was implemented through a Linear Mixed-Effect Model that used condition, transition, and their interaction term as the factorial predictors for consumption, while also included the animal identities to account for the within-subject variance. The analysis revealed a significant effect of condition, X2(1)=5.56, p=0.018, partial R2=0.00, and transition, X2(1)=8.00, p=0.0047, partial R2=0.00. Post hoc comparisons using estimated marginal means with Tukey correction and Kenward-Roger df method indicated that the mean consumption for the pre-transition (M = 0.5, SE = 0.041) and post-transition (M = 0.37, SE = 0.041) valued condition was significantly different, t(24.5) = 2.78, p = 0.010. The pre-transition valued (M = 0.50, SE = 0.041) and water (M = 0.39, SE = 0.048) conditions also had significantly different mean consumption, t(24.5) = 2.46, p = 0.021. Therefore, animals generally drink more in valued than devalued condition, and in pre- than post-transition. The absence of a significant group × transition interaction indicates that these effects are additive rather than group-specific responses to the transition. Due to low residual variance in repeated-measure ANOVA, a low R2 for the main effects suggests the differences are consistent but small, yet still warrants further analysis.

To test the effect of liquid consumption amount on hit rate, we performed linear regression with the two variables. Hit rate and consumption amount were negatively correlated (Supplementary Fig. 4c). To verify that the correlation between consumption and hit rate was uniform across animals, we compared different random slope models to the random intercept model currently employed by the analysis. Including condition, X2 (2) = 1.93, p = 0.38, transition, X2 (2) = 1.02, p = 0.60, or both, X2 (5) = 5.17, p = 0.40, as within-subject effects did not significantly increase the overall model fit. Therefore, we proceeded with the random intercept fixed slope model, which considered only the subject identities as the random effect.

We then proceeded to perform a 2-way repeated-measure ANCOVA to analyze the effect of condition and transition on hit rate, with consumption as a covariate. The slope of the consumption covariate was extracted to calculate the normalized hit rate. The homogeneity of regression slopes assumption was not violated (insignificant interaction effect between the predictors and the covariate, 2-way ANOVA, X2(3, N = 10) = 2.67, p = .45). Since the ANCOVA was implemented through a Linear Mixed-Effects Model, the slope of the consumption covariate (-0.75) was extracted to calculate the normalized hit rate with the formula below:
Hit Rate_normalized = Hit Rate_raw - slope × (Consumption - Consumption_average)

For the 2-way repeated-measure ANOVA on normalized hit rate. Post hoc comparisons using estimated marginal means with Tukey correction and Kenward-Roger degree of freedom method indicated that for pre-transition, the mean normalized rate of valued condition and that of the devalued condition were significantly different, t(24.5) = 2.36, p = .027. The mean normalized rate of the pre-transition and post-transition devalued condition was significantly different, t(24.5) = -4.42, p = .00020. Effect size was calculated as partial R2 for the repeated-measure two-way ANOVA, and Cohen’s d for the t-test.

The devaluation index for each animal was calculated based on the normalized hit rate with the formula below:
Devaluation Index = (Hit Rate_valued - Hit Rate_devalued) / (Hit Rate_valued + Hit Rate_devalued)

A devaluation index score of 1 suggests perfect goal-directed behavior85,95. In contrast, a score of 0 is habitual. Since there existed a clear prediction that the post-transition devaluation index is higher than that of the pre-transition, we performed a one-tailed dependent sample t-test to compare the pre- and post-transition devaluation index96–98, which yielded significant result, t(6) = 2.42, p = 0.026, two-sided 90%CI [0.078,0.71], Cohen’s d =0.92.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "The authors have improved the manuscript, and their replies are reasonable, but this revision has not significantly altered my assessment. I think the lack of mechanistic insight (e.g. reliance on lesions as the main causal manipulation) and limited conceptual advance remain weaknesses. As I pointed out earlier, on the whole it's an interesting paper with some intriguing data in a field that has not seen significant advances in recent years. Whether the conceptual advance is significant enough for publication at this journal I will let the editors decide.", "reviewer_2": "The authors have made some revisions to the manuscript and softened their claims in places to address the previous reviews.\nIn my view, there are several fundamental issues with the task and interpretation that remain despite the arguments presented in the rebuttal. I do not believe these can be addressed by further text revision unless the entire framing of the paper is changed. As it stands the conceptual advance is limited. Reviewer 1 shared this view.\nBelow I highlight the most significant weaknesses of the manuscript. Additional discussion of these points can be found in my previous review.\n1. I still remain unconvinced that the water consumption is not driven by a physiological need. Even if the animals are not dying of thirst, this does not mean that a motivational state is not engaged much the same as in other water or food restriction regimens. As mentioned before, deprivation of either food or water to where animals are at 90% of their free-feeding weights (historically 85% was common) is commonly used elsewhere to motivate animals to respond for food or water. The fact that animals won't do the task if water access is unrestricted obviates this point.\nTo me this is not just about modifying the wording, as the authors have done, it's a fundamental issue that this study is not as unique from other work as is suggested. I appreciate that water restriction and the newer CA method are commonly used (don't need a list of references to support that) but I disagree with the assertion that animals are not motivated by a physiological need and do not feel the argument that animals engage with the task out of interest or volition is convincing.\n2. Without direct support, the authors continue to argue other (“typical”) restriction protocols (that produce similar weight loss) produce strong constant motivation that is somehow different to what is produced in the current study. Where is the proof of this? having run many of these types of tasks, rodents do not do things like lever-press uniformly across a session. Maybe it looks this way averaging across many animals but I don't understand the basis for this claim or the assertion that the current study is fundamentally different to many that have gone before in terms of restriction and motivating animals to execute a task.\n3. It remains equivocal whether the behaviour is instrumental. Experimenters can set up whatever arrangements they like but it doesn’t mean that’s what the animal learns or uses to control behaviour (e.g., despite programming a requirement for the animal to lick to get a reward, there is a cue and a context that are present that are also associated with water and could elicit licking, thus Pavlovian processes are not ruled out). Without testing that the R-O contingency is actually essential for behavioural control (e.g. impose an omission schedule) we simply don’t know. However, if the behaviour is not instrumental in nature, it cannot be used as a model of habits.\nThe Authors present several other reasons why they believe the behaviour is instrumental. I respond to each of these points as follows:\n- S+/S- discrimination. The authros argue that the animals had to learn to suppress licking to the S- cue. However, animals will routinely respond to a CS- early in training and gradually learn not to respond to it. This discrimination or time required to learn it does not rule out Pavlovian processes.\n- Pavlovian performance is sensitive to changes in motivational state and so “volitional engagement fluctuations” don’t rule out Pavlovian processes either.\n4. I previously raised the issue that the photometry analyses are focused on activity that correlates with the stimulus and the outcome and a signal related to the response cannot be disentangled from any signal related to the outcome. For me, it’s not adequate to add this as a limitation in the discussion. This reveals a fundamental flaw in the experimental design; if the goal was to study instrumental behaviour the task should be able to identify neural activity related to the response clean of other confounds. If others adopt this task, they will face the same problem."}, "author_response": "Response to reviewers (Round 3):\nReviewer #1 (Remarks to the Author):\n1) The authors have improved the manuscript, and their replies are reasonable, but\nthis revision has not significantly altered my assessment. I think the lack of\nmechanistic insight (e.g. reliance on lesions as the main causal manipulation) and\nlimited conceptual advance remain weaknesses. As I pointed out earlier, on the\nwhole it's an interesting paper with some intriguing data in a field that has not seen\nsignificant advances in recent years. Whether the conceptual advance is\nsignificant enough for publication at this journal I will let the editors decide.\nWe appreciate that Reviewer 1 acknowledges the improvements made during revision,\nand finds that our paper ‘on the whole’ is interesting and our data intriguing. We would\nlike to address Reviewer 1's concerns about mechanistic insight and conceptual advance.\nMechanistic insight is drawn from two complementary experiments: DLS lesions and dual\nfiber photometry throughout learning and as individual animal transition to a habit. Our\nbroad question focused on the nature, timing, and properties of habit transitions. We\nexploited DLS lesions given the >50 years of evidence demonstrating that such lesions\nprevent or reduce habitual behavior. This was one of several measures (i.e., sensory-\nspecific outcome devaluation, motor automaticity, pupillary measures, and neural\nrecordings in the dorsal striatum) used to demonstrate that our behavioral transition was,\nin fact, a transition from goal-directed to habit. Moreover, lesion experiments establish\nthat the DLS is necessary for habit formation, a conclusion that would not be further\nstrengthened with optogenetic inhibition. As a result, we opted to perform dual fiber\nphotometry recordings from the DLS and DMS throughout learning and across habit\ntransitions to gain mechanistic insight: it revealed that DLS neurons exhibit an abrupt,\nstate-like reduction of outcome-related signaling precisely at the moment of habitual\ntransition. For the first time, we show that both the DLS and DMS are being ‘setup’ in\nparallel as an animal learns the task, and it’s not only that behavior is ‘expressed’ as\nabrupt, but rather that the underlying neural activity exhibits an abrupt shift as well. This\nobservation provides a major conceptual advance of the underlying (systems-level)\nneural mechanism and calls into question core dogma within the field that the DLS\ngradually strengthens until reaching a threshold and instead suggests that both the DLS\nand DMS have stable patterns of contingency-specific activity prior to a habit transition.\nSecond, the evidence for the importance of the work has been validated in a highly\ncompetitive external venue. We recently (July 2025) were awarded an NIH R01\napplication based on this behavioral study on first submission with a 6th percentile score\nin the Neurobiology of Motivated Behavior study section. This panel comprises individuals\nexceptionally qualified to evaluate the significance of work in this area. Notably, all three\nreviewers specifically praised the paradigm as highly innovative, and especially relevant\nfor exploring habit formation:\n‘This R01 application concerns a highly novel and important new\nopportunity to evaluate the transition from goal-direct to habitual\nbehavior in real time. Currently methodology is limited to performing\ndiscrete probe trials, limiting our ability to see the circuit dynamics\nsurrounding this transition. In addition, probe trials interrupt training\ndays and can only occur so many times before inducing their own\nlearning effects. The significance for the field would substantially\nopen up new opportunities to reveal mechanisms for behavioral\ntransitioning from goal to habit. Their key innovation is to create a\nmotivation that is strong enough to motivate action for the reward, but\nnot as severely. They do this by not water restricting, but putting less\nappetitive hydration solution in the home cage (citric acid solution) and\nhaving pure water as the training reward. (Reviewer 1)’\n‘The concept of volitional engagement is highly conceptually\ninnovative and appealing for enabling more real-time studies of\nhabit. Neural circuits for habit formation are important to\nunderstand, but current approaches have been limiting. These\nexperiments will help the field compare between hypotheses of habit\nformation that posit parallel acquisition of learning by habit and goal-\ndirected systems, with hypotheses that posit a slower time course of\nhabit formation. The framework of volitional engagement is highly\nintriguing and experimentally potentially very useful. (Reviewer 2)’\n‘The Principle Investigator offers a fresh perspective with a new\nprotocol (citric acid in home cage) to induce and monitor putative\nhabit transition with sharper temporal resolution than previously\npossible. (Reviewer 3)’\nThese comments underscore that our paradigm addresses a longstanding gap in the field,\none recognized by independent experts with appropriate expertise. Notably, the R01\nstudy section explicitly referenced our bioRxiv preprint (prior to the three rounds of\nrevision) during their evaluation, meaning the behavioral approach described in this\nmanuscript has now been positively assessed by three additional qualified reviewers. If\nwe consider those alongside Reviewer 3, who shifted from initial skepticism to strong\nsupport, that makes four independent experts who view this work as both conceptually\nnovel and impactful.\nIn the end, the idea of ‘conceptual advance’ depends on each person’s own subjective\nevaluation. Given our strong efforts to address all prior critiques, and no remaining\nsubstantive critiques, we think the final decision on ‘conceptual advance’ should come\nfrom the field once the manuscript is published.\nReviewer #2 (Remarks to the Author):\nThe authors have made some revisions to the manuscript and softened their claims in\nplaces to address the previous reviews.\nIn my view, there are several fundamental issues with the task and interpretation that\nremain despite the arguments presented in the rebuttal. I do not believe these can be\naddressed by further text revision unless the entire framing of the paper is changed. As it\nstands the conceptual advance is limited. Reviewer 1 shared this view.\nBelow I highlight the most significant weaknesses of the manuscript. Additional discussion\nof these points can be found in my previous review.\nWe thank the reviewer for recognizing our revisions to the manuscript, and for\nacknowledging the effort we made to soften our language following their previous\nsuggestions. We address the concerns noted below point-by-point.\n1.1) I still remain unconvinced that the water consumption is not driven by a\nphysiological need. Even if the animals are not dying of thirst, this does not mean\nthat a motivational state is not engaged much the same as in other water or food\nrestriction regimens. As mentioned before, deprivation of either food or water to\nwhere animals are at 90% of their free-feeding weights (historically 85% was\ncommon) is commonly used elsewhere to motivate animals to respond for food or\nwater. The fact that animals won't do the task if water access is unrestricted\nobviates this point.\nWe wish to clarify that we make no such claim in the revised manuscript that was\nreviewed. In fact, we explicitly removed this framing during revision in response to\nReviewer 2’s earlier feedback. We state that this approach “reduces—but does not\neliminate—reward-seeking for plain water droplets [Lines 16-18],” which is a factual\nobservation based on behavioral data presented in the paper and was not contested by\nany reviewer. We make no categorical claims about the presence or absence of\nphysiological need in this version of the manuscript. That discussion appeared in prior\nversions but was removed after careful consideration of reviewer concerns. The core\nnovelty of our paradigm is not that animals engage and disengage (which may occur in\nother contexts), but that we quantify this variability and use it as an operational index of\ndecision control.\n1.2) To me this is not just about modifying the wording, as the authors have done, it's\na fundamental issue that this study is not as unique from other work as is\nsuggested. I appreciate that water restriction and the newer CA method are\ncommonly used (don't need a list of references to support that) but I disagree with\nthe assertion that animals are not motivated by a physiological need and do not\nfeel the argument that animals engage with the task out of interest or volition is\nconvincing.\nAs stated before, in the third version of the manuscript, we did not make any argument\nthat the animals are not motivated by a physiological need, so this critique is not based\non the content of the reviewed manuscript.\nIn response to the reviewer’s strong opinion on 'volitional', we now completely remove\nthe word “volitional” from all text:\n[Line 16-17]: “To address this, we gave mice free access to citric-acid water...”\n[Line 148-149]: “…suggesting that mice are intermittently engaging in the task”.\n[Lines 229-230]: “…at two discrete time points during the behavioral training…”\n[Line 250]: “…to further test the validity of this paradigm…”\n[Lines 334-335]: “…when animals occasionally did not consume the reward).”\n[Lines 456-457]: “Here, we show one approach to address this question by giving\nanimals ad libitum access to CA water…”\n[Line 459]: “…animals occasionally engage and disengage from the task…”\n[Lines 501-502]: “…toward the habitual nature of post-transition behavior in\nintermittently engaged mice. This novel approach adds a powerful en passant tool to\nstudy decision control.”\n[Line 545]: “The spontaneous and abrupt appearance of habits in our paradigm…”\n2.1) Without direct support, the authors continue to argue other (“typical”) restriction\nprotocols (that produce similar weight loss) produce strong constant motivation\nthat is somehow different to what is produced in the current study. Where is the\nproof of this?\nIn the version of the manuscript reviewed (June 2025, third version), we never use the\nterm “typical restriction protocol”, and no longer make any such comparative claims about\nmotivational state. As a result, we honestly do not know what the Reviewer is referring to,\nunless they are considering the very first version they reviewed >2 years ago in June\n2023. In fact, we make nearly the opposite point in Lines 595-599:\n‘While our task differs from traditional instrumental training paradigms\nused to explore goal-directed and habitual control of behavior, the\nmain value of our approach lies not in the level of restriction or the\nexistence of behavioral variability, but in the recognition and\nquantification of this variability in reward-seeking as a proxy for\nunderlying decision control. This provides a complementary approach\nto existing procedures for probing the temporal dynamics and neural\ndrivers of habit transitions in real-time.’\n2.2) having run many of these types of tasks, rodents do not do things like lever-press\nuniformly across a session. Maybe it looks this way averaging across many\nanimals but I don't understand the basis for this claim or the assertion that the\ncurrent study is fundamentally different to many that have gone before in terms of\nrestriction and motivating animals to execute a task.\nThere seems to be a core misunderstanding. The key novelty of our paradigm is not\nthat animals engage and disengage nor the level of restriction (which may have\noccurred in prior studies and in the Reviewer’s own experiments they refer to), but that\nwe quantify this variability and use it as an operational index of decision control. If past\nstudies averaged across animals and thereby obscured intra-individual fluctuations (as\nstated by the Reviewer), then this reinforces, rather than undermines, our central claim.\nOur central methodological contribution is to precisely avoid the practice of group\naveraging by operationalizing within-animal variability as a window into understanding\nreal-time decision control within individual animals. We are not critiquing the field as a\nwhole but proposing a complementary approach that makes these individual patterns\nvisible. We have previously written about the limitations of averaging and the\nimportance of embracing within-subject behavioral dynamics (Moore & Kuchibhotla,\nIBRO Reports, 2022), and this manuscript reflects a direct extension of that perspective.\n3.1) It remains equivocal whether the behaviour is instrumental. Experimenters can set\nup whatever arrangements they like but it doesn’t mean that’s what the animal\nlearns or uses to control behaviour (e.g., despite programming a requirement for\nthe animal to lick to get a reward, there is a cue and a context that are present that\nare also associated with water and could elicit licking, thus Pavlovian processes\nare not ruled out). Without testing that the R-O contingency is actually essential for\nbehavioural control (e.g. impose an omission schedule) we simply don’t know.\nHowever, if the behaviour is not instrumental in nature, it cannot be used as a\nmodel of habits.\nThe Authors present several other reasons why they believe the behaviour is instrumental.\nI respond to each of these points as follows:\n- S+/S- discrimination. The authros argue that the animals had to learn to suppress licking\nto the S- cue. However, animals will routinely respond to a CS- early in training and\ngradually learn not to respond to it. This discrimination or time required to learn it does\nnot rule out Pavlovian processes.\n– Pavlovian performance is sensitive to changes in motivational state and so “volitional\nengagement fluctuations” don’t rule out Pavlovian processes either.\nWe must respectfully disagree. The Reviewer has across three rounds of review\nbrought up different aspects of this concern all of which we addressed with new\nexperiments, analysis, or conceptual justification. This was satisfactory for Reviewer 3\nwho shared many of the same concerns. Rather than re-state what we have already\ndone, we would like to frame the key issue: is the R-O contingency essential for\nbehavioral control? In prior reviews, the strongest counter-arguments that this was\npotentially Pavlovian included the below comments from Reviewer 2:\n‘But even with lever-pressing, it can also be controlled by Pavlovian contingencies (e.g.\nsign-tracking behaviour).’ -> This comment suggests that the Reviewer believes the\nbehavior could be a form of Pavlovian sign-tracking.\n‘Just to be clear, Pavlovian responses can be sensitive to outcome devaluation. Such\ndata were used to support arguments for S-S (vs. S-R) encoding in Pavlovian\nparadigms and were quite important in that regard.’ -> This comment suggests the\nreviewer is pointing to a broad finding that Pavlovian conditioned responses are\nsensitive to outcome-specific devaluation.\nWe observe that animals in our task are devaluation-sensitive at an earlier time-point\n(Session 7-8) and insensitive at a later time-point (Session 18-21). To our knowledge,\nnearly all Pavlovian accounts suggest that devaluation sensitivity is stable across\nlearning (Holland, 1998; Holland, 2004 among others). In the most direct test we’re\naware of that looks at devaluation sensitivity at early (Sessions 7-8) versus late\n(Sessions 17-18) in a Pavlovian task (Keefer...Calu, 2020), goal-tracking rats remained\ndevaluation-sensitive both early and late (consistent with all prior reports), while\ninterestingly, sign-tracking rats were insensitive early and sensitive late. In neither\ncase did Pavlovian conditioning produce the sensitive → insensitive pattern (in fact,\nsign-tracking rats produced the exact opposite result). By contrast, that is precisely the\ntrajectory long associated with instrumental behavior transitioning to habit under\nextended training. Our within-subject results follow this canonical instrumental pattern.\nThe only justifiable conclusion (based on consistency with prior evidence) is behavior in\nour study is instrumental goal-directed control pre-transition and habitual control post-\ntransition, rather than an ad hoc explanation that must account for multiple additional\nfactors (e.g., learned suppression to the S-, fluctuations in engagement within a\nsession) to produce a just-so Pavlovian account.\nMore broadly, as the Reviewer knows, framing this as a binary, ‘settled’ distinction\nbetween Pavlovian and instrumental control obscures what actually moves the field\nforward: data-driven evidence that builds on past findings. Our within-subject\ndevaluation results provide exactly that: outcome sensitivity before the transition and\noutcome insensitivity after. This data are incompatible with all prior accounts of\ndevaluation sensitivity for Pavlovian conditioning but entirely consistent with >60+ years\nof instrumental conditioning literature.\nFinally, we would like to note that the equivocality of the behavior being under\ninstrumental control was not shared by Reviewer 3 after the second revision. Moreover,\nas noted in response to Reviewer 1, our framing of this behavior as instrumental was\nnot viewed as equivocal by three additional expert reviewers at the NMB study section.\nWe think, at this point, these findings should be further adjudicated by interested parties\npost-publication.\n4.1) I previously raised the issue that the photometry analyses are focused on activity\nthat correlates with the stimulus and the outcome and a signal related to the\nresponse cannot be disentangled from any signal related to the outcome. For me,\nit’s not adequate to add this as a limitation in the discussion. This reveals a\nfundamental flaw in the experimental design; if the goal was to study instrumental\nbehaviour the task should be able to identify neural activity related to the response\nclean of other confounds. If others adopt this task, they will face the same problem.\nAgain, we must respectfully disagree. We explicitly analyze trials in which animals\nexecute the operant lick but do not consume the reward (no-reward trials) and those\nwhere the reward is consumed, thereby isolating the outcome-related neural activity.\nThese analyses are in Extended Data Fig. 6j-k and described in both Results and\nMethods. What we acknowledge in the discussion is that ‘we use two time periods when\nevaluating neural signals: cue-related, which may include an action component and\noutcome-related, which may include both outcome and evaluative signals.’ Most\nimportantly, for the purpose of this study, the temporal resolution we have is sufficient\ngiven the question, as we focus on neural activity dynamics in a trial-based manner\naround the transition: namely, the sharpening of cue-evoked response and suppression\nof outcome-related signals. Therefore, we believe it is inadequate to completely dismiss\nour added discussion. And as we note, future work using electrophysiology can more fully\ndisentangle the response-related signaling, so if others adopt our paradigm using different\ntechnical approach, they will not “face the same problem”. More broadly, ‘others’ have\nagency and will decide what is useful from our study and what is not, this is not something\nanyone can prescribe. We can only say that we have fielded multiple inquiries about our\nparadigm with fellow scientists particularly interested in how we operationalize inter- and\nintra-animal variability in response rate as a proxy measure of decision control."}, {"round": 2, "reviewer_comments": {"reviewer_1": "As I said in the previous round, the authors made a reasonable attempt to address my concerns, but I'm not sure about the degree of conceptual advance here. The authors quote some grant reviewers who are enthusiastic about the findings, but I'm not sure how relevant that is. Clearly, there might be some in the field who find the current findings interesting. And clearly there might be others, like R2, who strongly disagree with the interpretation. While I often find R2's arguments cogent, my overall evaluation is not as negative. For me, the authors tried their best to perform some innovative experiments and also tried their best to be balanced in their presentation. Controversy should not prevent publication. So again I leave it to the editors."}, "author_response": "We thank the reviewer for this perspective and for noting our balanced approach to describing our results. We agree that different viewpoints (including Reviewer 2’s) are part of a healthy scientific debate, and we appreciate the Reviewer’s point that controversy alone should not preclude publication."}]

Decision letter