Devaluation of response-produced safety signals reveals circuits for goal-directed versus habitual avoidance in dorsal striatum

Domain: neuroscience
Published: 2026-02-09
Editorial outcome: major_revision

Abstract
Active avoidance responses (ARs) are instrumental behaviors that prevent harm. Adaptive ARs may contribute to active coping, whereas maladaptive avoidance habits are implicated in anxiety and obsessive-compulsive disorders. The AR learning mechanism has remained elusive, as successful avoidance trials produce no obvious reinforcer. We used a novel outcome-devaluation procedure in rats to show that ARs are positively reinforced by response-produced feedback cues that develop into safety signals during training. Males were sensitive to feedback devaluation after moderate training, but not overtraining, consistent with a transition from goal-directed to habitual avoidance. Using chemogenetics and feedback devaluation, we also show that goal-directed vs. habitual ARs depend on dorsomedial vs. dorsolateral striatum, suggesting a significant overlap between the mechanisms of avoidance and rewarded instrumental behavior. Females were insensitive to feedback devaluation due to a remarkable context-dependence of counter-conditioning. However, degrading the contingency between avoidance and feedback suggests that both sexes rely on safety signals to perform goal-directed ARs.

Introduction
Active avoidance is a form of response learning that allows subjects to suppress fear and neutralize threats. Avoidance likely evolved to expand the defensive behavior repertoire beyond a narrow set of innate, inflexible reactions (i.e., fight, flight, freezing). Avoidance response (AR) learning may even underlie complex defensive behaviors that could not have evolved under predatory pressure. For instance, commuters may buckle their seatbelt to protect against future injury or slam their brakes to prevent imminent harm. Clinically, active avoidance is important because adaptive ARs likely contribute to active coping and resilience1. Conversely, inappropriate or persistent avoidance habits are thought to protect irrational fears from extinction2 and contribute to disorders of control like OCD3.

Active avoidance is typically studied by presenting subjects with a warning stimulus (WS; e.g., sound) that predicts the delivery of an aversive electric shock. This transforms the WS into a conditioned threat that arouses fear and triggers defensive reactions like freezing. Subjects eventually emit the AR (e.g., bar press) and gain exposure to avoidance contingencies. As training continues, innate fear reactions to the WS are progressively replaced by ARs.

Theoretical disputes about the reinforcement mechanism and misconceptions about the mediating role of fear stalled progress in understanding the associative structures and neurobiological mechanisms of active avoidance4,5. Prominent instrumental theories focused on stimulus-response learning via negative reinforcement (i.e., fear-reduction when warning stimuli terminated and/or shock frequency reduction)6,7. Avoidance research also failed to fully capitalize on modern conceptions of instrumental behavior developed using appetitive reinforcers8,9. This work moved beyond the notion that instrumental training produces only reflexive stimulus-response habits, demonstrating that subjects can also perform more cognitive, goal-directed responses that depend on knowledge of action-outcome associations and motivation to obtain valued outcomes. Goal-directed responding is typically diagnosed with outcome-devaluation or contingency degradation tests8,9. Thus, barpressing for food is goal-directed if reducing the food’s value via satiation or counter-conditioning reduces barpressing. The same can be said if food delivered non-contingently (independent of behavior) reduces barpressing. Habits are usually inferred by insensitivity to these tests. However, the situation is more complicated for active avoidance, as there is little agreement about what reinforces avoidance. If the removal or absence of aversive stimuli contributes (i.e., warning stimulus termination or shock omission), it is unclear how one could verifiably devalue these outcomes or deliver more of them. Identifying a precise reinforcement mechanism would establish that active avoidance is instrumental and help differentiate between goal-directed vs. habitual ARs.

A less studied hypothesis is that ARs are positively reinforced by response-produced safety signals that are generated via their inverse relationship with the warning stimulus and shock during avoidance training7,10–12. Explicit feedback cues that immediately follow ARs are known to develop conditioned inhibition (safety) and reinforcing properties that promote avoidance10,13–16. Here, “reinforcing” is used in a general way, though many early studies assumed strengthening of stimulus-response associations. It is unknown, however, whether safety signals behave as valued outcomes supporting goal-directed avoidance via action-outcome associations, akin to food in a barpressing experiment. To our knowledge, no active avoidance studies have attempted to devalue response-produced safety signals. To examine this, we designed a devaluation protocol where AR feedback cues are counter-conditioned by pairing with shock. We also present data using a contingency degradation test, where non-contingent safety signals are presented during avoidance.

If ARs are positively reinforced by safety, this raises the exciting possibility that active avoidance relies on instrumental brain circuits already identified by appetitive studies. Moderately trained instrumental responses are sensitive to outcome devaluation and suppressed by manipulations impairing neural activity in posterior dorsomedial striatum (pDMS)17,18. In contrast, instrumental responses overtrained or trained on interval schedules are insensitive to outcome devaluation and suppressed by manipulations impairing dorsolateral striatum (DLS)18,19. Importantly, however, this latter effect is only evident in subjects that also received outcome devaluation or contingency degradation. This is because goal-directed responding mediated by a parallel pDMS circuit regains control when DLS-mediated habits are impaired19. Thus, if goal-directed and habitual avoidance memories co-exist in different brain circuits, a tool like outcome devaluation may be necessary to identify and study these circuits. Impairing either circuit alone may produce no change in overt avoidance behavior.

Here, we show that counterconditioning of response-produced safety signals impairs moderately trained, but not overtrained, ARs in male rats. We also demonstrate that moderately trained ARs depend on pDMS, and overtrained ARs depend on DLS, as in appetitive studies. Further, we found that ARs in female rats are insensitive to feedback devaluation due to a remarkable context dependence of counter-conditioning. Nonetheless, we show contingency degradation data suggesting that both sexes rely on safety signals for goal-directed avoidance.

Results
Avoidance is acquired similarly in male and female rats
Overtraining is frequently used to induce habitual responding in studies where responses are rewarded20. To study the development of avoidance habits, we used a signaled active avoidance shuttlebox procedure in rats and varied the amount of training. Figure 1 shows avoidance training data from multiple behavioral experiments. The percentage of good avoiders was similar in male and female rats (Fig. 1b). Poor avoiders were excluded from further analyses. Good avoiders approached 100% success by the fifth training session, and average avoidance latencies settled around 5s after WS onset (Fig. 1c–f). Nearly identical patterns of learning and performance were observed for males and females in both training cohorts. Two-way ANOVAs support this for moderately trained (Avoidance, Session: F 3.03,178.5 = 132.0, p < 0.0001, Sex: F 1,59 = 0.34, p = 0.5601, Session × Sex: F 4,236 = 2.28, p = 0.0613; Latency, Session: F 3.13,184.7 = 91.09, p < 0.0001, Sex: F 1,59 = 1.13, p = 0.2929, Session × Sex: F 4,236 = 0.73, p = 0.5741) and overtrained (Avoidance, Session: F 3.12,93.5 = 73.39, p < 0.0001, Sex: F 1,30 = 0.23, p = 0.6367, Session × Sex: F 19,570 = 0.34, p = 0.9965; Latency, Session: F 4.93,147.8 = 28.88, p < 0.0001, Sex: F 1,30 = 0.88, p = 0.3549, Session × Sex: F 19,570 = 0.995, p = 0.4645) rats. Interestingly, variation in successful avoidance latencies declined with overtraining even as average response latencies remained constant (Fig. 1g, h). This invariant response topography is a positive sign of automaticity21, and possibly habitual control22. For each animal, a significant large-sized session effect was apparent for AR latency (F 19,570 = 31.090, p < 0.0001, η2 = 0.18). We therefore calculated coefficients of variation to assess variability in latencies across sessions using a repeated measures mixed effects model. Here, coefficient of variation is inversely related to stereotyped behavior. Avoidance latency variability declined with overtraining for both sexes (Session: F 4.82,144.3 = 15.34, p < 0.0001; Sex: F 1,30 = 0.0196, p = 0.8895; Session × Sex: F 19,569 = 1.03, p = 0.425). Since asymptotic performance was achieved by session 5, we compared coefficients of variation during session 5 to all subsequent sessions (Dunnett’s test). Avoidance during sessions 7 through 20 was more stereotyped than avoidance during session 5 (p values ranged from 0.0005 to < 0.0001). The only sex differences observed during training were more shuttles during the acclimation and inter-trial intervals for females (Supplementary Fig. 1).

Devaluation of response-produced feedback impairs moderately trained avoidance in male rats
To test whether goal-directed avoidance depends on feedback (FB) safety signals, male rats received moderate shuttlebox training followed by paired or explicitly unpaired counterconditioning of the feedback tone in separate chambers (Fig. 2a). The paired procedure was used to reverse safety properties of the feedback tone in FB-Devalued rats. The unpaired procedure controlled for stimulus exposure during counterconditioning but preserved the safety value of the tone in FB-Valued rats. A FB-Control group was treated identically to the FB-Devalued group, except the feedback stimulus used during avoidance training differed (5s house light off). This allowed us to assess retardation of acquisition23 for FB-Devalued rats during counterconditioning compared to FB-Control rats that also had avoidance training, but the tone was novel. It also allowed us to assess whether any avoidance impairment induced by counterconditioning was specific to the feedback stimulus used during avoidance training. After counterconditioning, rats were returned to the shuttleboxes for two avoidance tests (15 presentations of the WS). Shuttling did not affect WS duration, and no feedback or shocks were presented.

Rats in all moderately trained groups displayed equally high levels of avoidance responding (ANOVA, F 2,24 = 2.94, p = 0.0722), similar AR latencies (F 2,24 = 1.60, p = 0.2225), and low levels of freezing to the WS (F 2,24 = 0.11, p = 0.9002) before feedback devaluation (Fig. 2b–d). Behaviors during the avoidance tests indicate that feedback devaluation significantly impaired avoidance. Shuttling during the WS was lower for FB-Devalued rats than both control groups (Fig. 2e left; Group × Block RM-ANOVA: F 8,96 = 4.41, p = 0.0001). This was clearest during the first testing block, before extinction obscured group differences (Dunnett’s tests: vs. FB-control: p = 0.0154, vs. FB-Valued: p = 0.0275). Since goal-directed responding is most clearly related to memory of the outcome value at the beginning of extinction tests24, Block 1 was also analyzed separately, confirming that FB-Devalued rats emitted fewer avoidance shuttles than controls (Fig. 2e right, ANOVA: F 2,24 = 4.45, p = 0.0228; Tukey’s tests: FB-Devalued vs. FB-control: p = 0.0466, FB-Devalued vs. FB-Valued: p = 0.034, FB-Valued vs. FB-Control: p = 0.8796). Consistent with this, AR latencies were significantly longer for FB-Devalued rats (Fig. 2f left; Group × Block RM-ANOVA: F 8,96 = 3.05, p = 0.0042), especially during Block 1 (Fig. 2f right; ANOVA, F 2,24 = 3.67, p = 0.0408. Tukey’s tests: FB-Devalued vs. FB-Control: p = 0.21, FB-Devalued vs. FB-Valued: p = 0.0337, FB-Valued vs. FB-Control: p = 0.4401). Finally, we also measured freezing to the WS during the first block of tests. Freezing is a fear-related behavior that competes with avoidance4 and multiple studies find that suppressing active avoidance disinhibits freezing25–27. This was also true in our experiment as FB-Devalued rats froze significantly more than both control groups during WS presentations (Fig. 2g; ANOVA: F 2,24 = 10.42, p = 0.0006, Tukey’s tests: FB-Devalued vs. FB-Control: p = 0.0204, FB-Devalued vs. FB-Valued: p = 0.0004, FB-Valued vs. FB-Control: p = 0.1041). These data indicate that counterconditioning of response-produced feedback stimuli impairs moderately trained avoidance even when the devalued feedback stimulus is not presented during the test. This is consistent with goal-directed responding, where the decision to avoid depends on a memory of the current outcome value. Importantly, data from the FB-control group show that our devaluation effect was specific; strong tone-shock conditioning did not affect ARs trained with a visual feedback stimulus.

Next, we conducted the same analyses on overtrained male rats. There were no group differences in avoidance or freezing behavior during the last session of overtraining (Fig. 2h–j; Unpaired, two-tailed, t-tests, Avoidance: t 13 = 1.31, p = 0.2138; Latency: t 13 = 2.13, p = 0.0525, WS-freezing: t 13 = 0.32, p = 0.7505). Unlike moderately trained rats, feedback counterconditioning did not significantly affect overtrained avoidance or WS-freezing during post-devaluation tests [Fig. 2k–m; Two-way RM-ANOVAs and t-tests (unpaired, two-tailed), Avoidance: F 1,13 = 0.015, p = 0.9040 (Group), F 2.361,30.70 = 11.06, p = 0.0001 (Block), F 4,52 = 2.07, p = 0.0981 (Group × Block), t 13 = 1.14, p = 0.2767 (Block 1); AR latency: F 1,13 = 0.056, p = 0.8165 (Group), F 2.910,37.84 = 11.21, p < 0.0001 (Block), F 4,52 = 0.81, p = 0.5244 (Group × Block), t 13 = 0.06, p = 0.9515 (Block 1); WS-freezing: t 13 = 1.56, p = 0.1439 (Block 1)]. These data indicate that ARs are insensitive to feedback devaluation after overtraining, most likely because they are mediated by stimulus-response associations that bypass outcome value when controlling behavior.

Feedback devaluation fails to impair avoidance in female rats
Male data are reported in Fig. 2, showing sensitivity to feedback devaluation after moderate training but not overtraining. An identical analysis in females shows insensitivity to feedback devaluation, regardless of the amount of training. Females assigned to FB-Valued or FB-devalued groups showed equally high avoidance rates (Unpaired, two-tailed, t-tests; Fig. 3b: t 18 = 0.61, p = 0.5508; Fig. 3h: t 15 = 1.42, p = 0.176), similar AR latencies (Fig. 3c: t 18 = 0.34, p = 0.7414; Fig. 3i: t 15 = 0.07, p = 0.9432), and equally low WS-freezing (Fig. 3d: t 18 = 0.76, p = 0.4582; Fig. 3j: t 15 = 0.08, p = 0.9359) before feedback devaluation. During post-devaluation avoidance tests, no group differences were observed for shuttles during the WS [Two-way RM-ANOVAs and Unpaired, two-tailed, t-tests; Fig. 3e, left: F 1,18 = 2.41, p = 0.1374 (Group), F 1.86,33.45 = 10.11, p = 0.0005 (Block), F 4,72 = 1.77, p = 0.1451 (Group × Block), right: t 18 = 1.64, p = 0.1183 (Block 1); Fig. 3k, left: F 1,15 = 0.34, p = 0.5675 (Group), F 2.689,40.33 = 25.14, p < 0.0001 (Block), F 4,60 = 0.33, p = 0.8582 (Group × Block), right: t 15 = 0.56, p = 0.5823 (Block 1)], avoidance latencies [Fig. 3f, left: F 1,18 = 1.78, p = 0.1985 (Group), F 3.35,60.30 = 6.37, p = 0.0005 (Block), F 4,72 = 0.38, p = 0.8232 (Group × Block), right: t 18 = 0.97, p = 0.3446 (Block 1); Fig. 3l, left: F 1,15 = 0.004, p = 0.9527 (Group), F 2.659,39.88 = 14.12, p < 0.0001 (Block), F 4,60 = 0.81, p = 0.5262 (Group × Block), right: t 15 = 0.29, p = 0.7794 (Block 1)], or freezing during the WS [Fig. 3g: t 18 = 0.85, p = 0.406 (Block 1), Fig. 3m: t 15 = 0.36, p = 0.7273 (Block 1)].

Expression of safety signal devaluation is context-dependent in female rats
There are many possible explanations for the failure to impair moderately trained avoidance with feedback devaluation in females. For instance, counterconditioning of safety cues may be less effective in females, safety learning to AR feedback may be weaker in females, or expression of safety signal counterconditioning may be context-dependent in females. We analyzed freezing behavior during the devaluation sessions and subsequent tone fear tests in multiple contexts to assess these.

Counterconditioning was effective in both sexes, regardless of the amount of avoidance training. Separate Three-way (Trial × Training × Group) ANOVAs assessing tone-freezing during the devaluation session found no effects for amount of avoidance training [Males trained with a FB-tone: F 1,26 = 0.15, p = 0.7063 (Training), F 4,104 = 1.07, p = 0.3753 (Training × Trial), F 1,26 = 4.10, p = 0.0532 (Training × Devaluation), F 4,104 = 1.24, p = 0.2996 (Trial × Training × Devaluation); Females trained with a FB-tone: F 1,33 = 0.07, p = 0.7931 (Training), F 4,132 = 1.46, p = 0.219 (Training × Trial), F 1,33 = 0.73, p = 0.4002 (Training × Devaluation), F 4,132 = 0.60, p = 0.662 (Trial × Training × Devaluation)], so this factor was collapsed to conduct Two-way (Group × Trial) RM-ANOVAs with Šídák’s contrasts for each trial. Freezing to the feedback tone was rapidly acquired by FB-Devalued rats receiving tone-shock pairings, while freezing remained low for FB-Valued rats receiving the same stimuli but explicitly unpaired [Fig. 4b, Males trained with a FB-tone: F 4,112 = 15.95, p < 0.0001 (Group × Trial), p values ≤ 0.0009 for Trials 2–5 (FB-Valued vs. FB-Devalued); Fig. 4c, Females trained with a FB-tone: F 4,140 = 11.28, p < 0.0001 (Group × Trial), p values ≤ 0.0002 for Trials 3–5 (FB-Valued vs. FB-Devalued)]. A separate Three-way ANOVA, collapsing the training factor, confirmed there were no sex differences in tone-freezing during the devaluation session for rats trained with a FB-tone: F 1,63 = 0.14, p = 0.7089 (Sex), F 4,252 = 2.015, p = 0.0929 (Sex × Trial), F 1,63 = 1.06, p = 0.307 (Sex × Devaluation), F 4,252 = 1.12, p = 0.3478 (Sex × Devaluation × Trial).

The canonical tests for safety conditioning (conditioned inhibition) are summation and retardation of acquisition23. With aversive conditioning, summation refers to the ability of a safety cue to suppress defensive reactions to a separate threat cue. Retardation of acquisition refers to slower fear conditioning to a safety cue compared to a novel cue. We found evidence for both phenomena during the devaluation sessions. First, unpaired shocks rapidly conditioned freezing to the context in FB-Valued rats. Tone presentations powerfully suppressed this freezing in all FB-Valued rats, regardless of sex or the amount of avoidance training (Fig. 4d). A Three-way (Tone × Sex × Training) ANOVA revealed a Tone × Sex interaction (F 1,29 = 7.46, p = 0.0106), but no effects for the amount of avoidance training [F 1,29 = 0.94, p = 0.3393 (Training), F 1,29 = 3.95, p = 0.0563 (Tone × Training), F 1,29 = 0.03, p = 0.8699 (Sex × Training), F 1,29 = 2.79, p = 0.1059 (Tone × Sex × Training). We therefore collapsed the Training factor and conducted a Two-way ANOVA with Šídák’s post-tests. This revealed a significant suppression of context freezing by the tone (F 1,31 = 397.9, p < 0.0001) and a Tone × Sex interaction (F 1,31 = 6.60, p = 0.0152), due to higher freezing in males during the Pre-tone (p = 0.0166), but not Tone (p = 0.8562), intervals. Second, moderately trained males in the FB-Devalued and FB-Control groups received tone-shock pairings during the devaluation phase. The tone was novel, however, for FB-Controls, who received a visual feedback cue during avoidance training. FB-Devalued rats acquired tone-freezing more slowly than FB-Controls (Group × Trial RM-ANOVA: F 8,96 = 8.98, p < 0.0001), which was most evident on Trial 2 of counterconditioning (Fig. 4b; 70% vs. 93% freezing, Dunnett’s test: p = 0.0288). These findings indicate that avoidance conditioning transformed feedback tones into safety signals, and the degree of safety conditioning was comparable in males and females.

After the final avoidance tests, we assessed tone fear in an unfamiliar context (open field). An ANOVA with Tukey’s post-tests in moderately trained rats revealed an interesting pattern of group differences (Fig. 5b; F 4,42 = 20.64, p < 0.0001). First, FB-Devalued males froze less than FB-Control males (p = 0.0388), providing additional evidence that avoidance feedback stimuli are safety signals (retardation of acquisition). Second, FB-Devalued males froze significantly more than low-freezing FB-Valued males (p = 0.0186), confirming that safety signal counterconditioning is effective and expressed outside the devaluation context. However, in females, tone-freezing was low in both groups (p = 0.7064), with FB-Devalued females freezing significantly less than males (p = 0.0356). The same analysis was conducted for overtrained rats tested in the open field (Fig. 5c; F 3,28 = 13.80, p < 0.0001), and the pattern of findings was identical [FB-Valued vs. FB-Devalued, p = 0.0001 (Males), FB-Valued vs. FB-Devalued, p = 0.7228 (Females), Male vs. Female, p = 0.0005 (FB-Devalued)]. A very similar pattern was observed in the avoidance context (Fig. 5d; F 3,28 = 12.55, p < 0.0001); males, but not females, showed evidence for feedback devaluation [FB-Valued vs. FB-Devalued, p = 0.0001 (Males), FB-Valued vs. FB-Devalued, p = 0.9157 (Females), Male vs. Female, p = 0.0005 (FB-Devalued)]. This pattern changed in the devaluation context, where FB-Devalued rats of both sexes froze more than FB-Valued controls [Fig. 5e; F 3,28 = 23.60, p < 0.0001; FB-Valued vs. FB-Devalued, p < 0.0001 (Males), FB-Valued vs. FB-Devalued, p = 0.0006 (Females), Male vs. Female, p = 0.1461 (FB-Devalued)]. Together, these data suggest that feedback devaluation fails to impair moderately trained avoidance in females because expression of safety signal counterconditioning is context-dependent, unlike males. Attempts to circumvent this context-dependence by conducting devaluation in the shuttleboxes were unsuccessful (Supplementary Fig. 3). However, one finding is notable: when devaluation occurred in the shuttleboxes with the door between chambers closed, FB-Devalued females froze to the tone only when the door was closed (Supplementary Fig. 4), demonstrating a remarkable contextual gating of tone value (safety vs. threat cue). Because of this, male rats were used for subsequent studies interrogating the neural circuits of goal-directed vs. habitual avoidance.

pDMS is required for goal-directed active avoidance
To test the role of pDMS, anterior DMS (aDMS), and DLS in moderately trained avoidance, we used a viral κ-opioid receptor DREADD (KORD) strategy to reversibly suppress neuronal activity in the dorsal striatum. KORD is an engineered Gαi-coupled receptor with no natural ligand that hyperpolarizes neurons when bound to Salvinorin B (SalB), an otherwise inert compound28. We first expressed KORD unilaterally in dorsal striatum neurons (see “Methods”). We then monitored field activity in awake rats via bilateral chronically implanted electrodes before and after SalB (i.v., 2.5 mg/kg; Supplementary Fig. 5). Theta band oscillations are a primary local field potential signature of activity in dorsal striatum29,30, and neurons in dorsal striatum become entrained to theta frequencies during instrumental learning29. Based on an assessment of the full mean Fast Fourier Transfer spectrum, the most significant effect of SalB was suppression of theta (9–15 Hz) in the KORD hemisphere (Supplementary Fig. 5c). Spontaneous theta was significantly depressed compared to baseline and compared to the control hemisphere (Supplementary Fig. 5d), which was not affected by SalB. These data confirm that KORD+SalB selectively suppresses behaviorally relevant neural activity in dorsal striatum.

Next, we expressed KORD bilaterally in pDMS or DLS of male rats before moderate avoidance training (Fig. 6a–c). Each rat received two avoidance test sessions after Vehicle or SalB injections (counterbalanced). Tests consisted of 15 presentations of the WS where shuttling did not affect WS duration, and no feedback stimuli or shocks were presented. Note that we confirmed high test-retest reliability before pursuing this within-subjects strategy with KORD (Supplementary Fig. 2). At the end of avoidance training, rats assigned to receive Vehicle-first or SalB-first displayed equivalent avoidance (unpaired t-tests; Fig. 6d: t 10 = 0.89, p = 0.3969, Fig. 6j: t 5 = 0.73, p = 0.4976), avoidance latencies (Fig. 6e; t 10 = 0.21, p = 0.8394, Fig. 6k: t 5 = 0.05, p = 0.9635), and freezing during the WS (Fig. 6f; t 10 = 0.39, p = 0.7022, Fig. 6l: t 5 = 0.77, p = 0.4774). During the tests, rats expressing KORD in pDMS performed fewer avoidance shuttles after SalB vs. vehicle injections (avoidance; Fig. 6g, Left; Two-way ANOVA, Block × Drug: F 1.92,21.08 = 4.02, p = 0.0347), especially during the first testing block (Fig. 6g, Right; paired t-test: t 11 = 2.34, p = 0.0393). Avoidance latencies were longer after SalB compared to Vehicle treatment, especially during early testing blocks, but this did not reach statistical significance (Fig. 6h, Left, Block × Drug: F 2.92,32.08 = 1.66, p = 0.1967, Right, t 11 = 1.71, p = 0.1147). Consistent with an impairment of avoidance, freezing to the WS was increased by SalB (Fig. 6i, t 11 = 2.36, p = 0.0377). In contrast, SalB did not affect moderately trained avoidance or WS freezing in rats expressing KORD in aDMS (Supplementary Fig. 6) or DLS (Fig. 6m, Left, Drug: F 1,6 = 0.09, p = 0.7777, Right, t 6 = 0.35, p = 0.7412, 6n, Left, Drug: F 1,6 = 0.26, p = 0.6303, Right, t 6 = 0.53, p = 0.6133, 6o, t 6 = 0.66, p = 0.5343). Further, SalB also did not affect avoidance in overtrained rats expressing KORD bilaterally in pDMS (Supplementary Fig. 6), presumably because stimulus-response associations stored outside pDMS maintain responding. This aligns with studies showing that pDMS lesions fail to impair habitual responding trained with food rewards19. Note that it is possible that pDMS inhibition indirectly impaired ARs after moderate training by facilitating competing freezing reactions. However, we view this as unlikely because pDMS inhibition had no effect on WS freezing or avoidance after overtraining (Supplementary Fig. 6), and because overtraining does not appear to change the neural circuits mediating Pavlovian freezing31,32. Together, these findings indicate that pDMS is critical for performance of goal-directed, but not habitual, ARs. They also suggest that DMS-mediated control of goal-directed avoidance occurs before habits develop (Fig. 6p–r). Note that KORD/SalB occasionally reduced shuttles during acclimation or intertrial intervals of avoidance tests; however, these effects were inconsistent and cannot explain the patterns of avoidance and freezing we observed during warning stimuli (Supplementary Fig. 7). SalB did not affect locomotor activity in a novel open field (Supplementary Fig. 8) or avoidance behavior in rats that never received the KORD virus (Supplementary Fig. 9).

DLS is required for habitual active avoidance
According to dual process accounts of instrumental behavior, goal-directed and habitual associations co-exist after overtraining and are expressed via parallel circuits that include pDMS and DLS. Thus, impairing DLS activity may not reduce overt avoidance behavior, as this will disinhibit goal-directed responding (Fig. 7p, q). To implicate DLS in habitual responding, a tool like outcome devaluation or contingency degradation is necessary to undermine goal-directed actions and isolate the habit circuit (Fig. 7s).

To test whether DLS contributes to avoidance habits, we expressed KORD bilaterally in DLS, overtrained all rats, and used paired counterconditioning to devalue FB-tones in half the rats (FB-Devalued group). The remaining rats received unpaired counterconditioning (FB-Valued group). As above, rats then received two Tests consisting of 15 WS presentations after counterbalanced Vehicle or SalB injections. At the end of avoidance overtraining, rats assigned to receive Vehicle- or SalB-first displayed equivalent avoidance and freezing behavior (Unpaired t-tests; Fig. 7d: t 8 = 0.42, p = 0.6836, 7e: t 8 = 0.23, p = 0.8206, 7f: t 8 = 0.46, p = 0.6555, 7j: t 11 = 0.71, p = 0.4911, 7k: t 11 = 0.53, p = 0.6042, 7l: t 11 = 0.73, p = 0.4782). In the FB-Valued group, SalB had no effect on avoidance (Two-way ANOVAs and Paired t-tests; Fig. 7g, left, Block: F 2.06,18.58 = 3.38, p = 0.0547, Drug: F 1,9 = 3.30, p = 0.1026, Block × Drug: F 2.65,23.84 = 0.74, p = 0.5221, right, Block 1: t 9 = 1.1, p = 0.3002), avoidance latencies (Fig. 7h, left, Block: F 2.81,25.27 = 2.21, p = 0.1145, Drug: F 1,9 = 3.30, p = 0.1027, Block × Drug: F 3.01,27.04 = 1.04, p = 0.3925, right, Block 1: t 9 = 1.76, p = 0.1127), or WS-freezing (Fig. 7i, Block 1: t 9 = 1.83, p = 0.1000). However, in the FB-Devalued group, SalB impaired WS-shuttles (Fig. 7m, left, Block: F 2.48,29.73 = 11.27, p < 0.0001, Drug: F 1,12 = 37.06, p < 0.0001, Block × Drug: F 2.28,27.33 = 1.94, p = 0.1591, right, Block 1: t 12 = 3.14, p = 0.0085), increased avoidance latencies (Fig. 7n, left, Block: F 2.53,30.35 = 11.01, p < 0.0001, Drug: F 1,12 = 48.33, p < 0.0001, Block × Drug: F 2.34,28.02 = 1.29, p = 0.2934, right, Block 1: t 12 = 3.35, p = 0.0058), and increased WS-freezing (Fig. 7o, Block 1: t 12 = 4.61, p = 0.0006). This pattern of results supports the hypothesis that parallel circuits in dorsal striatum control goal-directed vs. habitual ARs, with DLS being critical for habitual ARs (Fig. 7p–s).

Degrading AR-safety contingency impairs moderately trained avoidance in both sexes
In appetitive studies, goal-directed responding is also sensitive to action-outcome contingency degradation. For instance, rats barpressing for food will reduce responding if food is delivered for free (non-contingently), even if food remains available upon pressing33. We designed a contingency degradation protocol using male rats shuttling in a Sidman unsignaled active avoidance paradigm26. During Sidman avoidance, shocks are scheduled for delivery every 5s. Each shuttle produces a feedback cue and delays foot shock by 30s. Shuttles during response-shock intervals are coded as ARs and shuttles following a shock are coded as escape responses. Since there is no explicit warning signal, Sidman training encourages rats to shuttle regularly, which may be ideal for assessing contingency degradation. Response-produced feedback cues are known to improve learning and become safety signals with Sidman training34,35. After reaching training criterion, subjects were matched into equal Control and Contingency Degradation (CD) groups (Two-way ANOVA, 8b, left, Group: F 1,15 = 0.05, p = 0.8290). Control rats received four additional Sidman training sessions. CD rats received similar sessions, except additional safety signals (+30s of safety from shock) were delivered non-contingently (Fig. 8a). This reduced ARs by about 50% (Two-way ANOVA; Fig. 8b, right, Group: F 1,15 = 4.57, p = 0.0494), further implicating safety signals in goal-directed avoidance.

Contingency degradation may allow for the assessment of goal-directed avoidance in females because it preserves the value of the outcome (feedback = safety) and occurs in the training context. To test this, different experimenters conducted parallel experiments at two institutes using identical protocols and equipment. Results are combined in Fig. 8 and shown separately in Supplementary Fig. 10. Rats were first matched into equal groups based on baseline training (Fig. 8c, left, Group: F 1,35 = 0.0005, p = 0.9815). Like males, CD females emitted significantly fewer ARs during the contingency degradation phase (Fig. 8c, right, Group: F 1,35 = 13.34, p = 0.0008).

Since females shuttled at a higher rate than males, we next analyzed average performance during contingency degradation sessions to capture relevant sex differences (Group × Sex ANOVAs followed by Holm–Šídák’s multiple comparisons tests). Females emitted significantly more avoidance shuttles than males during the contingency degradation phase (Fig. 8d, Sex: F 1,50 = 20.91, p < 0.0001), though free safety signals reduced avoidance in both sexes (Fig. 8d, Group: F 1,50 = 13.92, p = 0.0005, Group × Sex: F 1,50 = 0.26, p = 0.61; CTL vs. CD, Males: p = 0.0577, Females: p = 0.0016). Contingency degradation also reduced escape shuttles (Fig. 8e, Group × Sex: F 1,50 = 4.56, p = 0.038; CTL vs. CD, Males: p < 0.0001, Females: p = 0.0002) and increased the delivery of feedback safety signals (Fig. 8g, Group: F 1,50 = 48.63, p < 0.0001, Group × Sex: F 1,50 = 0.20, p = 0.66; CTL vs. CD, Males: p = 0.0008, Females: p < 0.0001) but had little effect on the number of shocks delivered in both sexes (Fig. 8f, Group: F 1,50 = 0.81, p = 0.37, Group × Sex: F 1,50 = 0.009, p = 0.92; CTL vs. CD, Males: p = 0.63, Females: p = 0.61). Though the overall patterns were very similar, the higher rate of avoidance by females led to more safety signals (Sex: F 1,50 = 24.90, p < 0.0001), fewer shocks (Sex: F 1,50 = 23.38, p < 0.0001), and fewer opportunities to escape (Sex: F 1,50 = 10.18, p = 0.003). Similar patterns of ARs, escapes, shocks, and safety signals were observed at both institutes (Supplementary Fig. 10), indicating that contingency degradation robustly suppresses avoidance. These data suggest that females, like males, rely on feedback safety signals for goal-directed avoidance after moderate training.

Methods
Rats
All procedures were performed in accordance with the National Institute of Health Guide for the Care and Use of Experimental Animals and were approved by the NKI and TAMU Animal Care and Use Committees. Most studies were conducted at NKI with a subset conducted at TAMU (Fig. 8 and Supplementary Fig. 10). Subjects were female and male Sprague-Dawley adult rats weighing 250–300 g upon arrival (NKI: Hilltop Laboratory Animals, Scottsdale, PA, USA; TAMU: Envigo, Indianapolis, IN, USA). Rats were pair-housed in transparent plastic HEPA-filtered cages with cob bedding on a 12:12 h (NKI) or (14:10) light-dark cycle within temperature and humidity-controlled environments. Food and water were available ad libitum throughout the duration of the experiment. Animals were allowed a minimum of five days to acclimatize to the vivarium before all experiments. Following surgery, animals were singly housed for a minimum of one week and returned to pair-housing once recovered.

Shuttlebox apparatus
All equipment was from Coulbourn Instruments. Active avoidance training and tests occurred in two-way shuttleboxes (Model H10-11R-SC) with in sound-attenuating chambers (SAC Model: H10-24A). Shuttle chambers were divided into two identical sections (25.4 depth × 50.8 width × 30.5 cm height) by metal dividers, which contained a small passage (8.0 cm wide, 8.5 cm high) that allowed rats to move freely between sides. In one experiment, the passage connecting the shuttlebox chambers was blocked (door closed) during counter-conditioning and one Tone Fear Test via a metal insert held in place with magnets. A house light with small 0.5 W bulbs in each compartment dimly illuminated the shuttle-boxes (25 cm high). A PC computer installed with Graphic State software (v. 3 or 4) and connected to the chambers via the Habitest Linc System (Model H01-01) delivered 80 dB white noise warning stimuli, feedback tones (5 kHz, 80 dB), and footshock unconditioned stimuli during behavioral sessions. Warning stimuli and feedback were produced by an audio generator (Model A12-33) and presented through speakers (Model H12-01R) located on opposing walls in each chamber (20 cm high). Scrambled shock (Model H13-15) was delivered to a grid floor consisting of 32 stainless steel bars arranged in parallel to the dividers (Model H10-11R-XX-SF). Shuttling was automatically registered by infrared beam breaks and recorded throughout the sessions (Model H20-95-X). Behavior was monitored by security video cameras mounted on the ceiling of each shuttlebox chamber and recorded to DVD.

Counterconditioning (devaluation) apparatus
All equipment was from Coulbourn Instruments. Counterconditioning and some Tone Fear Tests took place in rodent fear conditioning chambers (28.5 depth × 26 width × 28.5 height cm, Model H10-11R-TC) enclosed by sound-attenuating cubicles (Model H10-24A) and dimly lit by a house light with a 0.5 W bulb (25 cm high). A computer, installed with Graphic State software (v. 3) and connected to the chambers via the Habitest Linc System (Model H01-01), delivered tones (5 s, 5 kHz, 80 dB) and shock stimuli during behavioral sessions. Tones were produced by an audio generator (Model A12-33) and presented through a single speaker (Model H12-01R) located in the front panel of the chamber (20 cm high). Scrambled shocks (Model H13-15) were delivered to a grid floor containing 18 stainless steel bars arranged in parallel (H10-11R-TC-SF). Animal movement was monitored by a security video camera mounted on the ceiling of the cubicle and recorded to DVD.

Two-way signaled active avoidance training procedure
Each session began in illuminated shuttle boxes with a 5-min stimulus-free acclimation. On the first day of training only, after acclimation, rats received a single Pavlovian trial consisting of the WS (15 s white noise) paired with footshock (0.5 s; 1.0 mA males, 0.7 mA females), followed by 30 active avoidance trials. The single Pavlovian trial is necessary to ensure that subsequent shuttles during the WS are responses under threat. The remaining days of training (day 2+) did not include the initial Pavlovian trial. After the acclimation period on subsequent training days (day 2+), rats received 30 active avoidance trials with an average intertrial interval of 2 min (random assignment without replacement, range 90–150 s). Trials consisted of an 80 dB white noise WS presentation that lasted a maximum of 15 s. Shuttling during the WS led to immediate WS termination, prevented delivery of the scheduled footshock (0.5 s; 1.0 mA males, 0.7 mA females), and triggered the delivery of a pure tone feedback cue (5 s, 5 kHz, 80 dB), or in the case of FB-controls (see below), a 5 s period of darkness (house lights turned off). If no avoidance shuttle was performed during the WS, the WS terminated, the inescapable shock was delivered, and no feedback was presented. ARs were defined as shuttles during the WS. After 30 avoidance trials were completed, animals were given a 60 s final period in the shuttle box before removal. Training (one session/day) continued for either 5 days (moderately trained) or 20 days (overtrained) with 5 sessions occurring per week. Rats failing to reach 70% successful avoidance by session 5 (poor avoiders) were removed from experiments because it is challenging to study the mechanisms of avoidance in rats that avoid sporadically or not at all. This is standard practice in studies of active avoidance memory/performance4,5,16,25,26,42, though the phenomenon of poor avoidance presents an obstacle to studies of avoidance learning.

Counterconditioning procedure
During the last five days of avoidance training (days 1–5 for moderately-trained rats, days 16–20 for overtrained rats), animals were placed in the illuminated counterconditioning apparatus and given 30 min. stimulus-free habituation sessions. It is our impression that this habituation phase is critical for obtaining effective counter-conditioning of safety signals. 1 d after the final active avoidance training session, animals were placed in the illuminated counterconditioning boxes for a 3 min. acclimation period. FB-Devalued and FB-control rats received 40 trials of the 5 s tone (5 kHz, 80 dB) that co-terminated with a 0.5 s footshock (0.7 mA for females, 1.0 mA for males; avg. inter-trial interval: 180 s, random assignment without replacement, range 150–210 s). For the FB-control group, the tone was novel since they were avoidance-trained with a visual feedback cue. FB-Valued rats received 40 tones and shocks in an explicitly unpaired fashion. Trial 1 began with a footshock, with tones and shocks alternating thereafter. Shocks occurred on average every 3 min, with placement closer to the onset of the next tone than the termination of the previous tone (to prevent trace conditioning). Shock-to-tone intervals ranged from 28 to 88 s (mean: 58 s) and tone-to-shock intervals ranged from 87 to 147 s (mean: 117 s). All intervals were selected from lists at random without replacement. Total session times for all animals were approximately 125 min. Two experiments with female rats conducted counterconditioning in the shuttle boxes using only 20 tones & shocks (Paired vs. Unpaired, Supplementary Figs. 3 and 4). These sessions were the same as the standard procedure except the total session duration was shorter (~65 min).

Avoidance tests
All avoidance tests were conducted 10 min. after i.v. injections with 1 ml/kg injection volumes. For within-subject KORD studies, rats were injected with Vehicle or SalB (counterbalanced). For other purely behavioral experiments, rats were injected with Vehicle before both tests. This was done to control for stress before testing in all studies, and to allow team members to practice i.v. injections before the critical KORD studies. After acclimation in the shuttle boxes (avoidance training context) for 5 min, rats received 15 presentations of the 80 dB, 15 s WS (mean inter-trial interval: 120 s, random assignment without replacement, range 90–150 s). Shuttling was monitored throughout the session but had no effect on WS duration (no WS-termination). There were also no feedback (tones or house light-off) or footshock presentations. Avoidance tests occurred twice, with 1 d off between tests. Note that feedback stimuli were omitted during the final tests to ensure that avoidance performance reflected a memory of the outcome value rather than reinforcement/punishment during the test phase73.

Tone fear tests
After all devaluation tests were completed, rats from behavioral experiments were placed in a novel open field (90 × 90 × 40 cm) for a 1 min acclimation phase followed by 3 presentations of a 5 kHz, 80 dB, 60 s tone cue (1 min inter-trial intervals; Fig. 5 and Supplementary Fig. 4). Video was recorded to DVD using a digital camera and freezing responses were rated offline. Some rats were also tested in the shuttle boxes where avoidance training originally occurred using an identical protocol with the shuttlebox door open or closed (Fig. 5 and Supplementary Fig. 4). Some rats were subsequently tested in the fear conditioning boxes used for feedback counterconditioning/devaluation (Fig. 5) using an identical protocol. Tone fear tests occurred 2–3 d after the final Devaluation Test, with only one test/day (2–3 d interval between tests). Note that tone fear tests were only used to validate our devaluation procedure and were not conducted for subsequent KORD studies in males.

Open field locomotion tests
17 rats received two locomotion tests in the open field after counterbalanced Vehicle or SalB injections (i.v., 10-min pretreatment). Tests were conducted after the completion of all other behavioral tests and were separated by 48 h. Lines divided the open field into 36 square (15 × 15 cm) zones. Line crossings, defined as entry of the head into an adjacent zone, were counted for 15 min.

Sidman unsignaled avoidance and contingency degradation procedures
Rats were placed in the shuttle box, with offset of the house light indicating the beginning of the session. Shock (1.0 mA for males, 0.7 mA for females) was delivered every 5 s (shock-shock, or S-S interval) unless the subject shuttled to the opposite chamber. Every shuttle earned a feedback tone (0.5 s, 70 dB, 2 kHz) and a 30 s shock-free period (response-shock, or R-S interval). Shuttles during S-S intervals were considered escape responses and shuttles during R-S intervals were considered ARs. If the subject failed to shuttle during S-S or R-S intervals, shocks resumed every 5 s. All subjects received Sidman training until reaching “good avoider criterion” (2 consecutive sessions with 20 or more ARs)26. Rats failing to reach this criterion by session 10 were deemed poor avoiders and eliminated from the study. Session duration was 25 min. In this task, avoidance is expressed as the total number of ARs per session rather than a percentage. This is necessary because the number of Sidman trials varies depending on each rat’s behavior.

Contingency Degradation (CD) phase: Following Sidman avoidance training, animals were non randomly assigned to Control or CD groups based on comparable levels of criterion performance. Control animals received four more days of Sidman training, while experimental animals received four days of Sidman training with the action-outcome contingency degraded. For the latter, all parameters were the same as the Sidman protocol except during the shock-free period (regardless of whether it was earned by escape or avoidance), every 5 s there was a 1/3 chance that the rat would receive a “free” feedback safety signal (and additional shock-free period) in a way that was not contingent on behavior.

Surgery for DREADD experiments
Rats were anesthetized with a mixture of ketamine (100 mg/kg, i.p.) and xylazine (10 mg/kg, i.p.) and placed in a stereotaxic apparatus (David Kopf Instruments). Supplemental doses of the mixture were given as needed to maintain a deep level of anesthesia. Brain areas were targeted using coordinates from the Paxinos and Watson Brain Atlas. The skull was exposed and small burr holes were drilled above the target area. Rats were bilaterally injected with 0.5 µL AAV using a 1.0 µL Hamilton Neuros syringe (Hamilton Co.) at a rate of 0.1 µL/min. Following infusion, the syringe was left in place for an additional 5–8 min to allow fluid diffusion away from the needle tip before withdrawing. AAV injections consisted of a 1:10 cocktail of Cre [AAV5-CMV-HI-eGFP-Cre.WPRE.SV40 (Addgene plasmid no. 105545) packaged at UPenn Vector Core 2.5 × 10^14 GC ml−1] and KORD [AAV8-HSyn-DIO-HA-KORD-IRES-mCitrine (Addgene plasmid no. 65417), packaged at UPenn Vector Core, 8.9 × 10^13 GC ml−1], respectively. Injections were made into the anterior dorsomedial striatum (aDMS; y=+0.5 mm anterior to Bregma, x=+/− 2.1 mm lateral to midline, z=−4.5 mm ventral from skull surface), the posterior dorsomedial striatum (pDMS; y=−0.12 mm, x=+/− 2.45 mm, z=−4.9 mm), or the dorsolateral striatum (DLS; y=+0.75 mm, x=+/− 3.5 mm, z=−5.1, −5.6 mm). Immediately following surgery, animals were sutured (Ethicon Vicryl, Fisher Scientific), injected with analgesic (Buprenorphine-ER (s.c., 0.6–1.2 mg/kg), Wedgewood Pharmacy), and allowed to recover on a heat pad. Once awake and alert, rats were returned to the vivarium in a fresh, clean cage to recover for one week singly housed. After one week, rats were returned to pair housing for at least 14 additional days before behavioral training.

Drug injections
Salvinorin B (SalB, 2.5 mg/mL, Sigma) was dissolved in Vehicle (7% DMSO and 50% PEG in ddH2O) under low heat. Animals were restrained briefly and intravenously injected (i.v., lateral tail vein) with Vehicle or 2.5 mg/kg SalB (1 ml/kg volume) 10 min prior to all devaluation/AR Tests. Treatments (Vehicle vs. SalB) and order of treatment (Vehicle first vs. SalB first) were counterbalanced for all KORD studies evaluating DMS and DLS. Animals with poor or unsuccessful injections were removed from the study.

Implant surgery for electrophysiology
3–4 weeks after unilateral DMS viral infection using techniques and coordinates described above, 4 rats were surgically anesthetized with isoflurane, and a topical anesthetic (1% xylocaine) was administered locally around the wound site. Temperature was maintained at 37°C throughout the surgery using a heating pad. Animals were placed in a stereotaxic device and the skull was exposed. A 2-channel telemetry transmitter (HD-X02, Data Sciences International) was inserted under the skin of the back and stainless-steel electrodes (100 µ diameter, Teflon-coated, A-M Systems) connected to the telemetry leads were implanted in the DMS bilaterally targeting the region of KORD infection in the injected hemisphere and the same location in the contralateral uninfected hemisphere. Reference electrodes were implanted in the posterior cortex. Electrodes were attached to the skull with bone-anchor screws and dental cement and the scalp sutured closed. Following recovery from anesthesia, animals were singly housed and allowed to recover for at least 1 week before recording.

In vivo recording and analysis
Recordings were performed in the animal’s home cage by placing the cage on a telemetry receiver. Bilateral local field potentials were recorded in the DMS (amplification 200×, low-pass filtered at 200 Hz) and digitized (1000 Hz) for recording and analysis with Spike2 software (CED, Inc). Following at least 15 min of baseline recording, rats were injected with SalB (2.5 mg/kg, 1 ml/kg, i.v.) and returned to their home cage (<5 min post-injection) for continued recording of spontaneous activity for at least 30 min. LFP data were analyzed with FFT (2.4 Hz/bin) from at least 10 min. of continuous data pre-injection and starting at least 15 min. post-injection. The post-injection FFT spectrum was normalized to the baseline recording from that same hemisphere, providing a measure of SalB induced change in the infected DMS and the control, uninfected DMS within animals. Following the recording, animals were perfused for histological confirmation of viral expression and electrode placement. In one animal the electrode in the control hemisphere missed the DMS and entered the lateral ventricle, thus only data from the infected hemisphere of this animal was used in the analysis (Supplementary Fig. 3a–d).

Histology and immunohistochemistry
After behavioral testing was completed, DMS and DLS animals were anaesthetized with an overdose mixture of ketamine (100 mg/kg, i.p.) and xylazine (10 mg/kg, i.p.) and perfused transcardially with 4% paraformaldehyde (PFA) in 0.01 M phosphate-buffered saline (PBS). The brains were fixed in 4% PFA solution for 24 h, then post-fixed in 30% sucrose at 4°C for 48–72 h. Brains were then blocked and sectioned on a microtome (SM2000R, Leica) at 40 µm. These sections were kept in 0.01 M PBS+0.05% sodium azide (NaAz) until immunohistochemistry. Brain sections were blocked in 1% bovine serum albumin (BSA; Sigma) in 0.1 M PBS for 30–60 min at room temperature (RT) to block non-specific binding. KORD-injected brain sections were incubated overnight at RT in Rabbit Anti-GFP Polyclonal Primary Antibody (Invitrogen, #A-11122) in 1% BSA+0.2% Triton X-100 in 0.01 M PBS (1:1000) for verification of viral expression in neurons. Following primary antibody incubation, sections were rinsed with agitation three times for 5 min each in 0.01 M PBS at RT. Sections were then incubated in Goat Anti-Rabbit IgG (H+L) Cross-Adsorbed Secondary Antibody, Alexa Fluor 488 (Invitrogen, #A-11008) in 0.01 M PBS (1:200) at RT for 30–60 min. In some experiments primary and secondary antibody steps were combined using Alexa Fluor 488-tagged GFP Polyclonal Antibody, (Invitrogen, #A21311). After secondary incubation, sections were rinsed with agitation three times for 5 min each in 0.01 M PBS at RT, mounted on gelatin-coated slides, cover-slipped in Permount (Fisher, #SP-15) or Prolong Gold Antifade Reagent (Invitrogen, #P36930) and allowed to cure overnight at RT. Slides were placed at 4°C for long-term storage. Sections were imaged using a Leica TCS SP8 confocal microscope (Leica) or Olympus VS120 fluorescent microscope (Olympus). For all experiments, animals were excluded from analysis if virus expression was insufficient, too extensive, or outside regions of interest.

Assessment of freezing
Videos recorded during counterconditioning and testing sessions were analyzed for freezing by at least one rater who was blind to treatment condition when possible. Blinding was not possible for counterconditioning sessions where reactions to shock were obvious. Freezing was defined as cessation of all non-respiratory movement and is presented as a percentage of the interval assessed. For counterconditioning sessions, freezing was scored for the first 5 trials during pre-tone (5 s) or tone (Paired: 4.5 s, Unpaired: 5 s) intervals. For Devaluation and AR Tests sessions, freezing was scored during 15 s WS presentations for the first 3 trials (Block 1). For Tone Fear Tests, freezing was scored during each 60 s tone presentation.

Statistics and reproducibility
Data are typically presented as mean±standard error of the mean (s.e.m.) or mean plus individual values. Most data were analyzed using GraphPad Prism (v. 10.0.3) software. Figures were created with GraphPad Prism, Adobe Illustrator, Inkscape 1.2, and Biorender.com. Error bars in graphs represent ±s.e.m. Unpaired Student t-tests were used to compare means between two groups. Paired Student t-tests were used for within-subject comparisons at a single time point (e.g., Block 1 histograms). For comparing multiple groups at a single time point, ordinary ANOVAs were conducted, assuming equal standard deviations per group, followed by Dunnett’s test for comparisons to a common group or Tukey’s test for comparisons between all groups (single pooled variance). For comparing groups across multiple time points and training conditions, two- and 3-way repeated measures ANOVAs were performed with no assumption of sphericity (Geisser-Greenhouse correction). For experiments comparing multiple groups/timepoints to a common mean, this was followed by Dunnett’s test, with individual variances computed for each comparison to a mean of interest. For experiments with two groups, this was followed by Šídák’s multiple comparisons test, with individual variances computed for each comparison. For some sex difference analyses, Two-way ANOVAs without repeated measures were followed by Holm-Šídák’s multiple comparisons test (e.g., Fig. 8). To evaluate variability in avoidance latencies among animals across sessions (Fig. 1g), eta-squared (η2) was calculated for successful avoidance trials to determine effect size, where η2 ≤ 0.06, 0.07–0.14, and η2 > 0.14 indicated small, medium, and large effect sizes, respectively. For each animal, a significant large sized session effect (F 19,570 = 31.090, p < 0.0001, η2 = 0.18) was apparent. We therefore calculated coefficients of variation (CV) to assess changes in variability across sessions for males and female rats. Here, a repeated measures mixed effects model (REML) was used since elimination of failure trials led to missing values. This was followed by Dunnett’s post-tests comparing all session means to the session 5 mean, when avoidance reached asymptote. Here, CV is inversely related to stereotyped behavior. Effect size calculations were completed using R statistical software. P values < 0.05 were considered statistically significant.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "Sears et al., conducted a series of behavioral and chemogenetic experiments to demonstrate that active avoidance responses (AR) driven by safety signals exhibit sensitivity to devaluation. Furthermore, they observed that goal-directed versus habitual ARs are mediated differentially by dorsomedial (DMS) and dorsolateral (DLS) striatum, mirroring the neural substrates underlying instrumental action vs. habit associated with reward.\nMale and female rats underwent training in a two-way shuttle avoidance task involving a warning signal (WS) followed by active avoidance of scheduled shock, and the presentation of a safety signal (FB cue) upon a successful avoidance response (AR). After different training durations (5 vs. 20 sessions) aimed at promoting either outcome-driven or habitual action, the safety signal was devalued through counterconditioning (in another context). Subsequently, animals were placed back in the shuttle box and presented with 15 WS in the absence of FB or shock to assess the effects of devaluation on AR. While males exhibited sensitivity to FB devaluation following the shorter training (reduced AR, but increased freezing to WS), females failed to show this effect.\nChemogenetic inhibition of the dorsal medial striatum (DMS), particularly the posterior region, impaired ARs (and increased freezing to WS) during WS-only extinction sessions after moderate AR training. To demonstrate the disruption of habitual responding with DLS inhibition, the FB evaluation procedure was now introduced after extensive AR training, and DLS inhibited animals were found to be sensitive to devaluation.\nFinally, an unsignaled avoidance task revealed diminished responses in both sexes following contingency degradation, highlighting the importance of the association between AR and FB.\nThis is a compelling set of findings that potentially provide novel mechanistic insights into the behavioral and neural control of instrumental active avoidance, and will likely be of interest to an audience interested in the neurobiology of learning, anxiety, and function of the striatum. The statistical analyses are mostly appropriate, and the methodology is clearly written and reproducible on the whole, although more clarification and details are warranted in some parts. I have also outlined some major concerns including the need for further control experiments/groups that would need to be addressed for me to fully assess the validity of the claims.\nGiven the primary aim of the FB devaluation procedure is to assess changes in active avoidance behavior in response to the devalued safety signal, the omission of the safety signal during the extinction test sessions is a major limiting factor in evaluating the data. The claims would be greatly strengthened by the inclusion of the safety signal in the test.\nThe authors noted that SalB produced non-specific effects including locomotor depression –a SALB only control group that is typically included to control for off target effects is notably absent in this study, and should be justified, or included.\nI appreciate that the focus of the paper is on safety signals driving AR, but the paper would benefit from more consideration and discussion of the contribution of the WS to AR acquisition. The level of AR is evidently much lower in the non-signalled AA task – i.e., the absence of a WS (barely 50% in control males in Fig 6b), which speaks to the importance of WS in driving AR as well. The authors imply that the WS elicits a competing behavior, such as freezing. However, it would be crucial to demonstrate freezing behavior in response to the WS throughout active avoidance response (AR) acquisition, rather than solely during extinction sessions. More importantly, it is also unclear if AR behavior is attenuated after pDMS inhibition due to an impairment in goal directed AR driven by FB memory (as per the authors’ account) or due to an equally viable possibility that WS processing has become disrupted (potentiated freezing).\nIn general, the statistical reporting is minimalistic, and needs to be expanded to allow confirmation of the absence of effects. In many places, F stats are reported, with no accompanying explanation of which factors are involved. Here, I provide examples of such incidences:\n1) Figure 4D – F statistics for Block x Drug appears to be reported as being non-significant, but the F statistics for Drug needs to be reported as there seems to be a difference in the number of shuttles between drug vs. vehicle conditions.\n2) Ext Figure 2 illustrates the inter-test reliability; however, the statistical analyses do not directly compare performance between the two tests. A visual inspection of the performance in Tests 1 and 2 suggests potential differences that require statistical validation or clarification with Test as an additional factor particularly for Block 1. For instance, the performance of FB control rats in Block 1 in panels a and d, and performance of the males in Block 1 in panel b across the two tests are notably different. Could repeated testing be contributing to some seemingly different vehicle group performance between experiments (e.g., Ext Fig 3 h & m)?\n3) Fig 6 – there seems to be sex differences in avoidance shuttles in the unsignaled AA task, which could be investigated further with statistical analysis.\nThe concept of a safety signal serving as both a positive reinforcer and conditioned inhibitor is complex, and not very well differentiated in the introduction and discussion. The study presents evidence of the inhibitory properties of the tone safety signal, as demonstrated by the retardation of a new tone-foot shock pairing and the observed summation effect (freezing(context + safety signal) < freezing(context)). However, it is important that the authors clearly articulate how their study supports the claim that the tone safety signal serves as a positive reinforcer to the animals.\nMinor points\n- What was the magnitude of the shock given on the first day of Shuttlebox training\n- The results of tone fear tests in various contexts are provided for some experiments, and not others.\n- What was the justification for the 5min acclimatization period in each AA training session?\n- The decision to omit error bars for within-subject comparisons should be reconsidered. The line graphs in particular should contain some indicators or variance since individual data points are not shown.\n- Subjective adjectives such as ‘slightly’ should be voided when describing statistically significant results – e.g., in describing female data in text and Ext Fig 1\n- Why were the tone intervals during which freezing was scored different between the Paired and Unpaired groups (4.5 vs. 5s)?", "reviewer_2": "This manuscript by Sears and colleagues reports an interesting set of findings. The authors have designed a clever method for devaluing a safety signal (that signals successful avoidance) by pairing that safety signal with footshock. In addition, the authors adapt the classic contingency degradation procedure by providing non-contingent safety cues, although the training for this second procedure is distinct from the first design in which a discriminative stimulus signaled the opportunity to avoid shock. There are some interesting findings here that will be of interest to specialists in avoidance learning, as well as to the broader fields concerned with aversion and fear, animal learning, and functions of striatal circuitry. These findings additionally have relevance for understanding behavioral disorders in humans related to fear and avoidance. While the concept and experimental design are potentially of interest, and the topic is highly significant, the manuscript was somewhat burdensome to read because of the acronymns and the lack of clear presentation of the statistical analyses.\nSome of these issues are detailed below.\n1. There are aspects of data analysis and presentation that are not clear, missing, or confusing. First, the authors divide subjects into good and poor performers. There is no justification for such a stratification. Do these subjects improve following overtraining?\n2. Second, stats reported in the figure legends do not give the complete information. For example F-values are reported without significance levels. I recommend moving the statistical results to the results text so that appropriate text can be used around the statistical values to assist the reader. For example, some of the figures might suggest that there are significant interactions that would be of interest, but I can’t tell from the text, therefore I can not appropriately evaluate some of the findings.\n3. The training procedures used for safety-signal devaluation study and the contingency degradation procedure are different. Therefore they provide convergent evidence in general, but it is not possible to apply the neural findings to the contingency degradation task, something that should be considered. As well, it is not clear how well the neural findings will generalize to females. The possible lack of extension of the neural findings thus should be discussed in the Discussion.\n4. All of the acronymns make the reading slower as the reader must continually translate. Recommend avoiding those acronymns that are not broadly known in neuroscience. The figures could also benefit from more textual information in the panels themselves so readers understand what they are looking at.\n5. For figure 3, the histology images and schematics are too small to see.\n6. The US active avoidance/contingency data, while interesting, are disconnected from the rest of the study because they are placed last, after the neurobiology and the section on the females. The flow of the paper is also impacted by the placement of the data on the females, which is also placed after the striatal manipulations. You could change the order of the experiments – Cont degr, then devalutation, then neurobiological manipulations logically focusing on the males since they are the group showing short vs long training distinctions."}, "author_response": "We thank the reviewers for their thoughtful comments and suggestions. We address each point below. As requested, we made substantial changes to the order of data presentation and added statistics to the main text. We provide two versions of the revised manuscript. One with changes tracked and a clean version that is easier to read.\nReviewer 1\nSears et al., conducted a series of behavioral and chemogenetic experiments to demonstrate\nthat active avoidance responses (AR) driven by safety signals exhibit sensitivity to devaluation.\nFurthermore, they observed that goal-directed versus habitual ARs are mediated differentially by\ndorsomedial (DMS) and dorsolateral (DLS) striatum, mirroring the neural substrates underlying\ninstrumental action vs. habit associated with reward.\nMale and female rats underwent training in a two-way shuttle avoidance task involving a\nwarning signal (WS) followed by active avoidance of scheduled shock, and the presentation of a\nsafety signal (FB cue) upon a successful avoidance response (AR). After different training\ndurations (5 vs. 20 sessions) aimed at promoting either outcome-driven or habitual action, the\nsafety signal was devalued through counterconditioning (in another context). Subsequently,\nanimals were placed back in the shuttle box and presented with 15 WS in the absence of FB or\nshock to assess the effects of devaluation on AR. While males exhibited sensitivity to FB\ndevaluation following the shorter training (reduced AR, but increased freezing to WS), females\nfailed to show this effect. Chemogenetic inhibition of the dorsal medial striatum (DMS),\nparticularly the posterior region, impaired ARs (and increased freezing to WS) during WS-only\nextinction sessions after moderate AR training. To demonstrate the disruption of habitual\nresponding with DLS inhibition, the FB evaluation procedure was now introduced after extensive\nAR training, and DLS inhibited animals were found to be sensitive to devaluation. Finally, an\nunsignaled avoidance task revealed diminished responses in both sexes following contingency\ndegradation, highlighting the importance of the association between AR and FB.\nThis is a compelling set of findings that potentially provide novel mechanistic insights into the\nbehavioral and neural control of instrumental active avoidance, and will likely be of interest to an\naudience interested in the neurobiology of learning, anxiety, and function of the striatum. The\nstatistical analyses are mostly appropriate, and the methodology is clearly written and\nreproducible on the whole, although more clarification and details are warranted in some parts. I\nhave also outlined some major concerns including the need for further control\nexperiments/groups that would need to be addressed for me to fully assess the validity of the\nclaims.\n1. Given the primary aim of the FB devaluation procedure is to assess changes in active\navoidance behavior in response to the devalued safety signal, the omission of the safety signal\nduring the extinction test sessions is a major limiting factor in evaluating the data. The claims\nwould be greatly strengthened by the inclusion of the safety signal in the test.\nIn the early stages of this project, we agreed with the reviewer on this point. However, after\nconsulting with experts on devaluation like Andrew Delamater, we were persuaded that this is\nincorrect. Observing an avoidance decrement when the devalued reinforcer is omitted is more\ncompelling evidence that animals are using a cognitive strategy. If the devalued reinforcer is\npresented after each avoidance response, animals may stop responding due to punishment and\nS-R mechanisms cannot be ruled out. This is explained more clearly in Dickinson & Balleine’s\ninfluential 1994 paper Motivational control of goal-directed action:\n“…it is important to note that the test was conducted in “extinction”; neither outcome was\nactually presented during the test. If Adams and Dickinson had actually presented the food\npellets contingent on lever-pressing during the test, this devaluation effect could have been\nexplained in terms of the direct suppressive effects of the now-aversive positive outcome. By\ntesting in extinction, however, they ensured that the differential performance of the two groups\nmust have reflected the interaction of knowledge of the instrumental contingencies acquired\nduring initial training with some representation of the status of the outcome as a goal following\ndevaluation treatment.”\nWe added the following text to the Methods section to make our rationale for extinction tests\nclearer: “Note that feedback stimuli were omitted during the final tests to ensure that avoidance\nreflected a memory of the outcome value rather than reinforcement/punishment during the test\nphase1.”\n2. The authors noted that SalB produced non-specific effects including locomotor depression –a\nSALB only control group that is typically included to control for off target effects is notably\nabsent in this study, and should be justified, or included.\nThis is an important point that we did not adequately discuss in the original manuscript. We\naddress this in the revised manuscript in two ways. First, we tested a subset of rats from the 5-\nday DMS and 20-day DLS KORD experiments in the open field after counterbalanced Vehicle or\nSalB injections using a within-subjects design (n=17 Males). This was the last test conducted\nbefore brains were removed for histology. We left this out of the original submission because\nsome rats were ultimately excluded from the avoidance analyses (poor avoiders, poor bilateral\nKORD expression, mistargeted KORD expression). However, the data show no indication of\nimpaired locomotion after a 10-minute delay, suggesting that occasional suppressive effects of\nSalB after i.v. injections are transient and expressed mainly in the homecage. Second, we also\nincluded data in the initial submission showing many null effects on avoidance and freezing after\nSalB treatment (now Figs. 6, 7, and Extended Fig. 6). Although we occasionally observed fewer\nshuttles during the acclimation or ITI periods of avoidance tests, these were inconsistent and did\nnot show a pattern that could explain avoidance/freezing effects. For instance, SalB reduced\nacclimation shuttles in only two of six experiments (KORD in pDMS vs aDMS after moderate\ntraining). Of these two, SalB affected avoidance and freezing only in the pDMS experiment. The\nopposite profile was also observed; SalB affected avoidance and freezing in overtrained rats\nwith KORD in DLS after devaluation (Fig. 8) but had no effect on acclimation shuttles. A similar\nstory can be told about ITI shuttles. Together, these results strongly suggest that non-specific\neffects of SalB on locomotor behavior are transient and cannot explain the patterns we observe\nfor avoidance and freezing during test sessions. In this revision, we include new figures that\npresent the acclimation, ITI and Open Field data to address this issue more directly (Extended\nFigs. 7,8).\n3a. I appreciate that the focus of the paper is on safety signals driving AR, but the paper would\nbenefit from more consideration and discussion of the contribution of the WS to AR acquisition.\nThe level of AR is evidently much lower in the non-signalled AA task – i.e., the absence of a WS\n(barely 50% in control males in Fig 6b), which speaks to the importance of WS in driving AR as\nwell.\nThis appears to be a misunderstanding. Unsignaled (Sidman) active avoidance is expressed as\nthe total number of avoidance responses while signaled avoidance is expressed as a\npercentage. This is because the number of trials in the Sidman procedure necessarily varies for\neach rat depending on its behavior (every shuttle delays shock by 30s). We added the following\ntext to the Sidman methods section to clarify this: “In this task, avoidance is expressed as the\ntotal number of avoidance responses (shuttles) per session rather than a percentage. This is\nnecessary because the number of Sidman trials varies depending on each rat’s behavior.”\n3b. The authors imply that the WS elicits a competing behavior, such as freezing. However, it\nwould be crucial to demonstrate freezing behavior in response to the WS throughout active\navoidance response (AR) acquisition, rather than solely during extinction sessions.\nIn earlier publications, we demonstrated that WS-freezing is initially high and dramatically\ndeclines as avoidance is acquired2-4. We agree that we should show that WS-freezing is\nequivalent in our groups prior to manipulations. To address this, we analyzed WS-freezing for all\nexperiments during the final training sessions (Session 5 for moderate training experiments and\nSession 20 for overtraining experiments). WS-freezing was generally low at the end of\navoidance training and did not differ between groups in any experiment. These data are now\nincluded in the figures.\n3c. More importantly, it is also unclear if AR behavior is attenuated after pDMS inhibition due\nto an impairment in goal directed AR driven by FB memory (as per the authors’ account) or due to\nan equally viable possibility that WS processing has become disrupted (potentiated freezing).\nThis is an excellent point. We now highlight this possibility for the reader and argue that this\ninterpretation is unlikely. We added the following text to our discussion of pDMS inhibition: “Note\nthat it is possible that pDMS inhibition indirectly impaired avoidance after moderate training by\nfacilitating competing freezing reactions. However, we view this as unlikely because pDMS\ninhibition had no effect on WS-freezing after overtraining (Extended Fig. 6), and because\novertraining does not appear to change the neural circuits mediating Pavlovian freezing5,6.”\n4. In general, the statistical reporting is minimalistic, and needs to be expanded to allow\nconfirmation of the absence of effects. In many places, F stats are reported, with no\naccompanying explanation of which factors are involved. Here, I provide examples of such\nincidences:\nAll raw data and complete statistical analyses are included in the accompanying Figshare excel\nfile. We attempted to strike the right balance between statistical reporting and readability within\nthe journal’s word limit by using a recent Nature Neuroscience article on avoidance as a guide7.\nHowever, both reviewers found this inadequate, so we have added the requested statistics and\nreported full analyses in the main text and extended figures.\n4a. Figure 4D – F statistics for Block x Drug appears to be reported as being non-significant, but\nthe F statistics for Drug needs to be reported as there seems to be a difference in the number of\nshuttles between drug vs. vehicle conditions.\nFull statistics are now reported throughout the manuscript. For Figure 4d (left), now Figure 7g\n(left), the main effect for Drug was not statistically significant (F = 3.30, p=0.103).\n1,9\n4b. Ext Figure 2 illustrates the inter-test reliability; however, the statistical analyses do not\ndirectly compare performance between the two tests. A visual inspection of the performance in\nTests 1 and 2 suggests potential differences that require statistical validation or clarification with\nTest as an additional factor particularly for Block 1. For instance, the performance of FB control\nrats in Block 1 in panels a and d, and performance of the males in Block 1 in panel b across the\ntwo tests are notably different. Could repeated testing be contributing to some seemingly\ndifferent vehicle group performance between experiments (e.g., Ext Fig 3 h & m)?\nThis is an excellent point, and we now include a much more extensive analysis in Extended Fig.\n2 that addresses both within-session and between-session extinction during Avoidance Tests.\nWe also directly address the low level of avoidance behavior in Ext. Fig. 3h (compared to 3m,\nnow Ext. Fig. 6d,i) with the following text describing the results:\n“During the final tests, rats expressing KORD in aDMS showed no differences in avoidance or\nfreezing after receiving SalB vs. Vehicle injections [stats]. This aligns with appetitive studies\nshowing that aDMS is not necessary for goal-directed responding17. However, avoidance\nresponding was unusually low after Vehicle treatment in this study so this null effect should be\ninterpreted with caution.”\n4c. Fig 6 – there seems to be sex differences in avoidance shuttles in the unsignaled AA task,\nwhich could be investigated further with statistical analysis.\nThere are sex differences in the unsignaled avoidance results, though the overall patterns are\nvery similar. We now report these in the text and figure (now Fig. 9).\n5. The concept of a safety signal serving as both a positive reinforcer and conditioned inhibitor\nis complex, and not very well differentiated in the introduction and discussion. The study\npresents evidence of the inhibitory properties of the tone safety signal, as demonstrated by the\nretardation of a new tone-foot shock pairing and the observed summation effect\n(freezing(context + safety signal) < freezing(context)). However, it is important that the authors\nclearly articulate how their study supports the claim that the tone safety signal serves as a\npositive reinforcer to the animals.\nWe agree that this topic is complex and made several revisions in the Introduction and\nDiscussion to add some clarity. We added text emphasizing that prior studies already support\nthe hypothesis that response-produced safety signals increase avoidance behavior (generic\npositive reinforcement). However, many of these studies assumed positive reinforcement of\nstimulus-response associations that mediate habits. Our contribution is the demonstration that\navoidance can be goal-directed, that the goal is safety, and that devaluation of safety signals\ncan differentiate between goal-directed and habitual ARs. Impairing avoidance by devaluing a\nresponse-produced outcome is additional evidence for positive reinforcement, but it also goes\nfurther to demonstrate goal-directed control by an action-outcome association. Summation and\nRetardation tests were used only to confirm that feedback stimuli were safety signals\n(conditioned inhibitors) after avoidance training. Currently it is unknown how fear-inhibitory\nproperties of safety signals relate to positive reinforcement.\nIn the Introduction: A less studied hypothesis is that ARs are positively reinforced by response-\nproduced safety signals that are generated via their inverse relationship with the warning\nstimulus and shock during avoidance training7,10-12. Explicit feedback cues that immediately\nfollow ARs are known to develop conditioned inhibition (safety) and reinforcing properties that\npromote avoidance10,13-16. Here, “reinforcing” is used in a general way, though many early\nstudies assumed strengthening of stimulus-response associations. It is unknown, however,\nwhether safety signals behave as valued outcomes supporting goal-directed avoidance via\naction-outcome associations, akin to food in a barpressing experiment. To our knowledge, no\nactive avoidance studies have attempted to devalue response-produced safety signals. To\nexamine this, we designed a devaluation protocol where AR feedback cues are\ncounterconditioned by pairing with shock. We also present data on a novel contingency\ndegradation test, where non-contingent safety signals are presented during avoidance.\nIn the Discussion: Although negative reinforcement theories dominated during the heyday of\nactive avoidance research, the idea that safety signals may positively reinforce avoidance was\nintroduced as early as 1936 by Konorski and Miller. This was echoed by Soltysik, who called\navoidance feedback a “reward” that “stabilizes avoidance behavior and makes it virtually\nindependent of the noxious [unconditioned stimulus]”8. Studies since then are consistent with\nthe notion that response-produced safety signals improve or “reinforce” avoidance but were not\ndesigned to implicate goal-directed behavior9-14. Our finding that devaluation impairs moderately\ntrained avoidance during extinction tests (no shock or feedback) is the first direct demonstration\nthat response→feedback associations can control avoidance.\nIn the Discussion:Our results provide evidence for positive reinforcement in avoidance, but we\nacknowledge several limitations. First, “positive reinforcement” is an umbrella term that has\nbeen used to describe response-produced outcomes that promote goal-directed and/or habitual\ninstrumental behavior. We implicate safety signals in goal-directed avoidance, but it remains\nunclear if these also provide the teaching signal that stamps in stimulus-response associations\nunderlying habitual avoidance. One recent study suggests they do15. We also cannot rule out\nadditional roles for WS-termination and/or US-omission in the negative reinforcement of ARs,\nespecially early in training before feedback cues develop inhibitory properties. However,\nconditioned inhibition likely develops faster than is currently appreciated (Fig. S1 in reference16)\nand we suspect that ARs become safety-directed within a few trials. Also, for safety signals to\nbe considered true positive reinforcers, they should support new response learning in the\nabsence of shocks. One study failed to find evidence for this17, but more work is needed to\ndetermine if this is generally true or if the task needs optimization. Studies in rodents and\nhumans show that response-produced safety signals during avoidance induce dopaminergic\nprediction error signals and pleasant feelings like other positive reinforcers18-21.\nMinor points\n- What was the magnitude of the shock given on the first day of Shuttlebox training\nFor all experiments, shock magnitude was 1.0 mA for males and 0.7 mA for females. We added\nthis information to the methods describing Day 1 of shuttlebox avoidance training.\n- The results of tone fear tests in various contexts are provided for some experiments, and not\nothers.\nAll tone fear tests that we conducted are included in the manuscript. Our initial plan was to\nconduct tone fear tests only in a neutral context (open field). However, when devaluation was\nineffective in moderately trained females, we added additional tests in the counterconditioning\nboxes and shuttleboxes for overtraining experiments that were ongoing and took longer to\ncomplete. This is how we discovered that counterconditioning was effective in both sexes but\nonly transferred to other contexts in males. Once our devaluation protocol was validated in\nmales, there was no need to conduct tone fear tests in the KORD experiments. We added the\nfollowing sentence to the Methods to clarify: “Note that tone fear tests were only used to validate\nour devaluation procedure and were not conducted for subsequent KORD studies in males.” In\nthis revision, we present tone fear tests as standalone figures to make this clearer (new Fig. 5\nand Ext. Fig. 4).\n- What was the justification for the 5min acclimatization period in each AA training session?\nThis is somewhat arbitrary and was chosen to match prior work.\n- The decision to omit error bars for within-subject comparisons should be reconsidered. The\nline graphs in particular should contain some indicators or variance since individual data\npoints are not shown.\nWe now include standard error bars in all line graphs depicting group means.\n- Subjective adjectives such as ‘slightly’ should be voided when describing statistically\nsignificant results – e.g., in describing female data in text and Ext Fig 1\nGood point. We deleted these adjectives throughout the manuscript.\n- Why were the tone intervals during which freezing was scored different between the Paired\nand Unpaired groups (4.5 vs. 5s)?\nThe 0.5s shock co-terminated with the tone in the Paired group but was presented separately\nfor the Unpaired group. We scored freezing during the full 5s tone for the Unpaired group but\nstopped scoring when the shock elicited movement in the Paired group.\nReviewer 2\nThis manuscript by Sears and colleagues reports an interesting set of findings. The authors\nhave designed a clever method for devaluing a safety signal (that signals successful avoidance)\nby pairing that safety signal with footshock. In addition, the authors adapt the classic\ncontingency degradation procedure by providing non-contingent safety cues, although the\ntraining for this second procedure is distinct from the first design in which a discriminative\nstimulus signaled the opportunity to avoid shock. There are some interesting findings here that\nwill be of interest to specialists in avoidance learning, as well as to the broader fields concerned\nwith aversion and fear, animal learning, and functions of striatal circuitry. These findings\nadditionally have relevance for understanding behavioral disorders in humans related to fear\nand avoidance. While the concept and experimental design are potentially of interest, and the\ntopic is highly significant, the manuscript was somewhat burdensome to read because of the\nacronymns and the lack of clear presentation of the statistical analyses.\nSome of these issues are detailed below.\nWe have reduced the use of acronyms and moved key statistical analyses into the Results\nsection. We have also added Results-style text to the extended figures to improve the reading\nexperience.\n1. There are aspects of data analysis and presentation that are not clear, missing, or confusing.\nFirst, the authors divide subjects into good and poor performers. There is no justification for\nsuch a stratification. Do these subjects improve following overtraining?\nPoor avoiders sometimes improve with additional training; however, we have never attempted to\novertrain them. For this project, we were interested in how neural circuits may change when\nanimals who acquire avoidance at similar rate continue to encounter the threat and emit the\nresponse. The simplest way to do this was to study the large segment of the population that\nreadily acquires avoidance responses. We are also hesitant to expose animals to repeated\nshocks when they show little evidence of learning the task. This is not an issue for the good\navoiders as they experience few shocks after acquiring the response. We are very interested in\nthe poor performer population and study them in a separate project that does not involve\novertraining. For instance, it appears that many in this group learn the instrumental contingency\nbut are unable to avoid due to excessive freezing reactions. We can “rescue” avoidance\nresponding after moderate training with central amygdala lesions2,3,22 or propranolol23. Studies\nof the precise neural mechanisms of poor avoidance are ongoing24 and we view this as an\nimportant, but separate, question.\n2. Second, stats reported in the figure legends do not give the complete information. For\nexample F-values are reported without significance levels. I recommend moving the statistical\nresults to the results text so that appropriate text can be used around the statistical values to\nassist the reader. For example, some of the figures might suggest that there are significant\ninteractions that would be of interest, but I can’t tell from the text, therefore I can not\nappropriately evaluate some of the findings.\nWe have added more complete descriptions of the statistical analyses to the Results section as\nrequested by both reviewers. We also provide the data and statistical analyses for all figures in\nthe accompanying figshare Excel file.\n3. The training procedures used for safety-signal devaluation study and the contingency\ndegradation procedure are different. Therefore they provide convergent evidence in general,\nbut it is not possible to apply the neural findings to the contingency degradation task,\nsomething that should be considered. As well, it is not clear how well the neural findings will\ngeneralize to females. The possible lack of extension of the neural findings thus should be\ndiscussed in the Discussion.\nWe agree and have added the following text to a new “limitations” section in the Discussion:\n“…Finally, the question of whether females also rely on parallel circuits in dorsal striatum for\navoidance remains open. Our contingency degradation data suggest that females also rely on\nsafety signals for goal-directed avoidance, but this paradigm used a different avoidance task\nand method for manipulating safety. Thus, it cannot be assumed that the neural findings from\nmale devaluation studies apply to females.”\n4a. All of the acronymns make the reading slower as the reader must continually translate.\nRecommend avoiding those acronymns that are not broadly known in neuroscience.\nWe have greatly reduced the use of acronyms in the main text, though many remain in the\nfigures and are defined in the captions.\n4b. The figures could also benefit from more textual information in the panels themselves so\nreaders understand what they are looking at.\nWe redesigned the figures with more textual information to aid the reader.\n5. For figure 3, the histology images and schematics are too small to see.\nWe increased the size of all histology images and brain schematics.\n6. The US active avoidance/contingency data, while interesting, are disconnected from the rest\nof the study because they are placed last, after the neurobiology and the section on the\nfemales. The flow of the paper is also impacted by the placement of the data on the females,\nwhich is also placed after the striatal manipulations. You could change the order of the\nexperiments – Cont degr, then devalutation, then neurobiological manipulations logically\nfocusing on the males since they are the group showing short vs long training distinctions.\nThank you for this comment. We agree that the flow in the original submission was not ideal. We\nmoved the female devaluation data to the beginning of the paper where it is presented along\nwith the male data. The contingency degradation data are presented last as this paradigm is\nless developed and sets up a discussion of future directions. We feel these data are important\nbecause they strongly suggest that males and females rely on the same mechanism for goal-\ndirected avoidance and the sex difference only presents an issue for using devaluation as a tool\nto identify goal-directed behavior."}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have sufficiently addressed all my previous concerns, bar one extremely critical one that must be explicitly acknowledged as a severe limitation of the study and/or, ideally, addressed with additional controls: the absence of a SalB-only control group in animals that do not express KORD. This is a fundamentally important control in chemogenetics experiments, without which the strength of the causal claims of the striatal subregions made in the manuscript are substantially weakened. While the authors now include new behavioral data following SalB administration and show no consistent effect on shuttling or locomotion, these comparisons remain limited to KORD-expressing animals. Thus, it remains challenging to disentangle non-specific effects of SalB from effects mediated via KORD activation.\nLines 163-4: Please provide the p value for the comparison of WS-induced freezing between FB-control and FB-valued groups as there seems to be a difference there. I could not access the excel data sheet.\nLine 147: Please insert the word ‘equally’ before ‘high levels of avoidance responding’"}, "author_response": "We thank the reviewer for their suggestions. We address each point below. We also provide a revised manuscript file with changes tracked in reviewer mode.\nReviewer 1\n1. The authors have sufficiently addressed all my previous concerns, bar one extremely critical\none that must be explicitly acknowledged as a severe limitation of the study and/or, ideally,\naddressed with additional controls: the absence of a SalB-only control group in animals that do\nnot express KORD. This is a fundamentally important control in chemogenetics experiments,\nwithout which the strength of the causal claims of the striatal subregions made in the manuscript\nare substantially weakened. While the authors now include new behavioral data following SalB\nadministration and show no consistent effect on shuttling or locomotion, these comparisons\nremain limited to KORD-expressing animals. Thus, it remains challenging to disentangle non-\nspecific effects of SalB from effects mediated via KORD activation.\nWe conducted the requested control experiments using the same design as the KORD\nexperiments, except the rats never received the KORD virus. We found no effect of SalB on\navoidance or freezing to the warning stimulus. This was true for both moderately trained rats\nand overtrained rats subjected to feedback devaluation or the control procedure. These data are\npresented in Supplementary Figure 9, along with relevant statistical analyses in the caption.\nRaw data and analyses are also included in the accompanying figshare Excel data file. We also\nmade the following changes to the manuscript text, highlighting these findings:\nLines 306-308: SalB did not affect locomotor activity in a novel open field (Supplementary Fig.\n8) or avoidance behavior in rats that never received the KORD virus (Supplementary Fig. 9).\nLines 507-512: Although SalB is reported to be inert, we noticed effects of SalB injections in\nsome male rats, including some naïve rats not expressing KORD. Post-injection effects\nobserved in the home cage included reduced locomotor activity, squinting, and labored\nbreathing that did not occur with vehicle injections. These appeared to subside quickly (10-20\nminutes) and SalB did not impair locomotor activity in a novel open field (Supplementary Fig. 8)\nor avoidance behavior in rats that never received the KORD virus (Supplementary Fig. 9).\n2. Lines 163-4: Please provide the p value for the comparison of WS-induced freezing between\nFB-control and FB-valued groups as there seems to be a difference there. I could not access\nthe excel data sheet.\nOur original analysis used Dunnett’s tests to compare the FB-Devalued group to both control\ngroups. Thus, there was no comparison between the FB-Valued and FB-Control groups to\nreport. To address this question, we re-analyzed these data using Tukey’s tests that provide p\nvalues for all comparisons. The difference between FB-Valued and FB-Control rats is not\nstatistically significant (p=0.10). We now report this in the manuscript:\nLines 162-165: This was also true in our experiment as FB-Devalued rats froze significantly\nmore than both control groups during WS presentations (Fig. 2g; ANOVA: F =10.42,\n2,24\np=0.0006, Tukey’s tests: FB-Devalued vs. FB-Control: p=0.02, FB-Devalued vs. FB-Valued:\np=0.0004, FB-Valued vs. FB-Control: p=0.10).\nFor consistency, we also conducted Tukey’s tests to compare all groups in Figures 2e and 2f.\nThis did not change the pattern of results and these new analyses are also included in the\nrevised manuscript and accompanying figshare Excel data file.\n3. Line 147: Please insert the word ‘equally’ before ‘high levels of avoidance responding’\n‘equally’ has been added to Line 147"}, {"round": 3, "reviewer_comments": {"reviewer_1": "The authors have addressed my previous concerns. My one remaining comment is that the authors should avoid uncorrected Fisher LSD post hoc tests (line 734) and instead use corrected tests (e.g., Holm, Tukey)to maintain statistical rigor."}, "author_response": "We thank the reviewer for their suggested change to improve the rigor of our statistical\nanalyses. We provide a revised manuscript file with changes tracked in reviewer mode.\nReviewer 1\n1. The authors have addressed my previous concerns. My one remaining comment is that the\nauthors should avoid uncorrected Fisher LSD post hoc tests (line 734) and instead use\ncorrected tests (e.g., Holm, Tukey) to maintain statistical rigor.\nWe agree and have reanalyzed the data depicted in Figs. 8d-g using Holm-\nŠídák's multiple comparisons tests. The new analyses altered the p values for some\ncomparisons but did not change the pattern of results. To reflect his, we updated the Results\nsection (lines 360-371), Methods section (line 737), Fig. 8 asterisks, and the full statistical\nanalyses reported in the accompanying Figshare raw data file."}]

Decision letter