Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment
Authors: Christopher I Moore
Michael J Frank
Arif A Hamid
Journal: Cell (2021)
Discipline: MOLECULAR BIOLOGY & GENETICS
Keywords: agency learning; basal ganglia; credit assignment; decisionmaking; dopamine
Affiliations: Department of Neuroscience, Brown University, Providence, RI 02912, USA; Carney Institute for Brain Science, Brown University, Providence, RI 02912, USA. Electronic address: ., Department of Cognitive Linguistics & Psychological Sciences, Brown University, Providence, RI 02912, USA; Carney Institute for Brain Science, Brown University, Providence, RI 02912, USA. Electronic address: .

Abstract:  Significant evidence supports the view that dopamine shapes learning by encoding reward prediction errors. However, it is unknown whether striatal targets receive tailored dopamine dynamics based on regional functional specialization. Here, we report wave-like spatiotemporal activity patterns in dopamine axons and release across the dorsal striatum. These waves switch between activational motifs and organize dopamine transients into localized clusters within functionally related striatal subregions. Notably, wave trajectories were tailored to task demands, propagating from dorsomedial to dorsolateral striatum when rewards are contingent on animal behavior and in the opponent direction when rewards are independent of behavioral responses. We propose a computational architecture in which striatal dopamine waves are sculpted by inference about agency and provide a mechanism to direct credit assignment to specialized striatal subregions. Supporting model predictions, dorsomedial dopamine activity during reward-pursuit signaled the extent of instrumental control and interacted with reward waves to predict future behavioral adjustments. 

Retracted: No

PubPeer comments:
 I have some questions about panels F G in figure 4. I can t grasp how these panels show the same data (as claimed in the figure caption for G), considering that there seems to be much more overlap in the distribution of data points in F than reflected by the histograms in the polar plot in G. Moreover, I converted 1.57 radians (about the mean of all the data in F by my eye) and got 90 degrees. If I understand the inset in E right, 90 degrees would be pointing in posterior direction. With most of the points closer to 90 (1.57 rad) degrees than to 0 (0 rad) or 180 (3.14 rad), I would predict the histograms to be arranged around the posterior direction of the polar plot; an arrangement very different from what is shown in G. ![file] , 
 Dear Musa Serpentina, Thank you for your question. The data displayed in panels D, E, F, and G are all the same data. Panel G is the rawest angular data, and E F are actually processed (remeasured according to the description in the methods section, see quoted below). You can also see more unprocessed data from this example session in Panel D colormap, and also vector quivers in the right portion of D. Perhaps what may specifically help in reconciling your  visual reaction to F and G is to realize that the linearization or remeasuring is not direction-specific (see methods, but also indicated by the arrows in the inset of E). That is, a vector that falls in the top quadrants is re-measured clockwise, and if it were in the lower two quadrants, the angle is remeasured counterclockwise. This is like folding G about the X-axis. You would get an artificial inflation (doubling) of the quantity of angles observations near the midline, whereas those at the extremes medial lateral vectors do not. So, most of the overlap in F (near 1.57 rads ~90deg) are trials with average wave-direction further from the medial or lateral vectors (looking at quiver plots in the right portion of D may help). Anyway, I hope these answers help you interpret the data better. Please feel free to reach out at arifhamid.DA@gmail.com so I can provide you with actual numbers that go into that plot, and you can remeasure the unit vectors yourself to be more convinced. Ultimately, these angular transformations are meant to facilitate parametric statistical comparisons between conditions as such tests are not always straightforward to implement or interpret for angular data. Warm wishes, Arif Methods text from the paper:  While most figures show the angular orientation of wave directions relative to the imaging field of view (i.e., keeping AP ML consistent, e.g., Figures 2H, 3G, 3H, and 4G), we also utilized a linearization of wave directions to specifically quantify the extent of medial or lateral directionality of DA reward waves. To achieve this, the frame vector angle is remeasured relative to the medially oriented vector (u = u22121, v = 0) without regard to clockwise counterclockwise directionality (e.g., Figures 3I inset and 4E insets). This yielded relative wave-angles that were small if oriented in the medial direction (i.e., LM waves) and larger relative-angles for laterally oriented wave direction (i.e., ML waves), as shown in Figure 4E. , 
 #2 So are you trying to establish in E and F that the average wave directions under the Pavlovian and Instrumental conditions are not exactly opposite? I too find the presentation and explanation confusing. , 
 Dear Arif, It s my understanding that the following means you found the smallest absolute angle between the medial direction vector (0 degrees) and each data point: the frame vector angle is remeasured relative to the medially oriented vector (u = u22121, v = 0) without regard to clockwise counterclockwise directionality (e.g., Figures 3I inset and 4E insets). This yielded relative wave-angles that were small if oriented in the medial direction (i.e., LM waves) and larger relative-angles for laterally oriented wave direction (i.e., ML waves) I do not entirely understand your explanation. What is clear to me is that the smaller angle for the largest bar in the polar plot is for a bin spanning approximately 0 to 15 degrees. 15 degrees are about 0.26 radians. However, there is no data in plot F between 0 and 0.26 radians. I ve illustrated this in the attached file. ![file] Since we already had contact via email and you know who I am, please feel free to send me the data and, if you don t mind, the code. It might help me understand what I m missing. Regards , 
 **The text below and attached figure summarizes the email exchange with  Musa Serpentina clarifying exactly how the analysis was performed and his response via email afterward.** -- On Apr 30, 2021, at 22:17,  Arif A. Hamid arifhamid.da@gmail.com wrote: Dear X# %, Here is a more direct clarification. There is more detail thanu00a0you are appreciating, and nuance to why we did the linearization. The rose polar plots are meant to show the trueu00a0**angular**u00a0orientation distribution of the optic flow results. In contrast, the linearization moves the data into a slightly different coordinate system with the goal of assessingu00a0**medial-ness**u00a0-vs-u00a0**lateral-ness** of the wave directions. This is the u2018relative wave angle quantity plotted in Fig4E,F, H-K etc. The linearized data is related to, but not the same as the angular data so one may run into interpretation issues if not careful (b c not natural to linearize circularu00a0data). I hope you understand this broader point. 1) **Rose polar plots in the paper:** 4G is evaluatedu00a0the same way 3G 3H bottom right angular distribution panels are is evaluated. That is:u00a0 a) The optic flow analysis yieldsu00a0vectors oriented in the direction of detected flow for each pixel of each frame (eg. Fig 2B). For each frame, we calculated an average, net,u00a0**frameVector.** b) To do an analysis surrounding a 1sec window after reward, we took vectors in N frames corresponding to 1s, from T trials with rewards. So we have a matrix of NxT frameVector. c) For each trial, average the vectors across frames. End up with **trialVectors** with size T. d) angle(trialVector) will yield **trialAngles**u00a0with -pi to pi radians. Plot this in a rose polar plot and label with Medial Lateral Anterior Posterior labelu00a0asu00a0in 4G 3G 3H. hope this makes sense. 2) **Linearized angular data:** NOTE: The linearization (or relative angle calculation) is applied **framewise** to the NxT variableu00a0frameVectoru00a0from step 1b above. That is: a) Take the matrix of frameVector with size (N(frames) x T(trials).u00a0(this is what legend refers to as same data ). b) re-measure theu00a0angleu00a0of each vector relative to the Medially oriented vector,u00a0as shown in inset of 4E. This results in another matrix ofu00a0**frameRelativeAngles**u00a0(0 to pi radians) of size NxT. c) Commute the mean angle (circ_mean function) ofu00a0frameRelativeAnglesu00a0across frames for each trial. This yieldsu00a0**trialRelAngle**u00a0of size T,u00a0ranging in value from 0 to pi. This is what is plotted in 4E 4F. I hope these steps are straightforward to follow. I am sure you realize that there are many decisions that go into data analysis, to the extent that they are suited to make specific points in a broader narrative. We have spent a lot of time scrutinizingu00a0many alternatives to every analysis in every figure in the paper. So at the very least, you understand the distinction betweenu00a0**why**u00a0we are doing the linearization, andu00a0**whether**u00a0it replaces substitutes the original directional data.u00a0 Moreover, I think you assumed that the variableu00a0trialVectoru00a0from 1c was simply remeasured and plotted in 4F. Our goal was to compute a framewiseu00a0medial-nessu00a0-vs-u00a0lateral-nessu00a0score, so the remeasuring is performed on each frameVector. To satisfy your curiosity, here is a plot side-by-side of the two methods (left is original in the paper, the right plot is if remeasure as you expected), our method is a bit more conservative in yielding separation between the distributions. ![file] Please reach out again if you have more questions. Arif ---------------- On May 1, 2021, at 10:18, X# % AKA  Musa Serpentina wrote: Dear Arif, I indeed assumed that  same data meant you plotted the relative angle of the trialVectors (trialVectors being the data points, although converted to degrees, shown in G) in radians in 4F, instead of computing  new trial averages over frame-wise computed *relative* angles. In your second plot, where you replot literally the same data points as in G, I do see data even in the 0-15 degree area (0-0.26 radians), thus replicating what s shown in the polar rose plot. I feel this is a very important clarification of your methodology and you should post it on pubpeer, because I can t imagine others will not be confused about the lack of data around the extremes in F, considering that the rose plot does show data in bins  in direct contact with the lateral medial axis (i.e. less than 0.26 or more than 2.9 radians from the medial vector). This would seem like a very blatantly obvious problem to anyone who takes the time to convert between degrees and radians. I cc ed everybody else who s been following this again. Hope you don t mind. Thanks for taking the time to go into more detail and covering the remeasuring of the frame vectors for me. Regards, ,