DOI: 10.5061/dryad.3xsj3txq8
title: Informativeness, contingency and time scale invariance in associative learning

description: Contemporary theories guiding the search for neural mechanisms of learning
 and memory assume that associative learning results from the temporal
 pairing of cues and reinforcers resulting in coincident activation of
 associated neurons, strengthening their synaptic connection. While
 enduring, this framework has limitations: Temporal-pairing-based models of
 learning do not fit with many experimental observations and cannot be used
 to make quantitative predictions about behavior. Here we present
 behavioral data that supports an alternative, information-theoretic
 conception: The amount of information that cues provide about the timing
 of reward delivery predicts behavior. Furthermore, this approach accounts
 for the rate and depth of both inhibitory and excitatory learning across
 paradigms and species. We also show that dopamine release in the ventral
 striatum reflects cue–predicted changes in reinforcement rates consistent
 with subjects understanding temporal relationships between task events.
 Our results reshape the conceptual and biological framework for
 understanding associative learning.
description: Complete Methods are provided in the
 article: <strong>Learning Depends on the Information Conveyed by
 Temporal Relationships Between Events and is Reflected in the Dopamine
 Response to Cues. <br></strong>
 <strong>Briefly:<br></strong>
 Male, Sprague-Dawley rats were housed in groups of two in a
 colony room on a 12:12 hour light:dark cycle. Water was available ad-lib
 in the home cages. They were fed in their home cages for one hour after
 experimental sessions, which occurred 5 days per week. On weekends, they
 had ad-lib access to food until approximately 22 hours before the first
 weekday session. They were approximately two months old at the start of
 training and had been handled for one week before that.  They were trained
 in eight identical experimental chambers (30.5 cm x 24.1 cm x 21.0 cm)
 located in ventilated and soundproof boxes. Each chamber was equipped with
 a speaker, a house light, and a pellet dispenser (Model ENV-203, Med
 Associates), which delivered 20mg pellets into a head-entry-detecting
 trough (Models ENV-200R7 and ENV-254-CB, Med Associates). They initially
 received 2 sessions of magazine training, during which 40 pellets were
 delivered at random times during a 20-minute session (random time 30s
 schedule), followed by daily sessions with one of the experimental
 protocols. The time of occurrence of each head entry and the time of onset
 and termination of all stimulus events were recorded with 0.1s
 resolution.
description: # Informativeness, contingency, and time scale invariance in associative
 learning
 [https://doi.org/10.5061/dryad.3xsj3txq8](https://doi.org/10.5061/dryad.3xsj3txq8) This dataset includes the raw data from experiments conducted in the Laboratory of Peter Balsam at the New York State Psychiatric Institute between 2013 and 2017. Experimental details and results are presented in the article: **Learning Depends on the Information Conveyed by Temporal Relationships Between Events and is Reflected in the Dopamine Response to Cues**. ## Description of the data and file structure ## Guide to Raw Data Long Table: 10,092,421 row x 10 column CSV file. The time-stamped event codes in the final two columns (the raw data); the first 8 columns specify experiment, group, and protocol parameters * Col 1: The experiment number (6 experiments)Col 2: The group/protocol ID: 26 groups, but fewer protocols, because several protocols formed part of more than one experiment—see the 6 parameter columns * Cols 3:8 The parameters of the protocol * Col 3: Number of sessions * Col 4: Total number of CSs (“trials”) * Col 5: CS duration (s) * Col 6: Mean intertrial-interval (ITI) duration (s) * Col 7: Mean pellet-pellet (US-US) interval (s); only during ITIs except in truly random group in Experiment 1 * Col 8: Session duration (m) * Col 9: cumulative (across-sessions) time in the experimental chamber (s) * Col 10: Event code Sessionstart           113 Sessionend            114 Fanon                    117 Fanoff                    118 Redlighton             41 Redlightoff            31 ITIStart                121 ITIend                    122 Trialstart               111 Toneon                  61 Toneoff                  51 Pelletdelivered      21 Pokestart               1011 Pokeend                1001 ## Guide to Regression Tables There is one regression table for each of the 6 experiments in the 6 so-named files: E1RegressionTable.mat, E2RegressionTable.mat, …E6RegressionTable.mat. Each file contains a table variable named **RegT**. **These are different Matlab™ table variables. Do not load 2 or more of these files into the workspace at once, because loading a 2nd file will overwrite the RegT table already in the workspace!** The column headings describe the variable in that column: Exp *experiment #*; Grp *# of the group within the experiment i;* Inf *informativeness of the protocol; *FindPkRtDfs *the difference between the poke rate during the CSs and the poke rate during the ITIs, computed using the cumulative counts and times for the entire training time (all training sessions)*; lgUSsToAcq *the common log of the number of pellets delivered at the 5 different acquisition criteria: i) the maximum of the cumulative poke rate difference (first appearance of greater poke rate during ITI than during CS), ii) p&lt;.1, iii) p&lt;.05, iv) p&lt;.01, v) p&lt;.001; *lgUSsBtwMxSg *the difference between the log of the pellets delivered when the last (most stringent) of the 5 criteria was met and the number delivered when the first criterion was met; *lgTmtoAcq *the cumulative training time when the 5 criteria were met;* lgTmBtwMxSg *same as for # of pellets but for elapsed time*; lgTrlsToAcq *same as for previous 2 columns but trials rather than # of pellets or time; *MxPkRtDiffs * the maximum value of the cumulative poke rate difference—the # of pellets delivered or the time or the trial # gave the first of the 5 values in the preceding 3 columns. *When a subject did not reach the acquisition criterion, their cells for USsTOAcq, lgUSsBtwMxSg, TmToAcq, lgTmBtwMxSg, Trls to Acq are empty. The file named MasterRegressionTable contains the variable named RegTmaster.mat table variable, which is the concatenation of the 6 RegT tables into a Matlab™ table variable. The following additional columns have been inserted between Cols 2 & 3 of the RegT tables: GrpNm *the name of a group within the Experiment structure that contains the data and results from that group*; nses *the # of training sessions*; sesdur *the duration of a session*; CSdur *the duration of a CS*; ITIdur *the average duration of the exponentially distributed ITIs*; US_US *the average duration of the interval between two USs; except for the group Rnd in Experiment 1, USs were delivered only during ITIs and the intervals between them are in ITI time, that is, the intervals measured on a clock that ran only during ITIs. * The MasterRegressionTable.xlsx file contains the Excel version of the MasterRegressionTable, which was produced by first applying Matlab’s splitvars command to split the three 5-col “columns” in the Matlab table into separate columns. When a subject did not reach the acquisition criterion, their cells for USsTOAcq, lgUSsBtwMxSg, TmToAcq, lgTmBtwMxSg, Trls to Acq are empty. The first 8 columns in the MasterRegressionTable.xlsx file (Experiment #, group #, and protocol parameters) are highlighted in yellow. ## Sharing/Access information Links to other publicly accessible locations of the data: * [https://osf.io/sfwbu/](https://osf.io/sfwbu/) ## Code/Software Guide To Code December 2022 The data were analyzed using the custom Matlab™ Toolbox TSsystem created by Adam King and C.R. Gallistel for the analysis of data in the form of a 2-column array with time stamps in the first column and (numerical) event codes in the 2nd column. The raw data for a given experiment and all the results derived from it are stored in a 3-level Matlab™ structure variable, which must be named Experiment when it is in the workspace for the commands in the TSsystem to recognize it. The first level in the Experiment structure is the Experiment level; the second is the Subject level; the third is the Session level. The raw data from the MedPC files have been read into the TSDdata field at the Session level. In the Structures folder, there are 6 Experiment structures, each named Experiment and stored in a file named E[#]: E1, E2 …E6. The files named E[#]m, etc, contain the same structures but are renamed to E1, E2…E6. When you want to have all 6 Experiment structures in the workspace at the same time, load from these E1[#]m. When you load from the  E1, etc, files the variable that comes into the workspace is named Experiment and it overwrites whatever variable with that name may already be there. To have different Experiment structures in the workspace at the same time, all but one of them must *not* be named ‘Experiment’. Thus, when you want TSsystem commands to operate on one of them, you must rename it from e.g. ‘E1’ to ‘Experiment’. In doing so you, overwrite whatever Experiment variable may already be there!!! The TSfunctions (commands) are in the TSlib subfolder of the Code folder. It must be on Matlab’s search path when using them. The helper functions are in the Helper Functions folder, which must also be on the search path In the Code folder, there is a Matlab™ script for each of the 6* *experiments: ScriptE1, ScriptE2…ScriptE6. They use the custom functions in the TSsystem. The custom functions (aka commands) in the TSsystem read data and intermediate results from fields in this structure. The fields from which they read are specified in the second input argument. The input fields must all be at the same level. The TSfunctions write results to other fields at the same level as their input fields. The fields to which results are written are specified in the first input argument of a TSfunction. Its third argument is often a helper function. The scripts are in the Code folder. The 6 scripts are divided into **Cells**. A cell performs a single conceptual step in the analysis. The code in Cell 1 of a script concatenates the raw data in TSData fields at the Session level into a Field named tsd at the Subject level. This this field the time stamps cumulate across sessions, so a time stamp for an event is the total training time at which that event occurred. **Cell 4** in each script does the lion’s share of the data analysis with a *TSapplystat* command that calls the *Cums* helper function. The Cums helper function generates the following outputs, all of which are put into similarly or identically named fields at the Subject level: 1) Cms, 2) AcqTms, 3)FinDif, 4)USsToAcq, 5)TrlsToAcq, 6)TmToSig, 7)M, 8)FinDifLstN, 9) FinCSpkrate The Cms output goes into the Cums field at the Session level. It is the same length as the tsd field because it contains a row for every row in that 2-col raw-data array. The Cums field has 19 columns: They contain the following variables (in the order listed): Col  Variable     Description   1'   T_C         cumulative training time   2   #USs        cumulative USs   3   T_CS        cumulative USs during CSs   4   USs|CS     cumulative USs during ITIs (intertrial intervals)   5   T_ITI        cumulative ITI time   6   USs|ITI    cumulative USs during ITIs   7   lam_R|C   contextual reinforcement rate #USs/T_C   8   lam_R|CS      CS reinforcement rate  USs|CS/T_CS   9   lam_R|ITI      ITI reinforcement rate   USs|ITI/T_ITI  10  n_r|C        # pokes in context (total# of pokes)  11  n_r|CS      # pokes during CSs  12  n_r|ITI      # of pokes during ITIs  13  lam_r|CS     poke rate during CSs  n_r|CS/T_CS  14  lam_r|ITI     poke rate during ITIs  n_r|ITI/T_ITI  15  nDkl(lam_R|CS||lam_R|C)   *n*DKL for CS reinf rate vs contextual reinf rate  16  nDkl(lam_R|ITI||lam_R|C)   *n*DKL for ITI reinf rate vs contextual reinf rate  17  nDkl(lam_r|CS||lam_r|ITI)    *n*DKL for CS poke rate vs ITI poke rate  18  nDkl(lam_r|ITI||lam_r|CS     *n*DKL for ITI poke rate vs CS poke rate  19  cumsum(lam_r|CS–lam_r|ITI)}  cumulaltive difference in the CS & ITI poke rates The AcqTms,  USsToAcq, and TrlsToAcq outputs from the Cums helper function, which go into the fields with the same names, have 5 cols each, one column for each of 5 different acquisition criteria. In order of conservativeness, these criteria are: the maximum of the cumulative difference function (estimates onset of acquisition), and four increasingly stringent criteria on the strength of the evidence that acquisition has occurred (p&lt;.1, p&lt;.05,p,p&lt;.01 & p&lt;.001). The TmToSig output, which goes into the field with that name, is AcqTms(5:2) – AcqTms(1). In other words, the time it takes the evidence for acquisition to go from the initial appearance of acquisition to definitive evidence that it has occurred. The FinDif output is the final difference between the estimate of the CS poke rate and the estimate of the ITI poke rate (lam_r|CS – lam_r|ITI, where these estimates take all the data. The FinDifLstN output, in the field of the same name, is the same statistic but based only on the data from the last 4 sessions. The M output, which goes into the MxCmPkRateDiff field is the maximum cumulative poke rate difference; the longer the CS poke rate remains greater than the ITI poke rate, the larger this maximum is, because once the ITI poke rate estimate becomes consistently greater than the CS poke rate, this difference trends consistently downwards far into negative values. **Cell 7** generates plots of the cumulative poke rate difference as a function of the number of reinforcements. On the same plots are the nDkl for acquisition and vertical lines marking the 5 acquisition criteria (reinforcements to acquisition). One can study these plots to judge whether we have correctly estimated the acquisition **Cell 13** computes the within-CS response rate profiles and puts them in the PstAcqwiCSpkProfile field at the Subject level **Cell 14**  Aggregates them by group into group-labeled profile fields at the Experiment level **Cell 15**  plots the profiles for each group in an Experiment The results from all 6 experiments combined are combined and analyzed in the InhibAll.m script in the Code folder. Cell 1: Concatenates the 6 individual regression tables (each named RegT) into a master regression table named RegTmaster, which is saved in a file named MasterRegressionTable in the Structures folder. This table has 1 row for each of the 222 subjects Cell 2: Creates a cell array with the names that groups have within their Experiment structures. Fields with these names are fields 14 to 17 or 18 in the list of fields at the Experiment level in any Experiment structure **Cell 3:** Inserts the following columns between Col 2 and Col 3 of RegTmaster: nses, sesdur, CSdur, ITIdur (an average) US-US interval (also an average), the name of the group to which the subject belonged, the informativeness as computed from the values at the end of the Cums field for the rate of reinforcement during ITIs and the contextual rate of reinforcement Cell 4: Computes best-fitting regression parameters for a linear model of the dependence of the final poke rate difference as a function of log10(iota; log2(iota) gives the amount of information in bits that the offset of the CS transmits to a subject and plots them in the leftmost subplot of a 2-subplot figure. Also computes the nonlinear model for the dependence of pellets to acquisition on log10(iota):\ log10(pellets to acquisition) = log10(k/iota). This is the model that Gibbon & Balsam (1981) fit to their pigeon autoshaping data. Cell 5: **Creates** figure with 2 subplots showing the data and the regression lines for the regression models computed in Cell 4.