# [Re] Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study

Meropi Topalidou, Nicolas P. Rougier

► To cite this version:

Meropi Topalidou, Nicolas P. Rougier. [Re] Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study. The ReScience journal, 2015, 1 (1), 10.5281/zenodo.27944. [hal-01201790]

HAL Id: hal-01201790

https://inria.hal.science/hal-01201790

Submitted on 18 Sep 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# [Re] Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study

Meropi Topalidou¹, ², ³ and Nicolas P. Rougier¹, ², ³

1 INRIA Bordeaux Sud-Ouest, Bordeaux, France. 2 LaBRI, Université de Bordeaux, Institut Polytechnique de Bordeaux, Centre National de la Recherche Scientifique, UMR 5800, Talence, France. 3 Institut des Maladies Neurodégénératives, Université de Bordeaux, Centre National de la Recherche Scientifique, UMR 5293, Bordeaux, France.

Nicolas.Rougier@inria.fr

**Editor**

Tiziano Zito

**Reviewers**

Benoît Girard
Mehdi Khamassi

**Received** Jun 9, 2015

**Accepted** Aug 12, 2015

**Published** Aug 14, 2015

**Licence** CC-BY-SA

**Competing Interests:**

The authors have declared that no competing interests exist.

<figure><img src="image_2.png" /></figure>

Code repository

## A reference implementation of

→ *Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study*, M. Guthrie, A. Leblois, A. Garenne, and T. Boraud, Journal of Neurophysiology, 109, 2013.

## Introduction

We propose a reference implementation of [1] that introduces an action selection mechanism in cortico-basal ganglia loops based on a competition between the positive feedback, direct pathway through the striatum and the negative feedback, hyperdirect pathway through the subthalamic nucleus. The original implementation was made in Delphi (Object Pascal) whose sources are available on request to any of the author of the original article. We have used these sources to disambiguate ambiguous and missing information in the original article. The reference implementation we propose has been coded in Python for ease of reading and Cython for performances because the main result includes a batch of 250 experiments over 120 trials that would be too slow for regular Python scripts.

## Methods

We used the description of the model in the original article as well as the sources of the model (requested from author) that are made of a hundred files and 6,000 lines of Delphi for the main source. We have been unable to compile this original implementation but we were able to run the provided Windows executable. We found some factual errors in the original article that have been corrected in this implementation. The initialization of weights are defined in two different parts of the paper. First on page 3030 (second column) "Weights were initialized to a Gaussian distribution with a mean of 0.5 and a SD of 0.005 at the start of each simulation...", then on page 3031 in the caption of figure 4, "All synaptic weights were initialized to 0.5". It happened that both definitions are right but do not address the same projections. Cortico-striatal synaptic weights use Gaussian distribution while all other weights are set to 0.5. Furthermore, the Boltzmann equation given in the original paper uses a . instead of + between first term and second term.
One notable modification in our implementation is the reinforcement learning rule that has been greatly simplified. Original authors have been using quite a complex algorithm for ensuring that "*corticostriatal weights are bounded by a sigmoidal transfer function to represent physical constraints on synaptic growth with an absolute maximum of 0.75 and an absolute minimum of 0.25.*". This algorithm is not described in the article, but from sources, it appears that it is based on the estimation of the weight gradient along the sigmoid. We use instead an Oja-like rule given in the *Synapse* table.

We provide below the formal description of the model according to the proposition of Nordlie et al. [2] for reproducible descriptions of neuronal network models.

<table><caption>**Table 1: Model description following [2] prescription.**</caption><thead><tr><th>Table</th><th>Description</th></tr></thead><tbody><tr><td>Populations</td><td>Cortex (motor, associative &amp; cognitive),<br>Striatum (motor, associative &amp; cognitive),<br>GPi (motor &amp; cognitive),<br>STN (motor &amp; cognitive),<br>Thalamus (motor &amp; cognitive)</td></tr><tr><td>Topology</td><td>—</td></tr><tr><td>Connectivity</td><td>One to one, one to many (divergent), many to one (convergent)</td></tr><tr><td>Neuron model</td><td>Dynamic rate model</td></tr><tr><td>Channel model</td><td>—</td></tr><tr><td>Synapse model</td><td>Linear synapse</td></tr><tr><td>Plasticity</td><td>Reinforcement learning rule</td></tr><tr><td>Input</td><td>External current in cortical areas (motor, associative &amp; cognitive)</td></tr><tr><td>Recordings</td><td>Firing rate &amp; performances</td></tr></tbody></table>

<table><caption>**Table 2: Populations**</caption><thead><tr><th>Name</th><th>Elements</th><th>Size</th><th>Threshold</th><th>Noise</th><th>Initial state</th><th>τ</th></tr></thead><tbody><tr><td>Cortex motor</td><td>Linear neuron</td><td>1 × 4</td><td>-3</td><td>1.0%</td><td>0.0</td><td>10</td></tr><tr><td>Cortex cognitive</td><td>Linear neuron</td><td>4 × 1</td><td>-3</td><td>1.0%</td><td>0.0</td><td>10</td></tr><tr><td>Cortex associative</td><td>Linear neuron</td><td>4 × 4</td><td>-3</td><td>1.0%</td><td>0.0</td><td>10</td></tr><tr><td>Striatum motor</td><td>Sigmoidal neuron</td><td>1 × 4</td><td>0</td><td>0.1%</td><td>0.0</td><td>10</td></tr><tr><td>Striatum cognitive</td><td>Sigmoidal neuron</td><td>4 × 1</td><td>0</td><td>0.1%</td><td>0.0</td><td>10</td></tr><tr><td>Striatum associative</td><td>Sigmoidal neuron</td><td>4 × 4</td><td>0</td><td>0.1%</td><td>0.0</td><td>10</td></tr><tr><td>GPi motor</td><td>Linear neuron</td><td>1 × 4</td><td>+10</td><td>3.0%</td><td>0.0</td><td>10</td></tr><tr><td>GPi cognitive</td><td>Linear neuron</td><td>4 × 1</td><td>+10</td><td>3.0%</td><td>0.0</td><td>10</td></tr><tr><td>STN motor</td><td>Linear neuron</td><td>1 × 4</td><td>-10</td><td>0.1%</td><td>0.0</td><td>10</td></tr><tr><td>STN cognitive</td><td>Linear neuron</td><td>4 × 1</td><td>-10</td><td>0.1%</td><td>0.0</td><td>10</td></tr><tr><td>Thalamus motor</td><td>Linear neuron</td><td>1 × 4</td><td>-40</td><td>0.1%</td><td>0.0</td><td>10</td></tr><tr><td>Thalamus cognitive</td><td>Linear neuron</td><td>4 × 1</td><td>-40</td><td>0.1%</td><td>0.0</td><td>10</td></tr><tr><td>Values (V<sub>i</sub>)</td><td>Scalar</td><td>4</td><td>—</td><td>—</td><td>0.5</td><td>—</td></tr></tbody></table>

<table><caption>**Table 3: Connectivity**</caption><thead><tr><th>Source</th><th>Target</th><th>Pattern</th><th>Weight</th><th>Gain</th><th>Plastic</th></tr></thead><tbody><tr><td>Cortex motor</td><td>Thalamus motor</td><td>(1, i) → (1, i)</td><td>1.0</td><td>0.4</td><td>No</td></tr><tr><td>Cortex cognitive</td><td>Thalamus cognitive</td><td>(i, 1) → (i, 1)</td><td>1.0</td><td>0.4</td><td>No</td></tr></tbody></table>
<table><caption>**Table 4: Neuron Model (1)**</caption><thead><tr><th>Source</th><th>Target</th><th>Pattern</th><th>Weight</th><th>Gain</th><th>Plastic</th></tr></thead><tbody><tr><td>Cortex motor</td><td>STN motor</td><td>(1, i) → (1, i)</td><td>1.0</td><td>1.0</td><td>No</td></tr><tr><td>Cortex cognitive</td><td>STN cognitive</td><td>(i, 1) → (i, 1)</td><td>1.0</td><td>1.0</td><td>No</td></tr><tr><td>Cortex motor</td><td>Striatum motor</td><td>(1, i) → (1, i)</td><td>0.5</td><td>1.0</td><td>No</td></tr><tr><td>Cortex cognitive</td><td>Striatum cognitive</td><td>(i, 1) → (i, 1)</td><td>0.5</td><td>1.0</td><td>Yes</td></tr><tr><td>Cortex motor</td><td>Striatum associative</td><td>(1, i) → (i, i)</td><td>0.5</td><td>0.2</td><td>No</td></tr><tr><td>Cortex cognitive</td><td>Striatum associative</td><td>(i, 1) → (i, 1)</td><td>0.5</td><td>0.2</td><td>No</td></tr><tr><td>Cortex associative</td><td>Striatum associative</td><td>(i, j) → (i, j)</td><td>0.5</td><td>1.0</td><td>No</td></tr><tr><td>Thalamus motor</td><td>Cortex motor</td><td>(1, i) → (1, i)</td><td>1.0</td><td>1.0</td><td>No</td></tr><tr><td>Thalamus cognitive</td><td>Cortex cognitive</td><td>(i, 1) → (i, 1)</td><td>1.0</td><td>1.0</td><td>No</td></tr><tr><td>GPi motor</td><td>Thalamus motor</td><td>(1, i) → (1, i)</td><td>1.0</td><td>-0.5</td><td>No</td></tr><tr><td>GPi cognitive</td><td>Thalamus cognitive</td><td>(i, 1) → (i, 1)</td><td>1.0</td><td>-0.5</td><td>No</td></tr><tr><td>STN motor</td><td>GPi motor</td><td>(1, i) → (1, i)</td><td>1.0</td><td>1.0</td><td>No</td></tr><tr><td>STN cognitive</td><td>GPi cognitive</td><td>(i, 1) → (i, 1)</td><td>1.0</td><td>1.0</td><td>No</td></tr><tr><td>Striatum cognitive</td><td>GPi cognitive</td><td>(i, 1) → (i, 1)</td><td>1.0</td><td>-2.0</td><td>No</td></tr><tr><td>Striatum motor</td><td>GPi motor</td><td>(i, 1) → (i, 1)</td><td>1.0</td><td>-2.0</td><td>No</td></tr><tr><td>Striatum associative</td><td>GPi motor</td><td>(i, i) → (1, i)</td><td>1.0</td><td>-2.0</td><td>No</td></tr><tr><td>Striatum associative</td><td>GPi cognitive</td><td>(i, i) → (i, 1)</td><td>1.0</td><td>-2.0</td><td>No</td></tr></tbody></table>

<table><caption>**Table 5: Neuron Model (2)**</caption><tr><th colspan="2">Linear neuron</th></tr><tr><th>Type</th><th>Rate model</th></tr><tr><td>Membrane Potential</td><td>τdV/dt = -V + I<sub>syn</sub> + I<sub>ext</sub> - h<br>U = max(V, 0)</td></tr></table>

<table><caption>**Table 6: Synapse**</caption><tr><th colspan="2">Sigmoidal neuron</th></tr><tr><th>Type</th><th>Rate model</th></tr><tr><td>Membrane Potential</td><td>τdV/dt = -V + I<sub>syn</sub> + I<sub>ext</sub> - h<br>U = V<sub>min</sub> - (V<sub>max</sub> - V<sub>min</sub>)/(1 + e<sup>-V<sub>h</sub>/V<sub>c</sub>)<br>V<sub>min</sub> = 1, V<sub>max</sub> = 20, V<sub>h</sub> = 16, V<sub>c</sub> = 3</td></tr></table>

<table><caption>**Table 7: Plasticity**</caption><tr><th colspan="2">Linear synapse</th></tr><tr><th>Type</th><th>Weighted sum</th></tr><tr><td>Output</td><td>I<sup>B</sup><sub>syn</sub> = Σ<sub>A∈sources</sub>(G<sub>A→B</sub>W<sub>A→B</sub>U<sub>A</sub>)</td></tr></table>

<table><tr><th colspan="2">Reinforcement learning</th></tr><tr><th>Type</th><th>Delta rule</th></tr><tr><td>Delta</td><td>ΔW<sub>A→B</sub> = α × PE × U<sub>B</sub> × S<br>S = (W<sub>A→B</sub> - W<sub>min</sub>)(W<sub>max</sub> - W<sub>A→B</sub>)<br>PE = Reward - V<sub>i</sub></td></tr></table>
<table><caption>**Table 8: Recordings**</caption><tr><td>Reinforcement learning</td></tr><tr><td>α = 0.02 if PE &lt; 0 (LTD), α = 0.04 if PE &gt; 0 (LTP)</td></tr></table>

<table><caption>**Table 9: Input**</caption><thead><tr><td>Site</td><td>Type</td></tr></thead><tbody><tr><td>Cognitive cortex</td><td>Firing rate</td></tr><tr><td>Motor cortex</td><td>Firing rate</td></tr><tr><td>Cortico-striatal projections</td><td>Weights</td></tr></tbody></table>

<table><caption>**Table 10: Environment**</caption><thead><tr><td>Type</td><td>Description</td></tr></thead><tbody><tr><td>Cortical input</td><td>A trial is preceded by a settling period (500ms) and followed by a reset period. At time t = 0, two shapes are presented in cortical cognitive area (I<sub>ext</sub> = 7 at {i<sub>1</sub>, i<sub>2</sub>}</sub>) at two different locations in cortical motor area (I<sub>ext</sub> = 7 at {j<sub>1</sub>, j<sub>2</sub>}</sub>) and the cortical associate area is updated accordingly (I<sub>ext</sub> = 7 at {i<sub>1</sub>, i<sub>2</sub>} × {j<sub>1</sub>, j<sub>2</sub>}</sub>)</td></tr></tbody></table>

<table><thead><tr><td>Resources</td><td>Version</td></tr></thead><tbody><tr><td>OS</td><td>OSX 10.10 (yosemite)</td></tr><tr><td>Language</td><td>Python 2.7.6 (brew installation)</td></tr><tr><td rowspan="3">Libraries</td><td>Numpy 1.8.1 (pip installation)</td></tr><tr><td>Matplotlib 1.3.0 (pip installation)</td></tr><tr><td>Cython 0.22 (pip installation)</td></tr></tbody></table>

## Results

We did not reproduce all analyses of the original article but concentrated our efforts on the main results which are illustrated on figures 4 & 5 in the original article [1].

We first reproduce the activity in the cortical populations during a single trial, prior to learning. Noise has a great influence on the overall dynamic and it is not possible to exactly reproduce figure 4 in the original article without precise information on the underlying random generator(seed). Consequently, we can only report a qualitatively equivalent figure where the most critical feature is the bifurcation in cognitive and motor activities after stimulus onset. Since no learning has occured yet, it is also possible to have the motor decision to occur before the cognitive decision. Figure 1 shows an example of a decision dynamic with an oscillatory regime between time t=0 and time t=500ms that is characteristic of the model.

We also tested the learning capacity of the model by reproducing the same procedure as in the original article (250 experiments, 120 trials) but we used a modified and simpler learning rule (see Plasticity table) since the original learning rule used a sigmoidal transfer function but no actual details were given on how to enforce it.
<figure><img src="image_3.png" /><figcaption>**Figure 1:** Activity in the cortical population during a single trial of action selection. This is the reproduction of figure 4 in the original article.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Figure 2:** Learning time course over 120 trials, averaged over 250 simulations. The blue filled area indicates the standard deviation of the mean performance.</figcaption></figure>
Conclusion

After some minor corrections and modifications of the original description of the model,
we were able to reproduce the original results, confirming the correctness of the original
implementation of the model.

References

[1] M. Guthrie et al. "Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study". In: *Journal of Neurophysiology* 109 (2013), pp. 3025-3040. DOI: 10.1152/jn.00026.2013.

[2] E. Nordlie, M. Gewaltig, and H.E. Plesser. "Towards reproducible descriptions of neuronal network models". In: *PLoS Computational Biology* 5.8 (2009), e1000456. DOI: 10.1371/journal.pcbi.1000456.