# Advantage Race Diffusion mechanism to study the dissociated roles of Orbitofrontal Cortex subregions in instrumental learning

Bhargav Teja Nallapu, Frédéric Alexandre

► To cite this version:

Bhargav Teja Nallapu, Frédéric Alexandre. Advantage Race Diffusion mechanism to study the dissociated roles of Orbitofrontal Cortex subregions in instrumental learning. The 5th quadrennial meeting on orbitofrontal cortex function (OFC 2024), Apr 2024, Paris, France. . hal-04702416

HAL Id: hal-04702416

https://inria.hal.science/hal-04702416v1

Submitted on 19 Sep 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A computational model of the lateral and medial Orbitofrontal Cortex in instrumental learning tasks

Bhargav Teja NALLAPU¹,²,³, Frédéric ALEXANDRE¹,²,³

<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

## Abstract

**Orbitofrontal cortex (OFC)**: One of the crucial regions in the prefrontal cortex (PFC) involved in decision-making and instrumental learning

- The exact mechanisms through which the OFC encodes, or processes information related to decision-making and learning are unclear.

**Lateral and Medial OFC (ventromedial PFC):** Several lesion studies have shown that both the sub-regions have differential roles in the process of instrumental learning in the context of multi-alternative choice.

**Multi-alternative decision-making model:** We use a variation of drift diffusion models (DDM) based on an Advantage Race Diffusion (ARD) framework for choices among two or more options, that captures stimulus difficulty, speed-accuracy trade-off, and stimulus-response-mapping reversal effects (RL-ARD)

**Complementary roles of lateral and medial OFC:** The ARD based accumulators are driven by both *difference* and *sum* of the mean reward expectancies.

- We simulated lesions of lateral and medial OFC structures on a 3-arm bandit task with varying reward probabilities

The model showed differential effects in performance depending on the value difference of the presented options, thus accounting for recent experimental findings about the dissociate functional roles of both the sub-regions.

## Model and Experimentation

<table><caption>### Binary choice - Advantage Race Diffusion (ARD)</caption><thead><tr><th>DDM</th><th>Racing Diffusion</th><th>Advantage Racing Diffusion (ARD)</th></tr></thead><tbody><tr><td>dx = w(Q<sub>1</sub> - Q<sub>2</sub>)dt + sW</td><td>dx<sub>1</sub> = [V<sub>0</sub> + wQ<sub>1</sub>]dt + sW<br>dx<sub>2</sub> = [V<sub>0</sub> + wQ<sub>2</sub>]dt + sW</td><td>dx<sub>1</sub> = [V<sub>0</sub> + w<sub>d</sub>(Q<sub>1</sub> - Q<sub>2</sub>) + w<sub>s</sub>(Q<sub>1</sub> + Q<sub>2</sub>)]dt + sW<br>dx<sub>2</sub> = [V<sub>0</sub> + w<sub>d</sub>(Q<sub>2</sub> - Q<sub>1</sub>) + w<sub>s</sub>(Q<sub>1</sub> + Q<sub>2</sub>)]dt + sW</td></tr></tbody></table>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

(Miletic et al, 2021)

<figure><img src="image_7.png" /><figcaption>### Multi-alternative choice - 3-Arm bandit Task</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>### Advantage Linear Ballistic Accumulators (ALBAs)
(van Ravenzwaaij et al, 2020)</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>### Separate mechanisms in medial and lateral OFC - Lesion simulations, Results</figcaption></figure>

<figure><img src="image_10.png" /></figure>

(Noonan et al, 2010)

## References

[Miletic et al., 2021] Miletić, S., Boag, R. J., Trutti, A. C., Stevenson, N., Forstmann, B. U., & Heathcote, A. (2021). A new model of decision processing in instrumental learning tasks. eLife, 10, e63055. https://doi.org/10.7554/eLife.63055
[van Ravenzwaaij et al., 2020] van Ravenzwaaij, D., Brown, S. D., Marley, A. a. J., & Heathcote, A. (2020). Accumulating advantages: A new conceptualization of rapid multiple choice. Psychological Review, 127(2), 186–215. https://doi.org/10.1037/rev0000166
[Noonan et al., 2010] Noonan, M. P., et al. "Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex." Proceedings of the National Academy of Sciences 107.47 (2010): 20547-20552.