Motivational salience is a mechanism that determines URLanism's current level of attraction to or repulsion from a particular object, event, or outcome. Motivational salience is described by modulating the reward by an externally controlled parameter that remains constant within a single behavioral episode. The vector of perceived values of various outcomes determines motivation of URLanism toward different goals. Organism's behavior should be able to adapt to the varying-in-time motivation vector. Here, we propose a reinforcement learning framework that relies on neural networks to learn optimal behavior for different dynamically changing motivation vectors. First, we show that Q-learning neural networks can learn to navigate towards variable goals whose relative salience is determined by a multidimensional motivational vector . Second, we show that a Q-learning network with motivation can learn complex behaviors  towards several goals distributed in an environment. Finally, we show that firing patterns displayed by neurons in the ventral pallidum  , a basal ganglia structure playing a crucial role in motivated behaviors , are similar to the responses of neuronsin recurrent neural networks trained in similar conditions. Similarly to the pallidum neurons, artificial neural nets contain two different classes of neurons, tuned to reward and punishment. We conclude that reinforcement learning networks can efficiently learn optimal behavior in conditions when reward values are modulated by external motivational processes with arbitrary dynamics. Motivational salience can be viewed as a general-purpose model-free method identifying and capturing changes in subjective or objective values of multiple rewards. Networks with motivationmay also be parts of a larger hierarchical reinforcement learning system in the brain.
How can animals behave effectively in conditions involving different motivational contexts? Here, we propose how reinforcement learning neural networks can learn optimal behavior for dynamically changing motivational salience vectors. First, we show that Q-learning neural networks with motivation can navigate in environment with dynamic rewards . Second, we show that such networks can learn complex behaviors simultaneously directed towards several goals distributed in an environment. Finally, we show that in Pavlovian conditioning task, the responses of the neurons in our model resemble the firing patterns of neurons in the ventral pallidum (VP) , a basal ganglia structure involved in motivated behaviors . We show that, similarly to real neurons, recurrent networks with motivation are composed of two oppositely-tuned classes of neurons, responding to positive and negative rewards. Our model generates predictions for the VP connectivity. We conclude that networks with motivation can rapidly adapt their behavior to varying conditions without changes in synaptic strength when expected reward is modulated by motivation. Such networks may also provide a mechanism for how hierarchical reinforcement learning is implemented in the brain.
R
Motivational salience is a mechanism that determines URLanism's current level of attraction to or repulsion from a particular object, event, or outcome. Motivational salience is described by modulating the reward by an externally controlled parameter that remains constant within a single behavioral episode. The vector of perceived values of various outcomes determines motivation of URLanism toward different goals. Organism's behavior should be able to adapt to the varying-in-time motivation vector.
How can animals behave effectively in conditions involving different motivational contexts?
R
a reinforcement learning framework that relies on neural networks to
how reinforcement learning neural networks can
R
different dynamically changing motivation
dynamically changing motivational salience
R
can learn to navigate towards variable goals whose relative salience is determined by a multidimensional motivational vector
with motivation can navigate in environment with dynamic rewards
R
a Q-learning network with motivation
such networks
A
simultaneously directed
R
firing patterns displayed by
in Pavlovian conditioning task, the responses of the neurons in our model resemble the firing patterns of
A
(VP)
R
playing a crucial role
involved
R
, are similar to the responses of neuronsin recurrent neural networks trained in similar conditions. Similarly to the pallidum neurons, artificial neural nets contain two different
. We show that, similarly to real neurons, recurrent networks with motivation are composed of two oppositely-tuned
R
tuned to reward and punishment.
responding to positive and negative rewards. Our model generates predictions for the VP connectivity.
R
reinforcement learning networks can efficiently learn optimal behavior in conditions when reward values are modulated by external motivational processes with arbitrary dynamics. Motivational salience can be viewed as a general-purpose model-free method identifying and capturing changes in subjective or objective values of multiple rewards. Networks with motivationmay also be parts of a larger
networks with motivation can rapidly adapt their behavior to varying conditions without changes in synaptic strength when expected reward is modulated by motivation. Such networks may also provide a mechanism for how
R
system
is implemented