Nature Reviews Neuroscience | Aop, published online 13 January 2010; doi:10.1038/nrn2787

REVIEWS

The free-energy principle:a unified brain theory?
Karl Friston

Abstract | A free-energy principle has been proposed recently that accounts for action,
perception and learning. This Review looks at some key brain theories in the biological (forexample, neural Darwinism) and physical (for example, information theory and optimal
control theory) sciences from the free-energy perspective. Crucially, one key theme runs
through each of these theories — optimization. Furthermore, if we look closely at what is
optimized, the same quantity keeps emerging, namely value (expected reward, expectedutility) or its complement, surprise (prediction error, expected cost). This is the quantity thatis optimized under the free-energy principle, which suggests that several global brain
theories might be unified within a free-energy framework.
Free energy
An information theory measurethat bounds or limits (by being
greater than) the surprise on
sampling some data, given a
generative model.

Homeostasis
The process whereby an open
or closed system regulates its
internal environment tomaintain its states withinbounds.

Entropy
The average surprise ofoutcomes sampled from aprobability distribution or
density. A density with low
entropy means that, on
average, the outcome isrelatively predictable. Entropy
is therefore a measure of
uncertainty.

The Wellcome Trust Centre
for Neuroimaging,
University College London,
Queen Square, London,
WC1N 3BG, UK.
e‑mail:k.friston@fil.ion.ucl.ac.uk
doi:10.1038/nrn2787
Published online
13 January 2010

Despite the wealth of empirical data in neuroscience,
there are relatively few global theories about how thebrain works. A recently proposed free-energy principle
for adaptive systems tries to provide a unified accountof action, perception and learning. Although this principle has been portrayed as a unified brain theory 1, its
capacity to unify different perspectives on brain function
has yet to be established. This Review attempts to placesome key theories within the free-energy framework, inthe hope of identifying common themes. I first review
the free-energy principle and then deconstruct several
global brain theories to show how they all speak to thesame underlying idea.

The free-energy principle
The free-energy principle (BOX 1) says that any selforganizing system that is at equilibrium with its environment must minimize its free energy 2. The principle isessentially a mathematical formulation of how adaptive
systems (that is, biological agents, like animals or brains)resist a natural tendency to disorder 3–6. What follows is
a non-mathematical treatment of the motivation andimplications of the principle. We will see that although the
motivation is quite straightforward, the implications arecomplicated and diverse. This diversity allows the principle to account for many aspects of brain structure andfunction and lends it the potential to unify different perspectives on how the brain works. In subsequent sections,
I discuss how the principle can be applied to neuronalsystems as viewed from these perspectives. This Review
starts in a rather abstract and technical way but then tries
to unpack the basic idea in more familiar terms.

Motivation: resisting a tendency to disorder. The
defining characteristic of biological systems is thatthey maintain their states and form in the face of aconstantly changing environment 3–6. From the pointof view of the brain, the environment includes boththe external and the internal milieu. This maintenanceof order is seen at many levels and distinguishes biological from other self-organizing systems; indeed, thephysiology of biological systems can be reduced almost
entirely to their homeostasis7. More precisely, the repertoire of physiological and sensory states in which anorganism can be is limited, and these states define theorganism’s phenotype. Mathematically, this means thatthe probability of these (interoceptive and exteroceptive) sensory states must have low entropy; in otherwords, there is a high probability that a system will
be in any of a small number of states, and a low probability that it will be in the remaining states. Entropy
is also the average self information or ‘ surprise ’ 8(more formally, it is the negative log-probability of an
outcome). Here, ‘a fish out of water’ would be in a surprising state (both emotionally and mathematically).
A fish that frequently forsook water would have highentropy. Note that both surprise and entropy dependon the agent: what is surprising for one agent (forexample, being out of water) may not be surprising
for another. Biological agents must therefore minimize the long-term average of surprise to ensure that
their sensory entropy remains low. In other words,
biological systems somehow manage to violate thefluctuation theorem, which generalizes the second lawof thermodynamics9.

NATuRE REvIEWs | NeuroscieNce

ADvANcE oNlINE puBlIcATIoN | 1© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
Box 1 | The free-energy principle

Surprise(Surprisal or self information.)
The negative log-probability ofan outcome. An improbableoutcome (for example, waterflowing uphill) is therefore
surprising.

Fluctuation theorem(A term from statisticalmechanics.) Deals with theprobability that the entropyof a system that is far from thethermodynamic equilibrium
will increase or decrease overa given amount of time. It
states that the probability ofthe entropy decreasing
becomes exponentially smallerwith time.

Attractor
A set to which a dynamicalsystem evolves after a longenough time. Points that
get close to the attractor
remain close, even undersmall perturbations.

Kullback-Leibler divergence(Or information divergence,
information gain or crossentropy.) A non-commutative
measure of the non-negative
difference between two
probability distributions.

Recognition density(Or ‘approximating conditional
density’.) An approximate
probability distribution of thecauses of data (for example,
sensory input). It is the productof inference or inverting agenerative model.

Part a of the figure shows the dependencies among thequantities that define free energy. These include theinternal states of the brain μ(t) and quantities describing its
exchange with the environment: sensory signals (and theirmotion) s̃(t) = [s,s′,s″…]T plus action a(t). The environmentis described by equations of motion, which specify the
trajectory of its hidden states. The causes ϑ ⊃ {x̃, θ, γ } of
sensory input comprise hidden states x̃(t), parameters θand precisions γ controlling the amplitude of the random
fluctuations z̃(t) and w̃(t). Internal brain states and action
minimize free energy F(s̃,μ), which is a function of sensory
input and a probabilistic representation q(ϑ|μ) of its causes.
This representation is called the recognition density and is
encoded by internal states μ.
The free energy depends on two probability densities:
the recognition density q(ϑ|μ) and one that generates
sensory samples and their causes, p(s̃,ϑ|m). The latter
represents a probabilistic generative model (denoted bym), the form of which is entailed by the agent or brain.
Part b of the figure provides alternative expressions for thefree energy to show what its minimization entails: action
can reduce free energy only by increasing accuracy (that is,selectively sampling data that are predicted). Conversely,
optimizing brain states makes the representation an
approximate conditional density on the causes of sensory
input. This enables action to avoid surprising sensory
encounters. A more formal description is provided below.

aEnvironment

Agent
Sensationss~ = g(x~, ϑ) + z~

External statesẋ~ = f(x~, a, ϑ) + w~

Internal states
μ = arg min F(s~, μ)Action or control signals
a = arg min F(s~, μ)bFree-energy bound on surpriseF = −<ln p(s~, ϑ | m)> + <ln q(ϑ | μ)>q

Action minimizes prediction errors
F = D(q(ϑ | μ) || p(ϑ)) − <ln p(s~ (a) | ϑ, m)>qqa = arg max Accuracy
Perception optimizes predictions
F = D(q(ϑ | μ) || p(ϑ | s~ )) − ln p(s~ | m)
μ = arg max Divergence

optimizing the sufficient statistics (representations)
Reviews
Optimizing the recognition density makes it a posterior or conditional density on the causesNatureof sensory
data:| Neuroscience
this can beseen by expressing the free energy as surprise –In p(s̃,| m) plus a Kullback-Leibler divergence between the recognition andconditional densities (encoded by the ‘internal states’ in the figure). Because this difference is always positive, minimizing
free energy makes the recognition density an approximate posterior probability. This means the agent implicitly infers or
represents the causes of its sensory samples in a Bayes-optimal fashion. At the same time, the free energy becomes a tightbound on surprise, which is minimized through action.
optimizing action
Acting on the environment by minimizing free energy enforces a sampling of sensory data that is consistent with thecurrent representation. This can be seen with a second rearrangement of the free energy as a mixture of accuracy andcomplexity. Crucially, action can only affect accuracy (encoded by the ‘external states’ in the figure). This means thatthe brain will reconfigure its sensory epithelia to sample inputs that are predicted by the recognition density — in otherwords, to minimize prediction error.

In short, the long-term (distal) imperative — of maintaining states within physiological bounds — translates
into a short-term (proximal) avoidance of surprise.
surprise here relates not just to the current state, whichcannot be changed, but also to movement from one state
to another, which can change. This motion can be complicated and itinerant (wandering) provided that it revisits a small set of states, called a global random attractor10,that are compatible with survival (for example, driving a
car within a small margin of error). It is this motion thatthe free-energy principle optimizes.
so far, all we have said is that biological agents must
avoid surprises to ensure that their states remain within
physiological bounds (see supplementary information s1(box) for a more formal argument). But how do they
do this? A system cannot know whether its sensations
are surprising and could not avoid them even if it did
know. This is where free energy comes in: free energy isan upper bound on surprise, which means that if agentsminimize free energy, they implicitly minimize surprise.

crucially, free energy can be evaluated because it is afunction of two things to which the agent has access: its
sensory states and a recognition density that is encodedby its internal states (for example, neuronal activity
and connection strengths). The recognition density is a
probabilistic representation of what caused a particularsensation.
This (variational) free-energy construct was
introduced into statistical physics to convert difficult
probability-density integration problems into easier optimization problems 11. It is an information
theoretic quantity (like surprise), as opposed to athermodynamic quantity. variational free energy has
been exploited in machine learning and statistics tosolve many inference and learning problems12–14. In thissetting, surprise is called the (negative) model evidence.
This means that minimizing surprise is the same as
maximizing the sensory evidence for an agent’s existence, if we regard the agent as a model of its world. In
the present context, free energy provides the answer to

2 | ADvANcE oNlINE puBlIcATIoN

www.nature.com/reviews/neuro© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
a fundamental question: how do self-organizing adaptive systems avoid surprising states? They can do this byminimizing their free energy. so what does this involve?

Generative model
A probabilistic model (joint
density) of the dependencies
between causes andconsequences (data), fromwhich samples can begenerated. It is usuallyspecified in terms of thelikelihood of data, given theircauses (parameters of a model)
and priors on the causes.

Conditional density(Or posterior density.) The
probability distribution of
causes or model parameters,
given some data; that is, aprobabilistic mapping fromobserved data to causes.

Prior
The probability distribution or
density of the causes of datathat encodes beliefs about
those causes before observing
the data.

Bayesian surprise
A measure of salience basedon the Kullback-Leibler
divergence between therecognition density (which
encodes posterior beliefs) andthe prior density. It
measures the information thatcan be recognized in the data.

Bayesian brain hypothesis
The idea that the brain uses
internal probabilistic(generative) models to update
posterior beliefs, using sensory
information, in an(approximately) Bayes-optimal
fashion.

Analysis by synthesis
Any strategy (in speech coding)in which the parameters of asignal coder are evaluated bydecoding (synthesizing) thesignal and comparing it withthe original input signal.

Epistemological automata
Possibly the first theory for why
top-down influences (mediatedby backward connections in
the brain) might be importantin perception and cognition.

Empirical prior
A prior induced by hierarchicalmodels; empirical priors
provide constraints on therecognition density in the usual
way but depend on the data.

Implications: action and perception. Agents can
suppress free energy by changing the two things it depends
on: they can change sensory input by acting on the worldor they can change their recognition density by changing their internal states. This distinction maps nicely
onto action and perception (BOX 1). one can see what thismeans in more detail by considering three mathematicallyequivalent formulations of free energy (see supplementary
information s2 (box) for a mathematical treatment).
The first formulation expresses free energy as energy
minus entropy. This formulation is important for threereasons. First, it connects the concept of free energy as
used in information theory with concepts used in statistical thermodynamics. second, it shows that the freeenergy can be evaluated by an agent because the energy
is the surprise about the joint occurrence of sensations
and their perceived causes, whereas the entropy is simply that of the agent’s own recognition density. Third, itshows that free energy rests on a generative model of theworld, which is expressed in terms of the probability of asensation and its causes occurring together. This meansthat an agent must have an implicit generative model of
how causes conspire to produce sensory data. It is thismodel that defines both the nature of the agent and thequality of the free-energy bound on surprise.
The second formulation expresses free energy as
surprise plus a divergence term. The (perceptual) divergence is just the difference between the recognition density and the conditional density (or posterior density) of thecauses of a sensation, given the sensory signals. This conditional density represents the best possible guess about
the true causes. The difference between the two densities
is always non-negative and free energy is therefore an
upper bound on surprise. Thus, minimizing free energy
by changing the recognition density (without changing
sensory data) reduces the perceptual divergence, so thatthe recognition density becomes the conditional density
and the free energy becomes surprise.
The third formulation expresses free energy as complexity minus accuracy, using terms from the modelcomparison literature. complexity is the difference
between the recognition density and the prior density
on causes; it is also known as Bayesian surprise15 and is thedifference between the prior density — which encodes
beliefs about the state of the world before sensory data areassimilated — and posterior beliefs, which are encodedby the recognition density. Accuracy is simply the surprise about sensations that are expected under the recognition density. This formulation shows that minimizing
free energy by changing sensory data (without changing
the recognition density) must increase the accuracy ofan agent’s predictions. In short, the agent will selectively
sample the sensory inputs that it expects. This is known
as active inference16. An intuitive example of this process(when it is raised into consciousness) would be feeling
our way in darkness: we anticipate what we might touchnext and then try to confirm those expectations.

In summary, the free energy rests on a model of how
sensory data are generated and on a recognition density
on the model’s parameters (that is, sensory causes). Free
energy can be reduced only by changing the recognition
density to change conditional expectations about what is
sampled or by changing sensory samples (that is, sensory
input) so that they conform to expectations. In what follows, I consider these implications in light of some key
theories about the brain.

The Bayesian brain hypothesis
The Bayesian brain hypothesis17 uses Bayesian probability
theory to formulate perception as a constructive process
based on internal or generative models. The underlying
idea is that the brain has a model of the world18–22 thatit tries to optimize using sensory inputs23–28. This idea isrelated to analysis by synthesis20 and epistemological automata19. In this view, the brain is an inference machine that
actively predicts and explains its sensations18,22,25. central
to this hypothesis is a probabilistic model that can generate predictions, against which sensory samples are tested
to update beliefs about their causes. This generativemodel is decomposed into a likelihood (the probability of
sensory data, given their causes) and a prior (the a priori
probability of those causes). perception then becomes theprocess of inverting the likelihood model (mapping fromcauses to sensations) to access the posterior probability of
the causes, given sensory data (mapping from sensations
to causes). This inversion is the same as minimizing thedifference between the recognition and posterior densities to suppress free energy. Indeed, the free-energy formulation was developed to finesse the difficult problem
of exact inference by converting it into an easier optimization problem11–14. This has furnished some powerfulapproximation techniques for model identification andcomparison (for example, variational Bayes or ensemble
learning 29). There are many interesting issues that attend
the Bayesian brain hypothesis, which can be illuminatedby the free-energy principle; we will focus on two.
The first is the form of the generative model andhow it manifests in the brain. one criticism of Bayesiantreatments is that they ignore the question of how priorbeliefs, which are necessary for inference, are formed27.
However, this criticism dissolves with hierarchicalgenerative models, in which the priors themselves are
optimized26,28. In hierarchical models, causes in one
level generate subordinate causes in a lower level; sensory data per se are generated at the lowest level (BOX 2).
Minimizing the free energy effectively optimizes empirical priors (that is, the probability of causes at one level,
given those in the level above). crucially, because empirical priors are linked hierarchically, they are informedby sensory data, enabling the brain to optimize its priorexpectations online. This optimization makes every levelin the hierarchy accountable to the others, furnishing aninternally consistent representation of sensory causes atmultiple levels of description. Not only do hierarchicalmodels have a key role in statistics (for example, random effects and parametric empirical Bayes models30,31),
they may also be used by the brain, given the hierarchicalarrangement of cortical sensory areas32–34.

NATuRE REvIEWs | NeuroscieNce

ADvANcE oNlINE puBlIcATIoN | 3© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
Box 2 | Hierarchical message passing in the brain
ξv(i) = Πv(i) εv(i) = Πv(i) (μ(iv – 1) – g(μ (i) ))ξx(i) = Πx(i) εx(i) = Πx(i) (Dμ(ix ) – f(μ (i) ))Sensory
input

ξv(1)
ξx(1)

Forward:prediction
error
s~(t)

ξv(2)

ξx(2)

Backward:
predictions
μv(2)

μv(1)

μx(2)

μx(1)
Lower cortical areas
Synaptic plasticity
�μθ ij = −∂θ ijεTξξv(3)μ̇v(i) = Dμv(i) − (∂vε(i) )Tξ (i) − ξv(i + 1)μ̇x(i) = Dμx(i) − (∂xε(i) )Tξ (i)Higher cortical areas
Synaptic gainμ
� γi = ½tr(∂γi Π(ξξT − Π(μγ)))

The figure details a neuronal architecture that optimizes the conditional expectations ofcauses in hierarchical models of sensory input. It shows the putative cells of origin of forward| Neuroscience
driving connections that convey prediction error (grey arrows)Naturefrom aReviews
lower area(forexample, the lateral geniculate nucleus) to a higher area (for example, V1), and nonlinear
backward connections (black arrows) that construct predictions41. These predictions try to
explain away prediction error in lower levels. In this scheme, the sources of forward andbackward connections are superficial and deep pyramidal cells (upper and lower triangles),respectively, where state units are black and error units are grey. The equations represent a
gradient descent on free energy using the generative model below. The two upper equations
describe the formation of prediction error encoded by error units, and the two lower
equations represent recognition dynamics, using a gradient descent on free energy.

Generative models in the brain
To evaluate free energy one needs a generative model of how the sensorium is caused.
Such models p(s̃,ϑ) = p(s̃ | ϑ) p(ϑ) combine the likelihood p(s̃ | ϑ) of getting some data given
their causes and the prior beliefs about these causes, p(ϑ). The brain has to explain
complicated dynamics on continuous states with hierarchical or deep causal structure
and may use models with the following formU I Z  X   θ   \ · Z H Z  X   θ  

Y 

…

X Ks  I Z K X K  θ K  \ K
Z· K  H Z K X K  θ K  Y K…

Here, g(i) and f(i) are continuous nonlinear functions of (hidden and causal) states, with
Nature Reviews| Neuroscienceparameters θ(i). The random fluctuationsz(t)(i) andw(t)(i) play the part of observationnoise at the sensory level and state noise at higher levels. Causal states v(t)(i) linkhierarchical levels, where the output of one level provides input to the next. Hidden
states x(t)(i) link dynamics over time and endow the model with memory.
Gaussian assumptions about the random fluctuations specify the likelihood
and Gaussian assumptions about state noise furnish empirical priors in terms ofpredicted motion. These assumptions are encoded by their precision (or inverse
variance), П(i)(γ), which are functions of precision parameters γ.

recognition dynamics and prediction error
If we assume that neuronal activity encodes the conditional expectation of states, thenrecognition can be formulated as a gradient descent on free energy. Under Gaussian
assumptions, these recognition dynamics can be expressed compactly in terms
of precision-weighted prediction errors ξ(i) = П(i)(ε)(i) on the causal states and motion of
hidden states. The ensuing equations (see the figure) suggest two neuronal populations
that exchange messages: causal or hidden-state units encoding expected states anderror units encoding prediction error. Under hierarchical models, error units receive
messages from the state units in the same level and the level above, whereas state units
are driven by error units in the same level and the level below. These provide bottom-up
messages that drive conditional expectations μ(i) towards better predictions, which
explain away prediction error. These top-down predictions correspond to g(μ(i)) and f(μ(i)).
This scheme suggests that the only connections that link levels are forward connections
conveying prediction error to state units and reciprocal backward connections thatmediate predictions. See REFS 42,130 for details. Figure is modified from REF. 42.

The second issue is the form of the recognition density that is encoded by physical attributes of the brain,such as synaptic activity, efficacy and gain. In general,any density is encoded by its sufficient statistics (for example, the mean and variance of a Gaussian form). The way
the brain encodes these statistics places important constraints on the sorts of schemes that underlie recognition:
they range from free-form schemes (for example, particle
filtering 26 and probabilistic population codes 35–38),
which use a vast number of sufficient statistics, to simpler forms, which make stronger assumptions about
the shape of the recognition density, so that it can be
encoded with a small number of sufficient statistics. The
simplest assumed form is Gaussian, which requires only
the conditional mean or expectation — this is known
as the Laplace assumption39, under which the free energy
is just the difference between the model’s predictions
and the sensations or representations that are predicted.
Minimizing free energy then corresponds to explaining
away prediction errors. This is known as predictive coding
and has become a popular framework for understanding neuronal message passing among different levels ofcortical hierarchies40. In this scheme, prediction error
units compare conditional expectations with top-down
predictions to elaborate a prediction error. This prediction error is passed forward to drive the units in thelevel above that encode conditional expectations which
optimize top-down predictions to explain away (reduce)
prediction error in the level below. Here, explaining
away just means countering excitatory bottom-up
inputs to a prediction error neuron with inhibitory synaptic inputs that are driven by top-down predictions(see BOX 2 and REFS 41,42 for detailed discussion). The
reciprocal exchange of bottom-up prediction errors andtop-down predictions proceeds until prediction error
is minimized at all levels and conditional expectations
are optimized. This scheme has been invoked to explain
many features of early visual responses40,43 and provides
a plausible account of repetition suppression and mismatch responses in electrophysiology 44. FIGURE 1 provides an example of perceptual categorization that uses
this scheme.
Message passing of this sort is consistent with functional asymmetries in real cortical hierarchies45, where
forward connections (which convey prediction errors)
are driving and backwards connections (which model
the nonlinear generation of sensory input) have bothdriving and modulatory characteristics46. This asymmetrical message passing is also a characteristic featureof adaptive resonance theory 47,48, which has formal similarities to predictive coding.
In summary, the theme underlying the Bayesian brain
and predictive coding is that the brain is an inferenceengine that is trying to optimize probabilistic representations of what caused its sensory input. This optimization
can be finessed using a (variational free-energy) boundon surprise. In short, the free-energy principle entails
the Bayesian brain hypothesis and can be implemented
by the many schemes considered in this field. Almost
invariably, these involve some form of message passing
or belief propagation among brain areas or units. This

4 | ADvANcE oNlINE puBlIcATIoN

www.nature.com/reviews/neuro© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWSa Perceptual inference

Vocal centrev=

Syrinx

allows us to connect the free-energy principle to anotherprincipled approach to sensory processing, namely
information theory.

Sonogramv1
v2

18x2 − 18x1ẋ = f(x, v) = v1 x1 − 2x3 x1 − x2
2 x1 x2 − v2 x3

b Perceptual categorization
Song aFrequency (Hz)5,000Song bSong c

4,000
3,000
2,000

0.0 0.2 0.4 0.6 0.8 1.0

c

0.0 0.2 0.4 0.6 0.8 1.0

50

3.5

Estimated causes

c

µv1

40

3

a

30

b

20

c

10

b
a

2.5

v2
2

0

µ v1

1.5

–10
–20

0.0 0.2 0.4 0.6 0.8 1.0

Time (s)

0

0.2

0.4

0.6

Time (s)

0.8

1

1

10

15

20

25

30

35

v1

Figure 1 | Birdsongs and perceptual categorization. a | The generative model ofbirdsong used in this simulation comprises a Lorenz attractor with two control parameters
Nature Reviews | Neuroscience(or causal states) (v1,v2), which, in turn, delivers two control parameters (not shown) to asynthetic syrinx to produce ‘chirps’ that were modulated in amplitude and frequency (anexample is shown as a sonogram). The chirps were then presented as a stimulus to asynthetic bird to see whether it could infer the underlying causal states and thereby
categorize the song. This entails minimizing free energy by changing the internal
representation (μv1,μv2) of the control parameters. Examples of this perceptual inference or
categorization are shown below. b | Three simulated songs are shown in sonogram format.
Each comprises a series of chirps, the frequency and number of which fall progressively
from song a to song c, as a causal state (known as the Raleigh number; v1 in part a) isdecreased. c | The graph on the left depicts the conditional expectations (μv1,μv2) of thecausal states, shown as a function of peristimulus time for the three songs. It shows thatthe causes are identified after around 600 ms with high conditional precision (90%confidence intervals are shown in grey). The graph on the right shows the conditional
density on the causes shortly before the end of the peristimulus time (that is, the dottedline in the left panel). The blue dots correspond to conditional expectations and the grey
areas correspond to the 90% conditional confidence regions. Note that these encompass
the true values (red dots) of (v1,v2) that were used to generate the songs. These results
illustrate the nature of perceptual categorization under the inference scheme in BOX 2:
here, recognition corresponds to mapping from a continuously changing and chaotic
sensory input to a fixed point in perceptual space. Figure is reproduced, with permission,from REF. 130 © (2009) Elsevier.

The principle of efficient coding
The principle of efficient coding suggests that the brain
optimizes the mutual information (that is, the mutual
predictability) between the sensorium and its internal
representation, under constraints on the efficiency of
those representations. This line of thinking was articulated by Barlow 49 in terms of a redundancy reduction
principle (or principle of efficient coding) and formalized later in terms of the infomax principle50. It has been
applied in machine learning 51, leading to methods
like independent component analysis52, and in neurobiology, contributing to an understanding of the natureof neuronal responses53–56. This principle is extremely
effective in predicting the empirical characteristics ofclassical receptive fields53 and provides a principledexplanation for sparse coding 55 and the segregation ofprocessing streams in visual hierarchies57. It has been
extended to cover dynamics and motion trajectories58,59
and even used to infer the metabolic constraints on neuronal processing 60.
At its simplest, the infomax principle says thatneuronal activity should encode sensory information inan efficient and parsimonious fashion. It considers the
mapping between one set of variables (sensory states)
and another (variables representing those states). At
first glance, this seems to preclude a probabilistic representation, because this would involve mapping between
sensory states and a probability density. However, theinfomax principle can be applied to the sufficient statistics of a recognition density. In this context, the infomax principle becomes a special case of the free-energy
principle, which arises when we ignore uncertainty
in probabilistic representations (and when there is no
action); see supplementary information s3 (box) formathematical details). This is easy to see by noting that
sensory signals are generated by causes. This means that it
is sufficient to represent the causes to predict thesesignals. More formally, the infomax principle can beunderstood in terms of the decomposition of free energy
into complexity and accuracy: mutual information is
optimized when conditional expectations maximize
accuracy (or minimize prediction error), and efficiency
is assured by minimizing complexity. This ensures thatno excessive parameters are applied in the generativemodel and leads to a parsimonious representation of
sensory data that conforms to prior constraints on theircauses. Interestingly, advanced model-optimization
techniques use free-energy optimization to eliminate
redundant model parameters61, suggesting that freeenergy optimization might provide a nice explanation
for the synaptic pruning and homeostasis that take placein the brain during neurodevelopment 62 and sleep63.
The infomax principle pertains to a forward mappingfrom sensory input to representations. How does thissquare with optimizing generative models, which map
from causes to sensory inputs? These perspectives can bereconciled by noting that all recognition schemes based

NATuRE REvIEWs | NeuroscieNce

ADvANcE oNlINE puBlIcATIoN | 5© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS

Sufficient statistics
Quantities that are sufficient toparameterize a probability
density (for example, mean andcovariance of a Gaussian
density).

Laplace assumption(Or Laplace approximation or
method.) A saddle-point
approximation of the integralof an exponential function, thatuses a second-order Taylor
expansion. When the function
is a probability density, theimplicit assumption is thatthe density is approximately
Gaussian.

Predictive coding
A tool used in signal processing
for representing a signal using
a linear predictive (generative)model. It is a powerful speech
analysis technique and was
first considered in vision to
explain lateral interactions in
the retina.

Infomax
An optimization principle forneural networks (or functions)that map inputs to outputs. It
says that the mapping should
maximize the Shannon mutual
information between the inputs
and outputs, subject toconstraints and/or noise
processes.

Stochastic
Governed by random effects.

Biased competition
An attentional effect mediatedby competitive interactions
among neurons representing
visual stimuli; theseinteractions can be biased infavour of behaviourally relevant
stimuli by both spatial andnon-spatial and bothbottom-up and top-down
processes.

on infomax can be cast as optimizing the parameters of agenerative model64. For example, in sparse coding models55, the implicit priors posit independent causes thatare sampled from a heavy-tailed or sparse distribution42.
The fact that these models predict empirically observed
receptive fields so well suggests that we are endowedwith (or acquire) prior expectations that the causes ofour sensations are largely independent and sparse.
In summary, the principle of efficient coding saysthat the brain should optimize the mutual information
between its sensory signals and some parsimonious
neuronal representations. This is the same as optimizing
the parameters of a generative model to maximize theaccuracy of predictions, under complexity constraints.
Both are mandated by the free-energy principle, which
can be regarded as a probabilistic generalization of the
infomax principle. We now turn to more biologicallyinspired ideas about brain function that focus on neuronal dynamics and plasticity. This takes us deeper into
neurobiological mechanisms and the implementation of
the theoretical principles outlined above.

The cell assembly and correlation theory
The cell assembly theory was proposed by Hebb65 andentails Hebbian — or associative — plasticity, which is acornerstone of use-dependent or experience-dependent
plasticity 66, the correlation theory of von de Malsburg 67,68
and other formal refinements to Hebbian plasticity
per se69. The cell assembly theory posits that groups ofinterconnected neurons are formed through a strengthening of synaptic connections that depends on correlated pre- and postsynaptic activity; that is, ‘cells that fire
together wire together’. This enables the brain to distil
statistical regularities from the sensorium. The correlation theory considers the selective enabling of synaptic
efficacy and its plasticity (also known as metaplasticity 70) by fast synchronous activity induced by different
perceptual attributes of the same object (for example, a
red bus in motion). This resolves a putative deficiencyof classical plasticity, which cannot ascribe a presynaptic
input to a particular cause (for example, redness) in theworld67. The correlation theory underpins theoreticaltreatments of synchronized brain activity and its role inassociating or binding attributes to specific objects or
causes68,71. Another important field that rests on associative plasticity is the use of attractor networks as modelsof memory formation and retrieval72–74. so how do correlations and associative plasticity figure in the free-energy
formulation?
Hitherto, we have considered only inference on statesof the world that cause sensory signals, whereby conditional expectations about states are encoded by synaptic
activity. However, the causes covered by the recognition
density are not restricted to time-varying states (forexample, the motion of an object in the visual field):they also include time-invariant regularities that endow
the world with causal structure (for example, objects
fall with constant acceleration). These regularities areparameters of the generative model and have to beinferred by the brain — in other words, the conditional
expectations of these parameters that may be encodedby synaptic efficacy (these are μθ in BOX 2) have to be
optimized. This corresponds to optimizing connection
strengths in the brain — that is, plasticity that underlines learning. so what form would this learning take? Ittranspires that a gradient descent on free energy (that is,changing connections to reduce free energy) is formally
identical to Hebbian plasticity 28,42 (BOX 2). This is because
the parameters of the generative model determine how
expected states (synaptic activity) are mixed to form predictions. put simply, when the presynaptic predictions
and postsynaptic prediction errors are highly correlated,the connection strength increases, so that predictions
can suppress prediction errors more efficiently.
In short, the formation of cell assemblies reflects theencoding of causal regularities. This is just a restatement of cell assembly theory in the context of a specificimplementation (predictive coding) of the free-energy
principle. It should be acknowledged that the learning
rule in predictive coding is really a delta rule, which
rests on Hebbian mechanisms; however, Hebb’s wider
notions of cell assemblies were formulated from a nonstatistical perspective. Modern reformulations suggestthat both inference on states (that is, perception) andinference on parameters (that is, learning) minimize
free energy (that is, minimize prediction error) andserve to bound surprising exchanges with the world. sowhat about synchronization and the selective enablingof synapses?

Biased competition and attention
causal regularities encoded by synaptic efficacy
control the deterministic evolution of states in the world.
However, stochastic (that is, random) fluctuations inthese states play an important part in generating sensory data. Their amplitude is usually represented as precision (or inverse variance), which encodes the reliabilityof prediction errors. precision is important, especiallyin hierarchical schemes, because it controls the relative
influence of bottom-up prediction errors and top-down
predictions. so how is precision encoded in the brain?
In predictive coding, precision modulates the amplitudeof prediction errors (these are μγ in BOX 2), so that prediction errors with high precision have a greater impact
on units that encode conditional expectations. Thismeans that precision corresponds to the synaptic gain ofprediction error units. The most obvious candidates forcontrolling gain (and implicitly encoding precision) areclassical neuromodulators like dopamine and acetylcholine, which provides a nice link to theories of attention
and uncertainty 75–77. Another candidate is fast synchronized presynaptic input that lowers effective postsynaptic
membrane time constants and increases synchronous
gain78. This fits comfortably with the correlation theory
and speaks to recent ideas about the role of synchronous
activity in mediating attentional gain79,80.
In summary, the optimization of expected precisionin terms of synaptic gain links attention to synaptic gain
and synchronization. This link is central to theories ofattentional gain and biased competition80–85, particularlyin the context of neuromodulation86,87. The theories
considered so far have dealt only with perception.

6 | ADvANcE oNlINE puBlIcATIoN

www.nature.com/reviews/neuro© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
However, from the point of view of the free-energy
principle, perception just makes free energy a goodproxy for surprise. To actually reduce surprise we needto act. In the next section, we retain a focus on cell
assemblies but move to the selection and reinforcementof stimulus–response links.

Reentrant signalling
Reciprocal message passing
among neuronal groups.

Reinforcement learning
An area of machine learning
concerned with how an agent
maximizes long-term reward.
Reinforcement learningalgorithms attempt to find apolicy that maps states of theworld to actions performed by
the agent.

Optimal control theory
An optimization method(based on the calculus ofvariations) for deriving an
optimal control law in a
dynamical system. A control
problem includes a costfunction that is a function ofstate and control variables.

Bellman equation(Or dynamic programming
equation.) Named after
Richard Bellman, it is anecessary condition for
optimality associated withdynamic programming in
optimal control theory.

Neural Darwinism and value learning
In the theory of neuronal group selection88, the emergenceof neuronal assemblies is considered in the light of selective pressure. The theory has four elements: epigenetic
mechanisms create a primary repertoire of neuronalconnections, which are refined by experience-dependent
plasticity to produce a secondary repertoire of neuronal groups. These are selected and maintained throughreentrant signalling among neuronal groups. As in cell
assembly theory, plasticity rests on correlated pre- andpostsynaptic activity, but here it is modulated by value.
value is signalled by ascending neuromodulatory transmitter systems and controls which neuronal groups
are selected and which are not. The beauty of neuralDarwinism is that it nests distinct selective processes
within each other. In other words, it eschews a single unitof selection and exploits the notion of meta-selection(the selection of selective mechanisms; for example, see
REF. 89). In this context, (neuronal) value confers evolutionary value (that is, adaptive fitness) by selecting neuronal groups that meditate adaptive stimulus–stimulus
associations and stimulus–response links. The capacityof value to do this is assured by natural selection, in thesense that neuronal value systems are themselves subject
to selective pressure.
This theory, particularly value-dependent learning90,
has deep connections with reinforcement learning andrelated approaches in engineering (see below), such as
dynamic programming and temporal difference models91,92. This is because neuronal value systems reinforce
connections to themselves, thereby enabling the brain
to label a sensory state as valuable if, and only if, it leads toanother valuable state. This ensures that agents movethrough a succession of states that have acquired value toaccess states (rewards) with genetically specified innate
value. In short, the brain maximizes value, which may bereflected in the discharge of value systems (for example,
dopaminergic systems92–96). so how does this relate tothe optimization of free energy?
The answer is simple: value is inversely proportional
to surprise, in the sense that the probability of a phenotype being in a particular state increases with the valueof that state. Furthermore, the evolutionary value ofa phenotype is the negative surprise averaged over all
the states it experiences, which is simply its negative
entropy. Indeed, the whole point of minimizing freeenergy (and implicitly entropy) is to ensure that agents
spend most of their time in a small number of valuablestates. This means that free energy is the complement ofvalue, and its long-term average is the complement ofadaptive fitness (also known as free fitness in evolutionary biology 97). But how do agents know what is valuable? In other words, how does one generation tell thenext which states have value (that is, are unsurprising)?

value or surprise is determined by the form of an agent’s
generative model and its implicit priors — these specify
the value of sensory states and, crucially, are heritablethrough genetic and epigenetic mechanisms. This meansthat prior expectations (that is, the primary repertoire)can prescribe a small number of attractive states withinnate value. In turn, this enables natural selection to
optimize prior expectations and ensure they are consistent with the agent’s phenotype. put simply, valuablestates are just the states that the agent expects to frequent. These expectations are constrained by the form of
its generative model, which is specified genetically andfulfilled behaviourally, under active inference.
It is important to appreciate that prior expectations
include not just what will be sampled from the world but
also how the world is sampled. This means that natural
selection may equip agents with the prior expectationthat they will explore their environment until stateswith innate value are encountered. We will look at thismore closely in the next section, where priors on motion
through state space are cast in terms of policies inreinforcement learning.
Both neural Darwinism and the free-energy principle
try to understand somatic changes in an individual in
the context of evolution: neural Darwinism appeals to
selective processes, whereas the free energy formulationconsiders the optimization of ensemble or population
dynamics in terms of entropy and surprise. The key
theme that emerges here is that (heritable) prior expectations can label things as innately valuable (unsurprising);
but how can simply labelling states engender adaptive
behaviour? In the next section, we return to reinforcement learning and related formulations of action that try
to explain adaptive behaviour purely in terms of labelsor cost functions.

Optimal control theory and game theory
value is central to theories of brain function that are
based on reinforcement learning and optimum control. The basic notion that underpins these treatments
is that the brain optimizes value, which is expected
reward or utility (or its complement — expected loss
or cost). This is seen in behavioural psychology as reinforcement learning 98, in computational neuroscienceand machine learning as variants of dynamic programming such as temporal difference learning 99–101, and ineconomics as expected utility theory 102. The notion ofan expected reward or cost is crucial here; this is thecost expected over future states, given a particular policythat prescribes action or choices. A policy specifies thestates to which an agent will move from any given state(‘motion through state space in continuous time’). This
policy has to access sparse rewarding states using a costfunction, which only labels states as costly or not. The
problem of how the policy is optimized is formalizedin optimal control theory as the Bellman equation and its
variants99 (see supplementary information s4 (box)),which express value as a function of the optimal policy
and a cost function. If one can solve the Bellman equation, one can associate each sensory state with a value
and optimize the policy by ensuring that the next state

NATuRE REvIEWs | NeuroscieNce

ADvANcE oNlINE puBlIcATIoN | 7© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
is the most valuable of the available states. In general,
it is impossible to solve the Bellman equation exactly,
but several approximations exist, ranging from simple
Rescorla–Wagner models98 to more comprehensive formulations like Q-learning 100. cost also has a key role in
Bayesian decision theory, in which optimal decisions
minimize expected cost in the context of uncertainty
about outcomes; this is central to optimal decision theory(game theory) and behavioural economics102–104.
so what does free energy bring to the table? If one
assumes that the optimal policy performs a gradientascent on value, then it is easy to show that value isinversely proportional to surprise (see supplementary
information s4 (box)). This means that free energy is(an upper bound on) expected cost, which makes sense
as optimal control theory assumes that action minimizes expected cost, whereas the free-energy principle
states that it minimizes free energy. This is important

Optimal decision theory(Or game theory.) An area of
applied mathematics
concerned with identifying thevalues, uncertainties and otherconstraints that determine an
optimal decision.

Gradient ascent(Or method of steepest
ascent.) A first-order
optimization scheme that finds
a maximum of a function bychanging its arguments inproportion to the gradient of
the function at the current
value. In short, a hill-climbing
scheme. The opposite scheme
is a gradient descent.

ξx(1)Predictions
Prediction errors

ξ(2)
v
μv(1)

μx(1)

ξv(1)

Movement
trajectoryVsvisual =+ wvisualJ(0, 0)

Motorsignalsx1
sprop =

x1
+ wprop
x2

ξv(1)
a

Actionȧ = −∂aεTξ

J1
V = (v1, v2 , v3)
x2

J2

Jointed arm

J = J1 + J2 = ( j1, j2 )Figure 2 | A demonstration of cued reaching movements. The lower right part of the
figure shows a motor plant, comprising a two-jointed arm with two hidden states, each ofwhich corresponds to a particular angular position of the two
joints;Reviews
the current
position
Nature| Neuroscienceof the finger (red circle) is the sum of the vectors describing the location of each joint.
Here, causal states in the world are the position and brightness of the target (green
circle). The arm obeys Newtonian mechanics, specified in terms of angular inertia andfriction. The left part of the figure illustrates that the brain senses hidden states directlyin terms of proprioceptive input (Sprop) that signals the angular positions (x1,x2) of thejoints and indirectly through seeing the location of the finger in space (J1,J2). In addition,
through visual input (Svisual) the agent senses the target location (v1,v2) and brightness (v3).
Sensory prediction errors are passed to higher brain levels to optimize the conditional
expectations of hidden states (that is, the angular position of the joints) and causal (thatis, target) states. The ensuing predictions are sent back to suppress sensory prediction
errors. At the same time, sensory prediction errors are also trying to suppress themselvesby changing sensory input through action. The grey and black lines denote reciprocal
message passing among neuronal populations that encode prediction error andconditional expectations; this architecture is the same as that depicted in BOX 2. The
blue lines represent descending motor control signals from sensory prediction-error
units. The agent’s generative model included priors on the motion of hidden states that
effectively engage an invisible elastic band between the finger and target (when thetarget is illuminated). This induces a prior expectation that the finger will be drawn to
the target, when cued appropriately. The insert shows the ensuing movement trajectory
caused by action. The red circles indicate the initial and final positions of the finger,
which reaches the target (green circle) quickly and smoothly; the blue line is thesimulated trajectory.

because it explains why agents must minimize expected
cost. Furthermore, free energy provides a quantitativeand seamless connection between the cost functionsof reinforcement learning and value in evolutionary
biology. Finally, the dynamical perspective provides amechanistic insight into how policies are specified in thebrain: according to the principle of optimality99 cost is therate of change of value (see supplementary informations4 (box)), which depends on changes in sensory states.
This suggests that optimal policies can be prescribed byprior expectations about the motion of sensory states.
put simply, priors induce a fixed-point attractor, andwhen the states arrive at the fixed point, value will stopchanging and cost will be minimized. A simple example is shown in FIG. 2, in which a cued arm movement
is simulated using only prior expectations that the arm
will be drawn to a fixed point (the target). This figure
illustrates how computational motor control105–109 canbe formulated in terms of priors and the suppression of
sensory prediction errors (K.J.F., J. Daunizeau, J. Kilnerand s.J. Kiebel, unpublished observations). More generally, it shows how rewards and goals can be considered
as prior expectations that an action is obliged to fulfil16(see also REF. 110). It also suggests how natural selection
could optimize behaviour through the genetic specification of inheritable or innate priors that constrain thelearning of empirical priors (BOX 2) and subsequent goaldirected action.
It should be noted that just expecting to be attracted
to some states may not be sufficient to attain those states.
This is because one may have to approach attractors vicariously through other states (for example, to avoid obstacles) or conform to physical constraints on action. These
are some of the more difficult problems of accessing
distal rewards that reinforcement learning and optimum control contend with. In these circumstances,
an examination of the density dynamics, on which thefree-energy principle is based, suggests that it is sufficient
to keep moving until an a priori attractor is encountered(see supplementary information s5 (box)). This entails
destroying unexpected (costly) fixed points in the environment by making them unstable (like shifting to a newposition when sitting uncomfortably). Mathematically,
this means adopting a policy that ensures a positive
divergence in costly states (intuitively, this is like being
pushed through a liquid with negative viscosity or
friction). see FIG. 3 for a solution to the classicalmountain car problem using a simple prior that induces
this sort of policy. This prior is on motion through state
space (that is, changes in states) and enforces exploration
until an attractive state is found. priors of this sort may
provide a principled way to understand the exploration–
exploitation trade-off111–113 and related issues in evolutionary biology 114. The implicit use of priors to induce
dynamical instability also provides a key connection
to dynamical systems theory approaches to the brainthat emphasize the importance of itinerant dynamics,
metastability, self-organized criticality and winnerless competition115–123. These dynamical phenomena
have a key role in synergetic and autopoietic accounts ofadaptive behaviour 5,124,125.

8 | ADvANcE oNlINE puBlIcATIoN

www.nature.com/reviews/neuro© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWSb

The mountain car problem

Conditional expectations

Loss functions (priors)

0.7

5

0.6

0

25

0.5

–5

20

0.4

–10

ϕ(x)

–20

0.1

–25
-2

-1

0

1

2

c(x)

–15

0.2

0

30

Estimated states

0.3

Force

Height

a

−c(t)

15
10
5

μ(t)x

0

–30
–2

–1

0

1

2

–5

0

20

Position (x)Position (x)

40

60

80

100

120

Time (seconds)

Principle of optimality

Equations of motion

Exploration–exploitation
trade-offInvolves a balance between
exploration (of uncharted
territory) and exploitation (ofcurrent knowledge). In
reinforcement learning, it has
been studied mainly throughthe multi-armed bandit
problem.

Dynamical systems theory
An area of appliedmathematics that describes
the behaviour of complex(possibly chaotic) dynamicalsystems as described bydifferential or difference
equations.

Synergetics
Concerns the self-organizationof patterns and structures inopen systems far fromthermodynamic equilibrium. It
rests on the order parameter
concept, which was generalized
by Haken to the enslaving
principle: that is, the dynamics
of fast-relaxing (stable) modes
are completely determined bythe ‘slow’ dynamics of orderparameters (the amplitudes of
unstable modes).

Autopoietic
Referring to the fundamental
dialectic between structureand function.

Helmholtzian
Refers to a device or schemethat uses a generative model tofurnish a recognition density
and learns hidden structures indata by optimizing theparameters of generativemodels.

ẋx′=ẋ′ −∇xϕ − 1⁄8 x′ + σ(a)Action3x′21Velocityf=Trajectories
2

Control signal

An optimal policy has
the property that whatever the
initial state and initial decision,
the remaining decisions must
constitute an optimal policy
with regard to the state
resulting from the first decision.

0

–1

a(t)

1
0
–1

–2
–2
–2

–1

0

Position (x)

1

2

–3

0

20

40

60

80

100

120

Time (seconds)

Figure 3 | solving the mountain car problem with prior expectations. a | How paradoxical but adaptive behaviour (forNature
Reviews| Neuroscienceexample, moving away from a target to ensure that it is secured later) emerges from simple priors
on themotionof hidden
states in the world. Shown is the landscape or potential energy function (with a minimum at position x = –0.5) that exerts
forces on a mountain car. The car is shown at the target position on the hill at x =1, indicated by the red circle. The equationsof motion of the car are shown below the plot. Crucially, at x = 0 the force on the car cannot be overcome by the agent,because a squashing function –1≤σ≤1 is applied to action to prevent it being greater than 1. This means that the agent canaccess the target only by starting halfway up the left hill to gain enough momentum to carry it up the other side. b | The
results of active inference under priors that destabilize fixed points outside the target domain. The priors are encoded in acost function c(x) (top left), which acts like negative friction. When ‘friction’ is negative the car expects to go faster (seeSupplementary information S5 (box) for details). The inferred hidden states (upper right: position in blue, velocity in green
and negative dissipation in red) show that the car explores its landscape until it encounters the target, and that friction thenincreases (that is, cost decreases) dramatically to prevent the car from escaping the target (by falling down the hill). The
ensuing trajectory is shown in blue (bottom left). The paler lines provide exemplar trajectories from other trials, withdifferent starting positions. In the real world, friction is constant. However, the car ‘expects’ friction to change as it changes
position, thus enforcing exploration or exploitation. These expectations are fulfilled by action (lower right).

In summary, optimal control and decision (game)theory start with the notion of cost or utility and try toconstruct value functions of states, which subsequently
guide action. The free-energy formulation starts witha free-energy bound on the value of states, which is
specified by priors on the motion of hidden environmental states. These priors can incorporate any costfunction to ensure that costly states are avoided. stateswith minimum cost can be set (by learning or evolution) in terms of prior expectations about motion andthe attractors that ensue. In this view, the problem offinding sparse rewards in the environment is nature’s
solution to the problem of how to minimize the entropy(average surprise or free energy) of an agent’s states: byensuring they occupy a small set of attracting (that is,rewarding) states.

Conclusions and future directions
Although contrived to highlight commonalities, thisReview suggests that many global theories of brain
function can be united under a Helmholtzian perceptive of the brain as a generative model of the world it
inhabits18,20,21,25 (FIG. 4); notable examples include theintegration of the Bayesian brain and computational
motor control theory, the objective functions sharedby predictive coding and the infomax principle,
hierarchical inference and theories of attention, theembedding of perception in natural selection andthe link between optimum control and more exotic
phenomena in dynamical systems theory. The constanttheme in all these theories is that the brain optimizesa (free-energy) bound on surprise or its complement,
value. This manifests as perception (so as to change

NATuRE REvIEWs | NeuroscieNce

ADvANcE oNlINE puBlIcATIoN | 9© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
Attention and biased competition

Computational motor control

μγ = arg min ∫dtF

ȧ = −∂aεTξ

Optimization of synaptic gain
representing the precision(salience) of predictions

Minimization of sensory
prediction errors
Predictive coding and hierarchical inferenceμ̇v(i) = Dμv(i) − ∂vε (i) Tξ (i) − ξv (i + 1)

Associative plasticity
�μθ ij = −∂θ ijεTξ
Optimization of synaptic efficacy
Perceptual learning and memoryμθ = arg min ∫dtF
Optimization of synaptic efficacy
to represent causal structurein the sensorium
Probabilistic neuronal codingq(ϑ ) = N ( μ, Σ)
Encoding a recognition density
in terms of conditional
expectations and uncertainty

Minimization of prediction error
with recurrent message passing

Optimal control and value learninga, μ = arg max V (s~ | m)

The Bayesian brain hypothesis
μ = arg min D (q(ϑ) || (p(ϑ | s~ ))Optimization of a free-energy
bound on surprise or value

KL

Minimizing the difference between arecognition density and the conditional
density on sensory causes
The free-energy principle
a, μ, m = arg min F (s~, μ | m)

Infomax and the redundancy
minimization principle
μ = arg max {I (s~, μ ) − H(μ)}

Minimization of the free energy ofsensations and the representationof their causes

Maximization of the mutual
information between sensations
and representations

Model selection and evolution
m = arg min ∫dtF
Optimizing the agent’s model andpriors through neurodevelopment
and natural selection

Figure 4 | The free-energy principle and other theories. Some of the theoretical constructs considered in this Review
Reviews| Neuroscienceand how they relate to the free-energy principle (centre). The variables are described in BOXESNature
1,2 anda full explanationof the equations can be found in the Supplementary information S1–S4 (boxes).

predictions) or action (so as to change the sensations
that are predicted). crucially, these predictions dependon prior expectations (that furnish policies), which
are optimized at different (somatic and evolutionary)timescales and define what is valuable.
What does the free-energy principle portend for the
future? If its main contribution is to integrate established theories, then the answer is probably ‘not a lot’.
conversely, it may provide a framework in which current debates could be resolved, for example whether
dopamine encodes reward prediction error or surprise126,127 — this is particularly important for understanding conditions like addiction, parkinson’s disease
and schizophrenia. Indeed, the free-energy formulation
has already been used to explain the positive symptomsof schizophrenia in terms of false inference128. The freeenergy formulation could also provide new approaches

1.
2.

Huang, G. Is this a unified theory of the brain?
New Scientist 2658, 30–33 (2008).
Friston K., Kilner, J. & Harrison, L. A free energy
principle for the brain. J. Physiol. Paris 100, 70–87(2006).
An overview of the free-energy principle that
describes its motivation and relationship to
generative models and predictive coding. This

3.
4.
5.

to old problems that might call for a reappraisal of
conventional notions, particularly in reinforcement
learning and motor control.
If the arguments underlying the free-energy principle
hold, then the real challenge is to understand how it
manifests in the brain. This speaks to a greater appreciation of hierarchical message passing 41, the functional role of specific neurons and microcircuits andthe dynamics they support (for example, what is therelationship between predictive coding, attention
and dynamic coordination in the brain?129). Beyond
neuroscience, many exciting applications in engineering,robotics, embodied cognition and evolutionary biology
suggest themselves; although fanciful, it is not difficult toimagine building little free-energy machines that garnerand model sensory information (like our children) tomaximize the evidence for their own existence.

paper focuses on perception and theneurobiological infrastructures involved.
Ashby, W. R. Principles of the self-organising dynamic
system. J. Gen. Psychol. 37, 125–128 (1947).
Nicolis, G. & Prigogine, I. Self‑Organisation in Non‑
Equilibrium Systems (Wiley, New York, 1977).
Haken, H. Synergistics: an Introduction. Non‑
Equilibrium Phase Transition and Self‑Organisation in

10 | ADvANcE oNlINE puBlIcATIoN

6.
7.

Physics, Chemistry and Biology 3rd edn (Springer,
New York, 1983).
Kauffman, S. The Origins of Order: Self‑Organizationand Selection in Evolution (Oxford Univ. Press, Oxford,
1993).
Bernard, C. Lectures on the Phenomena Common
to Animals and Plants (Thomas, Springfield,
1974).

www.nature.com/reviews/neuro© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
8.
9.
10.
11.
12.

13.
14.
15.
16.
17.

18.
19.
20.
21.
22.
23.
24.

25.

26.
27.
28.
29.
30.
31.

32.

33.
34.
35.

Applebaum, D. Probability and Information: an
Integrated Approach (Cambridge Univ. Press,
Cambridge, UK, 2008).
Evans, D. J. A non-equilibrium free energy theorem
for deterministic systems. Mol. Physics 101,
15551–11554 (2003).
Crauel, H. & Flandoli, F. Attractors for random
dynamical systems. Probab. Theory Relat. Fields 100,
365–393 (1994).
Feynman, R. P. Statistical Mechanics: a Set of Lectures(Benjamin, Reading, Massachusetts, 1972).
Hinton, G. E. & von Cramp, D. Keeping neuralnetworks simple by minimising the description lengthof weights. Proc. 6th Annu. ACM Conf. Computational
Learning Theory 5–13 (1993).
MacKay. D. J. C. Free-energy minimisation algorithm
for decoding and cryptoanalysis. Electron. Lett. 31,
445–447 (1995).
Neal, R. M. & Hinton, G. E. in Learning in GraphicalModels (ed. Jordan, M. I.) 355–368 (Kluwer
Academic, Dordrecht, 1998).
Itti, L. & Baldi, P. Bayesian surprise attracts humanattention. Vision Res. 49, 1295–1306 (2009).
Friston, K., Daunizeau, J. & Kiebel, S. Active inference
or reinforcement learning? PLoS ONE 4, e6421(2009).
Knill, D. C. & Pouget, A. The Bayesian brain: the roleof uncertainty in neural coding and computation.
Trends Neurosci. 27, 712–719 (2004).
A nice review of Bayesian theories of perception
and sensorimotor control. Its focus is on Bayes
optimality in the brain and the implicit nature ofneuronal representations.
von Helmholtz, H. in Treatise on Physiological Optics
Vol. III 3rd edn (Voss, Hamburg, 1909).
MacKay, D. M. in Automata Studies (eds Shannon,
C. E. & McCarthy, J.) 235–251 (Princeton Univ. Press,
Princeton, 1956).
Neisser, U. Cognitive Psychology(Appleton-Century-Crofts, New York, 1967).
Gregory, R. L. Perceptual illusions and brain models.
Proc. R. Soc. Lond. B Biol. Sci. 171, 179–196 (1968).
Gregory, R. L. Perceptions as hypotheses. Philos.
Trans. R. Soc. Lond. B Biol. Sci. 290, 181–197 (1980).
Ballard, D. H., Hinton, G. E. & Sejnowski, T. J. Parallelvisual computation. Nature 306, 21–26 (1983).
Kawato, M., Hayakawa, H. & Inui, T. A forward-inverse
optics model of reciprocal connections between visualareas. Network: Computation in Neural Systems 4,
415–422 (1993).
Dayan, P., Hinton, G. E. & Neal, R. M. The Helmholtz
machine. Neural Comput. 7, 889–904 (1995).
This paper introduces the central role of generativemodels and variational approaches to hierarchicalself-supervised learning and relates this to thefunction of bottom-up and top-down corticalprocessing pathways.
Lee, T. S. & Mumford, D. Hierarchical Bayesianinference in the visual cortex. J. Opt. Soc. Am. A Opt.
Image Sci. Vis. 20, 1434–1448 (2003).
Kersten, D., Mamassian, P. & Yuille, A. Object
perception as Bayesian inference. Annu. Rev. Psychol.
55, 271–304 (2004).
Friston, K. J. A theory of cortical responses. Philos.
Trans. R. Soc. Lond. B Biol. Sci. 360, 815–836(2005).
Beal, M. J. Variational Algorithms for Approximate
Bayesian Inference. Thesis, University College London(2003).
Efron, B. & Morris, C. Stein’s estimation rule and its
competitors – an empirical Bayes approach. J. Am.
Stats. Assoc. 68, 117–130 (1973).
Kass, R. E. & Steffey, D. Approximate Bayesianinference in conditionally independent hierarchicalmodels (parametric empirical Bayes models). J. Am.
Stat. Assoc. 407, 717–726 (1989).
Zeki, S. & Shipp, S. The functional logic of corticalconnections. Nature 335, 311–317 (1988).
Describes the functional architecture of corticalhierarchies with a focus on patterns of anatomical
connections in the visual cortex. It emphasizes therole of functional segregation and integration (thatis, message passing among cortical areas).
Felleman, D. J. & Van Essen, D. C. Distributed
hierarchical processing in the primate cerebral cortex.
Cereb. Cortex 1, 1–47 (1991).
Mesulam, M. M. From sensation to cognition. Brain
121, 1013–1052 (1998).
Sanger, T. Probability density estimation for theinterpretation of neural population codes.
J. Neurophysiol. 76, 2790–2793 (1996).

36. Zemel, R., Dayan, P. & Pouget, A. Probabilistic
interpretation of population code. Neural Comput. 10,
403–430 (1998).
37. Paulin, M. G. Evolution of the cerebellum as aneuronal machine for Bayesian state estimation.
J. Neural Eng. 2, S219–S234 (2005).
38. Ma, W. J., Beck, J. M., Latham, P. E. & Pouget, A.
Bayesian inference with probabilistic population
codes. Nature Neurosci. 9, 1432–1438 (2006).
39. Friston, K., Mattout, J., Trujillo-Barreto, N.,
Ashburner, J. & Penny, W. Variational free energy andthe Laplace approximation. Neuroimage 34,
220–234 (2007).
40. Rao, R. P. & Ballard, D. H. Predictive coding in thevisual cortex: a functional interpretation of someextra-classical receptive field effects. Nature Neurosci.
2, 79–87 (1998).
Applies predictive coding to cortical processing toprovide a compelling account of extra-classicalreceptive fields in the visual system. It emphasizes
the importance of top-down projections inproviding predictions, by modelling perceptual
inference.
41. Mumford, D. On the computational architecture of the
neocortex. II. The role of cortico-cortical loops. Biol.
Cybern. 66, 241–251 (1992).
42. Friston, K. Hierarchical models in the brain. PLoS
Comput. Biol. 4, e1000211 (2008).
43. Murray, S. O., Kersten, D., Olshausen, B. A., Schrater, P.
& Woods, D. L. Shape perception reduces activity inhuman primary visual cortex. Proc. Natl Acad. Sci.
USA 99, 15164–15169 (2002).
44. Garrido, M. I., Kilner, J. M., Kiebel, S. J. & Friston,
K. J. Dynamic causal modeling of the response tofrequency deviants. J. Neurophysiol. 101,
2620–2631 (2009).
45. Sherman, S. M. & Guillery, R. W. On the actions that
one nerve cell can have on another: distinguishing“drivers” from “modulators”. Proc. Natl Acad. Sci. USA
95, 7121–7126 (1998).
46. Angelucci, A. & Bressloff, P. C. Contribution of
feedforward, lateral and feedback connections to theclassical receptive field center and extra-classicalreceptive field surround of primate V1 neurons.
Prog. Brain Res. 154, 93–120 (2006).
47. Grossberg, S. Towards a unified theory of neocortex:
laminar cortical circuits for vision and cognition.
Prog. Brain Res. 165, 79–104 (2007).
48. Grossberg, S. & Versace, M. Spikes, synchrony, andattentive learning by laminar thalamocortical circuits.
Brain Res. 1218, 278–312 (2008).
49. Barlow, H. in Sensory Communication (ed. Rosenblith, W.)
217–234 (MIT Press, Cambridge, Massachusetts,
1961).
50. Linsker, R. Perceptual neural organisation: someapproaches based on network models andinformation theory. Annu. Rev. Neurosci. 13,
257–281 (1990).
51. Oja, E. Neural networks, principal components, andsubspaces. Int. J. Neural Syst. 1, 61–68 (1989).
52. Bell, A. J. & Sejnowski, T. J. An information
maximisation approach to blind separation and blindde-convolution. Neural Comput. 7, 1129–1159(1995).
53. Atick, J. J. & Redlich, A. N. What does the retina know
about natural scenes? Neural Comput. 4, 196–210(1992).
54. Optican, L. & Richmond, B. J. Temporal encoding of
two-dimensional patterns by single units in primate
inferior cortex. III Information theoretic analysis.
J. Neurophysiol. 57, 132–146 (1987).
55. Olshausen, B. A. & Field, D. J. Emergence of simplecell receptive field properties by learning a sparsecode for natural images. Nature 381, 607–609(1996).
56. Simoncelli, E. P. & Olshausen, B. A. Natural image
statistics and neural representation. Annu. Rev.
Neurosci. 24, 1193–1216 (2001).
A nice review of information theory in visual
processing. It covers natural scene statistics andempirical tests of the efficient coding hypothesis inindividual neurons and populations of neurons.
57. Friston, K. J. The labile brain. III. Transients andspatio-temporal receptive fields. Philos. Trans. R. Soc.
Lond. B Biol. Sci. 355, 253–265 (2000).
58. Bialek, W., Nemenman, I. & Tishby, N. Predictability,
complexity, and learning. Neural Comput. 13,
2409–2463 (2001).
59. Lewen, G. D., Bialek, W. & de Ruyter van Steveninck,
R. R. Neural coding of naturalistic motion stimuli.
Network 12, 317–329 (2001).

NATuRE REvIEWs | NeuroscieNce

60. Laughlin, S. B. Efficiency and complexity in neuralcoding. Novartis Found. Symp. 239, 177–187(2001).
61. Tipping, M. E. Sparse Bayesian learning and theRelevance Vector Machine. J. Machine Learn. Res. 1,
211–244 (2001).
62. Paus, T., Keshavan, M. & Giedd, J. N. Why do many
psychiatric disorders emerge during adolescence?
Nature Rev. Neurosci. 9, 947–957 (2008).
63. Gilestro, G. F., Tononi, G. & Cirelli, C. Widespreadchanges in synaptic markers as a function of sleep andwakefulness in Drosophila. Science 324, 109–112(2009).
64. Roweis, S. & Ghahramani, Z. A unifying review of
linear Gaussian models. Neural Comput. 11, 305–345(1999).
65. Hebb, D. O. The Organization of Behaviour (Wiley,
New York, 1949).
66. Paulsen, O. & Sejnowski, T. J. Natural patterns ofactivity and long-term synaptic plasticity. Curr. Opin.
Neurobiol. 10, 172–179 (2000).
67. von der Malsburg, C. The Correlation Theory of Brain
Function. Internal Report 81–82, Dept. Neurobiology,
Max-Planck-Institute for Biophysical Chemistry(1981).
68. Singer, W. & Gray, C. M. Visual feature integration andthe temporal correlation hypothesis. Annu. Rev.
Neurosci. 18, 555–586 (1995).
69. Bienenstock, E. L., Cooper, L. N. & Munro, P. W.
Theory for the development of neuron selectivity:orientation specificity and binocular interaction invisual cortex. J. Neurosci. 2, 32–48 (1982).
70. Abraham, W. C. & Bear, M. F. Metaplasticity: theplasticity of synaptic plasticity. Trends Neurosci. 19,
126–130 (1996).
71. Pareti, G. & De Palma, A. Does the brain oscillate?
The dispute on neuronal synchronization. Neurol. Sci.
25, 41–47 (2004).
72. Leutgeb, S., Leutgeb, J. K., Moser, M. B. & Moser, E. I.
Place cells, spatial maps and the population code formemory. Curr. Opin. Neurobiol. 15, 738–746(2005).
73. Durstewitz, D. & Seamans, J. K. Beyond bistability:
biophysics and temporal dynamics of working memory.
Neuroscience 139, 119–133 (2006).
74. Anishchenko, A. & Treves, A. Autoassociative memory
retrieval and spontaneous activity bumps in smallworld networks of integrate-and-fire neurons.
J. Physiol. Paris 100, 225–236 (2006).
75. Abbott, L. F., Varela, J. A., Sen, K. & Nelson, S. B.
Synaptic depression and cortical gain control. Science
275, 220–224 (1997).
76. Yu, A. J. & Dayan, P. Uncertainty, neuromodulation
and attention. Neuron 46, 681–692 (2005).
77. Doya, K. Metalearning and neuromodulation. NeuralNetw. 15, 495–506 (2002).
78. Chawla, D., Lumer, E. D. & Friston, K. J. The
relationship between synchronization among neuronal
populations and their mean activity levels. NeuralComput. 11, 1389–1411 (1999).
79. Fries, P., Womelsdorf, T., Oostenveld, R. & Desimone, R.
The effects of visual stimulation and selective visual
attention on rhythmic neuronal synchronization in
macaque area V4. J. Neurosci. 28, 4823–4835(2008).
80. Womelsdorf, T. & Fries, P. Neuronal coherence during
selective attentional processing and sensory-motor
integration. J. Physiol. Paris 100, 182–193 (2006).
81. Desimone, R. Neural mechanisms for visual memory
and their role in attention. Proc. Natl Acad. Sci. USA
93, 13494–13499 (1996).
A nice review of mnemonic effects (such as
repetition suppression) on neuronal responses andhow they bias the competitive interactions between
stimulus representations in the cortex. It provides
a good perspective on attentional mechanisms inthe visual system that is empirically grounded.
82. Treisman, A. Feature binding, attention and object
perception. Philos. Trans. R. Soc. Lond. B Biol. Sci.
353, 1295–1306 (1998).
83. Maunsell, J. H. & Treue, S. Feature-based attention invisual cortex. Trends Neurosci. 29, 317–322 (2006).
84. Spratling, M. W. Predictive-coding as a model ofbiased competition in visual attention. Vision Res. 48,
1391–1408 (2008).
85. Reynolds, J. H. & Heeger, D. J. The normalization
model of attention. Neuron 61, 168–185 (2009).
86. Schroeder, C. E., Mehta, A. D. & Foxe, J. J.
Determinants and mechanisms of attentional
modulation of neural processing. Front. Biosci. 6,
D672–D684 (2001).

ADvANcE oNlINE puBlIcATIoN | 11
© 2010 Macmillan Publishers Limited. All rights reserved

REVIEWS
87. Hirayama, J., Yoshimoto, J. & Ishii, S. Bayesian
representation learning in the cortex regulated byacetylcholine. Neural Netw. 17, 1391–1400 (2004).
88. Edelman, G. M. Neural Darwinism: selection andreentrant signaling in higher brain function. Neuron
10, 115–125 (1993).
89. Knobloch, F. Altruism and the hypothesis of metaselection in human evolution. J. Am. Acad.
Psychoanal. 29, 339–354 (2001).
90. Friston, K. J., Tononi, G., Reeke, G. N. Jr, Sporns, O. &Edelman, G. M. Value-dependent selection in thebrain: simulation in a synthetic neural model.
Neuroscience 59, 229–243 (1994).
91. Sutton, R. S. & Barto, A. G. Toward a modern theory ofadaptive networks: expectation and prediction.
Psychol. Rev. 88, 135–170 (1981).
92. Montague, P. R., Dayan, P., Person, C. & Sejnowski,
T. J. Bee foraging in uncertain environments using
predictive Hebbian learning. Nature 377, 725–728(1995).
A computational treatment of behaviour thatcombines ideas from optimal control theory anddynamic programming with the neurobiology ofreward. This provided an early example of valuelearning in the brain.
93. Schultz, W. Predictive reward signal of dopamine
neurons. J. Neurophysiol. 80, 1–27 (1998).
94. Daw, N. D. & Doya, K. The computational
neurobiology of learning and reward. Curr. Opin.
Neurobiol. 16, 199–204 (2006).
95. Redgrave, P. & Gurney, K. The short-latency dopamine
signal: a role in discovering novel actions? Nature Rev.
Neurosci. 7, 967–975 (2006).
96. Berridge, K. C. The debate over dopamine’s role inreward: the case for incentive salience.
Psychopharmacology (Berl.) 191, 391–431 (2007).
97. Sella, G. & Hirsh, A. E. The application of statisticalphysics to evolutionary biology. Proc. Natl Acad. Sci.
USA 102, 9541–9546 (2005).
98. Rescorla, R. A. & Wagner, A. R. in Classical
Conditioning II: Current Research and Theory (edsBlack, A. H. & Prokasy, W. F.) 64–99 (Appleton
Century Crofts, New York, 1972).
99. Bellman, R. On the Theory of Dynamic Programming.
Proc. Natl Acad. Sci. USA 38, 716–719 (1952).
100. Watkins, C. J. C. H. & Dayan, P. Q-learning. Mach.
Learn. 8, 279–292 (1992).
101. Todorov, E. in Advances in Neural Information
Processing Systems (eds Scholkopf, B., Platt, J. &Hofmann T.) 19, 1369–1376 (MIT Press, 2006).
102. Camerer, C. F. Behavioural studies of strategic thinking
in games. Trends Cogn. Sci. 7, 225–231 (2003).
103. Smith, J. M. & Price, G. R. The logic of animal conflict.
Nature 246, 15–18 (1973).
104. Nash, J. Equilibrium points in n-person games.
Proc. Natl Acad. Sci. USA 36, 48–49 (1950).
105. Wolpert, D. M. & Miall, R. C. Forward models forphysiological motor control. Neural Netw. 9,
1265–1279 (1996).

106. Todorov, E. & Jordan, M. I. Smoothness maximization
along a predefined path accurately predicts the speed
profiles of complex arm movements. J. Neurophysiol.
80, 696–714 (1998).
107. Tseng, Y. W., Diedrichsen, J., Krakauer, J. W.,
Shadmehr, R. & Bastian, A. J. Sensory predictionerrors drive cerebellum-dependent adaptation ofreaching. J. Neurophysiol. 98, 54–62 (2007).
108. Bays, P. M. & Wolpert, D. M. Computational
principles of sensorimotor control that minimize
uncertainty and variability. J. Physiol. 578, 387–396(2007).
A nice overview of computational principles inmotor control. Its focus is on representing
uncertainty and optimal estimation when
extracting the sensory information required for
motor planning.
109. Shadmehr, R. & Krakauer, J. W. A computational
neuroanatomy for motor control. Exp. Brain Res. 185,
359–381 (2008).
110. Verschure, P. F., Voegtlin, T. & Douglas, R. J.
Environmentally mediated synergy between
perception and behaviour in mobile robots. Nature
425, 620–624 (2003).
111. Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stayor should I go? How the human brain manages thetrade-off between exploitation and exploration. Philos.
Trans. R. Soc. Lond. B Biol. Sci. 362, 933–942(2007).
112. Ishii, S., Yoshida, W. & Yoshimoto, J. Control of
exploitation-exploration meta-parameter inreinforcement learning. Neural Netw. 15, 665–687(2002).
113. Usher, M., Cohen, J. D., Servan-Schreiber, D.,
Rajkowski, J. & Aston-Jones, G. The role of locus
coeruleus in the regulation of cognitive performance.
Science 283, 549–554 (1999).
114. Voigt, C. A., Kauffman, S. & Wang, Z. G. Rational
evolutionary design: the theory of in vitro protein
evolution. Adv. Protein Chem. 55, 79–160 (2000).
115. Freeman, W. J. Characterization of state transitions inspatially distributed, chaotic, nonlinear, dynamicalsystems in cerebral cortex. Integr. Physiol. Behav. Sci.
29, 294–306 (1994).
116. Tsuda, I. Toward an interpretation of dynamic neuralactivity in terms of chaotic dynamical systems. Behav.
Brain Sci. 24, 793–810 (2001).
117. Jirsa, V. K., Friedrich, R., Haken, H. & Kelso, J. A.
A theoretical model of phase transitions in the humanbrain. Biol. Cybern. 71, 27–35 (1994).
This paper develops a theoretical model (based onsynergetics and nonlinear oscillator theory) that
reproduces observed dynamics and suggests a
formulation of biophysical coupling among brain
systems.
118. Breakspear, M. & Stam, C. J. Dynamics of aneural system with a multiscale architecture. Philos.
Trans. R. Soc. Lond. B Biol. Sci. 360, 1051–1074(2005).

12 | ADvANcE oNlINE puBlIcATIoN

119. Bressler, S. L. & Tognoli, E. Operational principles ofneurocognitive networks. Int. J. Psychophysiol. 60,
139–148 (2006).
120. Werner, G. Brain dynamics across levels oforganization. J. Physiol. Paris 101, 273–279 (2007).
121. Pasquale, V., Massobrio, P., Bologna, L. L.,
Chiappalone, M. & Martinoia, S. Self-organization andneuronal avalanches in networks of dissociated corticalneurons. Neuroscience 153, 1354–1369 (2008).
122. Kitzbichler, M. G., Smith, M. L., Christensen, S. R. &Bullmore, E. Broadband criticality of human brain
network synchronization. PLoS Comput. Biol. 5,e1000314 (2009).
123. Rabinovich, M., Huerta, R. & Laurent, G. Transient
dynamics for neural processing. Science 321 48–50(2008).
124. Tschacher, W. & Hake, H. Intentionality in nonequilibrium systems? The functional aspects of selforganised pattern formation. New Ideas Psychol. 25,
1–15 (2007).
125. Maturana, H. R. & Varela, F. De máquinas y seres
vivos (Editorial Universitaria, Santiago, 1972).
English translation available in Maturana, H. R. &Varela, F. in Autopoiesis and Cognition (Reidel,
Dordrecht, 1980).
126. Fiorillo, C. D., Tobler, P. N. & Schultz, W. Discrete
coding of reward probability and uncertainty by
dopamine neurons. Science 299, 1898–1902(2003).
127. Niv, Y., Duff, M. O. & Dayan, P. Dopamine,
uncertainty and TD learning. Behav. Brain Funct. 1, 6
(2005).
128. Fletcher, P. C. & Frith, C. D. Perceiving is believing: aBayesian approach to explaining the positive
symptoms of schizophrenia. Nature Rev. Neurosci. 10,
48–58 (2009).
129. Phillips, W. A. & Silverstein, S. M. Convergence of
biological and psychological perspectives on cognitive
coordination in schizophrenia. Behav. Brain Sci. 26,
65–82 (2003).
130. Friston, K. & Kiebel, S. Cortical circuits for perceptual
inference. Neural Netw. 22, 1093–1104 (2009).

Acknowledgments

This work was funded by the Wellcome Trust. I would like tothank my colleagues at the Wellcome Trust Centre for
Neuroimaging, the Institute of Cognitive Neuroscience and the
Gatsby Computational Neuroscience Unit for collaborations
and discussions.

Competing interests statement

The author declares no competing financial interests.

SUPPLEMENTARY INFORMATION
See online article: S1 (box) | S2 (box) | S3 (box) | S4 (box) |S5 (box)
All liNks Are AcTive iN The oNliNe pdf

www.nature.com/reviews/neuro© 2010 Macmillan Publishers Limited. All rights reserved