Decisions, Decisions: The Emerging Field of Neuroeconomics

Neuroeconomics is an emerging field that seeks to understand what computations are carried out by the brain to make different types of decisions, how these computations are implemented, and what the implications of this knowledge for human behavior are. The field is attempting to build a neurobiological theory of decision-making that is capable of providing a unifying model of human behavior for both the natural and the social sciences. In Why choose this book? How we Make Decisions, Read Montague provides a provocative and insightful introduction to some important ideas in the field.Montague begins with an exploration of the role of reinforcement learning in decision-making. In order to make good decisions, individuals need to learn which actions and states lead to desirable outcomes, and which ones lead to undesirable ones. A series of important research findings from the 1990s has begun to explain how the brain manages to do this. Wolfram Schultz and colleagues painstakingly characterized the response of the midbrain dopamine system to the reception and anticipation of rewards (see Schultz, 2007xSchultz, W. Annu. Rev. Neurosci. 2007; 30: 259–288Crossref | PubMed | Scopus (568)See all ReferencesSchultz, 2007 for an up-to-date review). They found that dopamine neurons fire when an unexpected reward is received, but they do not respond to fully anticipated rewards. Based on these results, Montague et al. (1995)xMontague, P.R., Dayan, P., and Sejnowski, T. J. Neurosci. 1995; 16: 1936–1947See all ReferencesMontague et al. (1995) and Schultz et al. (1997)xSchultz, W., Dayan, P., and Montague, P.R. Science. 1997; 275: 1593–1599Crossref | PubMed | Scopus (3436)See all ReferencesSchultz et al. (1997) proposed that a simple reinforcement learning model (Sutton and Barto, 1988xReinforcement Learning. Sutton, R.S. and Barto, A.G. See all ReferencesSutton and Barto, 1988) could account for many of the qualitative properties of the dopamine system and provide an explanation for its role in decision-making. In this view, the dopamine system encodes a “prediction error signal” that the brain uses to learn the value of stimuli and actions. The prediction error is activated whenever an unexpected reward is delivered and signals that the value of the states and actions that led to that reward should be increased, which increases the probability that they will be sought or chosen in the future. These findings have recently received additional support from human fMRI studies (O'Doherty, 2003xO'Doherty, J. Curr. Opin. Neurobiol. 2003; 14: 769–776Crossref | Scopus (686)See all ReferencesO'Doherty, 2003).Most of the decisions used to establish these properties of the dopamine system involve small choices repeated at high frequency (e.g., choosing between stimuli associated with the probability of obtaining a small amount of juice or a small monetary payoff). An open and important question is this: to what extent does dopamine-mediated reinforcement learning play a role in guiding other forms of decisions? In the book, Montague proposes a model of how “cognitive mechanisms” could modulate the dopamine system to generate behaviors that cannot be learned under standard reinforcement learning rules, such as the choice to risk life and limb for a good cause, or the decision to incur short-term losses to attain long-term rewards. It will be interesting to see how well this theoretical framework holds empirically in the upcoming years.View Large Image | View Hi-Res Image | Download PowerPoint SlideThe book also captures the excitement of this burgeoning field by describing several interesting and provocative studies that have been carried out in the Montague lab. For example, one study explored decision-making under conditions of risk and uncertainty by simulating the experience of trading in the stock market (Lohrenz et al., 2007xLohrenz, T., McCabe, K., Camerer, C.F., and Montague, P.R. Proc. Natl. Acad. Sci. USA. 2007; 104: 9493–9498Crossref | PubMed | Scopus (145)See all ReferencesLohrenz et al., 2007). The experiment showed that the brain's reward learning capabilities are quite sophisticated: it learns from experiencing the consequences of the actions that it takes, but also, and perhaps ultimately more important, by evaluating the outcomes of actions that are not taken. Another provocative study (McClure, 2004xMcClure, S.M. Neuron. 2004; 44: 379–387Abstract | Full Text | Full Text PDF | PubMed | Scopus (467)See all ReferencesMcClure, 2004) challenges the hypothesis that the reward experienced after consuming an item, say a soda, depends only on its physical properties and on the internal state of the individual. A clever experiment that brings the old Coke-Pepsi challenge to the world of fMRI showed that this hypothesis is incorrect: the reward circuitry of a Coke drinker responds more strongly to the same liquid when the subject is told that he is drinking his favorite brand. These findings have clear and important implications for neuroscience, economics, marketing, and public policy.The book is filled with interesting conjectures and insights about computational neuroscience and future applications of the field of neuroeconomics. Two of them are worth highlighting. First, Montague speculates that valuation mechanisms similar to those that influence decisions are responsible for organizing the brain's structure, activity, and computations. This is an important idea that deserves further study. Second, Montague contends that the introduction of computational modeling and neuroeconmic experimental paradigms into the field of psychiatry could lead to an improved understanding of the mechanisms that are altered in these disorders. Consider the case of addiction, which might be the quintessential decision-making disease. An understanding of how exposure to addictive substances impairs the computations of the decision-making circuitry might give us important clues into the disease's pathology, diagnosis, prevention, and treatment.There is much to like in this book, both for the practicing neuroeconomist and for those new to the field. However, the book also has some important limitations. The main problem is that it is aimed at too general an audience, which leads to an oversimplification of the exposition—the book has no figures, tables, or equations. Given the subtlety of many of the issues being discussed, I suspect that most neuroscientists, psychologists, and economists will be left wanting more details about the models, as well as the data and figures from the experiments. For the half of the book that describes existing studies, there is an easy solution: enthusiastic readers can complement the text with the extensive endnotes and the original publications. Unfortunately, such recourse is not available for the other half of the book, which contains some of the text's most novel and provocative ideas. These limitations notwithstanding, this book is an interesting and thought-provoking introduction to the field of neuroeconomics and to the views of one of its foremost exponents.