Abstract: Recent articles by  Busza et al. (BJSW)   and Dar et al. (DDH)  argue that astrophysical data can be used to establish small bounds on the risk of a "killer strangelet" catastrophe scenario in the RHIC and ALICE collider experiments. The case for the safety of the experiments set out by BJSW does not rely solely on these bounds, but on theoretical arguments, which BJSW find sufficiently compelling to firmly exclude any possibility of catastrophe. Nonetheless, DDH and other commentators (initially including BJSW) suggested that these empirical bounds alone do give sufficient reassurance. This seems unsupportable when the bounds are expressed in terms of expectation value-a good measure, according to standard risk analysis arguments. For example, DDH's main bound, p catastrophe < 2 × 10 −8 , implies only that the expectation value of the number of deaths is bounded by 120; BJSW's most conservative bound implies the expectation value of the number of deaths is bounded by 60,000. This article reappraises the DDH and BJSW risk bounds by comparing risk policy in other areas. For example, it is noted that, even if highly risk-tolerant assumptions are made and no value is placed on the lives of future generations, a catastrophe risk no higher than ≈10 −15 per year would be required for consistency with established policy for radiation hazard risk minimization. Allowing for risk aversion and for future lives, a respectable case can be made for requiring a bound many orders of magnitude smaller. In summary, the costs of small risks of catastrophe have been significantly underestimated by BJSW (initially), by DDH, and by other commentators. Future policy on catastrophe risks would be more rational, and more deserving of public trust, if acceptable risk bounds were generally agreed upon ahead of time and if serious research on whether those bounds could indeed be guaranteed was carried out well in advance of any hypothetically risky experiment, with the relevant debates involving experts with no stake in the experiments under consideration.

INTRODUCTION Speculative suggestions are occasionally made about ways in which new physics experiments could hypothetically bring about a catastrophe leading to the end of life on Earth. Some of these hypothetical catastrophes, including the "killer strangelet" sce-nario considered in this article, would also lead to the destruction of the planet and wider catastrophic consequences. In any case, the proposed catastrophe mechanisms generally rely on speculation about hypothetical phenomena for which there is no evidence, but which at first sight do not contradict the known laws of physics. Sometimes, such pessimistic hypotheses can be countered by arguments that show that the existence of the catastrophe mechanism is highly improbable, either because closer analysis shows Kent that the proposed mechanism does in fact contradict well-established physical principles, or because its existence would imply effects that we should almost certainly have observed but have not. Unfortunately, there is a difficulty in making an argument of this type sufficiently reassuring. One would like to be reassured that the chances of inadvertently triggering a global catastrophe are very small indeed before going ahead with an experiment. But finding arguments that justify this conclusion with the appropriate level of confidence may be very hard, if not impossible. Discouragingly few attempts to grapple with this issue have been made. In fact, even the obvious and fundamental question-how improbable does a catastrophe have to be to justify proceeding with an experiment?-seems never to have been seriously examined. The aim of this article is to face this question squarely, in the hope of stimulating further debate. The particular stimulus for this article was the debate over the safety of the RHIC supercollider experiments now underway at Brookhaven, and the ALICE experiments proposed by CERN. Speculation about possible disaster scenarios in these experiments led to some pressure for the experiments to be deferred or cancelled. In response, reports and articles were written that were used to justify commencing the RHIC experiments on the grounds that, inter alia, "Cosmic ray collisions provide ample reassurance that we are safe from a . . . catastrophe at RHIC"  (1)  and "Beyond reasonable doubt, heavy-ion experiments at RHIC will not endanger our planet."  (2)  Here I will contend that the risk bounds obtained are actually not small, taking into account the scale of the catastrophe, either according to standard risk analysis or when compared with other adopted standards for acceptable risk to the public. Since the criteria used by Brookhaven to justify proceeding were developed by theoretical physicists and administrators, not by broader representatives of the public or by professionals in risk management, it seems desirable to bring these issues before a wider audience for the purposes of informed discussion and the formation of sounder public policy in future.
 HISTORICAL EXAMPLES The first catastrophe mechanism seriously considered seems to have been the possibility, raised in the 1940s at Los Alamos before the first atomic bomb tests, that fission or fusion bombs might ignite the atmosphere or oceans in an unstoppable chain reaction. Investigation led to an analysis by Konopinski et al.  (3)  that fairly definitively refuted the possibility. Compton was later reported, in a published interview  (4)  with Pearl Buck, as saying that he had decided not to proceed with the bomb tests if it were proved that the chances of global catastrophe were greater than three in a million, but that in the event calculation proved the figures slightly less. It is hard to understand how any meaningful calculation could have produced such a risk figure. The analysis of Reference 3 gives convincing arguments against the possibility of a catastrophic chain reaction, based on well-established physical principles. It concludes that it is unreasonable to expect a chain reaction propagated by nitrogen-nitrogen fusion reactions, and that an unlimited chain reaction consuming the atmosphere is less likely still. Other possible reactions, involving protons in clouds of steam liberated from the oceans, are also considered and argued to be less dangerous still. Konopinski et al. do note the "distant probability" that the mode of propagation of the reaction in the atmosphere might be more complicated than their analysis allows, in which case its conclusions might not apply, and they suggest that the complexity of their argument and the absence of a satisfactory experimental basis for it makes further work on the subject highly desirable. However, they offer nothing resembling a catastrophe risk estimate, nor any results from which a quantitative estimate could be derived. Yet, so far as I know, Compton never made an attempt to correct Buck's account. Had she simply misunderstood, it would have been easy for Compton to disclaim the statement. And, had it not reflected his views, he would surely have wanted both to set the historical record straight and to defend his reputation against the charge of unwisely gambling with the future of humanity. The natural inference seems to be that Compton did indeed make the statement reported.  1  If so, although the risk figure itself appears unjustifiable, Compton presumably genuinely believed that an actual risk (not just a risk bound)  1  In April 2000, in an attempt to understand this puzzling statement of Compton's, I contacted Hans Bethe, a key figure in both the Los Alamos project and the theoretical work that led to the conclusion that the possibility of an atomic bomb explosion leading to global catastrophe was negligible. His view, relayed by an intermediary (Kurt Gottfried), was that the analysis of Konopinski et al. was definitive and does not allow one to make any meaningful statement about probabilities since the conditions that must be met cannot be reached in any plausible manner. Bethe suggested that the 3 × 10 −6 figure was made up by Compton "off the top of his head," and is "far, far too large." I am most grateful to Hans Bethe and Kurt Gottfried for this e-mail correspondence. of 3 × 10 −6 of global catastrophe was roughly at the borderline of acceptability, in the cause of the American and allied effort to develop atomic weapons during World War II. Apparently the figure did not worry 1959 American Weekly readers greatly, since no controversy ensued. It would be interesting to compare current opinion on the acceptability of a 3 × 10 −6 risk of global catastrophe, in the circumstances of the Los Alamos project or otherwise. Another hypothetical catastrophe was examined some time ago by Hut and Rees.  (5, 6)  They considered the possibility that the vacuum state we live in is not the true vacuum, but merely a local minimum of the effective potential. They asked whether, if this were the case, new generations of collider experiments could trigger a catastrophic transition to the true vacuum, destroying not only the Earth but (eventually) all presently stable forms of matter in the cosmos. They showed that the probability of this occurring artificially in present or foreseeable collider experiments is considerably smaller than the probability of it having occurred naturally within our past light-cone.  2  Most recently, in response to some (rather unfocused) public concern,  (7)  the possibility of some catastrophe arising from prospective experiments at the Brookhaven relativistic heavy ion collider (RHIC) was reviewed by Busza et al. (BJSW) and Dar et al. (DDH).  3  Both groups paid most attention to the "killer strangelet" catastrophe scenario, which would arise if negatively charged metastable strange matter existed and could be produced in the experiments. As well as giving theoretical arguments against the hypotheses involved, both groups offer risk bounds inferred from empirical evidence. BJSW's proposed bounds on the probability of catastrophe during the 10-year lifetime of RHIC, derived from the survival of the Moon for 4.5 billion years, range from 10 −5 to 2 × 10 −11 , depending on how conservative the assumptions made are.  (8)  DDH's bounds, derived from the observed rate of supernovae, range from 2 × 10 −6 (their pessimistic bound for a very slow catastrophe, in which the Earth is prematurely destroyed at some point in the billion years before it would anyway be consumed by the expanding Sun) to 2 × 10 −8 (their main bound).  (2)  As the quotations extracted in Section 4 attest, both groups originally  (1, 2)  suggested their empirical bounds alone were adequate to show that the experiments were safe. If correct, this conclusion would obviously be particularly welcome, since it would remove any need to evaluate the degree of confidence that should be placed in the theoretical arguments. The view that the empirical bounds were indeed adequate was also expressed in commentaries.  (9, 10)  However, there are good reasons, explained below, to believe that this conclusion is incorrect, and indeed the claim was withdrawn by BJSW, after criticisms from the author of this article. BJSW produced a second version of their preprint, removing the reassuring characterizations of their risk bounds and instead disavowing any attempt to decide what is an acceptable upper bound on p catastrophe . In this revised version, BJSW also accept that the arguments for their empirically derived risk bounds could be invalidated if some additional pessimistic hypotheses were correct.  4  BJSW's revision of their preprint was an adequate response, from a purely scientific perspective. The public policy implications are troubling, however. My understanding is that the U.S. government's authorization for the RHIC experiments to proceed was given partly on the basis of BJSW's original arguments,  (1)  whose discussions of risk were gravely flawed, as the quotations considered in Section 4 illustrate. Public concern was countered by widely publicized  (11, 12)  reassurances  (9, 10)  that the risk was negligible, also relying heavily on the risk appraisals given in BJSW's original preprint.  (1)  As far as I am aware, no effort was made by Brookhaven to reobtain authorization on the basis of BJSW's revised assessment, or to bring what is a significantly revised case to media and public attention.  5  In my opinion, such efforts should have been made. It should be noted that BJSW stress  (8)  in the revised version of their article that they regard the theoretical arguments alone as sufficiently compelling. This may be a defensible position, but it is not the case that was originally made and publicized.
 SCOPE OF THIS ARTICLE This article is meant as a contribution to the debates over hypothetical catastrophe scenarios in the RHIC and ALICE collider experiments and over other hypothetical or real global catastrophe risks. It focuses on the key question: What risk of catastrophe could be acceptable? Other relevant questions are not considered. In particular, in the case of the collider experiments, no attempt is made to infer quantitative risk bounds from the qualitative theoretical arguments against the possibility of catastrophe, or to consider BJSW's conclusion that the theoretical arguments alone offer sufficiently compelling reassurance.  (1, 8)  The interest in this debate is not, of course, purely or mainly intellectual. The aim is to improve future policy over catastrophe risks. In particular, lessons can and should be learned from the evident flaws in BJSW's and DDH's discussions of risk. It is obviously unsatisfactory that the question of what constitutes an acceptable catastrophe risk should continue to be decided, in an ad hoc way, according to the personal risk criteria of scientists whom those in charge of experimental facilities choose to consult. Those criteria, however sincerely held and thoughtfully constructed, may be unrepresentative of general opinion or of expert opinion in risk analysis. Worse still, history suggests the risk criteria actually used may not be at all thoughtfully constructed. Compton's reported opinion suggests, and the mischaracterizations of References 1 and 2 illustrate very clearly, that scientists whose expertise is not in risk analysis or public policy cannot necessarily be relied on either to interpret the risk implications of the science correctly or to consider elementary arguments that tend to suggest more cautious risk criteria than can easily be satisfied. Relying on such inexpert appraisals is neither in the public interest nor the longterm interests of science. Scientists and scientific institutions need to work to gain, maintain, and justify public trust. Arguments that suggest that an experiment should proceed simply because the global catastrophe risk appears fairly low, without comparison to any preexisting thresholds or guidelines, may not only fail to reassure, but may (not unreasonably) be interpreted as public relations exercises, intended to support a prejudged conclusion, rather than dispassionate scientific analyses. As Calogero notes,  (14)  this has a long-term cost for the credibility on questions of risk not only of those directly involved, but of all scientists, and the likely long-term consequence is less in-formed and more irrational public debate and public policy. It may well not be possible to reach a complete consensus on firm guidelines. It seems unlikely, for instance, that some clear agreement will emerge that global catastrophe risks are small enough to be of negligible concern if and only if lower than, say, 10 −20 . Life is more complicated than that, and democratic debate more multifaceted. Nonetheless-indeed, for this very reason-it would be valuable to have a spectrum of carefully argued opinion in the literature. I hope that the arguments below may spark further discussion.
 BJSW'S AND DDH'S RISK BOUNDS FOR THE "KILLER STRANGELET" SCENARIO We turn now to the particulars of the catastrophe risk concerns raised over the RHIC and ALICE experiments, and specifically to the hypothetical "killer strangelet" catastrophe scenario analyzed in some detail by BJSW and DDH. The "killer strangelet" scenario requires: (i) that stable strange matter exists, (ii) that a valley of stability exists for negatively charged strangelets, (iii) that negatively charged metastable strangelets could be produced in the ≈40 TeV Au-Au ion collisions planned at RHIC, (iv) that a strangelet so produced could survive collisions that bring it toward rest in surrounding matter, and (v) that it would then fuse with nuclei, producing larger negatively charged strangelets, in a runaway reaction that eventually consumes the Earth. The theoretical arguments  (2, 8)  against (i)-(iii) are generally regarded as convincing. If (i)-(iii) were nonetheless true, (iv) and (v) would also be plausible. If (i)-(v) were true, killer strangelets should also be produced in naturally occurring high energy heavy ion collisions, which take place when cosmic rays collide with one another or with heavy nuclei in celestial bodies. Naturally produced killer strangelets would be able to initiate runaway reactions capable of destroying asteroids, satellites such as the Moon, or stars. From the fact that the Moon has survived for 4.5 billion years, and from the fact that astronomical observations are not consistent with stars being converted into strange matter at any significant rate, bounds on the risk of catastrophe at RHIC can be derived.  (2, 8)  Unfortunately, these derivations require assumptions about the types of interaction that produce strangelets, the velocity distribution of the strangelets produced, their interactions with nuclei, and their stability.  (2, 8, 13)  For this reason, BJSW and DDH give various bounds, derived by making more or less conservative assumptions. Even the weakest of these requires some assumptions.  (8, 13)  Without knowing what level of confidence we can have in the relevant assumptions-a question that neither group addresses quantitatively-it is difficult to see how the bound figures can really be meaningful.  (13)  But even if the figures cannot really be justified, the comments made on them by BJSW, DDH, and others give an interesting and valuable insight into the risk criteria of physicists and administrators involved in RHIC policy. Assuming that RHIC runs for the scheduled 10 years, DDH obtain p catastrophe < 2 × 10 −8 for a fast catastrophic destruction of the Earth and p < 2 × 10 −6 for a slow destruction that would be completed in the billion years before the Sun expands beyond the Earth orbit. DDH describe these results as "a safe and stringent upper bound on the risk incurred in running  [RHIC] ." They add that the two bounds respectively imply that "it is safe to run RHIC for 500 million years" and that "running the RHIC experiments for five million years is . . . safe." These last two statements are, of course, incorrect. DDH's bounds, if valid, would establish only that it would be unlikely that the Earth would be destroyed very early in a RHIC experiment run over the relevant periods: the bounds are consistent with a high probability of destroying the Earth at some point during these hypothetically extended experiments.  6  In the first version  (1)  of their article, BJSW described DDH's result as "a factor of 10 8 below the value required for the safety of RHIC." This, of course, is also incorrect: a risk bound 10 8 times that of DDH's would be consistent with a high probability of destroying the Earth within five years of the RHIC experiment-a risk level that even the most gung-ho physicist could hardly describe as "safety." Using their own independent analyses, BJSW derive the following bounds from the survival of the Moon, given various assumptions (their Cases I-III) about strangelet production, and again assuming that RHIC runs for the scheduled 10 years: p catastrophe < 2 × 10 −10 , p catastrophe < 2 × 10 −5 , p catastrophe < 2 × 10 −4 . They described the second and third of these cases as still leaving "a comfortable margin of error." These comments, and that quoted at the start of this paragraph, are so obviously inapplicable-no sane person would seek to reassure the public by suggesting that a risk bound of one in 5,000 of destroying the Earth represented a comfortable margin of error-that I suspect they must reflect some surprising confusion on BJSW's part at the time of writing Reference 1. BJSW refined their calculations in the second version of their article, extracting an extra factor of 10 and producing bounds for a 10-year run of the RHIC experiment of (Cases I-III): p catastrophe < 2 × 10 −11 , p catastrophe < 2 × 10 −6 , p catastrophe < 2 × 10 −5 . In this revised version, which followed criticisms of the comments noted above, no judgment is made as to whether any of the bounds are satisfactory. To quote BJSW: "We do not attempt to decide what is an acceptable upper limit on [ p catastrophe ], nor do we attempt a 'risk analysis', weighing the probability of an adverse event against the severity of its consequences."  (8)  We use the revised bounds in the following discussion, referring to them simply as BJSW's bounds. DDH's main bound-p catastrophe < 2 × 10 −8 over the 10-year life of RHIC-has been widely referred to  (1, 2, 9, 10)  in terms that suggest that it alone would be sufficiently reassuring to require no further analysis or risk optimization. My impression is that many numerate and thoughtful people would disagree. My own reasons for doing so are given below.
 RISK BOUNDS AND RISK ESTIMATES: AN IMPORTANT CAVEAT It is important to stress that DDH's and BJSW's empirical arguments produce bounds on the risk of catastrophe, not estimates of that risk.  7  Their bounds are based on the fact that we do not observe something that we should expect to observe if the risk were larger than some value p. A negative result of this form tells us nothing about the actual value of the risk. Everything in DDH's and BJSW's analyses is consistent with the true risk of catastrophe being zero-and if current theoretical understanding is correct, the risk is indeed precisely zero. When the destruction of the Earth is in question, though, it would be preferable not to have to rely on theoretical expectations alone. As Glashow and Wilson put it,  (10)  "The word 'unlikely', however many times it is repeated, just isn't enough to assuage our fears of this total disaster." Hence the interest in looking at naturally occurring versions of the experiment, verifying that they have not resulted in catastrophes, and so producing firm bounds on the risk of catastrophe. Unfortunately, this approach has its pitfalls and limitations. Comparing the effects expected from hypothetical killer strangelets produced in naturally occurring heavy ion collisions and at RHIC is not completely straightforward. Theoretical assumptions need to be made in order to derive risk bounds. Unless we are very confident indeed in those assumptions, we cannot validly infer very small risk bounds this way.  (8, 13)  And in any case, nature may not necessarily have done versions of the experiment we are interested in often enough to produce sufficiently strong risk bounds. How do we begin to decide what constitutes a sufficiently strong risk bound? It seems to me that the correct approach in appraising risk bounds is to make worst-case assumptions. So, if we are assured that p catastrophe ≤ p 0 , and we have to decide whether that bound alone is sufficient reassurance, we have to ask whether we would be happy to proceed if we knew that p catastrophe = p 0 . If not, then the bound alone is not sufficiently reassuring. Such a bound might still form part of a compelling case for the safety of an experiment if it could be combined with other arguments. For instance, in the case of RHIC, it might be argued that a combination of the theoretical arguments and empirical bounds is sufficiently reassuring, even if neither would be alone.  8  I will not consider such arguments here. Nor-to reiterate-do I examine whether the theoretical arguments alone are sufficiently reassuring. The discussion below considers only the narrow question of whether the empirical bounds alone would suffice.
 RISK VERSUS EXPECTATION The destruction of the Earth would entail the death of the ≈ 6 × 10 9 human population and of all other species, the loss of the historical record of the evolution of its biosphere, and the loss of almost all record of the culture developed by humanity.  9  Added to these are the opportunity costs arising from the absence of future generations. Consider for the moment just the number of human deaths. If an experiment were expected to cause one human death, in the everyday use of the termthat is, it was likely that at least one person would die as a result of the experiment-its health and safety implications could not be said to be negligible. Now, when we are dealing with small risks of large catastrophes, we cannot directly use this measure: an experiment with small risk is expected to cause no human deaths, in the sense that this is the likeliest outcome. However, we can calculate a related measure: the statistical expectation value of the number of human deaths. The expectation value of the number of human deaths ensuing from an Earth-destroying catastrophe is E d = p catastrophe N, where N ≈ 6 × 10 9 is the current human population. So, if we accept E d as an appropriate measure of the seriousness of a risk-and the next section explains why we should-then any risk that does not satisfy p catastrophe 1.6 × 10 −10 (1) is not negligible. Of the above bounds, neither of DDH's bounds ensures that Equation (  1 ) is satisfied, nor do the second and third of BJSW's. BJSW's least conservative lunar survival bound (Case I) comes closer, but still fails unless a factor of 1/8-i.e., in this case a probability of 1/8 of causing one human death-is regarded as negligible. Making the comparison in terms of expectations, DDH's main (tighter) bound implies that the expectation value of the number of human fatalities caused by RHIC in more than 10 years will not exceed 120. Put this way, this bound seems far from adequately reassuring.
 IS THE EXPECTATION VALUE OF THE NUMBER OF FATALITIES RELEVANT? It might perhaps be argued that the preceding calculation is misleading. After all, DDH's bounds on p catastrophe represent probabilities small enough to be negligible in most circumstances. Most of us take 2 × 10 −8 risks of death in our stride: the risk of a typical U.S. citizen dying in a shark attack in any given  9  A few spacecraft, including the message-bearing Pioneer and Voyager craft, would survive, as would the-continually attenuating-electromagnetic signals generated on Earth. year is comparable. Translating the bound value into E d , the expected number of fatalities, makes it seem significant. But is it really reasonable to use expected fatalities as a measure of the safety of a risk bound? Actually, the next section argues that considering E d alone still greatly underestimates the cost. But first let us consider whether requiring E d < 1 gives a sensible upper bound on negligible risk, assuming that it is agreed that the certainty of causing one death would not be negligible. I believe most expert opinion would agree that it does, for the following reasons. First, everyone agrees that in carrying out any risk analysis we need the cost or utility of the various outcomes, not merely their probability: a 10 −3 chance of losing one dollar is better than a 10 −3 chance of losing one million dollars, and so on. Second, a fundamental principle of risk analysis is that in normal circumstances rational people are risk averse. If X represents a random process whose possible outcomes x i have probabilities p i , and if V(x i ) represents the value to the community of outcome x i , then the value V(X) of a single run of X-that is, the value of allowing precisely one of the x i to happen, with respective chances p i -is generally assumed to obey V(X) ≤ i p i V(x i ). ( The values of undesirable outcomes, of course, are negative: we refer to −V(x i ) as the cost of x i . A second principle is that the utility or cost function is concave. Applied to the cost of a loss of human lives, this implies that the cost to society of N deaths is at least N times as great as the cost of one death: if x N and x 1 represent the two events, then V(x N ) ≤ NV(x 1 ). (3) The principles of risk aversion and concave utility explain, for example, why it can often be rational to take out insurance, even though on average the insurance company expects to make a profit and the customer a loss. Similarly, it explains why investors almost universally require investments that involve higher risk to offer a higher expected profit in compensation. By considering a random process X with probability 2 × 10 −8 of killing 6 × 10 9 people and probability (1 − 2 × 10 −8 ) of killing no one, we see that the principles together imply that a 2 × 10 −8 chance of killing 6 × 10 9 people is at least as bad as the certainty of killing 120 people. In summary, to demonstrate that the risk is negligible, we would need to show that E d is considerably smaller than 1-precisely how much smaller depend-ing on precisely how risk averse one is when it comes to global catastrophe. Neither DDH's nor BJSW's bounds satisfy this criterion. To speak of the bounds being "safe and stringent" or guaranteeing "comfortable margins of error" is, on this analysis, simply incorrect. Similarly, to demonstrate that the risk is acceptable, it would be necessary (though not necessarily sufficient) to show that E d is small enough that the certainty of the experiment killing E d people would be acceptable. Put another way, if it would be unacceptable for the experiment to lead to the certain loss of E d lives, then a risk at the bound value would be unacceptable. Suppose, counterfactually, that we knew that the RHIC experiment were certain to kill precisely N people (and no more). What value of N would be acceptable? Answers will vary, but my guess is that most would be somewhere in the range <10 or so. In particular, I think it clear that RHIC would not have obtained political authorization if it was thought certain to kill precisely 120 people: that would be regarded as an unacceptably high cost. From the discussion of this section, it follows that a global catastrophe risk at the DDH bound value would be at least as unacceptable. Although the observations made in this section are elementary, it is worth noting that the CERN panel did not acknowledge their validity. The response of Alvaro de Rujula, the panel leader, is accurately summarized by his opinion, quoted in New Scientist,  (15)  that it is "absurd" to take the risk bound probability and multiply it by the global population. I recommend contemplation of this comment to anyone inclined to automatic faith in the risk analysis expertise of scientists chosen by institutions to argue for the safety of their experiments.
 COMPARISON WITH EXISTING RISK POLICIES DDH's and BJSW's risk bounds were presented and discussed by DDH  (2)  and BJSW  (1)  in a statement by John Marburger, then director of Brookhaven,  (9)  and in a commentary by Glashow and Wilson.  (10)  None of these discussions make comparisons with risk criteria or optimization procedures applied to other potentially hazardous activities. This is unfortunate, since risk comparisons are generally illuminating, and in this case suggest that regarding the risk bounds per se as adequate would be wildly inconsistent with at least some established policy. For example, the U.K. National Radiological Protection Board requires that the risk of serious deleterious health effects arising from a nuclear solid waste disposal facility must always be bounded at below 10 −5 per year, that risk optimization procedures should be continued until the risk is below 10 −6 per year, and that the risk of low probability natural events that could lead to serious deterministic health effects should be separately bounded at 10 −6 per year.  (16)  The risk figures apply to the critical group of individuals, typically numbering between a few and a few hundred, whose habits or location render them most at risk. Quite typically, the events whose risk is bounded would be expected to kill fewer than 10 people. In summary, according to established policy for these radiation hazards, it is not acceptable to incur a risk of greater than 10 −6 per year of killing ≈5 people. The risk aversion arguments of the last section suggest that a consistent policy on catastrophe risk should treat a risk greater than 10 −15 per year of killing the global population as even less acceptable. An acceptable risk bound should thus imply p catastrophe 10 −15 per year. (  4 )
 FUTURE LIVES The discussion so far has considered only the expected cost due to immediate human fatalities, neglecting the other costs mentioned earlier. These are very hard to quantify in any commensurable way. (What is the value of the rest of the biosphere compared to that of the human population? What price do we put on the historical record?) However, it is at least arguably possible to assign a sensible and commensurable value to one of these further costs-the loss of future generations-by estimating the number of future human lives that would not take place if the planet were destroyed in the near future. This line of argument cannot be followed without addressing two rather complex questions: Should we value our successors' lives as highly as those of our contemporaries? And can we say anything meaningful about the likely fate of humanity over the billion years of life the Earth has left (or beyond)? To the first, my own answer is "yes," partly because I cannot see any good reason to prefer an unknown contemporary to an unknown successor, and partly because it seems to me our lives have value in the first place largely because they form part of ongoing human history. This view finds some support in established policy: the U.K. National Radiological Protection Board guidelines cited above also explic-itly state that those living at any time in the future should be given a level of protection at least equivalent to that given to those alive now. As for futurology, there are obviously so many unknowns that attempting detailed analysis seems pointless. I offer only a crude calculation, which is obviously open to criticism, but at least suggests a starting point for discussion. Suppose, optimistically, that humanity has a reasonable chance of surviving (in some form) for the lifetime of the Earth. Suppose also that there is a reasonable chance of arranging things so that the sum global quality of life is at least at the level of today, and the global population is roughly today's: 10 10 in round figures. And suppose we neglect the possibility that the human lifespan may increase beyond 10 2 , on the grounds that it is arguably irrelevant: arguably, one can make a reasonable approximationreasonable, that is, given the uncertainties in the entire discussion-by considering the total number of person-years, so that for instance a 700-year life is equated to seven 100-year lives. Let us also, conservatively, neglect the effect of migration to other planets some time in the future, which would presumably (i) allow the human population to vastly increase over the next billion years and (ii) permit humanity to survive beyond a billion years. The cost of the destruction of the Earth today would then be roughly 10 10 10 −2 10 9 = 10 17 lives, or 10 7 greater than the cost earlier calculated. Including this factor in the earlier calculation derived from the NRPB's risk bounds would mean that an acceptable risk bound should imply p catastrophe 10 −22 per year. ( This further proposed tightening of risk bounds is controversial in all sorts of ways. To mention just one: regarding the loss of future lives as a separate cost raises the question-a cost to whom? To those potential future generations, deprived of the possibility of existence? To us, deprived of descendants and successors? Both, I think-but I concede that both lines of thought raise some difficulties. Another way of approaching the question is to ask a different hypothetical question: Would it be far worse if all of us (or all life on Earth) were killed than if almost all of us (or almost all life on Earth) were, supposing that in the second case the planet remained otherwise fit for life? Answering "yes" suggests placing a high relative value on future lives, since the number of immediate fatalities is almost the same in both cases. Those who answer "no" will presumably not find any of the arguments of this section convincing. My impression is that the question has not been widely enough debated for it to be possible to say which view (if either) reflects general opinion. For the moment, then, whichever view one holds on future lives, it is worth bearing in mind that the majority view, on which catastrophe risk policy should properly be based, may turn out to differ.
 SOME COUNTERARGUMENTS Calculations and comparisons are indispensable in rationalizing risk policy and in highlighting inconsistencies. However, there is no generally agreed set of principles from which we can decide policy in every instance. Politics do not form a subset of mathematics. The above arguments could well be opposed on many different grounds. I consider here some counterarguments that have been suggested to me in discussions. 1. One obvious criticism is that the arguments above consider the cost of a catastrophe risk but not the benefits gained by taking the risk. For that reason, it may be argued, they are bound to produce overcautious prescriptions. After all, no risk at all is worth taking unless there is some benefit. Without a costbenefit analysis, no sensible conclusion can be reached. This is a partially fair criticism, but only partially. It should be stressed that it does not apply to the argument of Section 6, since using the number of deaths as a measure of safety is justified by comparing the implicit cost-benefit tradeoffs in conclusions that would be generally agreed. It seems to me pretty uncontroversial that, despite the benefits of RHIC, the experiment would not be allowed to proceed if it were certain (say, because of some radiation hazard) to cause precisely 120 deaths among the population at large. If that is accepted, it follows that a risk at the DDH bound value would be unacceptably high, even when the benefits of RHIC were taken into account. That said, some forms of cost-benefit analysis might indeed suggest that requiring p catastrophe 10 −15 per year or lower may be overstringent. The likely immediate benefit of the RHIC experiments-advancing our understanding of basic science-is not negligible. Moreover, the possible benefits presumably include at least some probability-perhaps small, but not necessarily small compared to 10 −15 -of contributing in some presently unforeseen way to a discovery with a very large beneficial impact on future human lives. The foresight problem is particularly acute here, of course, since one can also imagine low probability outcomes, other than the catastrophe scenario, with a large negative impact. But, if one takes the view that scientific and technological progress have on balance been beneficial and are likely to continue to be, the small possibility of a benefit that would save (or enable) many future human lives gives, at least in principle, something to offset against the small possibility of a catastrophe. It is important to be clear, though, that by definition no cost-benefit argument could justify a claim that the risks involved are negligible. Rather, it would make the case for proceeding with RHIC by suggesting that the risks, though possibly not negligible, were justified by the benefits. This is not the case that has been made. Such a case might or might not be widely accepted. 2. It may be argued that the more stringent risk criteria suggested above for global catastrophe, even if rationally justifiable in theory, are impossibly utopian. If we took them seriously, and attempted to ensure that they were satisfied before proceeding with any enterprise, we would stop not only collider experiments, but many other human activities. Progress would become impossible; life might be made unliveable. Maybe-but I would be cautious about accepting this sort of defeatism too readily. We begin from a state where risk bounds of 10 −6 are used quite widely, for instance in the solid nuclear waste disposal guidelines cited above. It does not seem obvious to me that, with careful attention to the problems, we could not ensure that catastrophe risks associated to specific mechanisms are many further orders of magnitude smaller. On the contrary, it seems quite clear that in some cases catastrophe risk bounds could be substantially reduced. The RHIC experiments are an excellent example: had the problem of reducing the risk bounds been taken seriously, further theoretical research, perhaps combined with a tentative experimental program aimed at carefully testing our understanding of the new physics involved before running the full experiment, could almost certainly have reduced the bounds very significantly. Of course, this is not to say that risk avoidance is cost free. One has to accept that more stringent catastrophe risk criteria might indeed delay or preclude at least some interesting future experiments. It seems to me we just have to accept this as a fact of life. One cannot defensibly adopt a mindset that requires that every interesting experiment must be carried out, and that sees every risk analysis as an exercise in justifying this foregone conclusion. Human life, collectively as well as individually, is, after all, fragile. Our understanding of nature is limited, and there are surely many dangers we have not yet appreciated. Due caution is appropriate. We should not, in any case, rely on speculation about the implications of a more cautious catastrophe risk policy. If it were to become clear that it would be effectively impossible to apply a policy on catastrophe risks consistently, obviously that policy would need to be reconsidered. Unless and until carefully justified arguments are made, identifying specific examples of problematic catastrophe risks, it seems premature to worry. 3. The justifications given above for risk criteria may strike some as a bad policy guide, since they assume that preserving human lives is in some sense a primary value against which our actions should be judged. Actually, of course, we are guided by many other values. Few people consistently act so as to maximize their own life expectation, for example: many risky pleasures are widely indulged in. Perhaps we should accept that what applies to us as individuals applies also to us as a species: worrying about very small risks detracts too much from the quality of our existence to be the best course. This is certainly arguable. On the other hand, current risk policy tends to count the cost in human lives for a good reason: because that particular value seems to be more widely shared and more strongly held than most. It cannot possibly adequately represent the variety of individual values we bring to any policy debate, but it is a measure that, by general consensus, is very important. Making a generally acceptable policy for extinction risks on some other basis would require establishing a fairly firm consensus on what that basis should be. No such consensus seems to exist at the moment. 4. There is what might be termed the argument of dominant risk. We face many other extinction risks, some natural (large asteroid impact), some wholly or partly self-created (global nuclear war, catastrophic extinction of species as a result of human impact on the global ecosystem, catastrophic climate change as a result of human impact on the global environment). There is a view that suggests that a new artificial risk is acceptable if it is smaller than existing risks. A refinement of this view is that a new artificial risk is acceptable only if smaller than presently unavoidable natural risks. In two further common variants of these two views, "smaller than" is replaced by "very small compared to." Large asteroid impact seems to be the greatest known natural extinction risk that can be reasonably well estimated. The risk of the Earth being hit by an asteroid of diameter 10 km is estimated to be 10 −8 per year.  (17)  Such an impact would be so devastating that it is generally thought very likely that it would cause mass extinctions of species, and very plausible that we would be among the species extinguished. Accepting that last hypothesis, perhaps at the price of another order of magnitude, gives an estimate of 10 −8 -10 −9 per year for this natural extinction risk. Following the argument of dominant risk leads to the socalled asteroid test, according to which an artificial extinction risk is acceptable if smaller than ≈10 −9 per year, or in the more conservative version, very small compared to 10 −9 per year.  10  My impression from discussions is that many thoughtful people find some version of the argument of dominant risk reasonable, but that many equally thoughtful people find this line of argument entirely irrational. My sympathies are with the latter. Why should the existence of one risk, which may be distressingly  10  Versions of the "asteroid test" have been discussed as possible justifications for the acceptability of the BJSW and DDH risk bounds by several people involved in policy formation at CERN and Brookhaven: for example in W. Pratt et al., Brookhaven National Laboratory Memo to T. Ludlam and J. Marburger, 17.2.00. The test is also considered in Reference 14. high, justify taking another easily avoidable risk, which, even if much lower, may still be unacceptably high? Unavoidable natural risks are not normally believed to justify wilfully inflicting avoidable risks on third parties. Everyone now living is very likely to die within the next 120 years, and would be very likely to die of natural causes in that timespan even if exposed to no other risks. An industry that added slightly to the natural risk level, annually killing 10,000 people who had made no choice to accept the extra risk, would not find much sympathy for the defense that these extra deaths were more or less lost in the noise compared to natural wastage. A further problem specific to the asteroid test is that it assumes that asteroid extinction risks are either unavoidable or else, though avoidable in principle, small enough to be tolerable. In fact, the asteroid threat is not unavoidable with current and foreseeable technology, and passive and active countermeasures are being seriously considered. That said, let me reiterate that many people seem to be persuaded by some version of the argument of dominant risk. It no doubt deserves a more careful discussion than is given here. The above brief sketch of a counterargument is not meant to dismiss the "asteroid test" and related criteria out of hand, but rather just to note that there are serious counterarguments. I do not believe these criteria represent anything approaching a consensus view. Unless and until it becomes clear that they do, they cannot legitimately be used to justify catastrophe risks.
 FINAL COMMENT The particular artificial extinction risk considered in this article is hypothetical, and there are good arguments to suggest that the actual risk is small or zero. But, as already noted, we face other undoubtedly real and not necessarily small artificial extinction risks. The above arguments, which suggest that the true costs of extinction are generally underestimated, obviously apply generally. For instance, while the serious costs associated with artificially induced global warming are widely (albeit not widely enough) appreciated, the extra cost associated with the small risk of a truly catastrophic climate change does not seem to have been much con-sidered. Yet it might be that, with proper accounting, the cost of the risk of climatic catastrophe would be the greater. Similarly, although some (insufficient) attention is being paid to the costs associated with the loss of biodiversity caused by human impact on the environment, the cost of the risk of a catastrophic collapse of the global ecosystem seems to have been generally neglected. In these and other areas where modeling is possible, the above arguments suggest we should encourage and pay attention to research into unlikely but not inconceivable catastrophic outcomes, and try to quantify the risk they represent, rather than focusing only on likelier outcomes that may be very deleterious but are not truly catastrophic. 			 These results are reviewed in Jaffe et al. (8)  Whether they offer adequate reassurance that no forseeable collider experiment will be unacceptably risky deserves reconsideration in the light of the arguments set out below. 3  DDH also considered the ALICE experiments, scheduled to take place later at CERN. Their bounds on the risk of catastrophe ensuing from ALICE will not be considered here, though it is worth noting that even DDH regard them as inadequate and suggest further investigation.
			 Some further possible loopholes in those arguments are listed in Reference 13. 5  Indeed, the Brookhaven web pages continue, in April 2003, to direct readers to the original unamended version of BJSW's preprint: see http://www.bnl.gov/rhic/docs/rhicreport.pdf.
			 That these statements misrepresent their results was pointed out to DDH by the author in January 2000.
			 In contrast, Compton's reported statement on the risk of destroying life on Earth by a fission explosion is given in the form of a risk estimate-though, as noted above, it was not justifiable.
			 Arguments along these lines have been suggested in informal discussions, but to the best of my knowledge none has been set out in print. Such an argument would need to be made very carefully, since the theoretical arguments and empirical bounds are not independent. As already noted, the empirical bounds still rely on theoretical assumptions, and if theoretical expectations were incorrect, the derivation of the empirical bounds might also be affected.

/Users/janhendrikkirchner/code/2022/10/alignment-research-dataset/align_data/common/../../data/raw/nonarxiv_teis/Risk Analysis - 2004 - Kent - A Critical Look at Risk Assessments for Global Catastrophes.tei.xml


Abstract: Much of traditional neuroeconomics proceeds from the hypothesis that value is reified in the brain, that is, that there are neurons or brain regions whose responses serve the discrete purpose of encoding value. This hypothesis is supported by the finding that the activity of many neurons covaries with subjective value as estimated in specific tasks, and has led to the idea that the primary function of the orbitofrontal cortex is to compute and signal economic value. Here we consider an alternative: That economic value, in the cardinal, common-currency sense, is not represented in the brain and used for choice by default. This idea is motivated by consideration of the economic concept of value, which places important epistemic constraints on our ability to identify its neural basis. It is also motivated by the behavioral economics literature, especially work on heuristics, which proposes value-free process models for much if not all of choice. Finally, it is buoyed by recent neural and behavioral findings regarding how animals and humans learn to choose between options. In light of our hypothesis, we critically reevaluate putative neural evidence for the representation of value and explore an alternative: direct learning of action policies. We delineate how this alternative can provide a robust account of behavior that concords with existing empirical data.

The Case Against Economic Values in the Orbitofrontal Cortex (or Anywhere Else in the Brain)

The past 20 years have seen a great deal of interest in understanding how our brains implement economic choices  (Camerer et al., 2005; Glimcher & Fehr, 2013; Loewenstein et al., 2008; Padoa-Schioppa, 2011; Rangel et al., 2008; Rushworth et al., 2011) . Much research in this field of neuroeconomics rests on the assumption that choices between options rely on an explicit valuation process  (Kable & Glimcher, 2009; Levy & Glimcher, 2012; Montague & Berns, 2002; O'Doherty, 2014; Padoa-Schioppa, 2011) . That is, that the brain first assigns value to each option and then compares those values to determine choice. The concept of valuation-which stems from economic theory-is so ingrained that it may seem inevitable. How else would one literally compare apples to oranges? Indeed, the idea that value exists on a single cardinal scale, also called a "common currency," has been extended to encompass not only goods, but also effort costs and time delays; everything an agent needs to bundle into evaluation of a mode of action that is aimed at procuring a specific outcome. In parallel, the computational framework of reinforcement learning, which has been a cornerstone of neuroeconomics, also makes a scalar value signal central to its implementation  (Niv, 2009; Sutton & Barto, 2018) . Specifically, in reinforcement learning, the value of an option-a state or an action-is the expected sum of future rewards contingent upon that choice. As a sum of future rewards that may be of different types (and include costs as negative rewards), reinforcement-learning value is naturally calculated in some unitless common currency. However, not all reinforcement learning algorithms rely on or even calculate values  (Sutton & Barto, 2018) , and reinforcementlearning values, as sums of future rewards, are not synonymous with economic values of specific goods. Likewise, many empirically supported process models of choice get by with no valuation  (Gigerenzer & Gaissmaier, 2011; Miller et al., 2019; Vlaev et al., 2011) . The fact that the brain can compute values to compare apples and oranges does not mean that it routinely does so, or that valuation is the primary process underlying choice. In this opinion paper, we argue that in many choice scenarios the brain may not be computing values at all, despite appearing to do so. We will demonstrate that the rationale for a value signal is weak, as are both behavioral and neural evidence supporting value computation. Finally, we propose an alternative-direct learning of action policies-and suggest there is scant direct evidence for value learning that cannot be explained by this and other alternative theories. Our hypothesis is important as it suggests a different interpretation of previous data and requires that studies attempting to resolve mechanisms of value computation in the brain first establish that in the specific situation studied, valuation is indeed occurring. In particular, much work has associated the orbitofrontal cortex (OFC) in representing the expected economic value of different goods or options  (Bartra et al., 2013; Levy & Glimcher, 2012 )-a role that must be critically reconsidered if we agree that the brain may not necessarily be representing such values in many of the experiments so far used to test this hypothesis.
 Why Argue Against Value? Intuitively, our thesis is that while we may know that we prefer an orange to an apple (for one thing, oranges don't brown when exposed to oxygen), we may make this judgment without consulting an internal scalar (cardinal) or even a universal ordinal value signal. Consequently, we may be hard-pressed to express the precise value that we put on an orange. This is not merely an issue of conscious access to our internal value of oranges 1 , but may be due to the fact that deciding on preferences can utilize many alternative mechanisms that don't require or rely on calculation of such a value. Valuation is hard  (Payne et al., 1992) . It is also often unnecessary: when choosing between, say, an orange and a car, it is immediately clear that one is better than the other without calculating the precise value of either. If you are extremely thirsty, you might choose the orange, whereas if you need to go somewhere a car is the only relevant option. Arguably, many real-life choices are between options that are sufficiently different in the needs that they fulfill as to be more similar to this extreme example than they are to the choice between apples and oranges  (Juechems & Summerfield, 2019) . Valuation may only be necessary when choosing between two very similarly valued items. However, if the items are sufficiently similar in how they satisfy our needs, the brain may decide to choose randomly, according to some valuefree heuristic, or according to past choices-and move on  (Chater, 2018) . Alternatively, the brain can try to calculate the exact values of the options to the necessary precision that arbitrates between them. Indeed, choosing between similar options often takes longer than choosing between very different options, even when making rather trivial choices that do not warrant the time and effort invested  (Pirrone et al., 2018; Teodorescu et al., 2016 ). An example is the inordinate amount of time some of us may spend deciding what brand of tuna fish to buy, despite the fact that our time (as per our salary) is worth much more than the money we will save by correctly evaluating which brand provides the best "value for money." This scenario also illustrates how inept we are at comparing the value of different kinds-here, time and money-which should have been straightforward if the modus operandi of the brain was to evaluate everything in terms of some common currency (see below). So perhaps the brain can, when needed, calculate values. However, we argue that this is not the main means by which the brain makes decisions, and perhaps not the natural mode of decision making.
 What Is Value? Before moving on, we would like to delineate precisely how we are defining value for the sake of our argument. The term "value" has multiple uses (the problems this multiplicity raises are carefully laid out by O'Doherty, 2014). We consider "value" a hypothesized scalar variable that reflects the worth of a specific item or outcome. 2 Because it is scalar, it is necessarily abstract. It can refer to any good, and makes use of a common currency code that is comparable across goods of different types (e.g., food, water, recreation time). Value, by this definition, is cardinal, not ordinal, meaning it can be defined for an option per se, rather than solely relative to other options. That is, it reifies the idea of "utils"-quantifiable units of value, often used in a jocular manner in Economics classes. The idea of a common currency is key as it implies that all relative calculations have already happened-the value of an option is in some denomination that objectively defines it as compared to other such values. One might argue that this is too narrow a definition, but we believe this is what neuroeconomists have in mind when talking about representation of common currency values in the brain. For example, we know of no neuroeconomic model that imagines a neural implementation of an ordinal value scale. This having been said, even an ordinal scale should not change based on the comparison set, and should satisfy transitivity-which many of the neural signals attributed to value do not, as we detail below. Our definition of value, for the purpose of this paper, is different from other types of value that are discussed in reinforcement learning. Specifically, here we are discussing value as the reward worth of a single item/event, not the expected sum of all future rewards (R in reinforcement learning models, not V). One might argue that in order to compute such an expected sum V, the subjective worth of all individual rewards must be translated to a common currency that can be added. In this sense, reinforcement learning models do presuppose a common currency reward value. However, they don't necessarily commit to economic properties of such values, such as transitivity and consistency (see also  Juechems & Summerfield, 2019 ), and we discuss below a class of reinforcement learning algorithms that can make do with evaluations that are only relative to the current options, and not applicable in other situations. One important feature of our definition of value is that in our view, although value is inferred from choice, it is not strictly identical to choice, nor necessarily implied by choice. For instance, if we observe a consistent preference for A over B, A is assumed to have a higher value than B. But choice of A over B isn't sufficient to infer the existence of a self-consistent value function: a decisionmaker may adhere to a heuristic policy that results in stable preference for  Slovic, 2006) . Moreover, choice can be altered without manipulating the reward value of an item  (Schonberg & Katz, 2020) , which suggests that value, as inferred from choice, is not untarnished by processes that are not economic in nature. Thus, value can be inferred from behavior given certain assumptions, but preferences do not always lead to a value function. [A > B], [B > C], and [C > A] (Lichtenstein & This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
 The Common-Currency Hypothesis The notion that economic decision-makers make use of internal value functions to compare options is often dated to Daniel Bernoulli's proposal in 1738 of a logarithmic utility curve to explain preferences in the St. Petersburg Paradox  (Martin, 2011) . Here, a decision-maker is offered a chance to play a game in which they win $2 (n−1) with n being the first time, in a series of coin tosses, that a coin falls on "heads." The paradox is that although the expected monetary value of this game (that is, the sum of expected wins multiplied by their probability P ∞ n=1 2 ðn−1Þ • 0.5 n = P ∞ n=1 0.5) is infinity, people are not willing to pay even $10 to play this game  (Hayden & Platt, 2009) . The explanation that Bernoulli proposed proved foundational within microeconomic theory. He argued that decision-makers don't base calculations on the nominal, objective cash value of the potential gain, but rather on its subjective value. If the subjective value grows more slowly than the objective value (the idea of "diminishing marginal utility"), an optimizing decision-maker will appear riskaverse, for instance, when evaluating the St. Petersburg gamble. Utility, or subjective value, has been central to many if not most microeconomic models. These include the axiomatic approaches of Pareto, Von Neumann and Morgenstern, and Samuelson. It is also central to behavioral theories such as prospect theory and decision field theory  (Busemeyer & Townsend, 1993; Kahneman & Tversky, 1979) . Ironically, however, explaining choices in the St. Petersburg paradox using such a subjective value function requires utility for money that diminishes so rapidly that it does not generalize to choices in other contexts. Instead, heuristic accounts-accounts that don't rely on ideas about value maximization-provide better quantitative matches for St. Petersburg choices  (Hayden & Platt, 2009) . Despite its central importance in economics, economists are typically agnostic about whether the concept of value is just a convenient description, or whether it is instantiated in the brain. As Friedman argued, decision-makers behave as if we compute and compare values, but we cannot conclude that we actually do so  (Friedman, 1953) . In that work, he famously compared economic agents to a trained billiards player who makes excellent shots as if having a sophisticated grasp of Euclidean geometry, although in fact such a theoretical understanding is not necessary for good billiards skill. Economists typically stop at the point of saying that economic models are "as if" models, with some arguing that the question of the underlying reality of economic variables is outside the domain of economics  (Gul & Pesendorfer, 2008; Harrison, 2008) . In contrast, neuroeconomics has generally taken as a default assumption that these "as if" theories are reified in the brain  (Kable & Glimcher, 2009; Levy & Glimcher, 2012; Montague & Berns, 2002; O'Doherty, 2014; Padoa-Schioppa, 2011; Rich & Wallis, 2016) . Modern tools such as neuroimaging and single unit physiology provide the opportunity to assess the implementation of decision making directly. Indeed, neuroscientists have had little trouble identifying correlates of value in several brain areas  (Knutson et al., 2001; Levy & Glimcher, 2012; Plassmann et al., 2007; Wallis, 2007) , leading to the suggestion that the OFC is a nexus for representing the economic value of goods in the brain (see, e.g.,  Rushworth et al., 2011; Wallis, 2007) , alongside other brain areas that are important for computing and representing value such as the ventromedial prefrontal cortex  (Bartra et al., 2013)  and ventral striatum  (Haber & Knutson, 2010) . However, it is not clear that the signals identified in these studies actually represent value as proposed-on a common-currency scale, for comparing options and making choices. Moreover, so called "value signals" only correlate with value, so may be signaling other quantities, such as attention, action plans, vigor, or preference, which also correlate with value  (Maunsell, 2004; O'Doherty, 2014; Wallis & Rich, 2011) . Hence it is important to carefully consider alternative interpretations for these findings, as we do further below, after discussing some practical constraints and philosophical conundrums.
 Alternatives to Valuation in Decision Making While common-currency value provides a convenient and general mechanism for making choices, there are many possible alternatives  (Gigerenzer & Gaissmaier, 2011; Kahneman et al., 1982; Lichtenstein & Slovic, 2006; Vlaev et al., 2011) , some of which are quite general and robust. Consider for example the "priority heuristic"  (Brandstätter et al., 2006) . This de minimis heuristic approach proposes that decision-makers first identify a dimension along which options vary and then compare options along that dimension. If that results in a choice, they stop; otherwise, they move on to the next dimension. This heuristic can explain many phenomena, including Allais preferences  (Allais, 1953) , the reflection effect  (Fishburn & Kochenberger, 1979) , the certainty effect  (Kahneman & Tversky, 1979) , the fourfold pattern in risky choice that motivates prospect theory  (Tversky & Fox, 1995) , and several intransitivities-all without ever requiring computing of value  (Brandstätter et al., 2006) . And the priority heuristic is one of a large number of heuristics that do a remarkable job at describing behavior  (Gigerenzer & Gaissmaier, 2011) . These heuristics are generally motivated by psychological observations, and thus are consistent with known data from the psychology-if not the neuroscience-of choice. In particular, they reflect the assumption that calculating value is difficult, that humans typically use shortcuts whenever possible, and that heuristics are a good shortcut  (Gigerenzer & Gaissmaier, 2011; Lieder & Griffiths, 2020; Payne et al., 1992) . Moreover, these heuristics are not limited to humans, but apply to other species, including monkeys  (Heilbronner & Hayden, 2016; Marsh, 2002; Santos & Rosati, 2015; Shafir et al., 2002) . Importantly for our argument, the success of heuristic approaches demonstrates that value calculations are not a priori essential for neuroeconomic theories of choice. In particular, heuristic theories can solve many problems for which value is proposed to be needed  (Kahneman et al., 1982; Lichtenstein & Slovic, 2006; Piantadosi & Hayden, 2015; Stevens, 2016; Tversky, 1969; Vlaev et al., 2011) . Heuristics readily allow for comparison of multi-dimensional goods and bundles, and for choice across dissimilar goods. While heuristics are not perfect, and lead to many choice anomalies, choice is, empirically, full of anomalies. Moreover, several non-heuristic process models also eschew value computation steps, including decision by sampling, query theory, and fuzzy trace theory; these all account for a wide range of choice behavior as well  (Reyna, 2008; Stewart & Simpson, 2008; Stewart et al., 2006; Weber et al., 2007) .
 Practical Issues in the Neuroscience of Value Representation Despite the above, it is often considered axiomatic that an internal, neural, value scale must exist in the brain. The job of neuroscientists is then to find this signal-to pinpoint brain activity This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. that correlates with value. In this section, we challenge this viewpoint by considering some basic issues that come up in the neuroscience of value. The first problem is that it is impossible to precisely measure value. We can measure preferences between options and use the data to infer option values, but elicited preferences are noisy measurements that also reflect factors other than value. For example, depending on how preferences are elicited, they may also reflect the tendency to press the same button repeatedly rather than change actions, the tendency to switch between options due to a prior belief about depleting resources, or the amount of attention or looking time for each of the options  (Armel et al., 2008; Schonberg & Katz, 2020; Shimojo et al., 2003; Sugrue et al., 2005) . Finally, value may shift from trial to trial, even depending on recent outcomes, meaning that methods that average across trials produce misleading value estimates  (Sugrue et al., 2005) . One could, in theory, incorporate these factors into the inferred value of an option. For example, there may be inherent value in choosing the same thing twice in a row. However, it becomes unclear what the definition of the "option" is that is being evaluated: if the value of an apple after eating (or choosing, or even just viewing) an orange is different from the value of an apple without that proximal experience, can we determine the value of goods at all? And if we can change the value (read: preference) for an option just by directing more attention to it  (Salomon et al., 2018; Schonberg et al., 2014) , or inducing choice of it  (Izuma et al., 2010; Sharot et al., 2012; Voigt et al., 2017) , is preference really measuring the subjective economic worth of a good? The risk is circularity-if any choice behavior can be explained by supposing value that depends on local history, then the concept of value adds no additional explanatory power beyond that of recent events and choices. Different ways to measure value raise a second challenge of inconsistent measures. Indeed, if we had a single neural value function we called on, values elicited by different measures would match. Unfortunately, they do not  (Lichtenstein & Slovic, 2006) . To explain this, in their seminal work, Sarah Lichtenstein and Paul Slovic proposed that preferences do not arise from any internal value function, but instead are constructed at the time of elicitation (see also  Ariely et al., 2003; Payne et al., 1992; Tversky & Shafir, 1992) . That is, in the view of these and like-minded scholars, value doesn't sit in the brain waiting to be used; rather, preference is a complex and active process that takes place at the time the decision is made. Critically, in this theory we compute choices in a largely ad-hoc manner based on the available options, without an intermediary common-currency valuation stage. As such, there is no guarantee of consistency or reliability; any consistency or reliability observed may be explained as a result of strong attractor states in the way the system determines the choice, and deviations from consistency are evidence for the specific nature of the algorithm and its idiosyncrasies. This hypothesis, while denouncing value as a latent construct in the brain, nevertheless invites neuroscientific research to understand the active processes involved in choice, and how these relate to (relative) evaluation, if not to economic values. This having been said, experimenters can roughly estimate value, even if not measure it precisely. For example, in a given experiment they can examine choices and determine with confidence that a monkey (behaves as if it) places more value on a gamble as compared to a safe option with a matched expected value. One could then use this fact to try to identify a neural correlate of value. However, for this neural endeavor to be valid, it is important to identify all confounding variables and regress them out. O'Doherty's (2014) review delineates the practical difficulty of doing so, using the overall rubric of "visceral, autonomic, and skeletomotor" activity. In practice, confounding variables include both stimulus and outcome identity, information about the state or structure of the world, the surpriseness, informativeness, and informational value of stimuli, details of the action associated with selecting of consuming the reward, including its likelihood and vigor, and the attention and arousal engendered by the stimulus (e.g.,  Blanchard et al., 2015; Botvinik-Nezer et al., 2020; Niv et al., 2007; O'Doherty, 2007; Roesch & Olson, 2004; Roesch et al., 2006; Wilson et al., 2014; . Indeed, in studies that separately assess encoding of outcome identity versus outcome value, activity in brain areas that are often considered to be emblematic of economic value (in particular, the OFC) turns out to correlate with outcome identity instead  (Klein-Flügge et al., 2013) . A final insurmountable problem is that it may be impossible, even in theory, to obtain a brain measure of value that is independent of behavior. Suppose for example, that we identify a particular class of neurons whose firing rates are perfectly correlated with value down to our ability to measure it through preference. Supporting this idea, we observe that any procedure that modifies value (as inferred from behavior) changes the firing rate of these neurons in a manner consistent with our predictions. We may tentatively hypothesize that these neurons are (or are among) the value neurons of the brain. However, as shown by Schonberg and colleagues, preference can be changed irrespective of changing the economic worth of goods  (Schonberg & Katz, 2020) . Therefore, to differentiate value neurons from preference neurons, we would need to show that these neurons do not strictly follow expressed preference when the value function diverges from it. This is, of course, not possible if the value function never measurably diverges from preference. And if we assume that values can diverge from preference, it is not clear how to define values to start with. We call this "the neuroeconomic relativity problem" because, like Einstein's relativity problem, it reflects the fact that there is no external reference frame to which one can calibrate value inferences.
 Reconsidering the Motivation for Common Currency It may be worth asking, then, what does having a single common currency buy you? Why would the brain invest in such an organization? One advantage of a common-currency scale is that it simplifies comparing options that differ along multiple dimensions. For example, when hunting for an apartment, the options may differ along dimensions of price, area, neighborhood, and amenities. The logic is that these dimensions must be first combined into a single scalar per apartment so that the scalars can be compared. However, this is not the only way to solve the problem, and importantly, may not be the way humans make their decisions. For example, the apartment shopper may choose a single dimension and pick the winner along that dimension, as discussed above, or may compare separately on each dimension and choose the apartment that wins on most counts  (Tversky, 1972) . Laboratory studies where humans can choose what attributes to view indeed suggest that people don't uncover all the attributes of one option (to calculate its value) and then continue to the next option, but rather prefer to view information for all options attribute by attribute  (Fellows, 2006; Hunt et al., 2014) . Indeed, as mentioned, much empirical work indicates that human decision-makers broadly favor heuristic approaches that This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. THE CASE AGAINST ECONOMIC VALUES eschew a value stage in a large number of contexts  (Brandstätter et al., 2006; Gigerenzer & Gaissmaier, 2011; Kahneman et al., 1982; Lieder & Griffiths, 2020) . Notably, systems that make use of heuristics may generate internal variables that are conceptually distinct from value but that correlate with value, thus leading to an interpretational confound. Consider, for example, a relatively well-understood implementation of choice in a (non-brain) distributed system: the selection of hive sites in bees (Apis mellifera,  Seeley, 2010; Seeley & Buhrman, 1999; Seeley et al., 2006) . Bee swarms select a hive site by sending out scouts to investigate potential sites. Each site differs along roughly 20 dimensions of varying but measurable importance (size, safety from predators, exposure to sunlight, wind, etc.). Each scout bee that encounters the hive site performs an extremely poor estimation of the value of the site-it typically will only sample three-to-four of the dimensions and even then, estimate them poorly. The bee then returns and if the estimated site quality is sufficiently good, indicates an assessment of its quality to the other bees in the swarm. The quality it signals will be correlated with the overall value of the hive site, but only weakly, and, critically, will only integrate a subset of relevant value components. The overall comparison between options is indirect-the options race to attract adherents; the majority of the decision is made by a positive feedback process. This example is particularly relevant because scholars of perceptual decision making may recognize this process as strongly related to ideas about how the brain decides between options by racing to bounds, and there is evidence that deliberative processes in the brain follow the same principles as choice processes in bee swarms  (Eisenreich et al., 2017; Franks et al., 2003; Mitchell, 2009; Pais et al., 2013; Passino et al., 2008; Pirrone et al., 2018) .
 Evaluating the Neural Evidence for the Common-Currency Hypothesis Evidence for the common-currency hypothesis comes from the observation that firing rates of single neurons or hemodynamic responses of voxels correlate with values of offers and outcomes  (Kennerley et al., 2009 (Kennerley et al., , 2011 Levy & Glimcher, 2012; Padoa-Schioppa, 2011; Rangel et al., 2008) . These responses depend on multiple elements of offers (e.g., the expected reward, as well as the associated response costs), and are modulated by factors that affect subjective value, such as context or level of satiety (see  Conen & Padoa-Schioppa, 2019; O'Neill & Schultz, 2010; Rudebeck et al., 2017  for only a few examples). Often, variations in these firing rates predict variations in choice  (Conen & Padoa-Schioppa, 2015; Strait et al., 2014; Sugrue et al., 2004) . Such patterns have been taken as evidence that the neural signal encodes value, in particular implicating the OFC  (Bartra et al., 2013; Levy & Glimcher, 2012; O'Reilly, 2020) . However, such patterns of neural responding do not definitively demonstrate a common-currency value signal. For example, in the original finding of value-encoding neurons in the OFC  (Tremblay & Schultz, 1999) , neurons responded more strongly to a high-valued option than to a low-valued one. However, the principle of common currency only held within a given choice pair-responses for Option B were low when this option was paired with a better Option A, but were high when this same option was paired with a worse Option C. Other studies found value-correlated responses to be more stable across contexts  (Padoa-Schioppa & Assad, 2008) , but most findings support an encoding that depends on the alternatives (e.g.,  Kobayashi et al., 2010; Padoa-Schioppa, 2009; Zimmermann et al., 2018) . One might interpret this finding as reflecting simple range adaptation of neurons that have a finite firing rate; however, a more parsimonious explanation is that the firing pattern is consistent with a relative preference code rather than an abstract value code. This interpretation places the putative neural representation of value one stage later than we would expect from a common-currency codeimmediately after comparison (because a relative valuation is itself a comparison) rather than as an input to comparison, raising the question of where the input came from. An interpretation of this code as something other than economic value would obviate this worry. Importantly, with a code that depends on alternatives, there is no real sense in which we can read out the "true subjective value" of an option. We can only know if an option's value is higher than another value-similar to the information provided by preferences and choice behavior. Another corollary of such a relative code is that it is unclear to what extent we can read out a meaningful value signal when only one option is available. Indeed, presenting a subject with one option at a time should, theoretically, provide the best ability to read out option values from neural activity in areas associated with value, such as OFC, ventromedial prefrontal cortex, and dorsal and subgenual anterior cingulate cortices. Doing so reveals that while neural activity in these regions does correlate with the value of the first option, neural responses to the second (alternative) option (presented on its own) correlate with the value difference, that is, they reflect the result of value comparison rather than valuation per se  Hunt et al., 2018; Strait et al., 2014) . Moreover, even responses to the first offer do not simply encode its value, but also contain information about the likelihood that option will be chosen (presumably relative to the expected value of the second offer,  Azab & Hayden, 2017) . These results suggest that even when evaluation is experimentally segregated from comparison, pure value encodings may not exist. In fact, the idea that prefrontal neurons are selective or responsive to a single experimenter-defined variable has been increasingly falling out of favor  (Fusi et al., 2016; Raposo et al., 2014; Rigotti et al., 2013) , with "mixed selectivity" appearing to be a core operating principle of prefrontal cortex, including in ostensible value regions  (Blanchard et al., 2018; Hayden & Platt, 2010; Kimmel et al., 2020; . A third issue is that although the activity of some neurons in the OFC is correlated with value (but not necessarily linearly related to value), activity of other neurons in this same area is anticorrelated with value, with the majority of neurons showing no relationship at all with value. This raises the possibility that the neurons are encoding something other than value, for instance, a distributed representation of the identity of each of the three options, A, B, and C above, or the identity of the stimulus representing the offer. With the small number of options evaluated in most experiments, such a distributed code of outcome or stimulus identity (but not value) can easily result in some neurons randomly firing most strongly for the higher-valued of the three options and least for the worse option, while in other neurons the relationship would be in the opposite direction. This would also explain why many neurons show a nonmonotonic relationship between value and their firing. Recently, using ensemble recordings in the OFC that allow analysis on the level of a single trial, Wallis and colleagues have attempted a more direct test of the hypothesis that OFC encodes value  (Rich & Wallis, 2016) . In their task, monkeys were offered two options (denoted by images corresponding to the options) and asked to choose between them. On some trials, only one option was offered. Classifiers were trained to classify options corresponding to each of four offer-value levels (0.05, 0.10, 0.18, and 0.30 ml juice, or, in separate blocks, four levels of second-order reinforcer), aggregating over the two images corresponding to each value level. Using single-unit activity and local-field-potential recordings in the OFC, the authors could classify the offered value above chance, suggesting that OFC neurons encoded information about value. However, here too, offer values may have been encoded as different (outcome stimulus) identities, not different (cardinal) values in a common currency. One stringent test of the value-coding hypothesis would be to show that a combination of two offers of value level 0.05 ml resulted in similar activation pattern to a single offer of value level 0.10 ml. Ensemble recordings that give ample data in a single trial allow testing these hypotheses with novel combinations that have not been trained to predict the same identity of reward through multiple presentations, therefore testing the assumptions of the additivity of common currency directly. This critical test has not been conducted, to our knowledge. As mentioned, many factors that are conceptually distinct from value can influence choice-and are therefore closely correlated with value  (Maunsell, 2004) . Reward value drives attention, promotes both short-and long-term learning, primes behavioral adjustments, updates internal models, activates circuitry that detect both positive and negative surprises, and elicits mental computations of cost-benefit tradeoffs as well as comparisons with what could have been chosen. All of these are different from scalar, commoncurrency value, but are known to drive neural activity in the regions usually associated with economic value. As such, they confound that interpretation. This problem is a long-standing and notorious one in neuroeconomics  (Maunsell, 2004; O'Doherty, 2014; Roesch & Olson, 2003) . To overcome some of these potential confounds, a strong tradition in neuroeconomics research is to control for extraneous factors such as salience and response cost (although we note that many factors-for example, covert attention-cannot be controlled for). As a practical issue, it is difficult to control for alternative interpretations without making the task so convoluted that it becomes unnatural for the animal, in the ethological sense  (O'Doherty, 2014) . As a result, it is not clear if findings of value computation (if we believe them to be so) in these tasks would naturally translate to how the brain computes choice in naturalistic situations. But the deeper issue remains that value as inferred in most experiments is definitionally a summary of aggregated choice behavior, and therefore, any variable that influences choice behavior will necessarily be correlated with value. It is for this reason that we suggest that a stronger demonstration of scalar value coding in the brain should show mathematical properties such as the additivity of value, or separation from preferences as these are changed without changing value, as suggested above. If the response of neurons to the novel sum of two stimuli each promising 0.1 ml of juice is similar to the response of those same neurons to a different stimulus associated with 0.2 ml of juice, and especially if this response did not change when preference for an option was induced by means such as the mere exposure effect  (Schonberg & Katz, 2020) , it would be harder to argue that this is due to a shared motor plan, or attentional capture. We note that research to date has shown that the activity of brain areas associated with value, in particular the OFC, does change when preferences are modified through methods that should, in principle, not change economic value  (Botvinik-Nezer et al., 2020) .
 An Alternative View: Direct Learning of Policies Although reinforcement-learning models have contributed to the assumption that the brain computes values, many reinforcementlearning algorithms do not learn or estimate values for different actions. Instead, they directly learn action policies. In fact, in the reinforcement-learning literature, the goal of an agent is to obtain as much reward as possible by executing optimal actions-calculating values is only one means to achieve that end. In explaining reinforcement-learning methods,  Dayan and Abbott's (2001)  textbook begins with the "direct actor"-an actor that learns actions without computing their values. The Actor-Critic model, a prominent algorithm for reinforcement learning that has been linked to the brain  (Barto, 1995; Joel et al., 2002; Maia, 2010; O'Doherty et al., 2004; Takahashi et al., 2008) , does exactly that. In this algorithm, a Critic module learns values of states in terms of expected future rewards (here, the state includes all available actions, averaging over choices and explicitly not computing the value of each possible choice), and uses these to compute reward prediction errors. These prediction errors are used to learn an action policy in the Actor module: the probability of actions that are followed by positive prediction errors is increased, and the probability of actions that are followed by negative prediction errors is decreased. Under some reasonable conditions, this model learns correct reward-maximizing policies  (Sutton & Barto, 2018) . However, the quantities learned by the Actor-tendencies to perform one action over the other-cannot be read out as action values. In particular, due to the way the algorithm learns, ties are broken between equally good actions such that eventually agents learn deterministic policies. To be clear, if four actions were to lead to 1, 2, 3, and 3 drops of juice, respectively, the model may learn to always choose the third option, or to always choose the fourth (both optimal policies). At the end of learning, action weights in the Actor may be 0, 0, 1, 0 respectively, losing all information about value, or relative value. Even before convergence to a deterministic policy, weights may be 0, 0.1, 0.95, 0.2 respectively (or any other combinationweights here do not have to sum up to 1, and these are just illustrative numbers). Importantly, if the basal ganglia indeed implement an Actor-Critic learning algorithm in the brain, there is no sense in which we can glean action values from the Actor or the Critic. The Actor-Critic model is only one of a class of reinforcementlearning algorithms that learn policies directly, that is, without calculating option values (e.g.,  Sutton et al., 2000; Williams, 1992) . In their general form, these algorithms maintain an action policy, use experience to evaluate a gradient direction for this policy (that is, what change in policy would increase the overall obtained reward), and change the policy in that direction. Another notable model, developed to explain behavioral patterns in choices, is the Experience-Weighted Attraction model of  Camerer and Ho (1999) , which interpolates between value calculation and direct policy learning. Recent findings suggest that both behavior and learning signals measured in humans during decision making are more in line with a policy-learning algorithm than with value estimation. For instance, This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly. THE CASE AGAINST ECONOMIC VALUES  Li and Daw (2011)  had participants choose between two options that gave reward with different probabilities. After each choice, both the outcome of the chosen option and the counterfactual outcome of the unchosen option were displayed. Behavior, as well as neural signals corresponding to prediction errors in the basal ganglia, suggested that subjects were updating both options in opposite directions, learning relative choice propensities (a policy) rather than tracking the expected value of each option. In another task in which humans chose between pairs of options and were able to view both the outcome of their choice and the counterfactual outcome of the forgone option,  Palminteri et al. (2015)  had subjects learn which of two probabilistically rewarding options is better, and which of two probabilistically punishing options was better. At test, subjects were asked to choose between pairs from both the rewarding and the punishing contexts. Surprisingly, when choosing between the less rewarding option and the less punishing option, subjects tended to choose the less punishing option. This is consistent with policy learning, as that option had been the favored option in the punishment context, whereas the less rewarding option had been the disfavored one in the reward context. However, the value of a sometimes-rewarding option is clearly higher than that of a sometimes-punishing option, hence value learning cannot explain this fundamentally suboptimal choice pattern. Direct learning of policies is consistent with basic tenets of decision making, such as the fact that choices are stochastic even when no exploration is necessary or warranted (e.g., choosing between two gambles that are fully described;  Khaw, Li, et al., 2017) . Because policies are relative quantities (preference for one option implies unfavorability of another, as the probabilities of all choices have to sum to one), they also explain common violations of the independence axiom such as the effects of third-option "decoys" on choice  (Soltani et al., 2012) , and temporal and spatial contextual influences on choice  (Khaw, Glimcher, et al., 2017) , although these phenomena can also be explained by valuation models that involve range normalization.
 Conclusion The field of neuroeconomics started with the putative identification of pure value signals  (Platt & Glimcher, 1999) . The meaning of these signals was disputed early on  (Maunsell, 2004; Roesch et al., 2006) , but deep questions were set aside as researchers continued to identify brain areas with parts of the computational processes of choice, and in particular, identified the OFC with the seat of economic value. However, while those debates have subsided, the problems they raised have not been resolved. Progress on these issues will require additional data, but we stress that not every experiment that involves choices also implies valuation, and one must be careful in interpreting data not only because of potential confounds, but also because we must be wary of treating a hypothesis-that the brain computes value-as an axiom. We also suggest the need for additional philosophical work to define value in a way that is-at least in principle-dissociable from other factors that promote choices  (Juechems & Summerfield, 2019) . In other words, we argue for a return to the productive debates of the early days of the field, 20 years ago (for neuroeconomics), and earlier (for its psychological underpinnings). Bolstered by an additional 20 years of new data, such debates would surely benefit the field of neuroeconomics moving forward. They would also potentially help reconcile conflicting views on what the OFC might or might not be doing in decision making  (Stalnaker et al., 2015) . 			 We note here that, in neuroeconomics, value-based choices are generally thought to be part of explicit, aware, goal-directed, or model-based decision making that relies on frontal-cortex areas, in particular, the OFC.2 We consider subjective worth as revealed by preference, following neuroeconomic theory, but note the limiting focus on only the "wanting" not the "liking" side of value (Berridge, 1996) .
			 This document is copyrighted by the American Psychological Association or one of its allied publishers.This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.