Public Reason 5 (1): 3-13 © 2013 by Public Reason Social Norms: Repeated Interactions, Punishment, and Context Dependence Jonathan Grose & Cedric Paternotte1 University of Bristol, Ludwig-Maximilians-University Munich I. G A M E TH EORY A N D SOCI A L NOR MS Game-theoretic approaches to social norms have flourished in the recent years, and on first inspection theorists seem to agree on the broad lines that such accounts should follow.2 By contrast, this paper aims to show that the main two interpretations of social norms are at odds on at least one aspect of social norms, and both fail to account for another aspect. We are sympathetic to the broad project of using game theory to model social norms. Our aim is not to undermine this project or argue that no coherent framework or frameworks is possible in principle. Rather, we aim to show, with reference to two particular issues, that at this stage it is unclear if and how differing approaches can be integrated into a unified picture. The two issues we go on to discuss are, first, the role of punishment within the models and, second, how they deal with the context dependence of norm triggering. In each case it is the question of whether or not the model should be one of repeated or one-shot interactions that is the source of problems for integration. When it comes to different modelling approaches, the recent literature provides two presentations of social norms. Ken Binmore outlines a model that frames social norms in terms of repeated interactions (1994, 1998, 2005). In contrast, Cristina Bicchieri's model is of one-shot interactions and involves utility transformations, triggered by social context, that make norm conformity the rational behaviour (2006, 2008). Both of these authors repeatedly refer to each other's work as belonging to a common programme but, we argue, their difference on the question of repeated interaction models makes integration of their approaches problematic. We draw attention to the fact that differences on the question of whether we should think in terms of repeated encounters or not lie at the very foundations of the models. Such disagreement makes common approaches to punishment and context dependence deeply problematic. This is not to say that having multiple approaches is unwelcome in principle. Still, those unfamiliar with the literature could be forgiven for taking it that there are core similarities common to all game theoretic treatments of social norms. In 1] This paper stems from our earlier "Social Norms and Game Theory: Harmony or Discord?," (Paternotte & Grose, 2012). This paper shares its overall diagnosis but adds to it by providing a deeper analysis of several points. 2] Despite their disagreements on specific issues; see for instance the issue 9 (vol. 3) of Politics, Philosophy and Economics, 2006. Social Norms: Repeated Interactions, Punishment, and Context Dependence 4 fact, the only issue on which they unambiguously agree is that norms should be modelled as being equilibria of a game theoretic model. This leaves open the question of what is the correct game to be modelled and it is to that issue that we move in section two. II. TO R EPE AT OR NOT TO R EPE AT? First, note that it would be totally unreasonable to demand that all social interactions be modelled by the same game. Our point in this section is that a fundamental difference in the models presented by Binmore, Bicchieri and others makes it difficult to see them as compatible analyses that can sit on the shelf next to each other and be called upon depending on the strategic nature of the situation at hand. Rather, they disagree at a deep level about the way in which we should model norms and important features such as punishment. The core difference between these approaches to social norms that we consider is the type of interaction modelled. It has been common to identify the fundamental problem of social cooperation with the prisoner's dilemma (PD). For instance, it has been suggested Hobbes' Leviathan presents an informal analysis of PD interactions (1991). More recently, Brian Skyrms (2004) stresses the value of thinking about social interactions in terms of the "Stag hunt" game, citing Rousseau as a precursor. Previously, Skyrms (1996) devotes chapters to the PD and to Hawk-Dove / Chicken games. These studies reflect one aspect of norm-following that has been stressed in the literature, that being that norms pull us towards actions not in accordance with our narrow self-interest (Young 2008). In most of the above games the equilibrium outcome is at odds with that which is optimal from society's point of view. The exception being the stag hunt in which there is a coordination problem between two equilibria, one of which is Pareto superior to the other. Bicchieri's account of social norms is broadly in keeping with these approaches. She stresses that norms operate in "mixed motive" interactions. These are loosely characterised as those in which there is a conflict of interest and the potential for mutual gain. Given this loose characterisation, her definition appears to apply to a very wide class of games. Demanding that a game contains conflict to some degree can be taken as meaning that different players' payoffs are not perfectly correlated. Requiring the presence of a mutual benefit can be translated as requiring that there be a Pareto-dominant payoff profile or a Pareto undominated one, or even one which provides a better average payoff. Although her characterisation is imprecise we can see that the focus is on norms overcoming conflicts of interest. This becomes apparent when we examine the way in which the action of norms is modelled. Bicchieri is very explicit that, "the problem that a social norm is solving in the first place is never a coordination problem." (Bicchieri 2006, 34; her emphasis) When a social norm exists, it changes the agents' utility functions. Preference for conformity comes about via a transformation of player i's utility function which converts a mixed-motive Jonathan Grose & Cedric Paternotte 5 game to a coordination game in which all (or, more accurately, enough) players prefer to conform. Once the players' utilities have been transformed, mutual norm compliance becomes an equilibrium of the transformed game. When the norm is triggered, agents' utilities are modified for every result of the game as a function of the highest loss caused by anyone's deviations from the norm. My being sensitive to the norm makes me consider as less valuable the consequences of actions that do not follow it (whether these actions are mine or others'), in proportion to the highest loss that this combination of actions entails (Bicchieri 2008, 199, Appendix 9.1). When it comes to reasons for this preference transformation when a norm is triggered, Bicchieri makes a number of suggestions. Agents may be motivated by the desire to please others, recognition that others normative expectations are reasonable and, most importantly for our purposes, fear of punishment (Bicchieri 2006, 29). At this point we emphasise the feature of this account on which we subsequently focus. The model is of so-called one-shot games with no modelling of repeats of the interaction. Norm-following becomes rational, equilibrium behaviour in the one-shot game once the players' utilities have been transformed. In this respect Bicchieri's model is similar to Herbert Gintis's recent model in which a "normative disposition" discounts non norm-following acts (2010). In contrast, Binmore's account is explicitly one of repeated interactions. For him, one-shot interactions are not those for which we are prepared by either our biological or cultural heritage. The "game of life" is an indefinitely repeated game. (Binmore 1994, 25) It is this focus on repeated interactions that makes Binmore concerned with a different role for norms than that of transformation of preferences. He focuses on the coordinating role of social norms that helps us to make our actions fit appropriately with each other. In game theoretic terms this fit is framed in terms of behaviours being in equilibrium with each other, where neither player would want to unilaterally change their action. The "folk theorem" of repeated game theory demonstrates that indefinitely repeated games have multiple equilibria, that is, multiple different ways in which actions can be mutually appropriate, and thus players face an equilibrium selection problem. (Myerson 1991, §7.5) For Binmore, social norms solve this problem by making a particular behaviour salient in a particular context. Our shared cultural heritage is what allows us to coordinate on an equilibrium. Notice that in the case of Bicchieri's (and Gintis's) norms, the coordination role of norms need not arise because the utility transformation makes norm-conformity the one rational outcome. It should also be emphasised that Binmore's focus on repeated games makes his account totally general when it comes to the base games that can be repeated. PDs, stag hunts, hawk-dove, coordination games and so on can all fall under this approach and in each case the equilibrium selection is between equilibria of the repeated game rather than what would be the equilibria if the games were played just once. At this point it appears that there need be no incompatibility between these approaches to social norms and scope for them to be integrated. Preference transformations Social Norms: Repeated Interactions, Punishment, and Context Dependence 6 attempt to capture the notion of norm-following pulling people to act against their narrow self interest and their coordination role is reflected in the difficult equilibrium selection problem faced in repeated games. Integration would take the form of the Bicchieri's games forming the base games for Binmore's repeated interactions. However, we first argue, at least on the important issue of punishment, the different accounts cannot be straightforwardly integrated. We then show that they both fail to adequately describe how context-dependent social norms are, for related but different reasons. III. W H AT PL ACE FOR SA NCTIONS? There are at least three ways in which to model game theoretically the action of punishment or social sanctions. As we will see, different approaches to norms model them in different ways. In itself this is not a problem. As we stressed in section one, there is potential value in having multiple modelling approaches. The problem here is that the different ways of thinking about punishment makes them appear incompatible with each other in the sense that they cannot be integrated. This is of particular concern because punishment of non-compliance is one of the central features of any informal account of social norms. Returning to the three game theoretic models of punishment, one possibility is for the sanction to feature as a payoff alteration. For instance, if defection in a PD is punished, and payoffs following defection are reduced, mutual cooperation can become an equilibrium outcome. This payoff transformation option is certainly the one taken by Gintis. He emphasises the "choreographing" role of norms in (one-shot) games with multiple equilibria. However, where compliance is not already part of an equilibrium outcome, sanctions can play a formal role as an argument in the utility function, thereby tipping the balance in favour of norm-following being an equilibrium (2010). In the case of Bicchieri, punishment does not feature explicitly in her formalization. However, as raised in section two, sanctioning is cited as a reason for the utility transformation that is a central feature of her model. In fact, it is unclear precisely how punishment connects with her formal model and in this sense the role of sanctions is poorly integrated with that formal account. An obvious possibility is the payoff altering role we are considering here. In that case the discounting of payoffs associated with non-conformity with the norm would reflect punishment received. However, if we look closely at Bicchieri's formalization it seems that this cannot be a correct interpretation, something emphasised by Daniel Hausman (2008). Remember that the discounting of norm-breaking actions is a function of the negative consequences of all players' deviations from norm-following. If the discounting of my payoffs were due to potential punishment of me then we would expect such discounting to be a function only of my own deviations from the norm. So, while it is clear that, according to Bicchieri, sanctions play a role in transforming utilities, it is unclear precisely in what way this is cashed out in the model. Jonathan Grose & Cedric Paternotte 7 A second formalization of sanctioning is to expand the game to make the punishment action an explicit move. This has the advantage of prompting the modeller to pay attention to the possible consequences of punishing for the punisher. In particular, punishing is very often taken to involve paying costs oneself.3 We will not expand further on this option since it is not one taken by either Bicchieri, Gintis or Binmore, to whose repeated interaction model we now move. In models of repeated interactions punishment behaviour can be modelled in a way unavailable to one-shot models. In this case some plays of the base game constitute sanctioning behaviour. Take, for instance, the indefinitely repeated PD. Permanent mutual cooperation can be the outcome of equilibrium strategies. One, but not the only, possible equilibrium strategy is the famous "Tit-for-Tat."4 In this case cooperative behaviour is reciprocated but, importantly for its being an equilibrium, so is defection. Retaliatory defection is interpreted by Binmore as a punishment for breaking the cooperative norm. A less forgiving strategy is the so-called "GRIM strategy" (Binmore 1994, 197). This cooperates until it is defected against and then switches to the punishment of permanent defection. A strength of interpreting punishment in terms of strategies in repeated games is that it naturally makes explicit that punishment can take many forms in terms of its duration and by what it is provoked. For instance, "Tit for two Tats" requires two successive defections before it punishes with defection. However, a weakness is that punishment loses some of its special status found in informal accounts of social norms. What we mean by this is that there is nothing distinctive about punishing as represented by a repeated game strategy compared to any other behaviour that is conditional on ones partner's actions in previous rounds of play. Having raised some tensions in both Bicchieri and Binmore's framing of punishment we move on to the question of integrating their models since, even if the issues raised above are not fatal to their accounts, there remains a problem with bolting them together. Remember that the integration suggested at the end of section two was that the utility transformation role of norms acts on the base game and that we can then examine the coordination role for norms when repetitions of these games create, via the folk theorem result, an equilibrium selection problem. However, we have already seen that punishment plays a role in both accounts but is characterised in different ways. For Bicchieri (and Gintis), punishment triggers norm-following in a one-shot context, potentially via it being represented as altering players' payoffs. In contrast, in the repeated game framework, punishment is a constitutive part of the strategies themselves. A potential response to this difference is that the repeated game approach can be reinterpreted into the one-shot case by taking the expected payoffs of the repeated payoff 3] Something Gintis and others stress in their claims for a disposition in humans towards "strong reciprocity" (Gintis 2000). 4] See Binmore 1994, §3.2.5 for powerful arguments that TFT's importance as a repeated PD strategy has been overemphasised. Social Norms: Repeated Interactions, Punishment, and Context Dependence 8 stream as the payoffs of a new one-shot game. This approach is followed by Skyrms when he argues that the repeated PD can be rewritten as a one-shot stag hunt game (Skyrms 2004). However, this is a reframing of the repeated scenario rather than an integration of the two approaches. To reiterate, our concern is that in one case punishment features as a reason for action and in the other it is just one of the moves in the game. When attempting to apply an integrated model to actual cases of norm-following where social sanctions are present it would then be ambiguous in which of the two ways we should model those sanctions. At this point we are left with the question of whether one interpretation is to be preferred in general or in specific cases of norm-following or whether we should accept a plurality of modelling approaches. It is not our intention to adjudicate on this point here. What we stress is that such an adjudication (including agnosticism) would not be a move towards an integrated game theoretic account of social norms. Moreover, the following sections suggest that before any possible integration, both interpretations have to be completed in order to integrate the fact that norms are highly context-dependent. I V. CON TE XT DEPEN DENCE: TH E TR IGGER I NG A N D STR ATEGIC ROLES OF E X PECTATIONS Despite the striking variance in their analysis, as shown in the case of the nature and role of punishment, all accounts of social norms suffer from at least one common shortcoming, for related reasons. The basic problem is the following. A certain situation of interaction, formalised by a game (the sets of agents' actions and payoffs), may well trigger various social norms, depending on the background context of the interaction. When an interaction is repeated, should we say that the context changes so that a new social norm may appear, or that it does not since the base game describing each interaction is unchanged? In other words, when during a repeated interaction should we expect a social norm to be stable and when unstable? First, let us recall the characteristics of one-shot and repeated interaction approaches to social norms, respectively exemplified by Bicchieri's (2006) and Binmore's (1998, 2008). For Bicchieri, a social norm is cued in certain situations by the agents' beliefs about others' behaviour and expectations. As seen in section two, this triggers a change in preference that makes it rational for agents to choose certain actions. For Binmore, whether a norm is triggered depends on the similarity of the situation with a known one. Agents' actions are not explained by a preference change, but by their behaving as they are used to in a similar situation. In other words, when facing new situations in the laboratory, agents either conserve preferences or habits from their outside life (Woodward 2008). Do the starting game's characteristics constrain the existence of a social norm? Not really. Theorists usually reckon that cooperative norms can appear in any kind of cooperative dilemma, that is, of games containing an outcome that, although not an Jonathan Grose & Cedric Paternotte 9 equilibrium, Pareto-dominates an equilibrium.5 However, this expresses the theorists' interest for certain kind of games (those in which cooperative behaviour calls for an explanation) rather than a logical necessity or an empirical fact. Indeed, social norm theorists are disposed to see norms appear in almost any kind of game. Binmore does not discuss the issue, and we have seen in section two that Bicchieri holds that norms can appear in any 'mixed-motive games,' that is, games containing potential for mutual benefit and some conflict of interest; this description suits most games. Defenders of the competing 'group identity' explanation of cooperation are not any more precise. For instance, Bacharach (2006) only talks of situation of 'strong interdependence,' which is nothing else than the presence of a Pareto-dominated Nash equilibrium although he provides no explanation why this condition is paramount. Overall, virtually any game could cue a social norm. This makes context all the more important to social norms. The term "context" refers to anything that cannot be expressed by games' parameters. So the absence of constraints or payoff structures on the existence of social norms increases the importance of expectations or of the similarity with real-life situations in determining when social norms may appear.6 Of course, games still play a role in determining what social norms may appear. On this point though, somewhat unexpectedly, one-shot and repeated interaction approaches are not as separated as it seems. Expectations matter in the former, but even if they are not explicitly mentioned in the latter, their role is merely hidden. Suppose an agent is confronted with laboratory game A and deems it similar to reallife situation B, in which she would choose to do X. She will then do X in A for the same reason that she would have done so in B: because it is part of an equilibrium (she thinks mistakenly). But by definition, agents play according to an equilibrium if they believe that others will do the same. In a repeated game approach, expectations about others' behaviour and their expectations determine what an agent is going to do; the only difference is that they do not also lead to a preference change. Note that expectations also play this role in one-shot approaches, in addition to their triggering effect: once a game is transformed, expectations about others still influence a player's choice in the new game. In other words, expectations can have both a triggering role and a strategic role (this distinction will become important in the next section). One-shot and repeated interaction approaches to social norms both recognize the strategic role of expectations, but only the former also explicitly mentions their triggering role. A related reason why the two approaches are closer than it may seem is that the factors that cue norm-following behaviour are not necessarily different. In the one-shot interaction approach, certain expectations trigger preference change, but there is no 5] E.g. the Prisoner's Dilemma, in which the mutual cooperation outcome makes both players better off than the mutual defection one does, although only the latter is a Nash equilibrium. 6] An example of the effect of similarity between games on behaviour is found in Binmore 2010: when individuals had to play an Ultimatum game, "recognised" it as a familiar situation called "harambee" and then played accordingly. An example of the role of expectations can be found in Bicchieri 2008. Social Norms: Repeated Interactions, Punishment, and Context Dependence 10 constraint on the expectations that agents may or may not form. This reflects a widespread practice in rational choice theory: theorists determine what is the best, or rational choice for an agent given her preferences and beliefs, without asking whether those can be considered as rational or acceptable themselves. Now an agent's expectations about others may perfectly stem from the similarity of the laboratory game with a real-life situation: because A is similar to B, I may expect others to play according to B's equilibrium. So the role of game similarity could find a space even in one-shot approaches, as one possible origin of agents' expectations. In both approaches, similarity between games may cue a social norm and resulting expectations influence the agents' behaviour. What are the differences then? First, according to the one-shot interaction approach; other factors than similarity between games may trigger a social norm. However, the main difference lies in the link between the base game and the game that agents are actually playing. According to a repeated interaction approach, the laboratory game A is replaced by one representing the similar real-life situation B, in which players then play according to one of B's equilibria. According to a one-shot approach, the payoffs of laboratory game A are transformed into those of a game C. The difference is that there need not be a payoff transformation function that leads from A to B, and more precisely not one that corresponds to Bicchieri's description (2006, 52-54). In both cases, the characteristics of game A partly influence the game that players are really playing; only the way to determine the latter from the former varies. V. CON TE XT-DEPEN DENCE I N R EPE ATED G A M ES When agents interact only once, context-dependence is not deeply problematic, as it all depends on which expectations agents have or which real-life situations they deem similar to the one at hand. Surely, this makes social norm following behaviour hard to predict, as a theorist would need to know all possibly related real-life situations and all expectations linked to a social norm in a given population. Still, the analyses provided by both one-shot and repeated interaction accounts are clear. What happens to a social norm when the same game gets repeated? What makes agents keep sticking to it or start following another one along the way? On this point, the two accounts described above start to differ significantly, even if none of them provide a satisfactory answer. On the one hand, as the base game is just repeated and does not change, agents may well keep following the same norm. However, there are always multiple equilibria in a repeated game, and according to Binmore each of them could be a norm. There are also multiple ways in which the game's payoffs can be modified through preference change functions, and so just as many norms according to Bicchieri. As a game is repeated, agents observe each other's behaviour and consequently may see a change in their expectations; they may also start to understand the nature of the game they are playing and as a result adapt their behaviour to it. For these reasons, both one-shot and repeated interaction Jonathan Grose & Cedric Paternotte 11 accounts of social norms may predict that a social norm that agents follow changes as time passes. What does this change depend on? One way to put the problem is: when should observations of behaviour during repeated play lead to a change in the social norm an agent follows? It is a fact that most often, agents' behaviour varies with time when they play a repeated game. As may be expected, the repeated interaction account of social norms fits some of such cases well. For instance, the rate of cooperation in a repeated public good game tends to decrease with time (Camerer 2003, 59). In an Ultimatum game, offers can usually stabilize around one arbitrary value, although in the Dictator's game (when they cannot be refused) they get closer to zero. In these three cases, agents' strategies converge towards one of the game's Nash equilibria (no contribution in the public good game, any nonzero offer in the Ultimatum one, zero offer in the Dictator game). This is consistent with what the repeated interaction account predicts: when playing game A, agents may start to act as if they were playing game B, but as repetitions of the game accumulate, their understanding of the situation will improve and they will gradually learn to play according to A's equilibria. The effects of the context (that is, the existence of a similar real-life situation) can thus be offset by the success of an agent's strategy, in terms of its concrete payoff. Is this interpretation consistent with all experiments? It seems so. Consider Isaac and Walker's (1988) experiment (discussed by Bicchieri, 2006, 149ff.), consisting of two separated runs of ten successive public good games. Conversation between participants was allowed either only before the first sequence, only before the second sequence, or not at all. What was observed is although the level of contributions typically declines as the game is repeated, allowing conversation before a sequence led to higher, sometimes increasing contributions, and that the effect of a conversation before the first sequence carried over to the second one. The increasing effect is puzzling from a repeated interaction account, because in any equilibrium of a repeated social dilemma, there should be a decrease of the contribution level at least in the last round. If agents were slowly learning not to mistake the situation for a different one, their behaviour should converge towards such an equilibrium. It may be that the learning process needs longer than a handful of repetitions to kick in. However, such an argument threatens to render the explanation ad hoc: agents could be said to learn whenever they play according to an equilibrium (which are many in repeated games), and not to learn yet whenever they do not.7 Moreover, whenever agents do not play according to a finitely repeated game's equilibrium, such as when their contributions increase in the last period, one can always say that they are still behaving as if in real life, when it is hardly ever sure that an interaction will not be repeated some 7] Note that the carry over effect in itself need not be a problem for a repeated interaction account. There is a repeated game equilibrium in which agents make high contributions all the time, except towards the end. As long as the contribution level decreases at some point, it may stay high for a long time before, and there is no reason why this would not carry over between several sequences of games. Social Norms: Repeated Interactions, Punishment, and Context Dependence 12 time in the future. Any behaviour in the last repetition could thus be seen as part of an equilibrium of the infinitely repeated game. Even if players perfectly understand that the numbers of repetitions in an experiment is finite, they might be behaving partly intuitively, based on the similarity between laboratory and real-life situations. Put differently, to be satisfying the explanation should tell us when agents act strategically (by considering the payoffs and structure of the actual interaction) and when habitually. How does a one-shot interaction account fare? It actually faces a similar problem. Let us start with a difference: one-shot accounts based on preference transformations have no problem explaining that contribution levels in a social dilemma should not decrease in the last game. This is because the preferences of norm-following agents may be such that contributing zero is not an individually dominant strategy anymore (by contrast, without a preference change, payoffs are such that defecting is always individually beneficial). Still, it is just as difficult to say when agents should start following different social norms. The problem stems from the unknown balance between the triggering and strategic effect of expectations (as described in the previous section). When agents can have different preferences (or type) and are uncertain about others' types, a useful game-theoretic concept is that of a perfect Bayesian equilibrium (Osborne and Rubinstein 1994, 231-37). When applied to repeated games for instance, it says that agents start with a prior belief about everyone's possible type, which they will then update by Bayesian conditionalisation as they observe others' action. Observations constantly modify agents' beliefs about others' types and expectations about their behaviour. These changes of beliefs are part of the Bayesian Nash equilibrium: two strategies can only be at equilibrium if when agents implement them, the change of beliefs they cause is consistent with the strategies' prescriptions. This models the strategic role of expectations, that is, the way in which they help determine the behaviour agents who maximize their expected utility. Now recall that in Bicchieri's one-shot interaction account of social norms, expectations have a triggering role, that is, they can lead to a change in preferences. So a change in expectations may well cause an agent swapping types, and in particular can lead to the appearance of new types. This cannot be made part of Bayesian Nash equilibria, in which a list of possible types is set from the beginning and cannot evolve. The problem is that the theory can then explain virtually any observed behaviour: either agents follow a well-defined norm, in which case expectations play a triggering role at the start and then only a strategic role (preferences do not change as the game is repeated); or it does not and can be explained by the fact that the expectations changed over time and so triggered another norm. Overall, both kinds of accounts seem able to fit any data, thanks to the liberal definitions of context. Expectations can be part of the context; as they routinely change during any repeated interaction, they may trigger a change in norms at any time and thus allow one to explain any behaviour. Learning processes determine when the context's influence stops overcoming benefit-related considerations; but in the absence of a precise definition of such processes, the effect of context can also be used to explain any behaviour. Jonathan Grose & Cedric Paternotte 13 V I. CONCLUSION We have argued that despite surface-level similarities, game-theoretic accounts of social norms are not easily integrated. This is due to the existence of two main kinds of accounts, based on one-shot or on repeated interactions. This distinction gives rise to different treatments of the role played by punishment. Moreover, the problem is not merely to choose between them, as they both suffer similarly from difficulties to account for the context-dependence of social norms while conserving their explanatory power. jonathan.grose@bristol.ac.uk cedric.paternotte@lrz.uni-muenchen.de R EFER ENCES Bacharach, M. 2006. Beyond Individual Choice: Teams and Frames in Game Theory. Edited by N. Gold and R. Sugden. Princeton University Press. Bicchieri, C. 2006. The Grammar of Society The Nature and Dynamics of Social Norms. Cambridge: Cambridge University Press. ---. 2008. How Expectations Affect behavior: Fairness Preferences or Fairness Norms?. In Rationality and Responsibility, edited by J. Krueger. New York: Psychology Press. Binmore, K. 1994. Game Theory and the Social Contract. Volume 1: Playing Fair. M.I.T. Press. ---. 1998. Game Theory and the Social Contract. Volume 2: Just Playing. M.I.T. Press. ---. 2005. Natural Justice. Oxford University Press. ---. 2006. Why do people cooperate?. Politics, Philosophy and Economics 5 (1): 81-96. ---. 2008. Do conventions need to be common knowledge?. Topoi 27: 17-27. ---. 2010. Social norms or social preferences?. Mind & Society 9 (2): 139-57. Camerer, C. 2003. Behavioral Game Theory. Princeton University Press. Gintis, H. 2000. Strong reciprocity and human sociality. Journal of Theoretical Biology 206: 169-79. ---. 2010. Social Norms as Choreographer. Politics, Philosophy and Economics 9 (3): 251-64. Hausman, D. 2008. Fairness and Social Norms. Philosophy of Science 75: 850-60. Hobbes, T. 1991. Leviathan. Edited by R. Tuck. Cambridge: Cambridge University Press. Isaac, R., and J. Walker. 1988. Communication and Free-Riding Behaviour: The Voluntary Contribution Mechanism. Economic Inquiry 26: 585-608. Myerson, R. 1991. Game Theory, Analysis of Conflict. Harvard University Press. Osborne, M.J., and A. R. Rubinstein. 1991. A Course in Game Theory. M.I.T. Press. Paternotte, C., and J. Grose. 2012. Social Norms and Game Theory: Harmony or discord. British Journal for the Philosophy of Science. doi: 10.1093/bjps/axs024. Skyrms, B. 2004. The Stag Hunt and the Evolution of Social Structure. Cambridge: Cambridge University Press. ---. 1996. Evolution of the Social Contract. Cambridge: Cambridge University Press. Woodward, J. 2008. Social Preferences in Experimental Economics. Philosophy of Science 75 (5): 646-57. Young, H. P. 2008. Social Norms. In The New Palgrave Dictionary of Economics, edited by S. Durlauf, N. and L. Blume. Basingstoke: Palgrave Macmillan.


Non-Archimedean Preferences Over Countable Lotteries Jeffrey Sanford Russell March 2020 Abstract We prove a representation theorem for preference relations over countably infinite lotteries that satisfy a generalized form of the Independence axiom, without assuming Continuity. The representing space consists of lexicographically ordered transfinite sequences of bounded real numbers. This result is generalized to preference orders on abstract superconvex spaces. 1 Infinite Lotteries and Independence Let a (discrete) lottery over a set of outcomes X be a function λ ∶ X → [0, 1] such that ∑x∈X λ(x) = 1. Let ΔX be the set of all such lotteries. Let a preference order be a total preorder ≲ on ΔX. (That is, ≲ is assumed to be transitive, reflexive, and complete in the sense that for each pair of lotteries, λ ≲ μ or μ ≲ λ.) A standard axiom of rational preference is the Independence axiom (I) For any lotteries λ, λ′, μ ∈ ΔX and any probability 0 < p ≤ 1, λ < λ′ iff p ⋅ λ + (1 − p) ⋅ μ < p ⋅ λ′ + (1 − p) ⋅ μ Let an indexed family of lotteries be a function λ ∶ I → X, for any set I. For total preorders, Independence can be equivalently restated:1 Thanks to Yoaav Isaacs for many discussions of the ideas on which this paper is based, and to Tobias Fritz and the editors of the Journal of Mathematical Economics for comments. 1Hammond's (1998) label (D) for this principle stands for "dominance." But note that the principle is stronger than a natural analogous dominance principle for outcomes, since it applies generally to mixtures of lotteries. (See Buchak 2013, 162ff, for discussion of the normative import of this distinction.) 1 (D) For any finite set I of indices, any probability distribution p ∈ ΔI, and any indexed families of lotteries λ, μ ∶ I → X, if λi ≲ μi for each i ∈ I, then ∑ i p(i) ⋅ λi ≲ ∑ i p(i) ⋅ μi If furthermore λi < μi for some i such that p(i) > 0, this preference is strict. This naturally suggests a generalized principle, which simply drops the qualification "finite" (see Blackwell and Girshick 1954; Hammond 1998, sec. 8.4).2 (D*) For any set I of indices, any probability distribution p ∈ ΔI, and any families of lotteries λ, μ ∶ I → X, if λi ≲ μi for each i ∈ I, then ∑ i p(i) ⋅ λi ≲ ∑ i p(i) ⋅ μi If furthermore λi < μi for some i ∈ I such that p(i) > 0, this preference is strict. This very natural principle can be motivated by some of the same considerations as the original Independence principle and its close cousin, the Sure Thing Principle. For example, Savage (1954, sec. 2.7) writes: A businessman contemplates buying a certain piece of property. He considers the outcome of the next presidential election relevant to the attractiveness of the purchase. So, to clarify the matter for himself, he asks whether he would buy if he knew that the Republican candidate were going to win, and decides that he would do so. Similarly, he considers whether he would buy if he knew that the Democratic candidate were going to win, and again finds that he would do so. Seeing that he would buy in either event, he decides that he should buy, even though he does not know which event obtains ... [E]xcept possibly for the assumption of simple ordering, I know of no other extralogical principle governing decisions that finds such ready acceptance. The principle (D*) can be similarly motivated, by generalizing to questions with more than two possible answers. Let E1,E2, ... be countably many mutually exclusive and jointly exhaustive events. If you prefer to buy if E1 obtains, and you also 2Again, this label is Hammond's. Russell and Isaacs (forthcoming) call this principle "Countable Independence," to distinguish it more clearly from the weaker dominance principle concerning outcomes. The qualification "Countable" is included just because any discrete probability distribution p ∈ ΔI must have countable support. 2 prefer to buy if E2 obtains, and so on, then this seems like a strong reason to buy, unconditionally. Suppose that you assign probability λ(x) to the hypothesis that outcome x will result from buying (for each x ∈ X). Likewise, you assign conditional probability λi(x) to the hypothesis that outcome x will result from buying if event Ei obtains. By the law of total probability, λ = ∑i pi ⋅ λi, where pi is the probability you assign to event Ei. So if your conditional and unconditional preferences between acts accord with your preferences between these associated lotteries, (D*) follows from the countable generalization of Savage's principle.3 When the set of outcomes is infinite, (D*) genuinely strengthens the finitary condition (D), and the equivalent Independence axiom (I). There are preference orderings on ΔX satisfying Independence which are representable by unbounded real utilities (see Fine 2008). But no such orderings satisfy (D*). This arises from well-known cases like the St. Petersburg paradox and the two-envelope paradox (Menger 1934; see also Arntzenius and McCarthy 1997, 44; for a striking related result see Seidenfeld, Schervish, and Kadane 2009). One standard response to these puzzle cases is to impose the requirement that preferences can be represented by a bounded set of real numbers. But in fact, this is an overreaction. While (D*) is incompatible with preferences that are representable by unbounded real utilities, it is perfectly compatible with preferences that are not representable by bounded real utilities-because these preferences are not representable by real-valued utilities at all. A simple example is the lexicographic order on lotteries in Δ{0, 1, 2} with three possible outcomes. Preferences like these conflict with the standard Archimedean Axiom (or the Continuity axiom, which is equivalent in this context), which essentially rules out "infinite" or "infinitesimal" utilities. In the presence of the Archimedean Axiom, (D*) is indeed equivalent to representability by bounded real utilities (Blackwell and Girshick 1954; Hammond 1998, sec. 8.4 and 8.5). But while the combination of (D*) and the Archimedean Axiom is well-understood, it is also worthwhile to investigate the constraints imposed on preferences by (D*) in the general non-Archimedean case. This is for two reasons. First, while there are strong normative arguments for (D*), there are not such strong arguments in favor of the Archimedean Axiom; the latter may not be a genuine constraint on rational preference, but merely a technically convenient simplification (see for example Luce and Raiffa 1989, 27; Hájek and Nover 2008, 649ff). Second, this exploration helps us conceptually distinguish two very different kinds of "infinity problem" in decision theory. Some difficulties arise from unbounded utilities, like the St. Petersburg 3For a closely related "consequentialist" argument see Hammond (1998, sec. 8.4). For further discusssion see Russell and Isaacs (forthcoming, sec. 2). 3 puzzle and the two-envelope puzzle. Other difficulties arise from infinite utilities, like Pascal's wager (for discussion see Hájek 2003). If (D*) is a normative requirement on preferences, it turns out that this will spare rational agents the first kind of difficulty-but not the second.4 The principle (D*) imposes interesting structural constraints on preferences. In a sense we can make precise, this principle rules out unbounded finite utilities; but it is perfectly compatible with infinite utilities, and indeed, with infinitely many different "regimes" of infinite utilities. The structure of these "regimes" is highly constrained: in particular, (D*) rules out ascending chains of lotteries each of which is infinitely better than those before it. In other words, these utility "regimes" or dimensions have the structure of ordinals. Classic representation theorems for non-Archimedean preferences (such as Fishburn 1971) represent utilities as lexicographically ordered n-tuples of real numbers (u1, u2, ..., un), indexed by numbers up to some finite n.5 But not all preferences satisfying (D*) have utility representations given by finite sequences: in general, some preferences require infinite sequences. Moreover, they can require quite long infinite sequences. Some infinite sequences are indexed by just the finite numbers 1, 2, 3, .... Other sequences have further dimensions beyond those, and may be indexed by ordinal numbers 1, 2, ..., ω, ω + 1, ω + 2, ..., 2ω, 2ω + 1, ..., ω2, ω2 + 1, ... and so on, as far as you like.6 In principle, the length of the utility sequences we'll need is only limited by the cardinality of the set of outcomes X: if this is a large infinite set, then we may require very large infinite ordinal numbers. The key structural feature of the ordinals is that they are well-ordered. This tells us that for any two distinct sequences of real numbers indexed by ordinals, there is a first index at which they differ. Thus for any ordinal α, we can lexicographically order the sequences of real numbers indexed by the ordinals before α. 4Collapsing these issues might lead one to the natural but incorrect conclusion that the difficulties raised by the St. Petersburg game could be avoided by dropping the Archimedean Axiom, as suggested by Hájek and Nover (2008). 5Fishburn (1974) and Martínez-Legaz (1998) survey prior work on lexicographic utilities. Borie (2016) proves a utility representation theorem for non-Archimedean preferences without completeness. Fishburn's and Borie's theorems apply to finite mixture spaces, without considering constraints like (D*) on preference between infinite mixtures of lotteries. 6For an introduction to ordinals see for example Hrbacek and Jech (1999) or Potter (2004). Exactly how far the ordinals go is a deep question in the foundations of mathematics: it is equivalent to the question of what "large cardinals" there are, andmany different answers to this question are consistent with the standard axioms of set theory, ZFC. But these axioms do tell us that for any set whatsoever (of any cardinality) there are enough ordinals to index that set. 4 Here we will give a utility representation theorem for preference orders that satisfy (D*), without assuming the Archidemean Axiom: preference orders that satisfy (D*) are precisely those that can be represented by bounded lexicographic utilities. This representation theorem extends the classic results of Hausner (1952) andHausner and Wendel (1952) concerning non-Archimedean preference orders. Hausner and Wendel's results show that any preference order on lotteries that satisfies Independence is representable by a convex subset of a lexicographically ordered vector space. Our extension of their result shows that the infinitary generalization of Independence (D*) requires, first, that this lexicographically ordered vector space takes a special concrete form-namely, the Banach space of bounded sequences l∞(α) for some ordinal α-and second, that this convex subset is itself bounded (with respect to the ∞-norm). Restricting utilities to a bounded set of sequences blocks the construction of a St. Petersburg lottery. Nonetheless, this representation permits highly non-trivial "infinite utility" structure, since the space of utilites can have arbitrarily high dimension. 2 Representation Theorem In this section we will take lotteries to be concrete probability distributions. In Section 3 we will consider a more general approach where "lotteries" are elements of a space with an abstract countable mixture operation, called a superconvex space. As it turns out, this apparent additional generality does not actually add much: any ordering of such space is equivalent to an ordering on an ordinary lottery space. We begin with some basic definitions. Definition 1. (König 1986) Let V be a Banach space. A subset C ⊆ V is called σ-convex iff C is closed under countable mixtures, in the following sense: for each p ∈ Δω and sequence x ∈ Cω, the infinite sum ∑ i<ω pi ⋅ xi is well-defined and contained in C. We will also use the more compact notation Σ(p, x) for this mixture. If C ⊆ V and C′ ⊆ V are both σ-convex sets, a function f ∶ C → C′ is mixturepreserving iff for each p ∈ Δω and x ∈ Cω, f(Σ(p, x)) = ∑ i pi ⋅ f(xi) 5 The restriction of any continuous linearmap to a σ-convex set is mixture-preserving. In particular, this holds for any contraction (a linear map f ∶ V → V′ such that ‖f(v)‖V′ ≤ ‖v‖V; equivalently, f(BV) ⊆ BV′). Example 1. For any nonempty set X, the set ΔX of discrete lotteries with outcomes in X is a σ-convex subset of the Banach space l1(X) (consisting of all functions u ∶ X → R for which the 1-norm ‖u‖ = ∑x∈X |u(x)| is finite). Example 2. Let V be a Banach space. The closed unit ball in V BV = {x ∈ V | ‖x‖ ≤ 1} is a σ-convex subset of V. In particular, let α be an ordinal, and consider the Banach space l∞(α) of length α bounded ordinal sequences of real numbers, with the ∞-norm ‖x‖ = supi<α |xi|. The closed unit ball Bl∞(α), which consists of all α-sequences of numbers in the interval [−1, 1], is a σ-convex subset of l∞(α). Definition 2. (a) A preorder on a set X is a reflexive and transitive relation ≲ on X. We say x ∼ y iff x ≲ y and y ≲ x. We say x < y iff x ≲ y but not y ≲ x. A preorder is total iff for each x, y ∈ X, either x ≲ y or y ≲ x. A partial order is a preorder which is also antisymmetric (if x ∼ y then x = y). A total order is a partial order which is also total. (b) If (X, ≲) and (X′, ≲′) are preordered sets, a function f ∶ X → X′ is monotone iff for any x ≲ y in X, f(x) ≲′ f(y) in X′. It is strictly monotone iff the converse also holds, so for any x, y ∈ X, x ≲ y iff f(x) ≲′ f(y). (c) A preorder on a vector space V is linear iff vector addition andmultiplication by positive scalars are both monotone functions. That is, for any x, y, z ∈ V and a ∈ R+, if x ≲ y then x+z ≲ y+z and ax ≲ ay. A preordered/partially ordered/totally ordered vector space is a vector space equipped with a linear preorder/partial order/total order. (d) If V and V′ are preordered vector spaces, a function f ∶ V → V′ is positive iff for each x ∈ V such that x ≳ 0, f(x) ≳ 0, and strictly positive iff additionally for each x ∈ V such that x > 0, f(x) > 0. It is straightforward to check that a linear map between preordered vector spaces is (strictly) positive iff it is (strictly) monotone. Here is the main theorem we will prove: Theorem 1. For any set X, a total preorder on the set ΔX of discrete lotteries satisfies (D*) iff for some ordinal α there is a strictly monotone mixture-preserving function from ΔX to Bl∞(α) with its lexicographic order. 6 First, we will show that the order on lotteries can be represented in a totally ordered Banach space which a certain boundedness property. Second, we will show that we can define a strictly monotone contraction from any such ordered Banach space to l∞(α). We begin by extending the order on lotteries to the surrounding Banach space l1(X). Lemma 1. For any total preorder ≲ on ΔX that satisfies the Independence axiom (I), there is a unique linear total preorder ≲∗ on l1(X) such that (a) The embedding of (ΔX, ≲) in (l1(X), ≲∗) is strictly monotone; and (b) The function h ∶ l1(X) → R such that h(u) = ∑x∈X u(x) is positive. (In particular, for each lottery λ ∈ ΔX, we have h(λ) = 1 and thus λ >∗ 0.) Proof. First, for any nonzero u ∈ l1(X) such that h(u) = 0, there exist λ, μ ∈ ΔX such that u = ‖u‖ ⋅ (λ − μ). For x ∈ X let u+(x) = { u(x) if u(x) > 0 0 otherwise and u−(x) = { −u(x) if u(x) < 0 0 otherwise We have 0 = h(u) = ‖u+‖ − ‖u−‖ and ‖u‖ = ‖u+‖ + ‖u−‖. It follows that λ = 2u+/‖u‖ and μ = 2u−/‖u‖ are elements of ΔX such that u = ‖u‖(λ − μ). Next, we define the order on l1(X) lexicographically, first using the function h, and second using the order on lotteries. For u ∈ l1(X), let u ≳∗ 0 iff either of the following cases holds: (a) h(u) > 0, or (b) h(u) = 0 and u = ‖u‖ ⋅ (λ − μ) for some λ, μ ∈ ΔX such that λ ≳ μ. (In general, v ≳∗ u iff v − u ≳∗ 0.) It is straightforward to check that (I) implies that this order is linear and satisfies the two conditions of the lemma. For uniqueness, it suffices to check that, for any linear order ≲∗ on l1(X), and any nonzero u ∈ l1(X), if h(u) = 0 and u = ‖u‖(λ − μ), then λ ≳ μ iff u ≳∗ 0. Both directions follow from linearity. 7 In order to get to a totally ordered Banach space, the natural thing to do next is quotient out the set I of vectors which are equivalent to zero. The difficulty is that in general the quotient space may not retain the structure of a Banach space: this holds only if the subspace I is closed. It does turn out that our axiom (D*) guarantees this, but this takes a little bit of showing. Lemma 2. For any preordered Banach space V, let B+V = {x ∈ BV ∣ x > 0} If (ΔX, ≲) satisfies (D*), then B+l1(X) is σ-convex (with respect to the extended order ≲∗). Proof. Let p ∈ Δω and x1, x2, ... ∈ B+l1(X). Then h(xi) ≥ 0 for each i. If h(xi) > 0 for any i, then h(Σ(p, x)) = ∑ i pi ⋅ h(xi) > 0 so in that case Σ(p, x) is positive as well. Otherwise, h(xi) = 0 for every i ∈ ω. In that case, for each i there is some λi > μi in ΔX such that xi = ‖xi‖(λi − μi). Since ‖xi‖ ≤ 1, the series ∑i pi‖xi‖ converges to some value a ∈ R+. Then ∑ i pi ⋅ xi = a( ∑ i pi‖xi‖ a ⋅ λi − ∑ i pi‖xi‖ a ⋅ μi) By (D*) and the definition of ≲∗, this is positive. The following very simple fact really contains the core idea of the main theorem. The central "paradoxical" feature of the St. Petersburg lottery is that it is strictly preferred over every one of its outcomes. (D*) rules out this situation. Lemma 3. Let V be a preordered Banach space such that B+V is σ-convex. There is no p ∈ Δω and x1, x2, ... ∈ B+V such that xi < Σ(p, x) for each i ∈ ω Proof. If there were such p and x1, x2, ..., then the vectors 12 (Σ(p, x) − xi) would each be in B+V. This would imply that B+V also contains the mixture ∑ i pi ⋅ ( 1 2(Σ(p, x) − xi) = 1 2(Σ(p, x) − ∑i pi ⋅ xi) = 0 8 And this is false. This means that (D*) blocks the construction of St. Petersburg lotteries. There are two different versions of this. One kind of "improper mixture" arises from unbounded relatively finite utilities, and another arises from infinite ascending chains of relatively infinite utilities. Both kinds are captured by the following condition. Lemma 4. Let V be a preordered Banach space such that B+V is σ-convex. Then there is no supergeometric sequence x1, x2, ... ∈ B+V such that 2xi ≲ xi+1 for each i ∈ ω. Proof. This is the St. Petersburg paradox. Suppose x1, x2, ... ∈ B+V is a supergeometric sequence, and let w = ∑ i 2−i ⋅ xi We will prove that this is an improper mixture. Consider the truncated sum wn = ∑ 0<i≤2n+n 2−ixi ≳ ∑ n<i≤2n+n 2−ixi = 2−n ∑ 0<j≤2n 2−jxn+j ≳ 2−n ∑ 0<j≤2n xn = xn using linearity together with the facts that each xi is positive and xn ≲ 2−jxn+j for each j. Furthermore, w is the sum of wn and a mixture of the remaining terms xi, each of which are positive. Let N = 2n + n. Then: w = wn + ∑ i>N 2−i ⋅ xi = wn + 2−N ∑ i 2−i ⋅ xN+i > wn The last step uses our assumption that B+V is σ-convex, which implies that the mixture of elements of B+V is positive. Here is another way of putting this consequence. 9 Definition 3. A preordered vector space V is nice iff every norm-bounded subset of V is order-bounded. Lemma 5. If V is a totally preordered Banach space and B+V is σ-convex, then every subspace of V is nice. Proof. Let U be a subspace of V. It suffices to show that B+U has an order bound in U. If not, then for each u ∈ U there is some x ∈ B+U such that 2u < x. So in particular there is a supergeometric sequence x1, x2, ... in B+U (and thus in B+V) such that 2xi < xi+1 for each i. Now we will introduce a few useful concepts for working with non-Archimedean ordered vector spaces. (See Paulsen and Tomforde 2007; Emelyanov 2014.) Definition 4. For a preordered vector space V, define I(V) = {x ∈ V ∣ x ∼ 0} N(V) = {x ∈ V ∣ for some v ∈ V, for every a ∈ R+, −av ≲ x ≲ av} Intuitively, N(V) is the set of vectors which are infinitesimally small in comparision to some other vector. V is Archimedean iff N(V) = I(V). V is trivial iff I(V) = V. For any v ∈ V, define F(v) = {x ∈ V ∣ for some a ∈ R+, −av ≲ x ≲ av } Intuitively, F(v) is the set of vectors which are finite or infinitesimal in comparison to v. A vector e ∈ V is an order unit iff F(e) = V. The following facts are straightforward. Lemma 6. (a) For each v ∈ V, I(V), N(V), and F(v) are subspaces of V. (b) V is partially ordered iff I(V) = {0}. (c) If e is an order unit for V and e ∈ N(V), then V is trivial. (d) If V is a Banach space and e is an order bound for B+V, then e is an order unit for V. Lemma 7. If V is a totally preordered Banach space and B+V is σ-convex, then I(V) is closed. 10 Proof. Suppose v ∈ V is not in I(V): that is, v ≁ 0. We will show that v is not in the closure of I(V): that is, there is some ε > 0 such that for each u ∈ I(V), ‖v − u‖ ≥ ε. Without loss of generality, assume v ≴ 0. (Otherwise we can argue similarly regarding −v.) Consider the subspace F(v). By Lemma 5, this subspace contains an order bound for BF(v). This means there is some a ∈ R+ such that x ≲ av for each x ∈ BF(v). Now suppose u ∈ I(V): that is, u ∼ 0. Then (v − u) ∼ v, which implies that v − u ∈ F(v), and (v − u)/‖v − u‖ is in BF(v). Thus: v ‖v − u‖ ∼ v − u ‖v − u‖ ≲ av Thus (1 − a‖v − u‖)v ≲ 0 By assumption v ≴ 0, so we must have ‖v − u‖ ≥ 1/a. Lemma8. For any preordered vector spaceV, there is a strictly positive contraction from V onto some partially ordered vector space iff I(V) is closed. Proof. It is a standard fact that a subset of V is a closed subspace iff it is the kernel of a continuous linear surjection, and in particular a contraction f ∶ V → V′ (see for example Conway 1994, 70). We can define a partial order ≤′ on V′ with respect to which f is strictly positive: let 0 ≤′ f(x) iff 0 ≲′ x. The fact that ker f = I(V) implies that this is a well-defined partial order. (Compare Emelyanov 2014, sec. 1) Putting this much together completes the first part. Proposition 1. If ≲ is a total preorder on ΔX that satisfies (D*), then there is a strictly monotone mixture-preserving function from ΔX to B+V for some totally ordered Banach space V such that B+V is σ-convex. For the second part, we will prove a representation theorem for totally ordered Banach spaces of this type. The main idea of this part is that we can quotient out the infinitesimals from an ordered vector space. In the case of a non-trivial totally ordered nice vector space, the resulting quotient space is in fact isomorphic to R. Thus if every subspace is nice, we can keep descending to smaller and smaller infinitesimal scales, representing each regime by real numbers, until we run out. To prove this, we will appeal to the following fact, which is a variant of the HahnBanach Theorem for ordered vector spaces. 11 Lemma 9. (Paulsen and Tomforde 2007, Corollary 2.15) Let (V, ≤) be a partially ordered vector space with order unit e. If E is a subspace of V containing e, then any positive linear functional f ∶ E → R may be extended to a positive linear functional g ∶ V → R whose restriction to E is f. Corollary 1. If (V, ≤) is a totally ordered nice vector space, then there is a positive contraction f ∶ V → R such that ker f = N(V). Proof. Let e be an upper bound for BV, and thus an order unit in V. Assume without loss of generality that ‖e‖ ≥ 1. Let E = {ae ∣ a ∈ R}, and let f(ae) = a for each element ae ∈ E. Then let g be a positive linear extension of f to V. To show that g is a contraction, it suffices to show that for each x ∈ BV, |g(x)| ≤ 1. This is clear, since in that case −e ≲ x ≲ e, g is positive, and g(e) = 1. To show ker g = N(V), first suppose x ∈ N(V). It follows that −ae ≲ x ≲ ae for every a ∈ R+ Thus since g is positive and extends f, −a ≤ g(x) ≤ a for every a ∈ R+ and thus g(x) = 0. Conversely, if x ∉ N(V), then (since V is totally ordered) for some a ∈ R+, x ≳ ae or x ≲ −ae. So either g(x) ≥ a or g(x) ≤ −a, and in either case g(x) ≠ 0. Proposition 2. Let (V, ≤) be a totally ordered Banach space such that B+V is σconvex. Then for some ordinal α, there is a contraction from V to l∞(α) which is strictly positive with respect to the lexicographic order on l∞(α). Proof. The idea is to repeatedly apply Corollary 1, descending to smaller and smaller infinitesimal subspaces until we run out. First we define a descending chain of subspaces of V by transfinite recursion, as follows. V0 = V Vi+1 = N(Vi) Vλ = ⋂ i<λ Vi In other words, Vi consists of the "level i infinitesimals (or smaller)." This sequence is strictly decreasing as long as Vi is non-trivial: each subspace Vi is nice, which implies 12 that it has an order unit, which implies that N(Vi) ⊊ Vi unless Vi is trivial. So there must be some least ordinal α such that Vα is trivial. We'll consider the sequence of subspaces (Vi)i<α . For each non-zero vector x ∈ V, there is some i < α such that x ∈ Vi ∖ Vi+1. Intuitively, this is the "scale" of x: the largest utility regime to which xmakes a noninfinitesimal contribution. (We know x ∉ Vα , since x > 0 and Vα is trivial. So there is some i < α such that x ∉ Vi+1. So there is some least ordinal i such that x ∉ Vi+1. If i is a successor ordinal, this immediately implies x ∈ Vi. If i is a limit ordinal, then Vi is the intersection of all previous successor stages, each of which contains x, and so again x ∈ Vi.) Next, we can define each component of the function f ∶ V → l∞(α). Let i < α. By Corollary 1, there is a positive contraction g ∶ Vi → R such that ker g = N(Vi) = Vi+1. By the Hahn-Banach Extension Theorem (see for example Conway 1994, Theorem 6.2, p. 78), g can be extended to a contraction fi ∶ V → R defined on the whole space. Note that this extension fi is not generally positive, but we do have fi(x) > 0 for each x ∈ Vi ∖ Vi+1 such that x > 0 (In this case we know fi(x) = g(x) ≥ 0 because g is positive, and fi(x) ≠ 0 because x ∉ ker g.) The resulting linear map f ∶ V → l∞(α) with these components is a contraction: by definition, ‖f(x)‖∞ = supi |fi(x)|, and by construction |fi(x)| ≤ ‖x‖ for each i. To show that f is strictly positive, consider any x ∈ V such that x > 0. Let i be the "scale" of x such that x ∈ Vi ⧵ Vi+1. Since x ∈ Vi and the sequence is nested, for all j < i we have x ∈ Vj+1 = ker fj, which means fj(x) = 0. Since x ∈ Vi ∖ Vi+1 and x > 0, we also have fi(x) > 0. Thus f(x) > 0 according to the lexicographic ordering on l∞(α). This essentially completes the proof of one direction Theorem 1. For the other direction we just need a couple additional facts, which are straightforward to check. Lemma 10. Let C and C′ be σ-convex sets. If (C′, ≲′) satisfies (D*) and there is a strictly monotone mixture-preserving function f ∶ C → C′, then (C, ≲) satisfies (D*) as well. Lemma 11. The lexicographic order on Bl∞(α) satisfies (D*). 13 Proof of Theorem 1. One direction follows from Lemma 1, Proposition 1, and Proposition 2. The converse follows from Lemma 10 and Lemma 11. 3 Superconvex Spaces So far we have considered the concrete case of preference orders over lottery spaces: the space of all discrete probability distributions over a given set of outcomes. But we can alternatively consider a more abstract notion of a discrete mixture space in which the "lotteries" are not given as probability distributions, but rather are just some objects equipped with an abstract "discrete mixture" operation. This kind of structure is called a superconvex space (or superconvex module) (König 1986; Pümplun 1991; Nörtemann 2002). In general, there are superconvex spaces which are not isomorphic to σ-convex subsets of Banach spaces. Even so, it turns out that any ordering of a general superconvex space is equivalent, in a natural sense, to some ordering of a lottery space (Lemma 12). So we can still apply the same representation theorem even in this more general setting. For any set X, let δ ∶ X → ΔX be the function that takes each x ∈ X to the lottery that assigns probability one to x. Definition 5. A superconvex space is a set S together with an operation Σ ∶ Δω × Sω → S with the following properties: (a) For any sequence of elements x ∈ Sω and any index i ∈ ω, Σ(δ(i), x) = xi (b) For any probability distribution p ∈ Δω, any sequence of probability distributions q ∈ (Δω)ω, and any sequence of elements x ∈ Sω, ∑ i p(i) ⋅ ( ∑ j qi(j) ⋅ xj) = ∑ j ( ∑ i p(i) ⋅ qi(j)) ⋅ xj Principle (a) says that putting all of the weight on the ith element just gives you that same element xi. Principle (b) corresponds to simplification of compound lotteries: a mixture of mixtures of elements of S can be "flattened" to an equivalent single-level mixture. 14 Here is an alternative way of describing superconvex structure, which is a bit more elegant, since it eliminates the arbitrary indices. Instead of the operation Σ which takes an indexed sequence of elements and a distribution over the set of indices, we can define an operation σ ∶ ΔS → S which takes a distribution directly over the elements of the superconvex space, and mixes those elements together with the appropriate weights. Intuitively, an element of ΔS plays the role of a (discrete) random variable taking values in S, and σ plays the role of an expectation operator. For any function f ∶ X → Y, let Δf ∶ ΔX → ΔY be the obvious "lifted" function that preserves mixtures. (That is, for any lottery λ ∈ ΔX, the lottery Δf(λ) ∈ ΔY assigns to y ∈ Y the total probability that λ assigns to f−1(y).) Proposition 3. Let S be a superconvex space. For any distribution λ ∈ ΔS, there is a countable sequence x ∈ Sω such that λ = ∑i λ(xi) ⋅ δ(xi). Let σ ∶ ΔS → S be σ(λ) = ∑ i λ(xi) ⋅ xi This is well-defined (it does not depend on the choice of sequence x), and we have the two identities σ(δ(x)) = x for each x ∈ S σ(σ(κ)) = σ(Δσ(κ)) for each κ ∈ ΔΔS More succinctly: σ δ = 1S σ σΔS = σ Δσ (Here juxtaposition means function composition, 1S is the identity function on S, and σΔS ∶ ΔΔS → ΔS is the mixture operation on ΔS.) We can sum this up abstractly using resources from category theory (see Pümplun 1991): superconvex spaces are the Eilenberg-Moore algebras of the "lotterymonad" Δ.7 Any σ-convex subset of a Banach space is a superconvex space. In general, there are other kinds of superconvex spaces as well. For example, any countably complete semi-lattice is a superconvex space, with the operation Σ(p, x) = ⋁ i∶p(i)>0 xi 7This "lottery monad" consists of the functor that takes each set X to the set of distributions ΔX and each function f ∶ X → Y to the mixture-preserving function Δf, together with the two natural transformations δ ∶ 1 → Δ and σΔ ∶ Δ2 → Δ. (Compare the "Giry monad" of Giry 1982.) An Eilenberg-Moore algebra of this monad is precisely a pair of a set S and an operation σ ∶ ΔS → S satsifying the two conditions in Proposition 3. 15 (Consider, for example, a bounded set of integers with the maximum operation. This is clearly not a σ-convex subset of any Banach space.) There are also hybrid spaces that combine "lattice-like" and "vector-like" behavior (for interesting examples see Fritz 2015). Even so, we will show that any preorder on one of these general spaces can be represented perfectly well by some preorder on a lottery space. We can straightforwardly extend the idea of a mixture-preserving function to abstract superconvex spaces. (Pümplun (1991) calls such functions superaffine.) Here we require a slightly more general notion, which preserves mixtures modulo indifference. Definition 6. If S and S′ are preordered superconvex spaces, a function f ∶ S → S′ is essentially mixture-preserving iff for each p ∈ Δω and x ∈ Sω, f(Σ(p, x)) ∼ ∑ i pi ⋅ f(xi) Equivalently, f(σ(λ)) ∼ σ(Δf(λ)) for each λ ∈ ΔS Or more succinctly, fσ ∼ σ Δf (where for g, h ∶ X → S we write g ∼ h iff gx ∼ hx for each x ∈ X). Lemma 12. For any superconvex space S and any preorder ≲ on S, there is a unique preorder ≲∗ on ΔS with respect to which the function δ ∶ S → ΔS is strictly monotone and essentially mixture-preserving. Proof. For any λ, μ ∈ ΔS, define λ ≲∗ μ iff σ(λ) ≲ σ(μ) For x, y ∈ S, δ(x) ≲∗ δ(y) iff x = σ(δ(x)) ≲ σ(δ(y)) = y So δ is strictly monotone. For δ to be essentially mixture-preserving requires δσ ∼∗ σΔS Δδ. So it suffices to show: σδσ = σ σΔS Δδ Using the identities σδ = 1S and σ σΔS = σ Δσ, and the functor identities Δ(fg) = (Δf)(Δg) and Δ1S = 1ΔS, we have σδσ = σ = σ Δ(σδ) = σ Δσ Δδ = σ σΔS Δδ 16 For uniqueness, note first that in general σΔX Δδ = 1ΔX, which can be straightforwardly checked.8 Therefore, since δ is essentially mixture-preserving, 1ΔS = σΔS Δδ ∼∗ δ σ Thus, if δ is also strictly monotone, for any λ, μ ∈ ΔS we have λ ≲∗ μ iff δσ(λ) ≲∗ δσ(μ) iff σ(λ) ≲ σ(μ) Lemma 13. If (S, ≲) satisfies (D*), then so does (ΔS, ≲∗). Proof. Suppose λi ≲∗ μi for each i ∈ ω; so σλi ≲ σμi. The operation σ ∶ ΔS → S is itself mixture-preserving (this is what the identity σσΔS = σ Δσ tells us). So for any p ∈ Δω we have σΣ(p, λ) = ∑ i pi ⋅ σλi ≲ ∑ i pi ⋅ σμi = σΣ(p, μ) And thus Σ(p, λ) ≲∗ Σ(p, μ). The case where we also have strictly λi < μi for some i such that pi > 0 goes the same way. We can now put this fact together with the representation theorem for lottery spaces, to apply that theorem to arbitrary superconvex spaces. Corollary 2. A totally preordered superconvex space (S, ≲) satisfies (D*) iff there is a strictly monotonemixture-preserving function from S to Bl∞(σ), for some ordinal α. References Arntzenius, Frank, and David McCarthy. 1997. "The Two Envelope Paradox and Infinite Expectations." Analysis 57 (1). Oxford University Press: 42–50. Blackwell, David, and M. A. Girshick. 1954. Theory of Games and Statistical Decisions. NewYork: JohnWiley & Sons, Inc. https://onlinelibrary.wiley.com/doi/ abs/10.1002/nav.3800010313. 8This is one of the monad identity laws, since σΔ ∶ Δ2 → Δ is the monadic multiplication operation. 17 Borie, D. 2016. "Lexicographic Expected Utility Without Completeness." Theory and Decision 81 (2): 167–76. https://doi.org/10.1007/s11238-015-9523-y. Buchak, Lara. 2013. Risk and Rationality. Oxford University Press. Conway, John B. 1994. A Course in Functional Analysis. 2nd edition. New York: Springer. Emelyanov, Eduard Yu. 2014. "Archimedeanization of Ordered Vector Spaces." arXiv:1406.3657 [Math], June. http://arxiv.org/abs/1406.3657. Fine, Terrence L. 2008. "Evaluating the Pasadena, Altadena, and St Petersburg Gambles." Mind 117 (467). Oxford University Press: 613–32. Fishburn, Peter C. 1971. "A Study of Lexicographic Expected Utility." Management Science 17 (11): 672–78. https://doi.org/https:// doi.org/10.1287/mnsc.17.11.672. Fishburn, Peter C. 1974. "Lexicographic Orders, Utilities and Decision Rules: A Survey." Management Science 20 (11): 1442–71. https://www.jstor.org/ stable/2629975. Fritz, Tobias. 2015. "Convex Spaces I: Definition and Examples." arXiv:0903.5522 [Math], October. http://arxiv.org/abs/0903.5522. Giry, Michèle. 1982. "A Categorical Approach to Probability Theory." In Categorical Aspects of Topology and Analysis, edited by B. Banaschewski, 68–85. Lecture Notes in Mathematics. Berlin, Heidelberg: Springer. https://doi.org/10. 1007/BFb0092872. Hammond, Peter J. 1998. "Objective Expected Utility: A Consequentialist Perspective." In Handbook of Utility Theory, edited by Peter J. Hammond, Salvador Barberà, and Christian Seidl. Vol. I. Kluwer Academic. Hausner, M. 1952. "Multidimensional Utilities." Rand Corp, August. Hausner, M., and J. G. Wendel. 1952. "Ordered Vector Spaces." Proceedings of the American Mathematical Society 3 (6): 977–82. https://doi.org/10.2307/ 2031746. Hájek, Alan. 2003. "Waging War on Pascal's Wager." Philosophical Review 112 (1). Duke University Press: 27–56. Hájek, Alan, and Harris Nover. 2008. "Complex Expectations." Mind 117 (467). Oxford University Press: 643–64. Hrbacek, Karel, and Thomas Jech. 1999. Introduction to Set Theory, Revised and Expanded. 3 edition. New York: Marcel Dekker. 18 König, Heinz. 1986. "Theory and Applications of Superconvex Spaces." In NorthHolland Mathematics Studies, 122:79–118. Elsevier. https://doi.org/10.1016/ S0304-0208(08)71950-9. Luce, R. Duncan, and Howard Raiffa. 1989. Games and Decisions: Introduction and Critical Survey. Revised ed. edition. New York: Dover Publications. Martínez-Legaz, Juan E. 1998. "Lexicographic Utility and Orderings." In Handbook of Utility Theory, edited by Salvador Barberå, Peter J. Hammond, and Christian Seidl, 1: Principles:345–69. Dordrecht: Kluwer Academic Publishers. Menger, Karl. 1934. "Das Unsicherheitsmoment in der Wertlehre." Zeitschrift für Nationalökonomie 5 (4): 459–85. https://doi.org/10.1007/BF01311578. Nörtemann, Stefan. 2002. "The Hahn–Banach Theorem for Partially Ordered Totally Convex, Positively Convex and Superconvex Modules." Applied Categorical Structures 10 (4): 417–29. https://doi.org/10.1023/A:1016390813177. Paulsen, Vern, and Mark Tomforde. 2007. "Vector Spaces with an Order Unit." arXiv:0712.2613 [Math], December. http://arxiv.org/abs/0712.2613. Potter, Michael. 2004. Set Theory and Its Philosophy: A Critical Introduction. Oxford University Press. Pümplun, Dieter. 1991. "Banach Spaces and SuperconvexModules." InMathematics and Theoretical Physics, edited by Minaketan Behara, Rudolf Fritsch, and Rubens G. Lintz, 323–38. Walter de Gruyter. Russell, Jeffrey Sanford, and Yoaav Isaacs. Forthcoming. "Infinite Prospects." Philosophy and Phenomenological Research. Savage, Leonard J. 1954. The Foundations of Statistics. Wiley Publications in Statistics. Seidenfeld, Teddy, Mark J. Schervish, and Joseph B. Kadane. 2009. "Preference for Equivalent Random Variables: A Price for Unbounded Utilities." Journal of Mathematical Economics 45. Research Showcase @ Cmu: 329–40.