PRINCETON UNIVERSITY

CENTER OF INTERNATIONAL STUDIES

STAFF

Frederick S. Dunn, Director

Klaus Knorr, Associate Director

Faculty Associates

Gabriel A. Almond
Cyril E. Black
Gordon A. Craig
William Ebenstein
Edgar S. Furniss, Jr.
William W. Lockwood
Wilbert E. Moore

Research Associates

Arthur Lee Burns
Robert J. C. Butow
Miriam Camps
Bernard C. Cohen
Herman Kahn (2nd term)
Morton A. Kaplan
Dankwart A. Rustow
Glenn H. Snyder

Research Assistant

Sidney Verba

Editorial Assistant

Jean MacLachlan

Recent Policy Memoranda

No. 13. *Some Perspectives on American Military Assistance*, by Edgar S. Furniss, Jr.

No. 14. *Is the American Defense Effort Enough?* by Klaus Knorr

No. 15. *The European Common Market and Free Trade Area: A Progress Report*, by Miriam Camps

No. 16. *The State Department and the Foreign Service: The Wriston Report—Four Years Later*, by Zara S. Steiner

No. 17. *The First Year of the European Economic Community*, by Miriam Camps

Recent Research Monograph

No. 1. *Deterrence by Denial and Punishment*, by Glenn H. Snyder

Copies of the above memoranda and monograph can be obtained from the Center of International Studies, Woodrow Wilson Hall, Princeton, N. J., at 50 cents per copy.

# INTRODUCTION

There are some aspects of international politics that are essentially strategic in nature and that can be properly understood only if at least part of the understanding rests on strategic analysis. Unfortunately, the science of strategy, whether based on statistical or on game analysis, is not yet developed to the degree that models fully appropriate to the subject matter of international politics are available. We do not know at the present time whether this limitation is inherent or a merely temporary impediment. Yet if we accept the Aristotelian dictum that the mode of analysis ought to be appropriate to the matter subject to analysis, we can hardly turn elsewhere than to the theory of strategy for the analysis of matters that are essentially strategic.

Some of the difficulties with which the analyst of international politics must contend in the effort to subject his discipline to strategic analysis are formidable and fundamental. They extend to the interpretation of the numbers in the payoff boxes and, in the important bargaining and cooperative games, to the processes by means of which opponents can coordinate their strategies to achieve desirable outcomes in situations where, in the absence of coordination, the outcomes would be undesirable.

The two essays in this memorandum represent efforts to deal with or to explore a few of these fundamental difficulties. These difficulties cannot be evaded by common sense analysis; indeed, such analysis may obscure the nature of the difficulties. On the other hand, the direct application of models that are not fully appropriate could have even worse consequences than the application of common sense. These essays therefore have the modest purpose of contributing to an understanding of the strategic problem rather than of permitting the direct derivation of strategic solutions from mathematical formulas. If they are accepted in this limited way, it is my hope that they will aid in the penetration of a few of the dark areas that the mathematicians have left still largely unexplored.

-2-

# I. SOME IMPORTANT PROBLEMS IN NON-COOPERATIVE GAMES

Some important findings in regard to the non-cooperative game which are quite relevant to international politics are contained in the work of Nash, Luce and Raiffa, and Schelling. Technically, a non-cooperative game is a non-zero-sum game in which there is no pre-play communication.¹ As Schelling points out, there may in fact be tacit cooperation and therefore the label may be a trifle ambiguous. This sort of game has sufficient similarity to some real, although perhaps restricted, problems of politics to be worth attention.

The non-cooperative game can be employed to explore some of the most interesting problems of coordination. This factor of coordination, or tacit cooperation, to which Schelling calls our attention introduces many of the problems of the cooperative game. It turns out, however, that the context of the game is crucial for the interpretation of the game, and that games of this kind introduce difficulties in the use of utiles for payoffs or even indicate, under certain circumstances, at least some senses in which the use of utiles in the payoff boxes of the matrices of the game is illegitimate.

## The Zero-Sum Game

The non-cooperative game must be contrasted first with the competitive or zero-sum game. A zero-sum game is a game in which any change in the fortune of one player must be compensated for by an opposite and equal change in the fortune(s) of the other player(s). A zero-sum game can be represented by a matrix with a single number in each payoff box, as in Figure 1.

Figure 1

1. Duncan Luce and Howard Raiffa, *Games and Decisions*, New York, John Wiley & Sons, 1957; T. C. Schelling, "The Reciprocal Fear of Surprise Attack," RAND P-1342, 16 April 1958; idem, "For the Abandonment of Symmetry in the Theory of Cooperative Games," RAND P-1386, 29 May 1958; idem, "Re-interpretation of the Solution Concept for 'Non-Cooperative' Games," RAND P-1385, 2 June 1958.

-3-

Thus in this game Player A, by playing strategy 2, can ensure winnings of at least 3; and Player B, by playing his strategy 2, can hold his losses to a maximum of 3. These zero-sum games can be included within a general theory. The players use the minimax criterion, either by choosing a pure strategy in games with saddlepoints, like Figure 1, or by choosing a mixed strategy in games without saddlepoints.

Even in the zero-sum case, the use of utiles in the payoff boxes raises difficulties. In the single-shot game in which life--or some other very important consequence--is at issue, the substitution of utiles for raw outcomes has definite advantages. Yet, as the following examples will indicate, even this use can be pushed too far. Moreover, it may be important to use behavioral data in interpreting the game and this may introduce still further complications.

Consider the matrix represented by Figure 2. The payoffs

Figure 2

are given in terms of raw commodities. Consider the game as a zero-sum game in which a hostile opponent wants to kill player A and in which A must obtain at least 3.64 units of the commodity in order to survive. The minimax criterion calls for Player A to employ seven uses of his first strategy to four uses of his second strategy. The game has a value of 3.64 for A. The actual payoffs will be 4, 0, 3, or 10--depending on the strategies chosen--and not 3.64, which is the expected payoff. If Player A gets 4 or 10, he lives; if he gets 0 or 3, he dies. In effect, the unit-for-unit translation into utilities gives him a .364 probability of survival. If, on the other hand, he treats 10 and 4 as worth 1 and 3, and 0 as worth 0, his proper mixture is to use each of his strategies with .5 probability. This gives him a .5 probability of living--the best possible result he can obtain, given the nature of the game. If, however, Player A can play in a very large number of similar games and requires an average payoff of 3.64 units of the commodity, he must play the game in terms of the commodities and use the (7, 4) mixture, since there is no other way in which he can equal or improve this expected payoff.

-4-

Consider the game represented in Figure 3. Assume that utilities are linear with dollars, so that an immediate transformation may be made. No

Figure 3

Figure 4

saddlepoint exists in this zero-sum game. The proper mixed strategy of nine uses of A's second strategy to one use of his first strategy gives an expected minimum value of $19. Now suppose that a $100 will permit Player A to buy something he wants so much that the game is best represented by the matrix in Figure 4. This gives rise to a new mixed strategy consisting of 19 uses of the second strategy to one use of the first. Although this gives a minimum expected value to the game that is slightly higher (just under $20), it reduces by nearly one-half the possibility of getting the $100 payoff. Thus the more important a payoff becomes, the less likely it is that the proper mixed strategy will produce it.

Several things should be noted. The matrix representation is somewhat unfair, since the game is really not a zero-sum game if the players have a differential utility for money. Nonetheless, one only transfers an advantage to an opponent if he plays the game according to utiles rather than money. Thus games like poker are genuinely zero-sum games, although they would not be if the strict game formulation of the matrix were followed--that is, if utiles were introduced.²

2. In the pure zero-sum game, the minimax criterion is convincing. This is particularly true in games with saddlepoints, since in these games at least one of the players must have a dominant strategy. Once behavioral information is permitted, the minimax criterion is not obviously the best. Consider the matrix in Figure 5. In a single-shot game, Player A may very well prefer his second strategy if he has any reason to believe that

Figure 5

-5-

These examples give an insight into the utility axioms themselves. One reason why some players refuse gambles that are mathematically desirable stems from their satisfaction with their standard of life and their fear of wagering when they do not know that they can repeat the wager indefinitely. And will not the player who finds life as presently constituted not worth living be willing to wager against seemingly foolish odds in an attempt to better himself? This indicates that the relationship of the wager to the total resources of the player and to the opportunity for repeated wagers are important considerations in determining whether and how the raw payoffs should be translated into utiles.

(footnote 2 continued) Player B is not very intelligent or well-informed. Indeed, even if he knows that Player B is perfectly informed and rational, Player A may use his second strategy as a pure strategy so long as he has reason to believe that Player B cannot know he has made this choice. In a game like that in Figure 5, Player A may prefer to maximize his chance of getting $100 rather than to minimax.

In any event, in real life behavioral data cannot be ignored. Indeed, when the matrix is visualized, some logical possibilities are omitted either because they do not interest us or because we consider them so unlikely that they are not worth consideration. The representation of the game matrix depends upon the information available to the players. If behavioral information is permitted, it is possible even in the zero-sum game to devise strategies better than the minimax strategy which take advantage of certain kinds of behavioral information. It is nonetheless true that non-consideration of behavioral data simplified the development of the theory of the zero-sum game in a useful way. It is at least questionable whether similar considerations prevail with respect to a general treatment of the non-zero-sum game.

[LC]

-6-

## The Non-cooperative Game

The difficulty with the non-zero-sum game is that no general theory of the game exists. It is possible nonetheless to analyze specific kinds of non-zero-sum games, although as yet we possess no really good model for most aspects of the game of international politics. Some very interesting and even fascinating problems arise in the non-cooperative, non-zero-sum game and consideration of these special cases can prove instructive. Consider, for instance, the prisoners' dilemma, a famous problem in the literature, the matrix for which is represented by Figure 6.

Figure 6

Clearly the players would do well to coordinate on their first strategies. Yet, so long as they must make their decisions independently, they must choose strategies which lead to the (1, 1) payoff. The reasoning is simple. The "2" strategies dominate the "1" strategies. That is, if Player B plays strategy 1, Player A gains from his strategy 2. If B plays his strategy 2, A still gains from his strategy 2. Thus A does better by playing his strategy 2 regardless of what B does; the same reasoning holds for B. Dominance is the strongest possible strategic criterion. Therefore, in the single-shot game, there is no possible way to avoid an undesirable outcome without communication.

Nor does it help if there is a finite iteration of the game. One might think that the players could at least maximize over time by playing their "1's", but clearly the last play is equivalent to the single-shot game. Thus, in this case, the dominant strategy must be employed. If this is so, each player must protect himself by using the dominant strategy on the next to last play, which now becomes equivalent to the single-shot case. And so on. In short, the dominant strategy must be employed in the very first game and therefore both players get quite undesirable results.

If the game is repeated indefinitely over time and if some discount factor is introduced so that there is a finite summation of the payoffs, it becomes possible to produce a more desirable result. The most

-7-

elegant method, but the least satisfactory from any other point of view, is to
have the players choose symmetric strategies. Symmetric strategies undoubtedly
will produce coordination, but what reason is there to believe that the players
will hit upon such strategies? Non-symmetric strategies, however, are quite
likely to produce cycling or an undesirable equilibrium.

If one assumes that the players are capable of learning over time, the game
can be solved in a fashion that lacks elegance but may be close to life. One
player can play strategy 1, since the cost is slight, and then revert to strategy
2 if the other player does not follow suit. It hurts the other player more to
break this tacit agreement over the long run than he can gain from breaking it
in the single case. Thus by influencing the other player's estimates of the
probabilities with which one will make certain moves under specified circumstances,
a mutually desirable equilibrium can be introduced into the game.
Raiffa has shown that, in this game, a player can gain from playing dumb and
forcing the other player to teach him. But this is dangerous, for if both players
play dumb, neither will reach the desirable strategy pair.

The power of this result stems entirely from the fact that it is possible
to punish severely a player who strays from his strategy 1. If this threat
position does not exist, the result cannot be produced in this way. Much
has been made of other non-cooperative games where the threat position
does not exist but where dominance does not impose a prisoners' dilemma.
It is important to examine the case where an elegant strict solution does
exist, but where certain difficulties arise if the context of the game is not
kept clearly in mind.

In their discussion of the non-cooperative game, Luce and Raiffa have
proposed a strict solution concept, very important applications of which have
been made by Schelling. According to Luce and Raiffa, there is a strict
solution in the non-cooperative game if there exists an equilibrium pair
among the jointly admissible strategy pairs and if all jointly admissible
equilibrium pairs are both interchangeable and equivalent. An equilibrium
pair exists when for any pair (x, y) there is no pair (x', y) or (x, y') such
that one of the players can gain from the move. In Figure 7, the pairs
(2, 3) and (3, 2) are in equilibrium, because neither player prefers to move

<table>
  <tr>
    <td>A</td>
    <td></td>
  </tr>
  <tr>
    <td></td>
    <td>2, 3</td>
    <td>1, 1</td>
  </tr>
  <tr>
    <td>B</td>
    <td>1, 0</td>
    <td>3, 2</td>
  </tr>
</table>

Figure 7

[LC]

-8-

if the other player remains fixed. Pairs are jointly inadmissible if for any pair $(x, y)$ each player prefers the pair $(x', y')$. In Figure 7 both equilibrium pairs are jointly admissible. Thus the first condition of the strict solution is met.

Pairs are interchangeable if, when $(x, y)$ and$(x', y')$ are in equilibrium, $(x', y)$ and$(x, y')$ are also in equilibrium. This condition does not hold for Figure 7. Pairs are equivalent when the player has no preference between them. In Figure 3, however, Player A prefers (3, 2) to (2, 3), but Player B has the opposite preference. Therefore there is no strict solution for Figure 3. Figure 8 presents a non-cooperative game with a strict solution-- that is, all conditions are met. Figure 8 also presents problems that are

Figure 8

related to the context of the game. It is therefore worthwhile to introduce an interpretation of the game in order to analyze these problems.

Suppose two college students who are strangers to each other are seated in different rooms and that they are each placed at a desk with a buzzer. Suppose they are told that each will receive $10 if neither presses his buzzer for ten minutes; that if each presses the buzzer simultaneously, neither gets anything; and that if one presses the buzzer first, he gets $5 and the other must pay $5. If the players are reasonably intelligent, both will sit tight, without pressing the buzzer, and each will gain $10. Let us now double the sums and assume that each knows the other and each wants to date the same girl at the same time. This girl is a bit mercenary and will go out with the student who can spend the most money on her. Thus only one student can get the date with her. Now each student will press the buzzer as soon as the game starts, for if the other student has not pressed the buzzer he becomes sure of the date, and if the other student has pressed the buzzer the first student has at least prevented him from getting the date and is no worse off himself.

Let us change the context just a bit more. Let us assume that each student wants to buy a new high-fidelity set and that each would like to be the only student on the campus with such a set but values having the set more than being the unique possessor of the set. Suppose we now multiply

-9-

[LC]

the numbers in Figure 8 by a factor of 10 and assume that $50 will purchase the set. The chances are that each will press the buzzer immediately for the same reasons as in the earlier example. Repetition in time will not help in this case, unless we assume that the game consists of an exact number of plays necessary to purchase the set. (We must assume that there is no good alternative use of the money and no outside source of funds.) If the game has the exact number of plays necessary (that is, five plays, if we go back to the original figures), each player knows that if he retreats from the strict solution on any play but the last, the other will spite him and he will not get the set. But then each must keep the agreement on the last, or fifth, play also. If we put the price of the set at $45, however, each will keep the agreement until the fifth play and then break the agreement on the last play only. If, on the other hand, there is no finite limit, each player again will cooperate until the decisive play and then hold back, waiting for the other player to weaken. Without explicit cooperation they cannot get their sets.³

3. Of course the strict solution cannot be denied if the numbers in the payoff boxes are really utiles. But the situation may be contaminated by special knowledge of the situation, by knowledge of the likelihood with which the other player will choose given strategies, by a desire to aid or to injure the other player, or by a special valuation of the social rules concerning "fair" behavior. Indeed, society survives to some extent because men generally depend upon the existence of social rules that prevent "worst" solutions, so that even where each has an immediate interest in violating the rule, others have an interest in punishing him. But society has even more effective methods for ensuring the observance of its rules. The socialization process inculcates these social rules in man's psyche so that his conscience punishes him for violations. Thus social rules may often be considered as game payoffs, and we normally value them so highly that we are willing to take lower concrete payoffs at any particular time or to take risks of lower payoffs that we would be unwilling to accept if we had not internalized these social rules. And most men also have an interest in supporting this socialization process, for, although it constrains them, they are better off than if none was constrained.

-10-

Suppose we construct a new game, which is represented in Figure 9. This game again has a strict solution--namely, (0,0). Let the game now

Figure 9

be interpreted. The two players can fight or not fight. If both refrain from fighting, the status quo is maintained. If both attack simultaneously, 8 units of resources are lost apiece. If one attacks first, he loses 5 units and the other loses 10 units. This appears a clear case in which the strict solution will be accepted by rational participants. However, let us introduce a new complication. The game is replayed indefinitely. Neither opponent can withdraw from the game (otherwise the best solution would be to end the game by cooperative agreement). Finally, if either player attacks first, he gains an advantage at the next play. Suppose that A attacks first. The new matrix might look like Figure 10. In this game, if we assume that no

Figure 10

referee can stop the game and that no one can come to the aid of the losing player, we can well imagine a joint decision to attack. This is even more clear if we return to the matrix of Figure 8. The immediate attack assures a minimum score of 5, except in the case where the opponent attacks simultaneously; and, in that case, 0 is the alternative to -5.

If a player can generate a positive score in the long run by taking immediate losses, he is likely to do so. He may also have a preference for security rather than for profit maximization. Thus, even if the other player is not presently hostile or irrational, he may become so in the future. And, if there is any possibility that one can gain from the first attack, there is not even technically a strict solution. This may occur if the other player is more likely to accept a loss than to make the investment necessary to defeat an attack. A matrix

-11-

that might induce some players to behave that way is portrayed in Figure 11.

Figure 11

Thus, if the choices are not made simultaneously, the player who attacks a unit worth 4 which can be defended only by the expenditure of 10 units may think that he can force his opponent to accept defeat. This is not an unreasonable expectation because (a) one can always attack a position with such strong forces that the cost of defense would be higher than the independent value of the unit attacked, and (b) many people argue (incorrectly) that the cost of defense ought to be equated only with the isolated value of the objective.

Let us return to the discussion of the context of the game in order to clarify some of the problems. In the earlier problem of the students who are strangers, the strict solution is quite compelling because of the purity of the context. The utilities of the students must be independent, and unless one of them has an irrational utility function for money, each has strong reasons to accept the strict solution. Even in this case, however, if obtaining $5 becomes a matter of life or death, one of the students may decide to maximize his security by ringing the buzzer, for that strategy guarantees $5 unless the other player attacks simultaneously, in which case death is certain regardless of what one does. Since the opponent can press the buzzer at any moment within the allowed time interval, security is served best by immediately pressing the buzzer. Even so, this result could easily be changed by placing the students in the same room and by requiring an overt, time-consuming, and unambiguous move to ring the buzzer. Once this kind of visibility prevails, even great security needs would not cause a rational individual to move toward the buzzer.

Most games of the non-cooperative type are played in the context of a larger society with large numbers of individuals, and with many chances to recoup lost opportunities and subsequent punishment for non-socially approved behavior. And, in the long run, an individual who does not accept the strict solution will inflict far greater punishment on himself than on his unfortunate opposite number in any individual play. Unfortunately, the international society is quite different in character. The players are few in number and the opportunities to recoup few. Even so, in the single-shot case the strict solution has much to recommend it. If repeated plays and the possibility of cumulative and progressive advantage are introduced, the game changes character. The chief condition for not accepting the strict solution is that

-12

the cost of eliminating the other players not be too high in some sense. This
may mean that the potentiality of recouping in the long run in terms of resources
is high, or merely that if the cost cannot be recouped completely, the gain in
security is worth the cost.

## Problems of Interpretation

In the previous section, we have examined instances of the way in which the context "contaminates" the non-cooperative game. This, however, may merely inspire caution in the use of utiles, although the earlier examination of the zero-sum game indicated occasions when the use of utiles rather than raw payoffs was of dubious validity. This section will raise questions concerning the use of behavioral data in coordinating strategies in non-cooperative games, and the following section will raise questions concerning the legitimacy with which utiles may be employed in the payoff boxes.

Schelling, in one of his papers, has constructed some ingenious examples which demonstrate the ways in which the design of the matrix enables players to coordinate their strategies to their joint advantage without communication. In Figure 12, the players coordinate on the (9, 9) payoff, which is inferior to

<table><tr><td>10, 10</td><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>0, 0</td></tr><tr><td>0, 0</td><td>10, 10</td><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>0, 0</td></tr><tr><td>0, 0</td><td>0, 0</td><td>10, 10</td><td>0, 0</td><td>0, 0</td><td>0, 0</td></tr><tr><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>10, 10</td><td>0, 0</td><td>0, 0</td></tr><tr><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>9, 9</td><td>0, 0</td></tr><tr><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>0, 0</td><td>10, 10</td></tr></table>

Figure 12

the others in terms of absolute amounts, but the uniqueness of which presents an irresistibly attractive clue when failure to coordinate produces a (0, 0) payoff. Schelling makes an even subtler point with the aid of two additional examples, which are given in Figures 13 and 14. In Figure 13, the players

<table><tr><td>10, 10</td><td>0, 0</td></tr><tr><td>0, 0</td><td>10, 10</td></tr></table>

Figure 13

<table><tr><td>10, 10</td><td>0, 5</td></tr><tr><td>5, 0</td><td>10, 10</td></tr></table>

Figure 14

-13-

have no way of coordinating without information not included in the matrix. In Figure 14, however, the second strategies appear to represent some sort of natural choice. If the players choose their second strategy, they guarantee a minimum payoff of 5. Schelling contends nonetheless that this is not the reason for the choice, because neither player has any intention of gaining a payoff of 5. Rather the matrix gives each player an excuse to act as if the other player were provided with a clue which permits coordination on a (10,10) payoff, thereby giving each player as much as he can hope to get in this game.

Schelling's point is well-taken but perhaps pushed a trifle too far. Surely one player, if he had reason to believe that the other player would employ a random strategy, would choose his second strategy in order to secure a minimum gain. Moreover, if one knows that the opponent is not intelligent enough to recognize the kind of clue Schelling describes, one would choose one's second strategy. If, on the other hand, the opponent is a person who in the past has declared loudly that if he cannot get everything coming to him, he prefers nothing, the first strategy would be chosen. Players who like to "live dangerously" might also force the same kind of response.

Admittedly, considerations of this sort contaminate the game. The only difference between this sort of contamination and that introduced by Schelling is that his occurs within the matrix and the other occurs outside it. Yet, except in experimental cases where players in different rooms are presented with printed matrices, the difference between "inside the matrix" and "outside the matrix" is ambiguous at best. Admittedly, there is a distinction between the kind of information necessary to fill in the payoff boxes and the information necessary to set probability estimates for the opponent's strategy choices. But there is no clear and definite dichotomy between the two kinds of information which enter into the determinations or in the reliability of the two kinds of estimates. Estimates concerning the value which a player assigns to certain payoffs or outcomes can only be inferred from his actions in concrete circumstances. Conversely, the kinds of actions one is likely to take may be related, in part at least, to knowledge of his past preferences. The knowledge that a poker player repeatedly draws to inside straits may be much more reliable than any information concerning his preferences for sums of money. On the other hand, the knowledge that a person likes sport cars may not permit many inferences concerning his behavior in a game of chance that he has entered in order to win money to buy a sport car.

In most political situations, it is not possible to construct the matrix without a good deal of information about the past and potential future behavior of players. Something about behavior patterns must be known before the tile numbers can be placed in the payoff boxes of the matrices. One may minimize, but not entirely eliminate, this problem by placing signs for the concrete physical outcomes in the boxes. But one still has the

-14-

problem of picking the relevant physical payoffs. One of the great difficulties in intercultural contacts stems from lack of information concerning the relevancy of given actions or payoffs. Ruth Benedict points out that Japanese often refrain from doing favors for others in order to avoid placing them under an obligation. Thus even if one attempts to avoid fine valuations, it is necessary to include sufficient valuation to include those payoffs which have relevance for the scale of valuation.

This consideration goes beyond the assignment of numbers to the payoff boxes to the very construction of the matrix. Thus in cultures in which the mode of payment as well as the thing given has value, one must break down the move of "giving" to, for instance, the moves of "giving gracefully" and "giving insultingly." In this sense, the range of strategies available depends in part at least upon the range of goods that are relevant.

The construction of the matrix may also depend upon other kinds of information. For instance, consider the situation of a general who has forces sufficient to protect only one bridge and an enemy who is able to attack only one bridge in an area where there are two bridges. A matrix can be built and the matter be considered as a game. Note that one can build this matrix only on the basis of information which indicates that one force is likely to attack the other and that the strategic problem centers on the defense of the bridges.

The question does not hinge on the legitimacy of making inferences from past behavior, but rather on where it is most useful to make these inferences and where the inferences interfere with the proper analysis of the problem. This is a pragmatic matter and depends upon circumstances, but there are circumstances in which it is legitimate to place probability distributions over the possible moves or strategies of the players.

In the stochastic game, behavioral information is essential to the learning process and to strategies which maximize the payoffs for the players over time. The prisoners' dilemma, repeated indefinitely, is the clearest example of this game. An example I used elsewhere⁴ demonstrates the value of this consideration for games which are not prisoners' dilemmas but for which there are no strict solutions. A game of this type is represented in Figure 15. The

<table><thead><tr><th></th><th colspan="3">B</th></tr></thead><tbody><tr><th rowspan="3">A</th><td>1,5</td><td>4,4</td><td>2,3</td></tr><tr><td>5,1</td><td>1,1</td><td>2,6</td></tr><tr><td>6,2</td><td>6,3</td><td>1,1</td></tr></tbody></table>

Figure 15

4. Morton A. Kaplan, *System and Process in International Politics*, New York, John Wiley & Sons, 1957, p. 231.

-15-

original game was played in utiles and will first be played that way here. Every real game starts somewhere in time with expectations based upon past behavior and upon some estimates of the ways in which players will attempt to secure an advantage in the game. Let Player A estimate on the basis of Player B's past behavior and on the basis of the advantages that flow from coordination that his first, second, and third moves or strategies will elicit from Player B his first, second, and third moves or strategies, with the following probabilities: (.2, .6, .2; .1, .7, .2; and .1, .1, .8). The moves thus have the expected respective values of 3.0, 1.6, 2.2. Player B's respective estimates are (.1, .4, .5; .4, .1, .5; and .2, .2, .6), which give rise respectively to the expected values of 1.9, 3.2, 2.4. Player A maximizes by taking his first strategy and Player B maximizes by taking his second. The action of Player A reinforces the belief of Player B that A will choose his first strategy and therefore raises the expected value of B's second strategy. This serves therefore to stabilize B's behavior pattern.

The value of the consideration stems from the fact that under some circumstances tacit coordination by the players can produce better results than if they attempt to maximize under the circumstance of completely independent decision-making. This makes the degree of confidence the players have in each other essential to the play of the game. There must be some way of observing the behavior of the players in order to reach this degree of confidence. Depending upon the actual structure of the matrix, the presence of proper threats and inducements, and so forth, a desirable coordination may or may not exist. But examination of the matrix informs us that in these cases it is not rational to ignore past behavior in determining one's strategy and that this is especially true if one assumes rationality in the opponent. This is greatly different from the zero-sum case, in which behavioral observations become important only if one assumes irrationality in the opponent.

## The Use of Utiles

The difficulties of translating payoffs into utilities in the non-zero game will become clear in the discussion, but it may be worthwhile to summarize here at least some of the conditions under which the translation into utiles of the entries in the payoff boxes of the matrix is dubious or illicit. Translation into utiles of payoffs entered in the matrix boxes is misleading if probabilities are assigned to an opponent's use of strategies. In non-zero-sum games, where desirable coordination depends on the use of clues provided by the matrix, translation into utiles may be misleading, although theoretical difficulties must be faced in the use of raw payoffs. If utiles are used for bargaining games, the lack of a standard for interpersonal

-16-

comparisons will cause grave difficulties and in addition will introduce new complications flowing from the possibilities of deceit. Where repeated plays of a game are involved, the theoretical difficulties in summing the utiles gained after each play make the use of raw payoffs more meaningful.

Let us assume that we can place probability distributions over the strategies of the players. And let us construct a game with a matrix in which the numbers represent utiles. Let Figure 16 represent this game.

Figure 16

In this case, A must choose his strategy 1 because it has an expected payoff of 3, as contrasted with an expected payoff of 2 for the second strategy. Let us now assume that the utility of A is linear for dollars and therefore that the utile numbers can be translated immediately into dollars. It is entirely legitimate for us to transform the matrix into one where B has a single strategy of tossing a die, four sides of which will call into play the payoffs 5 and 2 for A and two sides of which call into play the payoffs -1 and 2 for A. Thus if A chooses strategy 1, he receives $5 two-thirds of the time and -$1 one-third of the time. If he chooses strategy 2, he receives $2. It is not clear that he prefers strategy 1 to strategy 2 or even that he is indifferent between them. Yet if the dollars are immediately transformed into utiles, as in Figure 16, there can be no question that strategy 1 has a higher expected value than strategy 2. But the new game is exactly equivalent to the game represented in Figure 16. There is something basically wrong when an equivalent game is capable of producing different payoffs and therefore different behavior. The use of the probability distribution brings this into stark outline, but does not introduce an element that was not present previously.

When von Neumann and Morgenstern introduced the lottery concept into utility theory, there was really no necessary reason to view the lottery as limited by the dimensions of the individual payoff boxes. One might consider a lottery in which a strategy choice was equivalent to a lottery ticket entitling one to a set of physical payoffs consisting of the entire space of the strategy as intersected by all the strategies available to the opponent. Thus the decision to raise in a two-handed poker game gives one a ticket to a lottery in which the prizes consist of the monetary payoffs associated

-17-

[LC]

with the other player's decision to re-raise, call, or fold, and in which the value of the payoff ticket depends upon the player's preference for the lottery resulting from his decision to raise. One can even calculate how much that ticket is worth--that is, how much one will sell the hand for at this point.

If we go back to Figure 15, it immediately becomes clear that if we translate the utility designations back into raw payoffs, the simple arithmetic weighting of outcomes or payoffs and probabilities sheers away from the most useful line of analysis. If we view the strategy as a lottery that gives rise to physical payoffs with certain probabilities, the evaluation of the strategy will depend upon the players' attitudes toward risks. In the case of the game represented by Figure 15, little difference may result. But in games in which one very low payoff--much lower than any payoff connected with any of the alternative strategies--occurs with a low probability, it is possible that that strategy would be avoided even if it provided a higher arithmetic total than any of the other strategies. This preference might be a consequence of a player's conservatism or might stem from the competitive nature of a game in which such an outcome would throw one into a disadvantageous succeeding game.

The lottery interpretation of the strategy is indeed justified. This is a game under conditions of uncertainty and the choice of a strategy buys a ticket to a lottery in which one believes that the outcomes have subjective probabilities attached to them. Although this is not a game against nature, neither is it a purely competitive game. One can therefore assign reasons why one has this degree of confidence in the probabilities of the outcomes even though the opposing player indeed has the possibility of changing his behavior pattern considerably. Moreover, these assumptions permit one to do better than does the decision not to depend upon observations of the behavior of the other players.

Note also that the lottery assumption subsumes the result that Schelling asserts for the game represented in Figure 12. If the (9,9) payoff is really stated in terms of utiles, one cannot account for the preferences of the players. After all, in utility theory, preferences do not stem from utilities, but rather utilities account for the patterns of preferences. If, instead, the (9,9) payoff is a raw outcome--for instance, dollars--then one can take into account its uniqueness as a coordinating device and prefer the strategy which gives rise to the ($9, $9) payoff to the strategies which give rise to the ($10, $10) payoffs. Yet there is a difficulty here. Unless we have information that the payoffs are in some sense desirable to the players, we have no ground on which to assume coordination.

Using raw payoffs rather than utiles in the payoff boxes solves some real difficulties in game theory which become clear in iterated games but which are present in milder form in all games. For instance, there is no reason

-18-

to believe that payoffs in utilities can be summed over a series of games, but certainly raw payoffs like dollars can be summed. Moreover, the object in ruin games is not to bankrupt the opponent in terms of utilities but in terms of resources. Thus in the discussion of nuclear deterrence, one of the important aspects of play concerns the relative resources of the players as the game progresses. The opposing players can make reasonable calculations of the consequences for future moves in terms of resources and bargaining strengths of present moves, but they would have much greater difficulties in doing so in terms of utilities. Indeed, in terms of utilities, bargaining positions would become indeterminable because of the incommensurability of the utility scales of different individuals. Moreover, each player would have an incentive to place as much value as possible on a given physical payoff, because the higher his valuation, the bigger the share of the physical profit that any bargaining criterion would give him. Such a game would soon become unstable. However, once the resource positions of the players are known, it is sufficient to have some impression of the players' attitudes toward risks to make reasonable calculations.

Clearly, not all difficulties are eliminated by this device. One still may not have a good way of estimating how the players think a given contested objective will influence their resource positions, for the same object may have different values in different systems. Thus, to choose an obvious example, a radio tube has use for someone with a radio but no use for someone without a radio unless he intends to sell it. Moreover, this approach cannot accommodate objects which have cultural value but little effect upon resource positions. Yet treating the payoffs in terms of resources and the strategies as tickets to lotteries does introduce the possibility of calculability with respect to some of the most important aspects of the problem in international politics.

The fact that a strategy is preferred immediately assigns it a high utility number—higher than any other strategy in the game. But one does not necessarily attempt to maximize dollars or resources. Instead, one may attempt to maximize security or may prefer some combination of things to other combinations.

Indeed, one can introduce much stronger and more realistic constraints on the coalition process if this device of using raw payoffs is adopted. Consider a classic coalition problem and the difficulties of placing constraints on solutions in that problem. There is one buyer who wants two ties. He will pay $3 for two ties, no more for additional ties, and nothing for fewer ties. There are three potential sellers with one tie each. How should they combine to secure a gain of $3? No seller acting alone can ensure a gain greater than 0. Any two sellers acting together can gain $3. A coalition of three sellers cannot do better than this. This

-19-

game is essential, in the game theoretic sense, for the value of the game for
any coalition is greater than the sum of the values which the players could
obtain individually.

In this problem, an imputation is a set of three numbers, standing for
payoffs to the sellers, each of which is equal to or greater than 0 and the
sum of which is 3. One imputation dominates another if there exists a
subset of players--namely, two--who can obtain more from this imputation
than from the other and if the value of the subset is not greater than 3. If
one compares the imputations (0, 1.5, 1.5), (1, 2, 0), and (1.5, 0, 1.5),
the first is dominated by the second and the second by the third, since, in
each case, two players can improve their position by moving from one impu-
tation to the next. However, the first imputation is not dominated by the
third; dominance is not necessarily transitive.

According to game theory, a solution is a set of imputations that meets given criteria of inner consistency and stability. A set of imputations constitutes a solution if no imputation in the set is dominated by another imputation in the set, and if any imputation outside the set is dominated by at least one imputation inside the set. The set of imputations (1.5, 1.5, 0), (1.5, 0, 1.5), and (0, 1.5, 1.5) meets these criteria.

There are a number of difficulties with this treatment since, despite
the mathematical stability of the solution, no constraints are established
to prevent successive replacements of imputations of the (1.5, 1.5, 0)
type by the (2, 0, 1) type and vice versa. So long as the numbers in the
imputations in the boxes are viewed as utiles, this must be the case.
Actually, however, coalitions form to divide dollars rather than utiles.
Once this is admitted, unique imputations do not necessarily follow, but
strong constraints are put on the coalition process. Suppose, for instance,
that, given the opportunity to speak first, one seller offers another a
partnership--that is, the opportunity to collect $1.50. Each of the two
sellers knows that if the agreement is rejected or accepted and then broken,
there is no mathematical halt to the process. Since each knows this, there
are strong constraints to stick at this offer and to reject offers of $2 even
though decisions be made independently. Once the accepted offer is re-
jected, if it is a "fair" offer, the process cannot halt logically. Indeed
the fear of being left out may be so strong that an initial $1 offer will be
accepted. No effort will be made to explore further the intricacies of the
coalition process, but the problem does become simpler and also more
realistic if raw payoffs rather than utiles are placed in the imputations.
Then the decision to accept the imputation itself acquires a utility based
in part on the raw payoff and in part on the reduction in uncertainty.

-20-

In the last analysis, one must look at the problem before one can know
what information is relevant and how one must proceed to maximize what-
ever one desires to maximize. In some games the players may have to
turn the play of the game over to a referee in order to maximize. Consider
Figure 17. In this game, the (0,0) payoff is quite likely, unless there is an

Figure 17

umpire with compulsory power. Although there is no strict solution, (5, 4)
is clearly the one upon which the players can most plausibly be expected to
agree. Yet if Player A takes his second strategy, Player B is likely to take
his second, which produces (-3, 5). To guard against this, Player A must
take his first strategy, which produces a payoff of (0, 0). Even though the
matrix of this game seems favorable to B, the only way he can get good re-
sults--assuming that the players are competitive and the game is iterative--
is to ensure A against a doublecross. Only an umpire can do this. Of
course, in reality there may be a side-payment of .5 by A to B to equalize
matters.

These considerations are quite relevant to international politics,
where the games are non-zero-sum games, iterative, and competitive.
The problem of the umpire is equivalent to situations where the players
can agree on the desirability of a payoff, except that once the agreement
is reached one player finds that the form of the matrix gives him an in-
centive to doublecross the other player. Both disarmament and the
problem of Germany may bear some relationship to these considerations.
For instance, let us assume--and this is a very dubious proposition--
that the neutralization of Germany is desirable. Even if the United States
and the Soviet Union could reach agreement in principle, the Soviet Union
might discover an incentive to doublecross in the fact that, once the
steam is taken out of NATO, the price of withdrawal from East Germany
and Poland no longer has to be paid to get the advantages that had been
hoped for from neutralization. Part of the international political game
may involve the search for situations of this kind and the effort to find
umpires or other devices which prevent or compensate for a doublecross
within the play of the game.

-21-

PROBLEMS OF COORDINATION

One of the more difficult problems for players in games that are not strictly cooperative is to coordinate their strategies in such a way that they can gain better payoffs than they otherwise would. Some of the most difficult obstacles to coordination occur in the prisoners' dilemma and some practical solutions to the dilemma have been suggested in the preceding essay in this memorandum. Moreover, the dilemma can be overcome, even in the single-shot case, if the prisoners must announce their strategies in the same room by a given time and if the decision to talk before the time runs out cannot be made without giving the other prisoner time to talk also. Under such conditions, the alternatives are effectively reduced to those of (talk, talk) and (not-talk, not-talk) and the latter alternative will dominate the former for both prisoners. This fact is recognized by police officers, who follow a standard practice of separating prisoners in order to question them effectively.

If choices must be made simultaneously, no pre-play agreement can affect the game, since no penalty can be imposed for breaking the agreement. Therefore, in games of this kind, the solution for the not strictly cooperative game is the solution for the cooperative game. On the other hand, if one of the players can commit himself to a given choice before play ends, the other player, depending upon the matrix, may be constrained in his choices by this commitment. For coordination to occur in the game used in the previous paragraph, it was necessary that the "talk" response require sufficient preparatory time for the other player also to have the opportunity to talk. Only then could the prisoners co-ordinate on a desirable solution in the single-shot case.

The form in which the game is presented is crucial for the players. This is not a trivial problem, although common sense might indicate that in real political problems players will naturally agree upon constraints that produce jointly desirable payoffs if the alternative consists of jointly undesirable payoffs. But it may turn out that the attempt to reach such agreement only pushes the problem back to a different level-- that is, to a dispute over the mode of cooperation--or there may be other constraints arising from internal political or economic conditions or the facts of nature that make agreement upon the constraints unstable.

The fact that the game consumes time and therefore that a series of iterated plays is involved may make coordination difficult by giving either

-22-

or both players a reason to doublecross before the end of the game. Consider the iterated game represented in Figure 18. This is a game which, if played out, can lead to one of two new games, $\zeta^1$ and $\zeta^2$. The new games can be viewed

Figure 18

simply as restructured matrices, one of which is good for both players and the other of which is bad for one and very good for the other. The (0,0) payoff is the starting state and the game can be ended at this point by either player. This may be treated as a refusal to negotiate. The other payoffs prior to the new games may be viewed as stages in the negotiations, with one player or the other being able to get a slight advantage by breaking off negotiations at a given move. By the time the last node leading to the new games is reached, the new game may require joint agreement. In the game represented by Figure 18, the player in the weaker position in the new game rationally must forego the cooperative solution because of lack of control at the crucial stage. This kind of situation may require agreement by the players on the use of penalties for non-performance that are practically automatic--that is, that penalize the player who opts for any game other than the jointly advantageous one.

Still other problems beset the players in iterative games. Not all situations can be governed by automatic penalties. For reasons which have been advanced in the previous essay, the outcome of the iterative game, where no strict solution exists, will depend on expectations concerning the behavior of the players. Consider a game similar to the prisoners' dilemma which is iterative and is played for dollars that are distributed after each move. Figure 19 may be taken as a representation

<table><tr><td>9, 9</td><td>0, 10</td></tr><tr><td>10, 0</td><td>1, 1</td></tr></table>

Figure 19

-23-

[LC]

of this game. If the game is repeated indefinitely in order to avoid regressive aspects of the dilemma and a discount factor is introduced to permit a summation, tacit cooperation between the players permits them to dollar maximize better than does following the strategy which is dominant in the single-shot case. If the players do not cooperate, however, they must retreat to the dominant strategies. Suppose each player decides to adopt a punishment strategy such that if the other player adopts the non-cooperative strategy, he will return to the dominant strategy until the other has played the cooperative move for two consecutive moves. If both players decide to use the cooperative move from the start until the other doublecrosses, no difficulties arise. If, however, one attempts to discover whether he can gain by forcing the other to teach him to cooperate, or if one attempts to discover whether he can get away with lack of cooperation, the strategies enforce cyclical behavior.

Intuitively, it is reasonably clear that strategies fixed in advance are not likely to produce cooperation. On the other hand, strategies based on estimates of the motivations which lie behind the moves may permit learning and eventual coordination. Yet one player may persistently gain by employing Raiffa's advice to play dumb in order to force the other player to teach him. Whether purposefully or not, the Soviet Union would seem to have gained a distinct advantage by forcing other nations to teach it to play according to the cooperative strategy. On the other hand, if both players persist in playing dumb, both will lose.

In this iterative game, communication and hence cooperative rather than strictly non-cooperative features will be necessary to the coordination process. Cues and signals, as well as direct and explicit communications, that influence expectations will play a role in the game. This aspect of game strategy has not been well explored beyond Schelling's pioneering work. Moreover, it is somewhat doubtful that a general theory can be developed. Instead it may be necessary to build a body of experience based on somewhat simplified models which can then be applied to some of the situations likely to arise in international politics. There will always be some hazard in this procedure, but if initial exploration by means of simplified models clarifies the utility of particular devices in specified situations, at least the errors of common sense may be avoided.

## Expectations and Signals

Two important elements are involved in the analysis of coordination problems in the learning game. First, it is necessary to build up a body of expectations that permits coordination. Learning is necessary to this end, and coordination in the non-cooperative game is unlikely without

-24-

[LC]

learning. Second, an effort must be made to agree on a body of signs, signals, or cues to be used in the learning process without foreclosing the possibility that a set of signs can be imposed that will be favorable to oneself. The two elements are linked, for expectations concerning behavior are necessarily linked to acts which serve as signs of future behavior. Signs are not signs except within some body of experience that gives them meaning, although the meaning is not entirely arbitrary. Thus holding out one's hand was originally a sign of peaceful intentions because it showed that no weapon was held in the hand. Obviously, the less confidence a situation permits, the more closely the sign must be linked to the physical potentialities of the situation.

For instance, before the First World War general mobilization was interpreted as an act of war. This sign did serve to protect to some extent nations which might be put to serious disadvantage if they did not react to a general mobilization by another nation. But the sign had a much greater advantage for Germany, since Germany could mobilize faster than Russia. This meant that Russia must be perpetually unprepared or trigger a war. On the other hand, insistence on that interpretation of the sign by Germany meant that whenever Russia felt impelled to mobilize, Germany would be forced to war or be revealed as a buffer.

It is obvious that control of the signals or of their meaning confers an
advantage upon the player having control. Control, however, does not imply
that the controlling player proposes the contents of the signals. It may very
well be that the player who proposes the contents does so out of weakness--
that is, that he has a greater incentive than the other player to avoid the
consequences that flow from failure to coordinate or a greater incentive to
seek the advantages of coordination. If this is so, the player who proposes
the signals may very well propose a set designed to elicit the cooperation
of the other player. Thus the signals established may favor the other
player when viewed from some standard of "fairness" that abstracts from
the bargaining strengths of the players.

It is also important to note that the appearance of weakness on the part
of one player changes the expectations of the other player and may very
well introduce cycling into the learning game rather than coordination.
To establish a pattern of activity, for instance, that leads the other player
to expect one to back down from bold positions in the face of threats may
lead to disastrous consequences in situations where one does not intend
to back down (unless the situation is both local and can be kept local and
one has a clear preponderance of capabilities). The obverse of this con-
sideration is that weakness should not be advertised, for this incites the
other player to raise his demands concerning either the direct objectives

-25-

[LC]

of the game or the rules or signals that should govern the play of the game.

Information concerning weakness or strength in bargaining over the establishment of a set of signals depends upon the signals that have already been established. Otherwise, coordination of expectations by the players would prove impossible. The game therefore is to use existing expectations in order to control future expectations.

The general condition underlying the ability rationally to impose a set of signals is that failure to accept the signals, even though not optimal from the standpoint of the player upon whom the imposition is made, must be more costly than acceptance. Put in this form, the statement is trivial. Yet it clearly underlies the blocking by the United States of the attempt by the Soviet Union to establish the potential attack-signal value of American bomber forays into Arctic regions in response to unidentified radar blips. By cooperating with the Soviet attempt to establish signals, the United States would become dangerously vulnerable to surprise attack. To insist upon interpreting the forays as signs of attack would force the Soviet Union into probable war to halt air forays which it knew with reasonable probability to be innocuous from the standpoint of its security interests.

In general, advantages in the imposition of signals stem from situational advantages. Schelling has investigated situational advantages in contrived situations, and reference to the kind of consideration involved there may prove helpful. Suppose two players can divide $100, provided they can agree on the division, but they get nothing if they cannot agree. Suppose the players can see each other but cannot hear each other except through a set of earphones. Let one player propose over the radio that a 60/40 division occur and visibly throw away his earphones. Most players would now accept the 60/40 division, but not all, for some would rather get nothing than let the other player profit from an "unfair" action. And, surely for all players, there would be some lower bounds on what they would give in to. For instance, how many would accept 99/1?

It may be objected that situations like this cannot occur in international politics, that the channel of communications cannot be closed. Yet even in highly important matters this is not necessarily so. For instance, one might establish an automatic missile system which would respond by attacking Odessa if Kansas City were attacked. This system might be made mechanically impervious to any communications except those stemming from nuclear explosions. The diplomatic ultimatum might establish political insulation sufficiently effective to constitute an interference with the communication process. By involving the prestige of the nation, the cost of backing down might be higher than the cost of initiating war in

-26-

[LC]

the event of refusal. Admittedly, such situations are likely to be more rare
in the modern world of nuclear weapons than in previous ages and are unlikely
to involve both the United States and the Soviet Union directly, although even
this possibility cannot be excluded.

The two examples given immediately above are instances of voluntary but
effective situational advantages. Other advantages may stem from involuntary
factors. Thus the fact that many nations friendly to the United States border
on the Soviet bloc and that the most convenient air routes between them and
this country come close to Soviet territory make forays by armed American
warplanes implicitly less threatening than similar Soviet flights to the border
regions of the United States. Other situational advantages may depend to a
large extent on the fact that the players are not consciously aware of their
nature. Consider, for instance, the attitude of many Arabs to the American
support of the legitimate government of Lebanon as interference in a civil
war. In the last hundred years or so, according to international law, it
has always been proper to support existing governments. Yet if popular
attitudes which view such aid as interference are sufficiently pervasive in
the affected area, then the aid does constitute a sign of interference and
may produce consequences as bad or worse than the absence of aid. Even-
more important in this respect is the widespread belief in Asian and Arab
areas that it is right and proper to aid the Algerian rebels but wrongful
interference to aid the Indonesian rebels. Such distinctions may even be
viewed as irrational rather than non-rational. But this fact alone is suf-
ficient to make them resistant to penalties and thus to make the signal
effective.

Seen in this perspective, much of present-day international politics is
a game to establish signals--that is, to control expectations. Thus, for
instance, much of Soviet foreign policy is designed not so much to seek
direct support for Soviet policy as to characterize United States or Western
policy in such a way that support is withdrawn from it. Thus, if the British-
French landing at Suez can be characterized as imperialist, a heavy penalty
is imposed. In the same way, if the American continuation of nuclear test-
ing can be characterized as a sign of a war-oriented policy, American
policy loses support. Or, if testing is seen as an injury to future genera-
tions rather than as a protection of existing institutions, the effect may be
felt both within other countries and within the home nation. The establish-
ment of signals of this sort does not necessarily prevent continuation of
the activities, but it effectively changes the game matrix by reducing the
payoffs. And, of course, if the payoff can be reduced sufficiently, the
activity itself may have to be discontinued.

-27-

[LC]

## Signaling in the Reprisal Situation

The problem of signaling in a reprisal situation is sufficiently important and interesting to deserve separate analysis. The meaning of reprisal has changed over time. Although it was once considered a proper means of implementing national policy in defined circumstances, more recent interpretations have led to its rejection. Reprisal is no longer interpreted as a limited response to a wrongful act and as a warning not to repeat the act. Instead, it is interpreted as a crime. Because reprisals are illicit, failure to respond to a reprisal with vigor may be interpreted as a sign of weakness. Or if reprisals are not expected in the first place, the reprisal may be interpreted as the sign of an unlimited attack and thus lead to an unlimited counterattack. If the theory that nuclear reprisals or warnings are the most effective deterrents to breaches of the peace by major nations is correct, changing the signaling value of the reprisal may be in order. The so-called advance in international behavior which has been fostered by many idealists may have eliminated from the arsenal of signals one of the most useful. Conceivably, if reprisals cannot be used, the alternatives may be reduced to surrender or nuclear war in case of aggression. This would give a decided advantage to an aggressive nation. Let us suppose, however, that the old interpretation of reprisal is restored. How can it best be used in a variety of situations?

In the first place, clarity is important in the reprisal situation if expectations are to be coordinated. One can hardly seek the cooperation of the other player if he has reason to suppose that an effort is being made to thrust him into an untenable position. Use by the players of nuclear reprisals as part of the strategy of the learning reprisal game⁵ depends upon a symmetry that permits each player confidently to believe that he can maintain an acceptable status quo. Its use also depends upon a clear and explicit relationship between the means used and the ends to be achieved. To use nuclear reprisals in cases where the threat to the status quo is political raises very serious questions. Political penetration rarely occurs without indigenous factors that are favorable. Moreover, the line between the admissible and inadmissible with respect to political activity is extremely difficult to define. Therefore, with some possible exceptions which are hard to establish in advance, nuclear reprisals cannot be used to counteract various forms of internal and external political pressures.

5. See Morton A. Kaplan, "The Calculus of Deterrence," *World Politics*, October 1958, pp. 20-44.

-28-

On the other hand, the military crossing of borders is clear and definite, although the causes of the crossing may be quite ambiguous. Therefore, the threat of reprisal can be used unambiguously to insist upon a withdrawal to the original borders. For instance, it would have been useless in June 1950 to have condemned the North Koreans for crossing the border into South Korea because the border had been crossed repeatedly by military forces of both sides in previous years. The real grievance lay in the refusal of the North Koreans to withdraw and to halt their military activities. Reprisal cannot be based upon a thorough examination of the matter (for this would almost always lead to ambiguity) but must be related clearly to an unambiguous criterion, like failure to withdraw to the original line of demarcation.⁶

The meaning of reprisal can be established more easily during times of peace than during an actual aggression. The reason is simple. If the reprisal strategy is established before the aggression, the potential aggressor must decide whether to risk unforeseen consequences. If one resorts to reprisal after an act of aggression, the decision to engage in aggression has already been made and at least part of the burden has been shifted to the other player by the aggressor. For the same reason, an announcement not merely of the intention to use reprisals but also of the schedule of reprisals--provided that the schedule is not so one-sided as to provide inherent reasons for rejecting it--will serve to shift the burden to the aggressor by establishing heavy costs for the defensive side if it does not carry through. In the latter case, the defenders would not only lose the objective but additionally would suffer the costs of being exposed as bluffers.

The meaning communicated by the reprisal strategy is of essential importance to its success. It must be clear that the strategy does not threaten the vital interests of the potential aggressor, that it does not threaten to impose a losing game upon him. It is also important that the strategy persuade the actual or potential aggressor to accept rather than to oppose it by appealing to his reason. Thus it is important to commit oneself to successive stages of retaliation should the first not be effective, so that the costs of not following through are worse than the costs of withdrawal from the action. In this way, the side that moves across the frontier and refuses to withdraw is forced to choose between "all-out war" and "withdrawal to the status quo" and the other side is not forced to choose between "unfavorable change of the status quo" and "all-out war." This cannot be accomplished unless the alternatives are established in advance. Temporization will not work here.

A predetermined schedule, of course, cannot work properly in ambiguous cases. It has earlier been mentioned that political disputes can only rarely, if ever, be sufficiently unambiguous for a reprisal strategy. It is not always the case that military events are sufficiently unambiguous. The Lebanese affair is an example of a cloudy situation. Border infiltration by small groups

6. Unfortunately, this criterion may not hold if trouble occurs in regard to Berlin, and a better or supplementary criterion may be needed for such cases.

-29-

may be real enough, but somehow may lack sufficient definiteness. This was
particularly true in 1958 in Lebanon, where tribal connections across the
borders clouded the issue. Although similar difficulties might seem to exist
in Germany and Korea because of common ethnic factors, army loyalties are
sufficiently great--and American intervention would not be interpreted
locally as imperialistic aggression--to permit effective counteraction. In
most significant areas of the world, aggression, to be effective, would re-
quire a degree of military activity that would be sufficiently clear to permit
the operation of the learning retaliatory strategy.

The Problem of Surprise

In general, relatively satisfied nations seek to avoid surprise, for the potential costs of surprise are considered more damaging than the corresponding potential gains are considered desirable. With respect to the retaliation strategy, the element of surprise as a danger to its effectiveness in ensuring coordination cannot be minimized. This may not always be the case, for once hard-based and submarine-based missiles are available in quantity, the deterrent capacity of a nation may not be vulnerable to a surprise knockout blow. In circumstances where the deterrent is not vulnerable to surprise, there will be time to determine the intentions of an attacking force, to find out whether it is pursuing a limited or unlimited objective. If, however, the deterrent system is vulnerable to surprise, it may be necessary to interpret some military formations as signifying the intention to attack, even though this may not be the case. For these reasons, under normal circumstances the deterrent strategy involves a serious danger of misinterpretation. Yet it may still be possible to use the deterrent successfully.

It is very important that reprisals be used neatly, that they accomplish
what they were designed to do and neither more nor less. Yet in ordinary
circumstances this is difficult. A lone bomber sent to an interior city
would have slight chance of reaching its target. Yet if, in order to ensure
delivery, the reprisal is carried out by a large force of bombers and if
the deterrent of the aggressor is vulnerable to massive attack, he may
have no alternative except to respond massively.

This seems to create an impasse and to preclude use of the reprisal
in situations in which the deterrent of the aggressor is vulnerable to a
first strike. However, this impasse could be got around if the retaliatory
bomber or missile were guided in by the aggressor. This may seem
absurd. But consider the alternatives. If a lone attack weapon were
launched, it would have little chance of reaching its target. Although
sending such a weapon might serve as a signal of seriousness and lead

-30-

to a restoration of the *status quo*, one is entitled to believe that such a gesture would appear more quixotic than serious. On the other hand, a mass incursion, even with the intention of dropping only one bomb, might do considerably more damage, might miss the target and hit a populated area, or might be regarded as a general nuclear attack, in which case instant retaliation could occur. However, if the aggressor cooperates with the reprisal, he can distinguish between surprise general attack and a limited retaliatory attack, while at the same time suffering minimal damage. This implies that optimally an observer of the aggressor nation should be present at the planning of the retaliatory operation and that a flight plan should be transmitted. Perhaps radio beams should guide the reprisal plane to the target. If the nation which launched its forces across a defined border should decide to restore the *status quo* after the reprisal was carried out, it could be permitted to launch a compensatory attack and the same procedural rules could be followed.

It may be argued that no nation would agree in advance to proposals for guiding retaliatory bombers to targets in its territory and that no nation would agree at the time of action, for, by failing to agree, it thrusts the burden of unforeseen consequences on the player engaged in retaliation. Suppose, however, that formal agreement is not required and that instead the flight plan of the sortie is announced publicly along with the threat of a larger raid if the side against whom the reprisal is directed does not co-operate. This thrusts the burden of unforeseen consequences on the aggressive side by establishing a severe penalty to the side announcing the use of reprisal for not following through with the heavier attack (loss of the objective and also exposure as a bluffer) if the reprisal is not accepted. Indeed, it thrusts the burdens so squarely upon the aggressor that one must expect his military withdrawal to the line of the *status quo* before it is necessary to complete the reprisal.

The procedure suggested for reprisal guarantees against the use of the retaliatory attack as a cover for a full-scale surprise attack, but it does not protect against other possibilities of surprise. An important paper by Schelling⁷ has demonstrated the close connection between the probability of surprise and the possibility of improving the warning system of the other player when it is not sufficiently good to protect against surprise; otherwise preventive war might become appealing. Warning systems may be improved not simply by radar-like gadgets but also by human factors. Hence the proposals for inspection. Szilard has made a seemingly outrageous proposal which has little chance of adoption but much intrinsic merit. He suggests a series of rewards of a very substantial nature for informers and also asylum for informers who provide correct information. This naturally clashes with the important value of patriotism. Nonetheless, this kind of system might provide the sort of information which most inspection teams could never ferret out unaided.

7. Schelling, "The Reciprocal Fear of Surprise Attack," loc. cit.

-31-

[LC]

One of the biggest problems, however, may stem from the possibility of technological developments that would undermine the mutually deterrent situation. This possibility makes for precarious stability. Such technological developments are not predictable in any meaningful sense, since they depend to some extent upon discoveries involving unforeseen basic relationships. If inspection were unlimited and informants used, this possibility of technological surprise might be minimized. It may be possible to develop some system which would give each side effective hostages and thereby nullify the advantages of technological discovery. Control of installations within the territory of the other player might accomplish this end, although care would have to be taken to ensure against aberrations on the part of the individuals who control these installations.

Perhaps one of the most important safeguards for a player against the possibility of surprise would be full access to all scientific and technical establishments of the other player. One of the chief present-day fears involves the consequences of unforeseen scientific or technological developments. Sudden shifts in military capabilities, or even possibly in economic capabilities, or the fear of such sudden shifts are among the most unstabilizing elements in the present situation. If the players are more interested in preventing defeat than in ensuring victory and if such developments are largely unpredictable, the elimination of scientific secrets between the United States and the Soviet Union (not between all nations, for the spread of nuclear secrets throughout the entire world would introduce dangerous complications without solving any existing problems) would constitute a joint interest of the two nations. Since technological developments could not be used to the important advantage of either side, this would de-emphasize the military aspects of research and emphasize the peaceful aspects, while at the same time restoring that openness of communication which is necessary to the scientific community.

Unlike simple disarmament, which has the disadvantage--for democracies at least--that rearmament is a painful and difficult task, adoption of proposals of this type would not undercut the retaliatory capabilities of the players, although it would tend to prevent the development of advantages that are unpredictable--and therefore hazardous to both players--and also tend to reduce those fears which make surprise attack seem an acceptable strategy. In effect, it would change the character of the strategic matrix in a way satisfactory to both players. Such solutions would work, however, only in situations where the joint cooperative solution was superior to any payoff that was likely to obtain if decisions were not coordinated.

-32-

[LC]

These suggestions should not be viewed as attempts to accomplish what no devices of this sort can--namely, to check social and political evolution. Certain kinds of economic, political, and social processes are going to continue to have an effect on military relationships and on the stability of any given status quo. The suggested devices are designed to safeguard against military attempts to change or to influence changes in existing political relationships and against the potentialities of surprise. But, even here, by posing conditions that are psychologically unacceptable, they may undercut the very purpose they are designed to serve. It is possible that some of these devices would give rise to fears so great that political upheavals would nullify them. Some could never be adopted in the first place and others might be unwise in view of their non-military consequences. Yet investigation of these problems still has a function insofar as it explores the conditions and requirements of military stability.

## Problems of the Pathway

Still other problems of cooperation in signaling involve the pathway. Even if so-called strict solutions exist, how can one reach them if the path to them lets one of the players achieve even greater advantages from breaking the agreement? In domestic situations, umpires and courts impose penalties. But, as Hobbes noted, in the state of nature within which nations operate no agencies of this type participate. Therefore, if pathways that are unstable must be traversed to achieve a desirable cooperative solution, ways must be found either to eliminate the need for umpires with compulsory powers or to provide substitutes for them.

It is doubtful whether any general theory will inform us about these situations. It is likely that efforts to solve problems of this kind will be ad hoc; at least, the present proposals are of that character. For instance, consider the matter of the placement of missile bases in Germany. Assume, for the sake of this discussion, the doubtful proposition that it is worthwhile to trade off these bases for the withdrawal of Soviet troops from Eastern Germany, Poland, and Czechoslovakia. If discussions begin in the present situation, the momentum behind Western German acceptance of missile bases is quite likely to collapse in the face of good expectations concerning the possibility of a detente in Europe. And, once this occurs, it is quite improbable that the decision to establish such bases could be made at a future time--particularly within what may be the critical time period for the maintenance of nuclear parity. Once, however, the probability of missile bases in Western Europe has been minimized, whatever incentive the Soviet Union may be presumed to have to reach an agreement that would disrupt her hold on the Eastern satellites will have vanished. Thus it cannot be assumed

-33-

[LC]

under these conditions that the talks would ever reach a successful conclusion. On the other hand, if the United States were to make it a condition of holding talks that work was to begin on the bases and that work would be halted or the bases dismantled only if the talks came to a successful conclusion, the incentives for the Soviet Union to reach agreement would become stronger than they were when the bases were only problematical. The incentive for the United States to continue the talks would decline, but only if one assumed, contrary to the original assumption, that Soviet withdrawal from the Eastern satellites was less preferable than the maintenance of the missile bases. Moreover, ways could be found to guard against this by giving the German government some sort of operational veto upon final use of the bases if the United States did not conduct the talks in a satisfactory fashion.

## The Role of the Expert as an Umpire

One great difficulty in coordination which may require the services of an expert as umpire stems from the unstable jockeying that may take place over which of several possible cooperative agreements are to be made. In this respect, it is important to note that it may be much easier to reach agreement on the general rules that will apply to such cases before they arise rather than afterward (or, alternately, that it may be easier to agree on the rules when their general but not specific consequences are known). Consider the games represented in Figures 20 and 21. In this

Figure 20

Figure 21

game (0, 0) is the alternative to agreement. The solution in this case is so obvious that we do not need to specify criteria. The (5, 5) payoff is the solution.

In Figure 21, according to the Nash criteria for bargains (invariance with respect to utility transformation, Pareto optimality, independence of irrelevant alternatives, and symmetry in the solution), the payoff should be (50, 5), assuming that side-payments are not permitted in this game. The player at the (0, 10) end of the scale may argue, however, that his bargaining position is much stronger since the other player has more to lose. He may therefore hold out for a (9.09, 9.09) solution. The other player may claim that one should look not at the absolute amount one may lose, but at its relationship to the amount one can gain in this game. He

["-34-", "may therefore hold out for the Nash solution or for some intermediate payoff.\nIrrational factors like stubbornness may further complicate the problem.", "If, however, the players are forced to choose the criteria to be applied\nbefore they know which roles they will play in the game, it will be consider-\nably easier to agree upon the criteria which the arbiter is to apply, even\nthough the choice of rules will still not be self-evident and problems may\narise in reaching agreement. For instance, one player may prefer the\nNash solution and another the (9.09, 9.09) solution without knowing which\nrole he will play in the actual game. Indeed, one or both of the players\nmay prefer the (100, 0) solution, on the thesis that he would rather accept
$100 when he wins and $0 when he loses (although intuitively it is doubtful\nthat the United States or the Soviet Union would accept such an option\nwhere important issues were involved). What seems acceptable may\ndepend upon whether there will be repeated plays. Nonetheless agreement\nis clearly easier before playing roles are assigned than afterward. More-\nover, if the problem is removed one stage further, the players might be\nasked to agree on rules that would govern a game in which they might or\nmight not play. Thus more general considerations of \"fairness\" might\nenter in and the problem of agreement would become less difficult.", "If raw payoffs are used in the matrix boxes, one problem in coordina-\ntion may stem from conflicts over inclusion of items. Inasmuch as the\nuse of utiles does not avoid this and, indeed, complicates it because of\nthe problem of interpersonal comparisons, the matter is quite knotty.\nIn \"The Calculus of Deterrence,\" ⁸ the matter is solved by fiat--that is,\nby the assumption that all units of resources have equal value for both\nplayers or, in effect, that all units of resources are interchangeable.\nIn practice, however, this is likely not to be the case. Therefore one\nplayer may hold out for an equal division in terms of physical units and\na second player may hold out for a division that equalizes the consequences\nfor the players rather than for an exact division of the raw units. This has\nnothing to do with utility, but concerns physical consequences like the\namount of steel a ton of ore will produce, which may be related to the\nplant into which it is fed. There will also be different estimates of the\nconsequences. Probably no general solution to this problem exists, but\nit is useful to call attention to it since it will constitute a serious hazard\nto cooperation. And, even here, giving broad rules to experts may\nsimplify the problem.", "8. Loc. cit. p. 23."]

-35-

One of the differences between real political problems involving the rules given to experts and abstract game discussions stems from the fact that in real political problems the players possess considerable information about the strength of their respective roles. For instance, one of the difficulties in disarmament situations is that the players start the game with playing strengths related to the technological progress they have made. Therefore agreement on "fair" criteria may depend considerably upon speed in recognizing where problems are likely to arise and in coming to agreement before too much information on respective advantages is possessed by any of the players. This is one reason why agreement on control of unforeseen technological developments may be easier than agreement on control of existing weapons systems, although there may be some eventual feedback if rules for future control prove workable. Once such rules are adopted, the players know they cannot reject the arbiter's award without foregoing the possibility of cooperation. And so long as failure to cooperate is worse than cooperation, the arbiter's award is likely to be accepted. Since the arbitration rules have been established, the effective choices are reduced to acceptance of the award or retreat to the non-cooperative solution, which is probably worse. Moreover, although in principle one might reject the award and attempt to negotiate new rules or principles of division, this entails a grave risk, for both theoretic and practical reasons, that the non-cooperative solution will be the actual outcome.⁹

These problems may be amenable to reasonable solution if they are turned over to experts who act as umpires. The players could establish quite general rules which meet clear standards of "fairness," such as "provide each player with equal protection against the dangers of surprise attack." The experts could then determine whether this rule could be implemented and, if so, propose a solution to the players.

A step in the indicated direction has already been made by the Geneva meeting of experts on the detection of nuclear explosions. In this case, however, the technicians are not empowered to decide whether nuclear explosions should be halted, nor, indeed, do the experts include those who would be best qualified to answer at least some of the questions involved--namely, technicians in the field of strategic analysis.

9. It is not unreservedly true, however, that even when the players know what their playing strengths are, agreement upon a cooperative solution is out of the question. In "The Reciprocal Fear of Surprise Attack," Schelling has shown that a nation which is vulnerable to surprise attack may be forced to opt for preventive war and that--at least in the case of a strict solution game--the other player has an incentive to improve the warning system of the first. Thus what at first may seem to represent strength may turn out to be disadvantageous.

-36-

The suggestion for turning policy questions over to technical experts runs counter to one of the tenets of democratic theory, which would have government leaders, as representatives of the people, make the policy decisions. Perhaps there is an area in which political leaders still have a legitimate function: that of issuing very broad ground rules to the experts. But these should be broad rules like "Attempt to give both players equal protection against the danger of surprise attack" or "Remove conditions which may rationally impel one of the players to launch a surprise attack." Beyond general instructions of this sort, the political function may be anachronous, because the politician is no longer in a position to understand the matter and because the interjection of political considerations is likely to lead to unstable jockeying that produces bad results for both players.

However, even if technical panels are established within the framework of broad governmental directives, there is no way to force governments to accept their awards, apart from the constraint that rejection would almost necessarily invoke the disadvantageous non-cooperative solution. Actually, the reference of tough problems to technical experts may prove politically advantageous to democratic governments in dealing with public opinion and legislatures. It may enable such governments to force acceptance of the arbitral award as preferable to the practicable alternatives while disclaiming responsibility except for the broad instructions to the experts. And usually those instructions will be of a character that will appeal to the public. Thus, by limiting its alternatives, a government may acquire political strength in the same way in which stubborn players acquire bargaining strength in conflicts with less stubborn opponents.

The proposal to use experts may seem unreasonable for a reason noted earlier. Experts have no power to enforce their decisions, while the United States and the Soviet Union possess such enormous capabilities that even the positions of other nations may not exercise significant influence on their decisions. Yet there are many situations where the cost of not cooperating is so high in general that serious efforts may be made to neutralize the situation. Claim rights in the Antarctic region may fall into this class. Provided that the umpire or arbiter has reasonably clear rules to follow--although one must not fall into the error of the advocates of mechanical jurisprudence in ignoring the fact that the choice of premises of rules of interpretation can never, in any practicable system, be devoid of problems of choice--some of the major difficulties may be overcome.

-37-

Final Considerations


Games requiring coordination are extremely technical in nature. And to play them properly may depend upon considerable control over nature. The nation as a player must have rational control over its resources in order to optimize the results in the game. But simple optimization is not as important as the fact that in this kind of game the rational control of resources and technical understanding of the game may make the difference between satisfactory and unsatisfactory payoffs, between cooperation and conflict. Coordination in strategy and restructuring of the matrix within which international decisions are made will depend upon great ingenuity in finding sets of signals which permit the players to have joint confidence in the safety of the cooperative solutions.

Anyone familiar with bargaining knows that clarity is difficult enough to obtain in the communication process and that the discovery of signals that permit safe use of cooperative strategies is doubly difficult. The discovery and implementation of such cooperative solutions will depend upon a receptiveness to novel modes of behavior that cut across the grain of our normal patterns of behavior. The willingness in international politics to give secrets of major importance to one's most deadly rival strikes at one of the guiding precepts of ordinary common sense and requires an understanding of the most important aspects of the cooperative and non-cooperative games. The seemingly fantastic or bizarre must become the focus of attention; and the "odd balls" who have sufficient flexibility and suppleness of mind to conceive the fantastic and to spin out streams of ideas--many of them genuinely odd or useless, but some of them essential to the cooperative solution--and to use their minds creatively must acquire the technical skills to work their ideas out. But this requires insulation from the ordinary and common, as well as the vulgar, pressures of politics and public opinion. This means that the right minds must be selected and, within the most general limitations of policy, given the authority to work out and to implement the solutions.

It is hardly necessary to add that if political practicability be the test of realism, most of the suggestions made here are not realistic. Indeed, further analysis may even establish technical difficulties or faults. Surely, however, there must be some virtue in the attempt to explore the nature of the strategies necessary to produce coordination and cooperation in the nuclear age.