Polynomial-Time Optimal Equilibria with a Mediator in Extensive-Form Games
Abstract
For common notions of correlated equilibrium in extensive-form games, computing an optimal ( e.g. , welfare-maximizing) equilibrium is NP-hard.
Other equilibrium notions— communication [11] and certiﬁcation [12] equilibria —augment the game with a mediator that has the power to both send and receive messages to and from the players—and, in particular, to remember the messages.
In this paper, we investigate both notions in extensive-form games from a computational lens.
We show that optimal equilibria in both notions can be computed in polynomial time, the latter under a natural additional assumption known in the literature.
Our proof works by constructing a mediator-augmented game of polynomial size that explicitly represents the mediator’s decisions and actions.
Our framework allows us to deﬁne an entire family of equilibria by varying the mediator’s information partition, the players’ ability to lie, and the players’ ability to deviate.
From this perspective, we show that other notions of equilibrium, such as extensive-form correlated equilibrium, correspond to the mediator having imperfect recall .
This shows that, at least among all these equilibrium notions, the hardness of computation is driven by the mediator’s imperfect recall.
As special cases of our general construction, we recover 1) the polynomial-time algorithm of Conitzer and Sandholm [8] for automated mechanism design in Bayes-Nash equilibria and 2) the correlation DAG algorithm of Zhang et al.
[31] for optimal correlation.
Our algorithm is especially scalable when the equilibrium notion is what we deﬁne as the full-certiﬁcation equilibrium, where players cannot lie about their information but they can be silent.
We back up our theoretical claims with experiments on a suite of standard benchmark games.
Introduction
Various equilibrium notions in general-sum extensive-form games are used to describe situations where the players have access to a trusted third-party mediator , who can communicate with the players.
Depending on the power of the mediator and the form of communication, these notions include the normal-form [1] and extensive-form correlated equilibrium (NFCE and EFCE) [29], the normal-form [25] and extensive-form [10] coarse-correlated equilibrium (NFCCE and EFCCE), the communication equilibrium [11], and the certiﬁcation equilibrium [12].
Several of these notions, in particular the EFCE and EFCCE, were deﬁned for mainly computational reasons: the EFCE as a computationally-reasonable relaxation to NFCE, and the EFCCE as a computationally-reasonable relaxation of EFCE.
When the goal is to compute a single correlated equilibrium, these relaxations are helpful: there are polynomial-time algorithms for computing an
EFCE [16].
However, from the perspective of computing optimal equilibria—that is, equilibria that maximize the expected value of a given function, such as the social welfare—even these relaxations fall short: for all of the correlation notions above, computing an optimal equilibrium of an extensive-form game is NP-hard [29, 10].
On the other hand, notions of equilibrium involving communication in games have arisen.
These differ from the notions of correlation in that the mediator can receive and remember information from the players, and therefore pass information between players as necessary to back up their suggestions.
Certiﬁcation equilibria [12] further strengthen communication equilibria by allowing players to prove certain information to the mediator.
To our knowledge, the computational complexity of optimal communication or certiﬁcation equilibria has never been studied.
We do so in this paper.
The main technical result of our paper is a polynomial-time algorithm for computing optimal communication and certiﬁcation equilibria (the latter under a certain natural condition about what messages the players can send).
This stands in stark contrast to the notions of correlation discussed above.
To prove our main result, we deﬁne a general class of mediator-augmented games , each having polynomial size, that is sufﬁcient to describe all of the above notions of equilibrium except the
NFCE 1 .
We also build on this main result in several ways.
We deﬁne the full-certiﬁcation equilibrium, which is the special case in which players cannot lie to the mediator (but can opt out of revealing their information).
In this case, the algorithm is a linear program whose size is almost linear in the size of the original game.
As such, this special case scales extremely well compared to all of the other notions.
We formalize notions for incorporating payments in the language of our augmented game.
By using payments, mediators can incentivize players to play differently than they otherwise would, possibly to the beneﬁt of the mediator’s utility function.
We deﬁne an entire family of equilibria using our augmented game, that includes as special cases the communication equilibrium, certiﬁcation equilibrium, NFCCE, EFCCE
and EFCE.
From this perspective, we show that other notions of equilibrium, such as extensiveform correlated equilibrium, correspond to the mediator having imperfect recall .
This shows that, at least among all these equilibrium notions, the hardness of computation is driven by the mediator’s imperfect recall.
We argue that, for this reason, many stated practical applications of correlated equilibria should actually be using communication or certiﬁcation equilibria instead, which are both easier to compute (in theory, at least) and better at modelling the decision-making process of a rational mediator.
We empirically verify the above claims via experiments on a standard set of game instances.
Applications and related work.
Correlated and communication equilibria have various applications that have been well-documented.
Here, we discuss just a few of them, as motivation for our paper.
For further discussion of related work, especially relating to automated dynamic mechanism design and persuasion, see Appendix F.
Bargaining, negotiation, and conﬂict resolution [4, 9].
Two parties with asymmetric information wish to arrive at an agreement, say, the price of an item.
A mediator, such as a central third-party marketplace, does not know the players’ information but can communicate with the players.
Crowdsourcing and ridesharing [13, 22, 31].
A group of players each has individual goals ( e.g. , to make money by serving customers at speciﬁc locations).
The players are coordinated by a central party ( e.g. , a ridesharing company) that has more information than any one of the players, but the players are free to ignore recommendations if they so choose.
Persuasion in games [17, 3, 23, 14, 30].
The mediator (in that literature, usually “sender”) has more information than the players (“receivers”), and wishes to tell information to the receivers so as to persuade them to act in a certain way.
Automated mechanism design [6, 8, 33, 35, 26, 34, 18, 19].
Players have private information unknown to the mediator.
The mediator wishes to commit to a strategy—that is, set a mechanism— such that players are incentivized to honestly reveal their information.
In fact, in Appendix E we will see that we recover the polynomial-time Bayes mechanism design algorithm of [6, 8] as a special case of our main result.
Some of the above examples are often used to motivate correlated equilibria.
However, when the mediator is a rational agent with the ability to remember information that it is told and pass the information between players as necessary, we will argue that communication or certiﬁcation equilibrium should be the notion of choice, for both conceptual and computational reasons.
Preliminaries
In this section, we discuss background on correlation in extensive-form games.
Extensive-form games.
An extensive-form game Γ with n players consists of the following.
A directed tree of nodes or histories H , whose root is denoted ∅ .
The depth of the tree will be denoted T .
The edges out of nodes are labeled with actions , and the set of such actions will be denoted A h .
Given a node h ∈ H and action a at h , the child reached by following action a at node h is denoted ha .
The set of terminal (leaf) nodes in H is denoted Z .
Terminal nodes will always be denoted z throughout the paper.
A partition H \ Z = H C (cid:116)
H 1 (cid:116) · · · (cid:116) . . .
H n of nodes, where H i is the set of all nodes at which player i plays and player H C is the set of chance nodes.
For each player i , a partition
I i of player
i ’s decision nodes, H i , into information sets or infosets .
Every node in a given information set I must have the same set of actions, denoted A I .
We will call the partition
I =
I 1 (cid:116) · · · (cid:116) I n the players’ information partition .
For each player i , a utility vector u i ∈ [0 , 1] Z , where u i [ z ] denotes the utility achieved by player i at terminal node z . 5.
For each chance node h ∈ H C , a probability distribution p ( ·| h ) over the children of h .
The sequence σ i ( h ) is the list of infosets reached by player i , and actions taken by the player i at those infosets, on the ∅ → h path, not including the infoset at h itself (if any).
We will assume that each player has perfect recall —that is, for each infoset I , the sequence of the player acting at I should be the same for each node in I .
We will denote this sequence σ ( I ) .
In perfect-recall games, nonempty sequences will be identiﬁed by the last infoset-action pair Ia in them.
We also will assume that games are timeable and ﬁxed-turn-order , that is, information sets do not span multiple levels of the tree, and all nodes in the same layer of the tree belong to the same player 2 .
We will use the following notation.
The relation (cid:22) denotes the natural precedence order induced by the tree H : we write h (cid:22) h (cid:48) means that h is an ancestor of h (cid:48) (or h = h (cid:48) ), and for sets S, S (cid:48) , we say S (cid:22) S (cid:48) if there are some h ∈ S, h (cid:48) ∈ S (cid:48)
such that h (cid:22) h (cid:48) .
The binary operation ∧ denotes the lowest common ancestor: h ∧ h (cid:48) is the lowest node u such that u (cid:22) h, h (cid:48) .
For sequences, σ ( h ) =
( σ 1 ( h ) , . . .
, σ n ( h )) denotes the joint sequence of all players at node h .
N ( σ ) denotes the set of possible next infosets following sequence σ , that is, N ( σ )
= { I : σ ( I ) = σ } .
The set Σ i denotes the set of sequences of player i , and Σ denotes the set of all sequences across all players (i.e., Σ = (cid:116) i Σ i ).
A pure strategy for a player i is a selection of one action for each information set I ∈
I i .
A pure proﬁle is a tuple of pure strategies.
A correlated proﬁle is a distribution over pure proﬁles.
We will generally work with strategies in realization form (see e.g., Koller et al.
Given a pure strategy x , we say that x plays to z ∈ Z if x plays every action on the ∅ → z path.
We will call the vector x ∈ { 0 , 1 } Z the realization form of x .
The realization form of a mixed strategy is the appropriate convex combination.
The set of mixed strategies forms a convex subset of R Z that, so long as the player has perfect recall, can be expressed using linearly many constraints and variables.
We will occasionally need to discuss changing information partitions of Γ .
If J = J 1 (cid:116) · · · (cid:116) J n is another valid information partition, we will use Γ J to denote the game Γ with its information partition replaced by J .
We will also occasionally need to talk about multiple games simultaneously; where this is the case, we will mark attributes of the game the same as the game itself.
For example, ˆ
H is the node set of game ˆΓ .
Communication and certiﬁcation equilibria.
Here, we review deﬁnitions related to communication equilibria , following Forges [11] and later related papers.
Deﬁnition 2.1.
Let S be a space of possible messages .
A pure mediator strategy is a map d : S ≤ T →
S , where S ≤ T denotes the set of sequences in S of length at most T .
A randomized mediator strategy (hereafter simply mediator strategy ) is a distribution over pure mediator strategies.
We will assume that the space of possible messages is large, but not exponentially so.
In particular, we will assume that {⊥} ∪ I ∪ (cid:83) h
A h ⊆ S ( i.e. , messages can at least be nothing, information, or actions)
and that | S | ≤ poly( |H| ) .
The latter assumption is mostly for cleanliness in stating results: we will give algorithms that need S as an input that we wish to run in time poly( |H| ) .
A mediator strategy augments a game as follows.
If the strategy is randomized, it ﬁrst samples a pure strategy d , which is hidden from the players.
At each timestep t , a player reaches a history h at which she must act, and observes the infoset I (cid:51) h .
She sends a message s t ∈ S to the mediator.
The mediator then sends a response d ( s 1 , . . .
, s t ) , which depends on the message s t as well as the messages sent by all other players prior to timestep t .
Then, the player chooses her action a ∈ A h .
We will call the sequence of messages sent and received between the mediator and player i , the transcript with player i .
A communication equilibrium 4 is a Nash equilibrium of the game Γ augmented with a mediator strategy.
The mediator is allowed to perform arbitrary communication with the players.
In particular, the mediator is allowed to pass information from one player to another .
Further, the players are free to send whatever messages they wish to the mediator, including false or empty messages.
These two factors distinguish communication equilibria from notions of correlated equilibria .
In Section 3.4 we will discuss this comparison in greater detail.
A useful property in the literature on communication equilibria is the revelation principle ( e.g. , [11]).
Informally, the revelation principle states that any outcome achievable by an arbitrary strategy proﬁle can also be achieved by a direct strategy proﬁle, in which the players tell the mediator all their information and are subsequently directly told by the mediator which action to play.
In order to be a communication equilibrium, the players still must not have any incentive to deviate from the protocol.
That is, the equilibrium must be robust to all messages that a player may attempt to send to the mediator, even if in equilibrium the player always sends the honest message.
Forges and Koessler [12] further introduced a form of equilibrium for Bayesian games which they called certiﬁcation equilibria .
In certiﬁcation equilibria, the messages that a player may legally send are dependent on their information; as such, some messages that a player can send are veriﬁable .
At each information set I ∈
I , let S
I ⊆
S denote the set of messages that the player at infoset I may send to the mediator.
We will always assume that I ∈ S I and ⊥ ∈ S I for all I .
That is, all players always have the options of revealing their true information or revealing nothing.
Extensive-form S -certiﬁcation equilibria
The central notion of interest in this paper is a generalization of the notion of certiﬁcation equilibria [12] to extensive-form games.
Deﬁnition 3.1.
Given an extensive-form game Γ and a family of valid message sets S =
{ S I : I ∈
I} , an S -certiﬁcation equilibrium is a Nash equilibrium of the game augmented by a randomized mediator, in which each player at each information set I is restricted to sending a message s ∈ S I .
The existence of S -certiﬁcation equilibria follows from the existence of Nash equilibria, which are the special case where the mediator does nothing.
We will need one extra condition on the message sets, which is known as the nested range condition (NRC) [15]: if I ∈ S I (cid:48) , then S I ⊆
S I (cid:48) .
That is, if a player with information I (cid:48) can lie by pretending to have information I , then that player can also emulate any other message she would have been able to send at I .
Equivalently, the honest message I should be the most certiﬁable message that a player can send at infoset I .
Our main result is the following.
Theorem 3.2.
Let u M ∈
R
Z be an arbitrary utility vector for the mediator.
Then there is a polynomial-time algorithm that, given a game Γ and a message set family S satisfying the nested range condition, computes an optimal S -certiﬁcation equilibrium, that is, one that maximizes E z u M [ z ] where the expectation is over playouts of the game under equilibrium.
In particular, by setting S I = S for all I , Theorem 3.2 implies that optimal communication equilibria can be computed in polynomial time.
The rest of the paper is organized as follows.
First, we will prove our main theorem.
Along the way, we will demonstrate a form of revelation principle for S -certiﬁcation equilibria.
We will then discuss comparisons to other known forms of equilibrium, including the extensive-form correlated equilibrium [29], and several other natural extensions of our model.
Finally, we will show experimental results that compare the computational efﬁciency and social welfare of various notions of equilibrium on some experimental game instances.
3.1 Proof of Theorem 3.2: The single-deviator mediator-augmented game
In this section, we construct a game ˆ Γ , with n + 1 players, that describes the game Γ where the mediator has been added as an explicit player.
This game has similar structure to the one used by Forges [11, Corollary 2], but, critically, has size polynomial in |H| .
This is due to two critical differences.
First, the players are assumed to either send ⊥ , or send messages that mediator cannot immediately prove to be off-equilibrium.
In particular, if the player’s last message was I and the mediator recommended action a
at I , the player must send a message I (cid:48) with σ ( I (cid:48) )
=
Ia .
If this is impossible, the player must send ⊥ .
Therefore, in particular, we will assume that S I consists of only ⊥ and information sets I (cid:48) at the same level as I .
Second, only one player is allowed to deviate.
Therefore, the strategy of the mediator is not deﬁned in cases where two or more players deviate.
We now formalize ˆΓ .
Nodes in ˆΓ will be identiﬁed by tuples ( h, τ , r ) where h ∈ H is a history in Γ , τ = ( τ 1 , . . .
, τ n ) is the collection of transcripts with all players, and r ∈ { REV , REC , ACT } is a stage marker that denotes whether the current state is one in which a player should be revealing information ( REV ), the mediator should be recommending a move ( REC ), or the player should be selecting an action ( ACT ).
The progression of ˆΓ is then deﬁned as follows.
We will use the notation τ
[ i · s ] to denote appending message s to τ i .
• The root node of ˆΓ is ( ∅ , ( ∅ , . . . , ∅ ) , REV ) .
• Nodes ( z, τ , REV ) for z ∈ Z are also terminal in Γ .
The mediator gets utility u M [ z ] , where u is the mediator’s utility function as in Theorem 3.2.
All other players i get utility u
i [ z ] .
• Nodes ( h, τ , REV ) for non-terminal h are decision nodes for the player i who acts at h . 1.
If i is chance, there is one valid transition, to ( h, τ , ACT ) .
If some other player j (cid:54) = i has already deviated (i.e., σ j ( h ) (cid:54) = τ j ) ), there is one valid transition, to ( h, τ
[ i · I ] , REC ) where I (cid:51) h . 3.
If player i has deviated or no one has deviated, then player i observes the infoset I (cid:51) h , and selects a legal message I (cid:48) ∈ S
I ∩ ( {⊥} ∪ N ( τ i )) to send to the mediator 5 .
Transition to ( h, τ
[ i · I (cid:48) ] , REV ) .
• At ( h, τ , REC ) where h ∈ H i , the mediator observes the transcript τ
i and makes a recommendation a .
If τ i contains any ⊥ messages, then a ⊥ .
Otherwise, a is a legal action
a ∈
A I , where I is the most recent message in τ i .
Transition to ( h, τ
[ i · a ] , ACT ) .
• Nodes ( h, τ , ACT ) for non-terminal h are decision nodes for the player i who acts at h . 1.
If i is chance, then chance samples a random action a ∼ p ( ·| h ) .
Transition to ( ha, τ , REV ) .
If some other player j (cid:54) = i has already deviated, there is one valid transition, to ( ha, τ , REC ) , where a is the action sent by the mediator.
If player i has deviated or no one has deviated, then player i observes the transcript τ i , and selects an action a {+(cid:
 )+} ∈
A h .
Transition to ( ha (cid:48) , τ , REV ) .
The action a (cid:48) need not be the recommended action.
Since at most one player can ever deviate by construction, and the length of the transcripts are ﬁxed because turn order is common knowledge, the transcripts τ can be identiﬁed with sequences σ i of the deviated player, if any.
We will make this identiﬁcation: we will use the shorthand h
σ i to denote the history ( h, ( σ − i ( h ) , σ i ) , REV ) , and h ⊥ for ( h, σ ( h ) , REV ) (i.e., no one has deviated yet).
Therefore, in particular, this game has at most O ( |H|| Σ | ) histories.
For each non-mediator player, there is a well-deﬁned direct strategy ˆ x ∗ i for that player: always report her true information
I (cid:51) h , and always play the action recommended by the mediator.
The goal of the mediator is to ﬁnd a strategy ˆ x M for itself that maximizes its expected utility, subject to the constraint that each player’s direct strategy is a best response —that is, ﬁnd ˆ x M such that ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) is a (strong) Stackelberg equilibrium of ˆΓ .
We claim that ﬁnding a mediator strategy ˆ x M that is a strong Stackelberg equilibrium in ˆΓ is equivalent to ﬁnding an optimal S -certiﬁcation equilibrium in Γ .
We prove this in two parts.
First, we prove a version of the revelation principle for S -certiﬁcation equilibria.
Deﬁnition 3.3.
An S -certiﬁcation equilibrium is direct if it satisﬁes the following two properties.
( Mediator directness )
If the transcript τ i of a player i is exactly some sequence of player i , and player i sends an infoset I with σ ( I )
=
τ i , then the mediator replies with an action a ∈ A I .
Otherwise 6 , the mediator replies ⊥
( Player directness )
In equilibrium, players always send their true information I , and, upon receiving an action
a ∈
A I , always play that action
Proposition 3.4 (Revelation principle for S -certiﬁcation equilibria under NRC) .
Assume that S satisﬁes the nested range condition.
For any S -certiﬁcation equilibrium, there is a realizationequivalent direct equilibrium.
Omitted proofs can be found in the appendix.
Since direct mediator strategies are exactly the mediator strategies in ˆΓ , and the player strategies are only limited versions of what they are allowed to do in S -certiﬁcation equilibrium, this implies that, for any S -certiﬁcation equilibrium, there is a mediator strategy ˆ x M in ˆΓ such that ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) is a Stackelberg equilibrium.
We will also need the converse of this statement.
Proposition 3.5.
Let ˆ x M be a strategy for the mediator in ˆΓ such that, in the strategy proﬁle ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) , every ˆ x ∗ i for i (cid:54) =
M is a best response.
Then there is an direct S -certiﬁcation equilibrium that is realization-equivalent to ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) .
Therefore, we have shown that the mediator strategies ˆ x M in ˆΓ for which ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) is a
Stackelberg equilibrium in ˆΓ correspond exactly to optimal S -certiﬁcation equilibria of Γ .
Such a
Stackelberg equilibrium can be found by solving the following program:
[Equation]
where ˆ X i is the sequence-form strategy space [20] of player i in ˆΓ .
The only variables in the program are ˆ x i for each player i and the mediator.
In particular, the direct strategies ˆ x ∗ i are constants.
Therefore, the objective is a linear function, and the inner maximization constraints are bilinear in ˆ x M and ˆ x j .
Therefore, this program can be converted to a linear program by dualizing the inner optimizations.
For more details on this conversion, see
Appendix B.
The result is a linear program of size O ( n | ˆ H| )
=
O ( n |H|| Σ | ) .
We have thus proved Theorem 3.2.
3.2 Extensions and special cases
In this section, we describe several extensions and interesting special cases of our main result.
Full-certiﬁcation equilibria.
One particular special case of S -certiﬁcation equilibria which is particularly useful.
We deﬁne a full-certiﬁcation equilibrium as an S -certiﬁcation equilibrium where
S I = {⊥ , I } .
Intuitively, this means that players cannot lie to the mediator, but they may withhold information.
We will call such an equilibrium full-certiﬁcation .
Removing valid messages from the players only reduces their ability to deviate and thus increases the space of possible equilibrium strategies.
As such, the full-certiﬁcation equilibria are the largest class of S -certiﬁcation equilibria.
For full-certiﬁcation equilibria, the size of game ˆΓ reduces dramatically.
Indeed, in all histories h Ia of ˆΓ , we must have I (cid:22) h .
Therefore, we have | ˆ H| ≤ |H| BD where B is the maximum branching factor and D is the depth of the game tree, i.e. , the size of ˆΓ goes from essentially quadratic to essentially quasilinear in |H| .
The mediator’s decision points in ˆΓ for a full-certiﬁcation equilibrium are the trigger histories used by Zhang et al.
[31] in their analysis of various notions of correlated equilibria.
Later, we will draw further connections between full certiﬁcation and correlation.
Changing the mediator’s information.
In certain cases, the mediator, in addition to messages that it is sent by the players, also has its own observations about the world.
These are trivial to incorporate into our model: simply change the information partition of the mediator in ˆΓ as needed.
Alternatively, one can imagine adding a “player”, with no rewards (hence no incentive to deviate), whose sole purpose is to observe information and pass it to the mediator.
For purposes of keeping the game small, it is easier to adopt the former method.
To this end, consider any reﬁnement partition
M of the mediator infosets in ˆΓ , and consider the game ˆΓ M created by replacing the mediator’s information partition in ˆΓ with M .
Then we make the following deﬁnition.
Deﬁnition 3.6.
An ( S , M ) -certiﬁcation equilibrium of Γ is a mediator strategy ˆ x M in ˆΓ M such that, in the strategy proﬁle (ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) , every x ∗ i for i (cid:54) =
M is a best response.
( S , M ) -certiﬁcation equilibria may not exist: indeed, if M is coarser than the mediator’s original information partition in ˆΓ , then the mediator may not have enough information to provide good recommendations under the restrictions of ˆΓ .
This can be remedied by allowing payments (see
Appendix E), or by making the assumption that the mediator at least knows the transcript of the player to whom she is making any nontrivial recommendation:
Deﬁnition 3.7.
A mediator partition M is direct if, at every mediator decision point ( h, τ , REC ) , so long as | A h | > 1 , the mediator knows the transcript of the player acting at h .
M is strongly direct if the mediator also observes the transcript when | A h | = 1 .
The condition | A h | > 1 in the deﬁnition allows the mediator to possibly not observe the full information of a player if she does not need to make a nontrivial recommendation to that player.
In particular, this allows players to sometimes have information that they only partially reveal to the mediator, so long as the player does not immediately need to act on such information.
Coarseness.
In literature on correlation, coarseness refers to the restriction that a player must obey any recommendation that she receives (but may choose to deviate by not requesting a recommendation and instead playing any other action).
Normal-form coarseness further adds the restriction that players can only choose to deviate at the start of the game—the mediator essentially takes over and plays the game on behalf of non-deviating players.
These notions can easily be expressed in terms of our augmented game, therefore also allowing us to express coarse versions of our equilibrium notions as augmented games.
3.3 The gap between polynomial and not polynomial
If players cannot send messages to the mediator at all, and the mediator has no other way of gaining any information, we recover the notion of autonomous correlated equilibrium (ACE) .
It is NP-hard to compute optimal ACE, even in Bayesian games (see e.g. , von Stengel and Forges [29]).
When M is direct and perfect recall, computing an optimal direct ( S , M ) -certiﬁcation equilibrium can be done in polynomial time using our framework.
When S obeys NRC and M satisﬁes a stronger condition 7 , the proof of the revelation principle (Propositions 3.4 and 3.5) works, and the resulting equilibrium is guaranteed to be optimal over all possible equilibria including those that may not be direct.
If NRC does not hold, one can still solve the program (1), and the solution is still guaranteed to be an optimal direct equilibrium by Proposition 3.5.
However, it is not guaranteed to be optimal over all possible communication structures.
Indeed, Green and Laffont [15, Theorem 1] give an instance in which, without NRC, there can be an outcome distribution that is not implementable by a direct mediator.
Our program cannot ﬁnd such an outcome distribution.
The counterexample does not preclude the possibility of efﬁcient algorithms for ﬁnding optimal certiﬁcation equilibria in more general cases, but does give intuition for why NRC is crucial to our construction.
We could also consider changing the mediator’s information partition so that the mediator does not have perfect recall.
This transformation allows us to recover notions of correlation in games.
Indeed, if we start from the full-certiﬁcation equilibrium and only allow the mediator to remember the transcript with the player she is currently talking to, we recover EFCE.
Adding coarseness similarly recovers EFCCE and NFCCE.
In this setting, the inability to represent the strategy space of an imperfect-recall player may result in the loss of efﬁcient algorithms.
3.4 A family of equilibria
By varying 1) what the mediator observes, 2) whether the mediator has perfect recall, 3) whether the players can lie or only withhold information, and 4) when and how players can deviate from the mediator’s recommended actions, we can use our framework to deﬁne a family consisting ofconceptually different equilibrium notions.
More can be generated by considering other variations in this design space, but we focus on the extreme cases in the table.
Some of these were already deﬁned in the literature; the remaining names are ours.
The result is Table 1.
An inclusion diagram for these notions can be found in Appendix G.
In the table, ex ante means that players have only a binary choice between deviating (in which case they can play whatever they want) and playing (in which case they must always be direct and obey recommendations).
With ex ante deviations, it does not matter whether lying is allowed because we can never get to that stage: either the player deviates immediately and never communicates with the mediator, or the player is direct.
If the mediator only remembers the current active player’s information, and players cannot lie, withholding and coarsely deviating are the same.
Mediator information advantage means that the mediator always learns the infoset of the current active player, and therefore requires no messages from the players.
This is equivalent to forcing players to truthfully report information.
A mediator with information advantage may still not have perfect information—for example, it will not know whether a player (or nature) has played an action until some other player observes the action.
In this setting, the mediator may also have extra private information (known to none of the players), leading to the setting of Bayesian persuasion [17].
In extensive-form games, there are two different reasonable notions of persuasion: one that stems from extending correlated equilibria, and one that stems from extending communication equilibria.
The distinction is that, in the former, the mediator has imperfect recall.
For a more in-depth discussion of Bayesian persuasion, see Appendix F.
Our framework allows optimal equilibria for all notions in the table to be computed.
For perfectrecall mediators, this is possible in polynomial time via the sequence form; for imperfect-recall mediators, the problem is NP-hard, in general, but the team belief DAG of Zhang et al.
[32] can be used to recover ﬁxed-parameter algorithms.
For the notions of correlated equilibrium, this method results in basically the same LP as the correlation DAG of Zhang et al.
We do not claim that all of these notions are easy to motivate.
For example, correlated equilibria are usually arrived at in the “truth known, imperfect recall” setting; the correlated equilibrium notions where lying is allowed are more difﬁcult to motivate in this respect.
Further, even the ﬁxedparameter algorithms of Zhang et al.
[31] would fail in this setting, because “public states” can no longer be treated as public due to the possibility of lying players.
We leave to future research the problem of ﬁnding a motivation for the notions that we do not reference elsewhere in the paper.
[Table] [Table]
Experiments
We ran our algorithm for communication and full-certiﬁcation equilibria on various two-player games
[Table]
and compared the results to those given by notions of optimal correlation in games.
The games used in the experiments are given in Appendix D. All experiments were allocated four CPU cores and 64 GB of RAM.
Linear programs were solved with Gurobi 9.5.
When payments are used, the allowable payment range is [0 , M ] where M is the reward range of the game.
Experimental results can be found in Table [Table]
In the battleship and sheriff instances
[Table]
there is not a signiﬁcant difference in performance between ﬁnding full-certiﬁcation equilibria and ﬁnding optimal correlated equilibria in terms of performance—this is because, unlike in the general case, optimal correlated equilibria in two-player games without chance can be found in polynomial time [29] anyway.
In the ridesharing instances, included here for purposes of comparison. When payments are used, the mediator is informed before making the payment of whether the player was honest [Table] and the optimization objective is the social welfare of the original terminal state, minus any payments made.
“oom” is out of memory.
[Table]
.02s 0.04s 0.01s 0.01s 0.02s
0.11s 0.08s 0.20s 0.85s
1.74s
S 2376 value
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
42.000 time 0.04s 0.23s 0.65s 0.03s 0.07s 0.12s 0.25s 0.46s [Table] .04s 1m13s 1m49s
S 5632 value
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
time
[Table]
oom RS
[Table]
value
[Table] [Table]
[Table] [Table] [Table] [Table]
[Table] [Table]
173 time
[Table]
91s
[Table]
77s
[Table]
4356 value 9.
[Table] [Table] [Table] [Table]
[Table] [Table] [Table] [Table]
[Table]
592 time
[Table]
82s
[Table]
55s
[Table]
RS
[Table]
value oom oom oom
[Table]
[Table] [Table] [Figure] [Figure]
[Figure]
u t i l i t
[Figure]
→
[Figure]
battleship
[Figure] sheriff [Figure] rideshare
[Figure]
computing optimal correlated equilibria is much more computationally intensive because the game contains non-public chance actions.
Computing optimal full-certiﬁcation equilibria is comparably easy, and this difference is clearly seen in the timing results.
Finding optimal communication equilibria is much more intensive than ﬁnding optimal fullcertiﬁcation equilibria, owing to the quadratic size of the augmented game for communication equilibria.
This often causes communication equilibria to be the hardest of the notions to compute in practice, despite optimal correlation being NP-hard.
In Figure 1, we have plotted the payoff spaces of some representative instances.
The plots show how the polytopes of communication and full-certiﬁcation equilibria behave relative to correlated equilibria.
In the battleship and sheriff instances, the space of communication equilibrium payoffs is a single point, which implies that the space of NFCE (and hence Nash) equilibrium payoffs is also that single point.
Unfortunately, that point is the Pareto-least-optimal point in the space of EFCEs.
In the ridesharing instances, communication allows higher payoffs.
This is because the mediator is allowed to “leak” information between players.
Conclusions and future research
We have shown that optimal communication and certiﬁcation equilibria in extensive-form games can be computed via linear programs of polynomial size, or almost-linear size in the full-certiﬁcation case.
We have used our machinery to derive an entire family of equilibrium concepts which we hope to be of use in the future.
Possible future directions include the following.
Are there efﬁcient online learning dynamics , in any reasonable sense of that term, that converge to certiﬁcation or communication equilibrium?
Is there a better-than-quadratic-size linear program for communication equilibria?
Is it possible to extend our augmented game construction to also cover normal-form correlated equilibria while maintaining efﬁciency?
Investigate further the comparison between communication and correlation in games.
For example, when and why do communication equilibria achieve higher social welfare than extensive-form correlated equilibria?
Acknowledgements
This material is based on work supported by the National Science Foundation under grants
IIS-1901403 and CCF-1733556, and the ARO under award W911NF2010081.