Polynomial-Time Optimal Equilibria with a Mediator in Extensive-Form Games
Abstract
For common notions of correlated equilibrium in extensive-form games, computing an optimal ( e.g. , welfare-maximizing) equilibrium is NP-hard.
Other equilibrium notions— communication [11] and certification [12] equilibria —augment the game with a mediator that has the power to both send and receive messages to and from the players—and, in particular, to remember the messages.
In this paper, we investigate both notions in extensive-form games from a computational lens.
We show that optimal equilibria in both notions can be computed in polynomial time, the latter under a natural additional assumption known in the literature.
Our proof works by constructing a mediator-augmented game of polynomial size that explicitly represents the mediator’s decisions and actions.
Our framework allows us to define an entire family of equilibria by varying the mediator’s information partition, the players’ ability to lie, and the players’ ability to deviate.
From this perspective, we show that other notions of equilibrium, such as extensive-form correlated equilibrium, correspond to the mediator having imperfect recall .
This shows that, at least among all these equilibrium notions, the hardness of computation is driven by the mediator’s imperfect recall.
As special cases of our general construction, we recover 1) the polynomial-time algorithm of Conitzer and Sandholm [8] for automated mechanism design in Bayes-Nash equilibria and 2) the correlation DAG algorithm of Zhang et al.
[31] for optimal correlation.
Our algorithm is especially scalable when the equilibrium notion is what we define as the full-certification equilibrium, where players cannot lie about their information but they can be silent.
We back up our theoretical claims with experiments on a suite of standard benchmark games.
Introduction
Various equilibrium notions in general-sum extensive-form games are used to describe situations where the players have access to a trusted third-party mediator , who can communicate with the players.
Depending on the power of the mediator and the form of communication, these notions include the normal-form [1] and extensive-form correlated equilibrium (NFCE and EFCE) [29], the normal-form [25] and extensive-form [10] coarse-correlated equilibrium (NFCCE and EFCCE), the communication equilibrium [11], and the certification equilibrium [12].
Several of these notions, in particular the EFCE and EFCCE, were defined for mainly computational reasons: the EFCE as a computationally-reasonable relaxation to NFCE, and the EFCCE as a computationally-reasonable relaxation of EFCE.
When the goal is to compute a single correlated equilibrium, these relaxations are helpful: there are polynomial-time algorithms for computing an
EFCE [16].
However, from the perspective of computing optimal equilibria—that is, equilibria that maximize the expected value of a given function, such as the social welfare—even these relaxations fall short: for all of the correlation notions above, computing an optimal equilibrium of an extensive-form game is NP-hard [29, 10].
On the other hand, notions of equilibrium involving communication in games have arisen.
These differ from the notions of correlation in that the mediator can receive and remember information from the players, and therefore pass information between players as necessary to back up their suggestions.
Certification equilibria [12] further strengthen communication equilibria by allowing players to prove certain information to the mediator.
To our knowledge, the computational complexity of optimal communication or certification equilibria has never been studied.
We do so in this paper.
The main technical result of our paper is a polynomial-time algorithm for computing optimal communication and certification equilibria (the latter under a certain natural condition about what messages the players can send).
This stands in stark contrast to the notions of correlation discussed above.
To prove our main result, we define a general class of mediator-augmented games , each having polynomial size, that is sufficient to describe all of the above notions of equilibrium except the
NFCE 1 .
We also build on this main result in several ways.
We define the full-certification equilibrium, which is the special case in which players cannot lie to the mediator (but can opt out of revealing their information).
In this case, the algorithm is a linear program whose size is almost linear in the size of the original game.
As such, this special case scales extremely well compared to all of the other notions.
We formalize notions for incorporating payments in the language of our augmented game.
By using payments, mediators can incentivize players to play differently than they otherwise would, possibly to the benefit of the mediator’s utility function.
We define an entire family of equilibria using our augmented game, that includes as special cases the communication equilibrium, certification equilibrium, NFCCE, EFCCE
and EFCE.
From this perspective, we show that other notions of equilibrium, such as extensiveform correlated equilibrium, correspond to the mediator having imperfect recall .
This shows that
, at least among all these equilibrium notions, the hardness of computation is driven by the mediator’s imperfect recall.
We argue that, for this reason, many stated practical applications of correlated equilibria should actually be using communication or certification equilibria instead, which are both easier to compute (in theory, at least) and better at modelling the decision-making process of a rational mediator.
We empirically verify the above claims via experiments on a standard set of game instances.
Applications and related work.
Correlated and communication equilibria have various applications that have been well-documented.
Here, we discuss just a few of them, as motivation for our paper.
For further discussion of related work, especially relating to automated dynamic mechanism design and persuasion, see Appendix F.
Bargaining, negotiation, and conflict resolution [4, 9].
Two parties with asymmetric information wish to arrive at an agreement, say, the price of an item.
A mediator, such as a central third-party marketplace, does not know the players’ information but can communicate with the players.
Crowdsourcing and ridesharing [13, 22, 31].
A group of players each has individual goals ( e.g. , to make money by serving customers at specific locations).
The players are coordinated by a central party ( e.g. , a ridesharing company) that has more information than any one of the players, but the players are free to ignore recommendations if they so choose.
Persuasion in games [17, 3, 23, 14, 30].
The mediator (in that literature, usually “sender”) has more information than the players (“receivers”), and wishes to tell information to the receivers so as to persuade them to act in a certain way.
Automated mechanism design [6, 33, 35, 26].
Players have private information unknown to the mediator.
The mediator wishes to commit to a strategy—that is, set a mechanism—such that players are incentivized to honestly reveal their information.
In fact, in Appendix E we will see that we recover the polynomial-time Bayes-Nash randomized mechanism design algorithm of [8] as a special case of our main result.
Some of the above examples are often used to motivate correlated equilibria.
However, when the mediator is a rational agent with the ability to remember information that it is told and pass the information between players as necessary, we will argue that communication or certification equilibrium should be the notion of choice, for both conceptual and computational reasons.
Preliminaries
In this section, we discuss background on correlation in extensive-form games.
Extensive-form games.
An extensive-form game Γ with n players consists of the following.
A directed tree of nodes or histories H , whose root is denoted ∅ .
The depth of the tree will be denoted T .
The edges out of nodes are labeled with actions , and the set of such actions will be denoted A h .
Given a node h ∈ H and action a at h , the child reached by following action a at node h is denoted ha .
The set of terminal (leaf) nodes in H is denoted Z .
Terminal nodes will always be denoted z throughout the paper.
A partition H \ Z =
H C ⊔
H 1 ⊔ · · · ⊔ . . .
H n of nodes, where H i is the set of all nodes at which player i plays and player H C is the set of chance nodes.
For each player i , a partition
I i of player
i ’s decision nodes, H i , into information sets or infosets .
Every node in a given information set I must have the same set of actions, denoted A I .
We will call the partition
I =
I 1 ⊔ · · · ⊔ I n the players’ information partition .
For each player i , a utility vector u i ∈ [0 , 1] Z , where u i [ z ] denotes the utility achieved by player i at terminal node z . 5.
For each chance node h ∈ H C , a probability distribution p ( ·| h ) over the children of h .
The sequence σ i ( h ) is the list of infosets reached by player i , and actions taken by the player i at those infosets, on the ∅ → h path, not including the infoset at h itself (if any).
We will assume that each player has perfect recall —that is, for each infoset I , the sequence of the player acting at I should be the same for each node in I .
We will denote this sequence σ ( I ) .
In perfect-recall games, nonempty sequences will be identified by the last infoset-action pair Ia in them.
We also will assume that games are timeable and fixed-turn-order , that is, information sets do not span multiple levels of the tree, and all nodes in the same layer of the tree belong to the same player 2 .
We will use the following notation.
The relation ⪯ denotes the natural precedence order induced by the tree H : we write h ⪯ h ′ means that h is an ancestor of h ′ (or h = h ′ ), and for sets S, S ′ , we say S ⪯ S
′ if there are some h ∈ S, h ′ ∈ S
′ such that h ⪯ h ′ .
The binary operation ∧ denotes the lowest common ancestor: h ∧ h ′ is the lowest node u such that u ⪯ h, h ′ .
For sequences, σ ( h ) =
( σ 1 ( h ) , . . .
, σ n ( h )) denotes the joint sequence of all players at node h .
N ( σ ) denotes the set of possible next infosets following sequence σ , that is, N ( σ )
= { I : σ ( I ) = σ } .
The set Σ i denotes the set of sequences of player i , and Σ denotes the set of all sequences across all players (i.e., Σ = ⊔ i Σ i ).
A pure strategy for a player i is a selection of one action for each information set I ∈
I i .
A pure profile is a tuple of pure strategies.
A correlated profile is a distribution over pure profiles.
We will generally work with strategies in realization form (see e.g., Koller et al.
Given a pure strategy x , we say that x plays to z ∈ Z if x plays every action on the ∅ → z path.
We will call the vector x ∈ { 0 , 1 } Z the realization form of x .
The realization form of a mixed strategy is the appropriate convex combination.
The set of mixed strategies forms a convex subset of R Z that, so long as the player has perfect recall, can be expressed using linearly many constraints and variables.
We will occasionally need to discuss changing information partitions of Γ .
If J = J 1 ⊔ · · · ⊔ J n is another valid information partition, we will use Γ J to denote the game Γ with its information partition replaced by J .
We will also occasionally need to talk about multiple games simultaneously; where this is the case, we will mark attributes of the game the same as the game itself.
For example, ˆ
H is the node set of game ˆΓ .
Communication and certification equilibria.
Here, we review definitions related to communication equilibria , following Forges [11] and later related papers.
Definition 2.1.
Let S be a space of possible messages .
A pure mediator strategy is a map d : S ≤ T →
S , where S ≤ T denotes the set of sequences in S of length at most T .
A randomized mediator strategy (hereafter simply mediator strategy ) is a distribution over pure mediator strategies.
We will assume that the space of possible messages is large, but not exponentially so.
In particular, we will assume that {⊥} ∪ I ∪ (cid:83) h
A h ⊆ S ( i.e. , messages can at least be nothing, information, or actions)
and that | S | ≤ poly( |H| ) .
The latter assumption is mostly for cleanliness in stating results: we will give algorithms that need S as an input that we wish to run in time poly( |H| ) .
A mediator strategy augments a game as follows.
If the strategy is randomized, it first samples a pure strategy d , which is hidden from the players.
At each timestep t , a player reaches a history h at which she must act, and observes the infoset I ∋ h .
She sends a message s t ∈ S to the mediator.
The mediator then sends a response d ( s 1 , . . .
, s t ) , which depends on the message s t as well as the messages sent by all other players prior to timestep t .
Then, the player chooses her action a ∈ A h .
We will call the sequence of messages sent and received between the mediator and player i , the transcript with player i .
A communication equilibrium 4 is a Nash equilibrium of the game Γ augmented with a mediator strategy.
The mediator is allowed to perform arbitrary communication with the players.
In particular, the mediator is allowed to pass information from one player to another .
Further, the players are free to send whatever messages they wish to the mediator, including false or empty messages.
These two factors distinguish communication equilibria from notions of correlated equilibria .
In Section 3.4 we will discuss this comparison in greater detail.
A useful property in the literature on communication equilibria is the revelation principle ( e.g. , [11]).
Informally, the revelation principle states that any outcome achievable by an arbitrary strategy profile can also be achieved by a direct strategy profile, in which the players tell the mediator all their information and are subsequently directly told by the mediator which action to play.
In order to be a communication equilibrium, the players still must not have any incentive to deviate from the protocol.
That is, the equilibrium must be robust to all messages that a player may attempt to send to the mediator, even if in equilibrium the player always sends the honest message.
Forges and Koessler [12] further introduced a form of equilibrium for Bayesian games which they called certification equilibria .
In certification equilibria, the messages that a player may legally send are dependent on their information; as such, some messages that a player can send are verifiable .
At each information set I ∈
I , let S
I ⊆
S denote the set of messages that the player at infoset I may send to the mediator.
We will always assume that I ∈ S I and ⊥ ∈ S I for all I .
That is, all players always have the options of revealing their true information or revealing nothing.
Extensive-form S -certification equilibria
The central notion of interest in this paper is a generalization of the notion of certification equilibria [12] to extensive-form games.
Definition 3.1.
Given an extensive-form game Γ and a family of valid message sets S =
{ S I : I ∈
I} , an S -certification equilibrium is a Nash equilibrium of the game augmented by a randomized mediator, in which each player at each information set I is restricted to sending a message s ∈ S I .
The existence of S -certification equilibria follows from the existence of Nash equilibria, which are the special case where the mediator does nothing.
We will need one extra condition on the message sets, which is known as the nested range condition (NRC) [15]: if I ∈ S
I ′ , then S I
⊆ S
I ′ .
That is, if a player with information I ′ can lie by pretending to have information I , then that player can also emulate any other message she would have been are given these names, they carry no semantic meaning.
The revelation principle is used to assign natural meaning to the messages.
Previous models of communication in games [11, 12] usually worked with a model in which players send messages, receive messages, and play moves simultaneously , rather than in sequence as in the extensive-game model that we use.
The simultaneous-move model is easy to recreate in extensive form: by adding further “dummy nodes” at which players learn information but only have one legal action, we can effectively re-order when players ought to communicate their information to the mediator. able to send at I .
Equivalently, the honest message I should be the most certifiable message that a player can send at infoset I .
Our main result is the following.
Theorem 3.2.
Let u M ∈
R
Z be an arbitrary utility vector for the mediator.
Then there is a polynomial-time algorithm that, given a game Γ and a message set family S satisfying the nested range condition, computes an optimal S -certification equilibrium, that is, one that maximizes E z u M [ z ] where the expectation is over playouts of the game under equilibrium.
In particular, by setting S I = S for all I , Theorem 3.2 implies that optimal communication equilibria can be computed in polynomial time.
The rest of the paper is organized as follows.
First, we will prove our main theorem.
Along the way, we will demonstrate a form of revelation principle for S -certification equilibria.
We will then discuss comparisons to other known forms of equilibrium, including the extensive-form correlated equilibrium [29], and several other natural extensions of our model.
Finally, we will show experimental results that compare the computational efficiency and social welfare of various notions of equilibrium on some experimental game instances.
3.1 Proof of Theorem 3.2: The single-deviator mediator-augmented game
In this section, we construct a game ˆ Γ , with n + 1 players, that describes the game Γ where the mediator has been added as an explicit player.
This game has similar structure to the one used by Forges [11, Corollary 2], but, critically, has size polynomial in |H| .
This is due to two critical differences.
First, the players are assumed to either send ⊥ , or send messages that mediator cannot immediately prove to be off-equilibrium.
In particular, if the player’s last message was I and the mediator recommended action a
at I , the player must send a message I ′ with σ ( I ′ )
=
Ia .
If this is impossible, the player must send ⊥ .
Therefore, in particular, we will assume that S I consists of only ⊥ and information sets I ′ at the same level as I .
Second, only one player is allowed to deviate.
Therefore, the strategy of the mediator is not defined in cases where two or more players deviate.
We now formalize ˆΓ .
Nodes in ˆΓ will be identified by tuples ( h, τ , r ) where h ∈ H is a history in Γ , τ = ( τ 1 , . . .
, τ n ) is the collection of transcripts with all players, and r ∈ { REV , REC , ACT } is a stage marker that denotes whether the current state is one in which a player should be revealing information ( REV ), the mediator should be recommending a move ( REC ), or the player should be selecting an action ( ACT ).
The progression of ˆΓ is then defined as follows.
We will use the notation τ
[ i · s ] to denote appending message s to τ i .
• The root node of ˆΓ is ( ∅ , ( ∅ , . . . , ∅ ) , REV ) .
• Nodes ( z, τ , REV ) for z ∈ Z are also terminal in Γ .
The mediator gets utility u M [ z ] , where u is the mediator’s utility function as in Theorem 3.2.
All other players i get utility u
i [ z ] .
• Nodes ( h, τ , REV ) for non-terminal h are decision nodes for the player i who acts at h . 1.
If i is chance, there is one valid transition, to ( h, τ , ACT ) .
If some other player j ̸ = i has already deviated (i.e., σ j ( h ) ̸ = τ j ) ), there is one valid transition, to ( h, τ
[ i · I ] , REC ) where I ∋ h . 3.
If player i has deviated or no one has deviated, then player i observes the infoset I ∋ h , and selects a legal message I ′ ∈ S
I ∩ ( {⊥} ∪ N ( τ i )) to send to the mediator 5 .
Transition to ( h, τ
[ i · I ′ ] , REV ) .
• At ( h, τ , REC ) where h ∈ H i , the mediator observes the transcript τ
i and makes a recommendation a .
If τ i contains any ⊥ messages, then a = ⊥ .
Otherwise, a is a legal action
a ∈
A I , where I is the most recent message in τ i .
Transition to ( h, τ
[ i · a ] , ACT ) .
• Nodes (
h, τ , ACT ) for non-terminal h are decision nodes for the player i who acts at h .
If i is chance, then chance samples a random action a ∼ p ( ·| h ) .
Transition to ( ha, τ , REV ) .
If some other player j ̸ = i has already deviated, there is one valid transition, to ( ha, τ , REC ) , where a is the action sent by the mediator.
If player i has deviated or no one has deviated, then player i observes the transcript τ i , and selects an action a ′ ∈
A h .
Transition to ( ha ′ , τ , REV ) .
The action a ′ need not be the recommended action.
Since at most one player can ever deviate by construction, and the length of the transcripts are fixed because turn order is common knowledge, the transcripts τ can be identified with sequences σ i of the deviated player, if any.
We will make this identification: we will use the shorthand h
σ i to denote the history ( h, ( σ − i ( h ) , σ i ) , REV ) , and h ⊥ for ( h, σ ( h ) , REV ) (i.e., no one has deviated yet).
Therefore, in particular, this game has at most O ( |H|| Σ | ) histories.
For each non-mediator player, there is a well-defined direct strategy ˆ x ∗ i for that player: always report her true information I ∋ h , and always play the action recommended by the mediator.
The goal of the mediator is to find a strategy ˆ x M for itself that maximizes its expected utility, subject to the constraint that each player’s direct strategy is a best response —that is, find ˆ x M such that ( ˆ x M , ˆ x ∗ 1 , . .
, ˆ x ∗ n ) is a (strong) Stackelberg equilibrium of ˆΓ .
We claim that finding a mediator strategy ˆ x M that is a strong Stackelberg equilibrium in ˆΓ is equivalent to finding an optimal S -certification equilibrium in Γ .
We prove this in two parts.
First, we prove a version of the revelation principle for S -certification equilibria.
Definition 3.3.
An S -certification equilibrium is direct if it satisfies the following two properties.
( Mediator directness )
If the transcript τ i of a player i is exactly some sequence of player i , and player i sends an infoset I with σ ( I )
=
τ i , then the mediator replies with an action a ∈ A I .
Otherwise 6 , the mediator replies ⊥ .
( Player directness )
In equilibrium, players always send their true information I , and, upon receiving an action a
∈
A I , always play that action.
Proposition 3.4 (Revelation principle for S -certification equilibria under NRC) .
Assume that S satisfies the nested range condition.
For any S -certification equilibrium, there is a realizationequivalent direct equilibrium.
Omitted proofs can be found in the appendix.
Since direct mediator strategies are exactly the mediator strategies in ˆΓ , and the player strategies are only limited versions of what they are allowed to do in S -certification equilibrium, this implies that, for any S -certification equilibrium, there is a mediator strategy ˆ x M in ˆΓ such that ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) is a Stackelberg equilibrium.
We will also need the converse of this statement.
Proposition 3.5.
Let ˆ x M be a strategy for the mediator in ˆΓ such that, in the strategy profile ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) , every ˆ x ∗ i for i ̸
= M is a best response.
Then there is an direct S -certification equilibrium that is realization-equivalent to ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) .
Therefore, we have shown that the mediator strategies ˆ x M in ˆΓ for which ( ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) is a
Stackelberg equilibrium in ˆΓ correspond exactly to optimal S -certification equilibria of Γ .
Such a
Stackelberg equilibrium can be found by solving the following program:
[Equation]
where ˆ X i is the sequence-form strategy space [20] of player i in ˆΓ .
The only variables in the program are ˆ x i for each player i and the mediator.
In particular, the direct strategies ˆ x ∗ i are constants.
Therefore, the objective is a linear function, and the inner maximization constraints are bilinear in ˆ x M and ˆ x j .
Therefore, this program can be converted to a linear program by dualizing the inner optimizations.
For more details on this conversion, see
Appendix B.
The result is a linear program of size O ( n | ˆ H| )
=
O ( n |H|| Σ | ) .
We have thus proved Theorem 3.2.
3.2 Extensions and special cases
In this section, we describe several extensions and interesting special cases of our main result.
Full-certification equilibria.
One particular special case of S -certification equilibria which is particularly useful.
We define a full-certification equilibrium as an S -certification equilibrium where
S I = {⊥ , I } .
Intuitively, this means that players cannot lie to the mediator, but they may withhold information.
We will call such an equilibrium full-certification .
Removing valid messages from the players only reduces their ability to deviate and thus increases the space of possible equilibrium strategies.
As such, the full-certification equilibria are the largest class of S -certification equilibria.
For full-certification equilibria, the size of game ˆΓ reduces dramatically.
Indeed, in all histories h Ia of ˆΓ , we must have I ⪯ h .
Therefore, we have | ˆ H| ≤ |H| BD where B is the maximum branching factor and D is the depth of the game tree, i.e. , the size of ˆΓ goes from essentially quadratic to essentially quasilinear in |H| .
The mediator’s decision points in ˆΓ for a full-certification equilibrium are the trigger histories used by Zhang et al.
[31] in their analysis of various notions of correlated equilibria.
Later, we will draw further connections between full certification and correlation.
Changing the mediator’s information.
In certain cases, the mediator, in addition to messages that it is sent by the players, also has its own observations about the world.
These are trivial to incorporate into our model: simply change the information partition of the mediator in ˆΓ as needed.
Alternatively, one can imagine adding a “player”, with no rewards (hence no incentive to deviate), whose sole purpose is to observe information and pass it to the mediator.
For purposes of keeping the game small, it is easier to adopt the former method.
To this end, consider any refinement partition
M of the mediator infosets in ˆΓ , and consider the game ˆΓ M created by replacing the mediator’s information partition in ˆΓ with M .
Then we make the following definition.
Definition 3.6.
An ( S , M ) -certification equilibrium of Γ is a mediator strategy ˆ x M in ˆΓ M such that, in the strategy profile (ˆ x M , ˆ x ∗ 1 , . . .
, ˆ x ∗ n ) , every x ∗ i for i ̸
= M is a best response.
( S , M ) -certification equilibria may not exist: indeed, if M is coarser than the mediator’s original information partition in ˆΓ , then the mediator may not have enough information to provide good recommendations under the restrictions of ˆΓ .
This can be remedied by allowing payments (see
Appendix E), or by making the assumption that the mediator at least knows the transcript of the player to whom she is making any nontrivial recommendation:
Definition 3.7.
A mediator partition M is direct if, at every mediator decision point ( h, τ , REC ) , so long as | A h | > 1 , the mediator knows the transcript of the player acting at h .
M is strongly direct if the mediator also observes the transcript when | A h | = 1 .
The condition | A h | > 1 in the definition allows the mediator to possibly not observe the full information of a player if she does not need to make a nontrivial recommendation to that player.
In particular, this allows players to sometimes have information that they only partially reveal to the mediator, so long as the player does not immediately need to act on such information.
Coarseness.
In literature on correlation, coarseness refers to the restriction that a player must obey any recommendation that she receives (but may choose to deviate by not requesting a recommendation and instead playing any other action).
Normal-form coarseness further adds the restriction that players can only choose to deviate at the start of the game—the mediator essentially takes over and plays the game on behalf of non-deviating players.
These notions can easily be expressed in terms of our augmented game, therefore also allowing us to express coarse versions of our equilibrium notions as augmented games.
3.3 The gap between polynomial and not polynomial
If players cannot send messages to the mediator at all, and the mediator has no other way of gaining any information, we recover the notion of autonomous correlated equilibrium (ACE) .
It is NP-hard to compute optimal ACE, even in Bayesian games (see e.g. , von Stengel and Forges [29]).
When M is direct and perfect recall, computing an optimal direct ( S , M ) -certification equilibrium can be done in polynomial time using our framework.
When S obeys NRC and M satisfies a stronger condition 7 , the proof of the revelation principle (Propositions 3.4 and 3.5) works, and the resulting equilibrium is guaranteed to be optimal over all possible equilibria including those that may not be direct.
If NRC does not hold, one can still solve the program (1), and the solution is still guaranteed to be an optimal direct equilibrium by Proposition 3.5.
However, it is not guaranteed to be optimal over all possible communication structures.
Indeed, Green and Laffont [15, Theorem 1] give an instance in which, without NRC, there can be an outcome distribution that is not implementable by a direct mediator.
Our program cannot find such an outcome distribution.
The counterexample does not preclude the possibility of efficient algorithms for finding optimal certification equilibria in more general cases, but does give intuition for why NRC is crucial to our construction.
We could also consider changing the mediator’s information partition so that the mediator does not have perfect recall.
This transformation allows us to recover notions of correlation in games.
Indeed, if we start from the full-certification equilibrium and only allow the mediator to remember the transcript with the player she is currently talking to, we recover EFCE.
Adding coarseness similarly recovers EFCCE and NFCCE.
In this setting, the inability to represent the strategy space of an imperfect-recall player may result in the loss of efficient algorithms.
3.4 A family of equilibria
By varying 1) what the mediator observes, 2) whether the mediator has perfect recall, 3) whether the players can lie or only withhold information, and 4) when and how players can deviate from the mediator’s recommended actions, we can use our framework to define a family consisting ofconceptually different equilibrium notions.
More can be generated by considering other variations in this design space, but we focus on the extreme cases in the table.
Some of these were already defined in the literature; the remaining names are ours.
The result is Table 1.
An inclusion diagram for these notions can be found in Appendix G.
In the table, ex ante means that players have only a binary choice between deviating (in which case they can play whatever they want) and playing (in which case they must always be direct and obey recommendations).
With ex ante deviations, it does not matter whether lying is allowed because we can never get to that stage: either the player deviates immediately and never communicates with the mediator, or the player is direct.
If the mediator only remembers the current active player’s information, and players cannot lie, withholding and coarsely deviating are the same.
Mediator information advantage means that the mediator always learns the infoset of the current active player, and therefore requires no messages from the players.
This is equivalent to forcing players to truthfully report information.
A mediator with information advantage may still not have perfect information—for example, it will not know whether a player (or nature) has played an action until some other player observes the action.
In this setting, the mediator may also have extra private information (known to none of the players), leading to the setting of Bayesian persuasion [17].
In extensive-form games, there are two different reasonable notions of persuasion: one that stems from extending correlated equilibria, and one that stems from extending communication equilibria.
The distinction is that, in the former, the mediator has imperfect recall.
For a more in-depth discussion of Bayesian persuasion, see Appendix F.
Our framework allows optimal equilibria for all notions in the table to be computed.
For perfectrecall mediators, this is possible in polynomial time via the sequence form; for imperfect-recall mediators, the problem is NP-hard, in general, but the team belief DAG of Zhang et al.
[32] can be used to recover fixed-parameter algorithms.
For the notions of correlated equilibrium, this method results in basically the same LP as the correlation DAG of Zhang et al.
We do not claim that all of these notions are easy to motivate.
For example, correlated equilibria are usually arrived at in the “truth known, imperfect recall” setting; the correlated equilibrium notions where lying is allowed are more difficult to motivate in this respect.
Further, even the fixedparameter algorithms of Zhang et al.
[31] would fail in this setting, because “public states” can no longer be treated as public due to the possibility of lying players.
We leave to future research the problem of finding a motivation for the notions that we do not reference elsewhere in the paper.
[Table]
[Table]
Experiments
We ran our algorithm for communication and full-certification equilibria on various two-player games
[Table]
and compared the results to those given by notions of optimal correlation in games.
The games used in the experiments are given in Appendix D. All experiments were allocated four CPU cores and 64 GB of RAM.
Linear programs were solved with Gurobi 9.5.
When payments are used, the allowable payment range is [0 , M ] where M is the reward range of the game.
Experimental results can be found in Table 2.
In the battleship and sheriff instances [Table] there is not a significant difference in performance between finding full-certification equilibria and finding optimal correlated equilibria in terms of performance—this is because, unlike in the general case, optimal correlated equilibria in two-player games without chance can be found in polynomial time [29] anyway.
In the ridesharing instances, the mediator is informed before making the payment of whether the player was honest, and the optimization objective is the social welfare of the original terminal state, minus any payments made. “oom” is out of memory. [Table]
−
[Table]
938 −
[Table]
.−
[Table]
487 −
[Table]
.−
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
P l a y e r 1 ’ s u t i l i t y →
Player2’sutility → battleship −
[Table]
[Table] [Table]
[Table]
P l a y e r 1 ’ s u t i l i t y →
Player2’sutility → sheriff
[Table]
[Table] [Table]
[Table] [Table]
P l a y e r 1 ’
[Table]
u t i l i t y
→
Player2’sutility → rideshare
[Figure]
computing optimal correlated equilibria is much more computationally intensive because the game contains non-public chance actions.
Computing optimal full-certification equilibria is comparably easy, and this difference is clearly seen in the timing results.
Finding optimal communication equilibria is much more intensive than finding optimal fullcertification equilibria, owing to the quadratic size of the augmented game for communication equilibria.
This often causes communication equilibria to be the hardest of the notions to compute in practice, despite optimal correlation being NP-hard.
In Figure 1, we have plotted the payoff spaces of some representative instances.
The plots show how the polytopes of communication and full-certification equilibria behave relative to correlated equilibria.
In the battleship and sheriff instances, the space of communication equilibrium payoffs is a single point, which implies that the space of NFCE (and hence Nash) equilibrium payoffs is also that single point.
Unfortunately, that point is the Pareto-least-optimal point in the space of EFCEs.
In the ridesharing instances, communication allows higher payoffs.
This is because the mediator is allowed to “leak” information between players.
Conclusions and future research
We have shown that optimal communication and certification equilibria in extensive-form games can be computed via linear programs of polynomial size, or almost-linear size in the full-certification case.
We have used our machinery to derive an entire family of equilibrium concepts which we hope to be of use in the future.
Possible future directions include the following.
Are there efficient online learning dynamics , in any reasonable sense of that term, that converge to certification or communication equilibrium?
Is there a better-than-quadratic-size linear program for communication equilibria?
Is it possible to extend our augmented game construction to also cover normal-form correlated equilibria while maintaining efficiency? 4.
Investigate further the comparison between communication and correlation in games.
For example, when and why do communication equilibria achieve higher social welfare than extensive-form correlated equilibria?
Acknowledgements
This material is based on work supported by the National Science Foundation under grants
IIS-1901403 and CCF-1733556, and the ARO under award W911NF2010081.
A Omitted proofs A.1 Proposition 3.
Proof.
We follow the usual structure of revelation principle proofs.
Given an S -certification equilibrium, consider augmenting the mediator d by devices d i , one per player, that acts as follows.
d i internally keeps track of the current sequence σ i of player i .
Whenever it is player
i ’s turn to act, d i expects to be told an information set I ′ with σ ( I ′ ) =
Ia .
Then d i samples what message s player i would have honestly sent to the mediator at infoset I ′ , and forwards this to d .
When d replies with a message s ′ , d i samples the action a ′ the player would have played, sends that to the player, and updates her internal state to σ i :
= I ′ a ′ .
If d i ever receives an invalid message (i.e., anything except an infoset I ′ with σ ( I ′ ) =
Ia ), it resorts to always sending ⊥ to both d and player i for the remainder of the game.
To see that this results in an equilibrium, note that player i can always simulate d i when playing with the original mediator d —therefore, any deviation she can perform against the direct equilibrium can also be performed with d (This is where NRC is used: since the acting player may in reality have lied about her information, without the NRC, it could be the case that the simulated message s would not be legal for player i to send).
A.2 Proposition 3.
Proof.
Consider a mediator for the full game that acts according to ˆ x M .
If the player sends messages that are not in ˆΓ , then the mediator acts as if the player sent ⊥ .
If the mediator reaches a state in which at least two players have provably deviated (i.e., the mediator has reached a state not in ˆΓ ), then the mediator acts arbitrarily.
This mediator strategy, along with the players’ direct strategies, forms a strategy profile.
The only difference between ˆΓ and the true communication protocol induced by S is, in ˆΓ , players are not able to send certain messages.
But, in any case, those messages are ones that are never sent by an honest player; therefore, the mediator will always act as if the player sent ⊥ if it receives such a message.
Therefore, the player gains nothing by having extra messages that she can send.
This completes the proof.
B Full linear program
In this section, we present the full LP formulation implied by the discussion in Section 3.1.
Dropping the hats for notational cleanliness, and letting X j = { x j : F j x j = f j , x j ≥ 0 } be the realization-form representation of player
j ’s decision space, the program (1) has the form
[Equation]
Dualizing the inner maximizations, we have the linear program
[Equation]
where v j are dual variables.
Intuitively, the dual variables represent best-response values : the vector v j will be indexed by information sets I for player j , and v j
[ I ] will be the best-response value for player j at infoset I (assuming all other players are direct).
This sort of dualization of an inner optimization problem to create a linear program is fairly standard—see Koller et al.
[20] for an application to two-player zero-sum games and Farina et al.
[9] for the same approach applied to the special case of correlation.
C Example of game in which full-certification equilibria dominate NFCCEs
We now give an example that demonstrates the difference between NFCCEs (where the mediator has imperfect recall) and the perfect-recall notions.
In particular, we will show a game where there is an
H
T
(extensive-form) persuasion strategy for the mediator that leads to outcomes Pareto-dominating every NFCCE of Γ .
Since persuasion can be recovered from full certification (or even communication) by adding a dummy player who communicates information to the mediator and has no incentives (thus is always direct), this also shows that communication equilibria can Pareto-dominate NFCCEs.
Consider the game whose tree is depicted in Figure 2.
At the root, chance selects a subgame s ∈ { ▲ , ▼ , H , T } .
If chance selects s ∈ { ▲ , ▼ } , then player s is offered a chance to exit the game; if she does, she scores 0 and the other player scores 1 .
If she does not exit, the two players play matching pennies with the other player (not s ) playing first (this will be relevant when we take perfect-recall refinements).
The winner scores 1 point and the loser − 1 .
If chance selects s ∈ { H , T } , then ▲ attempts to guess which one was selected.
If successful, both players score 1 point; otherwise, both scores 0 .
The following properties can be directly verified:
• For every NFCCE in both Γ and its perfect-information refinement, both ▲ and ▼ have value exactly 1 / 2 .
• The following mediator pure strategy is persuasive in Γ , where both players achieve value 3 / 4 : both players exit when possible, and if s ∈ { H , T } , the mediator informs ▲ which is the case.
If the matching pennies subgame is entered, the mediator recommends that each player play independently and uniformly at random.
Intuitively, this outcome exists because the mediator has the power to selectively tell players information when it is to their benefit ( e.g. in the case s ∈ { H , T } , where the mediator can inform ▲ of chance’s choice to allow ▲ to score a point) and withhold information to incentivize cooperation ( e.g. , in the case s = ▲ , where the mediator can withhold ▼ ’s matching pennies bit from ▲ to prevent ▲ from deviating).
D Descriptions of games in experiments
• B is a two-player Battleship game [9] with a grid of size h × w and r rounds, made into a nonzero-sum game by having each player value their own ships greater than their opponents’.
The three numbers afterward are, in order, the length, width, and number of shots per side.
Each player has a single ship of unit size.
• S is a simplified Sheriff of Nottingham game [9], a small game modeling a negotiation between two players.
The first two numbers roughly correspond to how much power each player has in negotiation; the final number is the number of rounds of negotiation.
• RS is a small ridesharing game [31], played on a graph, in which the players walk around the graph attempting to serve “customers” who reside at certain nodes.
The first number (1 or 2) denotes the specific graph on which the game is played; the second number is the number of steps in the game.
E Mechanisms with payments
The mediator may be able to take its own actions during the game, separate from the players.
In our framework, this can be modelled by introducing an auxiliary player that has no incentives of her own and therefore no reason to disobey the mediator recommendations.
One particular use case of such a player is payments : the auxiliary player can collect payments from or give payments to players, as a means of incentivizing players to perform certain actions.
We can make this concrete as follows.
At the end of the game ˆΓ , chance picks a random player i ∗ ∈ [ n ] , reveals it to the mediator.
The mediator then selects a payment p ∈ { L, U } to be given to the player, where L, U ∈ R are a minimum or maximum payment (that may be functions of the mediator’s information).
The player gains utility np and the mediator loses utility np .
By varying the probability with which the mediator makes the two payments, the mediator can (in expectation) pay any player any amount in the range [ L, U ] .
The size of the mediator-augmented game increases by a factor of O ( n ) .
If U − L is at least the reward range of the game, and the mediator always learns whether players have acted directly, then any strategy at all can be enforced in equilibrium simply by giving each player a sufficiently large payoff for cooperating.
The ability of the mediator to commit to a strategy is critical when payments are involved: if a mediator could not credibly commit, then it would never make a payment because it has negative incentive to do so.
Automated mechanism design.
A consequence of the above analysis is that we recover the BayesNash 8 randomized automated mechanism design algorithm of Conitzer and Sandholm [6] as a special case, as follows.
That paper considers a Bayesian game with T types per player and n players.
A type assignment θ ∈ [ T ] n is sampled from some joint distribution, and each type θ i is revealed privately to player i (for ex interim incentive compatibility) or publicly to all players (for ex post incentive compatibility).
A round of communication ensues, in which each player informs the mediator about her own type (or lies about it).
The mediator then chooses an outcome o ∈
[ O ] , and each player receives a utility that is a function of their own type θ i and the outcome o .
The resulting game has size T n O , and each player has at most T n sequences, so the overall LP (1), after including payments, has size poly( O, T n ) , which is the same result observed by Conitzer and Sandholm [8].
Costly messaging.
Suppose that players have a strict preference for not revealing information over revealing it, or for sending certain messages over others.
In the game ˆΓ , we can easily express this preference by changing the utilities of player i .
In this setting, direct equilibria may not exist in the general case: indeed, consider a single-player game where the player has perfect information and therefore no need to tell the mediator anything.
Nonetheless, direct equilibria can still exist in some games if the mediator has some power to persuade the players to reveal their information—that is, if a player’s information will help the mediator to give the player a better outcome.
If the mediator knows whether players have been direct, there always exist payments large enough to incentivize direct behavior (or, indeed, any behavior), so payments can also be used to recover equilibrium existence.
This algorithm does not take into account the possibility that the mediator or players may be better off not knowing certain information than paying the price of gathering it, which is often the case in preference elicitation [ e.g. , 21, 27] and has more recently been studied for automated mechanism design Zhang et al. [34], Kephart and Conitzer [18, 19]; it will only compute the optimal equilibrium for which the mediator gives sufficient incentive for all players to always honestly reveal information.
Finding optimal equilibria in settings when the revelation principle fails is beyond the scope of this paper.
F Additional related research
Our results are extremely general across settings and applications, most notably mechanism design and Bayesian persuasion (information design).
Many special cases of our main algorithm have been discussed in the literature.
Here, we give an overview of some of them.
Automated mechanism design.
In mechanism design, an uninformed mediator (the mechanism) takes actions based on information given to them by player(s) who have information but cannot take actions.
In the previous section, we showed how our framework can be used to generalize the automated mechanism design algorithm of Conitzer and Sandholm [6].
We will now discuss several other very recent papers on automated mechanism design in dynamic settings, and how our paper relates to them.
Zhang and Conitzer [33] consider an automated dynamic mechanism design setting in a Markov game with one player.
Their main positive result is an LP for the setting of short-horizon MDPs, which can be viewed as a special case of our framework by simply unrolling the (short-horizon)
MDP into an extensive-form game.
Zhang et al.
[35] consider an automated mechanism design setting in which the agent’s only power is, at each timestep, to quit the decision process.
They study Markov games, which in this setting are significantly involved difficult than extensive-form games.
In extensive form, their setting can be formulated as an augmented game very similar to our framework, in which the “exit” action is explicitly added into the augmented game and the principal selects the action.
Papadimitriou et al.
[26] study a dynamic auction design setting.
In their most general setup, there are k players (bidders) with independent valuations for each of D items to be sold in sequence.
The agents in their setting know all their valuations upfront, but only report their valuations for the item currently being sold.
That constraint can be expressed in our framework in the language of ( S , M ) - certification equilibria, by setting M such that the mediator only learns the player’s valuation of the current item.
As such, that setting is also a special case of ours—in particular, our algorithm matches their positive results, Theorems 7 and 8, that use a “dynamic programming LP” to compute the optimal randomized mechanism.
Sandholm et al.
[28] study multi-stage mechanism design, but they use multiple stages for the purpose of reducing the amount of communication necessary for preference elicitation in what would otherwise be a single-stage setting. Their work is therefore orthogonal to ours. Automated mechanism
with partially-verifiable types. Zhang et al.
[34] give an algorithm for finding the optimal deterministic, direct mechanism in a single-agent, single-stage setting.
Our work differs from that one in that we consider extensive-form (multi-stage) settings, randomized mechanisms, and a more general setup (in which the players can also take actions).
However, we share in common with that paper the fact that we compute the optimal direct equilibrium, and rely on the revelation principle holding for that equilibrium to be optimal across communication structures.
Kephart and Conitzer [18] analyze mechanism design in a single-stage, single-agent setting with costly reporting and not assuming the revelation principle.
Their goal is to, given a social choice function, determine whether that function can be implemented.
They analyze a large spectrum of cases, and discuss for each case whether this implementation problem is easy, or (NP)-hard.
In a follow-up paper [19], the same authors study the revelation principle in the presence of reporting costs, and carefully investigate when the revelation principle does or does not hold.
When the revelation principle does hold, our framework implies a polynomial-time algorithm for computing whether a social choice function is implementable, simply by adding the appropriate linear constraints to our linear program formulation.
However, when the revelation principle fails, that algorithm will fail to find an implementation for any social choice function that cannot be implemented by a direct mechanism.
Bayesian persuasion (information design). In Bayesian persuasion, also commonly referred to as information resign [17], the roles of the mediator and player are reversed compared to automated mechanism design: the mediator (“principal”) has informational advantage, and the player(s) take the actions.
The difference between the two settings lies in who has the commitment power: in automated mechanism design, the side with the power to take actions has the commitment power.
Celli et al.
[3] study persuasion in extensive-form games with multiple players, in which the principal can only send a single signal at the beginning of the game to the players.
They focus on a setting where the mediator can only send a single signal to the players, which is far more restrictive than our setup of persuasion.
Their setting of Bayes coarse -correlated equilibria is equivalent to what we call normal-form coarse imperfect-recall persuasion , and their results in this setting fall out as special case of our framework (using the correlation DAG of Zhang et al.
[31] to represent the mediator’s decision space).
Their discussion of Bayes- correlated equilibria is not captured by our framework for the same reason that NFCEs are not captured.
Gan et al.
[14] study persuasion in Markov games with a single player, in which the principal can send messages to the player at every time step.
The extensive-form analogue of their setting is, once again, a special case of our framework.
The complexity gap between myopic, advice-myopic, and far-sighted players, discussed in that paper for Markov games, disappears in extensive-form games because extensive form naturally allows history-dependent strategies for both the mediator and the players.
Wu et al.
[30] study persuasion in Markov games with myopic players.
That is, a new player arrives at every timestep.
After performing a single action, the player gains a reward dependent on the true state (which, of course, the player may not actually know) and her action, and then leaves the system forever.
The authors devise an online learning algorithm that provably has low regret for the sender.
Compared to their setting, our setting is significantly more general—allowing multiple non-myopic agents as well as other forms of limited information and communication—but ours is based on linear programming instead of online learning, and works with extensive-form games instead of Markov games.
We are not the first to point out associations between communication equilibria and other settings such as mechanism design and persuasion.
In particular, Bergemann and Morris [2] discuss in great depth the relationship between communication equilibria, mechanism design, and persuasion in Bayesian games.
On top of that paper, our contribution is that we extend their ideas of unification to the more general setting of arbitrary extensive-form games and provide efficient algorithms for all of those cases.
Preference elicitation from multiple agents.
There has been significant work on preference elicitation from multiple agents, starting in the context of combinatorial auctions [5].
Often the elicitor knows that some preference information from an agent is not needed for optimally allocating (and pricing) given what other agents have already revealed about their preferences.
This is the case already in private-values settings.
In quasilinear settings, if the elicitor receives enough information from the agents to determine the welfare-maximizing allocation and the Vickrey-Clarke-Groves payments, it is an ex post equilibrium for all players to tell the truth [5].
This is despite the fact that the elicitor’s queries leak information across agents.
Similar leakage also happens in preference elicitation in voting, and there are certain restrictions that can be imposed on the elicitation policy that eliminate any strategic effects that stem from that [7].
Our paradigm is fundamentally different from the above preference elicitation setting, in several ways.
First, all our complexity results are at least linear in the size of the game tree, whereas in a combinatorial auction, the auctioneer (mediator) may pick one of exponentially many outcomes.
Second, we always allow mixed strategies, which are typically not used in preference elicitation.
Finally, in voting settings one often desires an equilibrium in weakly dominant strategies, which our paradigm also does not cover.
G
hierarchy
[Figure]
persuasion coarse persuasion
NF coarse persuasion
IR persuasion coarse
IR persuasion
NF
[Figure]
IR persuasion
[Figure]