Abstract: Graphical games have been proposed as a game-theoretic model of large-scale distributed networks of non-cooperative agents. When the number of players is large, and the underlying graph has low degree, they provide a concise way to represent the players' payoffs. It has recently been shown that the problem of finding Nash equilibria in a general degree-3 graphical game with two actions per player is complete for the complexity class PPAD, indicating that it is unlikely that there is any polynomial-time algorithm for this problem. In this paper, we study the complexity of graphical games with two actions per player on bounded-degree trees. This setting was first considered by Kearns, Littman and Singh, who proposed a dynamic programming-based algorithm that computes all Nash equilibria of such games. The running time of their algorithm is exponential, though approximate equilibria can be computed efficiently. Later, Littman, Kearns and Singh proposed a modification to this algorithm that can find a single Nash equilibrium in polynomial time. We show that this modified algorithm is incorrect -- the output is not always a Nash equilibrium. We then propose a new algorithm that is based on the ideas of Kearns et al. and computes all Nash equilibria in quadratic time if the input graph is a path, and in polynomial time if it is an arbitrary graph of maximum degree 2. Moreover, our algorithm can be used to compute Nash equilibria of graphical games on arbitrary trees, but the running time can be exponential, even when the tree has bounded degree. We show that this is inevitable -- any algorithm of this type will take exponential time, even on bounded-degree trees with pathwidth 2. It is an open question whether our algorithm runs in polynomial time on graphs with pathwidth 1, but we show that finding a Nash equilibrium for a 2-action graphical game in which the underlying graph has maximum degree 3 and constant pathwidth is PPAD-complete (so is unlikely to be tractable).

Nash Equilibria in Graphical Games on Trees Revisited ∗ Edith Elkind University of Warwick Coventry, CV4 7AL, U.K. Leslie Ann Goldberg University of Warwick Coventry, CV4 7AL, U.K. Paul Goldberg University of Warwick Coventry, CV4 7AL, U.K. ABSTRACT Graphical games have been proposed as a game-theoretic model of large-scale distributed networks of non-cooperative agents.
When the number of players is large, and the underlying graph has low degree, they provide a concise way to represent the players'' payoffs.
It has recently been shown that the problem of finding Nash equilibria in a general degree-3 graphical game with two actions per player is complete for the complexity class PPAD, indicating that it is unlikely that there is any polynomial-time algorithm for this problem.
In this paper, we study the complexity of graphical games with two actions per player on bounded-degree trees.
This setting was first considered by Kearns, Littman and Singh, who proposed a dynamic programming-based algorithm that computes all Nash equilibria of such games.
The running time of their algorithm is exponential, though approximate equilibria can be computed efficiently.
Later, Littman, Kearns and Singh proposed a modification to this algorithm that can find a single Nash equilibrium in polynomial time.
We show that this modified algorithm is incorrect - the output is not always a Nash equilibrium.
We then propose a new algorithm that is based on the ideas of Kearns et al. and computes all Nash equilibria in quadratic time if the input graph is a path, and in polynomial time if it is an arbitrary graph of maximum degree 2.
Moreover, our algorithm can be used to compute Nash equilibria of graphical games on arbitrary trees, but the running time can be exponential, even when the tree has bounded degree.
We show that this is inevitable - any algorithm of this type will take exponential time, even on bounded-degree trees with pathwidth 2.
It is an open question whether our algorithm runs in polynomial time on graphs with pathwidth 1, but we show that finding a Nash equilibrium for a 2-action graphical game in which the underlying graph has maximum degree 3 and constant pathwidth is PPAD-complete (so is unlikely to be tractable).
Categories and Subject Descriptors F.2 [Theory of Computation]: Analysis of Algorithms and Problem Complexity; J.4 [Computer Applications]: Social and Behavioral Sciences-economics General Terms Algorithms, Economics, Theory 1.
INTRODUCTION Graphical games were introduced in the papers of Kearns et al. [8] and Littman et al. [9] as a succinct representation of games with a large number of players.
The classical normal form (or matrix form) representation has a size that is exponential in the number of players, making it unsuitable for large-scale distributed games.
A graphical game associates each player with a vertex of an underlying graph G, and the payoff to that player is a function of the actions chosen by himself and his neighbours in G; if G has low degree, this is a concise way to represent a game with many players.
The papers [8, 9] give a dynamic-programming algorithm for finding Nash equilibria in graphical games where there are two actions per player and G is a tree.
The first of these papers describes a generic algorithm for this problem that can be specialized in two ways: as an algorithm that computes approximations to all Nash equilibria in time polynomial in the input size and the approximation quality, or as an exponential-time algorithm that allows the exact computation of all Nash equilibria in G.
In [9], the authors propose a modification to the latter algorithm that aims to find a single Nash equilibrium in polynomial time.
This does not quite work, as we show in Section 3, though it introduces a useful idea.
1.1 Background The generic algorithm of [8] consists of two phases which we will refer to as the upstream pass and the downstream pass; 1 the former starts at the leaves of the tree and ends at the root, while the latter starts at the root and ends at the leaves.
It is assumed that each player has two pure strategies (actions), which are denoted by 0 and 1; it follows that any mixed strategy can be represented as a single number x ∈ [0, 1], where x is the probability that the player selects 1.
During the upstream pass, each vertex V computes the set of its potential best responses to every mixed strategy w of its parent W ; a strategy v is a potential best response to w if 1 Note that the terminology upstream and downstream are reversed in [8, 9] - our trees are rooted at the top.
100 there is a Nash equilibrium in the graphical game downstream of V (inclusive) given that W plays w (for a more technical definition, the reader is referred to Section 2).
The output of this stage can be viewed as a (continuous) table T(w, v), where T(w, v) = 1 if and only if v is a potential best response to w; we refer to this table as the best response policy for V .
The generic algorithm does not address the problem of representing the best response policy; in fact, the most important difference between the two instantiations of the generic algorithm described in [8] is in their approach to this issue.
The computation is performed inductively: the best response policy for V is computed based on the best response policies of V ``s children U1, ... , Uk.
By the end of the upstream pass, all children of the root have computed their best response policies.
In the beginning of the downstream pass, the root selects its strategy and informs its children about its choice.
It also selects a strategy for each child.
A necessary and sufficient condition for the algorithm to proceed is that the strategy of the root is a best response to the strategies of its children and, for each child, the chosen strategy is one of the pre-computed potential best responses to the chosen strategy of the root.
The equilibrium then propagates downstream, with each vertex selecting its children``s actions.
The action of the child is chosen to be any strategy from the pre-computed potential best responses to the chosen strategy of the parent.
To bound the running time of this algorithm, the paper [8] shows that any best response policy can be represented as a union of an exponential number of rectangles; the polynomial time approximation algorithm is obtained by combining this representation with a polynomial-sized grid.
The main idea of [9] is that it is not necessary to keep track of all rectangles in the best response policies; rather, at each step of the upstream pass, it is possible to select a polynomial-size subset of the corresponding policy (in [9], this subset is called a breakpoint policy), and still ensure that the downstream pass can proceed successfully (a sufficient condition for this is that the subset of the best response policy for V stored by the algorithm contains a continuous path from w = 0 to w = 1).
1.2 Our Results One of the main contributions of our paper is to show that the algorithm proposed by [9] is incorrect.
In Section 3 we describe a simple example for which the algorithm of [9] outputs a vector of strategies that does not constitute a Nash equilibrium of the underlying game.
In Sections 4, 5 and 6 we show how to fix the algorithm of [9] so that it always produces correct output.
Section 4 considers the case in which the underlying graph is a path of length n. For this case, we show that the number of rectangles in each of the best response policies is O(n2 ).
This gives us an O(n3 ) algorithm for finding a Nash equilibrium, and for computing a representation of all Nash equilibria.
(This algorithm is a special case of the generic algorithm of [8] - we show that it runs in polynomial time when the underlying graph is a path.)
We can improve the running time of the generic algorithm using the ideas of [9].
In particular, we give an O(n2 ) algorithm for finding a Nash equilibrium of a graphical game on a path of length n. Instead of storing best response policies, this algorithm stores appropriately-defined subsets, which, following [9], we call breakpoint policies (modifying the definition as necessary).
We obtain the following theorem THEOREM 1.
There is an O(n2 ) algorithm that finds a Nash equilibrium of a graphical game with two actions per player on an n-vertex path.
There is an O(n3 ) algorithm that computes a representation of all Nash equilibria of such a game.
In Section 5 we extend the results of Section 4 to general degree2 graphs, obtaining the following theorem.
THEOREM 2.
There is a polynomial-time algorithm that finds a Nash equilibrium of a graphical game with two actions per player on a graph with maximum degree 2.
In Section 6 we extend our algorithm so that it can be used to find a Nash equilibrium of a graphical game on an arbitrary tree.
Even when the tree has bounded degree, the running time can be exponential.
We show that this is inevitable by constructing a family of graphical games on bounded-degree trees for which best response policies of some of the vertices have exponential size, and any twopass algorithm (i.e., an algorithm that is similar in spirit to that of [8]) has to store almost all points of the best response policies.
In particular, we show the following.
THEOREM 3.
There is an infinite family of graphical games on bounded-degree trees with pathwidth 2 such that any two-pass algorithm for finding Nash equilibria on these trees requires exponential time and space.
It is interesting to note that the trees used in the proof of Theorem 3 have pathwidth 2, that is, they are very close to being paths.
It is an open question whether our algorithm runs in polynomial time for graphs of pathwidth 1.
This question can be viewed as a generalization of a very natural computational geometry problem - we describe it in more detail in Section 8.
In Section 7, we give a complexity-theoretic intractability result for the problem of finding a Nash equilibrium of a graphical game on a graph with small pathwidth.
We prove the following theorem.
THEOREM 4.
Consider the problem of finding a Nash equilibrium for a graphical game in which the underlying graph has maximum degree 3 and pathwidth k.
There is a constant k such that this problem is PPAD-complete.
Theorem 4 limits the extent to which we can exploit path-like properties of the underlying graph, in order to find Nash equilibria.
To prove Theorem 4, we use recent PPAD-completeness results for games, in particular the papers [7, 4] which show that the problem of finding Nash equilibria in graphical games of degree d (for d ≥ 3) is computationally equivalent to the problem of solving r-player normal-form games (for r ≥ 4), both of which are PPAD-complete.
2.
PRELIMINARIES AND NOTATION We consider graphical games in which the underlying graph G is an n-vertex tree.
Each vertex has two actions, which are denoted by 0 and 1.
A mixed strategy is given by a single number x ∈ [0, 1], which denotes the probability that the player selects action 1.
Fur the purposes of the algorithm, the tree is rooted arbitrarily.
For convenience, we assume without loss of generality that the root has a single child, and that its payoff is independent of the action chosen by the child.
This can be achieved by first choosing an arbitrary root of the tree, and then adding a dummy parent of this root, giving the new parent a constant payoff function.
Given an edge (V, W ) of the tree G, and a mixed strategy w for W , let G(V,W ),W =w be the instance obtained from G by (1) deleting all nodes Z which are separated from V by W (i.e., all nodes Z such that the path from Z to V passes through W ), and (2) restricting the instance so that W is required to play mixed strategy w. Definition 1.
Suppose that (V, W ) is an edge of the tree, that v is a mixed strategy for V and that w is a mixed strategy for W .
101 We say that v is a potential best response to w (denoted by v ∈ pbrV (w)) if there is an equilibrium in the instance G(V,W ),W =w in which V has mixed strategy v.
We define the best response policy for V , given W , as B(W, V ) = {(w, v) | v ∈ pbrV (w), w ∈ [0, 1]}.
Typically, W is the parent of V , and this is just referred to as the best response policy for V .
The expression B(W, V )|V =v is used to denote the set B(W, V ) ∩ [0, 1]×{v}.
The upstream pass of the generic algorithm of [8] computes the best response policy for V for every node V other than the root.
With the above assumptions about the root, the downstream pass is straightforward: Let W denote the root and V denote its child.
The root selects any pair (w, v) from B(W, V ).
It decides to play mixed strategy w and it instructs V to play mixed strategy v.
The remainder of the downward pass is recursive.
When a node V is instructed by its parent to adopt mixed strategy v, it does the following for each child U - It finds a pair (v, u) ∈ B(V, U) (with the same v value that it was given by its parent) and instructs U to play u. 3.
ALGORITHM OF LITTMAN ET AL..
The algorithm of [9] is based on the following observation: to compute a single Nash equilibrium by a two-pass algorithm, it is not necessary to construct the entire best response policy for each vertex.
As long as, at each step of the downstream pass, the vertex under consideration can select a vector of strategies for all its children so that each child``s strategy is a potential best response to the parent``s strategy, the algorithm succeeds in producing a Nash equilibrium.
This can be achieved if, at the beginning of the downstream pass, we have a data structure in which each vertex V with parent W stores a set ˆB(W, V ) ⊆ B(W, V ) (called a breakpoint policy) which covers every possible w ∈ [0, 1].
We will show later that a sufficient condition for the construction of such a data structure is the invariant that, at every level of the upstream pass, ˆB(W, V ) contains a continuous path from w = 0 to w = 1.
In [9], it is suggested that we can select the breakpoint policy in a particular way.
Namely, the paper uses the following definition: Definition 2.
(cf. [9]) A breakpoint policy for a node V with parent W consists of an ordered set of W -breakpoints w0 = 0 < w1 < w2 < · · · < wt−1 < wt = 1 and an associated set of V -values v1, ... , vt..
The interpretation is that for any w ∈ [0, 1], if wi−1 < w < wi for some index i and W plays w, then V shall play vi; and if w = wi for some index i, then V shall play any value between vi and vi+1.
We say such a breakpoint policy has t − 1 breakpoints.
The paper then claims that any vertex V can compute its breakpoint policy with respect to its parent W given the breakpoint policies of its children U1, ... , Uk.
The proof proceeds by ordering the children``s breakpoints (i.e., the respective values of v) from left to right (it can be assumed without loss of generality that all these breakpoints are distinct) and considering them in turn; each such point vl ∈ {v1, ... , vL} corresponds to a fixed choice of strategies for k − 1 children and an interval of admissible strategies for one child.
Assume for convenience that this child is U1 and its interval of admissible strategies at vl is [a, b]; assume also that for Uj , j = 2, ... , k, their respective breakpoint policies prescribe them to play uj in response to vl.
Let P i (u, w), i = 0, 1, be the expected payoff for V when V plays i, U1 plays u, each Uj , j = 2, ... , k, plays uj, and W plays w, and consider the set Wl = {w ∈ [0, 1] | ∃u ∈ [a, b] s.t. P 0 (u, w) = P1 (u, w)}; note that for any w ∈ Wl we have vl ∈ pbrV (w).
v1 v2 v3 v4 v5 v6 v7 V W Figure 1: LKS: Trimming to find breakpoint policies.
The authors show that for any breakpoint vl, the set Wl is either empty, a single interval, or a union of two non-floating intervals (an interval is non-floating if one of its endpoints is 0 or 1); moreover, the union of all sets Wl, l = 1, ... , L, covers the interval [0, 1].
It follows easily that one can cover [0, 1] with at most L+2 intervals, each of which is a subset of some Wl.
The authors then claim that any such cover can be transformed into a breakpoint policy for V .
Namely, they say that for any two intervals Wl1 and Wl2 in the cover, Any overlap between Wl1 and Wl2 can be arbitrarily assigned coverage by Wl1 and Wl2 trimmed accordingly (cf. [9], p. 5).
They illustrate their approach in a figure, which is reproduced as Figure 1 here.
In the figure, the dashed horizontal lines represent the breakpoints v1, v2, ... , v7 and the solid intervals along these breakpoints are the sets W1, W2, ... , W7.
The thick connected path is the corresponding breakpoint policy.
It is chosen as follows: begin on the left, and always jump to the interval allowing greatest progress to the right.
To see why this approach does not work in general, consider a path of length 4 consisting of an indifferent root R, its child W , W ``s child V , and V ``s child U. Suppose that U receives a payoff of 1 if it plays differently to V and 0 otherwise.
Thus, if v denotes the mixed strategy of V (i.e., V plays 1 with probability v), then the expected payoff that U derives from playing 0 is given by P0 (U) = v and the expected payoff that U derives from playing 1 is given by P1 (U) = 1 − v. Suppose that V derives no payoff from playing 1 (so P1 (V ) = 0) and that its payoff matrix for playing 0 is 1 −9 9 −1 , so if u denotes the mixed strategy of U and w denotes the mixed strategy of W , the expected payoff that V derives from playing 0 is given by P0 (V ) = (1 − u)(1 − w) + (1 − u)w(−9) + u(1 − w)9 + uw(−1).
Using the techniques of [8] (or, alternatively, those of Section 4), it is not hard to verify that the best response policies for U and V (as in Definition 1) are given by the graphs in Figure 2.
The best response policy for U is a breakpoint policy for U (as in Definition 2) with V -breakpoints v0 = 0, v1 = 1/2 and v2 = 1 with associated values u1 = 1 and u2 = 0.
The best response policy for V is not a breakpoint policy (because of how the curve from w = 0 to w = 1 doubles back).
The LKS algorithm would trim to get a breakpoint policy such as the one in Figure 3.
Note that this breakpoint policy ˆB(W, V ) is invalid in the sense that it does not satisfy ˆB(W, V ) ⊆ B(W, V ).
102 1 10.5 0.5 1 10.1 0.9 u v v w Figure 2: Best response policies for U and V .
0.1 0.9 1 0.5 1 v w Figure 3: A trimmed policy for V The point is that the payoff matrix of W can now be chosen to prevent the LKS algorithm from finding a Nash equilibrium.
For example, suppose the payoffs are given so that P0 (W ) = v and P1 (W ) = (1−v)2.
The best response policy for W is a horizontal line at w = .1 (This is the value of w that allows v = 2/3 - see Figure 2, which makes P0 (W ) = P1 (W ).)
In the downward pass, the chosen values are w = .1, then, from the trimming, v = 0 and u = 1, which is not a Nash equilibrium since W prefers action 1.
The failure of the algorithm is not caused by the fact that the trimming policy goes as far to the right as possible.
Any other trimming would be just as bad.
For example, suppose the breakpoint policy for V has v = 0 until some point w∗ < .9 and then jumps to v = 1.
The algorithm is then defeated by the payoff matrix with P 0 (W ) = 2v and P1 (W ) = (1 − v) in which the best response policy for W is a horizontal line at w = .9.
The algorithm then gives w = .9, v = 1, and u = 0, which is not a Nash equilibrium since W prefers action 0.
We conclude that the LKS algorithm does not always find a Nash equilibrium.
In Sections 4 and 6 we show how to modify the algorithm so that it always finds a Nash equilibrium.
For the modified algorithm, we have to extend the definition of breakpoint policy (see Definition 3) so that it includes breakpoint policies such as the best response policy for V in Figure 2.
Unfortunately, such a breakpoint policy may be exponential in size (see Figure 7) so the corrected algorithm does not run in polynomial time on all trees.
In the next section, we show that it runs in polynomial time on a path.
4.
FINDING EQUILIBRIA ON A PATH In this section, we focus on the case when the underlying graph is a path, i.e., its vertex set is {V1, ... , Vn}, and its edge set is {(Vj , Vj+1) | j = 1, ... , n − 1}.
We show that in this case the best response policy for each vertex can be represented as a union of a polynomial number of rectangles, where a rectangle is defined by a pair of closed intervals (IV , IU ) and consists of all points in IV × IU ; it may be the case that one or both of the intervals IV and IU consists of a single point.
THEOREM 5.
For any j = 1, ... , n, the set B(Vj , Vj−1) can be represented as a disjoint union of at most (j + 4)2 rectangles.
Moreover, given such representation of B(Vj , Vj−1), one can compute a representation of B(Vj+1, Vj) in time O(j2 ).
PROOF.
For any set A ⊆ [0, 1]2 that is represented as a union of a finite number of rectangles, we say that a point u ∈ [0, 1] on the U-axis is a U-event point of A if u = 0 or u = 1 or A contains a rectangle of the form IV × IU and u is an endpoint of IU ; V -event points are defined similarly.
Observe that for any u ∈ [0, 1], the number of connected components of [0, 1]×{u} ∩ A is at most the number of V -event points of A.
We use induction on j to show that for each Vj the statement of the theorem holds and, additionally, each B(Vj , Vj−1) has at most 2j + 4 event points.
To simplify the base case, we modify the graphical game by appending a dummy vertex V0 to the beginning of the path: the only neighbour of V0 is V1, the payoffs of V0 are always equal to 0, and the payoffs of all other vertices (including V1) are the same as in the original game.
For j = 0, we have B(V1, V0) = [0, 1]2 , so the statement of the theorem is trivially true.
Now, suppose that j > 0, set V = Vj and let U = Vj−1 and W = Vj+1 be the vertices that precede and follow V , respectively.
The payoffs to V are described by a 2×2×2 matrix P: Pxyz is the payoff that V receives when U plays x, V plays y, and W plays z, where x, y, z ∈ {0, 1}.
Suppose that U plays 1 with probability u and W plays 1 with probability w.
Then V ``s expected payoff from playing 0 is P0 =(1−u)(1−w)P000+(1−u)wP001+u(1−w)P100+uwP101, while its expected payoff from playing 1 is P1 =(1−u)(1−w)P010+(1−u)wP011+u(1−w)P110+uwP111.
If P 0 > P1 , V strictly prefers to play 0, if P0 < P1 , V strictly prefers to play 1, and if P0 = P1 , V is indifferent, i.e., can play any (mixed) strategy.
Since P0 and P1 are linear in w and u, there exist some constants A1, A0, B1, and B0 that depend on the matrix P, but not on u and w, such that P0 − P1 = w(B1u + B0) − (A1u + A0).
(1) Depending on the values of A1, A0, B1, and B0, we subdivide the rest of the proof into the following cases.
• B1 = 0, B0 = 0.
In this case, P0 > P1 if and only if A1u + A0 < 0.
If also A1 = 0, A0 = 0, clearly, B(W, V ) = [0, 1]2 , and the statement of the theorem is trivially true.
Otherwise, the vertex V is indifferent between 0 and 1 if and only if A1 = 0 and u = −A0/A1.
Let V = {v | v ∈ (0, 1), −A0/A1 ∈ pbrU (v)}.
By the inductive hypothesis, V consists of at most 2(j − 1) + 4 segments and isolated points.
For any v ∈ V, we have B(W, V )|V =v = [0, 1]: no matter what W plays, as long as U is playing −A0/A1, V is content to play v. On the other hand, for any v ∈ (0, 1) \ V we have B(W, V )|V =v = ∅: when V plays v, U can only respond with u = −A0/A1, in which case V can benefit from switching to one of the pure strategies.
To complete the description of B(W, V ), it remains to analyze the cases v = 0 and v = 1.
The vertex V prefers to play 0 if A1 > 0 and u ≤ −A0/A1, or A1 < 0 and u ≥ −A0/A1, or 103 A1 = 0 and A0 < 0.
Assume for now that A1 > 0; the other two cases can be treated similarly.
In this case 0 ∈ pbrV (w) for some w ∈ [0, 1] if and only if there exists a u ∈ pbrU (0) such that u ≤ −A0/A1: if no such u exists, whenever V plays 0 either U``s response is not in pbrU (0) or V can improve its payoff by playing 1.
Therefore, either B(W, V )|V =0 = [0, 1] or B(W, V )|V =0 = ∅.
Similarly, B(W, V )|V =1 is equal to either [0, 1] or ∅, depending on pbrU (1).
Therefore, the set B(W, V ) consists of at most 2j + 4 ≤ (j + 4)2 rectangles: B(W, V ) ∩ [0, 1]×(0, 1) = [0, 1]×V contributes at most 2j + 2 rectangles, and each of the sets B(W, V )|V =0 and B(W, V )|V =1 contributes at most one rectangle.
Similarly, its total number of event points is at most 2j + 4: the only W -event points are 0 and 1, each V -event point of B(W, V ) is a V -event point of B(V, U), and there are at most 2j + 2 of them.
• B1u + B0 ≡ 0, A1 = αB1, A0 = αB0 for some α ∈ R.
In this case, V is indifferent between 0 and 1 if and only if w = α, or B1 = 0 and u = −B0/B1 = −A0/A1.
Similarly to the previous case, we can show that B(W, V )∩[0, 1]×(0, 1) consists of the rectangle {α}×[0, 1] and at most 2j + 2 rectangles of the form [0, 1]×IV , where each IV corresponds to a connected component of B(V, U)|U=−B0/B1 .
Furthermore, V prefers to play 0 if B1u + B0 > 0 and w ≥ α or B1u + B0 < 0 and w ≤ α.
Therefore, if B1u∗ + B0 > 0 for some u∗ ∈ pbrU (0), then B(W, V )|V =0 contains [α, +∞) ∩ [0, 1] and if B1u∗∗ + B0 < 0 for some u∗∗ ∈ pbrU (0), then B(W, V )|V =0 contains [−∞, α] ∩ [0, 1]; if both u∗ and u∗∗ exist, B(W, V )|V =0 = [0, 1].
The set B(W, V )|V =1 can be described in a similar manner.
By the inductive hypothesis, B(V, U) has at most 2j + 2 event points; as at least two of these are U-event points, it has at most 2j V -event points.
Since each V -event point of B(W, V ) is a Vevent point of B(V, U) and B(W, V ) has at most 3 W -event points (0, 1, and α), its total number of event points is at most 2j + 3 < 2j +4.
Also, similarly to the previous case it follows that B(W, V ) consists of at most 2j + 4 < (j + 4)2 rectangles.
• B1u + B0 ≡ 0, α(B1u + B0) ≡ A1u + A0.
In this case, one can define the indifference function f(·) as f(u) = A(u) B(u) = A1u+A0 B1u+B0 , where A(u) and B(u) never turn into zero simultaneously.
Observe that whenever w = f(u) and u, w ∈ [0, 1], V is indifferent between playing 0 and 1.
For any A ⊆ [0, 1]2 , we define a function ˆfV by ˆfV (A) = {(f(u), v) | (v, u) ∈ A}; note that ˆfV maps subsets of [0, 1]2 to subsets of R×[0, 1].
Sometimes we drop the subscript V when it is clear from the context.
LEMMA 1.
For any (w, v) ∈ [0, 1]×(0, 1) we have (w, v) ∈ B(W, V ) if and only if there exists a u ∈ [0, 1] such that (v, u) ∈ B(V, U) and w = f(u).
PROOF.
Fix an arbitrary v ∈ (0, 1).
Suppose that U plays some u ∈ pbrU (v), w = f(u) satisfies w ∈ [0, 1], and W plays w.
There exists a vector of strategies v1, ... , vj−1 = u, vj = v such that for each Vk, k < j, its strategy is a best response to its neighbours'' strategies.
Since w = f(u), V is indifferent between playing 0 and 1; in particular, it can play v. Therefore, if we define vj+1 = w, the vector of strategies (v1, ... , vj+1) will satisfy the conditions in the definition of potential best response, i.e., we have v ∈ pbrV (w).
Conversely, suppose v ∈ pbrV (w) for some w ∈ [0, 1], v = 0, 1.
Then there exists a vector of strategies v1, ... , vj−1, vj = v, vj+1 = w such that for each Vk, k ≤ j, its strategy is a best response to its neighbours'' strategies.
As v = 0, 1, V is, in fact, indifferent between playing 0 and 1, which is only possible if w = f(vj−1).
Choose u = vj−1; by construction, u ∈ pbrU (v).
Lemma 1 describes the situations when V is indifferent between playing 0 and playing 1.
However, to fully characterize B(W, V ), we also need to know when V prefers a pure strategy.
Define ˆf(0) = ∪u∈pbrU (0)Ru, where Ru = ´ [f(u), +∞)×{0} if B(u) > 0, (−∞, f(u)]×{0} if B(u) < 0.
and ˆf(1) = ∪u∈pbrU (1)Ru, where Ru = ´ [f(u), +∞)×{1} if B(u) < 0, (−∞, f(u)]×{1} if B(u) > 0.
LEMMA 2.
For any w ∈ [0, 1], we have (w, 0) ∈ ˆf(0) if and only if 0 ∈ pbrV (w) and (w, 1) ∈ ˆf(1) if and only if 1 ∈ pbrV (w).
PROOF.
Consider an arbitrary u0 ∈ pbrU (0).
If B(u0) > 0, for u = u0 the inequality P0 ≥ P1 is equivalent to w ≥ f(u0).
Therefore, when U plays u0 and W plays w, w ≥ f(u0), V prefers to play 0; as u0 ∈ pbrU (u), it follows that 0 ∈ pbrV (w).
The argument for the case B(u0) < 0 is similar.
Conversely, if 0 ∈ pbrV (w) for some w ∈ [0, 1], there exists a vector (v1, ... , vj−1, vj = 0, vj+1 = w) such that for each Vk, k ≤ j, Vk plays vk, and this strategy is a best response to the strategies of Vk``s neighbours.
Note that for any such vector we have vj−1 ∈ pbrU (0).
By way of contradiction, assume (w, 0) ∈ Ë u∈pbrU (0) Ru.
Then it must be the case that for any u0 ∈ pbrU (0) either f(u0) < w and Ru0 = (−∞, f(u0)]×{0} or f(u0) > w and Ru0 = [f(u0), +∞)×{0}.
In both cases, when V plays 0, U plays u0, and V plays w, the inequality between f(u0) and w is equivalent to P0 < P1 , i.e., V would benefit from switching to 1.
The argument for ˆf(1) is similar.
Together, Lemma 1 and Lemma 2 completely describe the set B(W, V ): we have B(W, V ) = ˆf(0) ∪ ˆf(B(V, U)) ∪ ˆf(1) [0, 1]2 .
It remains to show that B(W, V ) can be represented as a union of at most (j + 4)2 rectangles, has at most 2j + 4 event points, and can be computed in O(j2 ) time.
Set u∗ = −B0/B1.
2 Consider an arbitrary rectangle R = [v1, v2]×[u1, u2] ⊆ B(V, U).
If u∗ ∈ [u1, u2], the function f(·) is continuous on [u1, u2] and hence ˆf(R) = [fmin, fmax]×[v1, v2], where fmin = min{f(u1), f(u2)}, fmax = max{f(u1), f(u2)}, i.e., in this case ˆf(R) ∩ [0, 1]2 consists of a single rectangle.
Now, suppose that R is intersected by the line [0, 1]×{u∗ }; as was noted earlier, there are at most 2j+2 such rectangles.
Suppose that limu→u∗− f(u) = +∞; as f(·) is a fractional linear function, this implies that limu→u∗+ f(u) = −∞ and also f(u1) > f(u2).
Since f(·) is continuous on [u1, u∗ ) and (u∗ , u2], it is easy to see that ˆf([v1, v2]×[u1, u∗ )) = [f(u1), +∞)×[v1, v2] 2 The case B1 = 0 causes no special problems.
For completeness, set u∗ to be any value outside of [0, 1] in this case.
104 v u v u* 1 f(0) f(a)f(b) f(1) a b (0, 0) w v 2 v (0, 0) 1 1 1 v 2 v 1 1 Figure 4: f is increasing on (−∞, u∗ ) and (u∗ , +∞).
and ˆf([v1, v2]×(u∗ , u2]) = (−∞, f(u2)]×[v1, v2], i.e., in this case ˆf(R) ∩ [0, 1]2 consists of at most two rectangles.
The case limu→u∗− f(u) = −∞ is similar.
As ˆf(B(V, U)) = Ë R⊂B(V,U) ˆf(R), it follows that ˆf(B(V, U)) consists of at most (j + 3)2 + 2j + 2 rectangles.
Also, it is easy to see that both ˆf(0) and ˆf(1) consist of at most 2 line segments each.
We conclude that B(W, V ) can be represented as a union of at most (j + 3)2 + 2j + 6 < (j + 4)2 rectangles.
Moreover, if v is a V -event point of B(W, V ), then v is a Vevent point of B(V, U) (this includes the cases v = 0 and v = 1, as 0 and 1 are V -event points of B(V, U)) and if w is a W -event point of B(W, V ), then either w = 0 or w = 1 or there exists some u ∈ [0, 1] such that w = f(u) and u is a U-event point of B(V, U).
Hence, B(W, V ) has at most 2j + 4 event points.
The O(j2 ) bound on the running time in Theorem 5 follows from our description of the algorithm.
The O(n3 ) bound on the overall running time for finding a Nash equilibrium (and a representation of all Nash equilibria) follows.
4.1 Finding a Single Nash Equilibrium in O(n2 ) Time The upper bound on the running time of our algorithm is tight, at least assuming the straightforward implementation, in which each B(Vj+1, Vj) is stored as a union of rectangles: it is not hard to construct an example in which the size of B(Vj+1, Vj) is Ω(j2 ).
However, in some cases it is not necessary to represent all Nash equilibria; rather, the goal is to find an arbitrary equilibrium of the game.
In this section, we show that this problem can be solved in quadratic time, thus obtaining a proof of Theorem 1.
Our solution is based on the idea of [9], i.e., working with subsets of the best response policies rather than the best response policies themselves; following [9], we will refer to such subsets as breakpoint policies.
While it is not always possible to construct a breakpoint policy as defined in [9], we show how to modify this definition so as to ensure that a breakpoint policy always exists; moreover, we prove that for a path graph, the breakpoint policy of any vertex can be stored in a data structure whose size is linear in the number of descendants this vertex has.
Definition 3.
A breakpoint policy ˆB(V, U) for a vertex U whose parent is V is a non-self-intersecting curve of the form X1 ∪ Y1 ∪ · · · ∪ Ym−1 ∪ Xm, where Xi = [vi−1, vi]×{ui}, Yi = {vi}×[ui, ui+1] and ui, vi ∈ [0, 1] for i = 0, ... , m.
We say that a breakpoint policy is valid if v0 = 0, vm = 1, and ˆB(V, U) ⊆ B(V, U).
We will sometimes abuse notation by referring to ˆB(V, U) as a collection of segments Xi, Yi rather than their union.
Note that we do not require that vi ≤ vi+1 or ui ≤ ui+1; consequently, in any argument involving breakpoint policies, all segments are to be treated as directed segments.
Observe that any valid breakpoint policy ˆB(V, U) can be viewed as a continuous 1-1 mapping γ(t) = (γv(t), γu(t)), γ : [0, 1] → [0, 1]2 , where γ(0) = (0, u1), γ(1) = (1, um) and there exist some t0 = 0, t1, ... , t2m−2 = 1 such that {γ(t) | t2k ≤ t ≤ t2k+1} = Xk+1, {γ(t) | t2k+1 ≤ t ≤ t2k+2} = Yk+1.
As explained in Section 3, we can use a valid breakpoint policy instead of the best response policy during the downstream pass, and still guarantee that in the end, we will output a Nash equilibrium.
Theorem 6 shows that one can inductively compute valid breakpoint policies for all vertices on the path; the proof of this theorem can be found in the full version of this paper [6].
THEOREM 6.
For any V = Vj, one can find in polynomial time a valid breakpoint policy ˆB(W, V ) that consists of at most 2j + 1 segments.
5.
NASH EQUILIBRIA ON GRAPHS WITH MAXIMUM DEGREE 2 In this section we show how the algorithm for paths can be applied to solve a game on any graph whose vertices have degree at most 2.
A graph having maximum degree 2 is, of course, a union of paths and cycles.
Since each connected component can be handled independently, to obtain a proof of Theorem 2, we only need to show how to deal with cycles.
Given a cycle with vertices V1, ... , Vk (in cyclic order), we make two separate searches for a Nash equilibrium: first we search for a Nash equilibrium where some vertex plays a pure strategy, then we search for a fully mixed Nash equilibrium, where all vertices play mixed strategies.
For i ≤ k let vi denote the probability that Vi plays 1.
The first search can be done as follows.
For each i ∈ {1, ... , k} and each b ∈ {0, 1}, do the following.
1.
Let P be the path (Vi+1, Vi+2 ... , Vk, V1, ... , Vi−1, Vi) 2.
Let payoff to Vi+1 be based on putting vi = b (so it depends only on vi+1 and vi+2.)
3.
Apply the upstream pass to P 4.
Put vi = b; apply the downstream pass For each vertex, Vj, keep track of all possible mixed strategies vj 5.
Check whether Vi+1 has any responses that are consistent with vi = b; if so we have a Nash equilibrium.
(Otherwise, there is no Nash equilibrium of the desired form.)
For the second search, note that if Vi plays a mixed strategy, then vi+1 and vi−1 satisfy an equation of the form vi+1 = (A0 + A1vi−1)/(B0 + B1vi−1).
Since all vertices in the cycle play mixed strategies, we have vi+3 = (A0 +A1vi+1)/(B0 +B1vi+1).
Composing the two linear fractional transforms, we obtain vi+3 = (A0 +A1 vi−1)/(B0 +B1 vi−1).
for some new constants A0 , A1 , B0 , B1 .
Choose any vertex Vi.
We can express vi in terms of vi+2, then vi+4, vi+6 etc. and ultimately vi itself to obtain a quadratic equation (for vi) that is simple to derive from the payoffs in the game.
If the equation is non-trivial it has at most 2 solutions in (0, 1).
For an odd-length cycle all other vj ``s are derivable from those solutions, and if a fully mixed Nash equilibrium exists, all the vj should turn out to be real numbers in the range (0, 1).
For an even-length cycle, we obtain two quadratic equations, one for vi and another for 105 vi+1, and we can in the same way test whether any solutions to these yield values for the other vj , all of which lie in (0, 1).
If the quadratic equation is trivial, there is potentially a continuum of fully-mixed equilibria.
The values for vi that may occur in a Nash equilibrium are those for which all dependent vj values lie in (0, 1); the latter condition is easy to check by computing the image of the interval (0, 1) under respective fractional linear transforms.
6.
FINDING EQUILIBRIA ON AN (ARBITRARY) TREE For arbitrary trees, the general structure of the algorithm remains the same, i.e., one can construct a best response policy (or, alternatively, a breakpoint policy) for any vertex based on the best response policies of its children.
We assume that the degree of each vertex is bounded by a constant K, i.e., the payoff matrix for each vertex is of size O(2K ).
Consider a vertex V whose children are U1, ... , Uk and whose parent is W ; the best response policy of each Uj is B(V, Uj).
Similarly to the previous section, we can compute V ``s expected payoffs P0 and P1 from playing 0 or 1, respectively.
Namely, when each of the Uj plays uj and W plays w, we have P0 = L0 (u1, ... , uk, w), P 1 = L1 (u1, ... , uk, w), where the functions L0 (·, ... , ·), L1 (·, ... , ·) are linear in all of their arguments.
Hence, the inequality P0 > P1 can be rewritten as wB(u1, ... , uk) > A(u1, ... , uk), where both A(·, ... , ·) and B(·, ... , ·) are linear in all of their arguments.
Set u = (u1, ... , uk) and define the indifference function f : [0, 1]k → [0, 1] as f(u) = A(u)/B(u); clearly, if each Uj plays uj, W plays w and w = f(u), V is indifferent between playing 0 and 1.
For any X = X1 × · · · × Xk, where Xi ⊆ [0, 1]2 define ˆf(X) = {(f(u), v) | (v, ui) ∈ Xi, i = 1, ... , k} Also, set ˆf(0) = {(w, 0) | ∃u s.t. ui ∈ pbrUi (0) and wB(u) ≥ A(u)} and ˆf(1) = {(w, 1) | ∃u s.t. ui ∈ pbrUi (1) and wB(u) ≤ A(u)}.
As in previous section, we can show that B(W, V ) is equal to ˆf(0) ∪ ˆf(B(V, U1) × · · · × B(V, Uk)) ∪ ˆf(1) [0, 1]2 ; also, any path from w = 0 to w = 1 that is a subset of B(W, V ) constitutes a valid breakpoint policy.
6.1 Exponential Size Breakpoint Policy While the algorithm of Section 4 can be generalized for boundeddegree trees, its running time is no longer polynomial.
In fact, the converse is true: we can construct a family of trees and payoff matrices for all players so that the best response policies for some of the players consist of an exponential number of segments.
Moreover, in our example the breakpoint policies coincide with the best response policies, which means that even finding a single Nash equilibrium using the approach of [8, 9] is going to take exponentially long time.
In fact, a stronger statement is true: for any polynomial-time two-pass algorithm (defined later) that works with subsets of best response policies for this graph, we can choose the payoffs of the vertices so that the downstream pass of this algorithm will fail.
S 1 1 T S n−1 00 0000 11 1111 00 0000 11 1111 00 0000 11 1111 0 00 1 11 0 00 1 11 0 00 1 11 0 00 1 11 0 00 1 11 0 00 1 11 0 00 1 11 00 0000 11 1111 00 0000 11 1111 0 00 1 11 0000 00000000 00000000 0000 1111 11111111 11111111 1111 000 000000 000000 000 111 111111 111111 111 S S T T T 2 n−1 n 2 n 1 n−12 n VVVVV 0 Figure 5: The tree Tn that corresponds to exponential-size breakpoint policy.
In the rest of this subsection, we describe this construction.
Consider the tree Tn given by Figure 5; let Vn be the root of this tree.
For every k = 1, ... , n, let the payoffs of Sk and Tk be the same as those for the U and V described in Section 3; recall that the breakpoint policies for U and V are shown in Figure 2.
It is not hard to see that the indifference function for Tk is given by f(s) = .8s+.1.
The payoff of V0 is 1 if V1 selects the same action as V0 and 0 otherwise; V0``s best response policy is given by Figure 6.
LEMMA 3.
Fix k < n, and let u, t, v, and w denote the strategies of Vk−1, Tk, Vk, and Vk+1, respectively.
Suppose that Vk prefers playing 0 to playing 1 if and only if .5t + .1u + .2 > w.
Then B(Vk+1, Vk) consists of at least 3k segments.
Moreover, {(v, w) | (v, w) ∈ B(Vk+1, Vk), 0 ≤ w ≤ .2} = [0, .2]×{0} and {(v, w) | (v, w) ∈ B(Vk+1, Vk), .8 ≤ w ≤ 1} = [.8, 1]×{1}.
PROOF.
The proof proceeds by induction on k. For k = 0, the statement is obvious.
Now, suppose it is true for B(Vk, Vk−1).
One can view B(Vk+1, Vk) as a union of seven components: ˆf(0) ∩ [0, 1]×{0}, ˆf(1) ∩ [0, 1]×{1}, and five components that correspond to the segments of B(Vk, Tk).
Let us examine them in turn.
To describe ˆf(0)∩[0, 1]×{0}, note that f(u, t) = .5t+.1u+.2 is monotone in t and u and satisfies f(0, 0) = .2.
Also, we have pbrVk−1 (0) = {0} and pbrTk (0) = {0}.
For any w ∈ [0, 1] we have f(0, 0) ≥ w if and only if w ∈ [0, .2].
We conclude that ˆf(0) ∩ [0, 1]×{0} = [0, .2]×{0}.
Similarly, it follows that ˆf(1) ∩ [0, 1]×{1} = [.8, 1]×{1}.
Define S1 = {(f(u, 0), v) | (v, u) ∈ B(Vk, Vk−1) ∩ [0, .9]×[0, 1]}, S2 = {(f(u, .5), v) | (v, u) ∈ B(Vk, Vk−1) ∩ [.1, .9]×[0, 1]}, S3 = {(f(u, 1), v) | (v, u) ∈ B(Vk, Vk−1) ∩ [.1, 1]×[0, 1]}; these sets correspond to horizontal segments of B(Vk, Tk).
It is easy to see that S1, S2, S3 ⊂ B(Vk+1, Vk).
Since f is a continuous function, the number of segments in each Si is at least the number of segments in B(Vk, Vk−1)∩[.1, .9]×[0, 1], which is at least 3k−1 by induction hypothesis.
Moreover, as f is monotone in u and f(1, 0) < f(0, .5) < f(1, .5) < f(0, 1), all Si, i = 1, 2, 3, are disjoint.
Finally, the set B(Vk+1, Vk) contains two segments that correspond to the vertical segments of B(Vk, Tk), i.e., S4 = {(f(0, t), .1) | t ∈ [.5, 1]) = [.45, .7]×{.1} and S5 = {(f(1, t), .9) | t ∈ [0, .5]) = [.3, .55]×{.9}.
Clearly, S4 connects S2 and S3, S5 connects S1 and S2, and S4 and S5 do not intersect each other.
We conclude that B(Vk+1, Vk) 106 0 00 00 00 00 00 00 00 00 00 0 1 11 11 11 11 11 11 11 11 11 1 00000000001111111111 1 1 10.8 1 1 0.9 0.1 V V0.5 0.2 V V 21 10 Figure 6: Breakpoint policies for V0 and V1.
is a continuous line that consist of at least 3k segments and satisfies the condition of the lemma.
To complete the construction, we need to show that we can design the payoff matrix for Vk so that it prefers playing 0 to playing 1 if and only if .5t + .1u + .2 > w. To this end, we prove a more general statement, namely, that the indifference function of a vertex can be an arbitrary fractional multilinear function of its descendants'' strategies.
We say that a function of k variables is multilinear if it can be represented as a sum of monomials and each of these monomials is linear in all of its variables.
Note that this definition is different from a more standard one in that we do not require that all of the monomials have the same degree.
Recall that the payoffs of a vertex with k + 1 neighbours are described by matrices P0 and P1 , where Pj i0i1...ik is the payoff that V gets when it plays j, and its neighbours play i0, ... , ik, and j, i0, ... , ik ∈ {0, 1}.
Let P[j] = P[j](w, u1, ... , uk) be the expected payoff obtained by this vertex when it plays j and the (mixed) strategies of its neighbours are given by a vector (w, u1, ... , uk), i.e., P[j] = E[P j i0i1...ik ] where i0, ... , ik are independent Bernoulli random variables, each of which is 1 with the respective probabilities w, u1, ... , uk.
LEMMA 4.
Given a tree vertex V whose parent is W and whose children are U1, ... , Uk, for any function f = f(u1, ... , uk) that can be represented as a ratio of two multilinear functions f1, f2, i.e., f = f1(u1,...,uk) f2(u1,...,uk) , there exist payoff matrices P0 and P1 for V such that P[0] − P[1] = wf2(u1, ... , uk) − f1(u1, ... , uk).
The proof of this lemma is based on the fact that every monomial of the form as(u0)s0 ... (uk)sk , s1, ... , sk ∈ {0, 1}, can be represented as t=t0...tk∈Σk+1 Ct(u0)t0 (1 − u0)1−t0 ... (uk)tk (1 − uk)1−tk for some Ct, t ∈ {0, 1}k+1 .
The details can be found in the full version of this paper [6].
6.2 Irreducibility of the Best Response Policy for Tn While the best response policy constructed in the previous subsection has exponential size, it is not clear `a priori that it is necessary to keep track of all of its line segments rather than to focus on a small subset of these segments.
However, it turns out that for two-pass algorithms such as the algorithm of [8], the best response policy cannot be simplified.
More precisely, we say that an algorithm A is a two-pass algorithm if 0 0 0 00 0 0 0 0 00 0 0 0 0 00 0 0 0 0 1 1 1 11 1 1 1 1 11 1 1 1 1 11 1 1 1 1 0000000000000000000000000000011111111111111111111111111111 0.2 0.8 0.9 1 0.1 1 V V 2 3 S 1 S S S S 1 T 0 T 2 3 4 5 Figure 7: Breakpoint policy for V2.
• A consists of an upstream pass and a downstream pass.
• During the upstream pass, for each vertex V with parent W , A constructs a set BB(W, V ) ⊆ B(W, V ).
This set is produced from the sets {BB(V, U) | U is a child of V } by applying the procedure from the beginning of Section 6 (substituting BB(V, Uj ) for B(V, Uj) for all children Uj of V ) , and then possibly omitting some of the points of the resulting set (which is then stored explicitly).
• The downstream pass is identical to the downstream pass of [8] as described in Section 2 except that it operates on the sets BB(W, V ) rather than on the sets B(W, V ).
Theorem 7 demonstrates that any two-pass algorithm will fail during the downstream pass on Tn if there is an index j such that the set BB(Vj+1, Vj) omits any interior point of any of the (at least 3j ) segments of B(Vj+1, Vj).
This implies Theorem 3.
THEOREM 7.
For any two-pass algorithm A for which there exists an index j, j ∈ [1, n/4], a segment S of B(Vj , Vj−1), and an interior point (x, y) of S such that BB(Vj, Vj−1) does not contain (x, y), we can choose payoff matrices of the vertices Vj, ... , Vn so that the downstream pass of A will fail, and, additionally, payoffs to V4j , ... , Vn are identically 0.
We sketch the proof of Theorem 7; the details can be found in the full version of this paper [6].
We proceed by induction.
For j = 1, the argument is similar to that in Section 3.
For the inductive step, the main idea is that we can zoom in on any part of a best response policy (including the part that was omitted!)
by using an appropriate indifference function; this allows us to reduce the case j = j0 to j = j0 − 1.
7.
PPAD-COMPLETENESS OF BOUNDED PATHWIDTH GRAPHICAL GAMES In the previous section, we showed that for graphical games on trees that are almost but not quite paths, two-pass algorithms fail to find Nash equilibria in polynomial time.
We next show that a milder path-like graph property allows us to construct graphical games for which it is unlikely that any polynomial-time algorithm will find Nash equilibria.
7.1 Pathwidth A path decomposition of a graph G = (V, E) is a sequence of subset Si(V ) ⊆ V such that for each edge (v, v ) ∈ E, v, v ∈ Si(V ) for some i, and furthermore, for each v ∈ V , if v ∈ Si(V ) and v ∈ Sj(V ) for j > i, then v ∈ Sk(V ) for all i ≤ k ≤ j.
The path decomposition has width k if all sets Si(V ) have cardinality at most k + 1.
The pathwidth of G is the minimum width of any path decomposition of G. 107 Pathwidth is a restriction of treewidth (in which one would seek a tree whose vertices were the sets Si(V ), and the sets containing some vertex would have to form a subtree).
For any constant k it can be decided in polynomial time whether a graph has pathwidth (or treewidth) k. Furthermore many graph-theoretic problems seem easier to solve in polynomial time, when restricted to fixed treewidth, or pathwidth, graphs, see [1] for an overview.
Note that a path has pathwidth 1 and a cycle has pathwidth 2.
7.2 PPAD-completeness We review some basic definitions from the computational complexity theory of search problems.
A search problem associates any input (here, a graphical game) with a set of solutions (here, the Nash equilibria of the input game), where the description length of any solution should be polynomially bounded as a function of the description length of its input.
In a total search problem, there is a guarantee that at least one solution exists for any input.
Nash``s theorem assures us that the problem of finding Nash equilibria is total.
A reduction from search problem S to problem S is a mechanism that shows that any polynomial-time algorithm for S implies a polynomial-time algorithm for S.
It consists of functions f and g, computable in polynomial time, where f maps inputs of S to inputs of S , and g maps solutions of S to solutions of S, in such a way that if IS is an input to S, and SS is a solution to f(IS), then g(SS ) is a solution to IS.
Observe that total search problems do not allow the above reductions from problems such as CIRCUIT SAT (where the input is a boolean circuit, and solutions are input vectors that make the output true) due to the fact that CIRCUIT SAT and other NP-complete problems have inputs with empty solution sets.
Instead, recent work on the computational complexity of finding a Nash equilibrium [7, 4, 5, 2, 3] has related it to the following problem.
Definition 4.
END OF THE LINE.
Input: boolean circuits S and P, each having n input and n output bits, where P(0n ) = 0n and S(0n ) = 0n .
Solution: x ∈ {0, 1}n such that S(x) = x, or alternatively x ∈ {0, 1}n such that P(S(x)) = x. S and P can be thought of as standing for successor and predecessor.
Observe that by computing Si (0n ) (for i = 0, 1, 2, ...) and comparing with P(Si+1 (0n )), we must eventually find a solution to END OF THE LINE.
END OF THE LINE characterizes the complexity class PPAD (standing for parity argument on a graph, directed version), introduced in Papadimitriou [11], and any search problem S is PPAD-complete if END OF THE LINE reduces to S.
Other PPAD-complete problems include the search for a ham sandwich hyperplane, and finding market equilibria in an exchange economy (see [11] for more detailed descriptions of these problems).
3-GRAPHICAL NASH is the problem of finding a Nash equilibrium for a graphical game whose graph has degree 3.
Daskalakis et al. [4] show PPAD-completeness of 3-GRAPHICAL NASH by a reduction from 3-DIMENSIONAL BROUWER, introduced in [4] and defined as follows.
Definition 5.
3-DIMENSIONAL BROUWER.
Input: a circuit C having 3n input bits and 2 output bits.
The input bits define a cubelet of the unit cube, consisting of the 3 coordinates of its points, given to n bits of precision.
The output represents one of four colours assigned by C to a cubelet.
C is restricted so as to assign colour 1 to cubelets adjacent to the (y, z)-plane, colour 2 to remaining cubelets adjacent to the (x, z)-plane, colour 3 to remaining cubelets on the (x, y)-plane, and colour 0 to all other cubelets on the surface of the unit cube.
A solution is a panchromatic vertex, a vertex adjacent to cubelets that have 4 distinct colours.
The reason why a solution is guaranteed to exist, is that an associated Brouwer function φ can be constructed, i.e. a continuous function from the unit cube to itself, such that panchromatic vertices correspond to fixpoints of φ.
Brouwer``s Fixpoint Theorem promises the existence of a fixpoint.
The proof of Theorem 4 uses a modification of the reduction of [4] from 3-DIMENSIONAL BROUWER to 3-GRAPHICAL NASH.
To prove the theorem, we begin with some preliminary results as follows.
Each player has 2 actions, denoted 0 and 1.
For a player at vertex V let p[V ] denote the probability that the player plays 1.
LEMMA 5.
[7] There exists a graphical game Gshift of fixed size having vertices V , V where p[V ] is the fractional part of 2p[V ].
COROLLARY 1.
There exists a graphical game Gn−shift of size Θ(n) of constant pathwidth, having vertices V , Vn where p[Vn] is the fractional part of 2n .
p[V ].
PROOF.
Make a chain of n copies of Gshift in Lemma 5.
Each subset of vertices in the path decomposition is the vertices in a copy of Gshift.
Let In(x) denote the n-th bit of the binary expansion of x, where we interpret 1 as true and 0 as false.
The following uses gadgets from [7, 4].
COROLLARY 2.
There exists k such that for all n, and for all n1, n2, n3 ≤ n, there exists a graphical game of size O(n) with pathwidth k, having vertices V1, V2, V3 where p[V3] = p[V1] + 2−n3 (In1 p[V1] ∧ In2 p[V2]).
PROOF OF THEOREM 4.
Let C be the boolean circuit describing an instance of 3-DIMENSIONAL BROUWER.
Let g1, ... , gp(n) be the gates of C indexed in such a way that the input(s) to any gate are the output(s) of lower-indexed gates.
g1, ... , g3n will be the 3n inputs to C. All players in the graphical game G constructed in [4] have 2 actions denoted 0 and 1.
The probability that V plays 1 is denoted p[V ].
G has 3 players Vx, Vy and Vz for which p[Vx], p[Vy] and p[Vz] represent the coordinates of a point in the unit cube.
G is designed to incentivize Vx, Vy and Vz to adjust their probabilities in directions given by a Brouwer function which is itself specified by the circuit C.
In a Nash equilibrium, p[Vx], p[Vy] and p[Vz] represent coordinates of a fixpoint of a function that belongs to the class of functions represented by 3-DIMENSIONAL BROUWER.
For 1 ≤ i ≤ p(n) we introduce a vertex V (i) C such that for 1 ≤ j ≤ i, Ij(p[V (i) C ]) is the output of gate gj; for i < j ≤ p(n), Ij(p[V (i) C ]) is 0.
Construct V (i) C from V (i−1) C using Corollary 2.
Let G(i) be the graphical game that does this.
Let S1(G(i) ), ... , Sn(G(i) ) be a length n path decomposition of G(i) , where V (i−1) C ∈ S1(G(i) ) and V (i) C ∈ Sn(G(i) ).
Then, a path decomposition of ∪1≤i≤p(n)G(i) is obtained by taking the union of the separate path decompositions, together with Sn(G(i−1) ) ∪ S1(G(i) ) for 2 ≤ i ≤ p(n).
Let GC be the above graphical game that simulates C. GC has 3n inputs, consisting of the first n bits of the binary expansions of p[Vx], p[Vy] and p[Vz].
Similarly to [4], the output of GC affects Vx, Vy and Vz as follows.
Colour 0 incentivizes Vx, Vy and Vz 108 to adjust their probabilities p[Vx], p[Vy] and p[Vz] in the direction (−1, −1, −1); colour 2 incentivizes them to move in direction (1, 0, 0); colour 2, direction (0, 1, 0); colour 3, direction (0, 0, 1).
We need to ensure that at points at the boundaries of adjacent cubelets, the change of direction will be approximately the average of directions of surrounding points.
That way, all four colors/directions must be nearby so that they can cancel each other out (and we are at a panchromatic vertex).
This is achieved using the same trick as [4], in which we make a constant number M of copies of GC, which differ in that each copy adds a tiny displacement vector to its copies of p[Vx], p[Vy] and p[Vz] (which are derived from the original using the addition gadget of [7]).
Using the addition and multiplication gadgets of [7] we average the directions and add a small multiple of this average to (p[Vx], p[Vy], p[Vz]).
At a Nash equilibrium the outputs of each copy will cancel each other out.
The pathwidth of the whole game is at most M times the pathwidth GC.
8.
OPEN PROBLEMS The most important problem left open by this paper is whether it is possible to find a Nash equilibrium of a graphical game on a bounded-degree tree in polynomial time.
Our construction shows that any two-pass algorithm that explicitly stores breakpoint policies needs exponential time and space.
However, it does not preclude the existence of an algorithm that is based on a similar idea, but, instead of computing the entire breakpoint policy for each vertex, uses a small number of additional passes through the graph to decide which (polynomial-sized) parts of each breakpoint policy should be computed.
In particular, such an algorithm may be based on the approximation algorithm of [8], where the value of is chosen adaptively.
Another intriguing question is related to the fact that the graph for which we constructed an exponential-sized breakpoint policy has pathwidth 2, while our positive results are for a path, i.e., a graph of pathwidth 1.
It is not clear if for any bounded-degree graph of pathwidth 1 the running time of (the breakpoint policybased version of) our algorithm will be polynomial.
In particular, it is instructive to consider a caterpillar graph, i.e., the graph that can be obtained from Tn by deleting the vertices S1, ... , Sn.
For this graph, the best response policy of a vertex Vk in the spine of the caterpillar is obtained by combining the best response policy of its predecessor on the spine Vk−1 and its other child Tk; since the latter is a leaf, its best response policy is either trivial (i.e., [0, 1]2 , [0, 1]×{0}, or [0, 1]×{1}) or consists of two horizontal segments and one vertical segment of the form {α}×[0, 1] that connects them.
Assuming for convenience that B(Vk, Tk) = [0, α]×{0} ∪ {α}×[0, 1] ∪ [α, 1]×{1}, and f is the indifference function for Vk, we observe that the best response policy for Vk consists of 5 components: ˆf(0), ˆf(1), and three components that correspond to [0, α]×{0}, {α}×[0, 1], and [α, 1]×{1}.
Hence, one can think of constructing B(Vk+1, Vk) as the following process: turn B(Vk, Vk−1) by π/2, cut it along the (now horizontal) line vk = α, apply a fractional linear transform to the horizontal coordinate of both parts, and reconnect them using the image of the segment {α}×[0, 1] under f.
This implies that the problem of bounding the size of the best response policy (or, alternatively, the breakpoint policy), can be viewed as a generalization of the following computational geometry problem, which we believe may be of independent interest: PROBLEM 1.
Given a collection of axis-parallel segments in R2 , consider the following operation: pick an axis-parallel line li (either vertical or horizontal), cut the plane along this line, and shift one of the resulting two parts by an arbitrary amount δi; as a result, some segments will be split into two parts.
Reconnect these parts, i.e., for each segment of the form [a, b] × {c} that was transformed into [a, t] × {c + δi} and [t, b] × {c}, introduce a segment {t}×[c, c+δi].
Is it possible to start with the segment [0, 1] and after n operations obtain a set that cannot be represented as a union of poly(n) line segments?
If yes, can it be the case that in this set, there is no path with a polynomial number of turns that connects the endpoints of the original segment?
It turns out that in general, the answer to the first question is positive, i.e., after n steps, it is possible to obtain a set that consists of Θ(cn ) segments for some c > 0.
This implies that even for a caterpillar, the best response policy can be exponentially large.
However, in our example (which is omitted from this version of the paper due to space constraints), there exists a polynomial-size path through the best response policy, i.e., it does not prove that the breakpoint policy is necessarily exponential in size.
If one can prove that this is always the case, it may be possible to adapt this proof to show that there can be an exponential gap between the sizes of best response policies and breakpoint policies.
9.
REFERENCES [1] H. Bodlaender and T. Kloks.
Efficient and constructive algorithms for the pathwidth and treewidth of graphs.
Journal of Algorithms, 21:358-402, 1996.
[2] X. Chen and X. Deng.
3-NASH is PPAD-complete.
Technical Report TR-05-134, Electronic Colloquium in Computational Complexity, 2005.
[3] X. Chen and X. Deng.
Settling the complexity of 2-player Nash equilibrium.
Technical Report TR-05-140, Electronic Colloquium in Computational Complexity, 2005.
[4] C. Daskalakis, P. Goldberg, and C. Papadimitriou.
The complexity of computing a Nash equilibrium.
In Proceedings of the 38th ACM Symposium on Theory of Computing, 2006.
[5] C. Daskalakis and C. Papadimitriou.
Three-player games are hard.
Technical Report TR-05-139, Electronic Colloquium in Computational Complexity, 2005.
[6] E. Elkind, L. Goldberg, and P. Goldberg.
Nash equilibria in graphical games on trees revisited.
Technical Report TR-06-005, Electronic Colloquium in Computational Complexity, 2006.
[7] P. Goldberg and C. Papadimitriou.
Reducibility among equilibrium problems.
In Proceedings of the 38th ACM Symposium on Theory of Computing, 2006.
[8] M. Kearns, M. Littman, and S. Singh.
Graphical models for game theory.
In Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence, 2001.
[9] M. Littman, M. Kearns, and S. Singh.
An efficient exact algorithm for singly connected graphical games.
In Proceedings of the 15th Annual Conference on Neural Information Processing Systems, 2001.
[10] L. Ortiz and M. Kearns.
Nash propagation for loopy graphical games.
In Proceedings of the 17th Annual Conference on Neural Information Processing Systems, 2003.
[11] C. Papadimitriou.
On the complexity of the parity argument and other inefficient proofs of existence.
J. Comput.
Syst.
Sci., 48(3):498-532, 1994.
109

Nash Equilibria in Graphical Games on Trees Revisited *
Graphical games have been proposed as a game-theoretic model of large-scale distributed networks of non-cooperative agents.
When the number of players is large, and the underlying graph has low degree, they provide a concise way to represent the players' payoffs.
It has recently been shown that the problem of finding Nash equilibria in a general degree-3 graphical game with two actions per player is complete for the complexity class PPAD, indicating that it is unlikely that there is any polynomial-time algorithm for this problem.
In this paper, we study the complexity of graphical games with two actions per player on bounded-degree trees.
This setting was first considered by Kearns, Littman and Singh, who proposed a dynamic programming-based algorithm that computes all Nash equilibria of such games.
The running time of their algorithm is exponential, though approximate equilibria can be computed efficiently.
Later, Littman, Kearns and Singh proposed a modification to this algorithm that can find a single Nash equilibrium in polynomial time.
We show that this modified algorithm is incorrect--the output is not always a Nash equilibrium.
We then propose a new algorithm that is based on the ideas of Kearns et al. and computes all Nash equilibria in quadratic time if the input graph is a path, and in polynomial time if it is an arbitrary graph of maximum degree 2.
Moreover, our algorithm can be used to compute Nash equilibria of graphical games on arbitrary trees, but the running time can be exponential, even when the tree has bounded degree.
We show that this is inevitable--any algorithm of this type will take exponential time, even on bounded-degree trees with pathwidth 2.
It is an open question whether our algorithm runs in polynomial time on graphs with pathwidth 1, but we show that finding a Nash equilibrium for a 2-action graphical game in which the underlying graph has maximum degree 3 and constant pathwidth is PPAD-complete (so is unlikely to be tractable).
* This research is supported by the EPSRC research grants "Algorithmics of Network-sharing Games" and "Discontinuous Behaviour in the Complexity of randomized Algorithms".
1.
INTRODUCTION
Graphical games were introduced in the papers of Kearns et al. [8] and Littman et al. [9] as a succinct representation of games with a large number of players.
The classical normal form (or matrix form) representation has a size that is exponential in the number of players, making it unsuitable for large-scale distributed games.
A graphical game associates each player with a vertex of an underlying graph G, and the payoff to that player is a function of the actions chosen by himself and his neighbours in G; if G has low degree, this is a concise way to represent a game with many players.
The papers [8, 9] give a dynamic-programming algorithm for finding Nash equilibria in graphical games where there are two actions per player and G is a tree.
The first of these papers describes a generic algorithm for this problem that can be specialized in two ways: as an algorithm that computes approximations to all Nash equilibria in time polynomial in the input size and the approximation quality, or as an exponential-time algorithm that allows the exact computation of all Nash equilibria in G.
In [9], the authors propose a modification to the latter algorithm that aims to find a single Nash equilibrium in polynomial time.
This does not quite work, as we show in Section 3, though it introduces a useful idea.
1.1 Background
The generic algorithm of [8] consists of two phases which we will refer to as the upstream pass and the downstream pass; 1 the former starts at the leaves of the tree and ends at the root, while the latter starts at the root and ends at the leaves.
It is assumed that each player has two pure strategies (actions), which are denoted by 0 and 1; it follows that any mixed strategy can be represented as a single number x ∈ [0, 1], where x is the probability that the player selects 1.
During the upstream pass, each vertex V computes the set of its potential best responses to every mixed strategy w of its parent W; a strategy v is a potential best response to w if 1Note that the terminology "upstream" and "downstream" are reversed in [8, 9]--our trees are rooted at the top.
there is a Nash equilibrium in the graphical game downstream of V (inclusive) given that W plays w (for a more technical definition, the reader is referred to Section 2).
The output of this stage can be viewed as a (continuous) table T (w, v), where T (w, v) = 1 if and only if v is a potential best response to w; we refer to this table as the best response policy for V.
The generic algorithm does not address the problem of representing the best response policy; in fact, the most important difference between the two instantiations of the generic algorithm described in [8] is in their approach to this issue.
The computation is performed inductively: the best response policy for V is computed based on the best response policies of V's children U1,..., Uk.
By the end of the upstream pass, all children of the root have computed their best response policies.
In the beginning of the downstream pass, the root selects its strategy and informs its children about its choice.
It also selects a strategy for each child.
A necessary and sufficient condition for the algorithm to proceed is that the strategy of the root is a best response to the strategies of its children and, for each child, the chosen strategy is one of the pre-computed potential best responses to the chosen strategy of the root.
The equilibrium then propagates downstream, with each vertex selecting its children's actions.
The action of the child is chosen to be any strategy from the pre-computed potential best responses to the chosen strategy of the parent.
To bound the running time of this algorithm, the paper [8] shows that any best response policy can be represented as a union of an exponential number of rectangles; the polynomial time approximation algorithm is obtained by combining this representation with a polynomial-sized grid.
The main idea of [9] is that it is not necessary to keep track of all rectangles in the best response policies; rather, at each step of the upstream pass, it is possible to select a polynomial-size subset of the corresponding policy (in [9], this subset is called a breakpoint policy), and still ensure that the downstream pass can proceed successfully (a sufficient condition for this is that the subset of the best response policy for V stored by the algorithm contains a continuous path from w = 0 to w = 1).
1.2 Our Results
One of the main contributions of our paper is to show that the algorithm proposed by [9] is incorrect.
In Section 3 we describe a simple example for which the algorithm of [9] outputs a vector of strategies that does not constitute a Nash equilibrium of the underlying game.
In Sections 4, 5 and 6 we show how to fix the algorithm of [9] so that it always produces correct output.
Section 4 considers the case in which the underlying graph is a path of length n. For this case, we show that the number of rectangles in each of the best response policies is O (n2).
This gives us an O (n3) algorithm for finding a Nash equilibrium, and for computing a representation of all Nash equilibria.
(This algorithm is a special case of the generic algorithm of [8]--we show that it runs in polynomial time when the underlying graph is a path.)
We can improve the running time of the generic algorithm using the ideas of [9].
In particular, we give an O (n2) algorithm for finding a Nash equilibrium of a graphical game on a path of length n. Instead of storing best response policies, this algorithm stores appropriately-defined subsets, which, following [9], we call breakpoint policies (modifying the definition as necessary).
We obtain the following theorem THEOREM 1.
There is an O (n2) algorithm that finds a Nash equilibrium of a graphical game with two actions per player on an n-vertex path.
There is an O (n3) algorithm that computes a representation of all Nash equilibria of such a game.
In Section 5 we extend the results of Section 4 to general degree2 graphs, obtaining the following theorem.
THEOREM 2.
There is a polynomial-time algorithm thatfinds a Nash equilibrium of a graphical game with two actions per player on a graph with maximum degree 2.
In Section 6 we extend our algorithm so that it can be used to find a Nash equilibrium of a graphical game on an arbitrary tree.
Even when the tree has bounded degree, the running time can be exponential.
We show that this is inevitable by constructing a family of graphical games on bounded-degree trees for which best response policies of some of the vertices have exponential size, and any twopass algorithm (i.e., an algorithm that is similar in spirit to that of [8]) has to store almost all points of the best response policies.
In particular, we show the following.
THEOREM 3.
There is an infinite family ofgraphical games on bounded-degree trees with pathwidth 2 such that any two-pass algorithm for finding Nash equilibria on these trees requires exponential time and space.
It is interesting to note that the trees used in the proof of Theorem 3 have pathwidth 2, that is, they are very close to being paths.
It is an open question whether our algorithm runs in polynomial time for graphs of pathwidth 1.
This question can be viewed as a generalization of a very natural computational geometry problem--we describe it in more detail in Section 8.
In Section 7, we give a complexity-theoretic intractability result for the problem of finding a Nash equilibrium of a graphical game on a graph with small pathwidth.
We prove the following theorem.
THEOREM 4.
Consider the problem offinding a Nash equilibrium for a graphical game in which the underlying graph has maximum degree 3 and pathwidth k.
There is a constant k such that this problem is PPAD-complete.
Theorem 4 limits the extent to which we can exploit "path-like" properties of the underlying graph, in order to find Nash equilibria.
To prove Theorem 4, we use recent PPAD-completeness results for games, in particular the papers [7, 4] which show that the problem of finding Nash equilibria in graphical games of degree d (for d> 3) is computationally equivalent to the problem of solving r-player normal-form games (for r> 4), both of which are PPAD-complete.
2.
PRELIMINARIES AND NOTATION
3.
ALGORITHM OF LITTMAN ET AL. .
4.
FINDING EQUILIBRIA ON A PATH
4.1 Finding a Single Nash Equilibrium in O (n2) Time
5.
NASH EQUILIBRIA ON GRAPHS WITH MAXIMUM DEGREE 2
6.
FINDING EQUILIBRIA ON AN (ARBITRARY) TREE
6.1 Exponential Size Breakpoint Policy
6.2 Irreducibility of the Best Response Policy for Tn
7.
PPAD-COMPLETENESS OF BOUNDED PATHWIDTH GRAPHICAL GAMES
7.1 Pathwidth
7.2 PPAD-completeness
8.
OPEN PROBLEMS
The most important problem left open by this paper is whether it is possible to find a Nash equilibrium of a graphical game on a bounded-degree tree in polynomial time.
Our construction shows that any two-pass algorithm that explicitly stores breakpoint policies needs exponential time and space.
However, it does not preclude the existence of an algorithm that is based on a similar idea, but, instead of computing the entire breakpoint policy for each vertex, uses a small number of additional passes through the graph to decide which (polynomial-sized) parts of each breakpoint policy should be computed.
In particular, such an algorithm may be based on the approximation algorithm of [8], where the value of e is chosen adaptively.
Another intriguing question is related to the fact that the graph for which we constructed an exponential-sized breakpoint policy has pathwidth 2, while our positive results are for a path, i.e., a graph of pathwidth 1.
It is not clear if for any bounded-degree graph of pathwidth 1 the running time of (the breakpoint policybased version of) our algorithm will be polynomial.
In particular, it is instructive to consider a "caterpillar" graph, i.e., the graph that can be obtained from Tn by deleting the vertices S1,..., Sn.
For this graph, the best response policy of a vertex Vk in the "spine" of the caterpillar is obtained by combining the best response policy of its predecessor on the spine Vk − 1 and its other child Tk; since the latter is a leaf, its best response policy is either trivial (i.e., [0, 1] 2, [0, 1] × {0}, or [0, 1] × {1}) or consists of two horizontal segments and one vertical segment of the form {α} × [0, 1] that connects them.
Assuming for convenience that B (Vk, Tk) = [0, α] × {0} ∪ {α} × [0, 1] ∪ [α, 1] × {1}, and f is the indifference function for Vk, we observe that the best response policy for Vk consists of 5 components: ˆf (0), fˆ (1), and three components that correspond to [0, α] × {0}, {α} × [0, 1], and [α, 1] × {1}.
Hence, one can think of constructing B (Vk +1, Vk) as the following process: turn B (Vk, Vk − 1) by π / 2, cut it along the (now horizontal) line vk = α, apply a fractional linear transform to the horizontal coordinate of both parts, and reconnect them using the image of the segment {α} × [0, 1] under f.
This implies that the problem of bounding the size of the best response policy (or, alternatively, the breakpoint policy), can be viewed as a generalization of the following computational geometry problem, which we believe may be of independent interest: PROBLEM 1.
Given a collection of axis-parallel segments in R2, consider the following operation: pick an axis-parallel line li (either vertical or horizontal), cut the plane along this line, and shift one of the resulting two parts by an arbitrary amount δi; as a result, some segments will be split into two parts.
Reconnect these parts, i.e., for each segment of the form [a, b] × {c} that was transformed into [a, t] × {c + δi} and [t, b] × {c}, introduce a segment {t} × [c, c + δi].
Is itpossible to start with the segment [0, 1] and after n operations obtain a set that cannot be represented as a union of poly (n) line segments?
Ifyes, can it be the case that in this set, there is no path with a polynomial number of turns that connects the endpoints of the original segment?
It turns out that in general, the answer to the first question is positive, i.e., after n steps, it is possible to obtain a set that consists of Θ (cn) segments for some c> 0.
This implies that even for a caterpillar, the best response policy can be exponentially large.
However, in our example (which is omitted from this version of the paper due to space constraints), there exists a polynomial-size path through the best response policy, i.e., it does not prove that the breakpoint policy is necessarily exponential in size.
If one can prove that this is always the case, it may be possible to adapt this proof to show that there can be an exponential gap between the sizes of best response policies and breakpoint policies.

Nash Equilibria in Graphical Games on Trees Revisited *
Graphical games have been proposed as a game-theoretic model of large-scale distributed networks of non-cooperative agents.
When the number of players is large, and the underlying graph has low degree, they provide a concise way to represent the players' payoffs.
It has recently been shown that the problem of finding Nash equilibria in a general degree-3 graphical game with two actions per player is complete for the complexity class PPAD, indicating that it is unlikely that there is any polynomial-time algorithm for this problem.
In this paper, we study the complexity of graphical games with two actions per player on bounded-degree trees.
This setting was first considered by Kearns, Littman and Singh, who proposed a dynamic programming-based algorithm that computes all Nash equilibria of such games.
The running time of their algorithm is exponential, though approximate equilibria can be computed efficiently.
Later, Littman, Kearns and Singh proposed a modification to this algorithm that can find a single Nash equilibrium in polynomial time.
We show that this modified algorithm is incorrect--the output is not always a Nash equilibrium.
We then propose a new algorithm that is based on the ideas of Kearns et al. and computes all Nash equilibria in quadratic time if the input graph is a path, and in polynomial time if it is an arbitrary graph of maximum degree 2.
Moreover, our algorithm can be used to compute Nash equilibria of graphical games on arbitrary trees, but the running time can be exponential, even when the tree has bounded degree.
We show that this is inevitable--any algorithm of this type will take exponential time, even on bounded-degree trees with pathwidth 2.
It is an open question whether our algorithm runs in polynomial time on graphs with pathwidth 1, but we show that finding a Nash equilibrium for a 2-action graphical game in which the underlying graph has maximum degree 3 and constant pathwidth is PPAD-complete (so is unlikely to be tractable).
* This research is supported by the EPSRC research grants "Algorithmics of Network-sharing Games" and "Discontinuous Behaviour in the Complexity of randomized Algorithms".
1.
INTRODUCTION
Graphical games were introduced in the papers of Kearns et al. [8] and Littman et al. [9] as a succinct representation of games with a large number of players.
The classical normal form (or matrix form) representation has a size that is exponential in the number of players, making it unsuitable for large-scale distributed games.
A graphical game associates each player with a vertex of an underlying graph G, and the payoff to that player is a function of the actions chosen by himself and his neighbours in G; if G has low degree, this is a concise way to represent a game with many players.
The papers [8, 9] give a dynamic-programming algorithm for finding Nash equilibria in graphical games where there are two actions per player and G is a tree.
The first of these papers describes a generic algorithm for this problem that can be specialized in two ways: as an algorithm that computes approximations to all Nash equilibria in time polynomial in the input size and the approximation quality, or as an exponential-time algorithm that allows the exact computation of all Nash equilibria in G.
In [9], the authors propose a modification to the latter algorithm that aims to find a single Nash equilibrium in polynomial time.
This does not quite work, as we show in Section 3, though it introduces a useful idea.
1.1 Background
The generic algorithm of [8] consists of two phases which we will refer to as the upstream pass and the downstream pass; 1 the former starts at the leaves of the tree and ends at the root, while the latter starts at the root and ends at the leaves.
there is a Nash equilibrium in the graphical game downstream of V (inclusive) given that W plays w (for a more technical definition, the reader is referred to Section 2).
The generic algorithm does not address the problem of representing the best response policy; in fact, the most important difference between the two instantiations of the generic algorithm described in [8] is in their approach to this issue.
The computation is performed inductively: the best response policy for V is computed based on the best response policies of V's children U1,..., Uk.
By the end of the upstream pass, all children of the root have computed their best response policies.
In the beginning of the downstream pass, the root selects its strategy and informs its children about its choice.
It also selects a strategy for each child.
A necessary and sufficient condition for the algorithm to proceed is that the strategy of the root is a best response to the strategies of its children and, for each child, the chosen strategy is one of the pre-computed potential best responses to the chosen strategy of the root.
The equilibrium then propagates downstream, with each vertex selecting its children's actions.
The action of the child is chosen to be any strategy from the pre-computed potential best responses to the chosen strategy of the parent.
To bound the running time of this algorithm, the paper [8] shows that any best response policy can be represented as a union of an exponential number of rectangles; the polynomial time approximation algorithm is obtained by combining this representation with a polynomial-sized grid.
1.2 Our Results
One of the main contributions of our paper is to show that the algorithm proposed by [9] is incorrect.
In Section 3 we describe a simple example for which the algorithm of [9] outputs a vector of strategies that does not constitute a Nash equilibrium of the underlying game.
In Sections 4, 5 and 6 we show how to fix the algorithm of [9] so that it always produces correct output.
Section 4 considers the case in which the underlying graph is a path of length n. For this case, we show that the number of rectangles in each of the best response policies is O (n2).
This gives us an O (n3) algorithm for finding a Nash equilibrium, and for computing a representation of all Nash equilibria.
(This algorithm is a special case of the generic algorithm of [8]--we show that it runs in polynomial time when the underlying graph is a path.)
We can improve the running time of the generic algorithm using the ideas of [9].
In particular, we give an O (n2) algorithm for finding a Nash equilibrium of a graphical game on a path of length n. Instead of storing best response policies, this algorithm stores appropriately-defined subsets, which, following [9], we call breakpoint policies (modifying the definition as necessary).
We obtain the following theorem THEOREM 1.
There is an O (n2) algorithm that finds a Nash equilibrium of a graphical game with two actions per player on an n-vertex path.
There is an O (n3) algorithm that computes a representation of all Nash equilibria of such a game.
In Section 5 we extend the results of Section 4 to general degree2 graphs, obtaining the following theorem.
THEOREM 2.
There is a polynomial-time algorithm thatfinds a Nash equilibrium of a graphical game with two actions per player on a graph with maximum degree 2.
In Section 6 we extend our algorithm so that it can be used to find a Nash equilibrium of a graphical game on an arbitrary tree.
Even when the tree has bounded degree, the running time can be exponential.
We show that this is inevitable by constructing a family of graphical games on bounded-degree trees for which best response policies of some of the vertices have exponential size, and any twopass algorithm (i.e., an algorithm that is similar in spirit to that of [8]) has to store almost all points of the best response policies.
In particular, we show the following.
THEOREM 3.
There is an infinite family ofgraphical games on bounded-degree trees with pathwidth 2 such that any two-pass algorithm for finding Nash equilibria on these trees requires exponential time and space.
It is interesting to note that the trees used in the proof of Theorem 3 have pathwidth 2, that is, they are very close to being paths.
It is an open question whether our algorithm runs in polynomial time for graphs of pathwidth 1.
This question can be viewed as a generalization of a very natural computational geometry problem--we describe it in more detail in Section 8.
In Section 7, we give a complexity-theoretic intractability result for the problem of finding a Nash equilibrium of a graphical game on a graph with small pathwidth.
We prove the following theorem.
THEOREM 4.
Consider the problem offinding a Nash equilibrium for a graphical game in which the underlying graph has maximum degree 3 and pathwidth k.
There is a constant k such that this problem is PPAD-complete.
Theorem 4 limits the extent to which we can exploit "path-like" properties of the underlying graph, in order to find Nash equilibria.
To prove Theorem 4, we use recent PPAD-completeness results for games, in particular the papers [7, 4] which show that the problem of finding Nash equilibria in graphical games of degree d (for d> 3) is computationally equivalent to the problem of solving r-player normal-form games (for r> 4), both of which are PPAD-complete.
8.
OPEN PROBLEMS
The most important problem left open by this paper is whether it is possible to find a Nash equilibrium of a graphical game on a bounded-degree tree in polynomial time.
Our construction shows that any two-pass algorithm that explicitly stores breakpoint policies needs exponential time and space.
However, it does not preclude the existence of an algorithm that is based on a similar idea, but, instead of computing the entire breakpoint policy for each vertex, uses a small number of additional passes through the graph to decide which (polynomial-sized) parts of each breakpoint policy should be computed.
In particular, such an algorithm may be based on the approximation algorithm of [8], where the value of e is chosen adaptively.
Another intriguing question is related to the fact that the graph for which we constructed an exponential-sized breakpoint policy has pathwidth 2, while our positive results are for a path, i.e., a graph of pathwidth 1.
It is not clear if for any bounded-degree graph of pathwidth 1 the running time of (the breakpoint policybased version of) our algorithm will be polynomial.
In particular, it is instructive to consider a "caterpillar" graph, i.e., the graph that can be obtained from Tn by deleting the vertices S1,..., Sn.
This implies that the problem of bounding the size of the best response policy (or, alternatively, the breakpoint policy), can be viewed as a generalization of the following computational geometry problem, which we believe may be of independent interest: PROBLEM 1.
Ifyes, can it be the case that in this set, there is no path with a polynomial number of turns that connects the endpoints of the original segment?
This implies that even for a caterpillar, the best response policy can be exponentially large.
However, in our example (which is omitted from this version of the paper due to space constraints), there exists a polynomial-size path through the best response policy, i.e., it does not prove that the breakpoint policy is necessarily exponential in size.
If one can prove that this is always the case, it may be possible to adapt this proof to show that there can be an exponential gap between the sizes of best response policies and breakpoint policies.

Nash Equilibria in Graphical Games on Trees Revisited *
Graphical games have been proposed as a game-theoretic model of large-scale distributed networks of non-cooperative agents.
When the number of players is large, and the underlying graph has low degree, they provide a concise way to represent the players' payoffs.
It has recently been shown that the problem of finding Nash equilibria in a general degree-3 graphical game with two actions per player is complete for the complexity class PPAD, indicating that it is unlikely that there is any polynomial-time algorithm for this problem.
In this paper, we study the complexity of graphical games with two actions per player on bounded-degree trees.
This setting was first considered by Kearns, Littman and Singh, who proposed a dynamic programming-based algorithm that computes all Nash equilibria of such games.
The running time of their algorithm is exponential, though approximate equilibria can be computed efficiently.
Later, Littman, Kearns and Singh proposed a modification to this algorithm that can find a single Nash equilibrium in polynomial time.
We show that this modified algorithm is incorrect--the output is not always a Nash equilibrium.
We then propose a new algorithm that is based on the ideas of Kearns et al. and computes all Nash equilibria in quadratic time if the input graph is a path, and in polynomial time if it is an arbitrary graph of maximum degree 2.
Moreover, our algorithm can be used to compute Nash equilibria of graphical games on arbitrary trees, but the running time can be exponential, even when the tree has bounded degree.
We show that this is inevitable--any algorithm of this type will take exponential time, even on bounded-degree trees with pathwidth 2.
It is an open question whether our algorithm runs in polynomial time on graphs with pathwidth 1, but we show that finding a Nash equilibrium for a 2-action graphical game in which the underlying graph has maximum degree 3 and constant pathwidth is PPAD-complete (so is unlikely to be tractable).
* This research is supported by the EPSRC research grants "Algorithmics of Network-sharing Games" and "Discontinuous Behaviour in the Complexity of randomized Algorithms".
1.
INTRODUCTION
Graphical games were introduced in the papers of Kearns et al. [8] and Littman et al. [9] as a succinct representation of games with a large number of players.
The classical normal form (or matrix form) representation has a size that is exponential in the number of players, making it unsuitable for large-scale distributed games.
A graphical game associates each player with a vertex of an underlying graph G, and the payoff to that player is a function of the actions chosen by himself and his neighbours in G; if G has low degree, this is a concise way to represent a game with many players.
The papers [8, 9] give a dynamic-programming algorithm for finding Nash equilibria in graphical games where there are two actions per player and G is a tree.
The first of these papers describes a generic algorithm for this problem that can be specialized in two ways: as an algorithm that computes approximations to all Nash equilibria in time polynomial in the input size and the approximation quality, or as an exponential-time algorithm that allows the exact computation of all Nash equilibria in G.
In [9], the authors propose a modification to the latter algorithm that aims to find a single Nash equilibrium in polynomial time.
This does not quite work, as we show in Section 3, though it introduces a useful idea.
1.1 Background
The generic algorithm of [8] consists of two phases which we will refer to as the upstream pass and the downstream pass; 1 the former starts at the leaves of the tree and ends at the root, while the latter starts at the root and ends at the leaves.
It is assumed that each player has two pure strategies (actions), which are denoted by 0 and 1; it follows that any mixed strategy can be represented as a single number x ∈ [0, 1], where x is the probability that the player selects 1.
During the upstream pass, each vertex V computes the set of its potential best responses to every mixed strategy w of its parent W; a strategy v is a potential best response to w if 1Note that the terminology "upstream" and "downstream" are reversed in [8, 9]--our trees are rooted at the top.
there is a Nash equilibrium in the graphical game downstream of V (inclusive) given that W plays w (for a more technical definition, the reader is referred to Section 2).
The output of this stage can be viewed as a (continuous) table T (w, v), where T (w, v) = 1 if and only if v is a potential best response to w; we refer to this table as the best response policy for V.
The generic algorithm does not address the problem of representing the best response policy; in fact, the most important difference between the two instantiations of the generic algorithm described in [8] is in their approach to this issue.
The computation is performed inductively: the best response policy for V is computed based on the best response policies of V's children U1,..., Uk.
By the end of the upstream pass, all children of the root have computed their best response policies.
In the beginning of the downstream pass, the root selects its strategy and informs its children about its choice.
It also selects a strategy for each child.
A necessary and sufficient condition for the algorithm to proceed is that the strategy of the root is a best response to the strategies of its children and, for each child, the chosen strategy is one of the pre-computed potential best responses to the chosen strategy of the root.
The equilibrium then propagates downstream, with each vertex selecting its children's actions.
The action of the child is chosen to be any strategy from the pre-computed potential best responses to the chosen strategy of the parent.
To bound the running time of this algorithm, the paper [8] shows that any best response policy can be represented as a union of an exponential number of rectangles; the polynomial time approximation algorithm is obtained by combining this representation with a polynomial-sized grid.
The main idea of [9] is that it is not necessary to keep track of all rectangles in the best response policies; rather, at each step of the upstream pass, it is possible to select a polynomial-size subset of the corresponding policy (in [9], this subset is called a breakpoint policy), and still ensure that the downstream pass can proceed successfully (a sufficient condition for this is that the subset of the best response policy for V stored by the algorithm contains a continuous path from w = 0 to w = 1).
1.2 Our Results
One of the main contributions of our paper is to show that the algorithm proposed by [9] is incorrect.
In Section 3 we describe a simple example for which the algorithm of [9] outputs a vector of strategies that does not constitute a Nash equilibrium of the underlying game.
In Sections 4, 5 and 6 we show how to fix the algorithm of [9] so that it always produces correct output.
Section 4 considers the case in which the underlying graph is a path of length n. For this case, we show that the number of rectangles in each of the best response policies is O (n2).
This gives us an O (n3) algorithm for finding a Nash equilibrium, and for computing a representation of all Nash equilibria.
(This algorithm is a special case of the generic algorithm of [8]--we show that it runs in polynomial time when the underlying graph is a path.)
We can improve the running time of the generic algorithm using the ideas of [9].
In particular, we give an O (n2) algorithm for finding a Nash equilibrium of a graphical game on a path of length n. Instead of storing best response policies, this algorithm stores appropriately-defined subsets, which, following [9], we call breakpoint policies (modifying the definition as necessary).
We obtain the following theorem THEOREM 1.
There is an O (n2) algorithm that finds a Nash equilibrium of a graphical game with two actions per player on an n-vertex path.
There is an O (n3) algorithm that computes a representation of all Nash equilibria of such a game.
In Section 5 we extend the results of Section 4 to general degree2 graphs, obtaining the following theorem.
THEOREM 2.
There is a polynomial-time algorithm thatfinds a Nash equilibrium of a graphical game with two actions per player on a graph with maximum degree 2.
In Section 6 we extend our algorithm so that it can be used to find a Nash equilibrium of a graphical game on an arbitrary tree.
Even when the tree has bounded degree, the running time can be exponential.
We show that this is inevitable by constructing a family of graphical games on bounded-degree trees for which best response policies of some of the vertices have exponential size, and any twopass algorithm (i.e., an algorithm that is similar in spirit to that of [8]) has to store almost all points of the best response policies.
In particular, we show the following.
THEOREM 3.
There is an infinite family ofgraphical games on bounded-degree trees with pathwidth 2 such that any two-pass algorithm for finding Nash equilibria on these trees requires exponential time and space.
It is interesting to note that the trees used in the proof of Theorem 3 have pathwidth 2, that is, they are very close to being paths.
It is an open question whether our algorithm runs in polynomial time for graphs of pathwidth 1.
This question can be viewed as a generalization of a very natural computational geometry problem--we describe it in more detail in Section 8.
In Section 7, we give a complexity-theoretic intractability result for the problem of finding a Nash equilibrium of a graphical game on a graph with small pathwidth.
We prove the following theorem.
THEOREM 4.
Consider the problem offinding a Nash equilibrium for a graphical game in which the underlying graph has maximum degree 3 and pathwidth k.
There is a constant k such that this problem is PPAD-complete.
Theorem 4 limits the extent to which we can exploit "path-like" properties of the underlying graph, in order to find Nash equilibria.
To prove Theorem 4, we use recent PPAD-completeness results for games, in particular the papers [7, 4] which show that the problem of finding Nash equilibria in graphical games of degree d (for d> 3) is computationally equivalent to the problem of solving r-player normal-form games (for r> 4), both of which are PPAD-complete.
2.
PRELIMINARIES AND NOTATION
We consider graphical games in which the underlying graph G is an n-vertex tree.
Each vertex has two actions, which are denoted by 0 and 1.
A mixed strategy is given by a single number x ∈ [0, 1], which denotes the probability that the player selects action 1.
Fur the purposes of the algorithm, the tree is rooted arbitrarily.
For convenience, we assume without loss of generality that the root has a single child, and that its payoff is independent of the action chosen by the child.
This can be achieved by first choosing an arbitrary root of the tree, and then adding a dummy "parent" of this root, giving the new parent a constant payoff function.
Given an edge (V, W) of the tree G, and a mixed strategy w for W, let G (V, W), W = w be the instance obtained from G by (1) deleting all nodes Z which are separated from V by W (i.e., all nodes Z such that the path from Z to V passes through W), and (2) restricting the instance so that W is required to play mixed strategy w. Definition 1.
Suppose that (V, W) is an edge of the tree, that v is a mixed strategy for V and that w is a mixed strategy for W.
We say that v is a potential best response to w (denoted by v ∈ pbrV (w)) if there is an equilibrium in the instance G (V, W), W = w in which V has mixed strategy v.
We define the best response policy for V, given W, as B (W, V) = {(w, v) | v ∈ pbrV (w), w ∈ [0, 1]}.
Typically, W is the parent of V, and this is just referred to as "the best response policy for V".
The expression B (W, V) | V = v is used to denote the set B (W, V) ∩ [0, 1] × {v}.
The upstream pass of the generic algorithm of [8] computes the best response policy for V for every node V other than the root.
With the above assumptions about the root, the downstream pass is straightforward: Let W denote the root and V denote its child.
The root selects any pair (w, v) from B (W, V).
It decides to play mixed strategy w and it instructs V to play mixed strategy v.
The remainder of the downward pass is recursive.
When a node V is instructed by its parent to adopt mixed strategy v, it does the following for each child U--It finds a pair (v, u) ∈ B (V, U) (with the same v value that it was given by its parent) and instructs U to play u.
3.
ALGORITHM OF LITTMAN ET AL. .
The algorithm of [9] is based on the following observation: to compute a single Nash equilibrium by a two-pass algorithm, it is not necessary to construct the entire best response policy for each vertex.
As long as, at each step of the downstream pass, the vertex under consideration can select a vector of strategies for all its children so that each child's strategy is a potential best response to the parent's strategy, the algorithm succeeds in producing a Nash equilibrium.
This can be achieved if, at the beginning of the downstream pass, we have a data structure in which each vertex V with parent W stores a set ˆB (W, V) ⊆ B (W, V) (called a breakpoint policy) which covers every possible w ∈ [0, 1].
We will show later that a sufficient condition for the construction of such a data structure is the invariant that, at every level of the upstream pass, ˆB (W, V) contains a continuous path from w = 0 to w = 1.
In [9], it is suggested that we can select the breakpoint policy in a particular way.
Namely, the paper uses the following definition: Definition 2.
(cf. [9]) A breakpoint policy for a node V with parent W consists of an ordered set of W-breakpoints w0 = 0 <w1 <w2 <· · · <wt − 1 <wt = 1 and an associated set of V - values v1,..., vt. .
The interpretation is that for any w ∈ [0, 1], if wi − 1 <w <wi for some index i and W plays w, then V shall play vi; and if w = wi for some index i, then V shall play any value between vi and vi +1.
We say such a breakpoint policy has t − 1 breakpoints.
The paper then claims that any vertex V can compute its breakpoint policy with respect to its parent W given the breakpoint policies of its children U1,..., Uk.
The proof proceeds by ordering the children's breakpoints (i.e., the respective values of v) from left to right (it can be assumed without loss of generality that all these breakpoints are distinct) and considering them in turn; each such point vl ∈ {v1,..., vL} corresponds to a fixed choice of strategies for k − 1 children and an interval of admissible strategies for one child.
Assume for convenience that this child is U1 and its interval of admissible strategies at vl is [a, b]; assume also that for Uj, j = 2,..., k, their respective breakpoint policies prescribe them to play uj in response to vl.
Let P i (u, w), i = 0, 1, be the expected payoff for V when V plays i, U1 plays u, each Uj, j = 2,..., k, plays uj, and W plays w, and consider the set Wl = {w ∈ [0, 1] | ∃ u ∈ [a, b] s.t. P 0 (u, w) = P1 (u, w)}; note that for any w ∈ Wl we have vl ∈ pbrV (w).
Figure 1: LKS: Trimming to find breakpoint policies.
The authors show that for any breakpoint vl, the set Wl is either empty, a single interval, or a union of two non-floating intervals (an interval is non-floating if one of its endpoints is 0 or 1); moreover, the union of all sets Wl, l = 1,..., L, covers the interval [0, 1].
It follows easily that one can cover [0, 1] with at most L + 2 intervals, each of which is a subset of some Wl.
The authors then claim that any such cover can be transformed into a breakpoint policy for V.
Namely, they say that for any two intervals Wl1 and Wl2 in the cover, "Any overlap between Wl1 and Wl2 can be arbitrarily assigned coverage by Wl1 and Wl2 "trimmed" accordingly (cf. [9], p. 5).
They illustrate their approach in a figure, which is reproduced as Figure 1 here.
In the figure, the dashed horizontal lines represent the breakpoints v1, v2,..., v7 and the solid intervals along these breakpoints are the sets W1, W2,..., W7.
The thick connected path is the corresponding breakpoint policy.
It is chosen as follows: begin on the left, and always "jump" to the interval allowing greatest progress to the right.
To see why this approach does not work in general, consider a path of length 4 consisting of an indifferent root R, its child W, W's child V, and V's child U. Suppose that U receives a payoff of 1 if it plays differently to V and 0 otherwise.
Thus, if v denotes the mixed strategy of V (i.e., V plays 1 with probability v), then the expected payoff that U derives from playing 0 is given by P0 (U) = v and the expected payoff that U derives from playing 1 is given by P1 (U) = 1 − v. Suppose that V derives no payoff from playing 1 (so P1 (V) = 0) and that its payoff matrix for playing 0 is 1 − 9
9 − 1, so if u denotes the mixed strategy of U and w denotes the mixed strategy of W, the expected payoff that V derives from playing 0 is given by P0 (V) = (1 − u) (1 − w) + (1 − u) w (− 9) + u (1 − w) 9 + uw (− 1).
Using the techniques of [8] (or, alternatively, those of Section 4), it is not hard to verify that the best response policies for U and V (as in Definition 1) are given by the graphs in Figure 2.
The best response policy for U is a breakpoint policy for U (as in Definition 2) with V - breakpoints v0 = 0, v1 = 1/2 and v2 = 1 with associated values u1 = 1 and u2 = 0.
The best response policy for V is not a breakpoint policy (because of how the curve from w = 0 to w = 1 "doubles back").
The LKS algorithm would "trim" to get a breakpoint policy such as the one in Figure 3.
Note that this breakpoint policy ˆB (W, V) is invalid in the sense that it does not satisfy ˆB (W, V) ⊆ B (W, V).
Figure 2: Best response policies for U and V.
Figure 3: A "trimmed" policy for V
The point is that the payoff matrix of W can now be chosen to prevent the LKS algorithm from finding a Nash equilibrium.
For example, suppose the payoffs are given so that P0 (W) = v and P1 (W) = (1 − v) 2.
The best response policy for W is a horizontal line at w = .1 (This is the value of w that allows v = 2/3--see Figure 2, which makes P0 (W) = P1 (W).)
In the downward pass, the chosen values are w = .1, then, from the trimming, v = 0 and u = 1, which is not a Nash equilibrium since W prefers action 1.
The failure of the algorithm is not caused by the fact that the trimming policy goes as far to the right as possible.
Any other "trimming" would be just as bad.
For example, suppose the breakpoint policy for V has v = 0 until some point w ∗ <.9 and then jumps to v = 1.
The algorithm is then defeated by the payoff matrix with P0 (W) = 2v and P1 (W) = (1 − v) in which the best response policy for W is a horizontal line at w = .9.
The algorithm then gives w = .9, v = 1, and u = 0, which is not a Nash equilibrium since W prefers action 0.
We conclude that the LKS algorithm does not always find a Nash equilibrium.
In Sections 4 and 6 we show how to modify the algorithm so that it always finds a Nash equilibrium.
For the modified algorithm, we have to extend the definition of "breakpoint policy" (see Definition 3) so that it includes breakpoint policies such as the best response policy for V in Figure 2.
Unfortunately, such a breakpoint policy may be exponential in size (see Figure 7) so the corrected algorithm does not run in polynomial time on all trees.
In the next section, we show that it runs in polynomial time on a path.
4.
FINDING EQUILIBRIA ON A PATH
In this section, we focus on the case when the underlying graph is a path, i.e., its vertex set is {V1,..., Vn}, and its edge set is {(Vj, Vj +1) | j = 1,..., n − 1}.
We show that in this case the best response policy for each vertex can be represented as a union of a polynomial number of rectangles, where a rectangle is defined by a pair of closed intervals (IV, IU) and consists of all points in IV × IU; it may be the case that one or both of the intervals IV and IU consists of a single point.
THEOREM 5.
For any j = 1,..., n, the set B (Vj, Vj − 1) can be represented as a disjoint union of at most (j + 4) 2 rectangles.
Moreover, given such representation of B (Vj, Vj − 1), one can compute a representation of B (Vj +1, Vj) in time O (j2).
PROOF.
For any set A ⊆ [0, 1] 2 that is represented as a union of a finite number of rectangles, we say that a point u ∈ [0, 1] on the U-axis is a U-event point of A if u = 0 or u = 1 or A contains a rectangle of the form IV × IU and u is an endpoint of IU; V - event points are defined similarly.
Observe that for any u ∈ [0, 1], the number of connected components of [0, 1] × {u} ∩ A is at most the number of V - event points of A.
We use induction on j to show that for each Vj the statement of the theorem holds and, additionally, each B (Vj, Vj − 1) has at most 2j + 4 event points.
To simplify the base case, we modify the graphical game by appending a dummy vertex V0 to the beginning of the path: the only neighbour of V0 is V1, the payoffs of V0 are always equal to 0, and the payoffs of all other vertices (including V1) are the same as in the original game.
For j = 0, we have B (V1, V0) = [0, 1] 2, so the statement of the theorem is trivially true.
Now, suppose that j> 0, set V = Vj and let U = Vj − 1 and W = Vj +1 be the vertices that precede and follow V, respectively.
The payoffs to V are described by a 2 × 2 × 2 matrix P: Pxyz is the payoff that V receives when U plays x, V plays y, and W plays z, where x, y, z ∈ {0, 1}.
Suppose that U plays 1 with probability u and W plays 1 with probability w.
Then V's expected payoff from playing 0 is P0 = (1 − u) (1 − w) P000 + (1 − u) wP001 + u (1 − w) P100 + uwP101, while its expected payoff from playing 1 is P1 = (1 − u) (1 − w) P010 + (1 − u) wP011 + u (1 − w) P110 + uwP111.
If P0> P1, V strictly prefers to play 0, if P0 <P1, V strictly prefers to play 1, and if P0 = P1, V is indifferent, i.e., can play any (mixed) strategy.
Since P0 and P1 are linear in w and u, there exist some constants A1, A0, B1, and B0 that depend on the matrix P, but not on u and w, such that
Depending on the values of A1, A0, B1, and B0, we subdivide the rest of the proof into the following cases.
• B1 = 0, B0 = 0.
In this case, P0> P1 if and only if A1u + A0 <0.
If also A1 = 0, A0 = 0, clearly, B (W, V) = [0, 1] 2, and the statement of the theorem is trivially true.
Otherwise, the vertex V is indifferent between 0 and 1 if and only if A1 = ~ 0 and u = − A0/A1.
Let V = {v | v ∈ (0, 1), − A0/A1 ∈ pbrU (v)}.
By the inductive hypothesis, V consists of at most 2 (j − 1) + 4 segments and isolated points.
For any v ∈ V, we have B (W, V) | V = v = [0, 1]: no matter what W plays, as long as U is playing − A0/A1, V is content to play v. On the other hand, for any v ∈ (0, 1) \ V we have B (W, V) | V = v = ∅: when V plays v, U can only respond with u = ~ − A0/A1, in which case V can benefit from switching to one of the pure strategies.
To complete the description of B (W, V), it remains to analyze the cases v = 0 and v = 1.
The vertex V prefers to play 0 if A1> 0 and u ≤ − A0/A1, or A1 <0 and u ≥ − A0/A1, or
A1 = 0 and A0 <0.
Assume for now that A1> 0; the other two cases can be treated similarly.
In this case 0 ∈ pbrV (w) for some w ∈ [0, 1] if and only if there exists a u ∈ pbrU (0) such that u ≤ − A0/A1: if no such u exists, whenever V plays 0 either U's response is not in pbrU (0) or V can improve its payoff by playing 1.
Therefore, either B (W, V) | V = 0 = [0, 1] or B (W, V) | V = 0 = ∅.
Similarly, B (W, V) | V = 1 is equal to either [0, 1] or ∅, depending on pbrU (1).
Therefore, the set B (W, V) consists of at most 2j + 4 ≤ (j + 4) 2 rectangles: B (W, V) ∩ [0, 1] × (0, 1) = [0, 1] × V contributes at most 2j + 2 rectangles, and each of the sets B (W, V) | V = 0 and B (W, V) | V = 1 contributes at most one rectangle.
Similarly, its total number of event points is at most 2j + 4: the only W-event points are 0 and 1, each V - event point of B (W, V) is a V-event point of B (V, U), and there are at most 2j + 2 of them.
• B1u + B0 ≡ ~ 0, A1 = αB1, A0 = αB0 for some α ∈ R.
In this case, V is indifferent between 0 and 1 if and only if w = α, or B1 = ~ 0 and u = − B0/B1 = − A0/A1.
Similarly to the previous case, we can show that B (W, V) ∩ [0, 1] × (0, 1) consists of the rectangle {α} × [0, 1] and at most 2j + 2 rectangles of the form [0, 1] × IV, where each IV corresponds to a connected component of B (V, U) | U = - B0 / B1.
Furthermore, V prefers to play 0 if B1u + B0> 0 and w ≥ α or B1u + B0 <0 and w ≤ α.
Therefore, if B1u * + B0> 0 for some u * ∈ pbrU (0), then B (W, V) | V = 0 contains [α, + ∞) ∩ [0, 1] and if B1u ** + B0 <0 for some u ** ∈ pbrU (0), then B (W, V) | V = 0 contains [− ∞, α] ∩ [0, 1]; if both u * and u ** exist, B (W, V) | V = 0 = [0, 1].
The set B (W, V) | V = 1 can be described in a similar manner.
By the inductive hypothesis, B (V, U) has at most 2j + 2 event points; as at least two of these are U-event points, it has at most 2j V - event points.
Since each V - event point of B (W, V) is a V event point of B (V, U) and B (W, V) has at most 3 W-event points (0, 1, and α), its total number of event points is at most 2j + 3 <2j + 4.
Also, similarly to the previous case it follows that B (W, V) consists of at most 2j + 4 <(j + 4) 2 rectangles.
• B1u + B0 ≡ ~ 0, α (B1u + B0) ≡ ~ A1u + A0.
In this case, one can define the indifference function f (·) as
into zero simultaneously.
Observe that whenever w = f (u) and u, w ∈ [0, 1], V is indifferent between playing 0 and 1.
For any A ⊆ [0, 1] 2, we define a function ˆfV by ˆfV (A) = {(f (u), v) | (v, u) ∈ A}; note that ˆfV maps subsets of [0, 1] 2 to subsets of R × [0, 1].
Sometimes we drop the subscript V when it is clear from the context.
PROOF.
Fix an arbitrary v ∈ (0, 1).
Suppose that U plays some u ∈ pbrU (v), w = f (u) satisfies w ∈ [0, 1], and W plays w.
There exists a vector of strategies v1,..., vj-1 = u, vj = v such that for each Vk, k <j, its strategy is a best response to its neighbours' strategies.
Since w = f (u), V is indifferent between playing 0 and 1; in particular, it can play v. Therefore, if we define vj +1 = w, the vector of strategies (v1,..., vj +1) will satisfy the conditions in the definition of potential best response, i.e., we have v ∈ pbrV (w).
Conversely, suppose v ∈ pbrV (w) for some w ∈ [0, 1], v = ~ 0, 1.
Then there exists a vector of strategies v1,..., vj-1, vj = v, vj +1 = w such that for each Vk, k ≤ j, its strategy is a best response to its neighbours' strategies.
As v = ~ 0, 1, V is, in fact, indifferent between playing 0 and 1, which is only possible if w = f (vj-1).
Choose u = vj-1; by construction, u ∈ pbrU (v).
Lemma 1 describes the situations when V is indifferent between playing 0 and playing 1.
However, to fully characterize B (W, V), we also need to know when V prefers a pure strategy.
LEMMA 2.
For any w ∈ [0, 1], we have (w, 0) ∈ ˆf (0) if and only if 0 ∈ pbrV (w) and (w, 1) ∈ fˆ (1) if and only if 1 ∈ pbrV (w).
PROOF.
Consider an arbitrary u0 ∈ pbrU (0).
If B (u0)> 0, for u = u0 the inequality P0 ≥ P1 is equivalent to w ≥ f (u0).
Therefore, when U plays u0 and W plays w, w ≥ f (u0), V prefers to play 0; as u0 ∈ pbrU (u), it follows that 0 ∈ pbrV (w).
The argument for the case B (u0) <0 is similar.
Conversely, if 0 ∈ pbrV (w) for some w ∈ [0, 1], there exists a vector (v1,..., vj-1, vj = 0, vj +1 = w) such that for each Vk, k ≤ j, Vk plays vk, and this strategy is a best response to the strategies of Vk's neighbours.
Note that for any such vector we have vj-1 ∈ pbrU (0).
By way of contradiction, assume (w, 0) ∈ ~ UuEpbrU (0) Ru.
Then it must be the case that for any u0 ∈ pbrU (0) either f (u0) <w and Ru0 = (− ∞, f (u0)] × {0} or f (u0)> w and Ru0 = [f (u0), + ∞) × {0}.
In both cases, when V plays 0, U plays u0, and V plays w, the inequality between f (u0) and w is equivalent to P0 <P1, i.e., V would benefit from switching to 1.
The argument for ˆf (1) is similar.
Together, Lemma 1 and Lemma 2 completely describe the set B (W, V): we have
It remains to show that B (W, V) can be represented as a union of at most (j + 4) 2 rectangles, has at most 2j + 4 event points, and can be computed in O (j2) time.
Set u * = − B0/B1.
2 Consider an arbitrary rectangle R = [v1, v2] × [u1, u2] ⊆ B (V, U).
If u * ∈ ~ [u1, u2], the function f (·) is continuous on [u1, u2] and hence fˆ (R) = [fmin, fmax] × [v1, v2], where fmin = min {f (u1), f (u2)}, fmax = max {f (u1), f (u2)}, i.e., in this case fˆ (R) ∩ [0, 1] 2 consists of a single rectangle.
Now, suppose that R is intersected by the line [0, 1] × {u *}; as was noted earlier, there are at most 2j + 2 such rectangles.
Suppose that limu_u ∗ − f (u) = + ∞; as f (·) is a fractional linear function, this implies that limu_u ∗ + f (u) = − ∞ and also f (u1)> f (u2).
Since f (·) is continuous on [u1, u *) and (u *, u2], it is easy to see that
2The case B1 = 0 causes no special problems.
For completeness, set u * to be any value outside of [0, 1] in this case.
Figure 4: f is increasing on (− ∞, u *) and (u *, + ∞).
and ˆf ([v1, v2] × (u *, u2]) = (− ∞, f (u2)] × [v1, v2], i.e., in this case ˆf (R) ∩ [0, 1] 2 consists of at most two rectangles.
The case limu -.
u ∗ − f (u) = − ∞ is similar.
As fˆ (B (V, U)) = URCS (V, U) ˆf (R), it follows that ˆf (B (V, U)) consists of at most (j + 3) 2 + 2j + 2 rectangles.
Also, it is easy to see that both fˆ (0) and fˆ (1) consist of at most 2 line segments each.
We conclude that B (W, V) can be represented as a union of at most (j + 3) 2 + 2j + 6 <(j + 4) 2 rectangles.
Moreover, if v is a V - event point of B (W, V), then v is a V event point of B (V, U) (this includes the cases v = 0 and v = 1, as 0 and 1 are V - event points of B (V, U)) and if w is a W-event point of B (W, V), then either w = 0 or w = 1 or there exists some u ∈ [0, 1] such that w = f (u) and u is a U-event point of B (V, U).
Hence, B (W, V) has at most 2j + 4 event points.
The O (j2) bound on the running time in Theorem 5 follows from our description of the algorithm.
The O (n3) bound on the overall running time for finding a Nash equilibrium (and a representation of all Nash equilibria) follows.
4.1 Finding a Single Nash Equilibrium in O (n2) Time
The upper bound on the running time of our algorithm is tight, at least assuming the straightforward implementation, in which each B (Vj +1, Vj) is stored as a union of rectangles: it is not hard to construct an example in which the size of B (Vj +1, Vj) is Ω (j2).
However, in some cases it is not necessary to represent all Nash equilibria; rather, the goal is to find an arbitrary equilibrium of the game.
In this section, we show that this problem can be solved in quadratic time, thus obtaining a proof of Theorem 1.
Our solution is based on the idea of [9], i.e., working with subsets of the best response policies rather than the best response policies themselves; following [9], we will refer to such subsets as breakpoint policies.
While it is not always possible to construct a breakpoint policy as defined in [9], we show how to modify this definition so as to ensure that a breakpoint policy always exists; moreover, we prove that for a path graph, the breakpoint policy of any vertex can be stored in a data structure whose size is linear in the number of descendants this vertex has.
Definition 3.
A breakpointpolicy ˆB (V, U) for a vertex U whose parent is V is a non-self-intersecting curve of the form
where Xi = [vi_1, vi] × {ui}, Yi = {vi} × [ui, ui +1] and ui, vi ∈ [0, 1] for i = 0,..., m.
We say that a breakpoint policy is valid if v0 = 0, vm = 1, and ˆB (V, U) ⊆ B (V, U).
We will sometimes abuse notation by referring to ˆB (V, U) as a collection of segments Xi, Yi rather than their union.
Note that we do not require that vi ≤ vi +1 or ui ≤ ui +1; consequently, in any argument involving breakpoint policies, all segments are to be treated as directed segments.
Observe that any valid breakpoint policy ˆB (V, U) can be viewed as a continuous 1--1 mapping γ (t) =
As explained in Section 3, we can use a valid breakpoint policy instead of the best response policy during the downstream pass, and still guarantee that in the end, we will output a Nash equilibrium.
Theorem 6 shows that one can inductively compute valid breakpoint policies for all vertices on the path; the proof of this theorem can be found in the full version of this paper [6].
THEOREM 6.
For any V = Vj, one canfind in polynomial time a valid breakpoint policy ˆB (W, V) that consists of at most 2j + 1 segments.
5.
NASH EQUILIBRIA ON GRAPHS WITH MAXIMUM DEGREE 2
In this section we show how the algorithm for paths can be applied to solve a game on any graph whose vertices have degree at most 2.
A graph having maximum degree 2 is, of course, a union of paths and cycles.
Since each connected component can be handled independently, to obtain a proof of Theorem 2, we only need to show how to deal with cycles.
Given a cycle with vertices V1,..., Vk (in cyclic order), we make two separate searches for a Nash equilibrium: first we search for a Nash equilibrium where some vertex plays a pure strategy, then we search for a fully mixed Nash equilibrium, where all vertices play mixed strategies.
For i ≤ k let vi denote the probability that Vi plays 1.
The first search can be done as follows.
For each i ∈ {1,..., k} and each b ∈ {0, 1}, do the following.
1.
Let P be the path (Vi +1, Vi +2..., Vk, V1,..., Vi_1, Vi) 2.
Let payoff to Vi +1 be based on putting vi = b (so it depends only on vi +1 and vi +2.)
3.
Apply the upstream pass to P 4.
Put vi = b; apply the downstream pass For each vertex, Vj, keep track of all possible mixed strategies vj 5.
Check whether Vi +1 has any responses that are consistent with vi = b; if so we have a Nash equilibrium.
(Otherwise, there is no Nash equilibrium of the desired form.)
For the second search, note that if Vi plays a mixed strategy, then vi +1 and vi_1 satisfy an equation of the form vi +1 = (A0 + A1vi_1) / (B0 + B1vi_1).
Since all vertices in the cycle play mixed strategies, we have vi +3 = (A' 0 + A' 1vi +1) / (B' 0 + B' 1vi +1).
Composing the two linear fractional transforms, we obtain vi +3 = (A' ' 0 + A' ' 1 vi_1) / (B' ' 0 + B' ' 1 vi_1).
for some new constants A' ' 0, A' ' 1, B' ' 0, B' ' 1.
Choose any vertex Vi.
We can express vi in terms of vi +2, then vi +4, vi +6 etc. and ultimately vi itself to obtain a quadratic equation (for vi) that is simple to derive from the payoffs in the game.
If the equation is non-trivial it has at most 2 solutions in (0, 1).
For an odd-length cycle all other vj's are derivable from those solutions, and if a fully mixed Nash equilibrium exists, all the vj should turn out to be real numbers in the range (0, 1).
For an even-length cycle, we obtain two quadratic equations, one for vi and another for
vi +1, and we can in the same way test whether any solutions to these yield values for the other vj, all of which lie in (0, 1).
If the quadratic equation is trivial, there is potentially a continuum of fully-mixed equilibria.
The values for vi that may occur in a Nash equilibrium are those for which all dependent vj values lie in (0, 1); the latter condition is easy to check by computing the image of the interval (0, 1) under respective fractional linear transforms.
6.
FINDING EQUILIBRIA ON AN (ARBITRARY) TREE
For arbitrary trees, the general structure of the algorithm remains the same, i.e., one can construct a best response policy (or, alternatively, a breakpoint policy) for any vertex based on the best response policies of its children.
We assume that the degree of each vertex is bounded by a constant K, i.e., the payoff matrix for each vertex is of size O (2K).
Consider a vertex V whose children are U1,..., Uk and whose parent is W; the best response policy of each Uj is B (V, Uj).
Similarly to the previous section, we can compute V's expected payoffs P0 and P1 from playing 0 or 1, respectively.
Namely, when each of the Uj plays uj and W plays w, we have P0 = L0 (u1,..., uk, w), P1 = L1 (u1,..., uk, w), where the functions L0 (·,..., ·), L1 (·,..., ·) are linear in all of their arguments.
Hence, the inequality P0> P1 can be rewritten as
where both A (·,..., ·) and B (·,..., ·) are linear in all of their arguments.
Set ut = (u1,..., uk) and define the indifference function f: [0, 1] k ~ → [0, 1] as f (u) = A (i) / B (i); clearly, if each Uj plays uj, W plays w and w = f (u), V is indifferent between playing 0 and 1.
For any X = X1 × · · · × Xk, where Xi ⊆ [0, 1] 2 define
and ˆf (1) = {(w, 1) | ∃ u s.t. ui ∈ pbrUi (1) and wB (u) ≤ A (u)}.
As in previous section, we can show that B (W, V) is equal to
also, any path from w = 0 to w = 1 that is a subset of B (W, V) constitutes a valid breakpoint policy.
6.1 Exponential Size Breakpoint Policy
While the algorithm of Section 4 can be generalized for boundeddegree trees, its running time is no longer polynomial.
In fact, the converse is true: we can construct a family of trees and payoff matrices for all players so that the best response policies for some of the players consist of an exponential number of segments.
Moreover, in our example the breakpoint policies coincide with the best response policies, which means that even finding a single Nash equilibrium using the approach of [8, 9] is going to take exponentially long time.
In fact, a stronger statement is true: for any polynomial-time two-pass algorithm (defined later) that works with subsets of best response policies for this graph, we can choose the payoffs of the vertices so that the downstream pass of this algorithm will fail.
Figure 5: The tree T. that corresponds to exponential-size breakpoint policy.
In the rest of this subsection, we describe this construction.
Consider the tree T. given by Figure 5; let V. be the root of this tree.
For every k = 1,..., n, let the payoffs of Sk and Tk be the same as those for the U and V described in Section 3; recall that the breakpoint policies for U and V are shown in Figure 2.
It is not hard to see that the indifference function for Tk is given by f (s) = .8 s +.1.
The payoff of V0 is 1 if V1 selects the same action as V0 and 0 otherwise; V0's best response policy is given by Figure 6.
LEMMA 3.
Fix k <n, and let u, t, v, and w denote the strategies of Vk-1, Tk, Vk, and Vk +1, respectively.
Suppose that Vk prefers playing 0 to playing 1 if and only if .5 t + .1 u + .2> w.
Then B (Vk +1, Vk) consists of at least 3k segments.
Moreover, {(v, w) | (v, w) ∈ B (Vk +1, Vk), 0 ≤ w ≤ .2} = [0, .2] × {0} and {(v, w) | (v, w) ∈ B (Vk +1, Vk), .8 ≤ w ≤ 1} = [.8, 1] × {1}.
these sets correspond to horizontal segments of B (Vk, Tk).
It is easy to see that S1, S2, S3 ⊂ B (Vk +1, Vk).
Since f is a continuous function, the number of segments in each Si is at least the number of segments in B (Vk, Vk-1) ∩ [.1, .9] × [0, 1], which is at least 3k-1 by induction hypothesis.
Moreover, as f is monotone in u and f (1, 0) <f (0, .5) <f (1, .5) <f (0, 1), all Si, i = 1, 2, 3, are disjoint.
Finally, the set B (Vk +1, Vk) contains two segments that correspond to the vertical segments of B (Vk, Tk), i.e.,
Clearly, S4 connects S2 and S3, S5 connects S1 and S2, and S4 and S5 do not intersect each other.
We conclude that B (Vk +1, Vk)
Figure 6: Breakpoint policies for V0 and V1.
is a continuous line that consist of at least 3k segments and satisfies the condition of the lemma.
To complete the construction, we need to show that we can design the payoff matrix for Vk so that it prefers playing 0 to playing 1 if and only if .5 t + .1 u + .2> w. To this end, we prove a more general statement, namely, that the indifference function of a vertex can be an arbitrary fractional multilinear function of its descendants' strategies.
We say that a function of k variables is multilinear if it can be represented as a sum of monomials and each of these monomials is linear in all of its variables.
Note that this definition is different from a more standard one in that we do not require that all of the monomials have the same degree.
Recall that the payoffs of a vertex with k + 1 neighbours are described by matrices P0 and P1, where Pj i0i1...ik is the payoff that V gets when it plays j, and its neighbours play i0,..., ik, and j, i0,..., ik ∈ {0, 1}.
Let P [j] = P [j] (w, u1,..., uk) be the expected payoff obtained by this vertex when it plays j and the (mixed) strategies of its neighbours are given by a vector (w, u1,..., uk), i.e., P [j] = E [Pji0i1...ik] where i0,..., ik are independent Bernoulli random variables, each of which is 1 with the respective probabilities w, u1,..., uk.
LEMMA 4.
Given a tree vertex V whose parent is Wand whose children are U1,..., Uk, for any function f = f (u1,..., uk) that can be represented as a ratio of two multilinear functions f1, f2,
The proof of this lemma is based on the fact that every monomial of the form as (u0) s0...(uk) sk, s1,..., sk ∈ {0, 1}, can be represented as
for some Ct, t ∈ {0, 1} k +1.
The details can be found in the full version of this paper [6].
6.2 Irreducibility of the Best Response Policy for Tn
While the best response policy constructed in the previous subsection has exponential size, it is not clear a ` priori that it is necessary to keep track of all of its line segments rather than to focus on a small subset of these segments.
However, it turns out that for two-pass algorithms such as the algorithm of [8], the best response policy cannot be simplified.
More precisely, we say that an algorithm A is a two-pass algorithm if
Figure 7: Breakpoint policy for V2.
• A consists of an upstream pass and a downstream pass.
• During the upstream pass, for each vertex V with parent W,
A constructs a set BB (W, V) ⊆ B (W, V).
This set is produced from the sets {BB (V, U) | U is a child of V} by applying the procedure from the beginning of Section 6 (substituting BB (V, Uj) for B (V, Uj) for all children Uj of V), and then possibly omitting some of the points of the resulting set (which is then stored explicitly).
• The downstream pass is identical to the downstream pass of [8] as described in Section 2 except that it operates on the sets BB (W, V) rather than on the sets B (W, V).
Theorem 7 demonstrates that any two-pass algorithm will fail during the downstream pass on Tn if there is an index j such that the set BB (Vj +1, Vj) omits any interior point of any of the (at least 3j) segments of B (Vj +1, Vj).
This implies Theorem 3.
THEOREM 7.
For any two-pass algorithm A for which there exists an index j, j ∈ [1, n/4], a segment S of B (Vj, Vj_1), and an interior point (x, y) of S such that BB (Vj, Vj_1) does not contain (x, y), we can choose payoff matrices of the vertices Vj,..., Vn so that the downstream pass of A will fail, and, additionally, payoffs to V4j,..., Vn are identically 0.
We sketch the proof of Theorem 7; the details can be found in the full version of this paper [6].
We proceed by induction.
For j = 1, the argument is similar to that in Section 3.
For the inductive step, the main idea is that we can "zoom in" on any part of a best response policy (including the part that was omitted!)
by using an appropriate indifference function; this allows us to reduce the case j = j0 to j = j0 − 1.
7.
PPAD-COMPLETENESS OF BOUNDED PATHWIDTH GRAPHICAL GAMES
In the previous section, we showed that for graphical games on trees that are almost but not quite paths, two-pass algorithms fail to find Nash equilibria in polynomial time.
We next show that a milder path-like graph property allows us to construct graphical games for which it is unlikely that any polynomial-time algorithm will find Nash equilibria.
7.1 Pathwidth
A path decomposition of a graph G = (V, E) is a sequence of subset Si (V) ⊆ V such that for each edge (v, v') ∈ E, v, v' ∈ Si (V) for some i, and furthermore, for each v ∈ V, if v ∈ Si (V) and v ∈ Sj (V) for j> i, then v ∈ Sk (V) for all i ≤ k ≤ j.
The path decomposition has width k if all sets Si (V) have cardinality at most k + 1.
The pathwidth of G is the minimum width of any path decomposition of G.
Pathwidth is a restriction of treewidth (in which one would seek a tree whose vertices were the sets Si (V), and the sets containing some vertex would have to form a subtree).
For any constant k it can be decided in polynomial time whether a graph has pathwidth (or treewidth) k. Furthermore many graph-theoretic problems seem easier to solve in polynomial time, when restricted to fixed treewidth, or pathwidth, graphs, see [1] for an overview.
Note that a path has pathwidth 1 and a cycle has pathwidth 2.
7.2 PPAD-completeness
We review some basic definitions from the computational complexity theory of search problems.
A search problem associates any input (here, a graphical game) with a set of solutions (here, the Nash equilibria of the input game), where the description length of any solution should be polynomially bounded as a function of the description length of its input.
In a total search problem, there is a guarantee that at least one solution exists for any input.
Nash's theorem assures us that the problem of finding Nash equilibria is total.
A reduction from search problem S to problem S' is a mechanism that shows that any polynomial-time algorithm for S' implies a polynomial-time algorithm for S.
It consists of functions f and g, computable in polynomial time, where f maps inputs of S to inputs of S', and g maps solutions of S' to solutions of S, in such a way that if Is is an input to S, and Ss is a solution to f (Is), then g (Ss) is a solution to Is.
Observe that total search problems do not allow the above reductions from problems such as CIRCUIT SAT (where the input is a boolean circuit, and solutions are input vectors that make the output true) due to the fact that CIRCUIT SAT and other NP-complete problems have inputs with empty solution sets.
Instead, recent work on the computational complexity of finding a Nash equilibrium [7, 4, 5, 2, 3] has related it to the following problem.
Definition 4.
END OF THE LINE.
Input: boolean circuits S and P, each having n input and n output bits, where P (0n) = 0n and S (0n) = ~ 0n.
Solution: x ∈ {0, 1} n such that S (x) = x, or alternatively x ∈ {0, 1} n such that P (S (x)) = ~ x. S and P can be thought of as standing for "successor" and "predecessor".
Observe that by computing Si (0n) (for i = 0, 1, 2, ...) and comparing with P (Si +1 (0n)), we must eventually find a solution to END OF THE LINE.
END OF THE LINE characterizes the complexity class PPAD (standing for parity argument on a graph, directed version), introduced in Papadimitriou [11], and any search problem S is PPAD-complete if END OF THE LINE reduces to S.
Other PPAD-complete problems include the search for a ham sandwich hyperplane, and finding market equilibria in an exchange economy (see [11] for more detailed descriptions of these problems).
3-GRAPHICAL NASH is the problem of finding a Nash equilibrium for a graphical game whose graph has degree 3.
Daskalakis et al. [4] show PPAD-completeness of 3-GRAPHICAL NASH by a reduction from 3-DIMENSIONAL BROUWER, introduced in [4] and defined as follows.
Definition 5.
3-DIMENSIONAL BROUWER.
Input: a circuit C having 3n input bits and 2 output bits.
The input bits define a "cubelet" of the unit cube, consisting of the 3 coordinates of its points, given to n bits of precision.
The output represents one of four colours assigned by C to a cubelet.
C is restricted so as to assign colour 1 to cubelets adjacent to the (y, z) - plane, colour 2 to remaining cubelets adjacent to the (x, z) - plane, colour 3 to remaining cubelets on the (x, y) - plane, and colour 0 to all other cubelets on the surface of the unit cube.
A solution is a panchromatic vertex, a vertex adjacent to cubelets that have 4 distinct colours.
The reason why a solution is guaranteed to exist, is that an associated Brouwer function 0 can be constructed, i.e. a continuous function from the unit cube to itself, such that panchromatic vertices correspond to fixpoints of 0.
Brouwer's Fixpoint Theorem promises the existence of a fixpoint.
The proof of Theorem 4 uses a modification of the reduction of [4] from 3-DIMENSIONAL BROUWER to 3-GRAPHICAL NASH.
To prove the theorem, we begin with some preliminary results as follows.
Each player has 2 actions, denoted 0 and 1.
For a player at vertex V let P [V] denote the probability that the player plays 1.
LEMMA 5.
[7] There exists a graphical game 9shif t offixed size having vertices V, V' where P [V'] is the fractional part of 2P [V].
COROLLARY 1.
There exists a graphical game 9n-shift ofsize Θ (n) of constant pathwidth, having vertices V, Vn where P [Vn] is the fractional part of 2n.
P [V].
PROOF.
Make a chain of n copies of 9shift in Lemma 5.
Each subset of vertices in the path decomposition is the vertices in a copy of 9shif t. Let In (x) denote the n-th bit of the binary expansion of x, where we interpret 1 as true and 0 as false.
The following uses gadgets from [7, 4].
COROLLARY 2.
There exists k such that for all n, and for all n1, n2, n3 <n, there exists a graphical game of size O (n) with pathwidth k, having vertices V1, V2, V3 where P [V3] = P [V1] + 2-n3 (In1 P [V1] ∧ In2 P [V2]).
PROOF OF THEOREM 4.
Let C be the boolean circuit describing an instance of 3-DIMENSIONAL BROUWER.
Let g1,..., gp (n) be the gates of C indexed in such a way that the input (s) to any gate are the output (s) of lower-indexed gates.
g1,..., g3n will be the 3n inputs to C. All players in the graphical game 9 constructed in [4] have 2 actions denoted 0 and 1.
The probability that V plays 1 is denoted P [V].
9 has 3 players Vx, Vy and Vz for which P [Vx], P [Vy] and P [Vz] represent the coordinates of a point in the unit cube.
9 is designed to incentivize Vx, Vy and Vz to adjust their probabilities in directions given by a Brouwer function which is itself specified by the circuit C.
In a Nash equilibrium, P [Vx], P [Vy] and P [Vz] represent coordinates of a fixpoint of a function that belongs to the class of functions represented by 3-DIMENSIONAL BROUWER.
C using Corollary 2.
Let 9 (i) be the graphical game that does this.
Let S1 (9 (i)),..., Sn (9 (i)) be a length n path decomposition of 9 (i), where VC (i-1) ∈ S1 (9 (i)) and V C (i) ∈ Sn (9 (i)).
Then, a path decomposition of ∪ 1 <i <p (n) 9 (i) is obtained by taking the union of the separate path decompositions, together with Sn (9 (i-1)) ∪ S1 (9 (i)) for 2 <i <p (n).
Let 9C be the above graphical game that simulates C. 9C has 3n inputs, consisting of the first n bits of the binary expansions of P [Vx], P [Vy] and P [Vz].
Similarly to [4], the output of 9C affects Vx, Vy and Vz as follows.
Colour 0 incentivizes Vx, Vy and Vz
to adjust their probabilities p [Vx], p [Vy] and p [Vz] in the direction (− 1, − 1, − 1); colour 2 incentivizes them to move in direction (1, 0, 0); colour 2, direction (0, 1, 0); colour 3, direction (0, 0, 1).
We need to ensure that at points at the boundaries of adjacent cubelets, the change of direction will be approximately the average of directions of surrounding points.
That way, all four colors/directions must be nearby so that they can cancel each other out (and we are at a panchromatic vertex).
This is achieved using the same trick as [4], in which we make a constant number M of copies of GC, which differ in that each copy adds a tiny displacement vector to its copies of p [Vx], p [Vy] and p [Vz] (which are derived from the original using the addition gadget of [7]).
Using the addition and multiplication gadgets of [7] we average the directions and add a small multiple of this average to (p [Vx], p [Vy], p [Vz]).
At a Nash equilibrium the outputs of each copy will cancel each other out.
The pathwidth of the whole game is at most M times the pathwidth GC.
8.
OPEN PROBLEMS
The most important problem left open by this paper is whether it is possible to find a Nash equilibrium of a graphical game on a bounded-degree tree in polynomial time.
Our construction shows that any two-pass algorithm that explicitly stores breakpoint policies needs exponential time and space.
However, it does not preclude the existence of an algorithm that is based on a similar idea, but, instead of computing the entire breakpoint policy for each vertex, uses a small number of additional passes through the graph to decide which (polynomial-sized) parts of each breakpoint policy should be computed.
In particular, such an algorithm may be based on the approximation algorithm of [8], where the value of e is chosen adaptively.
Another intriguing question is related to the fact that the graph for which we constructed an exponential-sized breakpoint policy has pathwidth 2, while our positive results are for a path, i.e., a graph of pathwidth 1.
It is not clear if for any bounded-degree graph of pathwidth 1 the running time of (the breakpoint policybased version of) our algorithm will be polynomial.
In particular, it is instructive to consider a "caterpillar" graph, i.e., the graph that can be obtained from Tn by deleting the vertices S1,..., Sn.
For this graph, the best response policy of a vertex Vk in the "spine" of the caterpillar is obtained by combining the best response policy of its predecessor on the spine Vk − 1 and its other child Tk; since the latter is a leaf, its best response policy is either trivial (i.e., [0, 1] 2, [0, 1] × {0}, or [0, 1] × {1}) or consists of two horizontal segments and one vertical segment of the form {α} × [0, 1] that connects them.
Assuming for convenience that B (Vk, Tk) = [0, α] × {0} ∪ {α} × [0, 1] ∪ [α, 1] × {1}, and f is the indifference function for Vk, we observe that the best response policy for Vk consists of 5 components: ˆf (0), fˆ (1), and three components that correspond to [0, α] × {0}, {α} × [0, 1], and [α, 1] × {1}.
Hence, one can think of constructing B (Vk +1, Vk) as the following process: turn B (Vk, Vk − 1) by π / 2, cut it along the (now horizontal) line vk = α, apply a fractional linear transform to the horizontal coordinate of both parts, and reconnect them using the image of the segment {α} × [0, 1] under f.
This implies that the problem of bounding the size of the best response policy (or, alternatively, the breakpoint policy), can be viewed as a generalization of the following computational geometry problem, which we believe may be of independent interest: PROBLEM 1.
Given a collection of axis-parallel segments in R2, consider the following operation: pick an axis-parallel line li (either vertical or horizontal), cut the plane along this line, and shift one of the resulting two parts by an arbitrary amount δi; as a result, some segments will be split into two parts.
Reconnect these parts, i.e., for each segment of the form [a, b] × {c} that was transformed into [a, t] × {c + δi} and [t, b] × {c}, introduce a segment {t} × [c, c + δi].
Is itpossible to start with the segment [0, 1] and after n operations obtain a set that cannot be represented as a union of poly (n) line segments?
Ifyes, can it be the case that in this set, there is no path with a polynomial number of turns that connects the endpoints of the original segment?
It turns out that in general, the answer to the first question is positive, i.e., after n steps, it is possible to obtain a set that consists of Θ (cn) segments for some c> 0.
This implies that even for a caterpillar, the best response policy can be exponentially large.
However, in our example (which is omitted from this version of the paper due to space constraints), there exists a polynomial-size path through the best response policy, i.e., it does not prove that the breakpoint policy is necessarily exponential in size.
If one can prove that this is always the case, it may be possible to adapt this proof to show that there can be an exponential gap between the sizes of best response policies and breakpoint policies.

graphic game
degre
nash equilibrium
ppad-complet
larg-scale distribut network
dynam program-base algorithm
bound-degre tree
gener algorithm
respons polici
downstream pass
breakpoint polici