interest outcom featur
suppli-chain trade
empir mechan
paramet set
observ behavior
two-stage game
player
particip
gametheoret model
analysi
nash equilibrium
empir mechan design
game theori

Empirical Mechanism Design: Methods, with Application to a Supply-Chain Scenario Yevgeniy Vorobeychik, Christopher Kiekintveld, and Michael P. Wellman University of Michigan Computer Science & Engineering Ann Arbor, MI 48109-2121 USA { yvorobey, ckiekint, wellman }@umich.
edu ABSTRACT Our proposed methods employ learning and search techniques to estimate outcome features of interest as a function of mechanism parameter settings.
We illustrate our approach with a design task from a supply-chain trading competition.
Designers adopted several rule changes in order to deter particular procurement behavior, but the measures proved insufficient.
Our empirical mechanism analysis models the relation between a key design parameter and outcomes, confirming the observed behavior and indicating that no reasonable parameter settings would have been likely to achieve the desired effect.
More generally, we show that under certain conditions, the estimator of optimal mechanism parameter setting based on empirical data is consistent.
Categories and Subject Descriptors I.6 [Computing Methodologies]: Simulation and Modeling; J.4 [Computer Applications]: Social and Behavioral Sciences-Economics General Terms Algorithms, Economics, Design 1.
MOTIVATION We illustrate our problem with an anecdote from a supply chain research exercise: the 2003 and 2004 Trading Agent Competition (TAC) Supply Chain Management (SCM) game.
TAC/SCM [1] defines a scenario where agents compete to maximize their profits as manufacturers in a supply chain.
The agents procure components from the various suppliers and assemble finished goods for sale to customers, repeatedly over a simulated year.
As it happened, the specified negotiation behavior of suppliers provided a great incentive for agents to procure large quantities of components on day 0: the very beginning of the simulation.
During the early rounds of the 2003 SCM competition, several agent developers discovered this, and the apparent success led to most agents performing the majority of their purchasing on day 0.
Although jockeying for day-0 procurement turned out to be an interesting strategic issue in itself [19], the phenomenon detracted from other interesting problems, such as adapting production levels to varying demand (since component costs were already sunk), and dynamic management of production, sales, and inventory.
Several participants noted that the predominance of day-0 procurement overshadowed other key research issues, such as factory scheduling [2] and optimizing bids for customer orders [13].
After the 2003 tournament, there was a general consensus in the TAC community that the rules should be changed to deter large day-0 procurement.
The task facing game organizers can be viewed as a problem in mechanism design.
The designers have certain game features under their control, and a set of objectives regarding game outcomes.
Unlike most academic treatments of mechanism design, the objective is a behavioral feature (moderate day-0 procurement) rather than an allocation feature like economic efficiency, and the allowed mechanisms are restricted to those judged to require only an incremental modification of the current game.
Replacing the supplychain negotiation procedures with a one-shot direct mechanism, for example, was not an option.
We believe that such operational restrictions and idiosyncratic objectives are actually quite typical of practical mechanism design settings, where they are perhaps more commonly characterized as incentive engineering problems.
In response to the problem, the TAC/SCM designers adopted several rule changes intended to penalize large day-0 orders.
These included modifications to supplier pricing policies and introduction of storage costs assessed on inventories of components and finished goods.
Despite the changes, day-0 procurement was very high in the early rounds of the 2004 competition.
In a drastic measure, the GameMaster imposed a fivefold increase of storage costs midway through the tournament.
Even this did not stem the tide, and day-0 procurement in the final rounds actually increased (by some measures) from 2003 [9].
The apparent difficulty in identifying rule modifications that effect moderation in day-0 procurement is quite striking.
Although the designs were widely discussed, predictions for the effects of various proposals were supported primarily by intuitive arguments or at best by back-of-the-envelope calculations.
Much of the difficulty, of course, is anticipating the agents'' (and their developers'') responses without essentially running a gaming exercise for this purpose.
The episode caused us to consider whether new ap306 proaches or tools could enable more systematic analysis of design options.
Standard game-theoretic and mechanism design methods are clearly relevant, although the lack of an analytic description of the game seems to be an impediment.
Under the assumption that the simulator itself is the only reliable source of outcome computation, we refer to our task as empirical mechanism design.
In the sequel, we develop some general methods for empirical mechanism design and apply them to the TAC/SCM redesign problem.
Our analysis focuses on the setting of storage costs (taking other game modifications as fixed), since this is the most direct deterrent to early procurement adopted.
Our results confirm the basic intuition that incentives for day-0 purchasing decrease as storage costs rise.
We also confirm that the high day-0 procurement observed in the 2004 tournament is a rational response to the setting of storage costs used.
Finally, we conclude from our data that it is very unlikely that any reasonable setting of storage costs would result in acceptable levels of day-0 procurement, so a different design approach would have been required to eliminate this problem.
Overall, we contribute a formal framework and a set of methods for tackling indirect mechanism design problems in settings where only a black-box description of players'' utilities is available.
Our methods incorporate estimation of sets of Nash equilibria and sample Nash equilibria, used in conjuction to support general claims about the structure of the mechanism designer``s utility, as well as a restricted probabilistic analysis to assess the likelihood of conclusions.
We believe that most realistic problems are too complex to be amenable to exact analysis.
Consequently, we advocate the approach of gathering evidence to provide indirect support of specific hypotheses.
2.
PRELIMINARIES A normal form game2 is denoted by [I, {Ri}, {ui(r)}], where I refers to the set of players and m = |I| is the number of players.
Ri is the set of strategies available to player i ∈ I, with R = R1 ×...×Rm representing the set of joint strategies of all players.
We designate the set of pure strategies available to player i by Ai, and denote the joint set of pure strategies of all players by A = A1 ×...×Am.
It is often convenient to refer to a strategy of player i separately from that of the remaining players.
To accommodate this, we use a−i to denote the joint strategy of all players other than player i. Let Si be the set of all probability distributions (mixtures) over Ai and, similarly, S be the set of all distributions over A.
An s ∈ S is called a mixed strategy profile.
When the game is finite (i.e., A and I are both finite), the probability that a ∈ A is played under s is written s(a) = s(ai, a−i).
When the distribution s is not correlated, we can simply say si(ai) when referring to the probability player i plays ai under s. Next, we define the payoff (utility) function of each player i by ui : A1 ×· · ·×Am → R, where ui(ai, a−i) indicates the payoff to player i to playing pure strategy ai when the remaining players play a−i.
We can extend this definition to mixed strategies by assuming that ui are von Neumann-Morgenstern (vNM) utilities as follows: ui(s) = Es[ui], where Es is the expectation taken with respect to the probability distribution of play induced by the players'' mixed strategy s. 2 By employing the normal form, we model agents as playing a single action, with decisions taken simultaneously.
This is appropriate for our current study, which treats strategies (agent programs) as atomic actions.
We could capture finer-grained decisions about action over time in the extensive form.
Although any extensive game can be recast in normal form, doing so may sacrifice compactness and blur relevant distinctions (e.g., subgame perfection).
Occasionally, we write ui(x, y) to mean that x ∈ Ai or Si and y ∈ A−i or S−i depending on context.
We also express the set of utility functions of all players as u(·) = {u1(·), ... , um(·)}.
We define a function, : R → R, interpreted as the maximum benefit any player can obtain by deviating from its strategy in the specified profile.
(r) = max i∈I max ai∈Ai [ui(ai, r−i) − ui(r)], (1) where r belongs to some strategy set, R, of either pure or mixed strategies.
Faced with a game, an agent would ideally play its best strategy given those played by the other agents.
A configuration where all agents play strategies that are best responses to the others constitutes a Nash equilibrium.
DEFINITION 1.
A strategy profile r = (r1, ... , rm) constitutes a Nash equilibrium of game [I, {Ri}, {ui(r)}] if for every i ∈ I, ri ∈ Ri, ui(ri, r−i) ≥ ui(ri, r−i).
When r ∈ A, the above defines a pure strategy Nash equilibrium; otherwise the definition describes a mixed strategy Nash equilibrium.
We often appeal to the concept of an approximate, or -Nash equilibrium, where is the maximum benefit to any agent for deviating from the prescribed strategy.
Thus, (r) as defined above (1) is such that profile r is an -Nash equilibrium iff (r) ≤ .
In this study we devote particular attention to games that exhibit symmetry with respect to payoffs, rendering agents strategically identical.
DEFINITION 2.
A game [I, {Ri}, {ui(r)}] is symmetric if for all i, j ∈ I, (a) Ri = Rj and (b) ui(ri, r−i) = uj (rj, r−j) whenever ri = rj and r−i = r−j 3.
THE MODEL We model the strategic interactions between the designer of the mechanism and its participants as a two-stage game.
The designer moves first by selecting a value, θ, from a set of allowable mechanism settings, Θ.
All the participant agents observe the mechanism parameter θ and move simultaneously thereafter.
For example, the designer could be deciding between a first-price and second-price sealed-bid auction mechanisms, with the presumption that after the choice has been made, the bidders will participate with full awareness of the auction rules.
Since the participants play with full knowledge of the mechanism parameter, we define a game between them in the second stage as Γθ = [I, {Ri}, {ui(r, θ)}].
We refer to Γθ as a game induced by θ.
Let N(θ) be the set of strategy profiles considered solutions of the game Γθ.3 Suppose that the goal of the designer is to optimize the value of some welfare function, W (r, θ), dependent on the mechanism parameter and resulting play, r.
We define a pessimistic measure, W ( ˆR, θ) = inf{W (r, θ) : r ∈ ˆR}, representing the worst-case welfare of the game induced by θ, assuming that agents play some joint strategy in ˆR.
Typically we care about W (N(θ), θ), the worst-case outcome of playing some solution.4 On some problems we can gain considerable advantage by using an aggregation function to map the welfare outcome of a game 3 We generally adopt Nash equilibrium as the solution concept, and thus take N(θ) to be the set of equilibria.
However, much of the methodology developed here could be employed with alternative criteria for deriving agent behavior from a game definition.
4 Again, alternatives are available.
For example, if one has a probability distribution over the solution set N(θ), it would be natural to take the expectation of W (r, θ) instead.
307 specified in terms of agent strategies to an equivalent welfare outcome specified in terms of a lower-dimensional summary.
DEFINITION 3.
A function φ : R1 × · · · × Rm → Rq is an aggregation function if m ≥ q and W (r, θ) = V (φ(r), θ) for some function V .
We overload the function symbol to apply to sets of strategy profiles: φ( ˆR) = {φ(r) : r ∈ ˆR}.
For convenience of exposition, we write φ∗ (θ) to mean φ(N(θ)).
Using an aggregation function yields a more compact representation of strategy profiles.
For example, suppose-as in our application below-that an agent``s strategy is defined by a numeric parameter.
If all we care about is the total value played, we may take φ(a) = Pm i=1 ai.
If we have chosen our aggregator carefully, we may also capture structure not obvious otherwise.
For example, φ∗ (θ) could be decreasing in θ, whereas N(θ) might have a more complex structure.
Given a description of the solution correspondence N(θ) (equivalently, φ∗ (θ)), the designer faces a standard optimization problem.
Alternatively, given a simulator that could produce an unbiased sample from the distribution of W (N(θ), θ) for any θ, the designer would be faced with another much appreciated problem in the literature: simulation optimization [12].
However, even for a game Γθ with known payoffs it may be computationally intractable to solve for Nash equilibria, particularly if the game has large or infinite strategy sets.
Additionally, we wish to study games where the payoffs are not explicitly given, but must be determined from simulation or other experience with the game.5 Accordingly, we assume that we are given a (possibly noisy) data set of payoff realizations: Do = {(θ1 , a1 , U1 ), ... , (θk , ak , Uk )}, where for every data point θi is the observed mechanism parameter setting, ai is the observed pure strategy profile of the participants, and Ui is the corresponding realization of agent payoffs.
We may also have additional data generated by a (possibly noisy) simulator: Ds = {(θk+1 , ak+1 , Uk+1 ), ... , (θk+l , ak+l , Uk+l )}.
Let D = {Do, Ds} be the combined data set.
(Either Do or Ds may be null for a particular problem.)
In the remainder of this paper, we apply our modeling approach, together with several empirical game-theoretic methods, in order to answer questions regarding the design of the TAC/SCM scenario.
4.
EMPIRICAL DESIGN ANALYSIS Since our data comes in the form of payoff experience and not as the value of an objective function for given settings of the control variable, we can no longer rely on the methods for optimizing functions using simulations.
Indeed, a fundamental aspect of our design problem involves estimating the Nash equilibrium correspondence.
Furthermore, we cannot rely directly on the convergence results that abound in the simulation optimization literature, and must establish probabilistic analysis methods tailored for our problem setting.
4.1 TAC/SCM Design Problem We describe our empirical design analysis methods by presenting a detailed application to the TAC/SCM scenario introduced above.
Recall that during the 2004 tournament, the designers of the supplychain game chose to dramatically increase storage costs as a measure aimed at curbing day-0 procurement, to little avail.
Here we systematically explore the relationship between storage costs and 5 This is often the case for real games of interest, where natural language or algorithmic descriptions may substitute for a formal specification of strategy and payoff functions.
the aggregate quantity of components procured on day 0 in equilibrium.
In doing so, we consider several questions raised during and after the tournament.
First, does increasing storage costs actually reduce day-0 procurement?
Second, was the excessive day-0 procurement that was observed during the 2004 tournament rational?
And third, could increasing storage costs sufficiently have reduced day-0 procurement to an acceptable level, and if so, what should the setting of storage costs have been?
It is this third question that defines the mechanism design aspect of our analysis.6 To apply our methods, we must specify the agent strategy sets, the designer``s welfare function, the mechanism parameter space, and the source of data.
We restrict the agent strategies to be a multiplier on the quantity of the day-0 requests by one of the finalists, Deep Maize, in the 2004 TAC/SCM tournament.
We further restrict it to the set [0,1.5], since any strategy below 0 is illegal and strategies above 1.5 are extremely aggressive (thus unlikely to provide refuting deviations beyond those available from included strategies, and certainly not part of any desirable equilibrium).
All other behavior is based on the behavior of Deep Maize and is identical for all agents.
This choice can provide only an estimate of the actual tournament behavior of a typical agent.
However, we believe that the general form of the results should be robust to changes in the full agent behavior.
We model the designer``s welfare function as a threshold on the sum of day-0 purchases.
Let φ(a) = P6 i=1 ai be the aggregation function representing the sum of day-0 procurement of the six agents participating in a particular supply-chain game (for mixed strategy profiles s, we take expectation of φ with respect to the mixture).
The designer``s welfare function W (N(θ), θ) is then given by I{sup{φ∗ (θ)} ≤ α}, where α is the maximum acceptable level of day-0 procurement and I is the indicator function.
The designer selects a value θ of storage costs, expressed as an annual percentage of the baseline value of components in the inventory (charged daily), from the set Θ = R+ .
Since the designer``s decision depends only on φ∗ (θ), we present all of our results in terms of the value of the aggregation function.
4.2 Estimating Nash Equilibria The objective of TAC/SCM agents is to maximize profits realized over a game instance.
Thus, if we fix a strategy for each agent at the beginning of the simulation and record the corresponding profits at the end, we will have obtained a data point in the form (a, U(a)).
If we also have fixed the parameter θ of the simulator, the resulting data point becomes part of our data set D.
This data set, then, contains data only in the form of pure strategies of players and their corresponding payoffs, and, consequently, in order to formulate the designer``s problem as optimization, we must first determine or approximate the set of Nash equilibria of each game Γθ.
Thus, we need methods for approximating Nash equilibria for infinite games.
Below, we describe the two methods we used in our study.
The first has been explored empirically before, whereas the second is introduced here as the method specifically designed to approximate a set of Nash equilibria.
4.2.1 Payoff Function Approximation The first method for estimating Nash equilibria based on data uses supervised learning to approximate payoff functions of mech6 We do not address whether and how other measures (e.g., constraining procurement directly) could have achieved design objectives.
Our approach takes as given some set of design options, in this case defined by the storage cost parameter.
In principle our methods could be applied to a different or larger design space, though with corresponding complexity growth.
308 anism participants from a data set of game experience [17].
Once approximate payoff functions are available for all players, the Nash equilibria may be either found analytically or approximated using numerical techniques, depending on the learning model.
In what follows, we estimate only a sample Nash equilibrium using this technique, although this restriction can be removed at the expense of additional computation time.
One advantage of this method is that it can be applied to any data set and does not require the use of a simulator.
Thus, we can apply it when Ds = ∅.
If a simulator is available, we can generate additional data to build confidence in our initial estimates.7 We tried the following methods for approximating payoff functions: quadratic regression (QR), locally weighted average (LWA), and locally weighted linear regression (LWLR).
We also used control variates to reduce the variance of payoff estimates, as in our previous empirical game-theoretic analysis of TAC/SCM-03 [19].
The quadratic regression model makes it possible to compute equilibria of the learned game analytically.
For the other methods we applied replicator dynamics [7] to a discrete approximation of the learned game.
The expected total day-0 procurement in equilibrium was taken as the estimate of an outcome.
4.2.2 Search in Strategy Profile Space When we have access to a simulator, we can also use directed search through profile space to estimate the set of Nash equilibria, which we describe here after presenting some additional notation.
DEFINITION 4.
A strategic neighbor of a pure strategy profile a is a profile that is identical to a in all but one strategy.
We define Snb(a, D) as the set of all strategic neighbors of a available in the data set D. Similarly, we define Snb(a, ˜D) to be all strategic neighbors of a not in D. Finally, for any a ∈ Snb(a, D) we define the deviating agent as i(a, a ).
DEFINITION 5.
The -bound, ˆ, of a pure strategy profile a is defined as maxa ∈Snb(a,D) max{ui(a,a )(a )−ui(a,a )(a), 0}.
We say that a is a candidate δ-equilibrium for δ ≥ ˆ.
When Snb(a, ˜D) = ∅ (i.e., all strategic neighbors are represented in the data), a is confirmed as an ˆ-Nash equilibrium.
Our search method operates by exploring deviations from candidate equilibria.
We refer to it as BestFirstSearch, as it selects with probability one a strategy profile a ∈ Snb(a, ˜D) that has the smallest ˆin D. Finally we define an estimator for a set of Nash equilibria.
DEFINITION 6.
For a set K, define Co(K) to be the convex hull of K. Let Bδ be the set of candidates at level δ.
We define ˆφ∗ (θ) = Co({φ(a) : a ∈ Bδ}) for a fixed δ to be an estimator of φ∗ (θ).
In words, the estimate of a set of equilibrium outcomes is the convex hull of all aggregated strategy profiles with -bound below some fixed δ.
This definition allows us to exploit structure arising from the aggregation function.
If two profiles are close in terms of aggregation values, they may be likely to have similar -bounds.
In particular, if one is an equilibrium, the other may be as well.
We present some theoretical support for this method of estimating the set of Nash equilibria below.
Since the game we are interested in is infinite, it is necessary to terminate BestFirstSearch before exploring the entire space of strat7 For example, we can use active learning techniques [5] to improve the quality of payoff function approximation.
In this work, we instead concentrate on search in strategy profile space.
egy profiles.
We currently determine termination time in a somewhat ad-hoc manner, based on observations about the current set of candidate equilibria.8 4.3 Data Generation Our data was collected by simulating TAC/SCM games on a local version of the 2004 TAC/SCM server, which has a configuration setting for the storage cost.
Agent strategies in simulated games were selected from the set {0, 0.3, 0.6, ... , 1.5} in order to have positive probability of generating strategic neighbors.9 A baseline data set Do was generated by sampling 10 randomly generated strategy profiles for each θ ∈ {0, 50, 100, 150, 200}.
Between 5 and 10 games were run for each profile after discarding games that had various flaws.10 We used search to generate a simulated data set Ds, performing between 12 and 32 iterations of BestFirstSearch for each of the above settings of θ.
Since simulation cost is extremely high (a game takes nearly 1 hour to run), we were able to run a total of 2670 games over the span of more than six months.
For comparison, to get the entire description of an empirical game defined by the restricted finite joint strategy space for each value of θ ∈ {0, 50, 100, 150, 200} would have required at least 23100 games (sampling each profile 10 times).
4.4 Results 4.4.1 Analysis of the Baseline Data Set We applied the three learning methods described above to the baseline data set Do.
Additionally, we generated an estimate of the Nash equilibrium correspondence, ˆφ∗ (θ), by applying Definition 6 with δ =2.5E6.
The results are shown in Figure 1.
As we can see, the correspondence ˆφ∗ (θ) has little predictive power based on Do, and reveals no interesting structure about the game.
In contrast, all three learning methods suggest that total day-0 procurement is a decreasing function of storage costs.
0 1 2 3 4 5 6 7 8 9 10 0 50 100 150 200 Storage Cost TotalDay-0Procurement LWA LWLR QR BaselineMin BaselineMax Figure 1: Aggregate day-0 procurement estimates based on Do.
The correspondence ˆφ∗ (θ) is the interval between BaselineMin and BaselineMax.
8 Generally, search is terminated once the set of candidate equilibria is small enough to draw useful conclusions about the likely range of equilibrium strategies in the game.
9 Of course, we do not restrict our Nash equilibrium estimates to stay in this discrete subset of [0,1.5].
10 For example, if we detected that any agent failed during the game (failures included crashes, network connectivity problems, and other obvious anomalies), the game would be thrown out.
309 4.4.2 Analysis of Search Data To corroborate the initial evidence from the learning methods, we estimated ˆφ∗ (θ) (again, using δ =2.5E6) on the data set D = {Do, Ds}, where Ds is data generated through the application of BestFirstSearch.
The results of this estimate are plotted against the results of the learning methods trained on Do 11 in Figure 2.
First, we note that the addition of the search data narrows the range of potential equilibria substantially.
Furthermore, the actual point predictions of the learning methods and those based on -bounds after search are reasonably close.
Combining the evidence gathered from these two very different approaches to estimating the outcome correspondence yields a much more compelling picture of the relationship between storage costs and day-0 procurement than either method used in isolation.
0 1 2 3 4 5 6 7 8 9 10 0 50 100 150 200 Storage Cost TotayDay-0Procurement LWA LWLR QR SearchMin SearchMax Figure 2: Aggregate day-0 procurement estimates based on search in strategy profile space compared to function approximation techniques trained on Do.
The correspondence ˆφ∗ (θ) for D = {Do, Ds} is the interval between SearchMin and SearchMax.
This evidence supports the initial intuition that day-0 procurement should be decreasing with storage costs.
It also confirms that high levels of day-0 procurement are a rational response to the 2004 tournament setting of average storage cost, which corresponds to θ = 100.
The minimum prediction for aggregate procurement at this level of storage costs given by any experimental methods is approximately 3.
This is quite high, as it corresponds to an expected commitment of 1/3 of the total supplier capacity for the entire game.
The maximum prediction is considerably higher at 4.5.
In the actual 2004 competition, aggregate day-0 procurement was equivalent to 5.71 on the scale used here [9].
Our predictions underestimate this outcome to some degree, but show that any rational outcome was likely to have high day-0 procurement.
4.4.3 Extrapolating the Solution Correspondence We have reasonably strong evidence that the outcome correspondence is decreasing.
However, the ultimate goal is to be able to either set the storage cost parameter to a value that would curb day-0 procurement in equilibrium or conclude that this is not possible.
To answer this question directly, suppose that we set a conservative threshold α = 2 on aggregate day-0 procurement.12 Linear 11 It is unclear how meaningful the results of learning would be if Ds were added to the training data set.
Indeed, the additional data may actually increase the learning variance.
12 Recall that designer``s objective is to incentivize aggergate day-0 procurement that is below the threshold α.
Our threshold here still represents a commitment of over 20% of the suppliers'' capacity for extrapolation of the maximum of the outcome correspondence estimated from D yields θ = 320.
The data for θ = 320 were collected in the same way as for other storage cost settings, with 10 randomly generated profiles followed by 33 iterations of BestFirstSearch.
Figure 3 shows the detailed -bounds for all profiles in terms of their corresponding values of φ.
0.00E+00 5.00E+06 1.00E+07 1.50E+07 2.00E+07 2.50E+07 3.00E+07 3.50E+07 4.00E+07 4.50E+07 5.00E+07 2.1 2.4 2.7 3 3.3 3.6 3.9 4.2 4.5 4.8 5.1 5.4 5.7 6 6.3 6.6 6.9 7.2 Total Day-0 Procurement ε−boundFigure 3: Values of ˆ for profiles explored using search when θ = 320.
Strategy profiles explored are presented in terms of the corresponding values of φ(a).
The gray region corresponds to ˆφ∗ (320) with δ =2.5M.
The estimated set of aggregate day-0 outcomes is very close to that for θ = 200, indicating that there is little additional benefit to raising storage costs above 200.
Observe, that even the lower bound of our estimated set of Nash equilibria is well above the target day-0 procurement of 2.
Furthermore, payoffs to agents are almost always negative at θ = 320.
Consequently, increasing the costs further would be undesirable even if day-0 procurement could eventually be curbed.
Since we are reasonably confident that φ∗ (θ) is decreasing in θ, we also do not expect that setting θ somewhere between 200 and 320 will achieve the desired result.
We conclude that it is unlikely that day-0 procurement could ever be reduced to a desirable level using any reasonable setting of the storage cost parameter.
That our predictions tend to underestimate tournament outcomes reinforces this conclusion.
To achieve the desired reduction in day-0 procurement requires redesigning other aspects of the mechanism.
4.5 Probabilistic Analysis Our empirical analysis has produced evidence in support of the conclusion that no reasonable setting of storage cost was likely to sufficiently curb excessive day-0 procurement in TAC/SCM ``04.
All of this evidence has been in the form of simple interpolation and extrapolation of estimates of the Nash equilibrium correspondence.
These estimates are based on simulating game instances, and are subject to sampling noise contributed by the various stochastic elements of the game.
In this section, we develop and apply methods for evaluating the sensitivity of our -bound calculations to such stochastic effects.
Suppose that all agents have finite (and small) pure strategy sets, A. Thus, it is feasible to sample the entire payoff matrix of the game.
Additionally, suppose that noise is additive with zero-mean the entire game on average, so in practice we would probably want the threshold to be even lower.
310 and finite variance, that is, Ui(a) = ui(a) + ˜ξi(a), where Ui(a) is the observed payoff to i when a was played, ui(a) is the actual corresponding payoff, and ˜ξi(a) is a mean-zero normal random variable.
We designate the known variance of ˜ξi(a) by σ2 i (a).
Thus, we assume that ˜ξi(a) is normal with distribution N(0, σ2 i (a)).
We take ¯ui(a) to be the sample mean over all Ui(a) in D, and follow Chang and Huang [3] to assume that we have an improper prior over the actual payoffs ui(a) and sampling was independent for all i and a.
We also rely on their result that ui(a)|¯ui(a) = ¯ui(a)−Zi(a)/[σi(a)/ p ni(a)] are independent with posterior distributions N(¯ui(a), σ2 i (a)/ni(a)), where ni(a) is the number of samples taken of payoffs to i for pure profile a, and Zi(a) ∼ N(0, 1).
We now derive a generic probabilistic bound that a profile a ∈ A is an -Nash equilibrium.
If ui(·)|¯ui(·) are independent for all i ∈ I and a ∈ A, we have the following result (from this point on we omit conditioning on ¯ui(·) for brevity): PROPOSITION 1.
Pr „ max i∈I max b∈Ai ui(b, a−i) − ui(a) ≤ `` = = Y i∈I Z R Y b∈Ai\ai Pr(ui(b, a−i) ≤ u + )fui(a)(u)du, (2) where fui(a)(u) is the pdf of N(¯ui(a), σi(a)).
The proofs of this and all subsequent results are in the Appendix.
The posterior distribution of the optimum mean of n samples, derived by Chang and Huang [3], is Pr (ui(a) ≤ c) = 1 − Φ ``p ni(a)(¯ui(a) − c) σi(a) # , (3) where a ∈ A and Φ(·) is the N(0, 1) distribution function.
Combining the results (2) and (3), we obtain a probabilistic confidence bound that (a) ≤ γ for a given γ.
Now, we consider cases of incomplete data and use the results we have just obtained to construct an upper bound (restricted to profiles represented in data) on the distribution of sup{φ∗ (θ)} and inf{φ∗ (θ)} (assuming that both are attainable): Pr{sup{φ∗ (θ)} ≤ x} ≤D Pr{∃a ∈ D : φ(a) ≤ x ∧ a ∈ N(θ)} ≤ X a∈D:φ(a)≤x Pr{a ∈ N(θ)} = X a∈D:φ(a)≤x Pr{ (a) = 0}, where x is a real number and ≤D indicates that the upper bound accounts only for strategies that appear in the data set D.
Since the events {∃a ∈ D : φ(a) ≤ x ∧ a ∈ N(θ)} and {inf{φ∗ (θ)} ≤ x} are equivalent, this also defines an upper bound on the probability of {inf{φ∗ (θ)} ≤ x}.
The values thus derived comprise the Tables 1 and 2.
φ∗ (θ) θ = 0 θ = 50 θ = 100 <2.7 0.000098 0 0.146 <3 0.158 0.0511 0.146 <3.9 0.536 0.163 1 <4.5 1 1 1 Table 1: Upper bounds on the distribution of inf{φ∗ (θ)} restricted to D for θ ∈ {0, 50, 100} when N(θ) is a set of Nash equilibria.
φ∗ (θ) θ = 150 θ = 200 θ = 320 <2.7 0 0 0.00132 <3 0.0363 0.141 1 <3.9 1 1 1 <4.5 1 1 1 Table 2: Upper bounds on the distribution of inf{φ∗ (θ)} restricted to D for θ ∈ {150, 200, 320} when N(θ) is a set of Nash equilibria.
Tables 1 and 2 suggest that the existence of any equilibrium with φ(a) < 2.7 is unlikely for any θ that we have data for, although this judgment, as we mentioned, is only with respect to the profiles we have actually sampled.
We can then accept this as another piece of evidence that the designer could not find a suitable setting of θ to achieve his objectives-indeed, the designer seems unlikely to achieve his objective even if he could persuade participants to play a desirable equilibrium!
Table 1 also provides additional evidence that the agents in the 2004 TAC/SCM tournament were indeed rational in procuring large numbers of components at the beginning fo the game.
If we look at the third column of this table, which corresponds to θ = 100, we can gather that no profile a in our data with φ(a) < 3 is very likely to be played in equilibrium.
The bounds above provide some general evidence, but ultimately we are interested in a concrete probabilistic assessment of our conclusion with respect to the data we have sampled.
Particularly, we would like to say something about what happens for the settings of θ for which we have no data.
To derive an approximate probabilistic bound on the probability that no θ ∈ Θ could have achieved the designer``s objective, let ∪J j=1Θj, be a partition of Θ, and assume that the function sup{φ∗ (θ)} satisfies the Lipschitz condition with Lipschitz constant Aj on each subset Θj.13 Since we have determined that raising the storage cost above 320 is undesirable due to secondary considerations, we restrict attention to Θ = [0, 320].
We now define each subset j to be the interval between two points for which we have produced data.
Thus, Θ = [0, 50] [ (50, 100] [ (100, 150] [ (150, 200] [ (200, 320], with j running between 1 and 5, corresponding to subintervals above.
We will further denote each Θj by (aj , bj].14 Then, the following Proposition gives us an approximate upper bound15 on the probability that sup{φ∗ (θ)} ≤ α.
PROPOSITION 2.
Pr{ _ θ∈Θ sup{φ(θ)} ≤ α} ≤D 5X j=1 X y,z∈D:y+z≤cj 0 @ X a:φ(a)=z Pr{ (a) = 0} 1 A × × 0 @ X a:φ(a)=y Pr{ (a) = 0} 1 A , where cj = 2α + Aj(bj − aj) and ≤D indicates that the upper bound only accounts for strategies that appear in the data set D. 13 A function that satisfies the Lipschitz condition is called Lipschitz continuous.
14 The treatment for the interval [0,50] is identical.
15 It is approximate in a sense that we only take into account strategies that are present in the data.
311 Due to the fact that our bounds are approximate, we cannot use them as a conclusive probabilistic assessment.
Instead, we take this as another piece of evidence to complement our findings.
Even if we can assume that a function that we approximate from data is Lipschitz continuous, we rarely actually know the Lipschitz constant for any subset of Θ.
Thus, we are faced with a task of estimating it from data.
Here, we tried three methods of doing this.
The first one simply takes the highest slope that the function attains within the available data and uses this constant value for every subinterval.
This produces the most conservative bound, and in many situations it is unlikely to be informative.
An alternative method is to take an upper bound on slope obtained within each subinterval using the available data.
This produces a much less conservative upper bound on probabilities.
However, since the actual upper bound is generally greater for each subinterval, the resulting probabilistic bound may be deceiving.
A final method that we tried is a compromise between the two above.
Instead of taking the conservative upper bound based on data over the entire function domain Θ, we take the average of upper bounds obtained at each Θj.
The bound at an interval is then taken to be the maximum of the upper bound for this interval and the average upper bound for all intervals.
The results of evaluating the expression for Pr{ _ θ∈Θ sup{φ∗ (θ)} ≤ α} when α = 2 are presented in Table 3.
In terms of our claims in maxj Aj Aj max{Aj ,ave(Aj)} 1 0.00772 0.00791 Table 3: Approximate upper bound on probability that some setting of θ ∈ [0, 320] will satisfy the designer objective with target α = 2.
Different methods of approximating the upper bound on slope in each subinterval j are used.
this work, the expression gives an upper bound on the probability that some setting of θ (i.e., storage cost) in the interval [0,320] will result in total day-0 procurement that is no greater in any equilibrium than the target specified by α and taken here to be 2.
As we had suspected, the most conservative approach to estimating the upper bound on slope, presented in the first column of the table, provides us little information here.
However, the other two estimation approaches, found in columns two and three of Table 3, suggest that we are indeed quite confident that no reasonable setting of θ ∈ [0, 320] would have done the job.
Given the tremendous difficulty of the problem, this result is very strong.16 Still, we must be very cautious in drawing too heroic a conclusion based on this evidence.
Certainly, we have not checked all the profiles but only a small proportion of them (infinitesimal, if we consider the entire continuous domain of θ and strategy sets).
Nor can we expect ever to obtain enough evidence to make completely objective conclusions.
Instead, the approach we advocate here is to collect as much evidence as is feasible given resource constraints, and make the most compelling judgment based on this evidence, if at all possible.
5.
CONVERGENCE RESULTS At this point, we explore abstractly whether a design parameter choice based on payoff data can be asymptotically reliable.
16 Since we did not have all the possible deviations for any profile available in the data, the true upper bounds may be even lower.
As a matter of convenience, we will use notation un,i(a) to refer to a payoff function of player i based on an average over n i.i.d. samples from the distribution of payoffs.
We also assume that un,i(a) are independent for all a ∈ A and i ∈ I.
We will use the notation Γn to refer to the game [I, R, {ui,n(·)}], whereas Γ will denote the underlying game, [I, R, {ui(·)}].
Similarly, we define n(r) to be (r) with respect to the game Γn.
In this section, we show that n(s) → (s) a.s. uniformly on the mixed strategy space for any finite game, and, furthermore, that all mixed strategy Nash equilibria in empirical games eventually become arbitrarily close to some Nash equilibrium strategies in the underlying game.
We use these results to show that under certain conditions, the optimal choice of the design parameter based on empirical data converges almost surely to the actual optimum.
THEOREM 3.
Suppose that |I| < ∞, |A| < ∞.
Then n(s) → (s) a.s. uniformly on S. Recall that N is a set of all Nash equilibria of Γ.
If we define Nn,γ = {s ∈ S : n(s) ≤ γ}, we have the following corollary to Theorem 3: COROLLARY 4.
For every γ > 0, there is M such that ∀n ≥ M, N ⊂ Nn,γ a.s. PROOF.
Since (s) = 0 for every s ∈ N, we can find M large enough such that Pr{supn≥M sups∈N n(s) < γ} = 1.
By the Corollary, for any game with a finite set of pure strategies and for any > 0, all Nash equilibria lie in the set of empirical -Nash equilibria if enough samples have been taken.
As we now show, this provides some justification for our use of a set of profiles with a non-zero -bound as an estimate of the set of Nash equilibria.
First, suppose we conclude that for a particular setting of θ, sup{ˆφ∗ (θ)} ≤ α.
Then, since for any fixed > 0, N(θ) ⊂ Nn, (θ) when n is large enough, sup{φ∗ (θ)} = sup s∈N (θ) φ(s) ≤ sup s∈Nn, (θ) φ(s) = sup{ˆφ∗ (θ)} ≤ α for any such n. Thus, since we defined the welfare function of the designer to be I{sup{φ∗ (θ)} ≤ α} in our domain of interest, the empirical choice of θ satisfies the designer``s objective, thereby maximizing his welfare function.
Alternatively, suppose we conclude that inf{ˆφ∗ (θ)} > α for every θ in the domain.
Then, α < inf{ˆφ∗ (θ)} = inf s∈Nn, (θ) φ(s) ≤ inf s∈N (θ) φ(s) ≤ ≤ sup s∈N (θ) φ(s) = sup{φ∗ (θ)}, for every θ, and we can conclude that no setting of θ will satisfy the designer``s objective.
Now, we will show that when the number of samples is large enough, every Nash equilibrium of Γn is close to some Nash equilibrium of the underlying game.
This result will lead us to consider convergence of optimizers based on empirical data to actual optimal mechanism parameter settings.
We first note that the function (s) is continuous in a finite game.
LEMMA 5.
Let S be a mixed strategy set defined on a finite game.
Then : S → R is continuous.
312 For the exposition that follows, we need a bit of additional notation.
First, let (Z, d) be a metric space, and X, Y ⊂ Z and define directed Hausdorff distance from X to Y to be h(X, Y ) = sup x∈X inf y∈Y d(x, y).
Observe that U ⊂ X ⇒ h(U, Y ) ≤ h(X, Y ).
Further, define BS(x, δ) to be an open ball in S ⊂ Z with center x ∈ S and radius δ.
Now, let Nn denote all Nash equilibria of the game Γn and let Nδ = [ x∈N BS(x, δ), that is, the union of open balls of radius δ with centers at Nash equilibria of Γ.
Note that h(Nδ, N) = δ.
We can then prove the following general result.
THEOREM 6.
Suppose |I| < ∞ and |A| < ∞.
Then almost surely h(Nn, N) converges to 0.
We will now show that in the special case when Θ and A are finite and each Γθ has a unique Nash equilibrium, the estimates ˆθ of optimal designer parameter converge to an actual optimizer almost surely.
Let ˆθ = arg maxθ∈Θ W (Nn(θ), θ), where n is the number of times each pure profile was sampled in Γθ for every θ, and let θ∗ = arg maxθ∈Θ W (N(θ), θ).
THEOREM 7.
Suppose |N(θ)| = 1 for all θ ∈ Θ and suppose that Θ and A are finite.
Let W (s, θ) be continuous at the unique s∗ (θ) ∈ N(θ) for each θ ∈ Θ.
Then ˆθ is a consistent estimator of θ∗ if W (N(θ), θ) is defined as a supremum, infimum, or expectation over the set of Nash equilibria.
In fact, ˆθ → θ∗ a.s. in each of these cases.
The shortcoming of the above result is that, within our framework, the designer has no way of knowing or ensuring that Γθ do, indeed, have unique equilibria.
However, it does lend some theoretical justification for pursuing design in this manner, and, perhaps, will serve as a guide for more general results in the future.
6.
RELATED WORK The mechanism design literature in Economics has typically explored existence of a mechanism that implements a social choice function in equilibrium [10].
Additionally, there is an extensive literature on optimal auction design [10], of which the work by Roger Myerson [11] is, perhaps, the most relevant.
In much of this work, analytical results are presented with respect to specific utility functions and accounting for constraints such as incentive compatibility and individual rationality.
Several related approaches to search for the best mechanism exist in the Computer Science literature.
Conitzer and Sandholm [6] developed a search algorithm when all the relevant game parameters are common knowledge.
When payoff functions of players are unknown, a search using simulations has been explored as an alternative.
One approach in that direction, taken in [4] and [15], is to co-evolve the mechanism parameter and agent strategies, using some notion of social utility and agent payoffs as fitness criteria.
An alternative to co-evolution explored in [16] was to optimize a well-defined welfare function of the designer using genetic programming.
In this work the authors used a common learning strategy for all agents and defined an outcome of a game induced by a mechanism parameter as the outcome of joint agent learning.
Most recently, Phelps et al. [14] compared two mechanisms based on expected social utility with expectation taken over an empirical distribution of equilibria in games defined by heuristic strategies, as in [18].
7.
CONCLUSION In this work we spent considerable effort developing general tactics for empirical mechanism design.
We defined a formal gametheoretic model of interaction between the designer and the participants of the mechanism as a two-stage game.
We also described in some generality the methods for estimating a sample Nash equilibrium function when the data is extremely scarce, or a Nash equilibrium correspondence when more data is available.
Our techniques are designed specifically to deal with problems in which both the mechanism parameter space and the agent strategy sets are infinite and only a relatively small data set can be acquired.
A difficult design issue in the TAC/SCM game which the TAC community has been eager to address provides us with a setting to test our methods.
In applying empirical game analysis to the problem at hand, we are fully aware that our results are inherently inexact.
Thus, we concentrate on collecting evidence about the structure of the Nash equilibrium correspondence.
In the end, we can try to provide enough evidence to either prescribe a parameter setting, or suggest that no setting is possible that will satisfy the designer.
In the case of TAC/SCM, our evidence suggests quite strongly that storage cost could not have been effectively adjusted in the 2004 tournament to curb excessive day-0 procurement without detrimental effects on overall profitability.
The success of our analysis in this extremely complex environment with high simulation costs makes us optimistic that our methods can provide guidance in making mechanism design decisions in other challenging domains.
The theoretical results confirm some intuitions behind the empirical mechanism design methods we have introduced, and increases our confidence that our framework can be effective in estimating the best mechanism parameter choice in relatively general settings.
Acknowledgments We thank Terence Kelly, Matthew Rudary, and Satinder Singh for helpful comments on earlier drafts of this work.
This work was supported in part by NSF grant IIS-0205435 and the DARPA REAL strategic reasoning program.
8.
REFERENCES [1] R. Arunachalam and N. M. Sadeh.
The supply chain trading agent competition.
Electronic Commerce Research and Applications, 4:63-81, 2005.
[2] M. Benisch, A. Greenwald, V. Naroditskiy, and M. Tschantz.
A stochastic programming approach to scheduling in TAC SCM.
In Fifth ACM Conference on Electronic Commerce, pages 152-159, New York, 2004.
[3] Y.-P.
Chang and W.-T.
Huang.
Generalized confidence intervals for the largest value of some functions of parameters under normality.
Statistica Sinica, 10:1369-1383, 2000.
[4] D. Cliff.
Evolution of market mechanism through a continuous space of auction-types.
In Congress on Evolutionary Computation, 2002.
[5] D. A. Cohn, Z. Ghahramani, and M. I. Jordan.
Active learning with statistical models.
Journal of Artificial Intelligence Research, 4:129-145, 1996.
[6] V. Conitzer and T. Sandholm.
An algorithm for automatically designing deterministic mechanisms without payments.
In 313 Third International Joint Conference on Autonomous Agents and Multi-Agent Systems, pages 128-135, 2004.
[7] D. Friedman.
Evolutionary games in economics.
Econometrica, 59(3):637-666, May 1991.
[8] R. Keener.
Statistical Theory: A Medley of Core Topics.
University of Michigan Department of Statistics, 2004.
[9] C. Kiekintveld, Y. Vorobeychik, and M. P. Wellman.
An analysis of the 2004 supply chain management trading agent competition.
In IJCAI-05 Workshop on Trading Agent Design and Analysis, Edinburgh, 2005.
[10] A. Mas-Colell, M. Whinston, and J. Green.
Microeconomic Theory.
Oxford University Press, 1995.
[11] R. B. Myerson.
Optimal auction design.
Mathematics of Operations Research, 6(1):58-73, February 1981.
[12] S. Olafsson and J. Kim.
Simulation optimization.
In E. Yucesan, C.-H.
Chen, J. Snowdon, and J. Charnes, editors, 2002 Winter Simulation Conference, 2002.
[13] D. Pardoe and P. Stone.
TacTex-03: A supply chain management agent.
SIGecom Exchanges, 4(3):19-28, 2004.
[14] S. Phelps, S. Parsons, and P. McBurney.
Automated agents versus virtual humans: an evolutionary game theoretic comparison of two double-auction market designs.
In Workshop on Agent Mediated Electronic Commerce VI, 2004.
[15] S. Phelps, S. Parsons, P. McBurney, and E. Sklar.
Co-evolution of auction mechanisms and trading strategies: towards a novel approach to microeconomic design.
In ECOMAS 2002 Workshop, 2002.
[16] S. Phelps, S. Parsons, E. Sklar, and P. McBurney.
Using genetic programming to optimise pricing rules for a double-auction market.
In Workshop on Agents for Electronic Commerce, 2003.
[17] Y. Vorobeychik, M. P. Wellman, and S. Singh.
Learning payoff functions in infinite games.
In Nineteenth International Joint Conference on Artificial Intelligence, pages 977-982, 2005.
[18] W. E. Walsh, R. Das, G. Tesauro, and J. O. Kephart.
Analyzing complex strategic interactions in multi-agent systems.
In AAAI-02 Workshop on Game Theoretic and Decision Theoretic Agents, 2002.
[19] M. P. Wellman, J. J. Estelle, S. Singh, Y. Vorobeychik, C. Kiekintveld, and V. Soni.
Strategic interactions in a supply chain game.
Computational Intelligence, 21(1):1-26, February 2005.
APPENDIX A. PROOFS A.1 Proof of Proposition 1 Pr „ max i∈I max b∈Ai\ai ui(b, a−i) − ui(a) ≤ `` = = Y i∈I Eui(a) '' Pr( max b∈Ai\ai ui(b, a−i) − ui(a) ≤ |ui(a)) = = Y i∈I Z R Y b∈Ai\ai Pr(ui(b, a−i) ≤ u + )fui(a)(u)du.
A.2 Proof of Proposition 2 First, let us suppose that some function, f(x) defined on [ai, bi], satisfy Lipschitz condition on (ai, bi] with Lipschitz constant Ai.
Then the following claim holds: Claim: infx∈(ai,bi] f(x) ≥ 0.5(f(ai) + f(bi) − Ai(bi − ai).
To prove this claim, note that the intersection of lines at f(ai) and f(bi) with slopes −Ai and Ai respectively will determine the lower bound on the minimum of f(x) on [ai, bi] (which is a lower bound on infimum of f(x) on (ai, bj ]).
The line at f(ai) is determined by f(ai) = −Aiai + cL and the line at f(bi) is determined by f(bi) = Aibi +cR.
Thus, the intercepts are cL = f(ai)+Aiai and cR = f(bi) + Aibi respectively.
Let x∗ be the point at which these lines intersect.
Then, x∗ = − f(x∗ ) − cR A = f(x∗ ) − cL A .
By substituting the expressions for cR and cL, we get the desired result.
Now, subadditivity gives us Pr{ _ θ∈Θ sup{φ∗ (θ)} ≤ α} ≤ 5X j=1 Pr{ _ θ∈Θj sup{φ∗ (θ)} ≤ α}, and, by the claim, Pr{ _ θ∈Θj sup{φ∗ (θ)} ≤ α} = 1 − Pr{ inf θ∈Θj sup{φ∗ (θ)} > α} ≤ Pr{sup{φ∗ (aj)} + sup{φ∗ (bj)} ≤ 2α + Aj(bj − aj )}.
Since we have a finite number of points in the data set for each θ, we can obtain the following expression: Pr{sup{φ∗ (aj)} + sup{φ∗ (bj)} ≤ cj } =D X y,z∈D:y+z≤cj Pr{sup{φ∗ (bj )} = y} Pr{sup{φ∗ (aj)} = z}.
We can now restrict attention to deriving an upper bound on Pr{sup{φ∗ (θ)} = y} for a fixed θ.
To do this, observe that Pr{sup{φ∗ (θ)} = y} ≤D Pr{ _ a∈D:φ(a)=y (a) = 0} ≤ X a∈D:φ(a)=y Pr{ (a) = 0} by subadditivity and the fact that a profile a is a Nash equilibrium if and only if (a) = 0.
Putting everything together yields the desired result.
A.3 Proof of Theorem 3 First, we will need the following fact: Claim: Given a function fi(x) and a set X, | maxx∈X f1(x) − maxx∈X f2(x)| ≤ maxx∈X |f1(x) − f2(x)|.
To prove this claim, observe that | max x∈X f1(x) − max x∈X f2(x)| = maxx f1(x) − maxx f2(x) if maxx f1(x) ≥ maxx f2(x) maxx f2(x) − maxx f1(x) if maxx f2(x) ≥ maxx f1(x) In the first case, max x∈X f1(x) − max x∈X f2(x) ≤ max x∈X (f1(x) − f2(x)) ≤ ≤ max x∈X |f1(x) − f2(x)|.
314 Similarly, in the second case, max x∈X f2(x) − max x∈X f1(x) ≤ max x∈X (f2(x) − f1(x)) ≤ ≤ max x∈X |f2(x) − f1(x)| = max x∈X |f1(x) − f2(x)|.
Thus, the claim holds.
By the Strong Law of Large Numbers, un,i(a) → ui(a) a.s. for all i ∈ I, a ∈ A.
That is, Pr{ lim n→∞ un,i(a) = ui(a)} = 1, or, equivalently [8], for any α > 0 and δ > 0, there is M(i, a) > 0 such that Pr{ sup n≥M(i,a) |un,i(a) − ui(a)| < δ 2|A| } ≥ 1 − α.
By taking M = maxi∈I maxa∈A M(i, a), we have Pr{max i∈I max a∈A sup n≥M |un,i(a) − ui(a)| < δ 2|A| } ≥ 1 − α.
Thus, by the claim, for any n ≥ M, sup n≥M | n(s) − (s)| ≤ max i∈I max ai∈Ai sup n≥M |un,i(ai, s−i) − ui(ai, s−i)|+ + sup n≥M max i∈I |un,i(s) − ui(s)| ≤ max i∈I max ai∈Ai X b∈A−i sup n≥M |un,i(ai, b) − ui(ai, b)|s−i(b)+ + max i∈I X b∈A sup n≥M |un,i(b) − ui(b)|s(b) ≤ max i∈I max ai∈Ai X b∈A−i sup n≥M |un,i(ai, b) − ui(ai, b)|+ + max i∈I X b∈A sup n≥M |un,i(b) − ui(b)| < max i∈I max ai∈Ai X b∈A−i ( δ 2|A| ) + max i∈I X b∈A ( δ 2|A| ) ≤ δ with probability at least 1 − α.
Note that since s−i(a) and s(a) are bounded between 0 and 1, we were able to drop them from the expressions above to obtain a bound that will be valid independent of the particular choice of s. Furthermore, since the above result can be obtained for an arbitrary α > 0 and δ > 0, we have Pr{limn→∞ n(s) = (s)} = 1 uniformly on S. A.4 Proof of Lemma 5 We prove the result using uniform continuity of ui(s) and preservation of continuity under maximum.
Claim: A function f : Rk → R defined by f(t) = Pk i=1 ziti, where zi are constants in R, is uniformly continuous in t.
The claim follows because |f(t)−f(t )| = | Pk i=1 zi(ti−ti)| ≤ Pk i=1 |zi||ti − ti|.
An immediate result of this for our purposes is that ui(s) is uniformly continuous in s and ui(ai, s−i) is uniformly continuous in s−i. Claim: Let f(a, b) be uniformly continuous in b ∈ B for every a ∈ A, with |A| < ∞.
Then V (b) = maxa∈A f(a, b) is uniformly continuous in b. To show this, take γ > 0 and let b, b ∈ B such that b − b < δ(a) ⇒ |f(a, b) − f(a, b )| < γ.
Now take δ = mina∈A δ(a).
Then, whenever b − b < δ, |V (b) − V (b )| = | max a∈A f(a, b) − max a∈A f(a, b )| ≤ max a∈A |f(a, b) − f(a, b )| < γ.
Now, recall that (s) = maxi[maxai∈Ai ui(ai, s−i) − ui(s)].
By the claims above, maxai∈Ai ui(ai, s−i) is uniformly continuous in s−i and ui(s) is uniformly continuous in s.
Since the difference of two uniformly continuous functions is uniformly continuous, and since this continuity is preserved under maximum by our second claim, we have the desired result.
A.5 Proof of Theorem 6 Choose δ > 0.
First, we need to ascertain that the following claim holds: Claim: ¯ = mins∈S\Nδ (s) exists and ¯ > 0.
Since Nδ is an open subset of compact S, it follows that S \ Nδ is compact.
As we had also proved in Lemma 5 that (s) is continuous, existence follows from the Weierstrass theorem.
That ¯ > 0 is clear since (s) = 0 if and only if s is a Nash equilibrium of Γ.
Now, by Theorem 3, for any α > 0 there is M such that Pr{ sup n≥M sup s∈S | n(s) − (s)| < ¯} ≥ 1 − α.
Consequently, for any δ > 0, Pr{ sup n≥M h(Nn, Nδ) < δ} ≥ Pr{∀n ≥ M Nn ⊂ Nδ} ≥ Pr{ sup n≥M sup s∈N (s) < ¯} ≥ Pr{ sup n≥M sup s∈S | n(s) − (s)| < ¯} ≥ 1 − α.
Since this holds for an arbitrary α > 0 and δ > 0, the desired result follows.
A.6 Proof of Theorem 7 Fix θ and choose δ > 0.
Since W (s, θ) is continuous at s∗ (θ), given > 0 there is δ > 0 such that for every s that is within δ of s∗ (θ), |W (s , θ) − W (s∗ (θ), θ)| < .
By Theorem 6, we can find M(θ) large enough such that all s ∈ Nn are within δ of s∗ (θ) for all n ≥ M(θ) with probability 1.
Consequently, for any > 0 we can find M(θ) large enough such that with probability 1 we have supn≥M(θ) sups ∈Nn |W (s , θ) − W (s∗ (θ), θ)| < .
Let us assume without loss of generality that there is a unique optimal choice of θ.
Now, since the set Θ is finite, there is also the second-best choice of θ (if there is only one θ ∈ Θ this discussion is moot anyway): θ∗∗ = arg max Θ\θ∗ W (s∗ (θ), θ).
Suppose w.l.o.g. that θ∗∗ is also unique and let ∆ = W (s∗ (θ∗ ), θ∗ ) − W (s∗ (θ∗∗ ), θ∗∗ ).
Then if we let < ∆/2 and let M = maxθ∈Θ M(θ), where each M(θ) is large enough such that supn≥M(θ) sups ∈Nn |W (s , θ)− W (s∗ (θ), θ)| < a.s., the optimal choice of θ based on any empirical equilibrium will be θ∗ with probability 1.
Thus, in particular, given any probability distribution over empirical equilibria, the best choice of θ will be θ∗ with probability 1 (similarly, if we take supremum or infimum of W (Nn(θ), θ) over the set of empirical equilibria in constructing the objective function).
315

Empirical Mechanism Design : Methods , with Application to a Supply-Chain Scenario
ABSTRACT
Our proposed methods employ learning and search techniques to estimate outcome features of interest as a function of mechanism parameter settings .
We illustrate our approach with a design task from a supply-chain trading competition .
Designers adopted several rule changes in order to deter particular procurement behavior , but the measures proved insufficient .
Our empirical mechanism analysis models the relation between a key design parameter and outcomes , confirming the observed behavior and indicating that no reasonable parameter settings would have been likely to achieve the desired effect .
More generally , we show that under certain conditions , the estimator of optimal mechanism parameter setting based on empirical data is consistent .
1 .
MOTIVATION
We illustrate our problem with an anecdote from a supply chain research exercise : the 2003 and 2004 Trading Agent Competition ( TAC ) Supply Chain Management ( SCM ) game .
TAC/SCM [ 1 ] defines a scenario where agents compete to maximize their profits as manufacturers in a supply chain .
The agents procure components from the various suppliers and assemble finished goods for sale to customers , repeatedly over a simulated year .1 1Information about TAC and the SCM game , including specifications , rules , and competition results , can be found at http://www.sics.se/tac .
As it happened , the specified negotiation behavior of suppliers provided a great incentive for agents to procure large quantities of components on day 0 : the very beginning of the simulation .
During the early rounds of the 2003 SCM competition , several agent developers discovered this , and the apparent success led to most agents performing the majority of their purchasing on day 0 .
Although jockeying for day-0 procurement turned out to be an interesting strategic issue in itself [ 19 ] , the phenomenon detracted from other interesting problems , such as adapting production levels to varying demand ( since component costs were already sunk ) , and dynamic management of production , sales , and inventory .
Several participants noted that the predominance of day-0 procurement overshadowed other key research issues , such as factory scheduling [ 2 ] and optimizing bids for customer orders [ 13 ] .
After the 2003 tournament , there was a general consensus in the TAC community that the rules should be changed to deter large day-0 procurement .
The task facing game organizers can be viewed as a problem in mechanism design .
The designers have certain game features under their control , and a set of objectives regarding game outcomes .
Unlike most academic treatments of mechanism design , the objective is a behavioral feature ( moderate day-0 procurement ) rather than an allocation feature like economic efficiency , and the allowed mechanisms are restricted to those judged to require only an incremental modification of the current game .
Replacing the supplychain negotiation procedures with a one-shot direct mechanism , for example , was not an option .
We believe that such operational restrictions and idiosyncratic objectives are actually quite typical of practical mechanism design settings , where they are perhaps more commonly characterized as incentive engineering problems .
In response to the problem , the TAC/SCM designers adopted several rule changes intended to penalize large day-0 orders .
These included modifications to supplier pricing policies and introduction of storage costs assessed on inventories of components and finished goods .
Despite the changes , day-0 procurement was very high in the early rounds of the 2004 competition .
In a drastic measure , the GameMaster imposed a fivefold increase of storage costs midway through the tournament .
Even this did not stem the tide , and day-0 procurement in the final rounds actually increased ( by some measures ) from 2003 [ 9 ] .
The apparent difficulty in identifying rule modifications that effect moderation in day-0 procurement is quite striking .
Although the designs were widely discussed , predictions for the effects of various proposals were supported primarily by intuitive arguments or at best by back-of-the-envelope calculations .
Much of the difficulty , of course , is anticipating the agents ' ( and their developers ' ) responses without essentially running a gaming exercise for this purpose .
The episode caused us to consider whether new ap
proaches or tools could enable more systematic analysis of design options .
Standard game-theoretic and mechanism design methods are clearly relevant , although the lack of an analytic description of the game seems to be an impediment .
Under the assumption that the simulator itself is the only reliable source of outcome computation , we refer to our task as empirical mechanism design .
In the sequel , we develop some general methods for empirical mechanism design and apply them to the TAC/SCM redesign problem .
Our analysis focuses on the setting of storage costs ( taking other game modifications as fixed ) , since this is the most direct deterrent to early procurement adopted .
Our results confirm the basic intuition that incentives for day-0 purchasing decrease as storage costs rise .
We also confirm that the high day-0 procurement observed in the 2004 tournament is a rational response to the setting of storage costs used .
Finally , we conclude from our data that it is very unlikely that any reasonable setting of storage costs would result in acceptable levels of day-0 procurement , so a different design approach would have been required to eliminate this problem .
Overall , we contribute a formal framework and a set of methods for tackling indirect mechanism design problems in settings where only a black-box description of players ' utilities is available .
Our methods incorporate estimation of sets of Nash equilibria and sample Nash equilibria , used in conjuction to support general claims about the structure of the mechanism designer 's utility , as well as a restricted probabilistic analysis to assess the likelihood of conclusions .
We believe that most realistic problems are too complex to be amenable to exact analysis .
Consequently , we advocate the approach of gathering evidence to provide indirect support of specific hypotheses .
2 .
PRELIMINARIES
A normalform game2 is denoted by [ I , { Ri } , { ui ( r ) } ] , where I refers to the set of players and m = | I | is the number of players .
Ri is the set of strategies available to player i ∈ I , with R = R1 × ... × Rm representing the set ofjoint strategies of all players .
We designate the set of pure strategies available to player i by Ai , and denote the joint set of pure strategies of all players by A = A1 × ... × Am .
It is often convenient to refer to a strategy of player i separately from that of the remaining players .
To accommodate this , we use a_i to denote the joint strategy of all players other than player i. Let Si be the set of all probability distributions ( mixtures ) over Ai and , similarly , S be the set of all distributions over A .
An s ∈ S is called a mixed strategy profile .
When the game is finite ( i.e. , A and I are both finite ) , the probability that a ∈ A is played under s is written s ( a ) = s ( ai , a_i ) .
When the distribution s is not correlated , we can simply say si ( ai ) when referring to the probability player i plays ai under s. Next , we define the payoff ( utility ) function of each player i by ui : A1 × · · · × Am -- + R , where ui ( ai , a_i ) indicates the payoff to player i to playing pure strategy ai when the remaining players play a_i .
We can extend this definition to mixed strategies by assuming that ui are von Neumann-Morgenstern ( vNM ) utilities as follows : ui ( s ) = Es [ ui ] , where Es is the expectation taken with respect to the probability distribution of play induced by the players ' mixed strategy s. 2By employing the normal form , we model agents as playing a single action , with decisions taken simultaneously .
This is appropriate for our current study , which treats strategies ( agent programs ) as atomic actions .
We could capture finer-grained decisions about action over time in the extensive form .
Although any extensive game can be recast in normal form , doing so may sacrifice compactness and blur relevant distinctions ( e.g. , subgame perfection ) .
Occasionally , we write ui ( x , y ) to mean that x ∈ Ai or Si and y ∈ A_i or S_i depending on context .
We also express the set of utility functions of all players as u ( · ) = { u1 ( · ) , ... , um ( · ) } .
We define a function , e : R -- + R , interpreted as the maximum benefit any player can obtain by deviating from its strategy in the specified profile .
where r belongs to some strategy set , R , of either pure or mixed strategies .
Faced with a game , an agent would ideally play its best strategy given those played by the other agents .
A configuration where all agents play strategies that are best responses to the others constitutes a Nash equilibrium .
When r ∈ A , the above defines a pure strategy Nash equilibrium ; otherwise the definition describes a mixed strategy Nash equilibrium .
We often appeal to the concept of an approximate , or e-Nash equilibrium , where e is the maximum benefit to any agent for deviating from the prescribed strategy .
Thus , e ( r ) as defined above ( 1 ) is such that profile r is an e-Nash equilibrium iff e ( r ) ≤ e .
In this study we devote particular attention to games that exhibit symmetry with respect to payoffs , rendering agents strategically identical .
3 .
THE MODEL
4 .
EMPIRICAL DESIGN ANALYSIS
4.1 TAC/SCM Design Problem
4.2 Estimating Nash Equilibria
4.2.1 PayoffFunction Approximation
4.2.2 Search in Strategy Profile Space
4.3 Data Generation
4.4 Results
4.4.1 Analysis of the Baseline Data Set
4.4.2 Analysis of Search Data
4.4.3 Extrapolating the Solution Correspondence
4.5 Probabilistic Analysis
5 .
CONVERGENCE RESULTS
6 .
RELATED WORK
The mechanism design literature in Economics has typically explored existence of a mechanism that implements a social choice function in equilibrium [ 10 ] .
Additionally , there is an extensive literature on optimal auction design [ 10 ] , of which the work by Roger Myerson [ 11 ] is , perhaps , the most relevant .
In much of this work , analytical results are presented with respect to specific utility functions and accounting for constraints such as incentive compatibility and individual rationality .
Several related approaches to search for the best mechanism exist in the Computer Science literature .
Conitzer and Sandholm [ 6 ] developed a search algorithm when all the relevant game parameters are common knowledge .
When payoff functions of players are unknown , a search using simulations has been explored as an alternative .
One approach in that direction , taken in [ 4 ] and [ 15 ] , is to co-evolve the mechanism parameter and agent strategies , using some notion of social utility and agent payoffs as fitness criteria .
An alternative to co-evolution explored in [ 16 ] was to optimize a well-defined welfare function of the designer using genetic programming .
In this work the authors used a common learning strategy for all agents and defined an outcome of a game induced by a mechanism parameter as the outcome of joint agent learning .
Most recently , Phelps et al. [ 14 ] compared two mechanisms based on expected social utility with expectation taken over an empirical distribution of equilibria in games defined by heuristic strategies , as in [ 18 ] .
7 .
CONCLUSION
Acknowledgments
8 .
REFERENCES
APPENDIX A. PROOFS A. 1 Proof of Proposition 1
A. 2 Proof of Proposition 2
A. 3 Proof of Theorem 3
A. 5 Proof of Theorem 6
A. 4 Proof of Lemma 5
A. 6 Proof of Theorem 7

Empirical Mechanism Design : Methods , with Application to a Supply-Chain Scenario
ABSTRACT
Our proposed methods employ learning and search techniques to estimate outcome features of interest as a function of mechanism parameter settings .
We illustrate our approach with a design task from a supply-chain trading competition .
Designers adopted several rule changes in order to deter particular procurement behavior , but the measures proved insufficient .
Our empirical mechanism analysis models the relation between a key design parameter and outcomes , confirming the observed behavior and indicating that no reasonable parameter settings would have been likely to achieve the desired effect .
More generally , we show that under certain conditions , the estimator of optimal mechanism parameter setting based on empirical data is consistent .
1 .
MOTIVATION
We illustrate our problem with an anecdote from a supply chain research exercise : the 2003 and 2004 Trading Agent Competition ( TAC ) Supply Chain Management ( SCM ) game .
TAC/SCM [ 1 ] defines a scenario where agents compete to maximize their profits as manufacturers in a supply chain .
As it happened , the specified negotiation behavior of suppliers provided a great incentive for agents to procure large quantities of components on day 0 : the very beginning of the simulation .
During the early rounds of the 2003 SCM competition , several agent developers discovered this , and the apparent success led to most agents performing the majority of their purchasing on day 0 .
Several participants noted that the predominance of day-0 procurement overshadowed other key research issues , such as factory scheduling [ 2 ] and optimizing bids for customer orders [ 13 ] .
After the 2003 tournament , there was a general consensus in the TAC community that the rules should be changed to deter large day-0 procurement .
The task facing game organizers can be viewed as a problem in mechanism design .
The designers have certain game features under their control , and a set of objectives regarding game outcomes .
Unlike most academic treatments of mechanism design , the objective is a behavioral feature ( moderate day-0 procurement ) rather than an allocation feature like economic efficiency , and the allowed mechanisms are restricted to those judged to require only an incremental modification of the current game .
Replacing the supplychain negotiation procedures with a one-shot direct mechanism , for example , was not an option .
We believe that such operational restrictions and idiosyncratic objectives are actually quite typical of practical mechanism design settings , where they are perhaps more commonly characterized as incentive engineering problems .
In response to the problem , the TAC/SCM designers adopted several rule changes intended to penalize large day-0 orders .
These included modifications to supplier pricing policies and introduction of storage costs assessed on inventories of components and finished goods .
Despite the changes , day-0 procurement was very high in the early rounds of the 2004 competition .
In a drastic measure , the GameMaster imposed a fivefold increase of storage costs midway through the tournament .
Even this did not stem the tide , and day-0 procurement in the final rounds actually increased ( by some measures ) from 2003 [ 9 ] .
The apparent difficulty in identifying rule modifications that effect moderation in day-0 procurement is quite striking .
Although the designs were widely discussed , predictions for the effects of various proposals were supported primarily by intuitive arguments or at best by back-of-the-envelope calculations .
proaches or tools could enable more systematic analysis of design options .
Standard game-theoretic and mechanism design methods are clearly relevant , although the lack of an analytic description of the game seems to be an impediment .
Under the assumption that the simulator itself is the only reliable source of outcome computation , we refer to our task as empirical mechanism design .
In the sequel , we develop some general methods for empirical mechanism design and apply them to the TAC/SCM redesign problem .
Our analysis focuses on the setting of storage costs ( taking other game modifications as fixed ) , since this is the most direct deterrent to early procurement adopted .
Our results confirm the basic intuition that incentives for day-0 purchasing decrease as storage costs rise .
We also confirm that the high day-0 procurement observed in the 2004 tournament is a rational response to the setting of storage costs used .
Finally , we conclude from our data that it is very unlikely that any reasonable setting of storage costs would result in acceptable levels of day-0 procurement , so a different design approach would have been required to eliminate this problem .
Overall , we contribute a formal framework and a set of methods for tackling indirect mechanism design problems in settings where only a black-box description of players ' utilities is available .
We believe that most realistic problems are too complex to be amenable to exact analysis .
Consequently , we advocate the approach of gathering evidence to provide indirect support of specific hypotheses .
2 .
PRELIMINARIES
Ri is the set of strategies available to player i ∈ I , with R = R1 × ... × Rm representing the set ofjoint strategies of all players .
We designate the set of pure strategies available to player i by Ai , and denote the joint set of pure strategies of all players by A = A1 × ... × Am .
It is often convenient to refer to a strategy of player i separately from that of the remaining players .
To accommodate this , we use a_i to denote the joint strategy of all players other than player i. Let Si be the set of all probability distributions ( mixtures ) over Ai and , similarly , S be the set of all distributions over A .
An s ∈ S is called a mixed strategy profile .
When the game is finite ( i.e. , A and I are both finite ) , the probability that a ∈ A is played under s is written s ( a ) = s ( ai , a_i ) .
This is appropriate for our current study , which treats strategies ( agent programs ) as atomic actions .
We could capture finer-grained decisions about action over time in the extensive form .
Although any extensive game can be recast in normal form , doing so may sacrifice compactness and blur relevant distinctions ( e.g. , subgame perfection ) .
We also express the set of utility functions of all players as u ( · ) = { u1 ( · ) , ... , um ( · ) } .
We define a function , e : R -- + R , interpreted as the maximum benefit any player can obtain by deviating from its strategy in the specified profile .
where r belongs to some strategy set , R , of either pure or mixed strategies .
Faced with a game , an agent would ideally play its best strategy given those played by the other agents .
A configuration where all agents play strategies that are best responses to the others constitutes a Nash equilibrium .
When r ∈ A , the above defines a pure strategy Nash equilibrium ; otherwise the definition describes a mixed strategy Nash equilibrium .
We often appeal to the concept of an approximate , or e-Nash equilibrium , where e is the maximum benefit to any agent for deviating from the prescribed strategy .
In this study we devote particular attention to games that exhibit symmetry with respect to payoffs , rendering agents strategically identical .
6 .
RELATED WORK
The mechanism design literature in Economics has typically explored existence of a mechanism that implements a social choice function in equilibrium [ 10 ] .
Additionally , there is an extensive literature on optimal auction design [ 10 ] , of which the work by Roger Myerson [ 11 ] is , perhaps , the most relevant .
In much of this work , analytical results are presented with respect to specific utility functions and accounting for constraints such as incentive compatibility and individual rationality .
Several related approaches to search for the best mechanism exist in the Computer Science literature .
Conitzer and Sandholm [ 6 ] developed a search algorithm when all the relevant game parameters are common knowledge .
When payoff functions of players are unknown , a search using simulations has been explored as an alternative .
One approach in that direction , taken in [ 4 ] and [ 15 ] , is to co-evolve the mechanism parameter and agent strategies , using some notion of social utility and agent payoffs as fitness criteria .
An alternative to co-evolution explored in [ 16 ] was to optimize a well-defined welfare function of the designer using genetic programming .
In this work the authors used a common learning strategy for all agents and defined an outcome of a game induced by a mechanism parameter as the outcome of joint agent learning .
Most recently , Phelps et al. [ 14 ] compared two mechanisms based on expected social utility with expectation taken over an empirical distribution of equilibria in games defined by heuristic strategies , as in [ 18 ] .

Empirical Mechanism Design : Methods , with Application to a Supply-Chain Scenario
ABSTRACT
Our proposed methods employ learning and search techniques to estimate outcome features of interest as a function of mechanism parameter settings .
We illustrate our approach with a design task from a supply-chain trading competition .
Designers adopted several rule changes in order to deter particular procurement behavior , but the measures proved insufficient .
Our empirical mechanism analysis models the relation between a key design parameter and outcomes , confirming the observed behavior and indicating that no reasonable parameter settings would have been likely to achieve the desired effect .
More generally , we show that under certain conditions , the estimator of optimal mechanism parameter setting based on empirical data is consistent .
1 .
MOTIVATION
We illustrate our problem with an anecdote from a supply chain research exercise : the 2003 and 2004 Trading Agent Competition ( TAC ) Supply Chain Management ( SCM ) game .
TAC/SCM [ 1 ] defines a scenario where agents compete to maximize their profits as manufacturers in a supply chain .
The agents procure components from the various suppliers and assemble finished goods for sale to customers , repeatedly over a simulated year .1 1Information about TAC and the SCM game , including specifications , rules , and competition results , can be found at http://www.sics.se/tac .
As it happened , the specified negotiation behavior of suppliers provided a great incentive for agents to procure large quantities of components on day 0 : the very beginning of the simulation .
During the early rounds of the 2003 SCM competition , several agent developers discovered this , and the apparent success led to most agents performing the majority of their purchasing on day 0 .
Although jockeying for day-0 procurement turned out to be an interesting strategic issue in itself [ 19 ] , the phenomenon detracted from other interesting problems , such as adapting production levels to varying demand ( since component costs were already sunk ) , and dynamic management of production , sales , and inventory .
Several participants noted that the predominance of day-0 procurement overshadowed other key research issues , such as factory scheduling [ 2 ] and optimizing bids for customer orders [ 13 ] .
After the 2003 tournament , there was a general consensus in the TAC community that the rules should be changed to deter large day-0 procurement .
The task facing game organizers can be viewed as a problem in mechanism design .
The designers have certain game features under their control , and a set of objectives regarding game outcomes .
Unlike most academic treatments of mechanism design , the objective is a behavioral feature ( moderate day-0 procurement ) rather than an allocation feature like economic efficiency , and the allowed mechanisms are restricted to those judged to require only an incremental modification of the current game .
Replacing the supplychain negotiation procedures with a one-shot direct mechanism , for example , was not an option .
We believe that such operational restrictions and idiosyncratic objectives are actually quite typical of practical mechanism design settings , where they are perhaps more commonly characterized as incentive engineering problems .
In response to the problem , the TAC/SCM designers adopted several rule changes intended to penalize large day-0 orders .
These included modifications to supplier pricing policies and introduction of storage costs assessed on inventories of components and finished goods .
Despite the changes , day-0 procurement was very high in the early rounds of the 2004 competition .
In a drastic measure , the GameMaster imposed a fivefold increase of storage costs midway through the tournament .
Even this did not stem the tide , and day-0 procurement in the final rounds actually increased ( by some measures ) from 2003 [ 9 ] .
The apparent difficulty in identifying rule modifications that effect moderation in day-0 procurement is quite striking .
Although the designs were widely discussed , predictions for the effects of various proposals were supported primarily by intuitive arguments or at best by back-of-the-envelope calculations .
Much of the difficulty , of course , is anticipating the agents ' ( and their developers ' ) responses without essentially running a gaming exercise for this purpose .
The episode caused us to consider whether new ap
proaches or tools could enable more systematic analysis of design options .
Standard game-theoretic and mechanism design methods are clearly relevant , although the lack of an analytic description of the game seems to be an impediment .
Under the assumption that the simulator itself is the only reliable source of outcome computation , we refer to our task as empirical mechanism design .
In the sequel , we develop some general methods for empirical mechanism design and apply them to the TAC/SCM redesign problem .
Our analysis focuses on the setting of storage costs ( taking other game modifications as fixed ) , since this is the most direct deterrent to early procurement adopted .
Our results confirm the basic intuition that incentives for day-0 purchasing decrease as storage costs rise .
We also confirm that the high day-0 procurement observed in the 2004 tournament is a rational response to the setting of storage costs used .
Finally , we conclude from our data that it is very unlikely that any reasonable setting of storage costs would result in acceptable levels of day-0 procurement , so a different design approach would have been required to eliminate this problem .
Overall , we contribute a formal framework and a set of methods for tackling indirect mechanism design problems in settings where only a black-box description of players ' utilities is available .
Our methods incorporate estimation of sets of Nash equilibria and sample Nash equilibria , used in conjuction to support general claims about the structure of the mechanism designer 's utility , as well as a restricted probabilistic analysis to assess the likelihood of conclusions .
We believe that most realistic problems are too complex to be amenable to exact analysis .
Consequently , we advocate the approach of gathering evidence to provide indirect support of specific hypotheses .
2 .
PRELIMINARIES
A normalform game2 is denoted by [ I , { Ri } , { ui ( r ) } ] , where I refers to the set of players and m = | I | is the number of players .
Ri is the set of strategies available to player i ∈ I , with R = R1 × ... × Rm representing the set ofjoint strategies of all players .
We designate the set of pure strategies available to player i by Ai , and denote the joint set of pure strategies of all players by A = A1 × ... × Am .
It is often convenient to refer to a strategy of player i separately from that of the remaining players .
To accommodate this , we use a_i to denote the joint strategy of all players other than player i. Let Si be the set of all probability distributions ( mixtures ) over Ai and , similarly , S be the set of all distributions over A .
An s ∈ S is called a mixed strategy profile .
When the game is finite ( i.e. , A and I are both finite ) , the probability that a ∈ A is played under s is written s ( a ) = s ( ai , a_i ) .
When the distribution s is not correlated , we can simply say si ( ai ) when referring to the probability player i plays ai under s. Next , we define the payoff ( utility ) function of each player i by ui : A1 × · · · × Am -- + R , where ui ( ai , a_i ) indicates the payoff to player i to playing pure strategy ai when the remaining players play a_i .
We can extend this definition to mixed strategies by assuming that ui are von Neumann-Morgenstern ( vNM ) utilities as follows : ui ( s ) = Es [ ui ] , where Es is the expectation taken with respect to the probability distribution of play induced by the players ' mixed strategy s. 2By employing the normal form , we model agents as playing a single action , with decisions taken simultaneously .
This is appropriate for our current study , which treats strategies ( agent programs ) as atomic actions .
We could capture finer-grained decisions about action over time in the extensive form .
Although any extensive game can be recast in normal form , doing so may sacrifice compactness and blur relevant distinctions ( e.g. , subgame perfection ) .
Occasionally , we write ui ( x , y ) to mean that x ∈ Ai or Si and y ∈ A_i or S_i depending on context .
We also express the set of utility functions of all players as u ( · ) = { u1 ( · ) , ... , um ( · ) } .
We define a function , e : R -- + R , interpreted as the maximum benefit any player can obtain by deviating from its strategy in the specified profile .
where r belongs to some strategy set , R , of either pure or mixed strategies .
Faced with a game , an agent would ideally play its best strategy given those played by the other agents .
A configuration where all agents play strategies that are best responses to the others constitutes a Nash equilibrium .
When r ∈ A , the above defines a pure strategy Nash equilibrium ; otherwise the definition describes a mixed strategy Nash equilibrium .
We often appeal to the concept of an approximate , or e-Nash equilibrium , where e is the maximum benefit to any agent for deviating from the prescribed strategy .
Thus , e ( r ) as defined above ( 1 ) is such that profile r is an e-Nash equilibrium iff e ( r ) ≤ e .
In this study we devote particular attention to games that exhibit symmetry with respect to payoffs , rendering agents strategically identical .
3 .
THE MODEL
We model the strategic interactions between the designer of the mechanism and its participants as a two-stage game .
The designer moves first by selecting a value , 0 , from a set of allowable mechanism settings , Θ .
All the participant agents observe the mechanism parameter 0 and move simultaneously thereafter .
For example , the designer could be deciding between a first-price and second-price sealed-bid auction mechanisms , with the presumption that after the choice has been made , the bidders will participate with full awareness of the auction rules .
Since the participants play with full knowledge of the mechanism parameter , we define a game between them in the second stage as Γθ = [ I , { Ri } , { ui ( r , 0 ) } ] .
We refer to Γθ as a game induced by 0 .
Let N ( 0 ) be the set of strategy profiles considered solutions of the game Γθ .3 Suppose that the goal of the designer is to optimize the value of some welfare function , W ( r , 0 ) , dependent on the mechanism parameter and resulting play , r .
We define a pessimistic measure , W ( ˆR , 0 ) = inf { W ( r , 0 ) : r ∈ ˆR } , representing the worst-case welfare of the game induced by 0 , assuming that agents play some joint strategy in ˆR .
Typically we care about W ( N ( 0 ) , 0 ) , the worst-case outcome of playing some solution .4 On some problems we can gain considerable advantage by using an aggregation function to map the welfare outcome of a game 3We generally adopt Nash equilibrium as the solution concept , and thus take N ( 0 ) to be the set of equilibria .
However , much of the methodology developed here could be employed with alternative criteria for deriving agent behavior from a game definition .
4Again , alternatives are available .
For example , if one has a probability distribution over the solution set N ( 0 ) , it would be natural to take the expectation of W ( r , 0 ) instead .
specified in terms of agent strategies to an equivalent welfare outcome specified in terms of a lower-dimensional summary .
We overload the function symbol to apply to sets of strategy profiles : φ ( ˆR ) = { φ ( r ) : r ∈ ˆR } .
For convenience of exposition , we write φ ∗ ( θ ) to mean φ ( N ( θ ) ) .
Using an aggregation function yields a more compact representation of strategy profiles .
For example , suppose -- as in our application below -- that an agent 's strategy is defined by a numeric parameter .
If all we care about is the total value played , we may take φ ( a ) = Pmi = 1 ai .
If we have chosen our aggregator carefully , we may also capture structure not obvious otherwise .
For example , φ ∗ ( θ ) could be decreasing in θ , whereas N ( θ ) might have a more complex structure .
Given a description of the solution correspondence N ( θ ) ( equivalently , φ ∗ ( θ ) ) , the designer faces a standard optimization problem .
Alternatively , given a simulator that could produce an unbiased sample from the distribution of W ( N ( θ ) , θ ) for any θ , the designer would be faced with another much appreciated problem in the literature : simulation optimization [ 12 ] .
However , even for a game Γθ with known payoffs it may be computationally intractable to solve for Nash equilibria , particularly if the game has large or infinite strategy sets .
Additionally , we wish to study games where the payoffs are not explicitly given , but must be determined from simulation or other experience with the game .5 Accordingly , we assume that we are given a ( possibly noisy ) data set of payoff realizations : Do = { ( θ1 , a1 , U1 ) , ... , ( θk , ak , Uk ) } , where for every data point θi is the observed mechanism parameter setting , ai is the observed pure strategy profile of the participants , and Ui is the corresponding realization of agent payoffs .
We may also have additional data generated by a ( possibly noisy ) simulator : Ds = { ( θk +1 , ak +1 , Uk +1 ) , ... , ( θk + l , ak + l , Uk + l ) } .
Let D = { Do , Ds } be the combined data set .
( Either Do or Ds may be null for a particular problem . )
In the remainder of this paper , we apply our modeling approach , together with several empirical game-theoretic methods , in order to answer questions regarding the design of the TAC/SCM scenario .
4 .
EMPIRICAL DESIGN ANALYSIS
Since our data comes in the form of payoff experience and not as the value of an objective function for given settings of the control variable , we can no longer rely on the methods for optimizing functions using simulations .
Indeed , a fundamental aspect of our design problem involves estimating the Nash equilibrium correspondence .
Furthermore , we can not rely directly on the convergence results that abound in the simulation optimization literature , and must establish probabilistic analysis methods tailored for our problem setting .
4.1 TAC/SCM Design Problem
We describe our empirical design analysis methods by presenting a detailed application to the TAC/SCM scenario introduced above .
Recall that during the 2004 tournament , the designers of the supplychain game chose to dramatically increase storage costs as a measure aimed at curbing day-0 procurement , to little avail .
Here we systematically explore the relationship between storage costs and 5This is often the case for real games of interest , where natural language or algorithmic descriptions may substitute for a formal specification of strategy and payoff functions .
the aggregate quantity of components procured on day 0 in equilibrium .
In doing so , we consider several questions raised during and after the tournament .
First , does increasing storage costs actually reduce day-0 procurement ?
Second , was the excessive day-0 procurement that was observed during the 2004 tournament rational ?
And third , could increasing storage costs sufficiently have reduced day-0 procurement to an `` acceptable '' level , and if so , what should the setting of storage costs have been ?
It is this third question that defines the mechanism design aspect of our analysis .6 To apply our methods , we must specify the agent strategy sets , the designer 's welfare function , the mechanism parameter space , and the source of data .
We restrict the agent strategies to be a multiplier on the quantity of the day-0 requests by one of the finalists , Deep Maize , in the 2004 TAC/SCM tournament .
We further restrict it to the set [ 0,1.5 ] , since any strategy below 0 is illegal and strategies above 1.5 are extremely aggressive ( thus unlikely to provide refuting deviations beyond those available from included strategies , and certainly not part of any desirable equilibrium ) .
All other behavior is based on the behavior of Deep Maize and is identical for all agents .
This choice can provide only an estimate of the actual tournament behavior of a `` typical '' agent .
However , we believe that the general form of the results should be robust to changes in the full agent behavior .
sum of day-0 purchases .
Let φ ( a ) = P6 We model the designer 's welfare function as a threshold on the i = 1 ai be the aggregation function representing the sum of day-0 procurement of the six agents participating in a particular supply-chain game ( for mixed strategy profiles s , we take expectation of φ with respect to the mixture ) .
The designer 's welfare function W ( N ( θ ) , θ ) is then given by I { sup { φ ∗ ( θ ) } ≤ α } , where α is the maximum acceptable level of day-0 procurement and I is the indicator function .
The designer selects a value θ of storage costs , expressed as an annual percentage of the baseline value of components in the inventory ( charged daily ) , from the set Θ = R + .
Since the designer 's decision depends only on φ ∗ ( θ ) , we present all of our results in terms of the value of the aggregation function .
4.2 Estimating Nash Equilibria
The objective of TAC/SCM agents is to maximize profits realized over a game instance .
Thus , if we fix a strategy for each agent at the beginning of the simulation and record the corresponding profits at the end , we will have obtained a data point in the form ( a , U ( a ) ) .
If we also have fixed the parameter θ of the simulator , the resulting data point becomes part of our data set D .
This data set , then , contains data only in the form of pure strategies of players and their corresponding payoffs , and , consequently , in order to formulate the designer 's problem as optimization , we must first determine or approximate the set of Nash equilibria of each game Γθ .
Thus , we need methods for approximating Nash equilibria for infinite games .
Below , we describe the two methods we used in our study .
The first has been explored empirically before , whereas the second is introduced here as the method specifically designed to approximate a set of Nash equilibria .
4.2.1 PayoffFunction Approximation
The first method for estimating Nash equilibria based on data uses supervised learning to approximate payoff functions of mech6We do not address whether and how other measures ( e.g. , constraining procurement directly ) could have achieved design objectives .
Our approach takes as given some set of design options , in this case defined by the storage cost parameter .
In principle our methods could be applied to a different or larger design space , though with corresponding complexity growth .
anism participants from a data set of game experience [ 17 ] .
Once approximate payoff functions are available for all players , the Nash equilibria may be either found analytically or approximated using numerical techniques , depending on the learning model .
In what follows , we estimate only a sample Nash equilibrium using this technique , although this restriction can be removed at the expense of additional computation time .
One advantage of this method is that it can be applied to any data set and does not require the use of a simulator .
Thus , we can apply it when Ds = ∅ .
If a simulator is available , we can generate additional data to build confidence in our initial estimates .7 We tried the following methods for approximating payoff functions : quadratic regression ( QR ) , locally weighted average ( LWA ) , and locally weighted linear regression ( LWLR ) .
We also used control variates to reduce the variance of payoff estimates , as in our previous empirical game-theoretic analysis of TAC/SCM -03 [ 19 ] .
The quadratic regression model makes it possible to compute equilibria of the learned game analytically .
For the other methods we applied replicator dynamics [ 7 ] to a discrete approximation of the learned game .
The expected total day-0 procurement in equilibrium was taken as the estimate of an outcome .
4.2.2 Search in Strategy Profile Space
When we have access to a simulator , we can also use directed search through profile space to estimate the set of Nash equilibria , which we describe here after presenting some additional notation .
defined as maxa ' ∈ Snb ( a , D ) max { ui ( a , a , ) ( a ) − ui ( a , al ) ( a ) , 0 } .
We say that a is a candidate δ-equilibrium for δ ≥ ˆ ~ .
When Snb ( a , ˜D ) = ∅ ( i.e. , all strategic neighbors are represented in the data ) , a is confirmed as an ˆ ~ - Nash equilibrium .
Our search method operates by exploring deviations from candidate equilibria .
We refer to it as `` BestFirstSearch '' , as it selects with probability one a strategy profile a ' ∈ Snb ( a , ˜D ) that has the smallest ~ ˆ in D. Finally we define an estimator for a set of Nash equilibria .
DEFINITION 6 .
For a set K , define Co ( K ) to be the convex hull of K. Let Bδ be the set of candidates at level δ .
We define ˆφ ∗ ( θ ) = Co ( { φ ( a ) : a ∈ Bδ } ) for a fixed δ to be an estimator of φ ∗ ( θ ) .
In words , the estimate of a set of equilibrium outcomes is the convex hull of all aggregated strategy profiles with ~ - bound below some fixed δ .
This definition allows us to exploit structure arising from the aggregation function .
If two profiles are close in terms of aggregation values , they may be likely to have similar ~ - bounds .
In particular , if one is an equilibrium , the other may be as well .
We present some theoretical support for this method of estimating the set of Nash equilibria below .
Since the game we are interested in is infinite , it is necessary to terminate BestFirstSearch before exploring the entire space of strat7For example , we can use active learning techniques [ 5 ] to improve the quality of payoff function approximation .
In this work , we instead concentrate on search in strategy profile space .
egy profiles .
We currently determine termination time in a somewhat ad hoc manner , based on observations about the current set of candidate equilibria .8
4.3 Data Generation
Our data was collected by simulating TAC/SCM games on a local version of the 2004 TAC/SCM server , which has a configuration setting for the storage cost .
Agent strategies in simulated games were selected from the set { 0 , 0.3 , 0.6 , ... , 1.5 } in order to have positive probability of generating strategic neighbors .9 A baseline data set Do was generated by sampling 10 randomly generated strategy profiles for each θ ∈ { 0 , 50 , 100 , 150 , 200 } .
Between 5 and 10 games were run for each profile after discarding games that had various flaws .10 We used search to generate a simulated data set Ds , performing between 12 and 32 iterations of BestFirstSearch for each of the above settings of θ .
Since simulation cost is extremely high ( a game takes nearly 1 hour to run ) , we were able to run a total of 2670 games over the span of more than six months .
For comparison , to get the entire description of an empirical game defined by the restricted finite joint strategy space for each value of θ ∈ { 0 , 50 , 100 , 150 , 200 } would have required at least 23100 games ( sampling each profile 10 times ) .
4.4 Results
4.4.1 Analysis of the Baseline Data Set
We applied the three learning methods described above to the baseline data set Do .
Additionally , we generated an estimate of the Nash equilibrium correspondence , ˆφ ∗ ( θ ) , by applying Definition 6 with δ = 2.5 E6 .
The results are shown in Figure 1 .
As we can see , the correspondence ˆφ ∗ ( θ ) has little predictive power based on Do , and reveals no interesting structure about the game .
In contrast , all three learning methods suggest that total day-0 procurement is a decreasing function of storage costs .
Figure 1 : Aggregate day-0 procurement estimates based on Do .
The correspondence ˆφ ∗ ( θ ) is the interval between `` BaselineMin '' and `` BaselineMax '' .
8Generally , search is terminated once the set of candidate equilibria is small enough to draw useful conclusions about the likely range of equilibrium strategies in the game .
9Of course , we do not restrict our Nash equilibrium estimates to stay in this discrete subset of [ 0,1.5 ] .
10For example , if we detected that any agent failed during the game ( failures included crashes , network connectivity problems , and other obvious anomalies ) , the game would be thrown out .
4.4.2 Analysis of Search Data
To corroborate the initial evidence from the learning methods , we estimated ˆφ ∗ ( 0 ) ( again , using S = 2.5 E6 ) on the data set D = { Do , Ds } , where Ds is data generated through the application of BestFirstSearch .
The results of this estimate are plotted against the results of the learning methods trained on Do11 in Figure 2 .
First , we note that the addition of the search data narrows the range of potential equilibria substantially .
Furthermore , the actual point predictions of the learning methods and those based on e-bounds after search are reasonably close .
Combining the evidence gathered from these two very different approaches to estimating the outcome correspondence yields a much more compelling picture of the relationship between storage costs and day-0 procurement than either method used in isolation .
Figure 2 : Aggregate day-0 procurement estimates based on search in strategy profile space compared to function approx
ˆ imation techniques trained on Do .
The correspondence φ ∗ ( 0 ) for D = { Do , Ds } is the interval between `` SearchMin '' and `` SearchMax '' .
This evidence supports the initial intuition that day-0 procurement should be decreasing with storage costs .
It also confirms that high levels of day-0 procurement are a rational response to the 2004 tournament setting of average storage cost , which corresponds to 0 = 100 .
The minimum prediction for aggregate procurement at this level of storage costs given by any experimental methods is approximately 3 .
This is quite high , as it corresponds to an expected commitment of 1/3 of the total supplier capacity for the entire game .
The maximum prediction is considerably higher at 4.5 .
In the actual 2004 competition , aggregate day-0 procurement was equivalent to 5.71 on the scale used here [ 9 ] .
Our predictions underestimate this outcome to some degree , but show that any rational outcome was likely to have high day-0 procurement .
4.4.3 Extrapolating the Solution Correspondence
We have reasonably strong evidence that the outcome correspondence is decreasing .
However , the ultimate goal is to be able to either set the storage cost parameter to a value that would curb day-0 procurement in equilibrium or conclude that this is not possible .
To answer this question directly , suppose that we set a conservative threshold α = 2 on aggregate day-0 procurement .12 Linear 11It is unclear how meaningful the results of learning would be if Ds were added to the training data set .
Indeed , the additional data may actually increase the learning variance .
12Recall that designer 's objective is to incentivize aggergate day-0 procurement that is below the threshold α .
Our threshold here still represents a commitment of over 20 % of the suppliers ' capacity for extrapolation of the maximum of the outcome correspondence estimated from D yields 0 = 320 .
The data for 0 = 320 were collected in the same way as for other storage cost settings , with 10 randomly generated profiles followed by 33 iterations of BestFirstSearch .
Figure 3 shows the detailed e-bounds for all profiles in terms of their corresponding values of
Figure 3 : Values of eˆ for profiles explored using search when
0 = 320 .
Strategy profiles explored are presented in terms of the corresponding values of φ ( a ) .
The gray region corresponds to ˆφ ∗ ( 320 ) with S = 2.5 M .
The estimated set of aggregate day-0 outcomes is very close to that for 0 = 200 , indicating that there is little additional benefit to raising storage costs above 200 .
Observe , that even the lower bound of our estimated set of Nash equilibria is well above the target day-0 procurement of 2 .
Furthermore , payoffs to agents are almost always negative at 0 = 320 .
Consequently , increasing the costs further would be undesirable even if day-0 procurement could eventually be curbed .
Since we are reasonably confident that φ ∗ ( 0 ) is decreasing in 0 , we also do not expect that setting 0 somewhere between 200 and 320 will achieve the desired result .
We conclude that it is unlikely that day-0 procurement could ever be reduced to a desirable level using any reasonable setting of the storage cost parameter .
That our predictions tend to underestimate tournament outcomes reinforces this conclusion .
To achieve the desired reduction in day-0 procurement requires redesigning other aspects of the mechanism .
4.5 Probabilistic Analysis
Our empirical analysis has produced evidence in support of the conclusion that no reasonable setting of storage cost was likely to sufficiently curb excessive day-0 procurement in TAC/SCM ' 04 .
All of this evidence has been in the form of simple interpolation and extrapolation of estimates of the Nash equilibrium correspondence .
These estimates are based on simulating game instances , and are subject to sampling noise contributed by the various stochastic elements of the game .
In this section , we develop and apply methods for evaluating the sensitivity of our e-bound calculations to such stochastic effects .
Suppose that all agents have finite ( and small ) pure strategy sets , A. Thus , it is feasible to sample the entire payoff matrix of the game .
Additionally , suppose that noise is additive with zero-mean the entire game on average , so in practice we would probably want the threshold to be even lower .
and finite variance , that is , Ui ( a ) = ui ( a ) + ˜ξi ( a ) , where Ui ( a ) is the observed payoff to i when a was played , ui ( a ) is the actual corresponding payoff , and ˜ξi ( a ) is a mean-zero normal random variable .
We designate the known variance of˜ξi ( a ) by σ2i ( a ) .
Thus , we assume that ˜ξi ( a ) is normal with distribution N ( 0 , σ2i ( a ) ) .
We take ¯ ui ( a ) to be the sample mean over all Ui ( a ) in D , and follow Chang and Huang [ 3 ] to assume that we have an improper prior over the actual payoffs ui ( a ) and sampling was independent for all i and a .
We also rely on their result that ui ( a ) | ¯ ui ( a ) =
¯ ui ( a ) − Zi ( a ) / [ σi ( a ) / ni ( a ) ] are independent with posterior distributions N ( ¯ ui ( a ) , σ2i ( a ) / ni ( a ) ) , where ni ( a ) is the number of samples taken of payoffs to i for pure profile a , and Zi ( a ) ∼ N ( 0 , 1 ) .
We now derive a generic probabilistic bound that a profile a ∈ A is an ~ - Nash equilibrium .
If ui ( · ) | ¯ ui ( · ) are independent for all i ∈ I and a ∈ A , we have the following result ( from this point on we omit conditioning on ¯ ui ( · ) for brevity ) :
where fui ( a ) ( u ) is the pdf of N ( ¯ ui ( a ) , σi ( a ) ) .
The proofs of this and all subsequent results are in the Appendix .
The posterior distribution of the optimum mean of n samples , derived by Chang and Huang [ 3 ] , is
where a ∈ A and Φ ( · ) is the N ( 0 , 1 ) distribution function .
Combining the results ( 2 ) and ( 3 ) , we obtain a probabilistic confidence bound that ~ ( a ) ≤ γ for a given γ .
Now , we consider cases of incomplete data and use the results we have just obtained to construct an upper bound ( restricted to profiles represented in data ) on the distribution of sup { φ ∗ ( θ ) } and inf { φ ∗ ( θ ) } ( assuming that both are attainable ) :
Table 2 : Upper bounds on the distribution of inf { φ ∗ ( θ ) } re
stricted to D for θ ∈ { 150 , 200 , 320 } when N ( θ ) is a set of Nash equilibria .
Tables 1 and 2 suggest that the existence of any equilibrium with φ ( a ) < 2.7 is unlikely for any θ that we have data for , although this judgment , as we mentioned , is only with respect to the profiles we have actually sampled .
We can then accept this as another piece of evidence that the designer could not find a suitable setting of θ to achieve his objectives -- indeed , the designer seems unlikely to achieve his objective even if he could persuade participants to play a desirable equilibrium !
Table 1 also provides additional evidence that the agents in the 2004 TAC/SCM tournament were indeed rational in procuring large numbers of components at the beginning fo the game .
If we look at the third column of this table , which corresponds to θ = 100 , we can gather that no profile a in our data with φ ( a ) < 3 is very likely to be played in equilibrium .
The bounds above provide some general evidence , but ultimately we are interested in a concrete probabilistic assessment of our conclusion with respect to the data we have sampled .
Particularly , we would like to say something about what happens for the settings of θ for which we have no data .
To derive an approximate probabilistic bound on the probability that no θ ∈ Θ could have achieved the designer 's objective , let ∪ Jj = 1Θj , be a partition of Θ , and assume that the function sup { φ ∗ ( θ ) } satisfies the Lipschitz condition with Lipschitz constant Aj on each subset Θj .13 Since we have determined that raising the storage cost above 320 is undesirable due to secondary considerations , we restrict attention to Θ = [ 0 , 320 ] .
We now define each subset j to be the interval between two points for which we have produced data .
Thus ,
with j running between 1 and 5 , corresponding to subintervals above .
We will further denote each Θj by ( aj , bj ] .14 Then , the following Proposition gives us an approximate upper bound15 on the probability that sup { φ ∗ ( θ ) } ≤ α .
where x is a real number and ≤ D indicates that the upper bound accounts only for strategies that appear in the data set D .
Since the events { ∃ a ∈ D : φ ( a ) ≤ x ∧ a ∈ N ( θ ) } and { inf { φ ∗ ( θ ) } ≤ x } are equivalent , this also defines an upper bound on the probability of { inf { φ ∗ ( θ ) } ≤ x } .
The values thus derived comprise the Tables 1 and 2 .
Table 1 : Upper bounds on the distribution of inf { φ ∗ ( θ ) } re
stricted to D for θ ∈ { 0 , 50 , 100 } when N ( θ ) is a set of Nash equilibria .
where cj = 2α + Aj ( bj − aj ) and ≤ D indicates that the upper bound only accounts for strategies that appear in the data set D.
Due to the fact that our bounds are approximate , we can not use them as a conclusive probabilistic assessment .
Instead , we take this as another piece of evidence to complement our findings .
Even if we can assume that a function that we approximate from data is Lipschitz continuous , we rarely actually know the Lipschitz constant for any subset of Θ .
Thus , we are faced with a task of estimating it from data .
Here , we tried three methods of doing this .
The first one simply takes the highest slope that the function attains within the available data and uses this constant value for every subinterval .
This produces the most conservative bound , and in many situations it is unlikely to be informative .
An alternative method is to take an upper bound on slope obtained within each subinterval using the available data .
This produces a much less conservative upper bound on probabilities .
However , since the actual upper bound is generally greater for each subinterval , the resulting probabilistic bound may be deceiving .
A final method that we tried is a compromise between the two above .
Instead of taking the conservative upper bound based on data over the entire function domain Θ , we take the average of upper bounds obtained at each Θj .
The bound at an interval is then taken to be the maximum of the upper bound for this interval and the average upper bound for all intervals .
The results of evaluating the expression for
Table 3 : Approximate upper bound on probability that some
setting of θ ∈ [ 0 , 320 ] will satisfy the designer objective with target α = 2 .
Different methods of approximating the upper bound on slope in each subinterval j are used .
this work , the expression gives an upper bound on the probability that some setting of θ ( i.e. , storage cost ) in the interval [ 0,320 ] will result in total day-0 procurement that is no greater in any equilibrium than the target specified by α and taken here to be 2 .
As we had suspected , the most conservative approach to estimating the upper bound on slope , presented in the first column of the table , provides us little information here .
However , the other two estimation approaches , found in columns two and three of Table 3 , suggest that we are indeed quite confident that no reasonable setting of θ ∈ [ 0 , 320 ] would have done the job .
Given the tremendous difficulty of the problem , this result is very strong .16 Still , we must be very cautious in drawing too heroic a conclusion based on this evidence .
Certainly , we have not `` checked '' all the profiles but only a small proportion of them ( infinitesimal , if we consider the entire continuous domain of θ and strategy sets ) .
Nor can we expect ever to obtain enough evidence to make completely objective conclusions .
Instead , the approach we advocate here is to collect as much evidence as is feasible given resource constraints , and make the most compelling judgment based on this evidence , if at all possible .
5 .
CONVERGENCE RESULTS
At this point , we explore abstractly whether a design parameter choice based on payoff data can be asymptotically reliable .
16Since we did not have all the possible deviations for any profile available in the data , the true upper bounds may be even lower .
As a matter of convenience , we will use notation un , i ( a ) to refer to a payoff function of player i based on an average over n i.i.d. samples from the distribution of payoffs .
We also assume that un , i ( a ) are independent for all a ∈ A and i ∈ I .
We will use the notation Γn to refer to the game [ I , R , { ui , n ( · ) } ] , whereas Γ will denote the `` underlying '' game , [ I , R , { ui ( · ) } ] .
Similarly , we define ~ n ( r ) to be ~ ( r ) with respect to the game Γn .
In this section , we show that ~ n ( s ) → ~ ( s ) a.s. uniformly on the mixed strategy space for any finite game , and , furthermore , that all mixed strategy Nash equilibria in empirical games eventually become arbitrarily close to some Nash equilibrium strategies in the underlying game .
We use these results to show that under certain conditions , the optimal choice of the design parameter based on empirical data converges almost surely to the actual optimum .
PROOF .
Since ~ ( s ) = 0 for every s ∈ N , we can find M large enough such that Pr { supn ≥ M sups ∈ N ~ n ( s ) < γ } = 1 .
By the Corollary , for any game with a finite set of pure strategies and for any ~ > 0 , all Nash equilibria lie in the set of empirical ~ - Nash equilibria if enough samples have been taken .
As we now show , this provides some justification for our use of a set of profiles with a non-zero ~ - bound as an estimate of the set of Nash equilibria .
First , suppose we conclude that for a particular setting of θ , sup { ˆφ ∗ ( θ ) } ≤ α .
Then , since for any fixed ~ > 0 , N ( θ ) ⊂ Nn , ~ ( θ ) when n is large enough ,
for any such n. Thus , since we defined the welfare function of the designer to be I { sup { φ ∗ ( θ ) } ≤ α } in our domain of interest , the empirical choice of θ satisfies the designer 's objective , thereby maximizing his welfare function .
Alternatively , suppose we conclude that inf { ˆφ ∗ ( θ ) } > α for ev
for every θ , and we can conclude that no setting of θ will satisfy the designer 's objective .
Now , we will show that when the number of samples is large enough , every Nash equilibrium of Γn is close to some Nash equilibrium of the underlying game .
This result will lead us to consider convergence of optimizers based on empirical data to actual optimal mechanism parameter settings .
We first note that the function ~ ( s ) is continuous in a finite game .
LEMMA 5 .
Let S be a mixed strategy set defined on a finite game .
Then ~ : S → R is continuous .
For the exposition that follows , we need a bit of additional notation .
First , let ( Z , d ) be a metric space , and X , Y C Z and define directed Hausdorffdistance from X to Y to be
Observe that U C X = : > .
h ( U , Y ) < h ( X , Y ) .
Further , define BS ( x , δ ) to be an open ball in S C Z with center x E S and radius δ .
Now , let Nn denote all Nash equilibria of the game Γn and let
that is , the union of open balls of radius δ with centers at Nash equilibria of Γ .
Note that h ( Nδ , N ) = δ .
We can then prove the following general result .
We will now show that in the special case when Θ and A are finite and each Γθ has a unique Nash equilibrium , the estimates 0ˆ of optimal designer parameter converge to an actual optimizer almost surely .
Let 0ˆ = arg maxθ ∈ Θ W ( Nn ( 0 ) , 0 ) , where n is the number of times each pure profile was sampled in Γθ for every 0 , and let 0 ∗ = arg maxθ ∈ Θ W ( N ( 0 ) , 0 ) .
The shortcoming of the above result is that , within our framework , the designer has no way of knowing or ensuring that Γθ do , indeed , have unique equilibria .
However , it does lend some theoretical justification for pursuing design in this manner , and , perhaps , will serve as a guide for more general results in the future .
6 .
RELATED WORK
The mechanism design literature in Economics has typically explored existence of a mechanism that implements a social choice function in equilibrium [ 10 ] .
Additionally , there is an extensive literature on optimal auction design [ 10 ] , of which the work by Roger Myerson [ 11 ] is , perhaps , the most relevant .
In much of this work , analytical results are presented with respect to specific utility functions and accounting for constraints such as incentive compatibility and individual rationality .
Several related approaches to search for the best mechanism exist in the Computer Science literature .
Conitzer and Sandholm [ 6 ] developed a search algorithm when all the relevant game parameters are common knowledge .
When payoff functions of players are unknown , a search using simulations has been explored as an alternative .
One approach in that direction , taken in [ 4 ] and [ 15 ] , is to co-evolve the mechanism parameter and agent strategies , using some notion of social utility and agent payoffs as fitness criteria .
An alternative to co-evolution explored in [ 16 ] was to optimize a well-defined welfare function of the designer using genetic programming .
In this work the authors used a common learning strategy for all agents and defined an outcome of a game induced by a mechanism parameter as the outcome of joint agent learning .
Most recently , Phelps et al. [ 14 ] compared two mechanisms based on expected social utility with expectation taken over an empirical distribution of equilibria in games defined by heuristic strategies , as in [ 18 ] .
7 .
CONCLUSION
In this work we spent considerable effort developing general tactics for empirical mechanism design .
We defined a formal gametheoretic model of interaction between the designer and the participants of the mechanism as a two-stage game .
We also described in some generality the methods for estimating a sample Nash equilibrium function when the data is extremely scarce , or a Nash equilibrium correspondence when more data is available .
Our techniques are designed specifically to deal with problems in which both the mechanism parameter space and the agent strategy sets are infinite and only a relatively small data set can be acquired .
A difficult design issue in the TAC/SCM game which the TAC community has been eager to address provides us with a setting to test our methods .
In applying empirical game analysis to the problem at hand , we are fully aware that our results are inherently inexact .
Thus , we concentrate on collecting evidence about the structure of the Nash equilibrium correspondence .
In the end , we can try to provide enough evidence to either prescribe a parameter setting , or suggest that no setting is possible that will satisfy the designer .
In the case of TAC/SCM , our evidence suggests quite strongly that storage cost could not have been effectively adjusted in the 2004 tournament to curb excessive day-0 procurement without detrimental effects on overall profitability .
The success of our analysis in this extremely complex environment with high simulation costs makes us optimistic that our methods can provide guidance in making mechanism design decisions in other challenging domains .
The theoretical results confirm some intuitions behind the empirical mechanism design methods we have introduced , and increases our confidence that our framework can be effective in estimating the best mechanism parameter choice in relatively general settings .
Acknowledgments
We thank Terence Kelly , Matthew Rudary , and Satinder Singh for helpful comments on earlier drafts of this work .
This work was supported in part by NSF grant IIS-0205435 and the DARPA REAL strategic reasoning program .
8 .
REFERENCES
APPENDIX A. PROOFS A. 1 Proof of Proposition 1
A. 2 Proof of Proposition 2
First , let us suppose that some function , f ( x ) defined on [ ai , bi ] , satisfy Lipschitz condition on ( ai , bi ] with Lipschitz constant Ai .
Then the following claim holds : Claim : infx ∈ ( ai , bi ] f ( x ) > 0.5 ( f ( ai ) + f ( bi ) − Ai ( bi − ai ) .
To prove this claim , note that the intersection of lines at f ( ai ) and f ( bi ) with slopes − Ai and Ai respectively will determine the lower bound on the minimum of f ( x ) on [ ai , bi ] ( which is a lower bound on infimum of f ( x ) on ( ai , bj ] ) .
The line at f ( ai ) is determined by f ( ai ) = − Aiai + cL and the line at f ( bi ) is determined by f ( bi ) = Aibi + cR .
Thus , the intercepts are cL = f ( ai ) + Aiai and cR = f ( bi ) + Aibi respectively .
Let x ∗ be the point at which these lines intersect .
Then ,
Since we have a finite number of points in the data set for each θ , we can obtain the following expression :
by subadditivity and the fact that a profile a is a Nash equilibrium if and only if ~ ( a ) = 0 .
Putting everything together yields the desired result .
A. 3 Proof of Theorem 3
First , we will need the following fact : Claim : Given a function fi ( x ) and a set X , | maxx ∈ X f1 ( x ) −
To prove this claim , observe that | max x ∈ X f1 ( x ) − max x ∈ X f2 ( x ) | = { maxx f1 ( x ) − maxx f2 ( x ) if maxx f1 ( x ) > maxx f2 ( x ) maxx f2 ( x ) − maxx f1 ( x ) if maxx f2 ( x ) > maxx f1 ( x ) In the first case ,
Thus , the claim holds .
By the Strong Law of Large Numbers , un , i ( a ) → ui ( a ) a.s. for
or , equivalently [ 8 ] , for any α > 0 and δ > 0 , there is M ( i , a ) > 0 such that
Thus , by the claim , for any n ≥ M ,
Now , recall that ~ ( s ) = maxi [ maxai ∈ Ai ui ( ai , s − i ) − ui ( s ) ] .
By the claims above , maxai ∈ Ai ui ( ai , s − i ) is uniformly continuous in s − i and ui ( s ) is uniformly continuous in s .
Since the difference of two uniformly continuous functions is uniformly continuous , and since this continuity is preserved under maximum by our second claim , we have the desired result .
A. 5 Proof of Theorem 6
Choose δ > 0 .
First , we need to ascertain that the following claim holds : Claim : ~ ¯ = mins ∈ S \ Nδ ~ ( s ) exists and ~ ¯ > 0 .
Since Nδ is an open subset of compact S , it follows that S \ Nδ is compact .
As we had also proved in Lemma 5 that ~ ( s ) is continuous , existence follows from the Weierstrass theorem .
That ~ ¯ > 0 is clear since ~ ( s ) = 0 if and only if s is a Nash equilibrium of Γ .
Now , by Theorem 3 , for any α > 0 there is M such that
with probability at least 1 − α .
Note that since s − i ( a ) and s ( a ) are bounded between 0 and 1 , we were able to drop them from the expressions above to obtain a bound that will be valid independent of the particular choice of s. Furthermore , since the above result can be obtained for an arbitrary α > 0 and δ > 0 , we have Pr { limn → ∞ ~ n ( s ) = ~ ( s ) } = 1 uniformly on S.
A. 4 Proof of Lemma 5
We prove the result using uniform continuity of ui ( s ) and preservation of continuity under maximum .
Claim : A function f : Rk → R defined by f ( t ) = Eki = 1 ziti , where zi are constants in R , is uniformly continuous in t .
The claim follows because | f ( t ) − f ( t ~ ) | = | Eki = 1 zi ( ti − t ~ i ) | ≤ Pk i = 1 | zi | | ti − t ~ i | .
An immediate result of this for our purposes is that ui ( s ) is uniformly continuous in s and ui ( ai , s − i ) is uniformly continuous in s − i. Claim : Let f ( a , b ) be uniformly continuous in b ∈ B for every a ∈ A , with | A | < ∞ .
Then V ( b ) = maxa ∈ A f ( a , b ) is uniformly continuous in b. To show this , take γ > 0 and let b , b ~ ∈ B such that ~ b − b ~ ~ < δ ( a ) ⇒ | f ( a , b ) − f ( a , b ~ ) | < γ .
Now take δ = mina ∈ A δ ( a ) .
Since this holds for an arbitrary α > 0 and δ > 0 , the desired result follows .
A. 6 Proof of Theorem 7
Fix θ and choose δ > 0 .
Since W ( s , θ ) is continuous at s ∗ ( θ ) , given ~ > 0 there is δ > 0 such that for every s ~ that is within δ of s ∗ ( θ ) , | W ( s ~ , θ ) − W ( s ∗ ( θ ) , θ ) | < ~ .
By Theorem 6 , we can find M ( θ ) large enough such that all s ~ ∈ Nn are within δ of s ∗ ( θ ) for all n ≥ M ( θ ) with probability 1 .
Consequently , for any ~ > 0 we can find M ( θ ) large enough such that with probability 1 we have supn ≥ M ( θ ) sups ~ ∈ Nn | W ( s ~ , θ ) − W ( s ∗ ( θ ) , θ ) | < ~ .
Let us assume without loss of generality that there is a unique optimal choice of θ .
Now , since the set Θ is finite , there is also the `` second-best '' choice of θ ( if there is only one θ ∈ Θ this discussion is moot anyway ) :
Then if we let ~ < ∆ / 2 and let M = maxθ ∈ Θ M ( θ ) , where each M ( θ ) is large enough such that supn ≥ M ( θ ) sups ~ ∈ Nn | W ( s ~ , θ ) − W ( s ∗ ( θ ) , θ ) | < ~ a.s. , the optimal choice of θ based on any empirical equilibrium will be θ ∗ with probability 1 .
Thus , in particular , given any probability distribution over empirical equilibria , the best choice of θ will be θ ∗ with probability 1 ( similarly , if we take supremum or infimum of W ( Nn ( θ ) , θ ) over the set of empirical equilibria in constructing the objective function ) .