Automated Dynamic Mechanism Design

1 Introduction . Consider the following scenario . A company assembles an internal research group to develop key technologies to be used in the company ’ s next-generation product in 5 years . The more progress the group makes , the more successful the product is likely to be . Since research progress is hard to monitor , the company manages the group based on its annual reports . At the beginning of each year , the group submits a report , summarizing its progress in the preceding year , as well as its needs for the current year . Taking into consideration this report ( and possibly also reports from previous years ) , the company then decides the compensation level and the headcount of the group in the current year . Moreover , after the product launches , the company may also pay a bonus to members of the group , depending on how successful the product is . For simplicity , suppose an annual report consists of two items : research progress ( satisfactory/unsatisfactory ) , and need to expand ( no request/request for an intern/request for a full-time employee ) . The company ’ s goal is to encourage and facilitate research progress while keeping the expenses reasonable . So , a natural managing strategy is to increase ( resp . decrease ) the compensation 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . level when the reported research progress is satisfactory ( resp . unsatisfactory ) , and to allow the group to expand only when necessary , i.e. , when the reported research progress is unsatisfactory . However , the research group may have a different goal than the company ’ s . Suppose members of the group do not care about the success of the product per se . Instead , their primary goal is to maximize the total compensation received from the company , and for this reason , they may be incentivized to misreport the situation . In other words , the company faces a dynamic mechanism design problem , where the principal ( i.e. , the company ) needs to implement ( and commit to ) a mechanism ( i.e. , a managing strategy ) that achieves its goal through repeated interactions , in the presence of strategic behavior of the agent ( i.e. , the research group ) . Indeed this problem is nontrivial . For example , if the company implements the above strategy , then the group will report satisfactory progress regardless of the actual situation , which maximizes the group ’ s total compensation over the 5 years , but also causes greater expenses for the company and jeopardizes the success of the product . To counter this , the company may additionally promise a significant bonus contingent on the success of the product . This creates incentives for the group to make more progress , and discourages overreporting the progress , because the group is not allowed to expand when the reported progress is satisfactory . That is , if actual progress is unsatisfactory , this introduces an incentive to report this truthfully so that the group may expand . However , this also runs the risk of encouraging the group to report unsatisfactory progress in order to expand even if actual progress is satisfactory , because more members always make more progress , which leads to a higher ( chance of ) bonus , whereas the cost of expanding is paid by the company and therefore irrelevant to the group . One may try to fix this by introducing more rules , possibly replacing existing ones . For example , the company may allow the group to recruit an intern , but not a full-time employee , when the reported progress is unsatisfactory . Then , in the next year , if the reported progress improves , the company allows the group to make a return offer to the intern as a full-time employee . Or alternatively , the company may unconditionally allow the group to recruit interns ( which are less costly ) , but never full-time employees . In addition to the above , the company could also temporarily decrease the compensation level when a new member joins , and later adjust the compensation based on how the reported progress improves . While all these ad hoc rules make intuitive sense , it is not immediately clear which ( combinations of ) rules are better , how to optimize parameters of these rules ( e.g. , the number of new members allowed per year and the amount by which the compensation is adjusted ) , or whether there is a better set of rules that look totally different . As demonstrated by the foregoing discussion , in general , the problem of finding an optimal mechanism in unstructured dynamic environments , such as the above example , turns out to be extremely rich and challenging . In such environments , the actions of the principal may go beyond the allocation of items to the agent , and affect the state of the world in arbitrary ways . Moreover , both the principal and the agent may have arbitrary valuations for these actions , which also depend on the current state of the world . In economic theory , the characterize-and-solve approach [ 25 , 13 , 30 ] to mechanism design has achieved spectacular success in both static and dynamic environments , by exploiting structure of the environment to construct a characterization of optimal mechanisms , often leading to closed-form or computationally tractable solutions . However , since the environments under consideration here are loosely structured at best , the classical characterize-and-solve approach does not seem particularly suited . When disregarding the agent ’ s incentives , one could treat the problem of finding an optimal strategy as a planning problem , which is known to be solvable efficiently [ 6 , 21 , 31 ] . However , as discussed above , the agent ’ s strategic behavior can ruin the performance of such a strategy . From a computational perspective , while numerous methods for automated mechanism design , which efficiently compute optimal mechanisms without heavily exploiting structures of the environment , have been proposed [ 10 , 11 , 12 ] , all existing methods work only for static environments with one-time interactions , and it is not immediately clear how to generalize these methods to dynamic environments . All this brings us to the following question : Can we efficiently compute optimal mechanisms in unstructured dynamic environments ? 1.1 Our Results . In this paper , we study the problem of computing optimal mechanisms in single-agent , discrete-time dynamic environments with a finite time horizon , without any further structural assumptions . Our main results ( presented in Section 3 ) can be summarized as follows : • Efficient algorithm : when the time horizon is fixed , there is a polynomial-time algorithm for computing optimal mechanisms , with or without payments , that maximize the principal ’ s utility facing a strategic agent . • Inapproximability : when the time horizon can be large , it is NP-hard to approximate the principal ’ s optimal utility within a factor of ( 7/8 + ε ) for any ε > 0 . To the best of our knowledge , our algorithm for constant time horizons is the first that efficiently computes optimal mechanisms in unstructured dynamic environments . The fact that our algorithm can not scale beyond constant time horizons is by no means surprising : optimal dynamic mechanisms generally depend on the entire history , and as a result , the straightforward description of such a mechanism is exponentially large in the time horizon . Our inapproximability result further rules out the possibility of computing succinct representations of approximately optimal mechanisms that can be efficiently evaluated . These results together paint a complete picture of the computational complexity of dynamic mechanism design in unstructured environments . 1.2 Further Related Work . Dynamic mechanism design . The problem we study can be situated in the broad area of dynamic mechanism design , and below we discuss some representative related work . For a more comprehensive exposition , see , e.g. , the survey by Pavan [ 29 ] and the one by Bergemann and Välimäki [ 8 ] . In the context of efficient ( i.e. , welfare-maximizing ) mechanisms , Bergemann and Välimäki [ 7 ] propose the dynamic pivot mechanism , which generalizes the VCG mechanism in static environments , and Athey and Segal [ 2 ] propose the team mechanism , which focuses on budget-balancedness . As for optimal ( i.e. , revenue-maximizing ) mechanisms , which are more closely related to our results , following earlier work [ 5 , 13 , 17 ] , Pavan et al . [ 30 ] generalize the classical characterization by Myerson [ 25 ] into dynamic environments , unifying previous results with continuous type spaces . Ashlagi et al . [ 1 ] study ex-post individual-rational dynamic mechanisms for repeated auctions , and give an efficient ( 1− ε ) -approximation to the optimal revenue for a single agent with independent valuations across items . Mirrokni et al . [ 24 ] study non-clairvoyant dynamic mechanism design , where future distributional knowledge is unavailable to the principal . All these results for optimal mechanisms follow the characterize-and-solve approach , which is quite different from the computational approach that we take . Particularly related to our results is the work by Papadimitriou et al . [ 28 ] , who study a setting where one item is sold at each time , and agents ’ valuations can be correlated across items . They show that designing an optimal deterministic mechanism is computationally hard even when there is only one agent and two items ( thereby ruling out the possibility of efficiently computing optimal deterministic mechanisms in our model , which is more general ) . And moreover , they give a polynomial-size linear program formulation for optimal randomized mechanisms for independent agents when the number of agents and the time horizon are both constant . Restricted to a single agent , their LP formulation can be viewed as a special case of our main result : they focus on revenue maximization with a single item to be allocated at each time , in a model where the principal ’ s actions can not affect the future valuations of the agent ; on the other hand , we allow the principal to care about actions as well as revenue , with actions being general and unstructured ( as opposed to allocation/no allocation ) , where the future state of the world can depend arbitrarily on the principal ’ s actions as well as the current state . Automated Mechanism Design . There is a rich body of research regarding automated mechanism design ( AMD ) in ( essentially ) static environments . Conitzer and Sandholm [ 10 , 11 ] initiated the study of automated mechanism design . They consider various specific static setups , and show that computing optimal deterministic mechanisms , even with a single agent , is often NP-hard ( which also rules out the possibility of efficiently computing optimal deterministic mechanisms in our model , since the 1-period case is a special case ) , while computing optimal randomized mechanisms is often tractable . Conceptually related to our model , Hajiaghayi et al . [ 20 ] consider a model where agents enter and leave the mechanism online ( but still have one-time interactions with the mechanism ) , and provide an algorithm for computing mechanisms that are competitive against the optimal allocation from hindsight . Sandholm et al . [ 35 ] study automated design of multistage mechanisms , but these are not for dynamic settings ; instead , the motivation is to implement static mechanisms using multiple rounds of queries in order to minimize the communication cost . Sandholm and Likhodedov [ 34 ] study automated design of combinatorial auction mechanisms , and Balcan et al . [ 3 , 4 ] study the sample complexity thereof . Kephart and Conitzer [ 22 , 23 ] and Zhang et al . [ 37 ] study AMD with partial verification and/or reporting costs . More recently , various methods have been proposed for automated mechanism design via machine learning [ 14 , 27 ] , and in particular , deep learning [ 15 , 18 , 36 , 32 ] . All these results are essentially for static environments , whereas in this paper , we focus solely on AMD in dynamic environments . Another emerging research direction is Bayesian persuasion in dynamic environments [ 16 , 33 ] . In particular , Celli et al . [ 9 ] study an algorithmic persuasion problem in extensive-form games , where a single signal is sent at the very beginning , and Gan et al . [ 19 ] study an algorithmic persuasion problem in infinite-horizon discounted MDPs , where a new signal is sent at every time . These persuasion problems can be viewed as a dual problem of ours : in our problem , the principal has the commitment power , and tries to encourage the agent to truthfully report their private information , whereas in ( dynamic ) Bayesian persuasion , the agent has the commitment power , and tries to induce the principal to act in favor of the agent by selectively revealing their private information .

Summary: The authors formulate and study dynamic mechanism design in an MDP-like setting. The motivation is principal-agent problems. The principal commits to a policy to choose actions given observations of states; the environment evolves according to some transitions dynamics based on these actions; the agent does not act but is free to misreport states to the principal. Agent and principal receive utility; the principal’s goal is to choose a policy that is incentive compatible and individually rational (standard goals of mechanism design) while maximizing its own expected total utility.  In general, from a full description of the environment and dynamics, the “planning” type problem to find an optimal mechanism is NP-hard (shown via reduction from MAXSAT). If the time horizon of the problem is treated as a constant, then it is possible to formulate a linear program to find the optimal mechanism in polynomial time, and doing so is a major portion of the paper.