Automated Dynamic Mechanism Design

1 Introduction . Consider the following scenario . A company assembles an internal research group to develop key technologies to be used in the company ’ s next-generation product in 5 years . The more progress the group makes , the more successful the product is likely to be . Since research progress is hard to monitor , the company manages the group based on its annual reports . At the beginning of each year , the group submits a report , summarizing its progress in the preceding year , as well as its needs for the current year . Taking into consideration this report ( and possibly also reports from previous years ) , the company then decides the compensation level and the headcount of the group in the current year . Moreover , after the product launches , the company may also pay a bonus to members of the group , depending on how successful the product is . For simplicity , suppose an annual report consists of two items : research progress ( satisfactory/unsatisfactory ) , and need to expand ( no request/request for an intern/request for a full-time employee ) . The company ’ s goal is to encourage and facilitate research progress while keeping the expenses reasonable . So , a natural managing strategy is to increase ( resp . decrease ) the compensation 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . level when the reported research progress is satisfactory ( resp . unsatisfactory ) , and to allow the group to expand only when necessary , i.e. , when the reported research progress is unsatisfactory . However , the research group may have a different goal than the company ’ s . Suppose members of the group do not care about the success of the product per se . Instead , their primary goal is to maximize the total compensation received from the company , and for this reason , they may be incentivized to misreport the situation . In other words , the company faces a dynamic mechanism design problem , where the principal ( i.e. , the company ) needs to implement ( and commit to ) a mechanism ( i.e. , a managing strategy ) that achieves its goal through repeated interactions , in the presence of strategic behavior of the agent ( i.e. , the research group ) . Indeed this problem is nontrivial . For example , if the company implements the above strategy , then the group will report satisfactory progress regardless of the actual situation , which maximizes the group ’ s total compensation over the 5 years , but also causes greater expenses for the company and jeopardizes the success of the product . To counter this , the company may additionally promise a significant bonus contingent on the success of the product . This creates incentives for the group to make more progress , and discourages overreporting the progress , because the group is not allowed to expand when the reported progress is satisfactory . That is , if actual progress is unsatisfactory , this introduces an incentive to report this truthfully so that the group may expand . However , this also runs the risk of encouraging the group to report unsatisfactory progress in order to expand even if actual progress is satisfactory , because more members always make more progress , which leads to a higher ( chance of ) bonus , whereas the cost of expanding is paid by the company and therefore irrelevant to the group . One may try to fix this by introducing more rules , possibly replacing existing ones . For example , the company may allow the group to recruit an intern , but not a full-time employee , when the reported progress is unsatisfactory . Then , in the next year , if the reported progress improves , the company allows the group to make a return offer to the intern as a full-time employee . Or alternatively , the company may unconditionally allow the group to recruit interns ( which are less costly ) , but never full-time employees . In addition to the above , the company could also temporarily decrease the compensation level when a new member joins , and later adjust the compensation based on how the reported progress improves . While all these ad hoc rules make intuitive sense , it is not immediately clear which ( combinations of ) rules are better , how to optimize parameters of these rules ( e.g. , the number of new members allowed per year and the amount by which the compensation is adjusted ) , or whether there is a better set of rules that look totally different . As demonstrated by the foregoing discussion , in general , the problem of finding an optimal mechanism in unstructured dynamic environments , such as the above example , turns out to be extremely rich and challenging . In such environments , the actions of the principal may go beyond the allocation of items to the agent , and affect the state of the world in arbitrary ways . Moreover , both the principal and the agent may have arbitrary valuations for these actions , which also depend on the current state of the world . In economic theory , the characterize-and-solve approach [ 25 , 13 , 30 ] to mechanism design has achieved spectacular success in both static and dynamic environments , by exploiting structure of the environment to construct a characterization of optimal mechanisms , often leading to closed-form or computationally tractable solutions . However , since the environments under consideration here are loosely structured at best , the classical characterize-and-solve approach does not seem particularly suited . When disregarding the agent ’ s incentives , one could treat the problem of finding an optimal strategy as a planning problem , which is known to be solvable efficiently [ 6 , 21 , 31 ] . However , as discussed above , the agent ’ s strategic behavior can ruin the performance of such a strategy . From a computational perspective , while numerous methods for automated mechanism design , which efficiently compute optimal mechanisms without heavily exploiting structures of the environment , have been proposed [ 10 , 11 , 12 ] , all existing methods work only for static environments with one-time interactions , and it is not immediately clear how to generalize these methods to dynamic environments . All this brings us to the following question : Can we efficiently compute optimal mechanisms in unstructured dynamic environments ? 1.1 Our Results . In this paper , we study the problem of computing optimal mechanisms in single-agent , discrete-time dynamic environments with a finite time horizon , without any further structural assumptions . Our main results ( presented in Section 3 ) can be summarized as follows : • Efficient algorithm : when the time horizon is fixed , there is a polynomial-time algorithm for computing optimal mechanisms , with or without payments , that maximize the principal ’ s utility facing a strategic agent . • Inapproximability : when the time horizon can be large , it is NP-hard to approximate the principal ’ s optimal utility within a factor of ( 7/8 + ε ) for any ε > 0 . To the best of our knowledge , our algorithm for constant time horizons is the first that efficiently computes optimal mechanisms in unstructured dynamic environments . The fact that our algorithm can not scale beyond constant time horizons is by no means surprising : optimal dynamic mechanisms generally depend on the entire history , and as a result , the straightforward description of such a mechanism is exponentially large in the time horizon . Our inapproximability result further rules out the possibility of computing succinct representations of approximately optimal mechanisms that can be efficiently evaluated . These results together paint a complete picture of the computational complexity of dynamic mechanism design in unstructured environments . 1.2 Further Related Work . Dynamic mechanism design . The problem we study can be situated in the broad area of dynamic mechanism design , and below we discuss some representative related work . For a more comprehensive exposition , see , e.g. , the survey by Pavan [ 29 ] and the one by Bergemann and Välimäki [ 8 ] . In the context of efficient ( i.e. , welfare-maximizing ) mechanisms , Bergemann and Välimäki [ 7 ] propose the dynamic pivot mechanism , which generalizes the VCG mechanism in static environments , and Athey and Segal [ 2 ] propose the team mechanism , which focuses on budget-balancedness . As for optimal ( i.e. , revenue-maximizing ) mechanisms , which are more closely related to our results , following earlier work [ 5 , 13 , 17 ] , Pavan et al . [ 30 ] generalize the classical characterization by Myerson [ 25 ] into dynamic environments , unifying previous results with continuous type spaces . Ashlagi et al . [ 1 ] study ex-post individual-rational dynamic mechanisms for repeated auctions , and give an efficient ( 1− ε ) -approximation to the optimal revenue for a single agent with independent valuations across items . Mirrokni et al . [ 24 ] study non-clairvoyant dynamic mechanism design , where future distributional knowledge is unavailable to the principal . All these results for optimal mechanisms follow the characterize-and-solve approach , which is quite different from the computational approach that we take . Particularly related to our results is the work by Papadimitriou et al . [ 28 ] , who study a setting where one item is sold at each time , and agents ’ valuations can be correlated across items . They show that designing an optimal deterministic mechanism is computationally hard even when there is only one agent and two items ( thereby ruling out the possibility of efficiently computing optimal deterministic mechanisms in our model , which is more general ) . And moreover , they give a polynomial-size linear program formulation for optimal randomized mechanisms for independent agents when the number of agents and the time horizon are both constant . Restricted to a single agent , their LP formulation can be viewed as a special case of our main result : they focus on revenue maximization with a single item to be allocated at each time , in a model where the principal ’ s actions can not affect the future valuations of the agent ; on the other hand , we allow the principal to care about actions as well as revenue , with actions being general and unstructured ( as opposed to allocation/no allocation ) , where the future state of the world can depend arbitrarily on the principal ’ s actions as well as the current state . Automated Mechanism Design . There is a rich body of research regarding automated mechanism design ( AMD ) in ( essentially ) static environments . Conitzer and Sandholm [ 10 , 11 ] initiated the study of automated mechanism design . They consider various specific static setups , and show that computing optimal deterministic mechanisms , even with a single agent , is often NP-hard ( which also rules out the possibility of efficiently computing optimal deterministic mechanisms in our model , since the 1-period case is a special case ) , while computing optimal randomized mechanisms is often tractable . Conceptually related to our model , Hajiaghayi et al . [ 20 ] consider a model where agents enter and leave the mechanism online ( but still have one-time interactions with the mechanism ) , and provide an algorithm for computing mechanisms that are competitive against the optimal allocation from hindsight . Sandholm et al . [ 35 ] study automated design of multistage mechanisms , but these are not for dynamic settings ; instead , the motivation is to implement static mechanisms using multiple rounds of queries in order to minimize the communication cost . Sandholm and Likhodedov [ 34 ] study automated design of combinatorial auction mechanisms , and Balcan et al . [ 3 , 4 ] study the sample complexity thereof . Kephart and Conitzer [ 22 , 23 ] and Zhang et al . [ 37 ] study AMD with partial verification and/or reporting costs . More recently , various methods have been proposed for automated mechanism design via machine learning [ 14 , 27 ] , and in particular , deep learning [ 15 , 18 , 36 , 32 ] . All these results are essentially for static environments , whereas in this paper , we focus solely on AMD in dynamic environments . Another emerging research direction is Bayesian persuasion in dynamic environments [ 16 , 33 ] . In particular , Celli et al . [ 9 ] study an algorithmic persuasion problem in extensive-form games , where a single signal is sent at the very beginning , and Gan et al . [ 19 ] study an algorithmic persuasion problem in infinite-horizon discounted MDPs , where a new signal is sent at every time . These persuasion problems can be viewed as a dual problem of ours : in our problem , the principal has the commitment power , and tries to encourage the agent to truthfully report their private information , whereas in ( dynamic ) Bayesian persuasion , the agent has the commitment power , and tries to induce the principal to act in favor of the agent by selectively revealing their private information .

Summary: This paper studies automated mechanism design in dynamic unstructured dynamic environments. In the setting of interest, a principal and agent repeatedly interact for a finite number of time steps (finite time horizon); at each time step, the agent reports the current state of the environment, the principal takes action, and the environment transitions to the new state. Importantly, the agent's and the principal's utilities are not necessarily the same, implying that the agent might misreport which further implies the need for a mechanism.  The paper considers a randomized dynamic mechanisms that consists of the policy of the principal and the mechanism's payment function. The main results are: a) an LP for finding optimal mechanisms (constant time horizon), b) a computational complexity result which shows that the principal's utility is hard to approximate within a certain factor for environments with large time horizons.  The paper has additional results, as mentioned in the abstract, but these are not presented in the main text.