Online Allocation and Learning in the Presence of Strategic Agents
Anonymous Author(s)
AfﬁliationAddress email
Abstract
We study the problem of allocating T sequentially arriving items among n homogeneous agents under the constraint that each agent must receive a pre-speciﬁedfraction of all items, with the objective of maximizing the agents’ total valuationof items allocated to them.
The agents’ valuations for the item in each round areassumed to be i.i.d.
but their distribution is a priori unknown to the central planner.
Therefore, the central planner needs to implicitly learn these distributions fromthe observed values in order to pick a good allocation policy.
However, an addedchallenge here is that the agents are strategic with incentives to misreport theirvaluations in order to receive better allocations.
This sets our work apart both fromthe online auction design settings which typically assume known valuation distributions and/or involve payments, and from the online learning settings that do notconsider strategic agents.
To that end, our main contribution is an online learningbased allocation mechanism that is approximately Bayesian incentive compatible,and when all agents are truthful, guarantees a sublinear regret for individual agents’utility compared to that under the optimal ofﬂine allocation policy.
Introduction
A classic sequential resource allocation problem is to allocate T sequentially arriving items to nagents, where each agent must receive a predetermined fraction of the items.
The goal is to maximizesocial welfare, i e., the agents’ total valuation of the items allocated to them.
This problem is nontrivial even when the agents’ valuations are stochastic and i i d. with a known distribution, the maindifﬁculty being that the allocations must be performed in real-time; speciﬁcally, an item must beallocated to an agent in the current round without knowledge of their future valuations.
A more challenging (and quite useful) extension of the problem which has been the focus of recentliterature considers the case where the distribution of the agents’ valuations is apriori unknown tothe planner.
In such settings, algorithms based on online learning can be used to adaptively learnthe valuation distribution from observed valuations in previous rounds, and improve the allocationpolicy over time (see [2, 10, 6, 5] for some examples).
However, these mechanisms implicitly assumethat the agents report their valuations truthfully, so that the mechanism can directly learn from thereported valuations in order to maximize the social welfare.
Many practical resource allocations settings do not conform with the truthful reporting assumption.
In particular, selﬁsh and strategic agents may have an incentive to misreport their valuations if that can lead to individual utility gain (possibly at the expense of social welfare).
Hence, an allocation policy that does not take such misreporting incentives into account can incur signiﬁcant loss in social welfare in presence of strategic agents.
For example, consider a simple setting with two agents whose true valuations are i.i.d.
and uniformly distributed between 0 and 1.
That is,
[Equation]
Each agent is pre-determined to receive an equal fraction of all the items.
The optimal welfaremaximizing allocation policy is to allocate the item to the agent with higher valuation in (almost)every round.
This policy results in T/ 3 expected utility ( E [ X 1 | X 1 > X 2 ] / 2 ) for each agent,and a social welfare of 2 T/ 3 .
However, suppose that the ﬁrst agent chooses to misreport in thefollowing way: the agent reports a high valuation of 1 whenever her true valuation is in [0 . 5 , 1] anda low valuation of 0 whenever her true valuation is in [0 , 0 . 5] .
Assuming the other agent remainstruthful, this will lead to the ﬁrst agent receiving all the items in her top 1 / 2 quantile, and therefore asigniﬁcantly increased utility of 3 T/ 8 ( E [ X 1 | X 1 > 0 .
5] / 2 ) compared to T/ 3 under truthful reporting.
The social welfare however, goes down to 5 T/ 8 in this case.
Thus under the optimal policy, eachagent has an incentive to misreport her valuations in order to gain individual utility.
The incentives tomisreport may be further ampliﬁed under an online learning based allocation algorithm that learnsapproximately optimal policies from the valuations observed in the previous rounds.
In such settings,the agents can potentially mislead the online learning algorithm to learn a more favorable policy overtime by repeatedly misreporting their values.
Motivated by these shortcomings, in this paper, we consider the problem of designing an onlinelearning and allocation mechanism in the presence of strategic agents .
Speciﬁcally, we considerthe problem of sequentially allocating T items to n strategic agents.
The problem proceeds in Trounds.
In each round t = 1 , . . .
, T , the agents’ true valuations
X
i,t , i = 1 , . . . , n for the t th item aregenerated i.i.d.
from a distribution F a priori unknown to the central planner.
However, the centralplanner can only observe a value
˜ X i,t reported by each agent i , which may or may not be the sameas her true valuation X i,t for the item.
Using the reported valuations from the current and previousrounds, the central planner needs to make an irrevocable decision of who to allocate the current item.
The allocations should be made in a way such that each agent at the end receives a ﬁxed fraction p ⇤ iof the T items, where p ⇤ i > 0 ,
P ni =
p
⇤ i
= 1 .
The objective of the central planner is to maximize thetotal utility of the agents, where utility of each agent is deﬁned as the sum total true valuations ofitems received by the agent.
Our main contribution is a mechanism that achieves both: (a) Bayesian incentive compatibility , i.e.,assuming all the other agents are truthful, with high probability no single agent can gain a signiﬁcantutility by deviating from the truthful reporting strategy, and; (b) near-optimal regret guarantees ,namely, the utility of each individual agent under the online mechanism is “close" to that achievedunder the optimal ofﬂine allocation policy.
Organization After discussing the related literature in some further detail in Section 2, we formallyintroduce the problem setting and some of the core concepts in Section 3.
Section 4 describesour online learning and allocation algorithm and provides formal statements of our main results(Theorem 1 and Theorem 2).
Section 5 and Section 6 provide an overview of the proofs of the abovetheorems.
All the missing details of the proofs are provided in the appendix.
Finally, in Section 7 wediscuss some limitations and future directions.
Literature review
Our work lies at the intersection of online learning and mechanism design.
From an online learningperspective, our setting is closely related to the recent work on constrained online resource allocationunder stochastic i.i.d.
rewards/costs (e.g., see [10, 2, 1, 6, 5]).
However, a crucial assumption in thosesettings is that the central planner can observe the true rewards/costs of an allocation, which in oursetting would mean that the central planner can observe agents’ true valuations of the items beingallocated.
Our work extends these settings to allow for selﬁsh and strategic agents who may haveincentives to misreport their valuations.
As discussed in the introduction, unless the online allocationmechanism design takes these incentives into account, selﬁsh agents may signiﬁcantly misreport theirvaluations to cause signiﬁcant loss in social welfare.
Incentives and strategic agents have been previously considered in online allocation mechanismdesign, however, most of that work has focused on auction design where payments are used as akey mechanism for limiting rational agents’ incentives to misreport their valuations.
For example,
Amin, Rostamizadeh, and Syed [3] study a posted-price mechanism in a repeated auction settingwhere buyers’ valuations are context dependent.
Golrezaei, Javanmard, and Mirrokni [12] extend thiswork to multi-buyer setting, using second-price auction with dynamic personalized reserve prices.
Kanoria and Nazerzadeh [17] study a similar problem in a non-contextual setting.
(There is also asigniﬁcant literature that studies learning in repeated auction settings from the bidder’s perspective.
Since our paper focuses on the central planner’s point of view, we omit references to that literature. )
All of the above-mentioned works are concerned with maximizing revenue for the seller, and usemoney/payments as a key instrument for eliciting private information about the bidder’s valuationsfor the items.
In this paper, we are concerned with online allocation without money, and the goal is tomaximize each agent’s utility.
Recently, there has been some work on studying reductions from mechanism design with money tothose without money.
Gorokh, Banerjee, and Iyer [13] provided a black-box reduction from any oneshot, BIC mechanism with money, to an approximately BIC mechanism without money.
However,their reduction relies crucially on knowing the true value distribution of agents and therefore is notapplicable to our setting.
Procaccia and Tennenholtz [18] consider a speciﬁc (one-round) facilityallocation problem and explicitly formulate the idea of designing mechanisms without money toachieve approximately optimal performance against mechanisms with money.
Subsequently, there is aseries of papers that extended the results on mechanism design without money in a single shot setting,when the bidders’ value distribution is unknown [14, 16, 8, 9, 8].
These papers either use a veryrestricted setting with just two agents, or use very speciﬁc/simple valuation functions for the agents.
Even in these basic settings, they show that the best one can hope for is a constant approximation towhat one can achieve with a mechanism that uses money.
It is not clear what kind of regret guaranteessuch reductions imply in a repeated online learning setting.
Therefore, we do not consider suchreductions from auction mechanisms with money to be a fruitful direction for achieving our goals ofboth incentive compatibility and low (sublinear) regret for our online allocation problem.
Finally, in a repeated allocation settings with known valuation distribution, there are more positiveresults for truthful mechanism design without money.
For example, Guo, Conitzer, and Reeves [15]and later Balseiro, Gurkan, and Sun [7] studied the problem of repeatedly allocating items to agentswith known value distributions; both use a state-based “promised utility” framework.
To summarize, to the best of our knowledge, this is the ﬁrst paper to incorporate strategic agents’incentives in the well-studied online allocation problem with stochastic i.i.d.
rewards and unknowndistributions.
Thus, it bridges the gap between the online learning and allocation literature whichfocuses on non-strategic inputs, and the work on learning in repeated auctions which focuses onallocation mechanisms that utilize money (payments) to achieve incentive compatibility.
Problem formulation
The ofﬂine problem
We ﬁrst state the ofﬂine version of the problem which will serve as our benchmark for the online problem.
There is a set of n agents, and a distribution F over X :=
[0 , ¯ x ] n .
Each draw X ⇠ F from this distribution represents the n agents’ valuations of one item: X =
[ X 1 , . . .
, X n ] .
We assume that the agents’ valuations are i.i.d., i.e.
[Equation]
A matching policy (aka allocation policy) maps, potentially with some exogenous randomness, a valuevector X to one of the agents i 2 { 1 , . . . , n } .
Speciﬁcally, given a realized value vector X 2 [0 , ¯ x ] n ,a (possibly randomized) policy ⇡ maps X to agent ⇡ ( X ) 2 { 1 , . . . , n } , with the probability ofagent i receiving an allocation given by P ( ⇡ ( X ) = i ) .
The ofﬂine optimization problem is to ﬁnd asocial welfare-maximizing policy ⇡
⇤ such that each agent i in expectation receives a predeterminedfraction p
⇤ i of the pool of items, where p ⇤ i > 0 ,
P i p
⇤ i = 1 .
The problem of ﬁnding optimal policycan therefore be stated as the following
[Equation]
where the expectations are taken both over X ⇠ F and any randomness in the mapping madeby policy ⇡ given X .
Solving the ofﬂine problem is non-trivial, as it is an inﬁnite dimensionaloptimization problem as stated in its’ current form in (1).
But it turns out to be closely related to
Semi-Discrete Optimal Transport, and that the dual of (1) can be written as
[Equation]
where L i is what is sometimes referred to as the Laguerre cell :
[Equation]
Let  ⇤ ( F ) denote an optimal solution to (2).
When it is clear from the context, we omit the distribution
F .
It is known that an optimal solution to (1) is given by the following deterministic policy deﬁnedby the Laguerre cell partition (Proposition 2.1 [4]):
[Equation]
More generally, we will refer to any policy deﬁned by a Laguerre cell partition as a greedy policy.
Deﬁnition 1 (Greedy allocation policy) .
Consider any allocation policy that partitions the domain[0 , ¯ x ] n as L i (  ) (as deﬁned in (4) ) for some  2 R n .
We refer to such a policy as the greedyallocation policy with parameter  .
Note that there are efﬁcient algorithms for solving (2) (see[4]) when the distribution F is known.
Therefore we will treat  ⇤ ( · ) as a black-box that can be computed efﬁciently for any given inputdistribution ˆ F .
The online problem: approximate Bayesian incentive compatibility and regret
We are interested in the case when items are sequentially allocated over T rounds, and that thedistribution F is initially unknown.
Speciﬁcally, in each round t = 1 , . .
, T , the agents’ truevaluations X t =
( X i,t , i = 1 , . . . , n ) are generated i.i.d.
from the distribution F a priori unknown tothe central planner.
However, the central planner does not observe X t but only observes the reportedvaluations ˜ X t
= ( ˜ X i,t , i = 1 , . . . , n ) which may or may not be the same as the true valuations.
An online allocation mechanism consists of a sequence of allocation policies ⇡ 1 , . . .
, ⇡ t where thepolicy ⇡ t at time t may be adaptively chosen based on the observed information until before time t :
[Equation]
Given allocation policy ⇡ t at time t , the agent i ’s utility at time t is given by
[Equation]
If an agent i has already reached his target allocation of p ⇤ i T items, then he cannot be allocated moreitems.
Note that since the allocation policy may be randomized, for any given value vector X , ⇡ t ( X )is a random variable.
To ensure truthful reporting in presence of strategic agents, we are interested inmechanisms that are (approximately) Bayesian incentive compatible.
Deﬁnition 2 (Approximate-BIC) .
For an online allocation mechanism, let ⇡ t , t = 1 , . .
, T be the sequence of allocations when all agents report truthfully, i.e., when ˜ X t =
X t , 8 t ; and let ˜ ⇡ it , t = 1 , . .
, T be the sequence when all agents except i report truthfully, i.e., X j,t = ˜
X j,t , 8 j 6 = i .
Then the online allocation mechanism is called ( ↵ ,  ) -approximate
Bayesian Incentive Compatible if, for all i = 1 , . .
. , n , with probability at least 1   ,
[Equation]
Here, the probability is with respect to the randomness in true valuations X t ⇠ F and any randomness in the online allocation policy.
For the online policy to be approximate-BIC, the statementshould hold for all possible misreporting of valuations ˜
X i,t 6 =
X i,t .
Therefore, if ↵ is small, then an individual agent has little incentive to strategize.
Note thatapproximate-BIC also implies that truthful reporting for all agents constitutes an approximateNash equilibrium.
Assuming that all agents are truthful, we are also interested in bounding eachindividual agent’s regret.
Deﬁnition 3 (Individual regret) .
We deﬁne an individual agent i ’s regret under an online allocationmechanism as the difference between agent i ’s realized utility over T rounds and the expected utilityachieved in the ofﬂine expected problem.
That is,
[Equation]
Here ⇡ 1 , . .
, ⇡ T denote the allocation policies used by the online allocation mechanism in roundt = 1 , . . .
, T .
Note that since social welfare is given by the sum of all agents’ utilities, a bound on individual regretimplies a bound on the regret in social welfare of the mechanism.
Algorithm and main results
We present an online allocation mechanism that is approximately-BIC, and further achieves low regretguarantees on individual regret when all agents are truthful.
Our algorithm contains two components:a learner, and a detector.
Intuitively, the detector makes sure that the mechanism is approximately BIC, and the learner adaptively learns utility-maximizing allocation policies assuming truthful agents.
The learner runs in epochs with geometrically increasing lengths.
The starting time of each epoch kis given by L k = 2 k , k = 0 , 1 , . .
. , which is also when the allocation policy is updated.
At the end ofeach epoch (i.e., at time t =
L k  1 for epoch k ), the learner takes all the previously reported valuesfrom all the agents, and uses them to construct an empirical distribution of the agents’ valuations.
The learner implicitly assumes truthful agents in its computations.
Therefore, since the agents’ truevaluations are i.i.d.
, it ﬁrst constructs a single, one-dimensional empirical distribution ˆ F t , and thenuses it to construct the corresponding n -dimensional distribution ˆ F t :
[Equation]
The learning algorithm then solves the ofﬂine problem (2) using ˆ F t , and uses the resulting greedyallocation policy characterized by  ⇤ ( ˆ F t ) to allocate the items in the following epoch.
In parallel to the learner, the detector constructs and monitors, in each time step t , and for each agenti , two empirical distributions.
One using the reported valuations from agent i : ¯ F t , and one usingthe reported valuations from all the other agents: ˜ F t .
The detection algorithm then computes thesupremum between the two empirical CDFs, sup x | ¯ F t ( x )  ˜ F t ( x ) | .
If this difference is greater thana predetermined threshold  t , then the detector raises a ﬂag that there has been a violation of truthfulreporting and the entire allocation game stops.
Otherwise, the process continues.
The threshold  t needs to be chosen such that if everyone is truthful, then with high probability thedetection algorithm will not pull the trigger.
At the same time, if someone deviates from truthfulreporting signiﬁcantly, then it should detect this with high probability.
The typical concentrationresult used in comparing empirical CDFs is the Dvoretzky-Kiefer-Wolfowitz (DKW) inequality[11].
However, since a strategic agent can adaptively change its’ misreporting strategy, we cannot directlyapply the DKW inequality, which assumes i.i.d.
samples.
Instead, we use martingale version of the
DKW inequality (Lemma 1), and use that to choose an appropriate threshold  t .
The details aregiven in Algorithm 1 and Algorithm 2.
Our main results are the following guarantees on incentive compatibility and regret of our onlineallocation algorithm.
Theorem 1 (Approximate-BIC) .
Algorithm 1 is ( O ( p nT log( nT/  )
Input: T,
Initialize:  = [0 , . .
. , 0] , k = 0 , K = log 2 ( T ) , L k = 2 k , k = 0 , . .
, K ; 1 for t 1 , 2 , 3 , . .
, T do
Observe X t .
Run Detection Algorithm (Algorithm 2) with sample set S =
{ X 1 , . . .
, X t } , and threshold  t =q 1 t log( 256 et  ) if Detection Algorithm Return Reject then
Terminates .
if one of the agents i 2 { 1 , . . . , n } has reached the allocation capacity p
⇤ i T then
Allocate randomly among agents who have not reached capacity else
Allocate item using greedy allocation policy  if t =
L k +1  1 then
Compute ˆ F t from samples { X 1 , . . .
, X t } as in (7).
k
Input: Sample set S =
{ X 1 , . . .
, X t } , threshold  t .
1 for i 1 , . . . , n do
Compute ¯ F t ( x ) = 1 t
P ts =
[ X si  x ] as the empirical CDF of the samples collected from agent
i
Compute ˜ F t ( x ) = 1 t ( n  1)
P ts =
P j 6 = i [ X sj  x ] be the empirical CDF of all reported values from the other agents.
if sup x | ˜ F t ( x )  ¯ F t ( x ) |   t 2 then
Return Reject 6 Return Accept
Since truthful reporting constitutes an approximate equilibrium, it is reasonable to then assume thatagents will act truthfully.
We show the following individual regret bound assuming truthfulness.
Theorem 2 (Individual Regret) .
Assuming all agents report their valuations truthfully, then under theonline allocation mechanism given by Algorithm 1, with probability 1   , every agent i ’s individualregret can be bounded as:
[Equation]
Showing approximate incentive compatibility, and then guaranteeing regret under the assumption oftruthfulness, is an approach commonly seen in the online mechanism design literature (e.g. Theorem 4in [17]).
In the next section, we describe the high level proof ideas for the main results above.
Proof ideas
Proof ideas for Theorem
We establish that the mechanism is approximately BIC by showingthat no single agent has incentive to signiﬁcantly deviate from reporting true valuations if all the otheragents are truthful.
The proof consists of two parts.
In Step 1, we prove that any signiﬁcant deviationfrom the truth can be detected and will lead the mechanism to terminate.
In Step 2,3, we prove that inorder to achieve a signiﬁcant gain in utility, an agent indeed has to report values that signiﬁcantlydeviate from the truth.
Step 1 Assuming that there is only one (unidentiﬁed) strategic agent while all the other agentsare truthful, we ﬁrst show that if the detector does not trigger a violation by time t then with highprobability, the empirical distribution of valuations reported by the strategic agent is no more than
O (1 / p t ) away from the true distribution (Lemma 3).
The key observation here is that since the agents’valuations are i.i.d.
, we can compare their reported values to detect if any single agent’s distribution issigniﬁcantly different from everyone else’s.
A technical challenge in making statistical comparisonshere is that the strategic agent can adaptively change their reporting strategy over time based on therealized outcomes.
Therefore, we derive a novel martingale version of the DKW inequality to showconcentration of the empirical distribution relative to the true underlying distribution.
Step
In a given round, given the history, the mechanism’s allocation policy is a ﬁxed greedyallocation policy given by  .
If the distribution of strategic agent’s reported values differs from thetrue distribution by at most  , then the agent’s expected utility gain in that round, compared toreporting truthfully, is at most O (  ) , (see Lemma 4).
Step
If over t rounds, the distribution of the strategic agent’s reported values is at most  awayfrom the true distribution, then the learning algorithm will, with high probability ﬁnd an allocationpolicy that is at most O ( p n  ) away (in terms of individual utility) from what it would have learnedif all the agents were truthful instead (see Lemma 2).
To understand the signiﬁcance and distinction between results in Step 2 vs. Step 3, note that astrategic agent has two separate ways to gain utility.
The ﬁrst is to report valuations in a way thatthe agent immediately wins more/better items under the central planner’s current allocation policy.
However, since the central planner is updating its’ allocation policy over time, the strategic agent canalso misreport in a way that beneﬁts its’ future utility , by “tricking” the central planner into learning apolicy that favors him later on.
Together, Step 2 and Step 3 show that the agent cannot gain signiﬁcantadvantage over being truthful in either manner.
In many existing works on online auctions mechanisms design, where the central planner dynamicallyadjusts the reserve price over time, these two types of strategic behaviors are in conﬂict: the agenteither sacriﬁces future utility to gain immediate utility; or sacriﬁces near-term utility for future utility.
The results in those settings therefore often rely on this observation to show approximate incentivecompatibility.
In our case however, since there is no money involved, it is not clear if such a conﬂictbetween short and long term utility exists.
Nonetheless, we are able to bound the agents’ ability tostrategize.
Step 2 bounds the agent’s short term incentive to be strategic, whereas Step 3 bounds thelonger term incentive to be strategic.
Combining these steps gives us a proof for Theorem 1.
Proof ideas for Theorem 2 Recall that here we assume all agents’ are truthful.
The proof involvestwo main steps.
Step 1 We show that uniformly for any t = 1 , . .
, T , with high probability, the empirical distribution constructed from the ﬁrst t samples is close (within a distance of ˜ O
(1 / p nt ) ) to the truedistribution F .
Here, the factor of 1 / p n comes from the fact that in each round we observe nindependent samples from the value distribution, one from each of the agents.
This also implies that ifall the agents are reporting truthfully, then, with high probability, the detector will not falsely trigger.
Step
We show that the allocation policy learned under the empirical distribution estimated fromthe samples is close to the the optimal allocation policy (Lemma 2).
Speciﬁcally, after t rounds ifthe empirical CDF is at most O (1 / p nt ) away from the true distribution, then each agents’ expectedutility in one round under the learned allocation policy is at most p n/t away (both from above andfrom below) from the optimal.
By using an epoch based approach we can then show that each agents’individual regret is with high probability bounded by O ( p nT ) over the entire planning horizon T .
Proof details
We will now outline our proof in more detail.
All missing proofs can be found in the Appendix.
Firstwe state the following martingale variation of the well-known Dvoretzky-Kiefer-Wolfowitz (DKW)inequality[11].
This is critical when dealing with strategic agents as they can adapt their strategy overtime, resulting in non-independent (reported) values.
Lemma 1 (Martingale Version of DKW Inequality) .
Given a sequence of random variables
Y 1 , . . .
, Y T , let F t =
( Y 1 , . .
, Y t ) , t = 1 , . . .
, T be the ﬁltration representing the information in the ﬁrst t variables.
Let F t ( y ) :
= Pr( Y t  y |F t  1 ) , and ¯ F T ( y ) :
= 1 T
P
Tt =
[ Y t  y ] .
Then,
[Equation]
Next we introduce a new notation to to denote the fraction of allocation that j receives under the greedy allocation policy with parameter  and valuation distribution F :
[Equation]
We start with proving Theorem 2, as we will use this to prove Theorem 1 later.
6.1 Individual Regret Bound (Theorem 2)
In Algorithm 1, the allocation policy is trained on the empirical distribution constructed from samples.
We want to show that this difference between empirical and population distribution will not impairthe performance of the resulting allocation policy too much.
Lemma 2.
Let G
= G 1 ⌦ . . .
⌦ G n , and F = F 1 ⌦ . . .
⌦ F n be two distributions over [0 , ¯ x ] n where the marginals on each coordinate are independent.
Suppose sup x | F i ( x )  G i ( x ) |   8 i .
Let  =  ⇤ ( G ) , and
⇤ =
⇤ ( F ) .
Then
[Equation]
Proof of Theorem
Now we have the main ingredients for Theorem 2.
We use the DKW inequalityto show that the empirical distribution constructed in (7) is close to the true distribution w.h.p..
Thenwe use Lemma 2 to show that the allocation policy selected by the learner based on the empiricaldistribution is almost optimal in expectation.
The details can be found in the Appendix B.
6.2 Approximate-Bayesian Incentive Compatibility (Theorem 1)
Theorem 2 says that online utility of each agent cannot be too far below the ofﬂine optimum ifeveryone behaves truthfully.
In order to show approximate-BIC, it sufﬁces to show that the strategicagent cannot gain too much more than the ofﬂine optimum.
To do so, we need to bound both theshort term and longer term incentives for the agent to be strategic.
6.2.1 Short term incentive
We start with bounding the short term strategic incentive.
We ﬁrst show that if agent reports from anaverage distribution that is very different from the truthful distribution, then with high probability
Algorithm 2 can detect that.
Note that given the strategic agent’s strategy in a given round, hisreported value is drawn from a distribution potentially different from F .
Lemma 3.
Fix a time step t .
Let  =
[Equation]
Let F s , s = 1 , . .
, t be the strategic agent’sreported value distributions in each time step given the history, i.e., F s ( x ) :
= P ( ˜ X i,s  x |H s ) .
Ifthe average distribution ¯ F = 1 t
P ts =1 F s is such that sup x | ¯ F ( x )  F ( x ) |   , then Algorithm 1 281 will terminate at or before time t with probability at least  .
Next, we show that if the agent restricts the reported distribution to not deviate more than  fromthe true distribution (so that the deviation may go undetected by the detection algorithm), then thepotential gain in the agent’s utility compared to truthful reporting is upper bounded by ¯ x  .
Thisbounds the agent’s incentive to be strategic.
Lemma 4.
Fix a round t and a single strategic agent i , so that the remaining agents are truthful, i.e., ˜ X j,t = X j,t , 8 j
=
i .
Let F r ( · ) denote the marginal distribution of values ˜
X
i,t reported by the strategic agent i at time t conditional on the history, i.e.,
[Equation]
Suppose that sup x | F ( x )  F r ( x ) |   .
Then, at any time t , given any greedy allocation policy  ,
[Equation]
Note that F r speciﬁes only the marginal distribution of ˜ X i,t |H t and not the joint distribution of( ˜ X t , X t ) |H
t .
Indeed the above lemma claims that the given bound on utility gain holds for allpossible joint distributions as long as the marginal F r of ˜ X i,t |H t is at most  away from F .
Intuitively, Lemma 3 and Lemma 4 together bound the agent’s short term incentive to be strategic:if the agent deviates from the truthful distribution too much, then the mechanism will terminateearly and the agent will lose out on all the future utility (Lemma 3); and given any greedy allocationstrategy set by the central planner, we have that if the agents deviates within the undetectable rangeof Algorithm 2, then the gain in utility compared to acting truthfully is small (Lemma 4).
Next, webound an agent’s incentive to lie in order to make the mechanism learn a suboptimal greedy allocationpolicy that is more favorable to the agent.
6.2.2 Long term incentive
In order to bound the longer term incentive to be strategic, we want to show that despite agent i beingstrategic, the central planner can still learn an allocation policy that closely approximates the ofﬂineoptimal allocation policy.
This means that the agent’s inﬂuence over the central planner’s allocationpolicies is limited.
Lemma 5.
Fix a round T 0  T and a strategic agent i .
If agent i is the only one being strategic, and
Algorithm 2 has not been triggered by the end of time T 0 , then with probability 1   , ˆ  :=
⇤ ( ˆ F T 0 ) satisﬁes
[Equation]
where  T 0 =
[Equation]
In particular, consider T
= L k .
Then the lemma above shows that if an agent was not kicked out bythe end of epoch k  1 , then with high probability the greedy allocation policy in epoch k will besuch that the agents’ expected utility by being truthful is close to what he would have received in theofﬂine optimal solution (Lemma 5).
We can now combine this result with Lemma 3 and Lemma 4 tobound the utility that any single strategic agent can gain over the entire trajectory.
Lemma 6.
If agent i is the only one being strategic, then with probability 1   , agent
i ’s online utility is upper bounded by
[Equation]
Here k t denotes the epoch number that time step t lies in, and ˜
k t denotes the allocation policy usedby the central planner in that epoch.
Proof of Theorem 1 We have already proven Theorem 2 which bounds individual regret deﬁnedas the difference T E [ u i ( X , X ,  ⇤ )]
P
Tt =1 u i ( X t , X t ,  k t ) , i.e., the difference between the utilityof agent i under the ofﬂine optimal policy and that under the allocation policy learned by thealgorithm when all the agents are truthful .
The proof of Theorem 1 follows from plugging in theupper bound on T E [ u i ( X , X ,  ⇤ )] from this theorem into Lemma 6, to obtain the desired boundon the expression
[Equation]
[Equation]
i.e., on the total gain in utilityachievable by misreporting under our mechanism.
Further details are in Appendix C.
Limitations and Future Directions
Although our goal is to develop mechanisms that are robust to selﬁsh and strategic agents, realapplications often involve bad faith actors that have extrinsic motivation to behave adversarially.
Assuch, deployment of such resource allocation mechanisms to critical applications requires signiﬁcantadditional validation.
In future work we would like to explore the limit of relaxing the i.i.d.
assumptionthat we place on the distribution of valuations across agents.
This is a natural relaxation because if oneagent thinks the item is good then it’s likely that other agents would like the item as well.
Furthermoreit is also conceivable that agents are heterogeneous and so have different value distributions for theitems.
However this seems to require a completely different strategy for detecting, and disincentivizestrategic behaviors, as we can no longer catch the strategic agent through comparing each agent’sreported distribution with that of others.
Checklist
For all authors
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] SeeSection
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots
, ifapplicable?
N /A](b)
Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board (IRB ) approvals, if applicable?
[N/A
[Table]
(c ) Did you include the estimated hourly wage paid to participants and
the total
amountspent on participant compensation?
[N/A