Online Allocation and Learning in the Presence of Strategic Agents
Abstract
We study the problem of allocating T sequentially arriving items among n homogeneous agents under the constraint that each agent must receive a pre-speciﬁed fraction of all items, with the objective of maximizing the agents’ total valuation of items allocated to them.
The agents’ valuations for the item in each round are assumed to be i.i.d.
but their distribution is a priori unknown to the central planner.
Therefore, the central planner needs to implicitly learn these distributions from the observed values in order to pick a good allocation policy.
However, an added challenge here is that the agents are strategic with incentives to misreport their valuations in order to receive better allocations.
This sets our work apart both from the online auction design settings which typically assume known valuation distributions and/or involve payments, and from the online learning settings that do not consider strategic agents.
To that end, our main contribution is an online learning based allocation mechanism that is approximately Bayesian incentive compatible, and when all agents are truthful, guarantees a sublinear regret for individual agents’ utility compared to that under the optimal ofﬂine allocation policy.
Introduction
A classic sequential resource allocation problem is to allocate T sequentially arriving items to n agents, where each agent must receive a predetermined fraction of the items.
The goal is to maximize social welfare, i e., the agents’ total valuation of the items allocated to them.
This problem is nontrivial even when the agents’ valuations are stochastic and i.i.d. with a known distribution, the main difﬁculty being that the allocations must be performed in real-time; speciﬁcally, an item must be allocated to an agent in the current round without knowledge of their future valuations
36th Conference on
A more challenging (and quite useful) extension of the problem which has been the focus of recent literature considers the case where the distribution of the agents’ valuations is apriori unknown to the planner.
In such settings, algorithms based on online learning can be used to adaptively learn the valuation distribution from observed valuations in previous rounds, and improve the allocation policy over time (see [2, 10, 6, 5] for some examples).
However, these mechanisms implicitly assume that the agents report their valuations truthfully, so that the mechanism can directly learn from the reported valuations in order to maximize the social welfare.
Many practical resource allocations settings do not conform with the truthful reporting assumption.
In particular, selﬁsh and strategic agents may have an incentive to misreport their valuations if that can lead to individual utility gain (possibly at the expense of social welfare).
Hence, an allocation policy that does not take such misreporting incentives into account can incur signiﬁcant loss in social welfare in presence of strategic agents.
For example, consider a simple setting with two agents whose true valuations are i.i.d.
and uniformly distributed between 0 and 1.
That is,
[Equation]
Each agent is pre-determined to receive an equal fraction of all the items.
The optimal welfare maximizing allocation policy is to allocate the item to the agent with higher valuation in (almost) every round.
This policy results in T/ 3 expected utility ( E [ X 1 | X 1 > X 2 ] / 2 ) for each agent, and a social welfare of 2 T/ 3 .
However, suppose that the ﬁrst agent chooses to misreport in the following way: the agent reports a high valuation of 1 whenever her true valuation is in [0 . 5 , 1] and a low valuation of 0 whenever her true valuation is in [0 , 0 . 5] .
Assuming the other agent remains truthful, this will lead to the ﬁrst agent receiving all the items in her top 1 / 2 quantile, and therefore a signiﬁcantly increased utility of 3 T/ 8 ( E [ X 1 | X 1 > 0 .
5] / 2 ) compared to T/ 3 under truthful reporting.
The social welfare however, goes down to 5 T/ 8 in this case.
Thus under the optimal policy, each agent has an incentive to misreport her valuations in order to gain individual utility.
The incentives to misreport may be further ampliﬁed under an online learning based allocation algorithm that learns approximately optimal policies from the valuations observed in the previous rounds.
In such settings, the agents can potentially mislead the online learning algorithm to learn a more favorable policy over time by repeatedly misreporting their values.
Motivated by these shortcomings, in this paper, we consider the problem of designing an online learning and allocation mechanism in the presence of strategic agents .
Speciﬁcally, we consider the problem of sequentially allocating T items to n strategic agents.
The problem proceeds in T rounds.
In each round t = 1 , . . .
, T , the agents’ true valuations
X
i,t , i = 1 , . . . , n for the t th item are generated i.i.d.
from a distribution F a priori unknown to the central planner.
However, the central planner can only observe a value
˜ X i,t reported by each agent i , which may or may not be the same as her true valuation X i,t for the item.
Using the reported valuations from the current and previous rounds, the central planner needs to make an irrevocable decision of who to allocate the current item.
The allocations should be made in a way such that each agent at the end receives a ﬁxed fraction p ⇤ i of the T items, where p ⇤ i > 0 ,
P [-ni =
-]n i =1 p
⇤ i
= 1 .
The objective of the central planner is to maximize the total utility of the agents, where utility of each agent is deﬁned as the sum total true valuations of items received by the agent.
Our main contribution is a mechanism that achieves both: (a) Bayesian incentive compatibility , i.e., assuming all the other agents are truthful, with high probability no single agent can gain a signiﬁcant utility by deviating from the truthful reporting strategy, and; (b) near-optimal regret guarantees , namely, the utility of each individual agent under the online mechanism is “close" to that achieved under the optimal ofﬂine allocation policy.
Organization After discussing the related literature in some further detail in Section 2, we formally introduce the problem setting and some of the core concepts in Section 3.
Section 4 describes our online learning and allocation algorithm and provides formal statements of our main results (Theorem 1 and Theorem 2).
Section 5 and Section 6 provide an overview of the proofs of the above theorems.
All the missing details of the proofs are provided in the appendix.
Finally, in Section 7 we discuss some limitations and future directions.
Literature review
Our work lies at the intersection of online learning and mechanism design.
From an online learning perspective, our setting is closely related to the recent work on constrained online resource allocation under stochastic i.i.d.
rewards/costs (e.g., see [10, 2, 1, 6, 5]).
However, a crucial assumption in those settings is that the central planner can observe the true rewards/costs of an allocation, which in our setting would mean that the central planner can observe agents’ true valuations of the items being allocated.
Our work extends these settings to allow for selﬁsh and strategic agents who may have incentives to misreport their valuations.
As discussed in the introduction, unless the online allocation mechanism design takes these incentives into account, selﬁsh agents may signiﬁcantly misreport their valuations to cause signiﬁcant loss in social welfare.
Incentives and strategic agents have been previously considered in online allocation mechanism design, however, most of that work has focused on auction design where payments are used as a key mechanism for limiting rational agents’ incentives to misreport their valuations.
For example,
Amin, Rostamizadeh, and Syed [3] study a posted-price mechanism in a repeated auction setting where buyers’ valuations are context dependent.
Golrezaei, Javanmard, and Mirrokni [12] extend this work to multi-buyer setting, using second-price auction with dynamic personalized reserve prices.
Kanoria and Nazerzadeh [17] study a similar problem in a non-contextual setting.
(There is also a signiﬁcant literature that studies learning in repeated auction settings from the bidder’s perspective.
Since our paper focuses on the central planner’s point of view, we omit references to that literature. )
All of the above-mentioned works are concerned with maximizing revenue for the seller, and use money/payments as a key instrument for eliciting private information about the bidder’s valuations for the items.
In this paper, we are concerned with online allocation without money, and the goal is to maximize each agent’s utility.
Recently, there has been some work on studying reductions from mechanism design with money to those without money.
Gorokh, Banerjee, and Iyer [13] provided a black-box reduction from any oneshot, BIC mechanism with money, to an approximately BIC mechanism without money.
However, their reduction relies crucially on knowing the true value distribution of agents and therefore is not applicable to our setting.
Procaccia and Tennenholtz [18] consider a speciﬁc (one-round) facility allocation problem and explicitly formulate the idea of designing mechanisms without money to achieve approximately optimal performance against mechanisms with money.
Subsequently, there is a series of papers that extended the results on mechanism design without money in a single shot setting, when the bidders’ value distribution is unknown [14, 16, 8, 9, 8].
These papers either use a very restricted setting with just two agents, or use very speciﬁc/simple valuation functions for the agents.
Even in these basic settings, they show that the best one can hope for is a constant approximation to what one can achieve with a mechanism that uses money.
It is not clear what kind of regret guarantees such reductions imply in a repeated online learning setting.
Therefore, we do not consider such reductions from auction mechanisms with money to be a fruitful direction for achieving our goals of both incentive compatibility and low (sublinear) regret for our online allocation problem.
Finally, in a repeated allocation settings with known valuation distribution, there are more positive results for truthful mechanism design without money.
For example, Guo, Conitzer, and Reeves [15] and later Balseiro, Gurkan, and Sun [7] studied the problem of repeatedly allocating items to agents with known value distributions; both use a state-based “promised utility” framework.
To summarize, to the best of our knowledge, this is the ﬁrst paper to incorporate strategic agents’ incentives in the well-studied online allocation problem with stochastic i.i.d.
rewards and unknown distributions.
Thus, it bridges the gap between the online learning and allocation literature which focuses on non-strategic inputs, and the work on learning in repeated auctions which focuses on allocation mechanisms that utilize money (payments) to achieve incentive compatibility.
Problem formulation The ofﬂine problem
We ﬁrst state the ofﬂine version of the problem which will serve as our benchmark for the online problem.
There is a set of n agents, and a distribution F over X : =
[0 , ¯ x ] n .
Each draw X ⇠ F from this distribution represents the n agents’ valuations of one item: X =
[ X 1 , . . .
, X n ] .
We assume that the agents’ valuations are i.i.d., i.e.
[Equation]
A matching policy (aka allocation policy) maps, potentially with some exogenous randomness, a value vector X to one of the agents i 2 { 1 , . . . , n } .
Speciﬁcally, given a realized value vector X 2 [0 , ¯ x ] n , a (possibly randomized) policy ⇡ maps X to agent ⇡ ( X ) 2 { 1 , . . . , n } , with the probability of agent i receiving an allocation given by P ( ⇡ ( X ) = i ) .
The ofﬂine optimization problem is to ﬁnd a social welfare-maximizing policy ⇡
⇤ such that each agent i in expectation receives a predetermined fraction p
⇤ i of the pool of items, where p ⇤ i > 0 ,
[Equation] = 1 .
The problem of ﬁnding optimal policy can therefore be stated as the following
[Equation]
where the expectations are taken both over X ⇠ F and any randomness in the mapping made by policy ⇡ given X .
Solving the ofﬂine problem is non-trivial, as it is an inﬁnite dimensional optimization problem as stated in its’ current form in (1).
But it turns out to be closely related to
Semi-Discrete Optimal Transport, and that the dual of (1) can be written as
[Equation]
where L i is what is sometimes referred to as the Laguerre cell :
[Equation]
Let  ⇤ ( F ) denote an optimal solution to (2).
When it is clear from the context, we omit the distribution
F .
It is known that an optimal solution to (1) is given by the following deterministic policy deﬁned by the Laguerre cell partition (Proposition 2.1 [4]):
[Equation]
More generally, we will refer to any policy deﬁned by a Laguerre cell partition as a greedy policy.
Deﬁnition 1 (Greedy allocation policy) .
Consider any allocation policy that partitions the domain [0 , ¯ x ] n as L i (  ) (as deﬁned in (4) ) for some  2 R n .
We refer to such a policy as the greedy allocation policy with parameter  .
Note that there are efﬁcient algorithms for solving (2) (see[4]) when the distribution F is known.
Therefore we will treat  ⇤ ( · ) as a black-box that can be computed efﬁciently for any given input distribution ˆ F .
The online problem: approximate Bayesian incentive compatibility and regret
We are interested in the case when items are sequentially allocated over T rounds, and that the distribution F is initially unknown.
Speciﬁcally, in each round t = 1 , . .
, T , the agents’ true valuations X t =
( X i,t , i = 1 , . . . , n ) are generated i.i.d.
from the distribution F a priori unknown to the central planner.
However, the central planner does not observe X t but only observes the reported valuations ˜ X t
= ( ˜ X i,t , i = 1 , . . . , n ) which may or may not be the same as the true valuations.
An online allocation mechanism consists of a sequence of allocation policies ⇡ 1 , . . .
, ⇡ t where the policy ⇡ t at time t may be adaptively chosen based on the observed information until before time t :
[Equation]
[Equation]
Given allocation policy ⇡ t at time t , the agent i ’s utility at time t is given by
[Equation]
If an agent i has already reached his target allocation of p ⇤ i T items, then he cannot be allocated more items since the allocation policy be
, for any given value vector X , ⇡ t ( X ) is a random variable.
To ensure truthful reporting in presence of strategic agents, we are interested in mechanisms that are (approximately) Bayesian incentive compatible.
Deﬁnition 2 (Approximate-BIC) .
For an online allocation mechanism, let ⇡ t , t = 1 , . .
, T be the sequence of allocations when all agents report truthfully, i.e., when ˜ X t =
X t , 8 t ; and let ˜ ⇡ it , t = 1 , . .
, T be the sequence when all agents except i report truthfully, i.e., X j,t = ˜
X j,t , 8 j 6 = i .
Then the online allocation mechanism is called ( ↵ ,  ) -approximate
Bayesian Incentive Compatible if, for all i = 1 , . .
. , n , with probability at least 1   ,
[Equation]
Here, the probability is with respect to the randomness in true valuations X t ⇠ F and any randomness in the online allocation policy.
For the online policy to be approximate-BIC, the statement should hold for all possible misreporting of valuations ˜
X i,t 6 =
X i,t .
Therefore, if ↵ is small, then an individual agent has little incentive to strategize.
Note that approximate-BIC also implies that truthful reporting for all agents constitutes an approximateNash equilibrium.
Assuming that all agents are truthful, we are also interested in bounding each individual agent’s regret.
Deﬁnition 3 (Individual regret) .
We deﬁne an individual agent i ’s regret under an online allocation mechanism as the difference between agent i ’s realized utility over T rounds and the expected utility achieved in the ofﬂine expected problem.
That is,
[Equation]
Here ⇡ 1 , . .
, ⇡ T denote the allocation policies used by the online allocation mechanism in round t = 1 , . . .
, T .
Note that since social welfare is given by the sum of all agents’ utilities, a bound on individual regret implies a bound on the regret in social welfare of the mechanism.
Algorithm and main results
We present an online allocation mechanism that is approximately-BIC, and further achieves low regret guarantees on individual regret when all agents are truthful.
Our algorithm contains two components: a learner, and a detector.
Intuitively, the detector makes sure that the mechanism is approximately
BIC, and the learner adaptively learns utility-maximizing allocation policies assuming truthful agents.
The learner runs in epochs with geometrically increasing lengths.
The starting time of each epoch k is given by L k = 2 k , k = 0 , 1 , . . .
, which is also when the allocation policy is updated.
At the end of each epoch (i.e., at time t =
L k  1 for epoch k ), the learner takes all the previously reported values from all the agents, and uses them to construct an empirical distribution of the agents’ valuations.
The learner implicitly assumes truthful agents in its computations.
Therefore, since the agents’ true valuations are i.i.d.
, it ﬁrst constructs a single, one-dimensional empirical distribution ˆ F t , and then uses it to construct the corresponding n -dimensional distribution ˆ F t :
[Equation]
The learning algorithm then solves the ofﬂine problem (2) using ˆ F t , and uses the resulting greedy allocation policy characterized by  ⇤ ( ˆ F t ) to allocate the items in the following epoch.
In parallel to the learner, the detector constructs and monitors, in each time step t , and for each agent i , two empirical distributions.
One using the reported valuations from agent i : ¯ F t , and one using the reported valuations from all the other agents: ˜ F t .
The detection algorithm then computes the supremum between the two empirical CDFs, sup x | ¯ F t ( x )  ˜ F t ( x ) | .
If this difference is greater than a predetermined threshold  t , then the detector raises a ﬂag that there has been a violation of truthful reporting and the entire allocation game stops.
Otherwise, the process continues.
The threshold  t needs to be chosen such that if everyone is truthful, then with high probability the detection algorithm will not pull the trigger.
At the same time, if someone deviates from truthful reporting signiﬁcantly, then it should detect this with high probability.
The typical concentration result used in comparing empirical CDFs is the Dvoretzky-Kiefer-Wolfowitz (DKW) inequality[11].
However, since a strategic agent can adaptively change its’ misreporting strategy, we cannot directly apply the DKW inequality, which assumes i.i.d.
samples.
Instead, we use martingale version of the
DKW inequality (Lemma 1), and use that to choose an appropriate threshold  t .
The details are given in Algorithm 1 and Algorithm 2.
Input: T,
Initialize:  = [0 , . .
. , 0] , k = 0 , K = log 2 ( T ) , L k = 2 k , k = 0 , . .
, K ; 1 for t 1 , 2 , 3 , . .
, T do
Observe X t .
Run Detection Algorithm (Algorithm 2) with sample set S =
{ X 1 , . . .
, X t } , and threshold  t =q 1 t log( 256 et  ) if Detection Algorithm Return Reject then
Terminates .
if one of the agents i 2 { 1 , . . . , n } has reached the allocation capacity p
⇤ i T then
Allocate randomly among agents who have not reached capacity else
Allocate item greedy allocation policy  if
1 then
Compute samples { X
, X   ⇤ ( ˆ F t ) k
k
+ Input: Sample
1 for i , n do
Compute
P ts =
[ X si  x ] as the empirical CDF of the samples
collected from agent i Compute ˜ F t ( x ) = 1 t ( n  1) P ts = P
j 6 = i [ X sj  x ] be the empirical CDF of all reported values from the other agents.
if sup x | ˜ F t ( x )  ¯ F t ( x ) |   t 2 then Return Reject 6 Return Accept
Our main results are the following guarantees on incentive compatibility and regret of our online allocation algorithm.
Theorem 1 (Approximate-BIC) .
Algorithm 1 is ( O ( p nT log( nT/  )) ,  ) -approximate BIC.
Since truthful reporting constitutes an approximate equilibrium, it is reasonable to then assume that agents will act truthfully.
We show the following individual regret bound assuming truthfulness.
Theorem 2 (Individual Regret) .
Assuming all agents report their valuations truthfully, then under the online allocation mechanism given by Algorithm 1, with probability 1   , every agent i ’s individual regret can be bounded as:
[Equation]
Showing approximate incentive compatibility, and then guaranteeing regret under the assumption of truthfulness, is an approach commonly seen in the online mechanism design literature (e.g. Theoremin [17]).
In the next section, we describe the high level proof ideas for the main results above.
Proof ideas
Proof ideas for Theorem
We establish that the mechanism is approximately BIC by showing that no single agent has incentive to signiﬁcantly deviate from reporting true valuations if all the other agents are truthful.
The proof consists of two parts.
In Step 1, we prove that any signiﬁcant deviation from the truth can be detected and will lead the mechanism to terminate.
In Step 2,3, we prove that in order to achieve a signiﬁcant gain in utility, an agent indeed has to report values that signiﬁcantly deviate from the truth.
Step 1 Assuming that there is only one (unidentiﬁed) strategic agent while all the other agents are truthful, we ﬁrst show that if the detector does not trigger a violation by time t then with high probability, the empirical distribution of valuations reported by the strategic agent is no more than
O (1 / p t ) away from the true distribution (Lemma 3).
The key observation here is that since the agents’ valuations are i.i.d.
, we can compare their reported values to detect if any single agent’s distribution is signiﬁcantly different from everyone else’s.
A technical challenge in making statistical comparisons here is that the strategic agent can adaptively change their reporting strategy over time based on the realized outcomes.
Therefore, we derive a novel martingale version of the DKW inequality to show concentration of the empirical distribution relative to the true underlying distribution.
Step
In a given round, given the history, the mechanism’s allocation policy is a ﬁxed greedy allocation policy given by  .
If the distribution of strategic agent’s reported values differs from the true distribution by at most  , then the agent’s expected utility gain in that round, compared to reporting truthfully, is at most O (  ) , (see Lemma 4).
Step
If over t rounds, the distribution of the strategic agent’s reported values is at most  away from the true distribution, then the learning algorithm will, with high probability ﬁnd an allocation policy that is at most O ( p n  ) away (in terms of individual utility) from what it would have learned if all the agents were truthful instead (see Lemma 2).
To understand the signiﬁcance and distinction between results in Step 2 vs. Step 3, note that a strategic agent has two separate ways to gain utility.
The ﬁrst is to report valuations in a way that the agent immediately wins more/better items under the central planner’s current allocation policy.
However, since the central planner is updating its’ allocation policy over time, the strategic agent can also misreport in a way that beneﬁts its’ future utility , by “tricking” the central planner into learning a policy that favors him later on.
Together, Step 2 and Step 3 show that the agent cannot gain signiﬁcant advantage over being truthful in either manner.
In many existing works on online auctions mechanisms design, where the central planner dynamically adjusts the reserve price over time, these two types of strategic behaviors are in conﬂict: the agent either sacriﬁces future utility to gain immediate utility; or sacriﬁces near-term utility for future utility.
The results in those settings therefore often rely on this observation to show approximate incentive compatibility.
In our case however, since there is no money involved, it is not clear if such a conﬂict between short and long term utility exists.
Nonetheless, we are able to bound the agents’ ability to strategize.
Step 2 bounds the agent’s short term incentive to be strategic, whereas Step 3 bounds the longer term incentive to be strategic.
Combining these steps gives us a proof for Theorem 1.
Proof ideas for Theorem 2 Recall that here we assume all agents’ are truthful.
The proof involves two main steps.
Step 1 We show that uniformly for any t = 1 , . .
, T , with high probability, the empirical distribution constructed from the ﬁrst t samples is close (within a distance of ˜ O
(1 / p nt ) ) to the true distribution F .
Here, the factor of 1 / p n comes from the fact that in each round we observe n independent samples from the value distribution, one from each of the agents.
This also implies that if all the agents are reporting truthfully, then, with high probability, the detector will not falsely trigger.
Step
We show that the allocation policy learned under the empirical distribution estimated from the samples is close to the the optimal allocation policy (Lemma 2).
Speciﬁcally, after t rounds if the empirical CDF is at most O (1 / p nt ) away from the true distribution, then each agents’ expected utility in one round under the learned allocation policy is at most p n/t away (both from above and from below) from the optimal.
By using an epoch based approach we can then show that each agents’ individual regret is with high probability bounded by O ( p nT ) over the entire planning horizon T .
Proof details
We will now outline our proof in more detail.
All missing proofs can be found in the Appendix.
First we state the following martingale variation of the well-known Dvoretzky-Kiefer-Wolfowitz (DKW) inequality[11].
This is critical when dealing with strategic agents as they can adapt their strategy over time, resulting in non-independent (reported) values.
Lemma 1 (Martingale Version of DKW Inequality) .
Given a sequence of random variables
Y 1 , . . .
, Y T , let F t =
( Y 1 , . .
, Y t ) , t = 1 , . . .
, T be the ﬁltration representing the information in the ﬁrst t variables.
Let F t ( y ) :
= Pr( Y t  y |F t  1 ) , and ¯ F T ( y ) :
= 1 T
P
Tt =
[ Y t  y ] .
Then,
[Equation]
Next we introduce a new notation to to denote the fraction of allocation that j receives under the greedy allocation policy with parameter  and valuation distribution F :
[Equation]
We start with proving Theorem 2, as we will use this to prove Theorem 1 later.
6.1 Individual Regret Bound (Theorem 2)
In Algorithm 1, the allocation policy is trained on the empirical distribution constructed from samples.
We want to show that this difference between empirical and population distribution will not impair the performance of the resulting allocation policy too much.
Lemma 2.
Let G
= G 1 ⌦ . . .
⌦ G n , and F = F 1 ⌦ . . .
⌦ F n be two distributions over [0 , ¯ x ] n where the marginals on each coordinate are independent.
Suppose sup x | F i ( x )  G i ( x ) |   8 i .
Let  =  ⇤ ( G ) , and
⇤ =
⇤ ( F ) .
Then
[Equation]
Proof of Theorem
Now we have the main ingredients for Theorem 2.
We use the DKW inequality to show that the empirical distribution constructed in (7) is close to the true distribution w.h.p..
Then we use Lemma 2 to show that the allocation policy selected by the learner based on the empirical distribution is almost optimal in expectation.
The details can be found in the Appendix B.
6.2 Approximate-Bayesian Incentive Compatibility (Theorem 1)
Theorem 2 says that online utility of each agent cannot be too far below the ofﬂine optimum if everyone behaves truthfully.
In order to show approximate-BIC, it sufﬁces to show that the strategic agent cannot gain too much more than the ofﬂine optimum.
To do so, we need to bound both the short term and longer term incentives for the agent to be strategic.
6.2.1 Short term incentive
We start with bounding the short term strategic incentive.
We ﬁrst show that if agent reports from an average distribution that is very different from the truthful distribution, then with high probability
Algorithm 2 can detect that.
Note that given the strategic agent’s strategy in a given round, his reported value is drawn from a distribution potentially different from F .
Lemma 3.
Fix a time step t .
Let  =
[Equation]
Let F s , s = 1 , . .
, t be the strategic agent’s reported value distributions in each time step given the history, i.e., F s ( x ) :=
P ( ˜ X i,s  x |H s ) .
If the average distribution ¯ F = 1 t
P ts =1 F s is such that sup x | ¯ F ( x )  F ( x ) |   , then Algorithm 1 will terminate at or before time t with probability at least  .
Next, we show that if the agent restricts the reported distribution to not deviate more than  from the true distribution (so that the deviation may go undetected by the detection algorithm), then the potential gain in the agent’s utility compared to truthful reporting is upper bounded by ¯ x  .
This bounds the agent’s incentive to be strategic.
Lemma 4.
Fix a round t and a single strategic agent i , so that the remaining agents are truthful, i.e., ˜ X j,t = X j,t , 8 j
=
i .
Let F r ( · ) denote the marginal distribution of values ˜
X
i,t reported by the strategic agent i at time t conditional on the history, i.e.,
[Equation]
Suppose that sup x | F ( x )  F r ( x ) |   .
Then, at any time t , given any greedy allocation policy  ,
[Equation]
Note that F r speciﬁes only the marginal distribution of ˜ X i,t |H t and not the joint distribution of ( ˜ X t , X t ) |H
t .
Indeed the above lemma claims that the given bound on utility gain holds for all possible joint distributions as long as the marginal F r of ˜ X i,t |H t is at most  away from F .
Intuitively, Lemma 3 and Lemma 4 together bound the agent’s short term incentive to be strategic: if the agent deviates from the truthful distribution too much, then the mechanism will terminate early and the agent will lose out on all the future utility (Lemma 3); and given any greedy allocation strategy set by the central planner, we have that if the agents deviates within the undetectable range of Algorithm 2, then the gain in utility compared to acting truthfully is small (Lemma 4).
Next, we bound an agent’s incentive to lie in order to make the mechanism learn a suboptimal greedy allocation policy that is more favorable to the agent.
6.2.2 Long term incentive
In order to bound the longer term incentive to be strategic, we want to show that despite agent i being strategic, the central planner can still learn an allocation policy that closely approximates the ofﬂine optimal allocation policy.
This means that the agent’s inﬂuence over the central planner’s allocation policies is limited.
Lemma 5.
Fix a round T 0  T and a strategic agent i .
If agent i is the only one being strategic, and
Algorithm 2 has not been triggered by the end of time T 0 , then with probability 1   , ˆ  :=
⇤ ( ˆ F T 0 ) satisﬁes
[Equation]
where  T 0 =
[Equation]
In particular, consider T
= L k .
Then the lemma above shows that if an agent was not kicked out by the end of epoch k  1 , then with high probability the greedy allocation policy in epoch k will be such that the agents’ expected utility by being truthful is close to what he would have received in the ofﬂine optimal solution (Lemma 5).
We can now combine this result with Lemma 3 and Lemma 4 to bound the utility that any single strategic agent can gain over the entire trajectory.
Lemma 6.
If agent i is the only one being strategic, then with probability 1   , agent
i ’s online utility is upper bounded by
[Equation]
Proof of Theorem 1 We have already proven Theorem 2 which bounds individual regret deﬁned as the difference T E [ u i ( X , X ,  ⇤ )]
P
Tt =1 u i ( X t , X t ,  k t ) , i.e., the difference between the utility of agent i under the ofﬂine optimal policy and that under the allocation policy learned by the algorithm when all the agents are truthful .
The proof of Theorem 1 follows from plugging in the upper bound on T E [ u i ( X , X ,  ⇤ )] from this theorem into Lemma 6, to obtain the desired bound on the expression P [Equation] P
[Equation]
i
[Equation]
X t , X t ,  k t ) , i.e., on the total gain in utility achievable by misreporting under our mechanism.
Further details are in Appendix C.
Limitations and Future Directions
Although our goal is to develop mechanisms that are robust to selﬁsh and strategic agents, real applications often involve bad faith actors that have extrinsic motivation to behave adversarially.
As such, deployment of such resource allocation mechanisms to critical applications requires signiﬁcant additional validation.
In future work we would like to explore the limit of relaxing the i.i.d.
assumption that we place on the distribution of valuations across agents.
This is a natural relaxation because if one agent thinks the item is good then it’s likely that other agents would like the item as well.
Furthermore it is also conceivable that agents are heterogeneous and so have different value distributions for the items.
However this seems to require a completely different strategy for detecting, and disincentivize strategic behaviors, as we can no longer catch the strategic agent through comparing each agent’s reported distribution with that of others. Acknowledgement
This work was supported in part by an NSF CAREER award [CMMI 1846792] awarded to author S.
Agrawal.
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See Section(c)
Did you discuss any potential negative societal impacts of your work?
[Yes] See Section(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[N/A] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]