Nonstationary Dual Averaging and Online Fair Allocation
Abstract
We consider the problem of fairly allocating sequentially arriving items to a set of individuals.
For this problem, the recently-introduced PACE algorithm leverages the dual averaging algorithm to approximate competitive equilibria and thus generate online fair allocations.
PACE is simple, distributed, and parameter-free, making it appealing for practical use in large-scale systems.
However, current performance guarantees for PACE require i.i.d. item arrivals.
Since real-world data is rarely i.i.d., or even stationary, we study the performance of PACE on nonstationary data.
We start by developing new convergence results for the general dual averaging algorithm under three nonstationary input models: adversariallycorrupted stochastic input, ergodic input, and block-independent (including periodic) input.
Our results show convergence of dual averaging up to errors caused by nonstationarity of the data, and recover the classical bounds when the input data is i.i.d.
Using these results, we show that the PACE algorithm for online fair allocation simultaneously achieves “best of many worlds” guarantees against any of these nonstationary input models as well as against i.i.d.
input.
Finally, numerical experiments show strong empirical performance of PACE against nonstationary inputs.
Introduction
In fair division, the goal is to allocate a set of items, typically assumed divisible, among a set of agents with heterogeneous preferences, while guaranteeing fairness and efﬁciency properties.
In this paper we are interested in how to fairly and efﬁciency allocate items that arrive online : at every time step one item arrives and we must irrevocably assign it to some agent.
Recently, there has been a growing literature on such online fair allocation problems (Azar et al., 2016; Balseiro et al., 2020;
Gao et al., 2021; Bateni et al., 2021; Sinclair et al., 2021; Banerjee et al., 2022).
Real-world systems that can be captured by such settings include Internet advertising systems, job recommender systems, cloud computing platforms, and many more.
One of the key challenges in such problems is to balance the (often conﬂicting) goals of overall efﬁcient resource utilization with fairness guarantees for the individual agents.
For this setting, Gao et al.
(2021) shows that a simple mechanism called PACE (Pace According to
Current Estimated utility) generates asymptotically fair and efﬁcient allocations when the item arrivals are drawn in an i.i.d. manner.
PACE gives each agent a per-time-step budget of faux currency, and the fair allocation is achieved by having agents participate in ﬁrst-price auctions for each item, using the faux money.
By guaranteeing that each agent asymptotically spends their budget at the correct rate, the resulting allocations and prices converge to what is known as a competitive equilibrium from equal incomes (CEEI), which guarantees both fairness and efﬁciency.
In PACE, each agent maintains a pacing multiplier to control their spending over time, and the pacing multipliers are updated based on buyers’ budgets and cumulative utilities.
This is similar to how budget-management systems work in Internet ad auctions.
PACE is highly decentralized due to its auction-based allocation, it does not require dividing the item, and it is also completely parameter free.
This makes it suitable for large-scale practical implementation.
Yet in many large-scale settings, such as the context of fair recommender systems (Kroer et al., 2021;
Kroer and Stier-Moses, 2022) or Internet advertising, we would not expect items to be drawn i.i.d.
from a single distribution.
One alternative is to assume that data arrives adversarially.
However, this leads to very pessimistic negative results and is not an accurate representation of the data one would expect to see in practice.
Instead, one would expect the data to have a strong stochastic component, but with changes over time, e.g., due to ﬂow of trafﬁc, breaking news events, or system updates (Esfandiari et al., 2018; Balseiro et al., 2020).
Motivated by the above considerations, we study online fair allocation when the data exhibits nonstationary behavior.
In particular, we focus on the performance of the PACE algorithm of Gao et al.
We ask
How does PACE behave when nonstationarity is present in the stream of items?
We show that, under several data-input models, the fairness and efﬁciency guarantees of the PACE algorithm are still preserved, up to errors due to the nonstationarity of the data input.
In this sense, we signiﬁcantly extend the main results in Gao et al.
To show these results, we ﬁrst consider the more general setting of nonstationary stochastic optimization and develop new performance guarantees for dual averaging in this setting.
Given the ubiquitous use of dual averaging in online and stochastic optimization, our results are of broader interest beyond (fair) resource allocation.
1.1 Summary of Contributions
Novel convergence results for dual averaging under three nonstationary settings.
We analyze the dual averaging (DA) algorithm for nonstationary stochastic optimization under different data input models, namely, (1) mildly corrupted, (2) ergodic and (3) periodic input data.
Specifically, we consider the composite dual averaging algorithm, where the composite term is strongly convex.
We show that, in all cases, the iterates generated by dual averaging (DA) converge to the optimal solution in mean square, where the bound on the mean-square error decomposes into two terms: i) the typical O (log t=t ) guarantee known from the i.i.d. case, and ii) a term that depends on the amount of nonstationarity in the data input model.
Our results recover the classical bounds under i.i.d.
data input as a special case.
Theoretical fairness and efﬁciency guarantees of PACE for nonstationary item arrivals.
We consider the online fair allocation problem where item arrivals follow any of the three data input models that we consider for DA; these settings generalize the i.i.d.
setting in Gao et al.
Utilizing our convergence results for DA under nonstationary data input models, we show that, for item arrivals following these models, PACE ensures convergence of the pacing multipliers, again with a decomposition into a O (log t=t ) term as well as a term depending on the nonstationarity.
We then show that the agents’ realized utilities, envy, regrets, and expenditures all obtain convergence bounds based on the convergence of pacing multipliers.
Our results show that PACE as an online fair resource allocation algorithm is robust against distributional uncertainty of the input and automatically adapts to many different data input models without any parameter tuning.
In Appendix F we provide numerical experiments which corroborate the above theory and demonstrate the practical efﬁciency of PACE under different data input models.
An extensive review of related work is provided in Appendix A.
Notation.
We use 1 t to denote the vector of ones of length t and e j to denote the vector with one in the j -th entry and zeros in the others.
We use ∆((cid:2)) to denote the space of probability measures on a measurable space (cid:2) , and ∆ n to denote the simplex in R n .
To measure the nonstationarity in the input data, we will use the total variation distance.
Given two probability measures P and Q , it is deﬁned as ∥ P (cid:0) Q ∥ TV :=
(1 = 2) ∫ j d
; where (cid:22) is a supporting measure
Preliminaries on Online Fair Allocation
An online fair allocation instance with inﬁnitely divisible items with n agents and a ﬁnite horizon t consists of a tuple A = ( n; t; (cid:2) ; Q; v ) , where (cid:2) is the (possibly uncountable) measurable space of all possible items, with an associated (cid:27) -algebra M and a probability measure (cid:22) , the distribution
Q 2 ∆((cid:2) t ) is the distribution over possible sequences of items (cid:13) =
( (cid:18) 1 ; : : : ; (cid:18) t ) 2 (cid:2) t , each of unit supply, and the set v = ( v 1 ; : : : ; v n ) 2 L 1+ ((cid:2))
n is the set of valuation functions of the n agents.
Here L 1+ ((cid:2)) is the space of positive integrable functions on (cid:2) .
Agent i sees a utility of v i ( (cid:18) ) in item (cid:18) 2 (cid:2) .
Abusing notation we let v i ( (cid:13) )
= ( v i ( (cid:18) 1 ) ; : : : ; v i ( (cid:18) t ) ) denote the valuation for agent i of items in the sequence (cid:13) .
Let Q (cid:28) be the marginal distribution of the item (cid:18) (cid:28) at time (cid:28) and (cid:22) Q
= (1 =t )
∑ t(cid:28)
=1 Q (cid:28) .
We assume ∫ (cid:2) v i d (cid:22) Q = 1 for all
i 2 [ n ] .
We further assume j v j 1 := max i ∥ v i ∥ 1 < 1 .
We stress that the PACE algorithm that we study is not going to require access to either the valuation functions v or the set of possible items (cid:2) ; these are only required in order to discuss the resulting bounds.
Given an instance A , the decision maker allocates the stream of items (cid:13) one at a time, in an irrevocable manner.
At time (cid:28) when item (cid:18) (cid:28) is revealed, the decision maker must choose an allocation x (cid:28) =
( x (cid:28) 1 ; : : : ; x (cid:28)n ) 2 ∆ n based on information available at that time, and allocate accordingly.
Here the i -th entry of x (cid:28) is the fraction of item (cid:18) (cid:28) allocated to agent i .
On receiving her fraction, agent i realizes a utility of u (cid:28)i := v i ( (cid:18) (cid:28) )
x (cid:28)i .
We let x = ( x 1 ; : : : ; x t ) denote the sequence of allocations made over time.
For agent i , let x i =
( x 1 i ; : : : ; x ti ) 2 R t denote the fraction of items given to agent i across time.
With this notation, the total utility of agent i is ⟨ x
i ; v i ( (cid:13) ) ⟩ .
The goal of the decision maker is to choose, in an online fashion, an allocation x such that it achieves some form of both efﬁciency and fairness guarantees.
2.1 Benchmark: The Hindsight Allocation
As a benchmark, we will consider the hindsight-optimal allocation.
Suppose all items are presented to the decision maker as opposed to arriving one by one.
In that case, a fair and efﬁcient allocation can be found by allocating using the Eisenberg-Gale (EG) convex program Eisenberg and Gale (1959).
EG picks the allocation that maximizes the sum of weighted logarithmic utilities (which is equivalent to maximizing the weighted geometric mean of utilities):
[Equation]
The weights B i represent the priority given to each agent, and they they can be interpreted as budgets in a market-based interpretation of the EG allocation.
We will focus on the case where B i =
t=n for all i , but all our results extend directly to the case of unequal weights, which can be useful in settings such as when buyers have quasilinear utilities Gao et al.
; Conitzer et al.
or when it is desirable to give a larger allocation to certain agents.
The PACE algorithm asymptotically converges to the optimal dual solution, which is
[Equation]
We will also be interested in the underlying problem implied by the average item supplies s =
d (cid:22) Q= d (cid:22) .
Letting ⟨ v i ; x i ⟩ := ∫ (cid:2) v i x i d (cid:22) , this leads to the inﬁnite-dimensional analogue of (1):
[Equation]
We let u (cid:3) denote the optimal utilities in Eq.
The inﬁnite-dimensional analogue of the dual (2) is the following.
For any (cid:14) 0 > 0 , The inﬁnite-dimensional analogue of (2) is the following.
For any (cid:14) 0 >
[Equation]
A rigorous mathematical treatment of the inﬁnite-dimensional program can be found in Gao and
Kroer (2021) and (Gao et al., 2021, Section 2).
Note the additional constraint in Eq.
(4) on (cid:12) does not affect the optimal solution since 1 =n (cid:20) (cid:12) (cid:3) i (cid:20) 1 ; see Lemma 1 in Gao and Kroer (2021).
It is well-known that the hindsight allocation generated by the EG program enjoys the following efﬁciency and fairness properties:
Pareto optimality: we cannot strictly increase any agent’s utility without decreasing some other agents’ utility.
Envy-freeness: each agent prefers their own allocation to that of any other agent: ⟨ v i ( (cid:13) ) ; x (cid:3) i ⟩ (cid:21) ⟨ v i ( (cid:13) ) ; x (cid:3) k ⟩ for all k ̸
= i .
Proportionality: every agent achieves at least as much utility as under the uniform allocation,
i
e. ⟨ v i ( (cid:13) ) ; x (cid:3) i ⟩ (cid:21) ⟨ v i ( (cid:13) ) ; (1 =n )1 t ⟩ .
Therefore the hindsight EG allocation is the gold standard that we assume the decision maker would use if she had known the sequence of items (cid:13) in advance.
However, in the online setting the decision maker does not know this sequence, and must therefore instead attempt to approximate an equally good allocation in online fashion.
For an item sequence (cid:13) , we let x (cid:13) denote the optimal hindsight allocation, which is an optimal solution to Eq.
and we denote the resulting total and average utility as
[Equation]
2.2 Performance Metrics
We measure the performance of an online allocation rule x on the instance (cid:13) via the following two quantities.
The regret of agent i is the difference between the total hindsight equilibrium utility U (cid:13)i and their realized utilities u (cid:28)i under x
[Equation]
The envy of agent i is the maximal extent to which they prefer the allocation of any other agent:
[Equation]
We seek to understand the worst-case behavior of an algorithm when facing a certain class of input distributions.
For a given input distribution class C (cid:26) ∆((cid:2) t ) , we will develop bounds on the worstcase regret and envy under any distribution in C :
[Equation]
2.3 The PACE Algorithm
In this section, we review the PACE (Pace According to Current Estimated Utility) dynamics (Gao et al., 2021).
In PACE, each item is allocated via ﬁrst-price auction, and each agent constructs bids by scaling their value by a pacing multiplier .
The pacing multipliers are maintained using simple, distributed updates that can be handled either by the agents or by the platform.
Algorithmic details are displayed in Algorithm 1.
Here (cid:5) [ ℓ;h ] [ x ] = max f ℓ; min f x; h gg .
At every time step (cid:28) an item (cid:18) (cid:28) is revealed.
At that point every agent comes up with a bid for that item, which is equal to their value for the item multiplied by their current pacing multiplier (cid:12) (cid:28)i .
Then, the agents submit these bids to a ﬁrst-price auction, and the item is allocated to the highest bidder.
For concreteness, we choose the bidder with the smallest index if a tie occurs, but any rule works.
Each agent then observes their realized utility, updates their average utility received so far, and updates their pacing multiplier accordingly.
As pointed out in Gao et al.
(2021), PACE is an instantiation of dual averaging Xiao (2010) applied to the dual of the hindsight allocation program in (4).
PACE has many attributes desirable in real-world applications.
Highlight 1. Decentralization.
The PACE dynamics can be run in either centralized (by having the mechanism designer emulate the pacing process for each agent) or decentralized fashion (since the auction-based allocation is the only centralized step at each iteration), and are therefore suitable for Internet-scale online fair division and online Fisher market applications.
Highlight 2.
Pure Allocation.
PACE allows each item to be fully allocated to a single agent, even though the hindsight performance metric is allowed to utilize fractional allocations.
While fractional allocations can be interpreted as randomized allocations in many large-scale settings, this may not always be desirable, for example when allocating food to food banks.
Highlight 3.
Tuning-free.
An important fact about the PACE dynamics is that each agent has no stepsize parameter whatsoever, which means that no stepsize tuning is required.
Moreover, PACE is robust against the types of item arrivals since the algorithm needs neither knowledge of the item distribution P nor the input type C .
In addition to the regret and the envy performance metrics, we will also derive results for the following two quantities that characterize the long-run behavior of PACE.
Let (cid:22) u t
=
(1 =t ) (cid:1) ∑ t(cid:28)
=1 u (cid:28) be the vector of average realized utilities for all agents.
We will show that the agents’ utilities converge to those associated to the underlying ofﬂine fair allocation problem , u (cid:3) , deﬁned in Eq.
(3), in a mean-square sense, i.e., E [ ∥ (cid:22) u t (cid:0) u (cid:3) ∥ 2 ] ! 0 ; as long as the error due to nonstationarity grows sublinearly in the number of time periods.
Secondly, deﬁne the expenditure of agent i at time (cid:28) by b (cid:28)i :=
(cid:12) (cid:28)i v i ( (cid:18) (cid:28) )
1 f i = i (cid:28) g : We will show (1 =t ) (cid:1)
∑ t(cid:28)
=1 b (cid:28)i !
=n in mean square as well, as long as the error due to nonstationarity grows sublinearly in the number of time periods.
Main Results
This section introduces the main results of this paper: the behavior of PACE under three different types of nonstationary input models.
All prior results on fair online allocation have been either for worst-case inputs (with much more conservative guarantees and not for the PACE algorithm)
(Azar et al., 2016; Banerjee et al., 2022) or for i.i.d.
input data Gao et al.
We ﬁrst introduce some notation that will be useful for describing these input models.
For s > (cid:28) (cid:21)let Q s ( (cid:18) 1: (cid:28) ) denote the conditional distribution of (cid:18) s given f (cid:18) 1 ; : : : ; (cid:18) (cid:28) g .
For a subset I of [ t ] let
Q I denote the joint distribution of the variables f (cid:18) (cid:28) g (cid:28) 2 I .
Let (cid:22) Q
= (1 =t ) (cid:1)
∑ t(cid:28)
=1 Q (cid:28) be the uniform mixture of f Q (cid:28) g (cid:28) .
We study three types of input: independent input with adversarial corruption, ergodic and Markov input, and periodic input.
For each input setting, we describe our main theorem for the performance guarantees of PACE here.
The proofs are given in Appendix C, because these results rely on developing a theory of nonstationary performance of DA, which is done in Section 4.
3.1 Independent Input with Adversarial Corruption
Adversarial perturbation of a ﬁxed item distribution models scenarios where the items generally behave in a predictable manner, but for some time steps the input behaves erratically.
Typically this is assumed to happen only for a small number of time steps.
Such perturbation could be malicious, for example when item arrivals are manipulated in favor of certain agents; or non-malicious, such as unpredictable surges of certain keywords on search engines (Esfandiari et al., 2018).
We study a type of adversarial perturbation where the item distribution at each time step might be corrupted by an arbitrary amount, but distributions at different time steps are independent of each other.
We assume the average corruption is bounded by (cid:14) , as measured in TV distance.
The set of distributions over sequences that we consider is then:
[Equation]
We use ~ O to hide numeric constants and polynomials of n , j v j 1 , and log t .
Our main fair online allocation result for the adversarial corruption case is:
Theorem 1 (Independent Case) .
For the adversarially corrupted and independent case, Algorithmguarantees that for an instance A = ( n; t; (cid:2)
; Q; v ) , we have
[Equation]
and
[Equation]
The result shows that the regret and envy performance metrics degrade linearly in the average corruption (cid:14) .
In the i.i.d. case where (cid:14) = 0 , we recover the p t regret rate and the 1 =t rate of convergence for utilities and expenditures in terms of the mean-square error from Gao et al.
If out of the t distributions of items in each time step only O ( p t ) are corrupted, each by a constant amount, then the p t regret and envy bounds, as well as 1 =t convergence rates, are also preserved.
3.2 Ergodic Input and Markov Processes
To handle correlation across time, we next study ergodic inputs.
For these inputs, strong correlation might be present for items sampled at nearby time steps, but the correlation between items decays as they are separated in time.
For any integer (cid:19) such that 1 (cid:20) (cid:19) (cid:20) t (cid:0) 1 , we measure the (cid:19) -step deviation from some distribution (cid:5) 2 ∆((cid:2)) by the quantity
[Equation]
Intuitively, this deﬁnition tells us that, no matter where and when we start the item arrival process, it takes only (cid:19) steps to get (cid:14) ( (cid:19) )
-close to the distribution (cid:5) .
We will consider the set of ergodic input distributions whose (cid:19) -step deviation is bounded by (cid:14) :
[Equation] ( (cid:14); (cid:19) )
:= {
[Equation] sup
[Equation] ∥ Q [Equation]
TV (cid:20) (cid:14); for some (cid:5) 2 ∆((cid:2)) } : (10)
Theorem 2 (Ergodic Case) .
For the ergodic case, Algorithm 1 guarantees that for an instance
A = ( n; t; (cid:2) ; Q; v ) , we have
[Equation]
Envy i [Equation]
(11) and sup
[Equation]
Remark 1 (Markov Input) .
We can specialize the result in Theorem 2 to fast mixing or Markov item sequences.
Fast mixing means the deviation (cid:14) decreases exponentially, i.e., for all 1 (cid:20) (cid:19) (cid:20) t (cid:0) 1 , sup
[Equation]
[Equation] (12) for some M > 0 , (cid:26) 2 [0 ; 1) , and (cid:5) is the stationary distribution.
Examples include ﬁnite statespace time-homogeneous Markov chains and uniformly ergodic Markov chains on general state spaces (Meyn and Tweedie, 2012, Chapter 16).
In these cases, setting (cid:19) = log t +log M log( (cid:26) (cid:0) 1 )
=
O ( log t log( (cid:26) (cid:0) 1 ) ) implies (cid:14) (cid:20)
=
t .
This means the Markov chain from which (cid:13) is generated takes O (log t ) steps to get (1 =t ) -close to stationarity.
The dominant term for the regret in Theorem 2 (further ignoring
M ) is then (1 + 1 log( (cid:26) (cid:0) 1 ) )
1 =p t: The term in the parenthesis reﬂects the inﬂation caused by input dependency.
To recover the case of i.i.d.
input, we simply send (cid:26) !
and the usual p t regret and envy rates and 1 =t utility and expenditure convergence rates are again recovered.
3.3 Periodic Input
Item sequences often exhibit statistical periodic structure.
For example, when allocating compute time to requestors, there will be more requests during weekdays and less on weekends.
The compute request patterns vary throughout the week, and yet the weekly pattern would repeat over time.
Similarly, Internet trafﬁc typically exhibits periodic structure.
Formally, assume that the horizon t divides into K blocks of time, each of size q .
This divides the item sequence (cid:13) into consecutive blocks of length q .
Within each block, we allow a distribution with arbitrary dependence between time steps, but we assume that the blocks, as a whole, are identically and independently distributed.
We deﬁne the set of periodic input distributions as follows:
[Equation]
Theorem 3 (Periodic Case) .
For the periodic case, Algorithm 1 guarantees that for an instance
A = ( n; t; (cid:2) ; Q; v ) , we have
[Equation]
and
[Equation]
If the length q of the blocks is of order o ( t ) then the time-averaged regret and envy are both vanishing.
For the i.i.d.
case, we can set q = 1 to recover the previous results.
Now we comment on dependence on the period length q .
Suppose the item sequence consists of
K blocks, and blocks are i.i.d.
We still allow arbitrary dependence within a block.
The proof of
Theorem 3 essentially relies on the result (Theorem 9) that DA produces iterates whose squared error is of order O ( q 2 =t ) .
Consider dual averaging with the knowledge of the block structure q .
Then the rate 1 =K
= q=t can be achieved by executing DA using one randomly chosen data point within a block, throwing away the rest in that same block.
Such selection produces K i.i.d.
samples from the distribution.
In comparison, the rate in O ( q 2 =t ) is worse off by a factor of q due to not knowing the block-structure information.
Proof Technique: Nonstationary Dual Averaging
The PACE dynamics can be cast as online dual averaging (Xiao, 2010) applied to the dual of the hindsight allocation program in (1).
This will be discussed in more detail in Section 4.2 and Appendix C.2.
However, in order to characterize its performance under various types of nonstationary input, we ﬁrst extend the general convergence results for dual averaging to incorporate nonstationary input.
The convergence results that we developed for dual averaging under three different types of input models are novel and are of independent interest.
We remark that the results for the stochastic setting given by Xiao (2010) cannot be used directly, since they rely on the stringent i.i.d.
assumption.
Duchi et al.
(2012) consider ergodic mirror descent (MD) for convex problems under some (but not all) of our nonstationary input models.
Their results and analysis cannot be used in our case either, since their results do not allow using the composite structure, whose strong convexity we leverage.
Moreover, unlike DA, an MD-based approach would require tuning parameters such as stepsizes.
In this section, after introducing the setup of DA in Section 4.1, we present a DA convergence result for independent but not identical input in Section 4.3, for which we outline the proof idea and technical challenges in Section 4.3.
Due to space limitations, we present DA convergence results for ergodic and periodic inputs in Appendix D.3.
In the nonstationary setup of DA, we emphasize that whenever we mention convergence, we mean convergence of DA iterates to the population-level optimum w (cid:3) (cid:5) (or sometimes the hindsight optimum), up to some error caused by nonstationarity.
4.1 Review: The DA Algorithm
We review the dual averaging setup in the strongly convex case (Xiao, 2010, §1.1).
Consider a stochastic optimization problem of the form
[Equation]
where w 2 ( R d ; ∥ (cid:1) ∥ ) is the variable, (cid:9) is a closed convex function with closed domain Dom (cid:9) :
= f w 2 R n : (cid:9)( w ) < 1g .
The expectation is taken over a probability distribution (cid:5) on a measurable space Z .
For each z 2 Z , the function f ( (cid:1) ; z ) is convex and subdifferentiable (a subgradient always exists) on Dom (cid:9) .
Let F ( w; z ) =
f ( z; w ) + (cid:9)( w ) .
Assumptions.
Let G ( w; z ) be a ﬁxed element in the set of subgradients @ w f ( w; z ) .
For almost every z , it holds ∥ G ( w; z ) ∥ (cid:3) (cid:20) G , where ∥ (cid:1) ∥ (cid:3) = max ∥ w ∥(cid:20) 1 ⟨ s; w ⟩ is the dual norm.
There exists an (cid:22) F 2 R such that F ( w; z ) (cid:20) (cid:22) F for all w 2 Dom (cid:9) and (almost every) z . 3.
(cid:9) is (cid:27) -strongly convex, i.e., (cid:9)( (cid:11)w + (1 (cid:0) (cid:11) ) u ) (cid:20) (cid:11) (cid:9)( w ) + (1 (cid:0) (cid:11) )(cid:9)( u ) (cid:0) (cid:27) 2 (cid:11) (1 (cid:0) (cid:11) )
∥ w (cid:0) u ∥ 2 for w; u 2 Dom (cid:9) .
Because of our strong convexity assumption, the solution to (15) is unique.
Associated with (cid:5) we deﬁne w (cid:3) (cid:5) :=
arg min E z (cid:24) (cid:5) [
F ( w; z ) ] :
The dual averaging algorithm (DA) (Xiao, 2010, Algorithm 1) aims to produce a sequence converging to the optimal point w (cid:3) (cid:5) or minimize the associated regret (Xiao, 2010, §1.2).
The algorithmic details for DA are presented in Algorithm 2.
Note that although Xiao (2010) only considers the case of i.i.d.
data, DA iterates are deﬁned for every input sequence f z (cid:28) g t(cid:28)
=1 , regardless of any distributional properties of the sequence.
The ﬁrst step in our analysis is a relationship between regret and the suboptimality ∥ w t (cid:0) w (cid:3) (cid:5) ∥ derived by Xiao (2010).
Consider the dual averaging algorithm with data f z (cid:28) g t(cid:28)
=1 .
We denote the one-step subgradient by g (cid:28) :
= G ( w (cid:28) ; z (cid:28) ) and the average subgradient by (cid:22) g (cid:28) =
( ∑ (cid:28)s =1 g s ) =(cid:28) .
Given data f z (cid:28) g t(cid:28) =1 , we deﬁne the regret and the sum of squared subgradient norms
[Equation]
( 5 ∥ g 1 ∥ 2 (cid:3) + t (cid:0) 1 ∑ (cid:28) =
∥ g (cid:28) +1 ∥ 2 (cid:3) =(cid:28) ) :
The bound ∆ t (cid:20) (6 + log t ) G 2 = (2 (cid:27) ) holds in a deterministic manner due to the bounded subgradient assumption.
Xiao (2010) shows the following bound on suboptimality of w t
Lemma 1 (Regret Bound, Section B.2 in Xiao (2010)) .
For any sequence f z (cid:28) g t(cid:28)
=1 , any w 2 Dom (cid:9) , any t = 1 ; 2 ; : : : , it holds ∥ w t +1 (cid:0) w ∥ 2 (cid:20) 2 (cid:27)t ( ∆ t (cid:0) R t ( w ) ) :
4.2 Review: PACE as Dual Averaging
In this section we review how to cast PACE as dual averaging applied to the problem (4).
This derivation was originally given in Gao et al.
Let f ( (cid:12); (cid:18) )
=
max i (cid:12) i v i ( (cid:18) ) , (cid:9)( (cid:12) ) =
(cid:0) 1 n ∑ ni =1 log (cid:12) i , in which case we get F ( (cid:12); (cid:18) )
=
f
( (cid:12); (cid:18) )
+ (cid:9)( (cid:12) )
=
max i 2 [ n ] { (cid:12) i v i ( (cid:18) ) } (cid:0) 1 n ∑ ni =1 log (cid:12) i :
Following (Gao and Kroer, 2021, §5), since f ( (cid:1) ; (cid:18) ) is a piecewise linear function, a subgradient is
G ( (cid:12); (cid:18) ) :
= v i (cid:28) ( (cid:18) )
e i (cid:28) 2 @ (cid:12) f
( (cid:12); (cid:18) ) ; where i (cid:28) =
min f arg max i (cid:12) i v i ( (cid:18) )
g is the index of the winning agent (see, e.g., Beck (2017, Theorem 3.50)).
Based on this instantiation of DA, we get that the iterates f (cid:12) (cid:28) g t +1 (cid:28) =1 generated by the PACE dynamics (Algorithm 1) are exactly the iterates w (cid:28) generated by DA( G ; (cid:9) ; (cid:13) )
(Algorithm 2).
A proof is given in Appendix C.2.
Before moving on to showing our results, let us ﬁrst touch on the fact that dual averaging provides worst-case regret guarantees.
Naively, one may expect that these regret guarantees would directly translate into regret guarantees on the primal performance, meaning a bound on sup (cid:13)
Reg i;t ( (cid:13) ) .
[Table] [Table] [Table] However, such dual regret bounds do not imply a worst-case primal regret bound.
From a technical perspective, it is unclear how a regret bound on the dual objective would translate to a regret bound on envy or utilities.
Secondly, based on results in the online fair allocation literature (Azar et al., 2016; Banerjee et al., 2022), it is known that not only can we not get a no-regret guarantee on the primal performance, it is not even possible to achieve a constant competitive ratio in the worst-case setting.
4.3 Independent Data with Adversarial Corruption
We present a DA convergence result with independent data in this section.
Theorem statements for the ergodic case and periodic case are presented in Appendix D.3.
Discarding the i.i.d.
assumption on the data f z (cid:28) g t(cid:28)
=1 , we let P be the joint distribution of f z (cid:28) g (cid:28) and let P (cid:28) be the marginal distribution of z (cid:28) .
We study the relationship between the DA iterates w t +1 and w (cid:3) (cid:5) by bounding the mean-square difference E f z (cid:28) g t(cid:28)
=1 (cid:24) P [ ∥
w t +1 (cid:0) w (cid:3) (cid:5) ∥ 2 ] , and thus demonstrate in what sense the data distribution P should stay close to the i.i.d.
distribution (cid:5) in order to preserve DA convergence.
We ﬁrst introduce a variant of C ID ( (cid:14) ) with a target distribution (cid:5) :
[Equation]
Theorem 4 (DA Convergence, Independent Data with Adversarial Corruption) .
If f z (cid:28) g t(cid:28) =1 (cid:24) P and P 2 C ID ( (cid:14); (cid:5)) .
Then for t (cid:21) 1 ,
[Equation]
Moreover, the rate ~ O ( (cid:14) +
=t ) applies to E [ ∥ w t +1 (cid:0) w (cid:3) (cid:13) ∥ 22 ] and E [ ∥ w (cid:3) (cid:13) (cid:0) w (cid:3) (cid:5) ∥ 22 ] (See Appendix D.1).
Remark 2.
From this result we can tell when DA retains last-iterate convergence.
Suppose the number of corrupted data points is of order o ( t ) (assuming corruption on each data is of the same order), then the corruption per item is (cid:14) =
o (1) and DA converges to the optimal solution as t ! 1 .
Furthermore, if the corruption per data point is of order (cid:14) =
O (1 =t ) , then the fast rate 1 =t is retained.
By Section 4.2, we get as a corollary that the PACE iterates f (cid:12) t g will converge to (cid:12) (cid:3) , the solution to the inﬁnite-dimensional dual program (4).
This is the building block for our results in Section 3.
While the full proof is too long to ﬁt in the paper, we now give a proof sketch to show the main ideas behind how we derive Theorem 4.
We begin the proof by noticing Lemma 1 is deterministic and valid for any f z (cid:28) g t(cid:28)
=1 .
Now set w = w (cid:3) (cid:5) in Lemma 1.
If the input data f z (cid:28) g t(cid:28) =1 were i.i.d.
from (cid:5) , i.e., P = (cid:5) (cid:10) t , then E [ R t ( w (cid:3) (cid:5) )] would be greater than zero, and we would obtain
[Equation]
However, in the nonstationary case, the regret E [ R t ( w (cid:3) (cid:5) )] might be negative.
At a high level, our results are achieved by introducing appropriate measures of the nonstationarity and then lower bounding E
[ R t ( w (cid:3)
To this end, we decompose the regret as follows.
Write
R t ( w (cid:3) (cid:5) )
=
t ∑
[Equation]
) |
[Equation]
z }
II :
By optimality of w (cid:3) (cid:5) we have II (cid:21) 0 .
Using the bound on the TV distance between f P (cid:28) g (cid:28) and (cid:5) , and boundedness of F we can control the other two terms.
The key is, conditional on F (cid:28) (cid:0) 1 , the iterate w (cid:28) is deterministic and the distribution of z (cid:28) is P (cid:28) due to independence assumption.
For each term in the ﬁrst summation, we condition on F (cid:28) (cid:0) 1 and obtain
[Equation]
E [
[Equation] [Equation]
The second sum can be handled similarly.
For the detailed proof and a generalization see Appendix D.2 in Appendix D.
We now discuss how this paper handles nonstationarity differently from the existing literature.
Let us use Balseiro et al.
(2020) as a reference point, since they consider very similar categories of nonstationary inputs.
From a online constrained optimization perspective, Balseiro et al.
(2020) relies on the fact that their objective is separable across timesteps in order to handle nonstationarity. In contrast, this structure does not exist for the online fair allocation problem, because the objective takes the logarithm of the utility over time.
Concretely, in Balseiro et al.
(2020), the objective is of the form ∑ t(cid:28) =1 f (cid:28) ( x ) .
This type of time-separability occurs in all works that we are aware of for non-stationary inputs, also e.g. the ergodic mirror descent paper Duchi et al.
(2012) (Duchi et al.
(2012) is not an online setting, but the expectation in their objective is analogous to time separability).
Time separability is used as a key property for deriving regret bounds in those papers.
The separability enables translating dual regret to primal regret by a weak duality argument (see e.g. Prop. 1 in Balseiro et al.
(2020) where a time-separable dual allows weak duality).
In contrast, our problem has time-separability only in the dual formulation, but not in the primal one, which is where we ultimately want guarantees (since we are interested in utilities converging).
Our contribution to handling nonstationarity in online fair allocation is showing that a dual approach works even without the separability structure.
We begin by deriving convergence guarantees on the last dual iterate, which we achieve by modifying the dual averaging proof to take into account nonstationarity and analyzing the stability of the dual variables in dual averaging.
We note that this technique depends on strong convexity, unlike for the time-separable case.
Next, to go from dual last-iterate convergence to primal regret bounds we use the ﬁrst-order optimality condition for
EG program, which are speciﬁc techniques for our problem (such techniques were also used in the previous PACE paper Gao et al. (2021)).
Conclusion
We established new convergence results for dual averaging under nonstationary data input models, namely, adversarial corruption, ergodic, and block-independent input models.
Leveraging these results, we showed that, for online fair allocation problems with item arrivals generated from the above nonstationary data input models, the PACE algorithm automatically adapts to them and achieves asymptotic fairness and efﬁciency without any parameter tuning.
Numerical experiments demonstrated the effectiveness of PACE under these data input models.
Acknowledgments and Disclosure of Funding
This research was supported by the Ofﬁce of Naval Research Young Investigator Program under grant N00014-22-1-2530, and the National Science Foundation award IIS-2147361.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information on how to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.
For example:
(cid:15)
Did you include the license to the code and datasets?
[Yes] See Section ?? .
(cid:15)
Did you include the license to the code and datasets?
[No] The code and the data are proprietary.
(cid:15)
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] See Section(b)
Did you describe the limitations of your work?
[No] Our work is mostly theoretical and studies the fair-allocation algorithm PACE under various nonstationary settings.
(c) Did you discuss any potential negative societal impacts of your work?
[No] Not applicable.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] We read them.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sectionand Section 4.
(b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix C, Appendix D and Appendix E. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See Appendix F. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See Appendix F. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[No] No applicable.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix F. 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[No] We use synthetic data.
(b) Did you mention the license of the assets?
[No] N/A. (c) Did you include any new assets either in the supplemental material or as a URL?
[No] N/A. (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[No] N/A. (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[No] N/A. 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[No] N/A. (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[No] N/A.
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[No] N/A.
A Related Work
Since our work studies competitive equilibrium computation, online fair resource allocation and stochastic optimization, while PACE employs the idea of pacing in auction mechanism design, we further discuss related work in these areas.
Convex optimization for computing competitive equilibria.
Convex optimization algorithm (especially ﬁrst-order methods) and their theory have been used to design and analyze algorithms for computing competitive equilibria, often through equilibrium-capturing convex programs Birnbaum et al.
; Cole et al.
(2017); Cheung et al.
; Gao and Kroer (2020);
Gao et al.
Applying a ﬁrst-order method to such a convex program often leads to (recovers) interpretable market dynamics that emulate real-world economic behaviors, such as the proportional response dynamics
Birnbaum et al.
(2011); Zhang (2011); Cheung et al.
; Gao and Kroer (2020) and tâtonnement (Cheung et al., 2020).
The PACE algorithm of Gao et al.
(2021) is no exception: it results from applying dual averaging to a speciﬁc convex program.
Discrete variants of these convex programs have also been used for fair indivisible allocation (Caragiannis et al., 2019), which yields some efﬁciency and fairness guarantees, though the discreteness breaks the connection to competitive equilibria.
(Online) fair resource allocation.
Azar et al.
(2016) consider an online Fisher market with arbitrary item arrivals.
They focus on a quality measure that is minimized at a competitive equilibrium and give an online algorithm that achieves a competitive ratio logarithmic in the size of the market and the ratio between the maximum and minimum (nonzero) buyer valuations over individual items.
This algorithm requires solving a nontrivial linear program per iteration and is not known to improve with stochastic arrivals.
Banerjee et al.
(2022) considers the problem of online allocation of divisible items to maximize Nash social welfare.
They show that, under arbitrary item arrivals but with access to meaningful predictions of each buyer’s total utility given all items, an online algorithm of the primal-dual type achieves a logarithmic competitive ratio.
Gkatzelis et al.
(2021) study the setting where items arrive online and with two agents.
They focus on satisfying the no-envy condition while maximizing social welfare, and show that one do this approximately by allocating items proportionally to valuations, assuming that valuations are normalized.
Manshadi et al.
(2021) studies the problem of rationing a social good and propose simple, implementable algorithms that promote fairness and efﬁciency.
In their setting, it is the agents’ demands rather than the supply that are sequentially realized and possibly correlated over time.
Bateni et al.
(2021) uses Gaussian processes to model item arrivals and consider a budget-weighted proportional fairness metric.
They propose a reoptimization policy that consumes buyers’ budgets and clears the market gradually while ensuring a competitive ratio in hindsight w.r.t.
this metric.
This policy periodically resolves the EisenbergGale (EG) convex program and does not require prior knowledge of future item arrivals.
Our work differs from the above literature as follows.
First, we consider practically-motivated nonstationary data input models for item arrivals that interpolate between fully adversarial and fully stochastic (i.i.d.).
Second, we show that the PACE algorithm, without any parameter tuning, adapts to different data input models and achieves strong performance guarantees that depend mildly on the “nonstationarity” of these models.
Given that PACE is scalable, interpretable and easy to implement this paper further ensures its effectiveness upon more realistic, non-i.i.d.
item arrival processes.
(Nonstationary) stochastic optimization.
Many stochastic optimization algorithms have been shown to attain nontrivial performance guarantees under under nonstationary data input (Duchi et al., 2012;
Balseiro et al., 2020; Besbes et al., 2015).
Motivated by high-dimensional and distributed optimization problems, Duchi et al.
(2012) analyzes stochastic mirror descent under ergodic data input.
Balseiro et al.
(2020) analyzes a version of mirror descent for online resource allocation.
They show that it achieve strong regret bounds under different data input models without knowing the model in advance.
The ergodic and periodic data input models in this paper are motivated by those considered in Duchi et al.
(2012) and Balseiro et al.
Different from these papers which focus on mirror descent, this paper focuses on the dual averaging algorithm, a different stochastic optimization algorithm particularly suitable for the equilibrium-capturing convex program we study.
Furthermore, we achieve stronger results than those past papers, by focusing on a setting where a composite term has strong convexity.
Pacing in auction mechanism design.
The PACE algorithm uses ﬁrst-price auctions with pacing.
As noted in Gao and Kroer (2020), the idea of pacing has also been used widely in budget management strategies for Internet advertising auctions, with strong revenue and incentive guarantees (see, e.g., Conitzer et al.
; Balseiro and Gur (2019)).
It is also used widely in practice, as reported in Conitzer et al.
As shown in Balseiro et al.
(2020), pacing strategies ensure individual bidders’ returns on their budgets and, if used by all buyers, lead to approximate Nash equilibria.
Similar to the analysis in Gao et al.
(2021), in this paper, we focus on competitive equilibrium and fairness properties of PACE, rather than game-theoretic (incentive) properties.
B Review of Linear Fisher Market
In fair division the goal is to perform this allocation in a fair way, while simultaneously also guaranteeing some form of efﬁciency, typically Pareto efﬁciency.
In the case of allocating m divisible goods to n agents, the competitive equilibrium from equal incomes (CEEI) allocation guarantees many fairness properties.
In CEEI, every agent is endowed with a unit budget of faux currency, a competitive equilibrium is computed, i.e., a set of item prices along with an allocation that clears the market, and the resulting allocation is used as the fair allocation (Varian, 1974).
This guarantees several fairness desiderata such as envy-freeness (every person prefers their own bundle to that of any other person), proportionality (every person prefers their own bundle over receiving their fair share 1 =n of every item), and Pareto optimality (we cannot make any person better off without making at least one other person worse off).
A linear Fisher market refers to the tuple F = ( n; m; B; v ) .
The market consists of n buyers and m items.
We assume each buyer has a budget of B i .
We use f 1 ; : : : ; m g to represent the set of items, each of unit supply.
The matrix v = ( v 1 ; : : : ; v n ) 2 ( R m + ) n consists of valuations, with v ji being the valuation of item j from buyer i .
For buyer i , an allocation of items, x i 2 R m + , gives a utility of u i ( x i ) :
= ⟨ v i ; x i ⟩ :=
∑ mj =1 v ji x ji .
Note we use different fonts to distinguish notions that appear in both the online allocation problem and the Fisher market.
Deﬁnition 1 (Demand) .
Given item prices p 2 R m + , the demand of buyer i is its set of utilitymaximizing allocations given the prices and budget:
[Equation]
Deﬁnition 2 (Market Equilibrium) .
The market equilibrium of F = ( n; m; B; v ) is an allocationprice pair ( x (cid:3) ; p (cid:3) ) 2 ( R m + ) n (cid:2) R m + such that the following holds.
Supply feasibility:
∑ ni =1 x (cid:3) i (cid:20) 1 m .
Buyer optimality: x (cid:3)
i 2 D
i ( p (cid:3) ) for all i .
Market clearance: ⟨ p (cid:3) ; 1 m (cid:0) ∑ n i =1 x (cid:3) i ⟩ = 0 .
Market equilibrium and fair allocation are related as follows.
In CEEI, we construct a mechanism for fair division by giving each agent the same budget of fake currency, i.e., B i =
B j for all i; j , computing what is called a market equilibrium under this new market, and using the corresponding allocation as our fair allocation rule.
It is known that an allocation x (cid:3) from the set of CEEI has many desirable properties.
It is Pareto optimal (every market equilibrium is Pareto optimal by the ﬁrst welfare theorem).
It has no envy: since each agent has the same budget in CEEI and every agent is buying something in their demand set, no envy must be satisﬁed, since they can afford the bundle of any other agent.
Finally, proportionality is satisﬁed, since each agent can afford the bundle where they get 1 =n of each good.
The ME is essentially a collection of optimization problems (Eq.
(17)) coupled through the constraint ∑ ni =1 x i (cid:20) 1 m .
A celebrated result is the Eisenberg-Gale convex program, which provides an equivalent characterization of ME.
[Equation]
That is, we maximize the sum of logarithmic utilities under the supply constraint.
The solution to the primal problem x (cid:3) =
( x (cid:3) 1 ; : : : ; x (cid:3) m ) along with the vector of dual variables p (cid:3) yields a market equilibrium.
The hindsight allocation Eq.
(1) is just the EG program of the linear Fisher market F A = ( n; t; B; v ) where entries of the valuation matrix v are deﬁned by v ji
= v i ( (cid:18) j ) for all
i 2 [ n ] ; j 2 [ t ] , and
B = ( t=n )1 n .
C PACE as Dual Averaging
In this section, we show how to cast PACE as dual averaging.
To this end, we will introduce inﬁnite-dimensional Eisenberg-Gale-type convex programs for the allocation of a (possibly inﬁnite/continuous) set of items.
Here, the item supplies correspond to the probability density d (cid:22)
Q= d (cid:22) of the average item arrival distribution (cid:22) Q .
They serve as intermediate “reference” convex programs that facilitate the use of DA convergence results developed in the previous section to analyze PACE.
When the item space is continuous, the supply function, allocation rules, and the price function in these convex programs are (measurable) functions over such item spaces, which can be inﬁnitedimensional objects.
When item arrivals are drawn from a (ﬁxed) distribution with density s , they correspond to the EG convex programs of the “underlying market” with item supplies s .
Note that when the item space (cid:2) is ﬁnite, the inﬁnite-dimensional analogues reduce to the classical ﬁnitedimensional EG convex programs.
After introducing these concepts we will show that our results on nonstationary DA allows us to derive comparable results on various PACE performance metrics.
The results of Gao et al.
cast PACE as dual averaging for the EG convex program of the underlying market, and show guarantees with respect to that program.
Here, we will show our results for that setting, as well as for the hindsight allocation problem.
Different from the presentation of DA in Xiao (2010), we do not need an additional regularizer.
This paper focuses on DA for strongly convex problems with an existing regularizer in the loss (and hence no auxiliary regularizer is needed), since this is the setting used in the design and analysis of the PACE algorithm; similar convergence results under our new input models can be derived for the general form of DA given in Xiao (2010, Algorithm 1) with an auxiliary regularizer for non-strongly convex problems.
C.1 The Dual of EG and the Inﬁnite-Dimensional Analogue
We derive the dual program of (1).
Introduce the dual variables (cid:12) i (cid:21) 0 with i 2 [ n ] for each constraint of the ﬁrst type and variables p (cid:28) (cid:21) 0 with (cid:28) 2 [ t ] for constraints of the second type.
The
Lagrangian L : R n (cid:2) t + (cid:2) R n + (cid:2) R n + (cid:2)
R t + !
R is given by
L ( x; U; (cid:12); p ) =
n ∑ i =
B i log U i + n ∑ i = (cid:12) i ( ⟨ v i ( (cid:13) ) ; x i ⟩ (cid:0) u i ) + t ∑ (cid:28) = p (cid:28) ( 1 (cid:0) n ∑ i = x
(cid:28)i ) =
t ∑ (cid:28) = p (cid:28) + n ∑ i = (
B i log U
i (cid:0) (cid:12) i u i ) + n ∑ i = ⟨ (cid:12) i v i ( (cid:13) )
(cid:0) p; x i ⟩ :
Maximizing out the variables ( x; U ) gives the dual program min
[Equation]
Dividing by t (recalling B i =
t=n ), ignoring constants and moving the constraint p (cid:21) (cid:12) i v i ( (cid:13) ) to the loss, we obtain the following equivalent optimization problem:
[Equation]
Let (cid:12) (cid:13) be the optimal solution.
To recover the corresponding optimal p (cid:13) we deﬁne p (cid:13);(cid:28) =
max i (cid:12) (cid:13)i v i ( (cid:18) (cid:28) ) for (cid:28) 2 [ t ] .
We recall the inﬁnite-dimensional analogue of (1):
[Equation]
where s =
d (cid:22) Q= d (cid:22) is the average item supply function, and ⟨ v i ; x i ⟩ := ∫ (cid:2) v i x i d (cid:22) .
The inﬁnitedimensional analogue of (2) is the following.
For any (cid:14) 0 > 0 ,
[Equation]
A rigorous mathematical treatment of the two inﬁnite-dimensional programs can be found in Gao and Kroer (2021) and (Gao et al., 2021, Section 2).
Note the additional constraint in (4) on (cid:12) does not affect the optimal solution since 1 =n (cid:20) (cid:12) (cid:3) i (cid:20) 1 ; see Lemma 1 in Gao and Kroer (2021).
The relationship between the ﬁnite and inﬁnite versions of (2) is that we have replaced the uniform averaging in (2) with an integral w.r.t.
the item average distribution (cid:22) Q in (4).
For notational simplicity, we suppress dependence on (cid:22) Q and let ( x (cid:3) ; u (cid:3) ; (cid:12) (cid:3) ) denote the optimal solutions to the inﬁnite-dimensional programs (3) and (4).
Deﬁne the corresponding optimal p (cid:3) in (4) by p (cid:3) :
= max i 2 [ n ] (cid:12) (cid:3) i v i .
C.2 PACE as Dual Averaging
In this section we review how to cast PACE as dual averaging applied to the problem (4).
This derivation was originally given in Gao et al.
Recall f
( (cid:12); (cid:18) )
=
max i (cid:12) i v i ( (cid:18) ) , (cid:9)( (cid:12) ) =
(cid:0)n ∑ ni =1 log (cid:12) i and then
[Equation] [Equation]
} (cid:0)
[Equation]
n
[Equation]
∑ i =
[Equation]
log (cid:12) i :
Following (Gao and Kroer, 2021, §5), since f ( (cid:1) ; (cid:18) ) is a piecewise linear function, a subgradient is
G ( (cid:12); (cid:18) ) :=
[Equation] [Equation]
f
[Equation]
(cid:12); (cid:18) ) ; where i (cid:28) =
min f arg max i (cid:12) i v i ( (cid:18) )
g is the index of the winning agent (see, e.g., (Beck, 2017, Theorem 3.50)).
Lemma 2 (PACE as Dual Averaging) .
The iterates f (cid:12) (cid:28) g t +1 (cid:28) =1 generated by the PACE dynamics (Algorithm 1) are exactly DA( G ; (cid:9) ; (cid:13) )
(Algorithm 2).
Proof of Lemma 2.
Interpret the DA updates using the following substitution: (cid:2) $ Z , (cid:18) (cid:28) $ z (cid:28) , (cid:12) (cid:28) +1 $ w (cid:28) +
and (cid:22) g (cid:28);i $ (cid:22) u (cid:28)i .
For initialization in DA, choose (cid:12) 1 to be the minimizer of (cid:9) over the cube [(1 + (cid:14) n ) (cid:0) 1 n (cid:0) 1 1 n ; (1 + (cid:14) 0 )1 n ] and set (cid:22) g
= (cid:22) u
= 0 .
(1) Subgradient computation , choose the winning bidder (Line 4 of
PACE
(2) Average subgradient , update current averaged utilities (Line 5 of PACE).
The i -th entry of
G ( (cid:12); (cid:18) ) is exactly the time- (cid:28) realized utility of agent i in PACE, that is, g (cid:28);i = v i ( (cid:18) (cid:28) )
1 f i = i (cid:28) g = u (cid:28)i .
Then the average gradient, (cid:22) g (cid:28) = (cid:28) (cid:0) 1 (cid:28) (cid:22) g (cid:28) (cid:0) 1 + 1 (cid:28) g (cid:28) , is the same as the time-averaged utilities:
[Equation]
f
[Equation]
(3) Solve regularized problem , update pacing multiplier (Line 6 of PACE).
The minimization problem is separable in agent index i and exhibits a simple and explicit solution.
Recall (cid:22) g (cid:28);i = (cid:22) u (cid:28) i : (cid:12) (cid:28) +
i
[Equation]
C.3 Performance Guarantees via Dual Averaging
Now that we have cast PACE as an instantiation of dual averaging and developed results for convergence in nonstationary settings, the following theorems follow easily from the general convergence results for DA.
Recall the hindsight optimum (cid:12) (cid:13) is deﬁned in (2), and its inﬁnite-dimensional counterpart (cid:12) (cid:3) is deﬁned in (4).
Theorem 5 (Convergence of PACE, the Independent Case) .
Assume the item sequence (cid:13) (cid:24) Q and
Q 2 C ID ( (cid:14) ) .
Choose (cid:14)
= 1 in PACE.
It holds for t (cid:21) 1 ,
[Equation]
Moreover, the rate ~ O ( (cid:14) + 1 =t ) applies to E [ ∥ (cid:12) (cid:13) (cid:0) (cid:12) (cid:3)
∥ 2 ] and E [ ∥ (cid:12) t +1 (cid:0) (cid:12) (cid:13) ∥ 2 ] .
Proof.
Set P = Q , (cid:5) =
(cid:22) Q , (cid:27) =
1 =n , and (cid:22) F =
j v j 1 in Theorem 4.
Here the convergence of (cid:12) t to the hindsight counterpart (cid:12) (cid:13) is of practical importance.
This is because (cid:12) (cid:13) can always be computed after the fact, while its inﬁnite-dimensional counterpart (cid:12) (cid:3) is not necessarily obtainable.
Theorem 6 (Convergence of PACE, the Ergodic and the Periodic Cases) .
For the erogdic case, i.e., (cid:13) (cid:24) Q and Q 2 C E ( (cid:14); (cid:19) ) , it holds for t (cid:21) 1 ,
[Equation]
: where C E; 1 = n 2 j v j
[Equation]
6 + log t
[Equation]
, C E; 2 = 4 n ( j v j 2 1 (1 + log t ) + j v j 1 ) and C E; 3 = 8 n j v j 1 .
For the periodic case, i.e., (cid:13) (cid:24) Q and Q 2 C P ( q ) , it holds for t (cid:21)
[Equation]
=
[Equation]
=
[Equation]
where C P; 1 = C E; 1 and C P; 2 = 2 n j v j 2 1 (1 + log t ) .
For both cases, similar convergence results can be stated for E [ ∥ (cid:12) (cid:13) (cid:0) (cid:12) (cid:3) ∥ 2 ] and E [ ∥ (cid:12) t +1 (cid:0) (cid:12) (cid:13) ∥ 2 ] and are omitted here.
Proof.
For the ﬁrst inequality, set P = Q , (cid:5)
= (cid:22) Q , (cid:27) =
1 =n , and (cid:22) F =
j v j 1 in Theorem 8.
For the second inequality, set additionally (cid:14) b = 0 and jPj 1 = q in Theorem 9.
C.4 From Dual EG Performance Bounds to Primal Performance Bounds Convergence of (cid:12) (cid:28)
to (cid:12) (cid:3) implies the convergence of the average utilities and expenditure to their inﬁnite-dimensional counterparts.
This follows almost directly from results developed by Gao et al.
In particular, they show:
Lemma 3 (PACE Long-Run Behavior, Gao et al.
For any distribution Q 2 ∆((cid:2) t ) , let (cid:13) (cid:24) Q .
It holds for t (cid:21) 1 ,
[Equation]
and
[Equation]
where C u = n(
[Equation]
Finally, we relate results in Appendix C.3 to the main quantities of interest: regret and envy, deﬁned in (6) and (7), as well as convergence to the hindsight utilities.
Similar results are given by Gao et al.
(2021), and our proof is almost identical to theirs, simply extended to the nonstationary case as well as to the hindsight allocation problem.
Lemma 4 (Regret and Envy) .
For any distribution Q 2 ∆((cid:2) t ) , let (cid:13) (cid:24) Q .
It holds for t (cid:21) 1 ,
E [ ∥ (cid:22) u t (cid:0) u (cid:13) ∥ 2 ] (cid:20)
C r; 1 (cid:1) E [ ∥ (cid:12) t +1 (cid:0) (cid:12) (cid:3) ∥ 2 ] + n (cid:1) C r; 2 (cid:1) (t t ∑ (cid:28) =
E [ ∥ (cid:12) (cid:28) (cid:0) (cid:12) (cid:3) ∥ 2 ] ) ;
E [ Reg i;t ( (cid:13) ) ]
(cid:20) t (cid:1) vuut
C r; 1 (cid:1) E [ ∥ (cid:12) t +1 (cid:0) (cid:12) (cid:3)
∥ 2 ] + C r; 2 (cid:1) (t t ∑ (cid:28) =
E [ ∥ (cid:12) (cid:28) (cid:0) (cid:12) (cid:3) ∥ 2 ] ) ; and
E [ Envy i;t ( (cid:13) ) ]
(cid:20) t (cid:1) vuut
C e; 1 (cid:1) E [ ∥ (cid:12) t +1 (cid:0) (cid:12) (cid:3) ∥ 2 ] + C e; 2 (cid:1) (t t ∑ (cid:28) =
E [ ∥ (cid:12) (cid:28) (cid:0) (cid:12) (cid:3) ∥ 2 ] ) ; where C r; 1 = 2 C u , C r; 2 = 2 n 2 j v j 2 1 , C e; 1 = 2(1 + n 2 ) C u and C e; 2 = 4 j v j 2 1 n 2 + 2 n 3 .
Now we have all the ingredients to prove the convergence of PACE.
Proof of Theorem 1, Theorem 2 and Theorem 3.
Combine Lemma 3 with Theorem 5 and Theorem 6 and we obtain the ﬁrst set of inequalities in Theorems 1 to 3.
Then combine Lemmawith Theorem 5 and Theorem 6, and we obtain the second set of inequalities.
D Proofs for Nonstationary Dual Averaging
The DA algorithm Algorithm 2 is obtained in Xiao (2010, §3.2) for the strongly convex case.
We simply set h
= (1 =(cid:27) )(cid:9) , (cid:12)
= (cid:27) and (cid:12) t = 0 for t (cid:21) 1 in (Xiao, 2010, Algorithm 1).
Recall P (cid:28) is the distribution of z (cid:28) .
For integers (cid:28) and (cid:28) ′
( (cid:28) ′ (cid:21) (cid:28) ), let P (cid:28) ′
( (cid:1)
j z 1: (cid:28) ) denote the distribution of z (cid:28)
′ if the process starts at z 1: (cid:28) =
f
z 1 ; : : : ; z (cid:28) g .
D.1 Remark: Convergence to the Hindsight Optimum
Before developing the nonstationary convergence theory, we digress a bit and introduce a simple deduction through which we can easily show the convergence of DA iterates to the optimum of the hindsight problem based on convergence to w (cid:3) (cid:5) .
Given data f z (cid:28) g t(cid:28) =1 , deﬁne the sum ϕ (cid:13) ( w ) =
(1 =t ) (cid:1)
∑ t(cid:28)
=1 F ( w; z (cid:28) ) and its unique minimizer w (cid:3) (cid:13) =
arg min ϕ (cid:13) ( w ) .
We claim all results developed for ∥ w t +1 (cid:0) w (cid:3) (cid:5) ∥ 2 will also hold for the hindsight suboptimality ∥ w t +1 (cid:0) w
[Equation]
∥
[Equation]
Note the following inequality:
[Equation]
the last term being exactly R t ( w (cid:3) (cid:5) ) .
Choose w = w (cid:3) (cid:13) in Lemma 1 and we obtain
[Equation]
It follows that all lower bounds for the regret E [
R t ( w (cid:3)
[Equation]
) ] can be turned into an upper bound for the hindsight suboptimality measure ∥ w t +1 (cid:0) w (cid:3) (cid:5) ∥ 2 .
Convergence to the hindsight optimum is of practical importance since the hindsight optimum w (cid:3) (cid:13) can typically be computed, where this is not always the case for the population optimum w (cid:3) (cid:5) .
A simple consequence of the deduction above is the following.
We note the inequality
[Equation]
Therefore convergence results for ∥ w t +1 (cid:0) w (cid:3) (cid:5) ∥ and
∥ w t +1 (cid:0) w (cid:3) (cid:13) ∥ 2 similarly hold for ∥ w (cid:3) (cid:13) (cid:0) w (cid:3) (cid:5) ∥ 2 .
D.2 Proof for Adversarial Corruption and Independent Data
Assume the data (cid:13) =
f z (cid:28) g t(cid:28)
=1 follows the distribution P with no further assumptions.
We let z 1:0 = ∅ and further let P (cid:28) ( (cid:1)
j z 1:0 ) = P (cid:28) , the marginal distribution of z (cid:28) .
Deﬁne the progressive deviation from (cid:5)
[Equation]
If the data are independent, then it holds
[Equation]
Note for the independent case, (cid:14) t = t (cid:1) (cid:14) where (cid:14) is in Eq.
If the data further has identical distribution (cid:5) then (cid:14) t = 0 .
If (cid:14) t =
O (log t ) we call data has mild corruption.
Theorem 7 (Corrupted Data, Generalization of Theorem 4) .
It holds
[Equation]
where O hides dependence on constants, G and (cid:22) F .
Recall (cid:27) is the strong convexity parameter of (cid:9) .
Remark 3.
In either the i.i.d. case or the mild corruption case ( (cid:14) t = O (log t ) ), we recover the usual
O (log t=t ) rate.
Proof of Theorem 7 In Lemma 1, the term ∆ t is upper bounded in a deterministic manner.
So it remains to handle R t .
In the i.i.d. case, E R t ( w (cid:3) (cid:5) ) is positive and thus can be dropped:
[Equation]
However, to handle corrupted data, we need to use
Lemma 5.
The regret can be lower bounded by the corruption parameter (cid:14) :
[Equation]
Plugging in the above lemma, we get
[Equation]
Thus, to complete the proof of Theorem 7 we only need to prove Lemma
[Equation]
Proof of Lemma 5.
Write
[Equation]
( w (cid:28) ) )
(I) + t ∑ (cid:28) =
[Equation] [Equation]
(cid:3)
[Equation] [Equation]
F (
[Equation]
(cid:3)
[Equation] [Equation]
(cid:28) ) )
(II) + t ∑ (cid:28)
[Equation]
( ϕ (cid:5) ( w (cid:28) ) (cid:0) ϕ (cid:5) ( w (cid:3) (cid:5) ) ) : (III)
By optimality of w (cid:3) (cid:5) we have III (cid:21) 0 .
Bounding I and II .
Conditional on F (cid:28) , the iterate w (cid:28) is deterministic and the distribution of z (cid:28) is
P (cid:28) ( (cid:1)
j z 1: (cid:28) (cid:0) 1 ) .
Note
[Equation]
[Table]
Let us investigate the inner expectation.
Conditional on F
[Table]
1 , the iterate w (cid:28) is deterministic
[Table]
and the distribution of z
[Table] [Table]
P
[Table]
by deﬁnition.
[Table]
[Equation]
where we use boundedness of F , i.e., sup w F ( w; z ) (cid:20) (cid:22) F for (cid:5) -almost every z ,
Next, sum over (cid:28) = 1 ; : : : ; t and move j (cid:1)
j inside the sum and the outer expectation.
[Equation]
Next consider j E [II] j .
The analysis goes through without the outer expectation.
Combining we get
[Equation]
This completes the proof of Lemma 5.
D.3 Theorem Statements
for
Markov Case
and Periodic Case
Results for other input types, C E ( (cid:14); (cid:19) ) and C P ( q ) , can be obtained by using more complicated regret decompositions and conditioning arguments.
We state the resulting convergence results here.
The proofs can be found in Appendix D.4 and Appendix D.5.
Theorem 8 (DA Convergence, Ergodic Case) .
For the input distribution P deﬁne the (cid:19) -step deviation from (cid:5) for an integer 1 (cid:20) (cid:19) (cid:20) t (cid:0) 1 : ϵ t ( (cid:19) ) :
= sup
[Equation]
1 ;:::;z t sup
[Equation]
(cid:0) (cid:19) ∥
P
[Equation]
+
[Equation] [Equation]
z 1: (cid:28) ) (cid:0) (cid:5) ∥ TV :
Then, for all t (cid:21) 1 and any 1 (cid:20) (cid:19) (cid:20) t (cid:0) 1 ,
E
[Equation]
z
[Equation]
t
[Equation]
1 (cid:24) P [ ∥
w t +1 (cid:0) w
[Equation]
(cid:5)
[Equation]
Moreover, the rate ~ O ( ϵ t ( (cid:19) )
+ (cid:19)=t ) applies to E [ ∥ w t +1 (cid:0) w (cid:3) (cid:13) ∥ 22 ] and E [ ∥ w (cid:3) (cid:13) (cid:0) w (cid:3) (cid:5) ∥ 22 ] by the deduction in Appendix D.1.
Remark 4 (Comparison with EMD Duchi et al.
Now we specialize Theorem 8 to the setting of Remark 1, and we brieﬂy compare our result with the Ergodic Mirror Descent (EMD) results of
Duchi et al.
EMD considers nonsmooth convex optimization problems of the form f (cid:3) =
min { f ( w ) =
E (cid:5)
[ F ( w ; (cid:24) )]
j w 2 W } for a closed convex set W .
Differently from our setting, they do not assume strong convexity in f , and do not allow a composite term (cid:9) which is not linearized.
Assume the Markov chain that generates f z (cid:28) g t(cid:28)
=1 are fast mixing with ϵ t ( (cid:19) )
(cid:20) M(cid:26) (cid:19)
for some M >and (cid:26) 2 [0 ; 1) , then the EMD algorithm produces iterates that satisfy the following convergence rate
[Equation]
As in Remark 1, for the same fast mixing Markov chain, we set (cid:19) =
O ( log t
[Equation]
and ϵ t ( (cid:19) )
= 1 =t in
Theorem 8, we obtain the rate
[Equation]
which is also the rate for E [ f ( w t +1 ) (cid:0) f (cid:3) ] .
Both results characterize the dependence of convergence rate on (cid:26) , the mixing parameter of the Markov chain.
However, our result exploits the strong convexity of the optimization problem and achieves the faster rate 1 =t , while also achieving convergence in iterates rather than only in function values.
Theorem 9 (DA Convergence, Block-Independent Case) .
Fix an integer K (cid:21) 1 .
Let f 1 = (cid:28) 1 < (cid:28) 2 < : : : (cid:28) K +
=
t g be an increasing subsequence of [ t ] .
Using each two consecutive points, form the interval I k :=
[ (cid:28) k ; (cid:28) k
+1 (cid:0) 1] .
Then P := f
I k g
Kk =1 is a partition of [ t ] .
Deﬁne by j I k j :=
(cid:28) k +1 (cid:0) (cid:28) k
(cid:21) 1 the length of the interval and jPj 1 :=
max k j I k
j the maximum length of the intervals.
Associated with the input distribution P and the partition P deﬁne the block-wise deviation from (cid:5) by
[Equation]
TV :
Assume f z (cid:28) g t(cid:28)
=1 are block-wise independent according to the partition P .
Then, for all t (cid:21) 1 ,
[Equation]
Moreover, the rate ~ O ( (cid:14) b + jPj 2 1 =t ) applies to E [ ∥ w t +1 (cid:0) w (cid:3) (cid:13) ∥ 22 ] and E [ ∥ w (cid:3) (cid:13) (cid:0) w (cid:3) (cid:5) ∥ 22 ] by the deduction in Appendix D.1.
Let us brieﬂy we comment on the dependence on j P j 1 .
Suppose there are in total K blocks, each of equal length jPj 1 = q , and blocks are i.i.d.
We still allow arbitrary dependence within a block.
Moreover, we choose (cid:5) =
(cid:22) P in the deﬁnition of (cid:14) b .
This implies (cid:14) b =
and then the rate in
Theorem 9 is q 2 =t .
Consider dual averaging with the knowledge of the block structure q .
Then the rate 1 =K
= q=t can be achieved by executing DA using one randomly chosen data point within a block, throwing away the rest in that same block.
Such selection produces K i.i.d.
samples from (cid:22) P .
In comparison, the rate in Eq.
(26) is worse off by a factor of q due to not knowing the block-structure information.
D.4 Proofs for Markov Case
Now we consider data that are not necessarily independent across time.
We restrict our attention to ergodic processes, meaning data tend to be independent as they grow apart in time.
Deﬁne the (cid:19) -step deviation from stationarity
[Equation]
An equivalent quantity is the ϵ -mixing time Duchi et al.
[Equation]
min
[Equation]
This means, no matter where and when we start the process, it takes only (cid:19) steps to get ϵ t ( (cid:19) ) -close to the stationary distribution (cid:5) .
One could expect the deviation ϵ t ( (cid:19) ) decreases as (cid:19) increases.
This makes sense because for large (cid:19) , the process has run long enough to reach stationarity.
Theorem 10 (Mixing Data, Restatement of Theorem 8) .
It holds for all t (cid:21) 1 and any 1 (cid:20) (cid:19) (cid:20) t (cid:0) 1 ,
[Equation]
[Equation]
where O ( (cid:1) ) hides dependence on constants, G and (cid:22) F .
Here there is a trade-off in (cid:19) in the last two terms.
Proof of Theorem 10 We use the proof in Duchi et al.
; see Eq.
(6.2) in the paper.
Decompose
R t ( w (cid:3) (cid:5) ) as follows.
[Equation]
By optimality of w (cid:3) (cid:5) we have C (cid:21) 0 .
By boundedness of F we get j D j (cid:20)
(cid:19) (cid:22) F . Remains to handle
A and B. We will show
[Equation]
Bounding A.
The key is z (cid:28) + (cid:19) is almost independent of F (cid:28) (cid:0) 1 if (cid:19) is moderately large.
For each (cid:28) ,
[Equation] [Equation]
(cid:1)
[Equation]
(cid:28) + (cid:19)
[Equation]
(cid:1)
j z 1: (cid:28) (cid:0) 1 ) (cid:0) (cid:5) ∥ TV ] (cid:20)
[Equation]
(cid:22) F (cid:1) sup z 1 ;:::;z (cid:28) (cid:0)∥ P (cid:28) + (cid:19) ( (cid:1)
j z 1: (cid:28) (cid:0) 1 ) (cid:0) (cid:5)
∥ TV (cid:20) 2 (cid:22)
Fϵ t ( (cid:19) ) :
Analysis for j E [ F ( w (cid:3) (cid:5) ; z (cid:28) + (cid:19) ) (cid:0) ϕ (cid:5) ( w (cid:3) (cid:5) )]
j is almost identical.
Next sum over (cid:28)
= 1 ; : : : ; t (cid:0) (cid:19) .
[Equation]
Bounding B.
The change in F by (cid:19) steps of updates, starting from w (cid:28) , is controlled by c (cid:1) (cid:19)G (cid:1) 1 (cid:28) where 1 =(cid:28) acting like a stepsize.
Lemma 6.
Let (cid:5) (cid:9) ; W ( g ) :=
arg min w 2W f⟨ g; w ⟩ + (cid:9)( w ) g .
If (cid:9) is (cid:27) -strongly convex, then
[Equation]
Proof.
See (Nesterov, 2003, Lemma 6.1.2).
Noting w
[Equation]
and w
[Equation]
) , Lemma 6 gives
[Equation]
It holds (cid:5) -a.s.
that for each (cid:28) , the map w 7!
F ( w; z (cid:28) + (cid:19) ) is Lipschitz with parameter G .
[Equation]
+ (cid:19) (cid:0) w (cid:28) ∥ ] (cid:20) G (cid:1) (cid:28) + (cid:19) (cid:0) 1 ∑ t ′ =
(cid:28)
E [ ∥ w t ′ +1 (cid:0) w t ′ ∥ ] (cid:20) G (cid:1) (cid:28) + (cid:19) (cid:0) 1 ∑ t ′ =
(cid:28) 2 G= ( (cid:27)t ′ )
(cid:20) G (cid:1) (cid:28) + (cid:19) (cid:0) 1 ∑ t ′ =
(cid:28) 2 G= ( (cid:27)(cid:28) )
=
2 G 2 (cid:19)=(cid:28) :
Summing over (cid:28) = 1 ; : : : ; t (cid:0) (cid:19) , we get
[Equation]
Putting together,
[Equation]
; and
[Equation]
We complete the proof of Theorem 10.
D.5 Proofs for Periodic Case
Assume f z (cid:28) g t(cid:28)
=1 are block-wise independent according to the partition P .
Given the partition P , deﬁne
[Equation]
Note we compute the deviation in a block-wise manner.
Note (cid:14) block t = t (cid:1) (cid:14) b with (cid:14) b deﬁned in
Theorem 9.
Theorem 11 (Block-wise Independent Data, Restatement of Theorem 9) .
It holds
[Equation]
where O ( (cid:1) ) hides dependence on constants, G and (cid:22) F .
Generally, compared with (cid:14) t deﬁned in Eq.
(23), our new notion of deviation can be much smaller for block-structured data.
This is especially true when each block of data, as a whole, forms a good estimate of (cid:5) , but each data point in the block deviates from (cid:5) by a constant amount.
The periodic case in Remark 7 exampliﬁes this.
Remark 5 (Extreme 1: Recover Independent Case) .
Setting jPj 1 = 1 and (cid:14) block t = (cid:14) t in Eq.
(23) we recover the usual rate under independence assumption (Theorem 7).
Remark 6 (Extreme 2: Fail to Recover Arbitrary Distribution Case) .
If we allow arbitrary dependence in the whole sequence (cid:13) =
f z (cid:28) g t(cid:28)
=1 , then we can only set jPj 1 = t and the bound is useless.
Remark 7 (The Gain from Block Structure) .
Although Theorem 7 applies to block-structure data, we obtain signiﬁcant improvement in Theorem 11.
Consider the periodic case where each block is of length q and blocks are i.i.d.
At the start of block
I k , we draw a sample from (cid:5) , i.e., z t k (cid:24) (cid:5) , and then let rest of the z (cid:28) ’s in that block equal z t k .
In this case (cid:14) block t = 0 because the marginal of every z (cid:28) is exactly (cid:5) .
Then the bound in Theorembecomes
[Equation]
(31) which converges to zero at the rate q 2 =t .
However, the bound in Theorem 7 fails to converge.
To see this let us estimate (cid:14) t in Eq.
Consider some (cid:28) in the interval
I k .
If (cid:28) ̸ = t k , then the conditional distribution P (cid:28) ( (cid:1)
j z 1: (cid:28) (cid:0) 1 ) is a point mass on z t k .
If (cid:28) =
t k then P (cid:28) ( (cid:1)
j z 1: (cid:28) (cid:0) 1 ) = (cid:5) .
Let c = sup z ∥ (cid:14) z (cid:0) (cid:5) ∥ TV .
The quantity c is positive unless (cid:5) is a point mass.
Then (cid:14) t = sup
z 1 ;:::
[Equation]
t
K
[Equation]
and then Theorem 7 becomes
[Equation]
Proof
of
Theorem
We decompose the regret by blocks.
[Equation] [Equation]
Rewrite the ﬁrst sum by adding and then subtracting the term ∑ Kk
[Equation]
∑
[Equation]
K ∑ k
= ∑ (cid:28)
I k (
F ( w (cid:28) ; z (cid:28) ) (cid:0) ϕ (cid:5) ( w (cid:28) k ) )
=
K
∑ k
= ( ∑ (cid:28)
I k (
F ( w (cid:28) ; z (cid:28) ) (cid:0) F ( w (cid:28) k ; z (cid:28) ) ) :
= B k ) (I) +
K
∑ k
= ( ∑ (cid:28)
I k (
F ( w (cid:28) k ; z (cid:28) ) (cid:0) ϕ (cid:5) ( w (cid:28) k ) ) :
= A k ) : (II)
[Table]
TV :
Then sum over k = 1 ; : : : ; K , and we have (cid:12)(cid:12) E
[I] (cid:12)(cid:12)
[Equation]
∑
[Equation]
∑
[Equation]
=
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)
∑ (cid:28)
[Equation]
k ( P (cid:28) (cid:0)
[Equation]
(cid:13)(cid:13)(cid:13)(cid:13)
TV (cid:20) 2 (cid:22) F(cid:14) block t :
Bounding B k .
Using Lemma 6 and Eq.
(28), we have
[Equation]
=
[Equation]
k :
Then sum over k = 1 ; : : : ; K , and we have (cid:12)(cid:12) E [II] (cid:12)(cid:12) (cid:20) K
∑ k = (cid:12)(cid:12) E
[ B k ] (cid:12)(cid:12)
[Equation]
It can be shown the second sum in the regret decomposition (Eq.
(32)) is upper bounded by 2 (cid:22)
F(cid:14) block t .
The third sum is (cid:21) 0 .
Using Lemma 1 we get
E
[Equation] [Equation]
We complete the proof of Theorem 11.
E Proofs for PACE E.1 Proof of Theorem 5 and Theorem
We show convergence of (cid:12) under different input assumption.
Recall (cid:12) t is the pacing multiplier generated by PACE, and (cid:12) (cid:3) is the solution to the optimization problem Eq.
The vector (cid:13) is the sequence of items.
Theorem 12 (Restatement of Theorem 5 and Theorem 6) .
For the independent case, i.e., (cid:13) (cid:24) Q and Q 2 C ID ( (cid:14) ) , it holds for t (cid:21)
[Equation]
and for t (cid:21) 3 ,
[Equation]
∑
[Equation]
=
[Equation]
For the erogdic case, i.e., (cid:13) (cid:24) Q and Q 2 C E ( (cid:14); (cid:19) ) , it holds for t (cid:21) 1 ,
E [ ∥
[Equation]
t +1 (cid:0) (cid:12) (cid:3) ∥
[Equation]
(cid:20)
[Equation]
37) and for t (cid:21) 3 , t t ∑
[Equation]
=
E
[Equation]
∥
[Equation]
O
[Equation] [Equation]
For the periodic case, i.e., (cid:13) (cid:24) Q and Q 2 C P ( q ) , it holds for t (cid:21)
[Equation]
(cid:3)
∥
[Equation]
] (cid:20)
[Equation] [Equation]
[Equation]
and for t (cid:21) 3 ,
[Equation]
Proof of Theorem 12.
Set (cid:27) = 1 =n and G = (cid:22) F = j v j 1 .
Eq.
(35) follows by Theorem 7 and speciﬁcally Eq.
The next inequality Eq.
(36) follows by (Xiao, 2010, Corollary 4): for t (cid:21) 3 ,
[Equation]
t ∑
[Equation]
Eq.
(37) follows by Theorem 10 and speciﬁcally Eq.
Following (Xiao, 2010, Corollary 4), we have for t (cid:21) 3 , t ∑
[Equation] [Equation]
; and thus Eq.
(38) holds.
The inequality Eq.
(39) follows from Theorem 11 and speciﬁcally Eq.
For the inequality
Eq.
(40), apply the same strategy: for t (cid:21) 3 , t ∑
[Equation]
=
[Equation] [Equation]
E.2 Proof of Lemma 3 and LemmaProof of Lemma
Lemma 3 follows from (Gao et al., 2021, Theorem 3 and 4).
Proof of Lemma
Deﬁne the hindsight average equilibrium utility u (cid:13)i :=
(1 =t ) (cid:1) U
(cid:13)i .
Although results in Gao et al.
(2021) were stated for i.i.d.
case, the proof in fact goes through for nonstationary input distributions.
Bounding E [ ∥ (cid:22) u t (cid:0) u (cid:13) ∥ 2 ] .
For the ﬁrst inequality we use the proof of (Gao et al., 2021, Theorem 6).
Follow that paper, we deﬁne r ti = max f 0 ; (cid:22) u ti (cid:0) u (cid:13)i g .
In Theorem 6 the authors show
[Equation]
In particular, the constant C r; 1 comes from the constant C in (Gao et al., 2021, Theorem 4) and C r;comes from (Gao et al., 2021, Equation (11))
Bounding Reg i;t .
Note Reg i;t = t (cid:1)
( u (cid:13)i (cid:0)
[Equation]
u
[Equation] [Equation]
(cid:1) r ti .
Then we use Cauchy-Schwarz.
[Equation]
Bounding Envy i;t .
For the second inequality we use the proof of Theorem 6 in the same paper.
Following that paper, we deﬁne
[Equation]
) ; x i ⟩ } :
During the course of proving Theorem 6, the authors show
[Equation]
In particular, the constant C e; 1 comes from (Gao et al., 2021, Theorem 4, Equations (5) and (13)) and C e; 2 comes from (Gao et al., 2021, Equations (5), (13) and (15))
Then using Cauchy-Schwarz,
[Equation]
This completes the proof of Lemma 4.
F Experiments
We conduct experiments on a market (a matrix of buyers’ valuations on items) generated from the
MovieLens dataset (Harper and Konstan, 2016) with n = 100 buyers and m = 300 items.
The process of turning the MovieLens dataset into the market instance is described in Kroer et al.
Here, we brieﬂy describe the experiment settings.
For more details on the experiment settings as well as all code and data to replicate the results, please refer to the Supplementary Material.
We generate item arrivals from the following data input models:
(cid:15) i.i.d.
: Every item (cid:18) t 2 [ m ] is sampled independently from a ﬁxed distribution s 0 2 ∆ m (an m -dimensional probability vector).
(cid:15) Mild corruption: (cid:18) t (cid:24) s t , where s t 2 ∆ m is a distribution such that ∥ s t (cid:0) s
∥ 1 = (cid:2)(1 =t ) for all t .
Here, s t is generated by randomly perturbing each coordinate of s 0 followed by normalization.
(cid:15)
Markov: ( (cid:18) t ) t (cid:21) 1 is sampled from an irreducible Markov chain starting from an initial distribution s 0 .
It is a special case of ergodic input.
Here, the Markov chain is given by a m (cid:2) m transition matrix (each row sums to 1 ), which we generated randomly (and rowwise normalized).
In this case, the “reference” item arrival distribution is the stationary distribution of this Markov chain which is in general different from the initial distribution.
(cid:15)
Periodic:
The period length is ℓ = 100 .
Let ( s k ) k 2 [ ℓ ] be a set of distributions (probability vectors).
Here, each s k is sampled randomly and normalized.
The item arrivals of each period is generated by sampling from each s k followed by a random permutation over the ℓ sampled items.
[Figure]
For each (ﬁxed data input model, we generate 10 sample paths of item arrivals and run PACE for T = 200 n = 20000 time steps on each sample path.
Then, we measure the convergence of the pacing multipliers and time-averaged cumulative utilities to their hindsight equilibrium values.
More speciﬁcally, we record the following relative differences: max i fj (cid:12) ti (cid:0) (cid:12) HS i j =(cid:12) HS i g and max i fj (cid:22) u ti (cid:0) u HS i j =u HS i g , where HS denote the hindsight equilibrium values of the “sample-path” market determined by the realized item arrivals.
Equivalently, u HS and (cid:12) HS are optimal solutions of the hindsight convex programs (1) and (2), respectively.
We also measure the performance of a proportional-share baseline solution that divides each arriving item among all buyers proportionally w.r.t.
their budgets: for an arrived item (cid:18) t , each buyer i gets B i amount of it and receives utility
B i v i ( (cid:18) t ) (in this paper, the buyers’ budgets are B i =
=n for all i ).
We compute the means and standard errors of the error measures across the 10 sample paths and plot them in Figure 1.
As can be seen, for all data input models, the pacing multipliers and buyers’ time-averaged utilities converge to their respective hindsight values and quickly outperform the baseline proportional-share solution.
Similar convergence behavior can also be observed when the error metrics are w.r.t.
to the true equilibrium values (cid:12) (cid:3) , u (cid:3) instead of the hindsight values.
We further conduct the following experiment to demonstrate the effect of nonstationarity on pacing multiplier convergence.
More speciﬁcally, we generate 10 sample paths from each of the following item arrival settings: i.i.d.
distributions ( (cid:18) t (cid:24) i : i : d : s ), perturbed distributions (small (cid:14) ), perturbed distributions (large (cid:14) ), where (cid:14) is the overall difference between the sequences of i.i.d.
distributions and perturbed distributions, as in ( ?? ).
We then run PACE on each sample path to obtain (cid:12) ti and (cid:22) u ti values.
For each setting and each time step, we plot mean values and standard error bars of the relative error metrics, where (cid:12) (cid:3) i and u (cid:3)
i are the equilibrium pacing multipliers (utility prices) and utilities of the market with supplies being the distribution s .
As can be seen, for perturbed distribution settings, PACE is able to bring (cid:12) t and (cid:22) u ti close to their equilibrium values, while the
[Figure]
      
[Figure]
convergence degrade as the perturbation amount (cid:14) increases.
Recall that, in terms of cumulative utility, the proportional-share baseline solution gives a relative error of around 0 : 45 .