SlowGuess/NeurIPS_2023_OCR huggingface dataset:
objective is to guarantee a sublinear regret for the allocation efficiency and a sublinear minimum guarantee violation penalty. The authors proposed an online policy that achieves this goal by using a weighted  $\alpha$ -fair allocation on each round while sequentially tuning the weights and the exponents of the  $\alpha$ -fairness function. Although they use round-wise  $\alpha$ -fair allocation as a tool, their objective is not to optimize the  $\alpha$ -fairness of the cumulative allocation, which is the focus of our paper.
Fairness in job scheduling: Apart from the papers cited above, the fair resource allocation problem in the context of job scheduling has also been extensively investigated by the wireless communication community, predominantly under stochastic assumptions [Stolyar, 2005a, Kushner and Whiting, 2002]. Furthermore, contrary to the online learning setting, these works typically assume that the current reward vectors (e.g., the channel states) are available to the scheduler before it makes scheduling decisions for each round. In this setting, the classic proportional fair scheduling algorithm emerges as a stochastic approximation-type gradient ascent algorithm for maximizing the logarithmic utility function [Stolyar, 2005a, Kelly et al., 1998]. Other notions of fair utility functions that have been studied in the context of wireless communication include max-min utility [Bertsekas and Gallager, 1991, Jaffe, 1981] and  $\alpha$ -fair utility [Mo and Walrand, 2000] (also called isoelastic utility function).
Fairness in matching: Fair matching algorithms have been studied in the context of resource allocation in online marketplaces in Bateni et al. [2021]. In this setting, a platform must dynamically allocate goods arriving at a platform in an online manner to customers who have their own utility and budget for each of these goods. The allocation of goods to customers must be fair to the customers while also maximizing the revenue for the platform. Different problems like online advertising where impressions must be served in an online manner to advertisers on a platform can be modeled under this setting. A proportionally fair stochastic approximation scheme was proposed in this paper. An algorithm with sublinear regret in a similar setting was proposed in Balseiro et al. [2021] where the fairness criteria were modeled as a regularizer for the objective function. Deng et al. [2023] studied how to incorporate machine-learned advice to improve fairness for bidders in the context of algorithmic bidding in online advertising. Competitive algorithms for optimizing both group and individual fairness in online matching markets which includes recommendation engines, crowdsourcing platforms etc. was proposed in Ma et al. [2023]. Fair matching algorithms for other domains like organ allocation Bertsimas et al. [2013] have also been proposed.
B Additional Examples
Beyond online fair caching, the NOfRA problem can capture a fair version of several other problems in the literature. In the following, we concretely present online fair job scheduling and online fair matching problems.
Example B.1 (Online Job Scheduling [Even-Dar et al., 2009]). In this problem, there are  $m$  machines that play the role of agents. A single job arrives at each round. The reward accrued by assigning the incoming job at round  $t$  to the  $i^{\text{th}}$  machine is given by  $x_{i}(t)$  where  $x_{i}(t) \in [\delta, 1]$ ,  $\forall i \in [m]$ , where  $\delta > 0$  is a small positive constant. Before the rewards for round  $t$  are revealed, an online allocation policy selects a probability distribution  $y_{t}$  on  $m$  machines such that
$$
\sum_ {i = 1} ^ {m} y _ {i} (t) = 1, y _ {i} (t) \geq 0, \forall i \in [ m ].
$$
The policy allocates a fraction  $y_{i}(t)$  of the job to the  $i^{\text{th}}$  machine  $\forall i \in [m]$ . As a result, the  $i^{\text{th}}$  machine accrues a reward of  $x_{i}(t)y_{i}(t)$  on round  $t$ . Hence, the cumulative reward accrued by the  $i^{\text{th}}$  machine in a time-horizon of length  $T$  is given by:
$$
R _ {i} (T) = 1 + \sum_ {t = 1} ^ {T} x _ {i} (t) y _ {i} (t), \forall i \in [ m ].
$$
The objective of the online fair job scheduling problem is to design an online allocation policy that achieves a sublinear regret with respect to the  $\alpha$ -fairness of the cumulative rewards defined in (4). From the above description, it is immediately clear that this problem is a special case of the general NOFRA problem. The online job scheduling problem occurs in many practical settings, e.g., in the targeted ad-campaigning problem discussed in the introduction, the ads can be modelled as jobs, and different groups of users can be modelled as machines. In OFDMA wireless systems, the
fair allocation of frequency resource blocks to different users where the wireless channels change dynamically can be straightforwardly modelled as an instance of the online job scheduling problem.
A Bipartite graph  $\mathcal{G}$
A Matching in  $\mathcal{G}$
Figure 4: Illustrating the Online Fair Matching (OFM) problem for zero-one demands. The existence of an edge denotes unit demand, and likewise, the absence of any edge implies zero demand.
Example B.2 (Online Matching). Consider an  $m \times m$  bipartite graph where the vertices on the left denote the agents and the vertices on the right denote the resources. On every round, each agent can be matched with one resource only, where we allow fractional matchings. The  $m \times 1$  demand vector of the agent  $i$  on round  $t$  is denoted by  $x_{i}(t)$ . The  $j^{\text{th}}$  component of the demand vector  $x_{i}(t)$  denotes the potential reward accrued by the  $i^{\text{th}}$  agent had it been completely matched with the  $j^{\text{th}}$  resource on round  $t$ . Let the binary action variable  $y_{ij}(t) \in [0,1]$  denote the amount by which the agent  $i$  is matched with resource  $j$  on round  $t$ . Hence, the reward accrued by the agent  $i$  on time  $t$  is given by  $\langle x_{i}(t), y_{i}(t) \rangle$ . Let  $\Delta$  denote the convex hull of all matchings. It is well known that  $\Delta$  can be succinctly represented by the set of all  $m \times m$  doubly stochastic matrices, a.k.a. the Birkhoff polytope [Ziegler, 2012]). In other words, in the OFM problem, the set of all feasible actions  $\Delta$  consists of all  $m \times m$  matrices  $(y_{ij})_{i,j}$  satisfying the following constraints:
$$
\sum_ {i = 1} ^ {m} y _ {i j} = 1, \forall j, \sum_ {j = 1} ^ {m} y _ {i j} = 1, \forall i, 0 \leq y _ {i j} \leq 1, \forall i, j. \tag {13}
$$
Let the variable  $R_{i}(t)$  denote the cumulative rewards accrued by the  $i^{\text{th}}$  agent up to round  $t$ . Clearly,
$$
R _ {i} (t) = R _ {i} (t - 1) + \left\langle x _ {i} (t), y _ {i} (t) \right\rangle , R _ {i} (0) = 1, \forall i \in [ m ], \tag {14}
$$
It can be verified that Assumption 2 holds in this problem with  $\mu = m^{-1}$  by noting that  $m^{-1}\mathbf{1}_{m\times m}\in \Delta$ . The objective of the Online Fair Matching (OFM) problem is to design an online matching policy that minimizes the  $c$ -Regret (4). From the above formulation, it can be immediately seen that the Online Matching problem is an instance of the NOFRA problem. Furthermore, it also generalizes the online scheduling problem described in Example B.1.
The online matching problem arises in numerous practical settings. For example, in the problem of online Ad Allocation, there are  $m$  different advertisers whose ads need to be placed in  $m$  different display slots on a webpage. Each slot can accommodate only one ad. On round  $t$ , a new user arrives and presents a reward vector  $x_{i}(t)$  for each advertiser. For example, the component  $x_{ij}(t)$  could denote potential revenue accrued by the advertiser if the ad  $i$  is placed on the  $j^{\text{th}}$  slot on round  $t$ . The objective of the allocation policy is to match the ads to the slots on each round so that the total earned revenue is fairly distributed among the advertisers. Similar problems arise in designing recommendation systems for crowd-sourcing, online dating websites [Tu et al., 2014], and fair channel assignment in wireless networks [Altman et al., 2010].
As before the fractional matching can be converted to a randomized integral matching via sampling. See Section C.6 for details.
C.1 Upper bound on the diameter of the admissible sets
Lemma 3. For the online shared caching problem the diameter of the admissible action set can be bounded as follows:
$$
\operatorname {D i a m} \left(\Delta_ {k} ^ {N}\right) \leq \sqrt {2 k}.
$$
Proof. Let  $x, y \in \Delta_k^N$ . We have
$$
\left| \left| x - y \right| \right| _ {2} ^ {2} = \sum_ {i = 1} ^ {N} {\left(x _ {i} - y _ {i}\right) ^ {2}} = \sum_ {i = 1} ^ {N} {\left| x _ {i} - y _ {i} \right| \left| x _ {i} - y _ {i} \right|} \stackrel {(a)} {\leq} \sum_ {i = 1} ^ {N} {\left| x _ {i} - y _ {i} \right|} \stackrel {(b)} {\leq} \sum_ {i = 1} ^ {N} {\left| x _ {i} \right|} + \sum_ {i = 1} ^ {N} {\left| y _ {i} \right|} \stackrel {(c)} {=} 2 k,
$$
where, in (a), we have used the fact that  $0 \leq x_{i}, y_{i} \leq 1$ , in (b), we have used triangle inequality, and in (c), we have used the fact that the sum of the components of each feasible vector is  $k$ .
Lemma 4. For the online matching problem, the diameter of the admissible action set can be bounded as follows:
$$
D i a m (\Delta) \leq \sqrt {2 m}.
$$
Let  $x, y \in \Delta$ , where  $\Delta$  is the feasible set for the OFM problem. We have
$$
\big | \big | x - y \big | \big | _ {2} ^ {2} = \sum_ {i, j} \big (x _ {i j} - y _ {i j} \big) ^ {2} \leq \sum_ {i, j} \big | x _ {i j} - y _ {i j} \big | \leq \sum_ {i j} x _ {i j} + \sum_ {i j} y _ {i j} \leq 2 m,
$$
where the inequalities follow from similar arguments as in the proof of the previous lemma.
Proof. The expression for  $\mathrm{Regret}_T(\beta^{1 - \alpha})$  from Eqn. (9) can be split into the difference of two terms  $A$  and  $B$  as follows:
$$
\operatorname {R e g r e t} _ {T} (\beta^ {1 - \alpha}) = \beta^ {- \alpha} \Big [ \underbrace {\sum_ {i} \phi^ {\prime} (R _ {i} (T)) \sum_ {t = 1} ^ {T} \langle x _ {i} (t) , y _ {i} ^ {*} \rangle} _ {(A)} - \beta \underbrace {\sum_ {i} \phi^ {\prime} (R _ {i} (T)) \sum_ {t = 1} ^ {T} \langle x _ {i} (t) , y _ {i} (t) \rangle} _ {(B)} \Big ].
$$
Also, denote the corresponding terms in the regret expression (10) for the surrogate learning problem by  $A'$  and  $B'$ . We will now separately bound each of the above two terms in terms of the corresponding terms in the regret expression (10) for the surrogate learning problem.
Proving  $A \leq A'$ : Recall that the utility function  $\phi(\cdot)$  is concave. Hence, its derivative is nonincreasing. Furthermore, under the action of any policy, the cumulative reward  $R_{i}(\cdot)$  is non-decreasing for each user  $i \in [m]$ . Thus, we have  $\phi'(R_{i}(t - 1)) \geq \phi'(R_{i}(T))$  for all  $t \in [T], i \in [m]$ . Hence,
$$
A \leq \sum_ {t = 1} ^ {T} \left\langle \sum_ {i} \phi^ {\prime} \left(R _ {i} (t - 1)\right) x _ {i} (t), y _ {i} ^ {*} \right\rangle \leq A ^ {\prime}. \tag {15}
$$
Proving  $B^{\prime}\leq (1 - \alpha)^{-1}(B + m)$  : We have
$$
\begin{array}{l} B ^ {\prime} = \sum_ {i} \sum_ {t = 1} ^ {T} \phi^ {\prime} \left(R _ {i} (t - 1)\right) \left\langle x _ {i} (t), y _ {i} (t) \right\rangle \\ = \sum_ {i} \sum_ {t = 1} ^ {T} \phi^ {\prime} \left(R _ {i} (t - 1)\right) \left(R _ {i} (t) - R _ {i} (t - 1)\right) \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(a)} {\leq} \sum_ {i} ^ {(a)} \int_ {0} ^ {R _ {i} (T)} \phi^ {\prime} (R) d R \\ = \sum_ {i} \phi (R _ {i} (T)) \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(b)} {=} \quad (1 - \alpha) ^ {- 1} \sum_ {i} \phi^ {\prime} (R _ {i} (T)) R _ {i} (T) \\ \stackrel {(c)} {=} \quad (1 - \alpha) ^ {- 1} \sum_ {i} \phi^ {\prime} (R _ {i} (T)) (\sum_ {t = 1} ^ {T} \langle x _ {i} (t), y _ {i} (t) \rangle + 1) \\ \leq \left(1 - \alpha\right) ^ {- 1} (B + m), \tag {16} \\ \end{array}
$$
where, in (a), we have used the fact that the derivative  $\phi'(\cdot)$  is non-increasing and  $R_{i}(t) - R_{i}(t - 1) = \langle x_{i}(t), y_{i}(t) \rangle \leq 1$  (see Appendix C.3 below for a proof by picture), in (b), we have used the explicit form of the  $\alpha$ -fair utility function to substitute  $x\phi'(x) = (1 - \alpha)\phi(x)$ , and in (c), we have used (2). Combining (15) and (16) and choosing  $\beta = (1 - \alpha)^{-1}$ , we conclude that, letting L-Regret $_T$  be the surrogate regret defined as in (10),
$$
\operatorname {R e g r e t} _ {T} \left(\left(1 - \alpha\right) ^ {- (1 - \alpha)}\right) \leq \left(1 - \alpha\right) ^ {\alpha} \cdot \mathrm {L} - \operatorname {R e g r e t} _ {T} + \left(1 - \alpha\right) ^ {- (1 - \alpha)} m.
$$
C.3 Graphical proof of the inequality:
$$
\phi^ {\prime} \left(R _ {i} (t - 1)\right) \left(R _ {i} (t) - R _ {i} (t - 1)\right) \leq \int_ {R _ {i} (t - 1) - 1} ^ {R _ {i} (t) - 1} \phi^ {\prime} (R) d R. \tag {17}
$$
Proof. We use the fact that  $\phi'(\cdot)$  is a non-increasing function and  $0 \leq R_i(t) - R_i(t - 1) \leq 1$ . Then the inequality (17) follows straightforwardly from the schematic below.
Figure 5: Graphical proof of (17)
The final inequality (a) then follows by summing up the above inequalities.
C.4 Proof of Lemma 2
Proof. We will be using the following adaptive regret bound for the online gradient descent policy with an appropriate adaptive step sizes sequence. We will see that the norm of the gradients diminishes
at a steady rate under the action of the OFA policy. Hence, the data-dependent regret bound plays a central role in a tight regret analysis of the OFA policy.
Theorem 3 (Theorem 4.14 of Orabona [2020]). Let  $\Delta \subset \mathbb{R}^d$  be a convex set with a diameter  $D$ . Let us consider a sequence of linear reward functions with gradients  $\{\pmb{g}_t\}_{t \geq 1}$ . Run the Online Gradient Ascent policy with step sizes  $\eta_t = \frac{D}{\sqrt{2\sum_{\tau=1}^{t} \|g_{\tau}\|_2^2}}$ ,  $1 \leq t \leq T$ . Then the standard regret under the OGA policy can be upper-bounded as follows:
$$
\operatorname {R e g r e t} _ {T} \leq D \sqrt {2 \sum_ {t = 1} ^ {T} \| \boldsymbol {g} _ {t} \| _ {2} ^ {2}}. \tag {18}
$$
Note that the gradient component  $g_{i}$  corresponding to the  $i^{\mathrm{th}}$  user for the surrogate problem (10) on round  $t$  is given by the vector  $g_{i}(t) = \phi'(R_{i}(t))x_{i}(t)$ . Using the above data-dependent static regret bound (18), the regret achieved by the OGA policy for the surrogate problem for any round  $T \geq 1$  can be upper-bounded as follows:
$$
\operatorname {L} - \operatorname {R e g r e t} _ {T} = O \left(\sqrt {\sum_ {t = 1} ^ {T} \sum_ {i} \phi^ {\prime} \left(R _ {i} (t)\right) ^ {2}}\right) = O \left(\sqrt {\sum_ {t = 1} ^ {T} \sum_ {i} \frac {1}{R _ {i} (t) ^ {2 \alpha}}}\right), \tag {19}
$$
where we have used the fact that the demand vectors at each round are bounded.
Clearly, the regret bound (19) depends on the sequence of the cumulative rewards  $\{\pmb{R}(t)\}_{t\geq 1}$ , which is implicitly controlled by the past actions of the online policy itself. By the definition of regret, for any fixed allocation  $y^{*}\in \Delta_{k}^{N}$ , we have for any time step  $T$ :
$$
\sum_ {t = 1} ^ {T} \sum_ {i} \left\langle \phi^ {\prime} \left(R _ {i} (t - 1)\right) x _ {i} (t), y _ {i} (t) \right\rangle \geq \sum_ {t = 1} ^ {T} \sum_ {i} \left\langle \phi^ {\prime} \left(R _ {i} (t - 1)\right) x _ {i} (t), y _ {i} ^ {*} \right\rangle - L - R e g r e t _ {T}, \tag {20}
$$
where  $\mathrm{L - Regret}_T$  denotes the worst-case cumulative regret of the adaptive OGD policy up to time  $T$ . Since the cumulative reward of each user is monotonically non-decreasing, we have:
$$
R _ {i} (T) \geq 1, \forall i, \forall T. \tag {21}
$$
Substituting this bound to the regret bound in (19), we obtain our first (loose) upper-bound for the regret of the fair allocation problem (10):
$$
\operatorname {L} - \operatorname {R e g r e t} _ {T} = O (\sqrt {T}). \tag {22}
$$
Note that this bound might be too loose as the offline benchmark itself could be smaller in magnitude than this regret bound. A closer inspection reveals the root cause for this looseness - the cumulative reward lower bound (21) is too loose for our purpose, as cumulative rewards grow steadily with time, depending on the online policy. We now strengthen the above upper-bound using a novel bootstrapping method, that simultaneously tightens the lower bound for the cumulative rewards (21) and, consequently, improves the regret upper bound (19).
Using the fact that  $\langle x_i(t),y_i(t)\rangle = R_i(t) - R_i(t - 1)$ , and following the same calculations up to the fourth step of (16), we have
$$
\sum_ {i} \sum_ {t = 1} ^ {T} \phi^ {\prime} \left(R _ {i} (t - 1)\right) \left\langle x _ {i} (t), y _ {i} (t) \right\rangle \leq \sum_ {i} \phi \left(R _ {i} (T)\right). \tag {23}
$$
Furthermore, lower bounding  $\phi^{\prime}(R_i(t - 1))$  by  $\phi^{\prime}(R_i(T))$ , we have
$$
\sum_ {t = 1} ^ {T} \sum_ {i} \left\langle \phi^ {\prime} \left(R _ {i} (t - 1)\right) x _ {i} (t), y _ {i} ^ {*} \right\rangle \geq \sum_ {i} \phi^ {\prime} \left(R _ {i} (T)\right) R _ {i} ^ {*} (T), \tag {24}
$$
where  $R_{i}^{*}(T) \equiv \sum_{t=1}^{T} \langle x_{i}(t), y_{i}^{*} \rangle$  is the cumulative reward accrued by a fixed allocation  $y^{*} \in \Delta$  up to time  $T$ . Using Assumptions 1 and 2 and choosing  $y_{i}^{*} = \mu \mathbb{1}_{N}$ , we have  $R_{i}^{*}(T) \geq \mu \delta T, \forall i$ . Hence, combining (23), (24), and (20), we have
$$
\sum_ {i} \phi \left(R _ {i} (T)\right) \geq \mu \delta T \sum_ {i} \phi^ {\prime} \left(R _ {i} (T)\right) - \mathrm {L} - \operatorname {R e g r e t} _ {T}. \tag {25}
$$
Figure 6: Illustrating the new Bootstrapping technique used in the proof of Lemma 2
Since  $0 < R_{i}(T) \leq T, \forall i$ , and  $\phi(\cdot)$  is monotone non-decreasing, for any user  $i \in [m]$ , the above inequality yields:
$$
\frac {m}{(1 - \alpha) T ^ {\alpha}} \geq \frac {\mu \delta}{R _ {i} ^ {\alpha} (T)} - \frac {\mathrm {L - R e g r e t} _ {T}}{T}.
$$
$$
R _ {i} ^ {\alpha} (T) \geq \mu \delta \left(\frac {\mathrm {L} - \operatorname {R e g r e t} _ {T}}{T} + \frac {m}{1 - \alpha} T ^ {- \alpha}\right) ^ {- 1} = \Omega \left(T ^ {\min  \left(1 / 2, \alpha\right)}\right), \forall i \in [ m ], \tag {26}
$$
where we have used our previous upper bound (22) on the regret. Eqn. (26) is our key equation for carrying out the bootstrapping process as it lower bounds the minimum cumulative reward of users in terms of the worst-case regret. Now we consider two cases:
Case-I:  $0 \leq \alpha \leq 1/2$
In this case, we immediately have  $R_{i}(T) = \Omega (T),\forall i,T$  . Substituting this bound on (19), we have
$$
\mathrm {L - R e g r e t} _ {T} = \left\{ \begin{array}{l} O (\sqrt {\log T}) \text {i f} \alpha = 1 / 2 \\ O (T ^ {1 / 2 - \alpha}) \text {i f} 0 <   \alpha <   1 / 2. \end{array} \right.
$$
Case-II:  $1/2 < \alpha < 1$
In this case, using the bound (26), we have  $R_{i}(T) = \Omega (T^{1 / 2\alpha}),\forall i,T$  Using the regret bound (19), the regret can be bounded as
$$
\mathrm {L} - \operatorname {R e g r e t} _ {T} = O (\sqrt {\sum_ {t = 1} ^ {T} \frac {1}{t}}) = O (\sqrt {\log T}).
$$
Substituting this bound again in (26), we have  $R_{i}(T)\geq \Omega (T)$ . This, in turn, yields the following regret bound:
$$
\mathrm {L} - \operatorname {R e g r e t} _ {T} = O \left(\sqrt {\sum_ {t = 1} ^ {T} \frac {1}{t ^ {2 \alpha}}}\right) = O (1).
$$
C.5 Simplified Pseudocode for the Online Fair Allocation Policy for the Caching Problem
Algorithm 2 Online Fair Allocation Caching (OPFC)
1: Input: Fairness parameter  $0 \leq \alpha < 1$ . Online file request sequence  $\{x_{i}(t)\}_{t=1}^{T}$  from user  $i, i \in [m]$ , Euclidean projection oracle  $\Pi_{\Delta_k^N}(\cdot)$  onto the feasible convex set  $\Delta_k^N$ .  
2: Output: Sequence of file inclusion probabilities  $\{y_t\}_{t=1}^T$  and samples from this distribution  
3:  $R_i \gets 1, \forall i \in [m], S \gets 0$ . ▷ Initialization  
4: for each round  $t = 2:T$ : do  
5:  $g \gets \sum_{i=1}^{m} \frac{x_i(t-1)}{R_i^\alpha}$   
6:  $S \gets S + ||g||_2^2$ ,  
7:  $y \gets \Pi_{\Delta_k^N}(y + \frac{k}{\sqrt{s}} g)$   
8: The users reveal their file requests for round  $t$ $\{x_i(t), i \in [m]\}$ .  
9:  $R_i \gets R_i + \langle x_i(t), y \rangle$ ,  $\forall i \in [m]$ .  
10: Sample a subset of  $k$  files by invoking Algorithm 3  
11: end for each
C.6 High-probability regret bound for randomized integral allocations
Theorem 2 establishes  $c_{\alpha} \equiv (1 - \alpha)^{-(1 - \alpha)}$ -regret guarantee for the Online Fair Allocation (OFA) policy for fractional allocations where the set of admissible actions  $\Delta$  is the convex hull of integral allocations. In the following, we show that a sub-linear  $c_{\alpha}$ -regret guarantee holds with high probability when randomized integral actions are chosen according to the optional step (9) in the OFA policy. Recall that for integral allocation, on each round  $t$  we independently sample an integral allocation  $Y_{t} \in \{0,1\}^{mN}$ , that matches with the fractional allocation  $y_{t}$  in expectation, i.e.,  $\mathbb{E}[Y_t] = y_t$  where  $y_{t}$  is the fractional allocation vector recommended by the OFA policy. A fractional allocation vector  $y_{t}$  in the convex hull of integral allocations can be turned into a randomized integral allocation by expressing  $y_{t}$  as a convex combination of integral allocations and randomly sampling one of the integral allocations with appropriate probabilities. As Appendix C.7 shows, for many problems with structure, such a decomposition can be efficiently computed.
Regret Analysis: By appealing to [Cesa-Bianchi and Lugosi, 2006, Lemma 4.1], it is enough to consider an oblivious adversary that fixes the sequence of demand vectors  $\{\pmb{x}(t)\}_{t\geq 1}$  before the game commences. Let the random variable  $\mathsf{R}_i(T)$  denote the random cumulative reward obtained by the  $i^{\mathrm{th}}$  user under the randomized integral allocation policy and the deterministic variable  $R_{i}(T)$  is defined as in Eqn. (2). By construction, we have  $\mathbb{E}[\mathsf{R}_i(T)] = R_i(T)$ . Furthermore, since  $\mathsf{R}_i(T)$  is the sum of  $T$  independent random variables, each of magnitude at most one, from the standard Hoeffding's inequality, we have
$$
\mathbb {P} \big (\left| \mathsf {R} _ {i} (T) - R _ {i} (T) \right| \geq \lambda \big) \leq \exp \big (- 2 \lambda^ {2} / T \big), \forall i \in [ m ].
$$
Hence, with probability at least  $1 - \mathrm{poly}(1 / T)$ , the aggregate  $\alpha$ -fair utility (3) accrued by the randomized policy can be lower bounded as:
$$
\begin{array}{l} (1 - \alpha) ^ {- 1} \sum_ {i} \mathsf {R} _ {i} ^ {1 - \alpha} (T) \geq (1 - \alpha) ^ {- 1} \sum_ {i} \left(R _ {i} (T) - O (\sqrt {T \log T})\right) ^ {1 - \alpha} \\ \stackrel {(a)} {\geq} \left(1 - \alpha\right) ^ {- 1} \sum_ {i} R _ {i} ^ {1 - \alpha} (T) - O \left(\left(T \log T\right) ^ {\frac {1 - \alpha}{2}}\right), \tag {27} \\ \end{array}
$$
where in inequality (a), we have used the fact that  $(x + y)^{1 - \alpha} \leq x^{1 - \alpha} + y^{1 - \alpha}, \forall 0 \leq \alpha \leq 1, x, y \geq 0$ . Combining the above with the approximate regret bound in Theorem 1, and taking the dominant term, we conclude that the  $c_{\alpha}$ -regret for the randomized integral allocation policy is upper-bounded by  $O\left(\left(T \log T\right)^{\frac{1 - \alpha}{2}}\right)$  w.h.p. for all  $0 \leq \alpha \leq 1$ . Unfortunately, unlike Theorem 1, the integral allocation policy incurs a non-trivial (but sublinear)  $c_{\alpha}$ -regret for the entire range of the fairness parameter  $0 \leq \alpha < 1$ .
C.7 Efficiently sampling an integral allocation from a mixed allocation in the convex hull