SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Proof outline: As discussed above, the surrogate problem greedily replaces the non-causal terminal gradient  $\phi'(R_i(T))$  in Eqn. (9) by the current gradient  $\phi'(R_i(t - 1)), \forall i$  on round  $t$ . To prove Lemma 1, we show that this transformation can be done by incurring only a small penalty factor to the overall regret bound. To show this, we split the regret expression (9) into the difference between two terms - term  $(A)$  corresponding to the cumulative reward accrued by the static allocation  $(\boldsymbol{y}^*)$  and term  $(B)$  corresponding to the reward accrued by the online policy. Next, we compare these two terms separately with the corresponding terms  $(A')$  and  $(B')$  in the regret expression for the surrogate online linear optimization problem (10). By exploiting the non-decreasing nature of the cumulative rewards, we first show that  $(A) \leq (A')$  (Eq. (15)). Next, by using properties of the  $\alpha$ -fair utility function, we show that  $(B') \leq (1 - \alpha)^{-1}(B + m)$  for any policy (Eq. (16)). Lemma 1 then follows by combining these two results. See Appendix C.2 for the proof.
3.2 Online Learning Policy for the Surrogate Problem and its Regret Analysis
Having established that the  $c_{\alpha}$ -regret of the original problem is upper-bounded by the regret of the surrogate learning problem, we now proceed to upper bound the regret of the surrogate problem under the action of a no-regret learner with a second-order regret bound. In the sequel, we use the projected Online Gradient Ascent (OGA) with adaptive step sizes [Orabona, 2020, Algorithm 2.2] for designing a no-regret policy for the surrogate problem. We call the resulting online learning policy ONLINE FAIR ALLOCATION (OFA). The pseudocode of the OFA policy is given in Algorithm 1.
The Online Fair Allocation policy (OFA): Since  $\phi'(R_i(t - 1)) = 1 / R_i(t - 1)^\alpha$ , at the end of round  $t$ , each user  $i$  computes its gradient component  $g_i$  by dividing its current demand vector  $x_i(t)$  with its current cumulative reward  $R_i(t - 1)$  raised to the power  $\alpha$  (line 5 of Algorithm 1). This formalizes the intuition that a user with a larger current cumulative reward has a smaller gradient component. In line 8, we take a projected gradient ascent step. Depending on the problem, the projection can often be efficiently computed, e.g., using variants of the Frank-Wolfe algorithm given access to an efficient LP
See Appendix C.1 for upper bounds on the Euclidean diameter of the action sets for Examples 2.1-2.3.
Algorithm 1 The Online Fair Allocation (OFA) Policy
1: Input: Fairness parameter  $0 \leq \alpha < 1$ , Demand/Reward vectors from the agents  $\{\pmb{x}(t)\}_{t=1}^{T}$ , Euclidean projection oracle  $\Pi_{\Delta}(\cdot)$  onto the feasible set  $\Delta$ , an upper-bound  $D$  to the Euclidean diameter of the feasible set.  
2: Output: Online resource allocation decisions  $\{y_t\}_{t=1}^T$   
3:  $R_i \gets 1, \forall i \in [m], S \gets 0$   
4: for each round  $t = 2:T$ : do  
5:  $g_i \gets \frac{x_i(t-1)}{R_i^\alpha}, \forall i \in [m]$  Computing the gradient components for each agents  
6:  $g \gets (g_1, g_2, \ldots, g_m)$  Computing the full gradient  
7:  $S \gets S + ||g||_2^2$ , Accumulating the norm of the gradients  
8:  $y \gets \Pi_{\Delta}\left(y + \frac{D}{2\sqrt{S}} g\right)$  Updating the inclusion probabilities using OGA  
9: [Optional] Sample a randomized integral allocation  $Y$  s.t.  $\mathbb{E}[Y] = y$ .  
10: The agents reveal their demand/reward vectors  $\{x_i(t)\}_{i \in [m]}$  for the current round.  
11:  $R_i \gets R_i + \langle x_i(t), y_i \rangle$ ,  $\forall i \in [m]$ . Updating cumulative rewards  
12: end for each
oracle. With additional structure, OFA can be simplified further with an even more efficient projection. For example, we give a simplified implementation for the online shared caching problem in Appendix C.5 by exploiting the fact that the action vector  $y(t)$  is the same for all users. Step 9 is an optional sampling step which is executed only if an integral allocation is required. We discuss this step in Appendix C.6. Finally, the cumulative rewards of all users are updated in Step 11. The following lemma gives an upper bound to the regret of the OFA policy for the surrogate problem.
Lemma 2. The Online Fair Allocation policy, described in Algorithm 1 achieves the following standard regret bound for the surrogate problem (10) for the  $\alpha$ -fair utility function:
$$
L \text {- R e g r e t} _ {T} = \left\{ \begin{array}{l l} O (T ^ {1 / 2 - \alpha}), & i f 0 <   \alpha <   ^ {1} / 2, \\ O (\sqrt {\log T}), & i f \alpha = ^ {1} / 2 \\ O (1), & i f ^ {1} / 2 <   \alpha <   1. \end{array} \right.
$$
Further, under this policy, the cumulative rewards of each user increase linearly with time, i.e.,  $R_{i}(T) = \Omega (T),\forall i,T.$
Proof outline: One of the major challenges in the regret analysis of the surrogate problem (10) is that the coefficients of the gradients  $\{\phi'(R_i(t - 1)), i \in [m]\}$  on round  $t$  depends on the past actions  $\{\pmb{y}(\tau)\}_{\tau = 1}^t$  of the policy itself. Since the second-order regret bound of any online linear optimization problem scales with the norm of the gradients, we now need to simultaneously control the regret and the norm of the gradients generated by the online policy. Surprisingly, the proof of Lemma 2 shows that the proposed OGA policy with adaptive step sizes not only provides a sublinear regret but also keeps the gradients small, which in turn helps keep the regret small. In fact, these two goals are well-aligned to each other, and our proof exploits the reinforcing nature of these two objectives via a new bootstrapping technique. See Appendix C.4 for the proof.
Finally, combining Lemma 1 and Lemma 2, we obtain the main result of this paper.
Theorem 1. The Online Fair Allocation (OFA) policy, described in Algorithm 1, achieves the following approximate regret bound for the NOFRA problem (4) with  $c_{\alpha} \equiv (1 - \alpha)^{-(1 - \alpha)}$ :
$$
\operatorname {R e g r e t} _ {T} \left(c _ {\alpha}\right) = (1 - \alpha) ^ {\alpha} \left\{ \begin{array}{l} O \left(T ^ {1 / 2 - \alpha}\right) \text {i f} 0 <   \alpha <   1 / 2, \\ O \left(\sqrt {\log T}\right) \text {i f} \alpha = 1 / 2 \\ O (1) \text {i f} 1 / 2 <   \alpha <   1, \end{array} \right. \tag {12}
$$
Remarks: 1. For the job scheduling problem in the reward maximization setting, Even-Dar et al. [2009, Lemma 4] showed that if the cumulative reward is concave and the offline optimal reward is convex, then their proposed approachability-based recursive policy, whose complexity scales exponentially with the number of machines  $m$ , achieves sublinear regret. In Section 7 of the same paper, the authors posed an open problem of attaining a relaxed goal when the above sufficient condition is violated. In Appendix C.8, we show that for the  $\alpha$ -fair utility function (3),
the offline optimal cumulative reward is non-convex in the regime  $0 < \alpha < 1$ . Hence, Theorem 1 gives a resolution to the above open problem by exhibiting a simple online policy with a sublinear approximate regret when the given sufficient condition is violated. Furthermore, the computational complexity of our policy is linear in the number of machines  $m$ , which is significantly lower than that of their approachability-based policy, whose complexity scales exponentially fast in  $m$ .
2. Observe that the regret bound given by Theorem 1 always remains non-vacuous, irrespective of the value of  $\alpha$  and the sequence of adversarial reward vectors. This follows from the fact that irrespective of the demand vectors, by choosing the constant action  $y = \mu \mathbf{1}_{N\times m}$  (which is feasibly by Assumption 2), each user can achieve a cumulative reward of  $\mu \delta T$ . Hence, the optimal offline value of the  $\alpha$ -fair utility function is  $\Omega(T^{1 - \alpha})$ . On the other hand, by Theorem 1, the OFA policy achieves a regret bound of  $O(T^{1/2 - \alpha})$ , which is always dominated by the optimal static offline objective.
3. When  $\alpha = 0$ , NOFRA corresponds to the cumulative reward maximization problem for all users. Hence, Theorem 1 recovers the well-known  $O(\sqrt{T})$  standard regret bound [Orabona, 2020].
The following converse result gives a universal lower bound to the approximation factor  $c$  for which it is possible to design an online policy for NOFRA with a sublinear  $c$ -regret.
Theorem 2 (Lower bound on approximation factor). Consider the online shared caching problem for the  $\alpha$ -fair reward function with  $m = 2$  users. Any online policy with a sublinear  $c_{\alpha}$ -regret must have
$$
c _ {\alpha} \geq \max  _ {0 \leq \eta \leq^ {1 / 2}} \frac {\eta^ {1 - \alpha} + (1 - \eta) ^ {1 - \alpha}}{(1 - \eta / 2) ^ {1 - \alpha} + (\eta / 2) ^ {1 - \alpha}} > 1, 0 <   \alpha <   1.
$$
See Appendix C.9 for the proof of Theorem 2. A numerical comparison between the upper and lower bounds on the approximation factor is shown in Figure 7 in the Appendix. Observe that for  $\alpha \in (0,1)$  the above lower bound is strictly greater than 1, improving on the prior best known lower bound of [Si Salem et al., 2022], which showed just that sublinear standard regret (i.e.,  $c_{\alpha} = 1$ ) is unachievable.
In this section, we report experimental results using the fair caching problem as a case study. Additional experiments on fair scheduling are provided in Appendix D.2. We compare the performance of our algorithms on two datasets against several baselines showing the effectiveness of our algorithm.
Setup and Dataset: We perform simulations on both synthetically generated data and on CDN traces from Berger [2018]. For our synthetic dataset, the request patterns of the users are highly homogeneous, letting us demonstrate the fairness behavior of different algorithms. In particular, we take time horizon  $T = 1000$  rounds, number of users  $m = 5$ , cache size  $C = 7$ , and library size  $N = 30$ . At each time step: (1) users 1 and 2 request a file from file numbers 0-29 uniformly at random; (2) user 3 repeatedly requests file numbers 0-3; (3) user 4 repeatedly requests file numbers 4-18; (4) user 5 repeatedly requests file numbers 19-27. Intuitively, the difficulty comes in ensuring a fair allocation of hits to users 1 and 2, who request files from a larger space of possibilities than the other users.
The CDN dataset Berger [2018] contains a request ID for each request, corresponding to the timestamp and the file ID of the requested file. We set  $T = 400$  rounds, number of users  $m = 4$ , cache size  $C = 10$  and, library size  $N = 50$ . For preprocessing, first, we sort the data according to the timestamp and discard all the files with an ID greater than  $N$ , and then only consider the first  $m \times T = 1600$  requests among the remaining requests. We then allocate files to users in decreasing order of their popularity, such that user 1 tends to request common files, while user 4 tends to request less common files. This makes ensuring a fair cache allocation difficult. See Appendix D.1 for details.
Comparison: We compare our algorithm with the Online Horizon-Fair (OHF) policy proposed by Si Salem et al. [2022]. Just like the OFA policy, OHF is also an online policy that ensures long-term fairness using the  $\alpha$ -fairness utility, which makes it an ideal candidate for comparison. We also compare our algorithm with the commonly used Least-Recently-Used (LRU) and Least-Frequently-Used (LFU) cache replacement policies. We compare our algorithm with the fixed offline optimal allocation and the maximin optimal allocation. Both policies have access to all the file requests ahead
The code is available at: https://github.com/AtivJoshinofra
(a) Average Hitrate
(b) Minimum Hitrate
(c) Jain's Index
Figure 2: Synthetic Dataset
(b) Minimum Hitrate
Figure 3: CDN Dataset
of time. The fixed optimal offline allocation is given by  $y^{*} = \operatorname{argmax}_{y\in \mathcal{V}}\sum_{i = 1}^{m}\phi \big(R_{i}(T)\big)$ , where  $\mathcal{V}$  is the set of feasible cache configurations (see Eqn. (5)) and  $R_{i}(T)$  is defined in Eqn. (6). The maximin optimal allocation is the fixed offline caching configuration that maximizes the minimum hitrate among all users. It can be computed via an LP. We compare these algorithms in terms of the average hit rate, minimum hit rate, and Jain's fairness index as the value of  $\alpha$  changes. Jain's fairness index is a common metric used to quantify if the users are receiving a fair share of network resources. In our case, the fairness index is given by
$$
\mathcal {J} (R _ {1} (T), \ldots , R _ {m} (T)) = \frac {(\sum_ {i = 1} ^ {m} R _ {i} (T)) ^ {2}}{m \sum_ {i = 1} ^ {m} R _ {i} ^ {2} (T)}.
$$
The index ranges from  $1 / m$  for minimum fairness to 1 for maximum fairness.
Observations: The average hitrates of each policy are shown in Figures 2a and 3a. For the synthetic dataset, despite being fair, the overall performance of the OFA policy is at par with the traditional caching algorithms like LRU and LFU policies. For the CDN dataset, the OFA policy initially performs close to the Si Salem et al. [2022] algorithm while for  $\alpha \geq 1.5$ , the OFA thoroughly outperforms it. For  $\alpha > 1$ , there is a drop in performance of OFA as compared to the "unfair" LRU and LFU algorithms. The minimum hitrates of each policy are shown in Figures 2b and 3b. For both datasets, the OFA policy outperforms Si Salem et al. [2022], LRU, and LFU. We can also see the minimum and average hitrates for the OFA policy come closer to each other as we increase  $\alpha$ . I.e., the algorithm becomes fairer at the cost of total hitrate. Finally, from the plots in Figures 2c and 3c, we can see that as  $\alpha$  increases, OFA outperforms all other algorithms in terms of Jain's fairness index. Also, Jain's index for OFA approaches the maximum value of 1 as we increase  $\alpha$ .
7Although regret guarantees for  $\alpha >1$  become vacuous, to clearly demonstrate the fairness characteristics of algorithms, we run simulations for  $\alpha \in [0,2]$
5 Conclusion and open problems
In this paper, we propose an efficient online resource allocation policy, Online Fair Allocation (OFA), that achieves a  $c_{\alpha}$ -approximate sublinear regret bound for the  $\alpha$ -fairness objective, where  $c_{\alpha} \equiv (1 - \alpha)^{-(1 - \alpha)} \leq 1.445$ , for  $0 < \alpha < 1$ . Our main technical contribution is to show that the non-additive  $\alpha$ -fairness function can be efficiently learned by greedily estimating the terminal gradients. An important follow-up problem is to investigate the extent to which the algorithmic and analytical methodologies introduced in this paper can be generalized. Note that, for the online scheduling problem, only a recursive approachability-based policy is known in the literature, whose complexity scales exponentially with the number of machines [Even-Dar et al., 2009]. The algorithm and analysis presented in this paper are specific to the  $\alpha$ -fair utility function. It would be interesting to investigate whether these ideas can be extended to learning general concave utility functions. Another related problem is to design an optimistic version of the proposed OFA policy that offers an improved regret bound by efficiently incorporating hints regarding the future demand sequence [Mhaisen et al., 2022, Bhaskara et al., 2020]. Finally, reducing the gap between the upper and lower bounds of the approximation factor in Figure 7 would be of interest.
This work is supported by a US-India NSF-DST collaborative grant coordinated by IDEAS-Technology Innovation Hub (TIH) at the Indian Statistical Institute, Kolkata, and as a supplementary fund of NSF CAREER 2045641. A. Sinha was additionally supported by the Qualcomm Research Grant IND-417880. M. Hajiesmaili's work is supported by CNS-2102963, CNS-2106299, and CPS-2136199. C. Musco was also partially supported by an Adobe Research grant.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd Conference on Innovations in Theoretical Computer Science (ITCS), ITCS '12, page 214-226, New York, NY, USA, 2012. Association for Computing Machinery. ISBN 9781450311151.
Karen Hao. Facebook's ad-serving algorithm discriminates by gender and race. MIT Technology Review, 2019.
Tian Lan, David Kao, Mung Chiang, and Ashutosh Sabharwal. An axiomatic theory of fairness in network resource allocation. IEEE, 2010.
Frank Kelly. Charging and rate control for elastic traffic. European Transactions on Telecommunications, 8(1):33-37, 1997.
Jeonghoon Mo and Jean Walrand. Fair end-to-end window-based congestion control. IEEE/ACM Transactions on Networking, 8(5):556-567, 2000.
Bozidar Radunovic and Jean-Yves Le Boudec. A unified framework for max-min and min-max fairness with applications. IEEE/ACM Transactions on Networking, 15(5):1073-1083, 2007.
Dritan Nace and Michal Pióro. Max-min fairness and its applications to routing and load-balancing in communication networks: a tutorial. IEEE Communications Surveys & Tutorials, 10(4):5-17, 2008.
Rajendra K Jain, Dah-Ming W Chiu, William R Hawe, et al. A quantitative measure of fairness and discrimination. *Eastern Research Laboratory*, Digital Equipment Corporation, Hudson, MA, 21, 1984.
Eitan Altman, Konstantin Avrachenkov, and Andrey Garnaev. Fair resource allocation in wireless networks in the presence of a jammer. Performance Evaluation, 67(4):338-349, 2010. ISSN 0166-5316. Performance Evaluation Methodologies and Tools: Selected Papers from VALUETOOLS 2008.
Elad Hazan. Introduction to online convex optimization. arXiv preprint arXiv:1909.05207, 2019.
Tareq Si Salem, Georgios Iosifidis, and Giovanni Neglia. Enabling long-term fairness in dynamic resource allocation. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 6(3):1-36, 2022.
Eyal Even-Dar, Robert Kleinberg, Shie Mannor, and Yishay Mansour. Online learning for global cost functions. In Proceedings of the 22nd Annual Conference on Computational Learning Theory (COLT), 2009.
Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Online learning: Beyond regret. In Proceedings of the 24th Annual Conference on Learning Theory, pages 559-594. JMLR Workshop and Conference Proceedings, 2011.
Shipra Agrawal, Nikhil R Devanur, and Lihong Li. An efficient algorithm for contextual bandits with knapsacks, and an extension to concave objectives. In Conference on Learning Theory, pages 4-18. PMLR, 2016.
Yuezhou Liu, Yuanyuan Li, Qian Ma, Stratis Ioannidis, and Edmund Yeh. Fair caching networks. Performance Evaluation, 143:102138, 2020. ISSN 0166-5316.
Günter M Ziegler. Lectures on polytopes, volume 152. Springer Science & Business Media, 2012.
Kun Tu, Bruno Ribeiro, David Jensen, Don Towsley, Benyuan Liu, Hua Jiang, and Xiaodong Wang. Online dating recommendations: matching markets and learning preferences. In Proceedings of the 23rd international conference on world wide web, pages 787-792, 2014.
Nicolo Cesa-Bianchi and Gábor Lugosi. Prediction, learning, and games. Cambridge University Press, 2006.
Víctor Valls, George Iosifidis, and Leandros Tassiulas. Birkhoff's decomposition revisited: Sparse scheduling for high-speed circuit switches. IEEE/ACM Transactions on Networking, 29(6):2399-2412, 2021.
Alexander L Stolyar. On the asymptotic optimality of the gradient scheduling algorithm for multiuser throughput allocation. Operations Research, 53(1):12-25, 2005b.
A. Sinha, A. Joshi, R. Bhattacharjee, C. Musco, M. Hajiesmaili
We provide a more comprehensive review of the fair machine learning literature in this section. Multiple different definitions have been used to quantify the fairness of machine learning algorithms. Hardt et al. [2016] introduced equality of opportunity as a fairness criterion, which ensures that individuals have an equal chance of being correctly classified by machine learning algorithms, regardless of their protected attributes like race or gender. Kleinberg et al. [2017] formalized three different notions of fairness and showed that no algorithm can satisfy these notions simultaneously, thus showing the inherent trade-offs in competing notions of fairness. Other prevalent fairness criteria include Price-of-fairness introduced by Bertsimas et al. [2011] which quantifies how much the aggregate utility is affected by enforcing fairness. Corbett-Davies and Goel [2016] discussed the limitations of different notions of fairness widely used in the fair machine learning literature. There have been different works that examined the societal harm caused by unfair algorithms in critical domains. For example, Chouldechova [2017] investigated the disparate impact of recidivism prediction algorithms used in the criminal justice system on different demographic groups. Since our work on online fair resource allocation is closely related to fairness in online algorithms, job scheduling, and matching, we provide a more detailed review of papers related to fair algorithms in each of these domains below.
Fairness in online algorithms: Several works have considered fairness specifically in the online setting using both regret and competitive ratio as performance metrics. Sinclair et al. [2022] considered the problem of allocating resources to individuals in an online setting, where the number and type of individuals arriving in each round are drawn from some fixed known distribution. The allocations must balance envy-freeness (requiring that each agent prefers the agent's own allocation over the other's allocation) and efficiency (the amount of consumed resources' budget). Banerjee et al. [2022] design competitive algorithms for the problem of allocating a set of divisible goods to agents in an online manner while maximizing the Nash social welfare, which is another metric that quantifies the trade-off between fairness and efficiency. They also study the setting when the algorithm is allowed to access some predictions related to the utility function of each agent over the goods. Arsenis and Kleinberg [2022] introduce notions of individual fairness in optimal stopping problems and provide competitive algorithms that satisfy these definitions of fairness.
There have been different works that consider fairness in terms of regret. Talebi and Proutiere [2018] studied a setting where tasks arriving dynamically need to be assigned to a set of servers where the probability of success of any job on any server is unknown. Using a stochastic multi-armed bandit setting, the authors showed that it is possible to achieve a proportionally fair allocation of jobs to servers in this setting. Multiple other works have studied the fairness problem in the stochastic multi-armed bandit setting [Patil et al., 2021, Joseph et al., 2016, Li et al., 2019, Chen et al., 2020]. There have also been some works recently that study algorithms with group fairness guarantees where we are concerned about being fair to a group of individuals (for example, a group might correspond to a race) instead of a single individual. [Freund et al., 2023] studies group fairness for algorithms used to assign refugees to different geographic locations. Baek and Farias [2021] studies the problem of sharing the cost of exploration in the stochastic multi-armed bandit model fairly among different groups.
Several prior works exist that design no-regret policies for specific non-additive functions. Blum and Burch [1997], Blum et al. [1999] used online learning techniques for solving the online paging and the Metrical Task System problem, which contains states. The notion of  $\alpha$ -fairness in the online setting has been considered in a couple of works including Si Salem et al. [2022] as explained previously in the introduction. Wang et al. [2022] considered an online resource allocation problem where the