Fast Algorithms for Packing Proportional Fairness and its Dual
Abstract
The proportional fair resource allocation problem is a major problem studied in ﬂow control of networks, operations research, and economic theory, where it has found numerous applications.
This problem, deﬁned as the constrained maximization of ∑︀ 𝑖 log 𝑥 𝑖 , is known as the packing proportional fairness problem when the feasible set is deﬁned by positive linear constraints and 𝑥 ∈ (cid:82) 𝑛 ≥ 0 .
In this work, we present a distributed accelerated ﬁrst-order method for this problem which improves upon previous approaches.
We also design an algorithm for the optimization of its dual problem.
Both algorithms are width-independent
Introduction
The assignment of bounded resources to several agents under some notions of fairness is a topic studied in networking, operations research, game theory, and economic theory.
The allocation obtained by the maximization of the function ∑︀ 𝑛𝑖 =1 log( 𝑥 𝑖 ) over a convex set 𝐶 ⊆ (cid:82) 𝑛 ≥ 0 , known as a proportional fair allocation , is an important solution that arises under a natural set of fairness axioms [BFT11; Lan+10].
It corresponds to Nash bargaining solutions [Nas50] and it also has applications to multi-resource allocation in compute clusters [BR15; JH18; Joe+12], rate control in networks [Kel97] and game theory [JV10; JV07].
Other important allocations are linear objectives (no fairness), the max-min allocations [MW00], or 𝛼 -fair allocations [Atk70; MW00; McC+14], which generalize all of the others.
Proportional fairness corresponds to 𝛼 = 1 .
A natural restriction, that many of these applications require, are positive linear constraints.
This results in the packing proportional fairness problem , also known as the 1 -fair packing problem .
The main focus of this paper is on solving this problem and its dual via ﬁrst-order methods.
Given 𝐴 ∈ ℳ 𝑚 × 𝑛
( (cid:82) ≥ 0 ) , the 1 -fair packing problem is
[Equation]
We also study the optimization of its Lagrange dual, that can be formulated, cf.
Lemma A.1, as
[Equation]
[Table]
Most of the notations in this work have a link to their deﬁnitions.
For example, if you click or tap on any instance of 𝑒 𝑖 , you will jump to the place where it is deﬁned as the 𝑖 -th vector of the canonical base.
36th Conference on Neural Information Processing Systems (NeurIPS 2022).
where ∆ 𝑚 def = { 𝜆 ∈ (cid:82) 𝑚 : ∑︀ 𝜆 𝑖 = 1 , 𝜆 ≥ 0 } is the 𝑚 -dimensional (probability) simplex.
We focus on width-independent algorithms that additively 𝜀 -approximate the optimum of those problems.
For the 1 -fair packing problem that means, respectively, that we can ﬁnd ¯ 𝑥 in time that depends at most polylogarithmically on the width 𝜌 of the matrix 𝐴 , and that it satisﬁes 𝑓 * − 𝑓 (¯ 𝑥 ) ≤ 𝜀 , where 𝑓 * is the optimal value.
Note that (1FP) has a unique optimizer, by strong concavity.
By the same reason, for every two minimizers 𝜆 1 , 𝜆 2 of (1FP-Dual), we have 𝐴 𝑇 𝜆 1 = 𝐴 𝑇 𝜆 2 .
The width 𝜌 of 𝐴 is deﬁned as max { 𝐴 𝑖𝑗 } / min 𝐴 𝑖𝑗 ̸
=
{ 𝐴 𝑖𝑗 } , the maximum ratio of the non-zero entries of 𝐴 .
Note that in general width-dependent algorithms are not polynomial.
Smoothness and Lipschitz constants of the objectives do not scale polylogarithmically with 𝜌 and thus, direct application of classical ﬁrst-order methods leads to non-polynomial algorithms.
As in packing and covering LP, an approximate solution for our primal problem does not necessarily yield one for the dual problem, cf.
[AK08], so we need to study them separately.
The current form of our techniques does not generalize to 𝛼 -fair problems with 𝛼 ̸ = 1 , but generalizing them to these settings is an interesting future direction of research.
We note that previous works treat 𝛼 in [0 , 1) , { 1 } , or (1 , ∞ ) separately, due to the structure of the problems being different.
Most works dealing with 𝛼 -fair functions assume, without loss of generality, that 𝐴 is given so that the minimum non-zero entry of 𝐴 is 1 and the maximum entry is 𝜌 .
However, in this work, we assume without loss of generality that
[Equation]
We can do so because, for our problem, we can rescale each primal coordinate multiplicatively, rescaling the columns of 𝐴 accordingly, which only changes the objectives by an additive constant.
Thus, the additive guarantees we will obtain are also satisﬁed in the non-scaled problem.
Our primal algorithm solves the problem in a distributed model of computation with 𝑛 agents.
Each agent 𝑗 ∈ [ 𝑛 ] controls variable 𝑥 𝑗 and only has access to global parameters like 𝑚, 𝑛, or the target accuracy 𝜀 , to the 𝑗 -th column of 𝐴 , and in each round it receives the slack ( 𝐴𝑥 ) 𝑖 − 1 of all the constraints 𝑖 in which 𝑗 participates.
This is a standard distributed model of computation.
We refer to [KY14; AK08] for its motivation and applications.
Notations We let 𝑒 𝑖 be the vector with 1 in coordinate 𝑖 and 0 elsewhere.
We denote by 𝐴 𝑖 a row of 𝐴 .
For 𝑘 ∈ (cid:78) , we use the notation [ 𝑘 ] def = { 1 , 2 , . . .
, 𝑘 } .
Throughout this work, log( · ) represents the natural logarithm.
For 𝑣 ∈ (cid:82) 𝑛 , the notation exp( 𝑣 ) means entrywise exponential.
We use ⊙ for the entrywise product.
Given a 1 -strongly convex map 𝜓 , we denote its Bregman divergence by 𝐷 𝜓 ( 𝑥, 𝑦 ) def = ∇ 𝜓 ( 𝑥 ) − ∇ 𝜓 ( 𝑦 ) − ⟨∇
𝜓
𝑦 ) , 𝑥 − 𝑦 ⟩ .
We denote by 𝑁 the number of non-zero entries of the matrix 𝐴 .
The notation ̃︀ 𝑂 ( · ) hides logarithmic factors with respect to 𝑚 , 𝑛 , 1 /𝜀 and 𝜌 .
But note that the rates of our algorithms do not depend on 𝜌 .
Related Work Despite the importance and widespread applicability of fairness objectives, widthindependent (and thus polynomial) algorithms for many 𝛼 -fair packing problems were not developed until recently.
Width-independent algorithms were ﬁrst designed for 0 -fair packing, i.e., for packing linear programming (LP), that have a longer history [LN93].
For this problem there are currently nearly linear-time width-independent iterative algorithms [AO19] and distributed algorithms [AO15;
DO17].
[MSZ16] studied the width-independent optimization of 𝛼 -fair packing problems for any 𝛼 ∈ [0 , ∞ ] with a stateless algorithm and [DFO20] gave better rates with a non-stateless algorithm.
Both works use the same distributed framework as ours.
For the particular case of 1 -fair packing, the latter work obtains an unaccelerated algorithm that runs in ̃︀ 𝑂 ( 𝑛 2 /𝜀 2 ) distributed iterations.
[Bec+14] study the optimization of the dual problem by using Nesterov’s accelerated method, and then they reconstruct a primal solution.
However, both primal and dual solutions depend on the smoothness constant of the dual problem, which in the worst case is proportional to 𝜌 2 , and therefore it is not a polynomial algorithm.
In contrast, our algorithms do not depend on 𝜌 at all.
Obtaining a priori lower bounds on each of the coordinates of the optimizer is of theoretical and practical interest, since it provides certain amount of resource that can be assigned to each agent before solving the problem.
These were studied in [MSZ16] and were improved by [All+18].
In Lemma B.1, we show a lower bound of this kind for our problem when it is normalized as in (1).
Contribution and Main Results Our contribution can be summarized as follows; See Table 1 for a comparison with previous works.
Accelerated algorithm for 1 -fair packing.
We design a distributed accelerated algorithm for 1 -fair packing by generalizing and extending an accelerated technique, designed for packing LP, that
Paper
[Table]
[Bec+14]
[Table]
[MSZ16]
[Table]
[DFO20]
[Table]
This paper (Theorem 2.5)
[Table]
[Bec
[Table] [Table]
This paper (Theorem 3.4)
[Table]
uses truncated gradients of a regularized objective [AO19].
In contrast with this technique, ours yields an algorithm and guarantees that are deterministic.
We exploit the structure of our problem to obtain a distributed solution, while for packing LP obtaining a distributed or just parallel algorithm that is accelerated and width-independent is an open question [DO17].
We make use of a different regularization and an analysis that yields additive error guarantees as opposed to multiplicative ones.
The dual problem.
We consider the dual of the 1 -fair packing problem.
We reduce the problem to optimizing a proxy function by using the Plotkin-Shmoys-Tardos (PST) framework [PST95; AHK12] with a novel geometric separation oracle.
Critical to obtaining fast convergence is showing that the oracle parameters decrease when we obtain better solutions.
This fact allows to reduce the dependence on 𝜀 , and as a result, our width-independent algorithm enjoys a convergence rate of ̃︀ 𝑂 ( 𝑛 2 /𝜀 )
iterations for this problem.
A distributed accelerated algorithm for 1-Fair Packing
In this section, we present the main steps of our algorithm for the primal problem, which is a deterministic accelerated descent method that optimizes an objective coming from the 1 -fair packing problem, and that encodes the constraints in the form of a barrier.
Our algorithm approximates the objective additively and allows to compute each iteration in a distributed manner.
We note that [DFO20] also made use of this objective for the 1 -fair packing problem with different constants, but as opposed to their solution, we allow to compute unfeasible solutions during the course of the algorithm, and we proceed with different techniques that allow to achieve acceleration and thus an algorithm with better convergence rates.
We defer the proofs to Appendix B. We reparametrize
Problem (1FP) so that the objective function is linear at the expense of making the constraints more complex.
That is, we deﬁne the function ˆ 𝑓 : (cid:82) 𝑛 → (cid:82) , 𝑥 ↦→ 𝑓 (exp( 𝑥 )) =
[Equation] 1 𝑛 , 𝑥 ⟩ .
The optimization problem becomes
[Equation]
Then, we regularize the negative of the reparametrized objective by adding a fast-growing barrier:
[Equation]
where 𝛽 def = 𝜀 6 𝑛 log(2 𝑚𝑛 2 /𝜀 )
In this way, we can work with an unconstrained minimization problem.
The resulting function is not globally smooth but when the absolute value of a coordinate of the gradient is large, it is positive, and in that case we are able to take a small gradient descent step and decrease the function considerably.
The intuition is that if the gradient is large, then the function value along the segment of the gradient step, as a function of the step, can decrease fast.
But it cannot increase fast since there are no large negative gradient coordinates.
We depict 𝑓 𝑟 in Figure 1 in
Appendix B. The barrier also allows to maintain almost feasibility, as we show in Proposition 2.
below.
It is chosen to grow fast enough so that a point satisfying ( 𝐴 exp( 𝑥 )) 𝑖 > 1 + 𝜀/𝑛 , for some 𝑖 ∈ [ 𝑛 ] , will have an optimality gap that is greater than the required accuracy.
On the other hand, the regularizer is very small in the feasible region that is not too close to the boundary.
Let ˆ 𝑥 * be the maximizer of ˆ 𝑓 , let 𝑥 * def = exp(ˆ 𝑥 * ) be the solution to Problem (1FP), and let 𝑥 * 𝑟 be the minimizer of 𝑓 𝑟 .
We have ˆ 𝑥 * ∈ [ − log( 𝑛 ) , 0] 𝑛 by Lemma B.1.
Let 𝜔 def = log( 𝑚𝑛/ (1 − 𝜀/𝑛 )) and deﬁne the box 𝐵 def =
[ − 𝜔, 0] 𝑛 .
We restrict ourselves to this domain and formulate our ﬁnal problem, that we will minimize with an accelerated method:
[Equation]
Note 𝑓
𝑟
𝑥 ) ≥
if 𝑥 ∈ 𝐵 .
We add the redundant and simple box constraints 𝐵 in order to later guarantee a bound on the regret of the mirror descent method that runs within the algorithm.
We show that it sufﬁces to obtain an 𝜀 -minimizer of Problem (1FP-primalReg) in order to obtain an 𝑂 ( 𝜀 ) -minimizer for the original Problem (1FP).
Proposition 2.1.
[ ↓ ] Let 𝜀 ∈ (0 , 𝑛/ 2] .
Let 𝑥 * 𝑟 be the minimizer of (1FP-primalReg) and let 𝑥 𝜀𝑟 ∈ 𝐵 be an 𝜀 -minimizer of this problem.
Then the point ¯ 𝑢 def = exp( 𝑥 𝜀𝑟 ) / (1 + 𝜀/𝑛 ) satisﬁes 𝑓 ( 𝑥 * ) − 𝑓 (¯ 𝑢 ) ≤ 5 𝜀
and 𝐴 ¯ 𝑢 ≤ 1 𝑚 , where 𝑥 * is the maximizer of 𝑓 .
The intuition about this proposition is that 𝑥 𝜀𝑟 is also a point with low ˆ 𝑓 value.
By the aforementioned barrier guarantees, it is almost feasible, i.e., 𝐴 exp( 𝑥 𝜀𝑟 ) ≤ 1 + 𝜀/𝑛 , and dividing the corresponding exp( 𝑥 𝜀𝑟 ) by 1 + 𝜀/𝑛 , and thus making it feasible, can only increase the objective 𝑓 by 𝜀 .
In the sequel, we will present the different parts of Algorithm 1 and their analyses.
In particular, the notation and deﬁnitions used are compatible with the choices in the algorithm and most of the parameter choices naturally occur throughout the arguments.
Our optimization algorithm starts at the points 𝑥 (0)
=
𝑦 (0) = 𝑧 (0)
= − log( 𝑚𝑛/ (1 − 𝜀/𝑛 ))
1 𝑛 and updates each of these variables 𝑥 ( 𝑘 ) , 𝑦 ( 𝑘 ) and 𝑧 ( 𝑘 ) once in each iteration.
They remain in 𝐵 , cf.
Lemma B.2.
The role of the three variables is the following: 𝑧 ( 𝑘 ) will be a mirror point and 𝑦 ( 𝑘 ) will be a gradient descent point, in the sense that in order to compute them we apply mirror descent and gradient descent.
Then, the point 𝑥 ( 𝑘 ) will be a convex combination of both, that will balance the regret of 𝑧 ( 𝑘 ) with the primal progress of 𝑦 ( 𝑘 ) , effectively coupling these two algorithms.
It is important to note that we do not use the gradient ∇ 𝑓 𝑟 ( 𝑥 ) for our mirror descent loss.
Instead, we use a truncation of the gradient.
More precisely, the loss we perform the mirror descent step on is the truncated gradient ∇ 𝑓 𝑟 ( 𝑥 ( 𝑘 ) ) ∈ (cid:82) 𝑛 deﬁned as
[Equation]
𝑖
[Equation] [Equation] Note that ∇ 𝑓 𝑟 ( 𝑥 ( 𝑘 ) ) ∈ [ − 1 , 1] 𝑛 because ∇ 𝑓 𝑟 ( 𝑥 ) ∈ [ − 1 , ∞ ] 𝑛 for any 𝑥 ∈ (cid:82) 𝑛 , as the regularizer has positive gradient; see also deﬁnition of 𝑓 𝑟 ( 𝑥 ) and its gradient.
The truncation allows mirror descent to control one part of the regret, which will not depend on the global Lipschitz constant.
Gradient descent will compensate for both such regret and the part that is not controlled by mirror descent.
Let Π 𝒳 ( · ) be the ‖ · ‖ 2 -projection map of a point onto a convex set 𝒳 .
The mirror descent update can be written in closed form as any of the two following equivalent ways
[Equation]
That is, projecting back to the box, in case of the ‖ · ‖ 2 , consists of simply clipping each coordinate.
We bound the regret coming from this mirror descent step by modifying the classical analysis of mirror descent, cf.
Lemma B.3.
Lemma 2.2 (Mirror Descent Guarantee).
[ ↓ ] Let 𝑢 ∈ 𝐵 and choose 𝐿 as in Algorithm 1.
We have:
[Equation]
Next, we will analyze the role of the gradient descent step.
We show in the following lemma a lower bound on the progress of our descent step.
Note that this progress could not be greater than ⟨∇ 𝑓 𝑟 ( 𝑥 ( 𝑘 ) ) , 𝑥 ( 𝑘 ) − 𝑦
( 𝑘 ) ⟩ , by convexity, so this is a strong descent condition.
Lemma 2.3 (Descent Lemma).
[ ↓ ] Given 𝑥 ( 𝑘 ) and 𝑦 ( 𝑘 ) as deﬁned in Algorithm 1, the following holds:
[Equation]
[Equation]
(cid:82) ≥ 0 ) normalized as in (1).
Accuracy 𝜀 ∈ (0 , 𝑛/ 2] .
1: 𝛽 ← 𝜀 6 𝑛 log(2 𝑚𝑛 2 /𝜀 ) ; 𝜔 ← log( 𝑚𝑛 1 − 𝜀/𝑛 ) ; 𝐿 = max {︁ 4 𝜔 (1+ 𝛽 ) 𝛽 , 16 𝑛 log(2 𝑚𝑛 ) 3 𝜀 + 13 }︁ = ̃︀ 𝑂 ( 𝑛/𝜀 ) 2: 𝜂 0 ← 13 𝐿 ; 𝐶 𝑘 = 3 𝜂 𝑘 𝐿 ; 𝜏 ← 𝜏 𝑘 = 𝜂 𝑘 /𝐶 𝑘 = 13 𝐿 .
3: 𝑇 ← ⌈ log( 4 𝑛 log(2 𝑚𝑛 ) 𝜀 ) / log( 11 − 𝜏 ) ⌉ ≤ ⌈ 3 𝐿 log( 4 𝑛 log(2 𝑚𝑛 ) 𝜀 ) ⌉ = ̃︀ 𝑂 ( 𝑛/𝜀 ) 4: 𝑥 (0) ← 𝑦 (0) ← 𝑧 (0) ← − 𝜔 1 𝑛 5: for 𝑘 = 1 to 𝑇 do 6: 𝜂
𝑘 ← 𝐶 𝑘 − 𝐶 𝑘 − 1 = 11 − 𝜏 𝜂 𝑘 −7: 𝑥 ( 𝑘 ) ← 𝜏𝑧 ( 𝑘 − 1)
+ (1 − 𝜏 ) 𝑦 ( 𝑘 − 1) 8: 𝑧 ( 𝑘 ) ← arg min 𝑧 ∈ 𝐵 {︀ 12 𝜔 ‖ 𝑧 − 𝑧 ( 𝑘 − 1)
‖ 22 + ⟨ 𝜂 𝑘 ∇ 𝑓 𝑟 ( 𝑥 ( 𝑘 ) ) , 𝑧 ⟩ }︀ ◇ Mirror descent step 9: 𝑦 ( 𝑘 ) ← 𝑥 ( 𝑘 ) + 1 𝜂 𝑘 𝐿 ( 𝑧 ( 𝑘 ) − 𝑧 ( 𝑘 − 1) ) ◇ Gradient descent step 10: end for 11: return ¯ 𝑥 def = exp( 𝑦 ( 𝑇 ) ) / (1 + 𝜀/𝑛 )
Output: 𝑓
¯ 𝑥 ) 𝑥 𝜀
¯ 𝑥 is feasible
¯ 𝑥
̃︀ 𝑂 𝑛 𝜀 ) to obtain an 𝑂 ( 𝜀 ) -approximate solution.
2.1 Coupling Mirror Descent and Gradient Descent
We ﬁrst prove a lemma that shows we can compensate for the regret coming from mirror descent as well as for the rest of the regret.
Note the total weighted instantaneous regret ⟨ 𝜂 𝑘 ∇ 𝑓 ( 𝑥 ( 𝑘 ) ) , 𝑧 ( 𝑘 − 1) − 𝑢 ⟩ is bounded by the left hand side of (5) up to a difference of potential functions, by Lemma 2.2.
This is a critical part of the analysis: using the truncated gradient for mirror descent makes its corresponding regret not to depend on the smoothness constant, but there is a remaining regret that, crucially, can be compensated by our strong descent condition.
Lemma 2.4.
[ ↓ ] Let 𝐶 𝑘 def =
𝜂 𝑘 𝐿 , and let 𝜈 ( 𝑘 ) def = ∇ 𝑓 𝑟 ( 𝑥 ( 𝑘 ) )
− ∇ 𝑓 𝑟 ( 𝑥 ( 𝑘 ) ) ∈ [0 , ∞ ) 𝑛 .
For all 𝑢 ∈ 𝐵 , we have
[Equation]
With these tools at hand, we can now use a linear coupling argument to establish an accelerated convergence rate.
Note that the algorithm takes the simple form of iterating a mirror descent step, gradient descent step and a coupling, after a careful choice of parameters.
All of which depend on known quantities.
Theorem 2.5.
[ ↓ ] Let 𝜀 ≤ 𝑛/ 2 and let 𝑥 * be the solution to (1FP) and let 𝑥 * 𝑟 be the minimizer of (1FP-primalReg) .
Algorithm 1 computes a point 𝑦
( 𝑇 ) ∈
𝐵 such that 𝑓 𝑟 ( 𝑦 ( 𝑇 ) ) − 𝑓 𝑟 ( 𝑥 * 𝑟 ) ≤ 𝜀 in a number of iterations 𝑇 = ̃︀ 𝑂 ( 𝑛/𝜀 ) .
Besides, ¯ 𝑥 def = exp( 𝑦 ( 𝑇 ) ) / (1 + 𝜀/𝑛 ) is a feasible point of (1FP) , i.e., 𝐴 ¯ 𝑥 ≤ 1 𝑚 , such that 𝑓 ( 𝑥 * ) − 𝑓 (¯ 𝑥 ) ≤ 5 𝜀 = 𝑂 ( 𝜀 ) .
Optimizing the dual problem
In this section, we reduce the dual problem (1FP-Dual) to the feasibility problem of a packing LP, which we call the proxy problem.
Then, we show how we can use the PST framework [PST95] to approximately solving the proxy.
Our algorithm relies on a carefully built geometric oracle whose width parameters can be guaranteed to decrease with the optimality gap.
We perform a sequence of restarts, and after each of them we can guarantee lower oracle width.
This allows for reducing the overall complexity.
3.1 The dual 1-fair packing problem as a feasibility packing LP
Let 𝒫 def = { 𝑥 ∈ (cid:82) 𝑛 ≥ 0 : 𝐴𝑥 ≤ 1 𝑚 } be the feasible region of the primal problem (1FP).
In this section, we identify (non-negative) covering constraints of the form ⟨ ℎ, 𝑥 ⟩ ≤ 1 with vector ℎ ∈ (cid:82) 𝑛 ≥ 0 .
Recall we assume without loss of generality that 𝐴 satisﬁes (1), that is, the maximum entry of each column is 1 .
This implies that 𝑒 𝑖 ∈ 𝒫 for all 𝑖 ∈ [ 𝑛 ] , and 𝒫 ⊆ [0 , 1] 𝑛 .
For this reason, we also assume in this section and without loss of generality that 𝐴 contains the constraints { 𝑒 𝑖 } 𝑛𝑖 =1 , i.e., 𝑥 𝑖 ≤ 1 , for 𝑖 ∈ [ 𝑛 ] .
For convenience, we assume they are the ﬁrst 𝑛 rows of 𝐴 .
Let 𝜆 * ∈ ∆ 𝑚 be an optimal solution of Problem (1FP-Dual) and let ℎ 𝜆 * def = 𝐴 𝑇 𝜆 * ∈ (cid:82) 𝑛 ≥ 0 .
By strong duality of the Lagrange dual, we can reconstruct the optimal primal solution as 𝑥 * = (1 / ( 𝑛ℎ 𝜆 * 1 ) , . .
, 1 / ( 𝑛ℎ 𝜆 * 𝑛 )) .
This motivates the deﬁnition of the centroid map :
[Equation]
where ℎ, 𝑥 ∈ (cid:82) 𝑛 ≥ 0 are constraints and points, respectively.
Despite the two maps above being the same function we distinguish between 𝑐 and 𝑐 − 1 to unambiguously refer to constraints or points.
The name of the centroid map is motivated by the fact that, for any constraint ℎ ∈ (cid:82) 𝑛 ≥ 0 , the point 𝑐 ( ℎ ) is the geometric centroid of the simplex { 𝑥 ∈ (cid:82) 𝑛 ≥ 0 : ⟨ ℎ, 𝑥 ⟩ = 1 } .
Given a 𝜆 ∈ ∆ 𝑚 , we deﬁne its associated constraint ℎ 𝜆 def = 𝐴 𝑇 𝜆 .
In addition, we deﬁne the centroid 𝑝 𝜆 def = 𝑐 ( 𝐴 𝑇 𝜆 ) = 𝑐 ( ℎ 𝜆 ) .
Note that we can efﬁciently compute ℎ 𝜆 , 𝑝 𝜆 from 𝜆 , and ℎ 𝜆 from 𝑝 𝜆 and viceversa.
However, we cannot obtain the coefﬁcients 𝜆 efﬁciently from ℎ 𝜆 or 𝑝 𝜆 , as this amounts to solving a linear program.
An important property is that ℎ 𝜆 * is the unique constraint of the form ℎ 𝜆 such that 𝑐 ( ℎ 𝜆 ) ∈
𝒫 , for some 𝜆 ∈ ∆ 𝑚 .
It is the optimizer because if ℎ 𝜆 * has 𝑐 ( ℎ 𝜆 * ) ∈ 𝒫 , then ( 𝑝 𝜆 * , 𝜆 * ) satisﬁes the optimality conditions.
It is unique because of strong convexity of − log( · ) ; any other dual optimizer ¯ 𝜆 * will have 𝐴 𝑇 𝜆 * = 𝐴 𝑇 ¯ 𝜆 * .
The following lemma shows an approximate version of this property.
Lemma 3.1.
[ ↓ ] Let ℎ 𝜆 for 𝜆 ∈ ∆ 𝑚 , such that 𝐴𝑝 𝜆 ≤ (1 + 𝜀 ) 1 𝑚 .
Let 𝜆 * be the minimizer of
Problem (1FP-Dual) , of objective function 𝑔 .
Then, 𝑔 ( 𝜆 ) − 𝑔 ( 𝜆 * ) ≤
𝑛 log(1 + 𝜀 ) ≤ 𝑛𝜀 .
This lemma motivates the minimization of max 𝑖 ∈ [ 𝑚 ] ⟨ 𝐴 𝑖 , 𝑐 ( 𝐴 𝑇 𝜆 ) ⟩ for 𝜆 ∈ ∆ 𝑚 as a proxy for solving Problem (1FP-Dual).
Furthermore, if we optimize over the set { 𝑝 𝜆 = 𝑐 ( 𝐴 𝑇 𝜆 ) : 𝜆 ∈ ∆ 𝑚 } , we end up with a feasibility problem in a packing LP.
However, this set is not convex in general, which is a requirement of the PST framework we intend to use.
Fortunately, we can optimize over a larger and convex set while preserving the minimizer.
Indeed, deﬁne the sets of constraints 𝒟 def = { 𝐴 𝑇 𝜆 : 𝜆 ∈ ∆ 𝑚 } = conv( { 𝐴 1 , . . .
, 𝐴 𝑚 } ) and 𝒟 + def = { 𝐴 𝑇 𝜆 − 𝜇 ≥ (cid:48) : 𝜆 ∈ ∆ 𝑚 , 𝜇 ∈ (cid:82) 𝑛 ≥ 0 } = { ℎ ∈ (cid:82) 𝑛 ≥ 0 : ℎ ≤ ℎ 𝜆 , for ℎ 𝜆 ∈ 𝒟} .
For a constraint ℎ ∈ (cid:82) ≥ 0 , we have by polyhedral duality that ℎ ∈ 𝒟
+ if and only if ⟨ ℎ, 𝑥 ⟩ ≤ 1 for all 𝑥 ∈ 𝒫 .
In other words, 𝒟 + is exactly the set of positive constraints ⟨ ℎ, 𝑥 ⟩ ≤ 1 with ℎ ∈ (cid:82) 𝑛 ≥ 0 that are feasible for 𝒫 .
We can think about the optimization of Problem (1FP-Dual) as
[Equation]
where 1 comes from the following observation: since ℎ ∈ 𝒟 + if and only if there is ℎ 𝜆 ∈ 𝒟 with ℎ ≤ ℎ 𝜆 , and the expression − ∑︀ 𝑛𝑖 =1 log( ℎ 𝑖 ) − 𝑛 log( 𝑛 ) is strictly decreasing in every ℎ 𝑖 then no ℎ ∈ 𝒟 + ∖ 𝒟 could ever minimize the right hand side.
Consequently, the minimizer of both problems is the same.
This results in the following proxy, motivated by Lemma 3.1:
[Equation]
By Lemma 3.1, the optimizer of this problem must be 𝑝 𝜆 * .
Note ˆ 𝑔 ( 𝑝 𝜆 * ) =
so we want a 𝑝 such that ˆ 𝑔 ( 𝑝 ) ≤ 1 + 𝜀/𝑛 .
We solve this as an approximate feasibility problem in a packing LP by using the PST framework over 𝑐 ( 𝒟 + ) , which is convex, according to the following lemma.
Lemma 3.2.
[ ↓ ] Let 𝑝 (1) , . .
, 𝑝 ( 𝑘 ) be points in 𝑐 ( 𝒟 ) , and let
𝜁 ∈ ∆ 𝑘 be coefﬁcients.
Then, we have [Equation]
𝑝 (1) + · · · + 𝜁 𝑘 𝑝 ( 𝑘 ) .
Consequently, 𝑐 ( 𝒟 + ) is convex.
3.2 The PST algorithm
Plotkin, Shmoys, and Tardos designed a framework (PST) for solving LP [PST95].
This result was improved by [AHK12] for the case of packing and covering LP.
We can use this framework to solve
Problem (1FPD-Proxy) if we can provide a good feasibility oracle, as explained in the sequel.
The
PST framework focuses on checking the feasibility of 𝐴𝑝 ≤ 1 𝑚 , with 𝑝 in a convex set 𝒳 .
Then, it obtains either a certiﬁcate of infeasibility of the problem or computes a point 𝑝 ∈ 𝒳 such that 𝐴𝑝 ≤ (1 + 𝜀 ) 1 𝑚 , for some given 𝜀 > 0 .
The PST framework works by calling an oracle which solves the simpler feasibility problem of ﬁnding 𝑝 ∈ 𝒳 such that ⟨ 𝐴 𝑇 Λ , 𝑝 ⟩ ≤ 1 , (6) for some distribution Λ ∈ ∆ 𝑚 .
That is, given a single constraint ℎ Λ , written as a convex combination of the constraints deﬁned by 𝐴 , the query asks for a point 𝑝 ∈ 𝒳 that satisﬁes the constraint.
In our case, we apply the framework to 𝒳 = 𝑐
( 𝒟 + ) to ﬁnd a point 𝑝 ∈ 𝑐
( 𝒟 + ) with 𝐴𝑝 ≤ (1 + 𝜀 ) 1 𝑚 .
We can apply PST because 𝑐 ( 𝒟 + ) is convex.
Our oracle subproblems are always solvable because 𝑝 𝜆 * ∈ 𝒫 satisﬁes them all.
We use the following formulation of the PST and multiplicative weights (MW) algorithms for packing LP, which is a slight variation of [AHK12].
The MW algorithm and its guarantees are presented in Lemma C.1.
Input: Matrix 𝐴 ∈ ℳ 𝑚 × 𝑛 ( (cid:82) ≥ 0 ) normalized as in (1).
Accuracy 𝜀 ∈ (0 , ( 𝑛 − 1) 𝑛 ] .
1: ¯ 𝜆 (0) ← concat( 1 𝑛 /𝑛, (cid:48) 𝑚 − 𝑛 ) ∈ ∆ 𝑚 ; 𝜀 − 1 = 𝑛 −◇ 𝑝 ¯ 𝜆 ( 𝑡 ) is an 𝜀 𝑡 − 1 -minimizer of ˆ 𝑔 2: for 𝑡 = 0 to 𝑇 def = max { 0 , ⌈ log 2 (2 / ( 𝜀/𝑛 ))
⌉} do 3:
𝐼 𝑡 ← { 𝑖 ∈ [ 𝑚 ] : ⟨ 𝐴 𝑖 , 𝑝 ¯ 𝜆 ( 𝑡 ) ⟩ ≥ 1+ 𝜀 𝑡 − 1 1+2 𝜀 𝑡 − 1 𝑛 + √ 2 𝜀 𝑡 − 1 𝑛 } ◇ Remove redundant constraints 4: if 𝑡 is 0 then 𝜀 𝑡 ← max { 2 , 𝜀/𝑛 } else 𝜀
𝑡 ← 𝜀 𝑡 − 1 /◇
Next target accuracy 5: Λ (1) ← 1 | 𝐼 𝑡 | ◇ Restart MW 6: for 𝑘 = 1 to 𝐾 𝑡 def = 32 𝜏 𝜀 𝑡 − 1 𝜎 𝜀 𝑡 − 1 log( | 𝐼 𝑡 | ) /𝜀 2 𝑡 = ̃︀ 𝑂 ( 𝑛/𝜀 𝑡 ) do 7: 𝜆 ( 𝑘 ) , 𝑝 𝜆 ( 𝑘 ) ← O ( query Λ ( 𝑘 ) /
‖ Λ ( 𝑘 ) ‖ 1 , current solution ¯ 𝜆 ( 𝑡 ) , index set 𝐼 𝑡 ) ∈ (∆ 𝑚 , 𝑐 ( 𝒟 )) 8: ℓ ( 𝑘 ) ← 1 | 𝐼 𝑡 | − 𝐴 𝐼 𝑡 𝑝 𝜆 ( 𝑘 ) 9: Λ ( 𝑘 +1) ← Λ ( 𝑘 ) ⊙ ( 1 | 𝐼 𝑡 | − ( 𝜀 𝑡 / (4 𝜏 𝜀 𝑡 − 1 𝜎 𝜀 𝑡 − 1 )) · ℓ ( 𝑘 ) )
◇ MW step 10: end for 11: ¯ 𝜆 ( 𝑡 +1) ← 1 𝐾 𝑡 ∑︀ 𝐾 𝑡 𝑘 =1 𝜆 ( 𝑘 ) 12: end for 13: return ¯ 𝜆 def = ¯ 𝜆 ( 𝑇 +1)
Output: 𝐴𝑝 ¯ 𝜆 ≤ (1 + 𝜀/𝑛 ) 1 𝑚 , that is, ˆ 𝑔 ( 𝑝 ¯ 𝜆 ) ≤ 1 + 𝜀/𝑛 .
Hence 𝑔 (¯ 𝜆 ) − 𝑔 ( 𝜆 * ) ≤ 𝜀 .
Lemma 3.3 (PST guarantee).
[ ↓ ] Let 𝜎, 𝜏 ∈ (cid:82) > 0 .
For a target accuracy 𝜀 ∈ (0 , 4 min { 𝜎, 𝜏 } ] , run the MW algorithm of Lemma C.1 with 𝛿 = 𝜀/ 2 , losses ℓ ( 𝑘 ) def = 1 𝑚 − 𝐴𝑝
( 𝑘 ) assumed to be in [ − 𝜎, 𝜏 ] 𝑚 , where 𝑝 ( 𝑘 ) is the point the oracle outputs at iteration 𝑘 when the constraint that is queried is given by 𝐴 𝑇 (Λ ( 𝑘 ) /
‖ Λ ( 𝑘 ) ‖ 1 ) , and where Λ ( 𝑘 ) are the weights computed by the MW algorithm.
Then, after 𝐾 = 32 𝜎𝜏 log( 𝑚 ) 𝜀iterations, we obtain a solution ¯ 𝑝 def = 1 𝐾 ∑︀ 𝐾𝑘 =1 𝑝 ( 𝑘 ) that satisﬁes ˆ 𝑔 (¯ 𝑝 ) ≤ 1 + 𝜀 .
In order to give a solution of Problem (1FP-Dual), we are also interested in recovering some 𝜆 ∈ ∆ 𝑚 such that ˆ 𝑔 ( 𝑝 𝜆 ) is small.
Assume the oracle returns a point 𝑝 ( 𝑘 ) =
𝑝 𝜆 ( 𝑘 ) for some 𝜆 ( 𝑘 ) it can also provide.
Then, even if ¯ 𝑝 ∈ 𝒟 + ∖ 𝒟 , we have by Lemma 3.2 that ¯ 𝜆 def = 1 𝐾 ∑︀ 𝐾𝑘 =1 𝜆 ( 𝑘 ) deﬁnes a point 𝑝 ¯ 𝜆 ∈ 𝒟 such that 𝑝 ¯ 𝜆 ≤ ¯ 𝑝 .
Hence ˆ 𝑔 ( 𝑝 ¯ 𝜆 ) ≤ ˆ 𝑔 (¯ 𝑝 ) ≤ 1 + 𝜀 , so ¯ 𝜆 can be used as our dual solution.
Algorithm O
Input: An approximate solution 𝑠 def = 𝐴 𝑇 𝜆
( 𝑠 ) , 𝜆 ( 𝑠 ) ∈
∆ 𝑚 , ˆ 𝑔 ( 𝑐 ( 𝑠 )) ≤ 1 + 𝛿 .
Query constraint 𝑞 =
𝐴 𝑇 𝜆 ( 𝑞 ) , 𝜆 ( 𝑞 ) ∈ ∆ 𝑚 .
Precision parameter 1 < 𝜔 ≤ 2 , default value 𝜔 = 2 .
Index set of non-redundant constraints 𝐼 = { 𝑖 ∈ [ 𝑚 ] : ⟨ 𝐴 𝑖 , 𝑐 ( 𝑠 ) ⟩ ≥ 1+ 𝛿 1+ 𝜔𝛿𝑛 + √ 𝜔𝛿𝑛 } .
Output: 𝜆 ( 𝑜 ) ∈ ∆ 𝑚 and point 𝑜 def = 𝑐 ( 𝐴 𝑇 𝜆 ( 𝑜 ) ) ∈ 𝑐
( 𝒟 )
such that
𝑜 is covered by 𝑞 ,
i
e., ⟨ 𝑞, 𝑜 ⟩ ≤ 1 . 2.
If 𝑖 ∈ 𝐼 then ⟨ 𝐴 𝑖 , 𝑜 ⟩ ∈ [1 − 𝜏, 1+ 𝜎 ] where 𝜎 = min( √ 𝜔𝛿𝑛 + 𝜔𝛿𝑛, 1+ 𝜔𝛿 1+ 𝛿 max 𝑖 ∈ [ 𝑛 ] 𝑠 − 1 𝑖 − 1) , 𝜏 = min(3 √ 𝜔𝛿𝑛, 1) .
It satisﬁes all redundant constraints, i e., ⟨ 𝐴 𝑖 , 𝑜 ⟩ ≤ 1 , if 𝑖 ∈ [ 𝑚 ] ∖ 𝐼 .
Thus our task is to construct an oracle with good enough 𝜎 and 𝜏 , which are called the width parameters of the oracle.
We also need to make sure that 𝜀 ≤ 4 min { 𝜎, 𝜏 } , and that we can output a point 𝑝 𝜆 in 𝒟 , and a corresponding 𝜆 .
Regardless, this algorithm runs in ̃︀ 𝑂 ( 𝜎𝜏/𝜀 2 ) iterations, which is slower than what we aim for, for constant 𝜎 and 𝜏 .
Our approach to obtain a fast algorithm is to design an adaptive feasibility oracle.
We provide our best solution ¯ 𝜆 ( 𝑡 ) to the oracle, which satisﬁes ˆ 𝑔 ( 𝑝 ¯ 𝜆 ( 𝑡 ) ) ≤ 1 + 𝛿 , for some 𝛿 > 0 .
Given such a 𝛿 -approximate solution we identify a set of indices 𝐼 𝑡 ⊆
[ 𝑛 ] such that the constraints { 𝐴 𝑖 , 𝑖 ̸∈ 𝐼 𝑡 } are redundant, in the sense that the oracle is guaranteed to return points satisfying them even if they are removed from the problem.
We remove these constraints since they would yield large values of 𝜏 .
Hence, we can run our MW algorithm with the remaining | 𝐼 𝑡 | constraints.
Denote 𝐴
𝐼 𝑡 the matrix that has { 𝐴 𝑖 : 𝑖 ∈ 𝐼 𝑡 } as rows, in increasing order of 𝑖 .
In this case if Λ ∈ ∆ | 𝐼 𝑡 | , we denote ℎ Λ =
𝐴 𝑇𝐼 𝑡 Λ and 𝑝 Λ = 𝑐 ( 𝐴 𝑇𝐼 𝑡 Λ) accordingly.
Our oracle, when given a query ℎ Λ , uses both constraints ℎ Λ and ℎ ¯ 𝜆 ( 𝑡 ) to return a point 𝑝 satisfying the oracle condition (6) and such that the loss 1 | 𝐼 𝑡 | − 𝐴 𝐼 𝑡 𝑝 is in [ − 𝜎 𝛿 , 𝜏 𝛿 ] | 𝐼 𝑡 | , with
[Equation]
In fact, 𝜎 𝛿 can be deﬁned as the minimum of its two expressions above, regardless of the value of 𝛿 .
But we shall use this deﬁnition for our algorithm.
In the next section, we present the construction and analysis of such an oracle and the set 𝐼 𝑡 .
We observe that, because the parameters depend on 𝛿 , we can restart the MW algorithm, and run it in several stages, indexed by 𝑡 = 0 , . . .
, 𝑇 .
This allows for incrementally reducing the width parameters and obtaining a better overall complexity, as we prove in the following theorem.
Theorem 3.4.
[ ↓ ] Let 𝜀 ∈ (0 , 𝑛 ( 𝑛 − 1)] .
Suppose we have a feasibility oracle O satisfying (6) and (7) , when we ﬁlter constraints according to Algorithm 2.
Then, Algorithm 2 computes, in ̃︀ 𝑂 ( 𝑛 2 /𝜀 ) iterations, a point ¯ 𝜆 which is an 𝜀 -minimizer of 𝑔 .
Moreover, 𝑝 ¯ 𝜆 is an ( 𝜀/𝑛 ) -minimizer of ˆ 𝑔 .
3.3 The PST oracle and the redundant constraints
In order to give a fast solution for Problem (1FPD-Proxy), we want to design an adaptive oracle.
That is, it should satisfy (6), and it should output better points if it already knows a good approximate solution.
Generically, assume that the oracle has access to a feasible constraint 𝑠 def =
𝐴 𝑇 𝜆 ( 𝑠 ) ∈ 𝒟 , with 𝜆 ( 𝑠 ) ∈ ∆ 𝑚 satisfying 𝑝 𝜆 ( 𝑠 ) =
𝑐 ( 𝑠 ) is a point with ˆ 𝑔 ( 𝑝 𝜆 ( 𝑠 ) ) ≤ 1 + 𝛿 .
In other words, 𝑐 ( 𝑠 ) is a 𝛿 -approximate solution to Problem (1FPD-Proxy).
Let 𝑞 def = 𝐴 𝑇 𝜆 ( 𝑞 ) ∈ 𝒟 , with 𝜆 ( 𝑞 ) ∈ ∆ 𝑚 be a query constraint.
In Algorithm 2, 𝜆 ( 𝑠 ) and 𝜆 ( 𝑞 ) correspond to ¯ 𝜆 ( 𝑡 ) and Λ ( 𝑘 ) / ‖ Λ ( 𝑘 ) ‖ 1 respectively, where the latter is interpreted as having coefﬁcients equal to zero for constraints 𝑖 ̸∈ 𝐼 𝑡 .
The main geometric idea is that by using the solution 𝑠 , we can deﬁne a region whose size depends on 𝛿 containing the optimum 𝑝 𝜆 * of Problem (1FPD-Proxy).
We will guarantee the oracle returns points in this region, which in turn means that the oracle returns points close to the optimum.
We call this geometric object the lens of a point :
[Equation]
where 𝜔 ∈ (1 , 2] is a parameter; we chose 𝜔 = 2 in the algorithm.
We depict the lens in Figurein Appendix C.
We apply this deﬁnition to the point 𝑣 def = 𝑐 ( 𝑠 ) / (1 + 𝛿 ) , which is in 𝒫 because ˆ 𝑔 ( 𝑠 ) ≤ 1 + 𝛿 implies that 𝑐 ( 𝑠 ) / (1 + 𝛿 ) ∈ 𝒫 .
We show in Lemma C.2 that, by using the bisection method, we can efﬁciently ﬁnd a convex combination (1 − 𝜇 ) 𝑠
+ 𝜇𝑞 that will result in a constraint, whose centroid is the point 𝑜 we output.
It satisﬁes the ﬁrst oracle condition, and is in the lens.
Also, we can recover 𝜆 ( 𝑜 ) as (1 − 𝜇 ) 𝜆 ( 𝑠 ) + 𝜇𝜆 ( 𝑞 ) .
If the oracle can output a point 𝑜 in the lens, we will have low width parameters for the constraints 𝐴 𝑖 that do not cover the lens, as we show in Proposition C.4.
That is, for one such constraint 𝐴 𝑖 , the corresponding loss is 1 − ⟨ 𝐴 𝑖 , 𝑜 ⟩ ∈ [ − 𝜎, 𝜏 ] .
The other constraints could be problematic in terms of width parameters because ⟨ 𝐴 𝑖 , 𝑜 ⟩ could be much smaller than 1 , forcing 𝜏 to be large.
However, we do not need to optimize over these constraints because ˆ 𝑔 ≥ 1 , so these constraints do not contribute to the max in its deﬁnition.
This leads to the set of non-redundant constraints 𝐼 , which for efﬁciency reasons, we relax to those indices of constraints 𝐴 𝑖 that do not cover a box that contains the lens, cf.
Lemma C.3.
We prove this is good enough in terms of the width parameters.
The computation of 𝐼 , which is done before each restart phase, requires 𝑂 ( 𝑁 ) operations and each query to the oracle takes 𝑂 ( 𝑛 log( 𝑛 ( 𝜔 − 1) 𝛿 + 𝑛𝜔 − 1 )) operations, cf.
Lemma C.2.
Acknowledgments and Disclosure of Funding
We thank Elias Koutsoupias for some suggestions and discussions that started the line of research of this work.
David Martínez-Rubio and Francisco Criado were partially funded by the DFG Cluster of
Excellence MATH+ (EXC-2046/1, project id 390685689) funded by the Deutsche Forschungsgemeinschaft (DFG).
David Martínez-Rubio was also partially supported by EP/N509711/1 from the
EPSRC MPLS division, grant No 2053152.
For all
authors..
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] See the introduction, where we talk about other 𝛼 -fair problems.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[N/A] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[N/A] (b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A] (d) Did you discuss whether
and how consent was obtained from people whose data you’re using/curating?
[N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[N/A] 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]