SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Filippo Vannella
KTH and Ericsson Research
vannella@kth.se
Alexandre Protiuere
Jaeseong Jeong
Ericsson Research
jaeseong.jeong@ericsson.com
We study the problem of regret minimization in Multi-Agent Multi-Armed Bandits (MAMABs) where the rewards are defined through a factor graph. We derive an instance-specific regret lower bound and characterize the minimal expected number of times each global action should be explored. This bound and the corresponding optimal exploration process are obtained by solving a combinatorial optimization problem whose set of variables and constraints exponentially grow with the number of agents, and cannot be exploited in the design of efficient algorithms. Inspired by Mean Field approximation techniques used in graphical models, we provide simple upper bounds of the regret lower bound. The corresponding optimization problems have a reduced number of variables and constraints. By tuning the latter, we may explore the trade-off between the achievable regret and the complexity of computing the corresponding exploration process. We devise Efficient Sampling for MAMAB (ESM), an algorithm whose regret asymptotically matches the approximated lower bounds. The regret and computational complexity of ESM are assessed numerically, using both synthetic and real-world experiments in radio communications networks.
The stochastic Multi-Agent Multi-Armed Bandits (MAMABs) [35, 2, 3] is a combinatorial sequential decision-making problem that generalizes the classical stochastic MAB problem by assuming that  $(i)$  a global action is defined by actions individually selected by a set of agents, and  $(ii)$  the reward function is defined through a factor graph, which defines inter-dependencies between agents. This reward structure arises naturally in applications where agents interact in a graph with the need to coordinate towards a common goal. MAMABs can model a wide range of real-world problems, from wind farm control [2, 37] to radio communication networks parameters optimization (see Fig. 1).
Despite the wide spectrum of their potential applications, MAMABs are extremely hard to solve, even when the reward function is known. The main challenge stems from the combinatorial structure of the action set (there are  $K^N$  possible global actions, where  $N$  is the number of agents and  $K$  is the number of actions per agent). This issue is exacerbated in the learning setting where the reward function has to be inferred. In this work, we study the regret minimization problem in MAMABs, and more specifically, the trade-off between statistical efficiency (the learner aims at achieving low regret), and computational efficiency (she will typically have to solve combinatorial optimization problems over the set of possible global actions while learning).
Contributions. We present statistically and computationally efficient algorithms for MAMABs. Our algorithms enjoy (in the worst case) regret guarantees scaling as  $\rho K^d\log (T)$ , where  $K$  is the number of actions per agent,  $\rho$  and  $d$  are the number of factors and the maximal degree of the graph defining the reward function. This scaling illustrates the gains one may achieve by exploiting the factor graph structure: without leveraging it, the regret would scale as  $K^{N}\log (T)$ . Our algorithms have controllable computational complexity and can be applied in large-scale MAMABs. More precisely, our contributions are as follows.
1) Regret lower bound. We derive a regret lower bound satisfied by any algorithm. The bound is defined through a convex program (the lower bound problem), whose solution provides an optimal exploration strategy. Unfortunately, because of the factored reward structure, this optimization problem contains an exponential number of variables and constraints, and is hard to use in practice.
2) Approximations of the lower bound problem. We devise approximations of the lower bound problem by combining variable and constraint reduction techniques inspired by methods in the probabilistic graphical model literature [39, 20]. To reduce the number of variables, we propose  $(i)$  locally tree-like approximation, a tight relaxation for MAMAB instances described by acyclic factor graphs, and  $(ii)$  Mean Field (MF) approximation for general graphs. The MF approximation yields an upper bound of the regret lower bound, scaling as  $\rho K^d\log (T)$  (where  $T$  is the time horizon).
Both approximations yield lower bound problems with a polynomial number of variables and exponential number of constraints (in  $N$ ). To reduce the number of constraints, we propose a technique that leverages an ordering of the  $m$  smallest gaps and a Factored Constraint Reduction (FCR) method to represent the exponentially many constraints in a compact manner. The corresponding optimization problems have a reduced number of variables and constraints. By tuning the latter, we may explore the trade-off between the achievable regret and the complexity of computing the corresponding exploration process.
3) The ESM algorithm. Based on this approximation, we devise Efficient Sampling for MAMABs (ESM), an algorithm whose regret provably matches our approximated regret lower bound. The algorithm trades off statistical and computational complexity by performing exploration as prescribed by the solution of the approximated lower bound problem. We test the performance of ESM numerically on both synthetic experiments and learn to coordinate the antenna tilts in a radio communication network. In both sets of experiments, ESM can solve problems with a large number of global actions in a statistical and computationally efficient manner.
Our work belongs to the framework of structured regret minimization in MABs, which encompasses a large variety of reward structures such as linear [21], unimodal [9], Lipschitz [11], etc. For general structured bandits, [8] propose Optimal Sampling for Structured Bandits (OSSB), a statistically optimal algorithm, i.e., matching the regret lower bound. The algorithm is computationally inefficient when applied to the MAMABs combinatorial structure. Our algorithm is inspired by OSSB, but relies on approximated lower bound problems to trade-off statistical and computational complexity.
A few studies investigate MAMABs with the same factored reward structure as ours [35, 2, 37]. These works focus on devising algorithms with regret guarantees using methods based on, e.g., Upper Confidence Bound (UCB) [35, 2] or Thompson Sampling (TS) [37]. For example, Stranders et al. [35] propose HEIST, an UCB-type algorithm whose asymptotic regret scales as  $O(K^{N}\Delta_{\mathrm{max}} / \Delta_{\mathrm{min}}\log (T))$ , where  $\Delta_{\mathrm{min}}$  and  $\Delta_{\mathrm{max}}$  are the minimal and maximal gaps, respectively. The MAUCE algorithm from Bargiacchi et al., [2] improves over [35] yielding asymptotic regret  $O(\rho^2 K^d\Delta_{\mathrm{max}}^2 /\Delta_{\mathrm{min}}^2\log (T))$ . Our worst approximation improves of a factor  $\Delta_{\mathrm{max}}$  w.r.t. this bound, a quantity that typically scales with  $\rho K^d$  (see App. M).
There is a large body of work [24, 10, 12, 13, 38] investigating regret minimization in the (linear) combinatorial semi-bandit feedback setting. Although our model can be interpreted as a particular instance of this setting (see App. E for details), the MAMAB combinatorial structure has never been explicitly considered in this context. The closest related work is [12], in which the authors study a regret lower bound problem with an exponentially large number of variables and constraints. They leverage [12, Assumption 6] to compactly represent the lower bound optimization problem and propose a gradient-based procedure to solve it in polynomial time. Unfortunately, for MAMABs, the
above-mentioned assumption only holds for rewards described by acyclic factor graphs (see App. M). We propose computationally efficient approximations valid for any factor graph while retaining statistical tightness in the case of acyclic factor graphs.
We consider the generic MAMAB model with factored structure introduced in [2]. The model is defined by the tuple  $\langle S, \mathcal{A}, r \rangle$ , where:
1.  $\mathcal{S} = [N] \triangleq \{1, \dots, N\}$  is a set of  $N$  agents;
2.  $\mathcal{A} = \times_{i\in [N]}\mathcal{A}_i$  is a set of global actions, which is the Cartesian product over  $i$  of the set  $\mathcal{A}_i$  of actions available to the agent  $i$ . We assume w.l.o.g. that  $|\mathcal{A}_i| = K$ , for all  $i\in [N]$ , and define  $A\triangleq |\mathcal{A}| = K^N$ ;
3.  $r$  is the reward function mapping the global action to the collected reward.
Rewards and their factor-graph representation. We model the collected rewards. There are  $\rho$  possibly overlapping groups of agents  $(S_e)_{e\in [\rho ]}$ , with  $S_{e}\subseteq S$  and  $|\mathcal{S}_e| = N_e$ . The local reward generated by group  $e$  depends on group actions  $a_{e}\triangleq (a_{i})_{i\in S_{e}}\in \mathcal{A}_{e}\triangleq \times_{i\in S_{e}}\mathcal{A}_{i}$  only. More precisely, each time  $a_{e}$  is selected, the collected local rewards are i.i.d. copies of a random variable  $r_e(a_e)\sim \mathcal{N}(\theta_e(a_e),1 / 2)$ . Rewards collected in various groups are independent. The global reward for action  $a$  is then  $r(a) = \sum_{e\in [\rho ]}r_e(a_e)$ , a random variable with expectation  $\theta (a) = \sum_{e\in [\rho ]}\theta_{e}(a_{e})$ . The number of possible group actions in group  $e$  is  $A_{e}\triangleq |\mathcal{A}_{e}| = K^{N_{e}}$ , and we define  $\tilde{A}\triangleq \sum_{e\in [\rho ]}A_{e}$ . The reward function can be represented using a factor graph [39]. Factor graphs are bipartite graphs with two types of node:  $N$  action nodes, one for each agent, and  $\rho$  factor nodes, one for each group. An edge between a factor  $r_e$  and an agent  $i$  exists if the action  $a_i$  selected by the agent  $i$  is an input of  $r_e$ :  $i\in S_e$ . Fig. 1 shows an example of a factor graph modeling interference in a radio communication network.
Figure 1: Factor graph in a radio communication network. An agent (represented by a circle) corresponds to a base-station (BS) whose transmissions cover a cell. The possible actions at a BS may correspond to different transmission power levels and antenna tilts (the physical angle of the antennas). The local rewards correspond to the throughput (in bit/s) achieved in a given cell, and hence each cell is associated with a factor (represented by a square). The throughput in a given cell depends on the action of the corresponding BS but also on those of neighboring BSs through interference. In the factor graph, each BS or agent has hence an edge to factors or cells it interferes.
Sequential decision process. The decision maker sequentially selects global actions based on the history of previous observations and receives a set of samples of the local rewards associated to the various groups. Specifically, in each round  $t \geq 1$ , the decision maker selects a global action  $a_{t} = (a_{t,1},\ldots ,a_{t,N})$  and observes the local rewards  $r_t = (r_{t,1},\dots,r_{t,\rho})$  from each group. The global action  $a_{t + 1}$  is selected based on the history of observations  $\mathcal{H}_t = (a_s,r_s)_{s\in [t]}$ . This type of interaction is known as semi-bandit feedback.
Regret minimization. The goal is to devise an algorithm  $\pi = (a_{t})_{t\geq 1}$ , i.e., a sequence of global actions  $a_{t}\in \mathcal{A}$  selected in each round  $t\geq 1$ , that minimizes the regret up to time  $T\geq 1$ , defined as
$$
R ^ {\pi} (T) = \mathbb {E} \left[ \sum_ {t = 1} ^ {T} \theta \left(a _ {\theta} ^ {\star}\right) - \theta \left(a _ {t}\right) \right],
$$
where  $a_{\theta}^{\star}\in \arg \max_{a\in \mathcal{A}}\theta (a)$  denotes the best global action. Throughout the paper, we assume that  $a_{\theta}^{\star}$  is unique and we use  $a^\star$  and  $a_{\theta}^{\star}$  interchangeably. We define the gap of a sub-optimal global action  $a$  by  $\Delta (a) = \theta (a^{\star}) - \theta (a)$
4 Regret Lower Bound
To derive instance-specific regret lower bounds, we restrict our attention to the class of uniformly good algorithms: An algorithm  $\pi$  is uniformly good if for any  $\theta, \forall \alpha > 0$ , we have that  $R^{\pi}(T) = o(T^{\alpha})$ .
Theorem 4.1. The regret of any uniformly good algorithm satisfies for any  $\theta$ ,  $\lim_{T\to \infty}\inf_{\log (T)}\frac{R^{\pi}(T)}{\log(T)}\geq C_{\theta}^{\star}$  where  $C_\theta^\star$  is the value of the following convex optimization problem
$$
\min  _ {v \in \mathbb {R} _ {\geq 0} ^ {A}} \sum_ {a \in \mathcal {A}} v _ {a} \Delta (a) \quad \text {s . t .} \quad \sum_ {e \in [ \rho ]: a _ {e} \neq a _ {e} ^ {\star}} \left(\sum_ {b \in \mathcal {A} \backslash \{a _ {\theta} ^ {\star} \}: b _ {e} = a _ {e}} v _ {b}\right) ^ {- 1} \leq \Delta (a) ^ {2}, \quad \forall a \in \mathcal {A}. \tag {1}
$$
The proof of this result leverages classical change-of-measure arguments [25] (see App. A.1 for details). If  $v^{\star}$  denotes the solution of the lower bound optimization problem, then for  $a \neq a_{\theta}^{\star}$ ,  $v_{a}^{\star}\log (T)$  can be interpreted as the asymptotic expected number of times the sub-optimal action  $a$  is explored under a uniformly good algorithm minimizing regret. We conclude this section by reformulating (1) using group variables  $\tilde{v}$ . Introduce the marginal cone:
$$
\tilde {\mathcal {V}} = \left\{\tilde {v} \in \mathbb {R} _ {\geq 0} ^ {\tilde {A}}: \exists v \in \mathbb {R} _ {\geq 0} ^ {A}, \forall e \in [ \rho ], a _ {e} \in \mathcal {A} _ {e}, \tilde {v} _ {e, a _ {e}} = \sum_ {b \in \mathcal {A} \backslash \{a _ {\theta} ^ {\star} \}: b _ {e} = a _ {e}} v _ {b}, \right\}.
$$
The set  $\tilde{\mathcal{V}}$  contains group variables  $\tilde{v} = (\tilde{v}_e)_{e\in [\rho ]}$  where  $\tilde{v}_{e} = (\tilde{v}_{e,a_{e}})_{a_{e}\in \mathcal{A}_{e}}$
Lemma 4.2. For any  $\theta$ ,  $C_{\theta}^*$  is the value of the following convex optimization problem
$$
\min  _ {\tilde {v} \in \tilde {\mathcal {V}}} \sum_ {e \in [ \rho ], a _ {e} \in \mathcal {A} _ {e}} \tilde {v} _ {e, a _ {e}} \left(\theta_ {e} \left(a _ {e} ^ {\star}\right) - \theta_ {e} \left(a _ {e}\right)\right) \quad \text {s . t .} \quad \sum_ {e \in [ \rho ]: a _ {e} \neq a _ {e} ^ {\star}} \tilde {v} _ {e, a _ {e}} ^ {- 1} \leq \Delta (a) ^ {2}, \quad \forall a \in \mathcal {A}. \tag {2}
$$
Again, if the solution of (2) is  $\tilde{v}^{\star}$ , then for any  $e\in [\rho ]$  and  $a_{e}\in \mathcal{A}_{e}$ ,  $\tilde{v}_{e,a_e}^{\star}\log (T)$  can be interpreted as the asymptotic expected number of times the group action  $a_{e}$  is selected under an optimal algorithm when it explores, i.e., when the global action  $a\neq a_{\theta}^{\star}$ .
5 Lower Bound Approximations
As suggested above, if we are able to solve (1) and hence obtain  $v^{\star}$ , the latter specifies the optimal exploration process. From there, we could devise an algorithm with minimal regret [8]. Unfortunately, solving (1) is an extremely hard task, even for relatively small problems. Indeed, the problem has  $K^{N}$  variables and  $K^{N}$  constraints, and using general-purpose solvers, e.g., based on the interior-point method, would require  $\mathrm{poly}(K^{N})\log (1 / \varepsilon)$  floating-point operations [33]. To circumvent this difficulty, we present approximations of the lower bound problem with a reduced number of variables and constraints. We will then leverage these approximations to design efficient algorithms.
5.1 Variable reduction
To reduce the number of variables, we apply approximation techniques inspired by methods in the probabilistic graphical model literature [39]. In Sec. 5.1.1, we first propose a locally tree-like reduction, yielding an optimization problem whose value  $C_{\theta}^{\mathrm{L}}$  exactly matches the true lower bound  $C_{\theta}^{\star}$  for MAMABs with acyclic factor graphs (see App. J for a formal definition and examples). For graphs containing cycles however, we have  $C_{\theta}^{\mathrm{L}} < C_{\theta}^{\star}$ , and hence for those graphs, it is impossible to devise an algorithm based on this reduction (such an algorithm would lead to a regret  $C_{\theta}^{\mathrm{L}}\log (T)$ , which contradicts the lower bound).
Instead, for general graphs, we propose in Sec. 5.1.2 the  $\psi$ -mean-field reduction, an approximation based on a local decomposition inspired by Mean Field (MF) methods [39]. The  $\psi$ -mean field reduction leads to an optimization problem whose value  $C_{\theta}^{\mathrm{MF}}$  provably upper bounds  $C_{\theta}^{\star}$ , and hence that can be used to devise an algorithm with regret approaching  $C_{\theta}^{\mathrm{MF}}\log (T)$ .
5.1.1 Locally tree-like reduction
This reduction imposes local consistency constraints between group variables  $\tilde{v}$  and local variables local variables  $w = (w_{i})_{i\in [N]}$ , where  $w_{i} = (w_{i,a_{i}})_{a_{i}\in \mathcal{A}_{i}}\in \mathbb{R}_{\geq 0}^{K}$ . Define the local cone as:
$$
\tilde {\mathcal {V}} _ {\mathrm {L}} = \left\{\tilde {v} \in \mathbb {R} ^ {\tilde {A}}: \exists w \in \mathbb {R} _ {\geq 0} ^ {K N}: \forall e \in [ \rho ], \forall i \in \mathcal {S} _ {e}, \forall a _ {i} \in \mathcal {A} _ {i}, w _ {i, a _ {i}} = \sum_ {b _ {e} \in \mathcal {A} _ {e}: b _ {e} \sim a _ {i}} \tilde {v} _ {b _ {e}} \right\},
$$
where the notation  $a_{e}\sim a_{i}$  means that the  $i^{\mathrm{th}}$  element of  $a_{e}$  equals  $a_{i}$ . The locally tree-like approximation, presented in the next lemma, is obtained by replacing  $\tilde{\mathcal{V}}$  by  $\tilde{\mathcal{V}}_{\mathrm{L}}$  in (2).
Lemma 5.1. For any  $\theta$  with rewards described by an acyclic factor graph, we have that  $C_{\theta}^{\star} = C_{\theta}^{\mathrm{L}}$ , where  $C_{\theta}^{\mathrm{L}}$  is the value of the following convex optimization problem:
$$
\min  _ {\tilde {v} \in \tilde {V} _ {L}} \sum_ {e \in [ \rho ], a _ {e} \in \mathcal {A} _ {e}} \tilde {v} _ {e, a _ {e}} \left(\theta_ {e} \left(a _ {e} ^ {\star}\right) - \theta_ {e} \left(a _ {e}\right)\right) \quad \text {s . t .} \quad \sum_ {e \in [ \rho ]: a _ {e} \neq a _ {e} ^ {\star}} \tilde {v} _ {e, a _ {e}} ^ {- 1} \leq \Delta (a) ^ {2}, \quad \forall a \in \mathcal {A}. \tag {3}
$$
The proof is presented in App. A.2. This approximation reduces the number of variables from  $K^{N}$  to  $\tilde{A} + KN$ . The lemma states that, for acyclic factor graphs, the locally tree-like approximation (3) is tight, i.e.,  $C_{\theta}^{\mathrm{L}} = C_{\theta}^{\star}$ . Unfortunately, for general graphs, we have that  $C_{\theta}^{\mathrm{L}} < C_{\theta}^{\star}$  (a direct consequence of [39, Prop. 4.1]), and hence it is impossible to devise algorithms based on this approximation.
5.1.2  $\psi$ -Mean-Field reduction
Our  $\psi$ -MF reduction is loosely inspired by MF approximation methods in graphical models [39]. It consists in decomposing global variables  $v$  as a function  $\psi$  of the local variables  $w = (w_{i})_{i\in [N]}$ . Specifically, the  $\psi$ -MF reduction introduces the following set of constraints:  $v_{a} = \psi_{a}(w), \forall a \neq a_{\theta}^{\star}$  where  $\psi_{a}: \mathbb{R}_{\geq 0}^{KN} \to \mathbb{R}_{\geq 0}$ . Let  $\mathcal{V}_{\psi} = \{v \in \mathbb{R}_{\geq 0}^{A}: \exists w \in \mathbb{R}_{\geq 0}^{KN}, v_{a} = \psi_{a}(w), \forall a \neq a_{\theta}^{\star}\}$ , and define the  $\psi$ -MF marginal cone as
$$
\tilde {\mathcal {V}} _ {\psi \text {- M F}} = \left\{\tilde {v} \in \mathbb {R} _ {\geq 0} ^ {\tilde {A}}: \exists v \in \mathcal {V} _ {\psi}, \forall e \in [ \rho ], a _ {e} \in \mathcal {A} _ {e}, \tilde {v} _ {e, a _ {e}} = \sum_ {b \in \mathcal {A} \backslash \{a _ {\theta} ^ {\star} \}: b _ {e} = a _ {e}} v _ {b}, \right\}.
$$
We get the  $\psi$ -MF approximation,  $C_{\theta}^{\psi -\mathrm{MF}}$  by replacing  $\tilde{\mathcal{V}}$  by  $\tilde{\mathcal{V}}_{\psi -\mathrm{MF}}$  in (2).
Lemma 5.2. For any  $\theta, \psi$ , we have that  $C_{\theta}^{\star} \leq C_{\theta}^{\psi - MF}$ , where  $C_{\theta}^{\psi - MF}$  is the value of the optimization problem:
$$
\min  _ {\tilde {v} \in \tilde {\mathcal {V}} _ {\psi - M F}} \sum_ {e \in [ \rho ], a _ {e} \in \mathcal {A} _ {e}} \tilde {v} _ {e, a _ {e}} \left(\theta_ {e} \left(a _ {e} ^ {\star}\right) - \theta_ {e} \left(a _ {e}\right)\right) \quad \text {s . t .} \quad \sum_ {e \in [ \rho ]: a _ {e} \neq a _ {e} ^ {\star}} \tilde {v} _ {e, a _ {e}} ^ {- 1} \leq \Delta (a) ^ {2}, \quad \forall a \in \mathcal {A}. \tag {4}
$$
Clearly, the tractability of the problem  $C_{\theta}^{\psi \cdot \mathrm{MF}}$  depends on the choice of  $\psi$ . A natural choice would be  $\psi_{a}(w) = \prod_{i\in [N]}w_{i,a_{i}}$ , as proposed, e.g., in approximate inference methods [39]. However, this choice leads to a non-convex program (see App. B). The following lemma proposes a choice of  $\psi$  which leads to a convex program over local variables  $w$  only.
Lemma 5.3. Let  $\psi_{a}(w) = \sum_{i\in [N]}w_{i,a_{i}},\forall a\neq a^{\star}$ . Then  $C_\theta^{\psi -MF}$  is the value of the following convex optimization problem:
$$
\min  _ {w \in \mathbb {R} _ {\geq 0} ^ {K N}} \sum_ {e \in [ \rho ], a _ {e} \in \mathcal {A} _ {e}} f _ {e, a _ {e}} (w) \left(\theta \left(a _ {e} ^ {\star}\right) - \theta_ {e} \left(a _ {e}\right)\right) \quad \text {s . t .} \sum_ {e \in [ \rho ]: a _ {e} \neq a _ {e} ^ {\star}} f _ {e, a _ {e}} (w) ^ {- 1} \leq \Delta (a) ^ {2}, \forall a \in \mathcal {A}, \tag {5}
$$
where  $f_{e,a_e}(w) = K^{N - |\mathcal{S}_e|}\sum_{i\in \mathcal{S}_e,a_i\in \mathcal{A}_i}w_{i,a_i} + K^{N - |\mathcal{S}_e| - 1}\sum_{i\notin \mathcal{S}_e}w_{i,a_i}$ . Furthermore, it holds that  $C_\theta^{\psi \cdot MF}\leq \rho \Delta_{\min}^{-2}\sum_{e\in [\rho ],a_e\in \mathcal{A}_e}(\theta (a_e^\star) - \theta_e(a_e))$ , where  $\Delta_{\mathrm{min}} = \min_{a\neq a^{\star}}\Delta (a)$ .
The proof is presented in App. A.3. The lemma provides a worst-case scaling of  $C_{\theta}^{\psi - \mathrm{MF}}$ : it scales at most as  $\tilde{A} = \sum_{e \in [\rho]} K^{|\mathcal{S}_e|}$  (remember that if we were considering a MAMAB as a standard bandit problem, the latter would have  $K^N$  arm and hence a regret scaling exponentially in  $N$ ). The number of variables involved in (5) is  $KN$ . The quantities  $(f_{e,a_e}(w))_{e \in [\rho], a_e \in \mathcal{A}_e}$  are group quantities interpreted as the group variables  $\tilde{v}_{e,a_e}$ , and uniquely determined by local variables  $w$ . In the following, we use the notation  $C_{\theta}^{\mathrm{MF}}$  to represent  $C_{\theta}^{\psi - \mathrm{MF}}$  for the function  $\psi$  defined in Lemma 5.3.
5.2 Constraint reduction
The remaining challenge is to reduce the number of constraints in (3), (4) or (5). For each global action  $a$ , the constraint writes  $\sum_{e\in [\rho ]}\tilde{v}_{e,a_e}^{-1}\leq \Delta (a)^2$ . The major issue is the non-linearity of the function appearing in the constraints w.r.t. group actions  $a_{e}$ . Upon inspection, it appears that the heterogeneity in the gaps (generally  $\Delta (a)\neq \Delta (b)$  for  $a\neq b$ ) is causing the non-linearity. To address this problem, we present, in the following lemma, a family of approximations leveraging an ordering of the first  $m$  smallest gaps. For  $m\in [K^N]$ , let  $a^{(m)}$  be the  $m^{\mathrm{th}}$  best global action and, for  $m\in [K^N -1]$ , let  $\Delta_m = \theta (a_\theta^\star) - \theta (a^{(m + 1)})$  be the  $m^{\mathrm{th}}$  minimal non-zero gap (with ties breaking arbitrarily).
Lemma 5.4. Let  $m \in [K^N - 1]$ , and  $\diamond \in \{\mathrm{L}, \mathrm{MF}\}$ . Let  $C_{\theta}^{\diamond}(m)$  be the value of the convex program:
$$
\min  _ {\tilde {v} \in \tilde {V} _ {\diamond}} \sum_ {e \in [ \rho ], a _ {e} \in \mathcal {A} _ {e}} \tilde {v} _ {e, a _ {e}} \left(\theta_ {e} \left(a _ {e} ^ {\star}\right) - \theta_ {e} \left(a _ {e}\right)\right) \tag {6}
$$
$$
\text {s . t .} \quad \sum_ {e \in [ \rho ]: a _ {e} ^ {(j + 1)} \neq a _ {e} ^ {\star}} \tilde {v} _ {e, a _ {e} ^ {(j + 1)}} ^ {- 1} \leq \Delta_ {j} ^ {2}, \quad \forall j \in [ m ] \tag {7}
$$