SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The intuition for their definition is the following. Letting  $i$  be a fixed arm, note that  $\mathrm{M}(i,j) > 0$  for some  $j$ , if and only if there exists a component  $d$  such that  $\mu_i^d > \mu_j^d$  i.e  $i$  is not dominated by  $j$ . Moreover, the larger  $\mathrm{M}(i,j)$ , the more  $i$  is non-dominated by  $j$  in the sense that there exists  $d$  such that  $\mu_i^d \gg \mu_j^d$ . Therefore,  $i$  is strictly optimal if and only if for all  $j \neq i$ ,  $\mathrm{M}(i,j) > 0$  i.e  $\alpha_i := \min_{j \neq i} \mathrm{M}(i,j) > 0$ . And the larger  $\alpha_i$ , the more  $i$  looks optimal in the sense that for each arm  $j \neq i$ , there exists a component  $d_j$  for which  $i$  is way better than  $j$ . As the  $\alpha_i$  are unknown, we define  $b_t$  as the maximizer of an optimistic estimate of the  $\alpha_i$ 's. We further restrict the maximization to arms for which we are not already convinced that they are optimal (by Lemma 1, the arms in  $\mathrm{OPT}^{\varepsilon_1}(t)$  are (nearly) Pareto optimal on the event  $\mathcal{E}$ ). Then, we note that for a fixed arm  $i$ ,  $\mathrm{M}(i,j) < 0$  if and only if  $i$  is strictly dominated by  $j$ . And the smaller  $\mathrm{M}(i,j)$ , the more  $j$  is close to dominate  $i$  (or largely dominates it): for any component  $d$ ,  $\mu_i^d - \mu_j^d$  is small (or negative). Thus, for a fixed arm  $i$ ,  $\operatorname*{argmin}_{j \neq i} \mathrm{M}(i,j)$  can be seen as the arm which is the closest to dominate  $i$  (or which dominates it by the largest margin). By minimizing a lower confidence bound on the unknown quantity  $\mathrm{M}(b_t,j)$ , our contender  $c_t$  can be interpreted as the arm which is the most likely to be (close to) dominating  $b_t$ . Gathering information on both  $b_t$  and  $c_t$  can be useful to check whether  $b_t$  can indeed be optimal.
Interestingly, we show in Appendix E that for  $D = 1$ , our sampling rule is close but not identical to the sampling rules used by existing confidence-based best arm identification algorithms.
Stopping condition	Recommendation	Objective
τε1	Z1ε1(t) > 0 ∧ Z2ε1(t) > 0	O(τε1)	ε1-PSI
τε1,ε2	Z1ε1,ε2(t) > 0 ∧ Z2ε1,ε2(t) > 0	OPTε1(τε1,ε2)	(ε1, ε2)-PSI
τk	|OPTε1(t)|≥ k	OPTε1(τk)	ε1-PSI-k
Table 1: Stopping conditions and associated recommendation
Stopping and recommendation rule(s) Depending on the objective, Adaptive Pareto Exploration can be plugged in with different stopping rules, that are summarized in Table I with their associated recommendations. To define those, we define for all  $i \in \mathbb{A}$ ,  $\varepsilon_1, \varepsilon_2 \geq 0$ ,
$$
g _ {i} ^ {\varepsilon_ {2}} (t) := \max  _ {j \neq i} \mathrm {m} ^ {-} (i, j, t) + \varepsilon_ {2} \mathbb {1} \{j \in \mathrm {O P T} ^ {\varepsilon_ {1}} (t) \} \quad \text {a n d} \quad h _ {i} ^ {\varepsilon_ {1}} (t) := \min  _ {j \neq i} \mathrm {M} ^ {-} (i, j, t) + \varepsilon_ {1}.
$$
and let  $g_{i}(t) \coloneqq g_{i}^{0}(t)$ . Introducing
$$
Z _ {1} ^ {\varepsilon_ {1}} (t) := \min  _ {i \in S (t)} h _ {i} ^ {\varepsilon_ {1}} (t), \text {a n d} Z _ {2} ^ {\varepsilon_ {1}} (t) := \min  _ {i \in S (t) ^ {\mathrm {c}}} \max  (g _ {i} (t), h _ {i} ^ {\varepsilon_ {1}} (t)),
$$
for  $\varepsilon_1$ -PSI, our stopping rule is  $\tau_{\varepsilon_1} \coloneqq \inf \{t \geq K : Z_1^{\varepsilon_1}(t) > 0 \land Z_2^{\varepsilon_1}(t) > 0\}$  and the associated recommendation is  $\mathcal{O}(\tau_{\varepsilon_1})$  where
$$
\mathcal {O} (t) := S (t) \cup \{i \in S (t) ^ {\mathrm {c}}: \not \exists j \neq i: \mathrm {m} ^ {-} (i, j, t) > 0 \}
$$
consists of the current empirical Pareto set plus some additional arms that have not yet been formally identified as sub-optimal. Those arms should be  $(\varepsilon_{1})$ -Pareto optimal.
For  $(\varepsilon_1,\varepsilon_2)$ -PSI we define a similar stopping rule  $\tau_{\varepsilon_1,\varepsilon_2}$  where the stopping statistics are respectively replaced with
$$
Z _ {1} ^ {\varepsilon_ {1}, \varepsilon_ {2}} (t) := \min  _ {i \in S (t)} \max  \left(g _ {i} ^ {\varepsilon_ {2}} (t), h _ {i} ^ {\varepsilon_ {1}} (t)\right) \text {a n d} Z _ {2} ^ {\varepsilon_ {1}, \varepsilon_ {2}} (t) := \min  _ {i \in S (t) ^ {\mathrm {c}}} \max  \left(g _ {i} ^ {\varepsilon_ {2}} (t), h _ {i} ^ {\varepsilon_ {1}} (t)\right)
$$
with the convention  $\min_{\emptyset} = +\infty$  , and the recommendation is  $\mathrm{OPT}^{\varepsilon_1}(\tau_{\varepsilon_1,\varepsilon_2})$
To tackle the  $\varepsilon_{1}$ -PSI- $k$  relaxation, we propose to couple  $\tau_{\varepsilon_1}$  with an additional stopping condition checking whether  $\mathrm{OPT}^{\varepsilon_1}(t)$  already contains  $k$  arms. That is, we stop at  $\tau_{\varepsilon_1}^k \coloneqq \min \left(\tau_{\varepsilon_1}, \tau^k\right)$  where  $\tau^k \coloneqq \inf \{t \geq K : |\mathrm{OPT}^{\varepsilon_1}(t)| \geq k\}$  with associated recommendation  $\mathrm{OPT}^{\varepsilon_1}(\tau^k)$ . Depending on the reason for stopping ( $\tau_{\varepsilon_1}$  or  $\tau^k$ ), we follow the corresponding recommendation.
Lemma 2. Assume  $\mathcal{E}$  holds. For  $\varepsilon_{1}$ -PSI (resp.  $(\varepsilon_{1},\varepsilon_{2})$ -PSI,  $\varepsilon_{1}$ -PSI- $k$ ), Adaptive Pareto Exploration combined with the stopping rule  $\tau_{\varepsilon_1}$  (resp.  $\tau_{\varepsilon_1,\varepsilon_2}$ , resp.  $\tau_{\varepsilon_1}^k$ ) outputs a correct subset.
Remark 1. We decoupled the presentation of the sampling rule to that of the "sequential testing" aspect (stopping and recommendation). We could even go further and observe that multiple tests could actually be run in parallel, for free. If we collect samples with APE (which only depends on  $\varepsilon_1$ ), whenever one of the three stopping conditions given in Table[1] triggers, for any values of  $\varepsilon_2$  or  $k$ , we can decide to stop and make the corresponding recommendation or continue and wait for another "more interesting" stopping condition to be satisfied. If  $\mathcal{E}$  holds, a recommendation made at any such time will be correct for the objective associated to the stopping criterion (third column in Table[1]).
3.2 Our instantiation
We propose to instantiate the algorithms with confidence interval on the difference of pair of arms. For any pair  $i,j\in \mathbb{A}$ , we define a function  $\beta_{i,j}$  such that for any  $d\in [D], U_{i,j}^{d}(t) = \widehat{\mu}_{i}^{d}(t) - \widehat{\mu}_{j}^{d}(t) + \beta_{i,j}(t)$  and  $L_{i,j}^{d}(t) = \widehat{\mu}_{i}^{d}(t) - \widehat{\mu}_{j}^{d}(t) - \beta_{i,j}(t)$ . We take from [20] the following confidence bonus for time-uniform concentration:
$$
\beta_ {i, j} (t) := 2 \sqrt {\left(C ^ {g} \left(\frac {\log \left(\frac {K _ {1}}{\delta}\right)}{2}\right) + \sum_ {a \in \{i , j \}} \log \left(4 + \log \left(T _ {a} (t)\right)\right)\right) \left(\sum_ {a \in \{i , j \}} \frac {1}{T _ {a} (t)}\right)}, \tag {4}
$$
where  $K_{1} \coloneqq K(K - 1)D / 2$  and  $C^g \approx x + \log (x)$  is a calibration function. They result in the simple expressions  $\mathrm{M}^{\pm}(i,j,t) = \mathrm{M}(i,j,t) \pm \beta_{i,j}(t)$  and  $\mathrm{m}^{\pm}(i,j,t) = \mathrm{m}(i,j,t) \pm \beta_{i,j}(t)$ . As an
Algorithm 1:  $\varepsilon_{1}$ -APE- $k$ : Adaptive Pareto Exploration for  $\varepsilon_{1}$ -PSI- $k$
Data: parameter  $\varepsilon_{1}\geq 0$ $k\in [K]$    
initialize: sample each arm once, set  $t = K$ $T_{i}(K) = 1$  for any  $i\in \mathbb{A}$    
for  $t = K + 1,\ldots ,$  do   
 $S(t) = \{i\in \mathbb{A}:\forall j\in \mathbb{A}\backslash \{i\} ,\mathrm{M}(i,j,t) > 0\} ;$ $\mathrm{OPT}^{\varepsilon_1}(t) = \{i\in \mathbb{A}:\forall j\in \mathbb{A}\backslash \{i\} ,\mathrm{M}(i,j,t) - \beta_{i,j}(t) + \varepsilon_1 > 0\} ;$    
if  $|\mathrm{OPT}^{\varepsilon_1}(t)|\geq k$  then   
 $\begin{array}{rl} & {\sqcup \mathrm{break}}\\ & {\sqcup \mathrm{break}}\\ & {\sqcup \mathrm{break}}\\ & {b_t:= \operatorname *{argmax}_{i\in \mathbb{A}\setminus \mathrm{OPT}^{\varepsilon_1}(t)}\min_{j\neq i}\left[\mathrm{M}(i,j,t) + \beta_{i,j}(t)\right];}\\ & {c_t:= \operatorname *{argmin}_{i\neq b_t}\left[\mathrm{M}(b_t,j,t) - \beta_{b_t,j}(t)\right];}\\ & {\mathrm{sample} a_t:= \operatorname *{argmin}_{i\in \{b_t,c_t\}}T_i(t);} \end{array}$
example, we state in Algorithm [1] the pseudo-code of APE combined the stopping rule suited for the  $k$ -relaxation of  $\varepsilon_{1}$ -PSI, which we refer to as  $\varepsilon_{1}$ -APE- $k$ .
In Appendix F, we also study a different instantiation based on confidence bounds of the form  $U_{i,j}(t) = U_i(t) - L_j(t)$  where  $[L_i(t), U_i(t)]$  is a confidence interval on  $\mu_i$ . This is the approach followed by LUCB for  $D = 1$  and prior work on Pareto identification [4, 27]. In practice we advocate the use of the pairwise confidence intervals defined above, even if our current analysis does not allow to quantify their improvement. For the LUCB-like instantiation, we further derive in Appendix F an upper bound on the expected stopping time of APE for the different stopping rules.
4 Theoretical analysis
In this section, we state our main theorem on the sample complexity of our algorithms and give a sketch of its proof.
First let us introduce some quantities that are needed to state the theorem. The sample complexity of the algorithm proposed by [4] for  $(\varepsilon_1)$ -Pareto set identification scales as a sum over the arms  $i$  of  $1 / (\Delta_i \vee \varepsilon_1)^2$  where  $\Delta_i$  is called the sub-optimality gap of arm  $i$  and is defined as follows. For a sub-optimal arm  $i \notin S^{\star}(\mathcal{X})$ ,
$$
\Delta_{i}:= \max_{j\in \mathcal{S}^{\star}}\mathrm{m}(i,j),
$$
which is the smallest quantity that should be added component-wise to  $\pmb{\mu}_i$  to make  $i$  appear Pareto optimal w.r.t  $\{\pmb{\mu}_i : i \in \mathbb{A}\}$ . For a Pareto optimal arm  $i \in S^*(\mathcal{X})$ , the definition is more involved:
$$
\Delta_ {i} := \left\{ \begin{array}{l l} \min  _ {j \in \mathbb {A} \setminus \{i \}} \Delta_ {j} & \text {i f} \mathcal {S} ^ {\star} := \{i \} \\ \min  (\delta_ {i} ^ {+}, \delta_ {i} ^ {-}) & \text {e l s e ,} \end{array} \right.
$$
$$
\delta_ {i} ^ {+} := \min  _ {j \in \mathcal {S} ^ {\star} \setminus \{i \}} \min  (\mathrm {M} (i, j), \mathrm {M} (j, i)) \text {a n d} \delta_ {i} ^ {-} := \min  _ {j \in \mathbb {A} \setminus \mathcal {S} ^ {\star}} \left\{\left(\mathrm {M} (j, i)\right) ^ {+} + \Delta_ {j} \right\}.
$$
For  $x \in \mathbb{R}, (x)^{+} \coloneqq \max(x, 0)$ . We also introduce some additional notion needed to express the contribution of the  $k$ -relaxation. Let  $1 \leq k \leq K$ . For any arm  $i$ , let  $\omega_{i} = \min_{j \neq i} \mathrm{M}(i, j)$  and define
$$
\omega^ {k} := \max  _ {i \in \mathbb {A}} ^ {k} \omega_ {i}, \qquad \mathcal {S} ^ {\star , k} := \operatorname * {a r g m a x} _ {i \in \mathbb {A}} ^ {1 \ldots k} \omega_ {i},
$$
with the  $k$ -th max and first to  $k$ -th argmax operators. Observe that  $\omega^k > 0$  if and only if  $|\mathcal{S}^\star(\mathcal{X})| \geq k$ .
Theorem 1. Fix a risk parameter  $\delta \in (0,1)$ ,  $\varepsilon_1 \geq 0$ , let  $k \leq K$  and  $\nu$  a bandit with 1-subgaussian marginals. With probability at least  $1 - \delta$ ,  $\varepsilon_1$ -APE- $k$  recommends a correct set for the  $\varepsilon_1$ -PSI- $k$  objective and stops after at most
$$
\sum_ {a \in \mathbb {A}} \frac {8 8}{\widetilde {\Delta} _ {a} ^ {2}} \log \left(\frac {2 K (K - 1) D}{\delta} \log \left(\frac {1 2 e}{\widetilde {\Delta} _ {a}}\right)\right),
$$
samples, where for each  $a\in \mathbb{A}$ $\widetilde{\Delta}_a\coloneqq \max (\Delta_a,\varepsilon_1,\omega^k)$
First, when  $k = K$ , observing that  $\varepsilon_{1}$ -APE-  $K$  provides a  $\delta$ -correct algorithm for  $\varepsilon_{1}$ -PSI, our bound improves the result of [4] for the  $\varepsilon_{1}$ -PSI problem in terms of constant multiplicative factors and  $\log \log \Delta^{-1}$  terms instead of  $\log \Delta^{-2}$ . It nearly matches the lower bound of Auer et al. [4] for the  $\varepsilon_{1}$ -PSI problem (Theorem 17 therein). It also shows the impact of the  $k$ -relaxation on the sample complexity. In particular, we can remark that for any arm  $i \in S^{\star} \backslash S^{\star, k}$ ,  $\max(\Delta_i, \omega_k) = \omega_k$ . Intuitively, it says that we shouldn't pay more than the cost of identifying the  $k$ -th optimal arm, ordered by the  $\omega_i'$ 's. A similar result has been obtained for the any  $k$ -sized subset of the best  $m$  problem [25]. But the authors have shown the relaxation only for the best  $m$  arms while our result shows that even the sub-optimal arms should be sampled less.
In Appendix D, we prove the following lower bound showing that in some scenarios,  $\varepsilon_{1}$ -APE- $k$  is optimal for  $\varepsilon_{1}$ -PSI- $k$ , up to  $D\log(K)$  and constant multiplicative terms. We note that for  $\varepsilon_{1}$ -PSI a lower bound featuring the gaps  $\Delta_{a}$  and  $\varepsilon_{1}$  was already derived by Auer et al. [4].
Theorem 2. There exists a bandit instance  $\nu$  with  $|\mathcal{S}^{\star}| = p\geq 3$  such that for  $k\in \{p,p - 1,p - 2\}$  any  $\delta$ -correct algorithm for 0-PSI- $k$  verifies
$$
\mathbb {E} _ {\nu} (\tau_ {\delta}) \geq \frac {1}{D} \log \left(\frac {1}{\delta}\right) \sum_ {a = 1} ^ {K} \frac {1}{\left(\Delta_ {a} ^ {k}\right) ^ {2}},
$$
where  $\Delta_{a}^{k}\coloneqq \Delta_{a} + \omega^{k}$  and  $\tau_{\delta}$  is the stopping time of the algorithm.
In Appendix C, we prove that Theorem 1 without the  $\omega^k$  terms also holds for  $(\varepsilon_1,\varepsilon_2)$ -APE. This does not justify the reduction in sample complexity when setting  $\varepsilon_{2} > 0$  in  $(\varepsilon_{1},\varepsilon_{2})$ -PSI observed in our experiments but it at least guarantees that the  $\varepsilon_{2}$ -relaxation doesn't make things worse.
Furthermore, since our algorithm allows  $\varepsilon_1 = 0$ , it is also an algorithm for BAI when  $D = 1, \varepsilon_1 = 0$ . We prove in Appendix E that in this case, the gaps  $\Delta_i^{\prime}s$  matches the classical gaps in BAI [2, 18] and we derive its sample complexity from Theorem 1 showing that it is similar in theory to UGap [9], LUCB [15] and LUCB++ [26] but have better empirical performance.
Sketch of proof of Theorem 1 Using Proposition 24 of [20] we first prove that the choice of  $\beta_{i,j}$  in (4) yields  $\mathbb{P}(\mathcal{E})\geq 1 - \delta$  for the good event  $\mathcal{E}$  defined in (1). Combining this result with Lemma 2 yields that  $\varepsilon_{1}$ -APE- $k$  is correct with probability at least  $1 - \delta$ .
The idea of the remaining proof is to show that under the event  $\mathcal{E}$ , if APE has not stopped at the end of round  $t$ , then the selected arm  $a_{t}$  has not been explored enough. The first lemma showing this is specific to the stopping rule  $\tau_{\varepsilon_1}^k$  used for  $\varepsilon_1$ -PSI- $k$ .
Lemma 3. Let  $\varepsilon_1 \geq$  and  $k \in [K]$ . If  $\mathcal{E}_t$  holds and  $t < \tau_{\varepsilon_1}^k$  then  $\omega^k \leq 2\beta_{a_t, a_t}(t)$ .
The next two lemmas are more general as they apply to different stopping rules.
Lemma 4. Let  $\varepsilon_1 \geq 0$ . Let  $\tau = \tau_{\varepsilon_1}^k$  for some  $k \in [K]$  or  $\tau = \tau_{\varepsilon_1, \varepsilon_2}$  for some  $\varepsilon_2 \geq 0$ . If  $\mathcal{E}_t$  holds and  $t < \tau$  then  $\Delta_{a_t} \leq 2\beta_{a_t, a_t}(t)$ .
Lemma 5. Let  $\varepsilon_1\geq 0$  and  $\tau$  be as in Lemma 4 If  $\mathcal{E}_t$  holds and  $t <   \tau$  then  $\varepsilon_{1}\leq 2\beta_{a_{t},a_{t}}(t)$
As can be seen in Appendix B the proofs of these three lemmas heavily rely on the specific definition of  $b_{t}$  and  $c_{t}$ . In particular, to prove Lemma 4 and 5 we first establish that when  $t < \tau$ , any arm  $j \in \mathbb{A}$  satisfies  $\mathrm{m}(b_t,j,t) \leq \beta_{b_t,j}(t)$ . The most sophisticated proof is then that of Lemma 4 which relies on a case distinction based on whether  $b_{t}$  or  $c_{t}$  belongs to the set of optimal arms.
These lemmas permit to show that, on  $\mathcal{E}_t$  if  $t < \tau_{\varepsilon_1}^k$  then  $\widetilde{\Delta}_{a_t} < 2\beta_{a_t,a_t}(t) \leq 2\beta^{T_{a_t}(t)}$ , where we define  $\beta^n$  to be the expression of  $\beta_{i,j}(t)$  when  $T_{i}(t) = T_{j}(t) = n$ . Then we have
$$
\begin{array}{l} \tau_ {\varepsilon_ {1}} ^ {k} \mathbb {1} \{\mathcal {E} \} \leq \sum_ {t = 1} ^ {\infty} \sum_ {a \in \mathbb {A}} \mathbb {1} \left\{a _ {t} = a \right\} \wedge \left\{\widetilde {\Delta} _ {a} \leq 2 \beta_ {a} ^ {T _ {a} (t)} \right\} \Bigg \} \\ \leq \sum_ {a \in \mathbb {A}} \inf  \left\{n \geq 2: \widetilde {\Delta} _ {a} > 2 \beta^ {n} \right\}. \tag {5} \\ \end{array}
$$
A careful upper-bounding of the RHS of (5) completes the proof of Theorem 1.
We evaluate the performance of Adaptive Pareto Exploration on a real-world scenario and on synthetic random Bernoulli instances. For a fair comparison, Algorithm 1 of [4], referred to as PSI-Unif-Elim and APE are both run with our confidence bonuses  $\beta_{i,j}(t)$  on pairs of arms, which considerably improve single-arm confidence bonuses. As anytime confidence bounds are known to be conservative, we use  $K_{1} = 1$  in (4) instead of its theoretical value coming from a union bound. Still, in all our experiments, the empirical error probability was (significantly) smaller than the target  $\delta = 0.1$ .
Real-world dataset COV-BOOST [24] is phase 2 trial which was conducted on 2883 participants to measure the immunogenicity of different Covid-19 vaccines as third dose (booster) in various combinations of initially received vaccines (first two doses). This resulted in a total of 20 vaccination strategies being assessed, each of them defined by the vaccines received as first, second and third dose. The authors have reported the average responses induced by each candidate strategy on cohorts of participants, measuring several immunogenicity markers. From this study, we extract and process the average response of each strategy to 3 specific immunogenicity indicators: two markers of antibody response and one of the cellular response. The outcomes are assumed to have a log-normal distribution [24]. We use the average (log) outcomes and their variances to simulate a multivariate Gaussian bandit with  $K = 20$ ,  $D = 3$ . We give in Appendix H.2 some additional details about the processing of the data, and report the means and variances of each arm. In Appendix H.1 we further explain how APE can be simply adapted when the marginals distributions of the arms have different variances. In this experiment, we set  $\varepsilon_1 = 0$ ,  $\delta = 0.1$  and compare PSI-Unif-Elim to 0-APE- $k$  (called
Figure 1: On the left is the log of the empirical sample complexity of PSI-Unif-Elim and APE on the real-world scenario plot (right) for 2 out of the 3 immunogenicity indicators.
APE- $k$  in the sequel) for different values of  $k$ . The empirical distribution of the sample complexity of the algorithms, averaged over 2000 independent runs, are reported in Figure 1. The results are shown in log-scale (y-axis is the log of the sample complexity) to fit in the same figure. As  $|S^{\star}| = 2$ , we first observe that, without the relaxation (i.e. for  $k > 3$ ), APE outperforms its state-of-the-art competitor PSI-Unif-Elim. Moreover for  $k = 1$  or  $k = 2$ , the sample complexity of APE- $k$  is significantly reduced. For  $k = 2$  when the stopping time  $\tau^{k}$  is reached some sub-optimal arms have possibly not yet been identified as such, while for  $k = 3$ , even if the optimal arms have been identified, the remaining arms have to be sampled enough to ensure that they are sub-optimal before stopping. This explains the gap in sample complexity between  $k = 2$  and  $k = 3$ . In Appendix H.3 we compare APE to an adaptation of PSI-Unif-Elim for the  $k$ -relaxation, showing that APE is always preferable.
Experiments on random instances To further illustrate the impact of the  $k$ -relaxation and to strengthen the comparison with PSI-Unif-Elim, we ran the previous algorithms on 2000 randomly generated multi-variate Bernoulli instances, with  $K = 5$  arms and different values of the dimension  $D$ . We set  $\delta = 0.1$  and  $\varepsilon_1 = 0.005$  (to have reasonable running time). The averaged sample complexities are reported in Table 2. We observe that APE (with  $k = K$ ) uses 20 to  $25\%$  less samples
3In their experiments, [4] already proposed the heuristic use of confidence bonuses of this form
than PSI-Unif-Elim and tends to be more efficient as the dimension increases (and likely the size of the Pareto set, since the instances are randomly generated). We also note that identifying a  $k$ -sized subset of the Pareto set requires considerably less samples than exact PSI. In Appendix H.3 we also provide examples of instances for which APE takes up to 3 times less samples than PSI-Unif-Elim.
ε1-APE-1	ε1-APE-2	ε1-APE-3	ε1-APE-4	ε1-APE-5	ε1-PSI-Unif-Elim
D=2	811	39530	109020	145777	150844	190625
D=4	214	6410	19908	68061	124001	157584
D=8	119	204	405	1448	20336	27270
Table 2: Average sample complexity over 2000 random Bernoulli instances with  $K = 5$  arms. On average the size of the Pareto set was (2.295, 4.0625, 4.931) respectively for the dimensions 2, 4, 8.
To illustrate the impact of the  $\varepsilon_{2}$  relaxation, setting  $\varepsilon_{1} = 0$  we report the sample complexity of APE associated with the stopping time  $\tau_{0,\varepsilon_2}$  for 20 equally spaced values of  $\varepsilon_{2}\in [0.01,0.05]$ , averaged over 2000 random Bernoulli instances. [Figure 2] shows the decrease of the average sample complexity when  $\varepsilon_{2}$  increases (left) and the average ratio of the size of the returned set to the size of the Pareto set (right). Note that for  $\varepsilon_{1} = 0$ , we have  $\mathcal{O}(\tau_{0,\varepsilon_2})\subset S^\star$ . The average sample complexity reported decreases up to  $86\%$  for the instance with  $K = 5$ ,  $D = 2$  while the returned set contains more than  $90\%$  of the Pareto optimal arms. In Appendix H.3 we further illustrate the behavior of APE with the  $\varepsilon_{2}$  relaxation on a fixed instance in dimension 2.
(a) Average sample complexity vs  $\varepsilon_{2}$
(b) Average ratio  $\left| \mathcal{O}(\tau_{0,\varepsilon_2}) \right| / |\mathcal{S}^*|$  vs  $\varepsilon_{2}$
Figure 2: APE with  $\tau_{0,\varepsilon_2}$  averaged over 2000 random Bernoulli instance with  $K = 5$  arms.
6 Conclusion and perspective
We proposed and analyzed APE, an adaptive sampling rule in multi-variate bandits models that when coupled with different stopping rules can tackle different relaxations of the fixed-confidence Pareto Set Identification problem. Our experiments revealed the good performance of the resulting algorithms compared to the state-of-the-art PSI algorithm as well as the great reductions in sample complexity brought by the relaxations. In future work, we intend to make our algorithms more practical for possible applications to clinical trials. For this purpose, as measuring efficacy takes time, we will investigate its adaptation to a batch setting, following, e.g. the work of [13] for BAI. We will also investigate the use of APE beyond the fixed-confidence setting, to the possibly more realistic fixed-budget [2] or anytime exploration [14] settings. To the best of our knowledge, no algorithm exists in the literature for PSI in such settings. Finally, following the works of [28, 4], we defined the  $\varepsilon_{1}, \varepsilon_{2}$  relaxations with scalar values, so that the same slack applies to all components. Although we could easily modify our algorithms to tackle vectorial values  $\varepsilon_{1}, \varepsilon_{2}$ , so far we could only prove a dependence on  $\min_d \varepsilon_1^d$  in the sample complexity. We intend to study the right quantification in the sample complexity when  $\varepsilon_{1}$  and  $\varepsilon_{2}$  are vectorial.
Cyrille Kone is funded by an Inria/Inserm PhD grant. Emilie Kaufmann acknowledges the support of the French National Research Agency under the projects BOLD (ANR-19-CE23-0026-04) and FATE (ANR22-CE23-0016-01).