SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
R (\Lambda) = \sum_ {k \in \mathcal {K}} \sum_ {m = 1} ^ {M} N _ {k} ^ {(m)} (\Lambda) \left(\frac {\lambda^ {(m)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right), \tag {22}
$$
where the inequality (a) is due to  $\Lambda >\sum_{t = 1}^{N}\lambda^{(m_t)}$ , the  $N_{k}^{(m)}(\Lambda)$  is the number of times that arm  $k$  is pulled in fidelity  $m$  after paying budget  $\Lambda$ . The  $N$  with subscript  $\forall k$  and superscript  $(\forall m)$  mean the pulling times of all arms and all fidelities respectively. Due to the multi-fidelity feedback, both decompositions are different from classic bandits' regret decomposition [24, Lemma 4.5], and, therefore, we need to non-trivially extend the known approaches to our scenario.
To prove the problem-independent lower bound  $\Omega (K^{1 / 3}\Lambda^{2 / 3})$  in Theorem 4.3, we utilize the decomposition in Eq.(21): In the RHS, the first term increases whether one choose fidelity other than the lowest; this is a novel term. The second term of RHS corresponds to the cost of pulling suboptimal arms which also appears in the classic MAB. With this observation, one can construct instance pairs such that it is unavoidable to do exploration at higher fidelities and, therefore, the first term is not negligible. Lastly, one needs to balance the magnitude of the above two terms case-by-case, which together bounds the regret as  $\Omega (\Lambda^{2 / 3})$ . To prove the problem-dependent lower bound in Theorem F.1, we utilize Eq.(22) to decompose the regret to each arm and bound each of them separately.
F.1 Proof of Theorem 4.3
Step 1. Construct instances and upper bound KL-divergence Fix a policy  $\pi$ . We construct two MF-MAB instances, each with  $K$  arms and  $M$  fidelities. For the pulling costs of different fidelities, we set  $\lambda^{(M)} \leqslant 2\lambda^{(1)}$ . For reward feedback, we assume all arms' reward distributions at any fidelity are Bernoulli, and denote  $\mu_{k}^{(m)}(1), \mu_{k}^{(m)}(2)$  as the reward means of these two instances. Let  $\mathbb{P}_1 = \mathbb{P}_{\pmb{\mu}(1),\pi}, \mathbb{E}_1 = \mathbb{E}_{\pmb{\mu}(1),\pi}$  and  $\mathbb{P}_2 = \mathbb{P}_{\pmb{\mu}(2),\pi}, \mathbb{E}_2 = \mathbb{E}_{\pmb{\mu}(2),\pi}$  be the probability measures and expectations on the canonical MF-MAB model induced by  $\Lambda$ -budget interconnection of  $\pi$  and  $\pmb{\mu}_1$  (and  $\pmb{\mu}_2$ ). Denote  $k' = \arg \min_{k \in \mathcal{K}} \mathbb{E}_1[N_k^{(m > 1)}(\Lambda)]$ .
The only difference between both instance pair is in the arm  $k'$ 's reward mean for fidelities  $m > 1$ , so that instance 1's optimal arm is arm 1 and instance 2's optimal arm is arm  $k'$ . The detailed reward means are listed as follows,
m=1	m>1
(μk(m)(1))k∈K	(1/2+Δ, 1/2, 1/2, ..., 1/2)	(1/2+Δ, 1/2, 1/2, ..., 1/2)
(μk(m)(2))k∈K	(1/2+Δ, 1/2, ..., 1/2, 1/2+2Δ, 1/2, ..., 1/2)
Denote the entry  $(M_t, I_t, X_t)$  as a tuple of the pulled fidelity, pulled arm, and observed reward random variables at time  $t$ , and  $\mathcal{H} \coloneqq (M_1, I_1, X_1; M_2, I_2, X_2; \ldots; M_N, I_N, X_N)$  as a sequence of applying policy  $\pi$ . We note that  $N$  is also a random variable depending on the sequence of fidelities in pulling arms. Next, we calculate the upper bound of the KL-divergence of the above two instances in this sequence.
$$
\begin{array}{l} \operatorname {K L} \left(\mathbb {P} _ {1}, \mathbb {P} _ {2}\right) = \mathbb {E} _ {1} \left[ \log \left(\frac {d \mathbb {P} _ {1}}{d \mathbb {P} _ {2}}\right) \right] \\ = \mathbb {E} _ {1} \left[ \log \left(\frac {d \mathbb {P} _ {1}}{d \mathbb {P} _ {2}} \left(M _ {1}, I _ {1}, X _ {1}; M _ {2}, I _ {2}, X _ {2}; \dots ; M _ {N}, I _ {N}, X _ {N}\right)\right) \right] \\ = \mathbb {E} _ {1} \left[ \mathbb {E} _ {1} \left[ \log \left(\frac {d \mathbb {P} _ {1}}{d \mathbb {P} _ {2}} \left(M _ {1}, I _ {1}, X _ {1}; M _ {2}, I _ {2}, X _ {2}; \dots ; M _ {N}, I _ {N}, X _ {N}\right)\right) \mid N \right] \right] \\ \stackrel {(a)} {=} \mathbb {E} _ {1} \left[ \mathbb {E} _ {1} \left[ \sum_ {t = 1} ^ {N} \log \left(\frac {p _ {1} (X _ {t} | M _ {t} , I _ {t})}{p _ {2} (X _ {t} | M _ {t} , I _ {t})}\right) \Bigg | N \right] \right] \\ = \mathbb {E} _ {1} \left[ \sum_ {t = 1} ^ {N} \mathbb {E} _ {1} \left[ \log \left(\frac {p _ {1} \left(X _ {t} \mid M _ {t} , I _ {t}\right)}{p _ {2} \left(X _ {t} \mid M _ {t} , I _ {t}\right)}\right) \Bigg | N \right] \right] \\ \stackrel {(b)} {=} \mathbb {E} _ {1} \left[ \sum_ {t = 1} ^ {N} \mathbb {E} _ {1} \left[ \operatorname {K L} \left(P _ {1} ^ {\left(M _ {t}, I _ {t}\right)}, P _ {2} ^ {\left(M _ {t}, I _ {t}\right)}\right) | N \right] \right] \\ = \sum_ {(m, k) \in \mathcal {M} \times \mathcal {K}} \mathbb {E} _ {1} \left[ \sum_ {t = 1} ^ {N} \mathbb {E} _ {1} \left[ \left. \mathbb {1} \{M _ {t} = m, I _ {t} = k \} \operatorname {K L} \left(P _ {1} ^ {\left(M _ {t}, I _ {t}\right)}, P _ {2} ^ {\left(M _ {t}, I _ {t}\right)}\right) \right| N \right] \right] \\ = \sum_ {(m, k) \in \mathcal {M} \times \mathcal {K}} \mathbb {E} _ {1} \left[ \operatorname {K L} \left(P _ {1} ^ {(m, k)}, P _ {2} ^ {(m, k)}\right) \sum_ {t = 1} ^ {N} \mathbb {E} _ {1} [ \mathbb {1} \{M _ {t} = m, I _ {t} = k \} | N ] \right] \\ = \sum_ {(m, k) \in \mathcal {M} \times \mathcal {K}} \operatorname {K L} \left(P _ {1} ^ {(m, k)}, P _ {2} ^ {(m, k)}\right) \mathbb {E} _ {1} \left[ \mathbb {E} _ {1} \left[ \sum_ {t = 1} ^ {N} \mathbb {1} \{M _ {t} = m, I _ {t} = k \} \mid N \right] \right] \\ \stackrel {(c)} {=} \sum_ {(m, k) \in \mathcal {M} \times \mathcal {K}} \operatorname {K L} \left(P _ {1} ^ {(m, k)}, P _ {2} ^ {(m, k)}\right) \mathbb {E} _ {1} \left[ T _ {k} ^ {(m)} (\Lambda) \right] \\ = \operatorname {K L} \left(P _ {1} ^ {(m, 2)}, P _ {2} ^ {(m, 2)}\right) \mathbb {E} _ {1} \left[ T _ {k ^ {\prime}} ^ {(m > 1)} (\Lambda) \right] \\ \stackrel {(d)} {\leqslant} \frac {1}{K} \cdot \mathbb {E} _ {1} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \right] \cdot \operatorname {K L} \left(\mathcal {B} \left(\frac {1}{2}\right), \mathcal {B} \left(\frac {1}{2} + 2 \Delta\right)\right) \\ \leqslant \mathbb {E} _ {1} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \right] \cdot \frac {9 \Delta^ {2}}{K}, \tag {23} \\ \end{array}
$$
where the equation (a) is due to
$$
\begin{array}{l} \frac {d \mathbb {P} _ {1}}{d (\rho \times \rho \times \lambda) ^ {N}} (m _ {1}, k _ {1}, x _ {1}; m _ {2}, k _ {2}, x _ {2}; \dots ; m _ {N}, k _ {N}, x _ {N}) \\ = p _ {\boldsymbol {\mu} _ {1}, \pi} (m _ {1}, k _ {1}, x _ {1}; m _ {2}, k _ {2}, x _ {2}; \dots ; m _ {N}, k _ {N}, x _ {N}) \\ = \prod_ {t = 1} ^ {N} \pi_ {t} \left(m _ {t}, k _ {t} | m _ {1}, k _ {1}, x _ {1}; \dots ; m _ {t - 1}, k _ {t - 1}, x _ {t - 1}\right) p _ {\boldsymbol {\mu} _ {1}} \left(x _ {t} | m _ {t}, k _ {t}\right), \\ \end{array}
$$
the equation (b) is due to
$$
\begin{array}{l} \mathbb {E} _ {1} \left[ \log \left(\frac {p _ {1} \left(X _ {t} \mid M _ {t} , I _ {t}\right)}{p _ {2} \left(X _ {t} \mid M _ {t} , I _ {t}\right)}\right) \Bigg | N \right] = \mathbb {E} _ {1} \left[ \mathbb {E} _ {1} \left[ \log \left(\frac {p _ {1} \left(X _ {t} \mid M _ {t} , I _ {t}\right)}{p _ {2} \left(X _ {t} \mid M _ {t} , I _ {t}\right)}\right) \Bigg | M _ {t}, I _ {t} \right] \Bigg | N \right] \\ = \mathbb {E} _ {1} \left[ \operatorname {K L} \left(P _ {1} ^ {\left(M _ {t}, I _ {t}\right)}, P _ {2} ^ {\left(M _ {t}, I _ {t}\right)}\right) \mid N \right], \\ \end{array}
$$
the equation (c) is due to the tower property as well, the inequality (d) is because arm  $k'$  is the arm with the smallest number of pulled with fidelity  $m > 1$ , and the inequality (e) is by calculating the KL-divergent between two Bernoulli distributions.
Step 2. Lower bound the regret We first note that the regret defined in Eq.(9) for instance 1 can be decomposed as follows,
$$
\begin{array}{l} \mathbb {E} _ {1} [ R (\Lambda) ] \stackrel {(a)} {\geqslant} \sum_ {t = 1} ^ {N} \left(\frac {\lambda^ {(m _ {t})} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} + \Delta_ {I _ {t}} ^ {(M)}\right) \\ = \sum_ {m = 2} ^ {M} T _ {\forall k} ^ {(m)} (\Lambda) \frac {\lambda^ {(m)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} + \sum_ {k \neq k _ {1} ^ {(M)}} T _ {k} ^ {(\forall m)} (\Lambda) \Delta_ {k} ^ {(M)} \\ \end{array}
$$
where inequality (a) is due to  $\Lambda >\sum_{t = 1}^{N}\lambda^{(m_t)}$ , the  $T_{k}^{(m)}(\Lambda)$  is the number of times that arm  $k$  is pulled in fidelity  $m$  (given total budget  $\overline{\Lambda}$ ).
Then, we lower bound the summation of the regrets under both instances,
$$
\begin{array}{l} \mathbb {E} _ {1} [ R (\Lambda) ] + \mathbb {E} _ {2} [ R (\Lambda) ] \\ = \mathbb {E} _ {1} \left[ \sum_ {m = 2} ^ {M} T _ {\forall k} ^ {(m)} (\Lambda) \frac {\lambda^ {(m)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} + \sum_ {k \neq k _ {1} ^ {(M)}} T _ {k} ^ {(\forall m)} (\Lambda) \Delta_ {k} ^ {(M)} \right] \\ + \mathbb {E} _ {2} \left[ \sum_ {m = 2} ^ {M} T _ {\forall k} ^ {(m)} (\Lambda) \frac {\lambda^ {(m)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} + \sum_ {k \neq k _ {1} ^ {(M)}} T _ {k} ^ {(\forall m)} (\Lambda) \Delta_ {k} ^ {(M)} \right] \\ \geqslant \mathbb {E} _ {1} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \frac {\lambda^ {(2)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} + \sum_ {k \neq k _ {1} ^ {(M)}} T _ {k} ^ {(\forall m)} (\Lambda) \Delta_ {k} ^ {(M)} \right] \\ + \mathbb {E} _ {2} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \frac {\lambda^ {(2)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} + \sum_ {k \neq k _ {1} ^ {(M)}} T _ {k} ^ {(\forall m)} (\Lambda) \Delta_ {k} ^ {(M)} \right] \\ \end{array}
$$
$$
\begin{array}{l} \stackrel {(a)} {\geqslant} \mathbb {E} _ {1} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \right] \frac {\lambda^ {(2)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} \\ + \Delta \min  \left\{\frac {\Lambda}{\lambda^ {(M)}} - \frac {\Lambda}{2 \lambda^ {(1)}}, \frac {\Lambda}{2 \lambda^ {(1)}} \right\} \left(\mathbb {P} _ {1} \left(T _ {1} ^ {(\forall m)} \leqslant \frac {\Lambda}{2 \lambda^ {(1)}}\right) + \mathbb {P} _ {2} \left(T _ {1} ^ {(\forall m)} > \frac {\Lambda}{2 \lambda^ {(1)}}\right)\right) \\ \end{array}
$$
$$
\stackrel {(b)} {\geqslant} \mathbb {E} _ {1} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \right] \frac {\lambda^ {(2)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} + \frac {\Lambda \Delta}{2} \min  \left\{\frac {1}{\lambda^ {(M)}} - \frac {1}{2 \lambda^ {(1)}}, \frac {1}{2 \lambda^ {(1)}} \right\} \exp (- \mathrm {K L} (\mathbb {P} _ {1}, \mathbb {P} _ {2}))
$$
$$
\begin{array}{l} \stackrel {(c)} {\geqslant} \mathbb {E} _ {1} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \right] \frac {\lambda^ {(2)} - \lambda^ {(1)}}{\lambda^ {(1)}} \mu_ {1} \\ + \frac {\Lambda \Delta}{2} \min \left\{\frac {1}{\lambda^ {(M)}} - \frac {1}{2 \lambda^ {(1)}}, \frac {1}{2 \lambda^ {(1)}} \right\} \exp \left(- 2 \Delta^ {2} K ^ {- 1} \mathbb {E} _ {1} \left[ T _ {\forall k} ^ {(m > 1)} (\Lambda) \right]\right), \\ \end{array}
$$
where inequality (a) uses the  $\lambda^{(M)}\leqslant 2\lambda^{(1)}$  condition, inequality (b) is by Bretagnolle-Huber inequality [24, Theorem 14.2], and inequality (c) is by Eq.(23).
Step 3. Obtain the  $\Omega(K^{\frac{1}{3}}\Lambda^{\frac{2}{3}})$  lower bound We show that  $\mathbb{E}_1[R(\Lambda)] + \mathbb{E}_2[R(\Lambda)] \geqslant \Omega(K^{\frac{1}{3}}\Lambda^{\frac{2}{3}})$  for any possible quantity of  $\mathbb{E}_1\left[T_{\forall k}^{(m > 1)}(\Lambda)\right]$  via categorized discussion as follows,
- Case 1: If  $\mathbb{E}_1\left[T_{\forall k}^{(m > 1)}(\Lambda)\right] = 0$ , then the last formula becomes  $C\Lambda \Delta$ . Letting  $\Delta$  be a constant (via sup), we have a  $\Omega(\Lambda)$  lower bound, which means  $\Omega(K^{\frac{1}{3}}\Lambda^{\frac{2}{3}})$  is also valid.
- Case 2: If  $\mathbb{E}_1\left[T_{\forall k}^{(m > 1)}(\Lambda)\right] \geqslant K^{\frac{1}{3}}\Lambda^{\frac{2}{3}}$ , then we have the  $\Omega(K^{\frac{1}{3}}\Lambda^{\frac{2}{3}})$  lower bound from the first term.
- Case 3: If  $0 < \mathbb{E}_1\left[T_{\forall k}^{(m > 1)}(\Lambda)\right] < K^{\frac{1}{3}}\Lambda^{\frac{2}{3}}$ , we choose  $\Delta = K^{\frac{1}{3}}\Lambda^{-\frac{1}{3}}$  and also obtain the  $\Omega (K^{\frac{1}{3}}\Lambda^{\frac{2}{3}})$  lower bound.
F.2 Proof of Theorem 4.4
In this proof, we prove that for any arm  $k \in \mathcal{K}$ , the total regret due to this arm  $k$  is lower bounded as follows,
$$
\liminf _ {\Lambda \to \infty} \frac {\mathbb {E} [ R _ {k} (\Lambda) ]}{\log (\Lambda)} \geqslant \min _ {m: \Delta_ {k} ^ {(m)} > 0} \left(\frac {\lambda^ {(m)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) \frac {C}{(\Delta_ {k} ^ {(m)}) ^ {2}}.
$$
Then, noticing that  $R(\Lambda) = \sum_{k\in \mathcal{M}}R_k(\Lambda)$  concludes the proof.
We construct instances  $\nu$  and  $\nu'$ . We set the reward distributions  $\nu = (\nu_k^{(m)})_{(k,m) \in \mathcal{K} \times \mathcal{M}}$  as Bernoulli and the reward means fulfill  $\mu_1^{(M)} > \mu_2^{(M)} \geqslant \mu_3^{(M)} \geqslant \ldots \geqslant \mu_K^{(M)}$ , where  $\mu_k^{(m)} = \mathbb{E}_{X \sim \nu_k^{(m)}}[X]$ . We let  $\nu'_k^{(m)}$  be the same to  $\nu_k^{(m)}$  for all  $k$  and  $m$ , except for that an arm  $\ell \neq 1$ . We set arm  $\ell$ 's reward means on fidelities  $m \in \mathcal{M}_k$  to be  $\nu'_\ell^{(m)} = \nu_1^{(M)} - \zeta^{(m)} + \epsilon$ . One can verify that the reward means of arm  $k$  under instance  $v'$  fulfill the condition that  $|\mu'_k^{(m)} - \mu'_k^{(M)}| \leqslant \zeta^{(m)}$  for any fidelity  $m$ . Notice that  $\mu'_k^{(M)} > \mu_{k_*}^{(M)}$ . Hence, under instance  $\mathcal{I}'$ , the optimal arm is  $k$ . We denote  $\mathbb{P}, \mathbb{E}$  and  $\mathbb{P}', \mathbb{E}'$  as the probability measures and expectations for instances  $\mathcal{I}$  and  $\mathcal{I}'$  respectively.
Next, we employ the (extended) key inequality from Garivier et al. [12] as follows,
$$
\sum_ {k \in \mathcal {K}} \sum_ {m \in \mathcal {M}} \mathbb {E} \left[ N _ {k} ^ {(m)} (\Lambda) \right] \operatorname {K L} \left(v _ {k} ^ {(m)}, v _ {k} ^ {\prime (m)}\right) \geqslant \operatorname {k l} \left(\mathbb {E} [ Z ], \mathbb {E} ^ {\prime} [ Z ]\right).
$$
Let  $Z = \lambda^{(M)}N_{\ell}^{(\forall m)}(\Lambda) / \Lambda$  in the above inequality, we have
$$
\begin{array}{l} \sum_ {m: \Delta_ {\ell} ^ {(m)} > 0} \mathbb {E} \left[ N _ {k} ^ {(m)} (\Lambda) \right] \operatorname {K L} \left(v _ {\ell} ^ {(m)}, v _ {\ell} ^ {\prime (m)}\right) \\ \geqslant \mathrm {k l} \left(\frac {\mathbb {E} \left[ N _ {\ell} ^ {(\forall m)} (\Lambda) \right]}{\Lambda}, \frac {\mathbb {E} ^ {\prime} \left[ N _ {\ell} ^ {(\forall m)} (\Lambda) \right]}{\Lambda}\right) \tag {24} \\ \stackrel {(a)} {\geqslant} \left(1 - \frac {\lambda^ {(M)} \mathbb {E} [ N _ {\ell} ^ {(\forall m)} (\Lambda) ]}{\Lambda}\right) \log \frac {\Lambda}{\Lambda - \lambda^ {(M)} \mathbb {E} ^ {\prime} [ N _ {\ell} ^ {(\forall m)} (\Lambda) ]} - \log 2 \\ \end{array}
$$
where inequality (a) is due to that for all  $(p,q)\in [0,1]^2,\mathrm{kl}(p,q)\geqslant (1 - p)\log (1 / (1 - q)) - \log 2$ . Notice that the regret attributed to any arm  $k$  can be decomposed and lower bounded as follows,
$$
R _ {\ell} (\Lambda) \geqslant \sum_ {m = 1} ^ {M} N _ {\ell} ^ {(m)} (\Lambda) \left(\frac {\lambda^ {(m)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {\ell} ^ {(M)}\right),
$$
with the constraint in Eq.(24). Since this is a linear programming, we know its solution is reached at its vertex. Therefore, we lower bound the regret as follows,
$$
\begin{array}{l} R _ {\ell} (\Lambda) \geqslant \min  _ {m: \Delta_ {k} ^ {(m)} > 0} \left(\frac {\lambda^ {(m)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {\ell} ^ {(M)}\right) \frac {1}{\mathrm {K L} \left(v _ {\ell} ^ {(m)} , v _ {\ell} ^ {\prime (m)}\right)} \tag {25} \\ \times \left(\left(1 - \frac {\lambda^ {(M)} \mathbb {E} [ N _ {\ell} ^ {(\forall m)} (\Lambda) ]}{\Lambda}\right) \log \frac {\Lambda}{\Lambda - \lambda^ {(M)} \mathbb {E} ^ {\prime} [ N _ {\ell} ^ {(\forall m)} (\Lambda) ]} - \log 2\right), \\ \end{array}
$$
Notice that the policy is consistent, that is,  $\mathbb{E}[N_{\ell}^{(\forall m)}(\Lambda)] = o(T^a)$  and  $\mathbb{E}'[N_k^{(\forall m)}(\Lambda)] = o(\Lambda^a)$  for any  $a \in (0,1]$  and any suboptimal arm  $k \neq \ell$ . We have  $\frac{\lambda^{(M)}\mathbb{E}[N_{\ell}^{(\forall m)}(\Lambda)]}{\Lambda} = o(1)$  and
$$
\Lambda - \lambda^ {(M)} \mathbb {E} ^ {\prime} \left[ N _ {\ell} ^ {(\forall m)} (\Lambda) \right] \leqslant \lambda^ {(M)} \sum_ {k \neq \ell} \mathbb {E} ^ {\prime} \left[ N _ {k} ^ {(\forall m)} (\Lambda) \right] = o \left(\Lambda^ {a}\right).
$$
Dividing both sides of Eq.(25) by  $\Lambda$ , and letting  $\Lambda$  go to infinity and  $a$  go to 1, we have
$$
\begin{array}{l} \liminf_{\Lambda \to \infty}\frac{\mathbb{E}[R_{\ell}(\Lambda)]}{\Lambda}\geqslant \min_{m:\Delta_{\ell}^{(m)} > 0}\left(\frac{\lambda^{(m)}}{\lambda^{(1)}}\mu_{1}^{(M)} - \mu_{\ell}^{(M)}\right)\frac{1}{\mathrm{KL}(v_{\ell}^{(m)},v^{\prime}_{\ell}^{(m)})} \\ \geqslant \min  _ {m: \Delta_ {\ell} ^ {(m)} > 0} \left(\frac {\lambda^ {(m)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {\ell} ^ {(M)}\right) \frac {1}{\mathrm {K L} (v _ {\ell} ^ {(m)} , v _ {1} ^ {(M)} - \zeta^ {(m)} + \epsilon)} \\ \end{array}
$$
To bound the optimal arm 1's sample cost, we use the same  $\nu$  as above and construct another instance  $\nu''$ . The instance  $\nu''$ s reward means are the same to  $\nu$  except for arm 1 whose reward means for fidelity  $m \in \mathcal{M}_1$  are set as  $\mu_{1}^{\prime \prime(m)} = \mu_2^{(m)} + \zeta^{(m)} - \epsilon$ . Then, with similar procedure as the above, we obtain
$$
\liminf_{\Lambda \to \infty}\frac{\mathbb{E}[R_{1}(\Lambda)]}{\Lambda}\geqslant \min_{m:\Delta_{k}^{(m)} > 0}\left(\frac{\lambda^{(m)}}{\lambda^{(1)}}\mu_{1}^{(M)} - \mu_{1}^{(M)}\right)\frac{1}{\mathrm{KL}(v_{1}^{(m)},v_{2}^{(m)} + \zeta^{(m)} - \epsilon)}
$$
F.3 Proof of Theorem 4.8
We first prove a problem dependent regret upper bound as follows and then convert this bound to the problem independent regret upper bound presented in Theorem 4.8.
Denote  $\Delta_k^{(M)} = \mu_1^{(M)} - \mu_k^{(M)}$ , and, especially,  $\Delta_1^{(M)} = 0$ .
Theorem F.1 (Problem-Dependent Regret Upper Bound). For any  $\varepsilon > 0$ , Algorithm 3's regret is upper bounded as follows,
$$
\begin{array}{l} \mathbb {E} [ R (\Lambda) ] \leqslant \sum_ {k: \Delta_ {k} ^ {(M)} > \varepsilon} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) \left(\frac {1 6}{(\Delta_ {k} ^ {(M)}) ^ {2}} \log \frac {\Lambda (\Delta_ {k} ^ {(M)}) ^ {2}}{1 6 \lambda^ {(M)}} + \frac {4 8}{(\Delta_ {k} ^ {(M)}) ^ {2}} + 1\right) + \frac {6 4}{\Delta_ {k} ^ {(M)}}\right) \\ + \sum_ {k: \Delta_ {k} ^ {(M)} \leqslant \varepsilon} \left(\left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) \left(\frac {1 6}{\varepsilon^ {2}} \log \left(\frac {\Lambda \varepsilon^ {2}}{1 6 \lambda^ {(M)}}\right) + \frac {3 2}{3 \varepsilon^ {2}} + 1\right) + \frac {6 4}{\varepsilon}\right) + \max  _ {k: \Delta_ {k} ^ {(M)} \leqslant \varepsilon} \frac {\Lambda}{\lambda^ {(1)}} \Delta_ {k} ^ {(M)}. \tag {26} \\ \end{array}
$$
Proof of Theorem F.1. By Hoeffding's inequality, we have
$$
\mathbb {P} \left(\hat {\mu} _ {k, p} ^ {(M)} > \mu_ {k} ^ {(M)} + 2 ^ {- p}\right) \leqslant \exp \left(- \frac {2 ^ {- 2 p}}{2 \times \frac {1}{2 \times 2 ^ {2 p} \log (\Lambda / 2 ^ {2 p} \lambda^ {(M)})}}\right) = \frac {2 ^ {2 p} \lambda^ {(M)}}{\Lambda}
$$
$$
\mathbb {P} \left(\hat {\mu} _ {k, p} ^ {(M)} <   \mu_ {k} ^ {(M)} - 2 ^ {- p}\right) \leqslant \exp \left(- \frac {2 ^ {- 2 p}}{2 \times \frac {1}{2 \times 2 ^ {2 p} \log (\Lambda / 2 ^ {2 p} \lambda^ {(M)})}}\right) = \frac {2 ^ {2 p} \lambda^ {(M)}}{\Lambda}.
$$
That is, the empirical mean  $\hat{\mu}_k^{(M)}$  is within the confidence interval  $(\mu_k^{(M)} - 2^{-p}, \mu_k^{(M)} + 2^{-p})$  with high probability.
Choose any  $\varepsilon >\frac{e}{\Lambda}$ . Let  $\mathcal{K}^{\prime} = \{k\in \mathcal{K}|\Delta_{k}^{(M)} > \varepsilon \}$ . Denote  $p_k\coloneqq \min \{p:2^{-p} < \frac{\Delta_k^{(M)}}{2}\}$ . From  $p_k$  's definition, we have the following inequality
$$
2 ^ {p _ {k}} <   \frac {4}{\Delta_ {k} ^ {(M)}} <   2 ^ {p _ {k} + 1}. \tag {27}
$$
We also note that the cost of pulling a suboptimal arm  $k \in \mathcal{K}'$  at highest fidelity  $M$  is upper bounded as  $\frac{\lambda^{(M)}}{\lambda^{(1)}}\mu_1^{(M)} - \mu_k^{(M)}$ , where the factor  $\frac{\lambda^{(M)}}{\lambda^{(1)}}$  is because the budget paying to pull an arm at fidelity  $M$  can be used to the arm at fidelity 1 for fractional times.
The rest of this proof consists of two steps. In the first step, we assume that all empirical means are in their corresponding confidence intervals at each phases, and show the algorithm can properly eliminate all suboptimal arms in  $\mathcal{K}'$  —the arm is eliminated in or before the phase  $p_k$ . In the second step, we upper bound the regret if there are any empirical estimates lying outside their corresponding confidence intervals.
Step 1. If all arms' empirical means lie in confidence intervals. That is, any suboptimal arm  $k$  is eliminated in or before the phase  $p_k$ . Because, if  $k \in \mathcal{C}_{p_k}$ , we have
$$
\begin{array}{l} \hat {\mu} _ {k} ^ {(M)} \leqslant \mu_ {k} ^ {(M)} + 2 ^ {- p _ {k}} = \mu_ {1} ^ {(M)} - \Delta_ {k} ^ {(M)} + 2 ^ {- p _ {k}} \\ \stackrel {(a)} {\leqslant} \mu_ {1} ^ {(M)} - 4 \times 2 ^ {- p _ {k} - 1} + 2 ^ {- p _ {k}} = \mu_ {1} ^ {(M)} - 2 ^ {- p _ {k}} <   \hat {\mu} _ {1} ^ {(M)} \leqslant \max  _ {k \in \mathcal {C} _ {p _ {k}}} \hat {\mu} _ {k} ^ {(M)}, \\ \end{array}
$$
where (a) is due to Eq.(27). That is, if this arm  $k$  haven't been eliminated before phase  $p_k$ , it must be eliminated in this phase. Therefore, the total pulling times of this arm  $k$  at highest fidelity  $M$  is upper bounded as follows,
$$
T _ {k} ^ {(M)} \leqslant \left[ 2 ^ {2 p _ {k}} \log \frac {\Lambda}{2 ^ {2 p _ {k}} \lambda^ {(M)}} \right] \stackrel {(a)} {\leqslant} \frac {1 6}{\left(\Delta_ {k} ^ {(M)}\right) ^ {2}} \log \left(\frac {\left(\Delta_ {k} ^ {(M)}\right) ^ {2} \Lambda}{1 6 \lambda^ {(M)}}\right) + 1, \tag {28}
$$
where (a) is due to Eq.(27).
We then handle the number of times of pulling arms  $k$  with  $\Delta_k^{(M)} \leqslant \varepsilon$  in fidelity  $M$ . Although these arms' total pulling times, eliminated in or before phase  $p_k$ , are also upper bounded by Eq.(28), their corresponding phases  $p_k$  is greater than  $\log_2\frac{2}{\varepsilon}$  and, therefore, cannot be reached. So, these arms' (including the optimal arm 1's) total pulling times in fidelity  $M$  is upper bounded by
$$
\frac {1 6}{\varepsilon^ {2}} \log \left(\frac {\Lambda \varepsilon^ {2}}{1 6 \lambda^ {(M)}}\right) + 1.
$$
Therefore, the cost due to pulling arms at fidelity  $M$  is upper bounded as follows,
$$
\sum_ {k \in \mathcal {K}} \left(\frac {\lambda^ {(M)}}{\lambda^ {(1)}} \mu_ {1} ^ {(M)} - \mu_ {k} ^ {(M)}\right) \left(\frac {1 6}{\left(\max  \left\{\varepsilon , \Delta_ {k} ^ {(M)} \right\}\right) ^ {2}} \log \left(\frac {\left(\max  \left\{\varepsilon , \Delta_ {k} ^ {(M)} \right\}\right) ^ {2} \Lambda}{1 6 \lambda^ {(M)}}\right) + 1\right). \tag {29}
$$
After the elimination process, arms with  $\Delta_k^{(M)}\leqslant \varepsilon$  may remain in the candidate arm set  $\mathcal{C}_p$  and are exploited in turn at fidelity  $m = 1$ . As some of them are not the optimal arm, this additional cost can be upper bounded as follows,
$$
\max  _ {k: \Delta_ {k} ^ {(M)} \leqslant \varepsilon} \frac {\Lambda}{\lambda^ {(1)}} \Delta_ {k} ^ {(M)}. \tag {30}
$$
Step 2. There are two cases that some arms are eliminated improperly: for an suboptimal arm  $k$ , either
2.1 The suboptimal arm  $k$  is not eliminated in (or before) the phase  $p_k$ , and the optimal arm 1 is in  $\mathcal{C}_{p_k}$  in phase  $p_k$ ; or
2.2 The suboptimal arm  $k$  is eliminated in (or before) the phase  $p_k$ , and the optimal arm 1 is not in  $\mathcal{C}_{p_k}$  in phase  $p_k$ .