Original: Multi-Fidelity Best-Arm Identiﬁcation
Revised: Multi-Fidelity Best-Arm Identiﬁcation

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: In several real-world applications, a learner has access to multiple environmentsimulators, each with a different precision (e.g., simulation accuracy) and cost(e.g., computational time).
Revised: In several real-world applications, a learner has access to multiple environment simulators, each with a different precision (e.g., simulation accuracy) and cost (e.g., computational time).

Original: In such a scenario, the learner faces the trade-offbetween selecting expensive accurate simulators or preferring cheap impreciseones.
Revised: In such a scenario, the learner faces the trade-off between selecting expensive accurate simulators or preferring cheap imprecise ones.

Original: We formalize this setting as a multi-ﬁdelity variant of the stochastic best- arm identiﬁcation problem, where querying the original arm is expensive, butmultiple and biased approximations (i.e., ﬁdelities) are available at lower costs.
Revised: We formalize this setting as a multi-ﬁdelity variant of the stochastic bestarm identiﬁcation problem, where querying the original arm is expensive, but multiple and biased approximations (i.e., ﬁdelities) are available at lower costs.

Original: The learner’s goal, in this setting, is to sequentially choose which simulator toquery in order to minimize the total cost, while guaranteeing to identify the optimalarm with high probability.
Revised: The learner’s goal, in this setting, is to sequentially choose which simulator to query in order to minimize the total cost, while guaranteeing to identify the optimal arm with high probability.

Original: We ﬁrst derive a lower bound on the identiﬁcation cost, assuming that the maximum bias of each ﬁdelity is known to the learner.
Revised: We ﬁrst derive a lower bound on the identiﬁcation cost, assuming that the maximum bias of each ﬁdelity is known to the learner.

Original: Then,we propose a novel algorithm, Iterative Imprecise Successive Elimination (IISE),which provably reduces the total cost w.r.t.
Revised: Then, we propose a novel algorithm, Iterative Imprecise Successive Elimination (IISE), which provably reduces the total cost w.r.t.

Original: algorithms that ignore the multi-ﬁdelitystructure and whose cost complexity upper bound mimics the structure of the lowerbound.
Revised: algorithms that ignore the multi-ﬁdelity structure and whose cost complexity upper bound mimics the structure of the lower bound.

Original: Furthermore, we show that the cost complexity of IISE can be furtherreduced when the agent has access to a more ﬁne-grained knowledge of the errorintroduced by the approximators.
Revised: Furthermore, we show that the cost complexity of IISE can be further reduced when the agent has access to a more ﬁne-grained knowledge of the error introduced by the approximators.

Original: Finally, we numerically validate IISE, showingthe beneﬁts of our method in simulated domains.
Revised: Finally, we numerically validate IISE, showing the beneﬁts of our method in simulated domains.

Original:  Introduction
Revised:  Introduction

Original: In the multi-armed bandit setting [26], the agent selects, at each interaction round, an arm andobserves a sample drawn from its reward distribution.
Revised: In the multi-armed bandit setting [26], the agent selects, at each interaction round, an arm and observes a sample drawn from its reward distribution.

Original: Unlike the regret minimization setting [26],where the agent tries to maximize the cumulative return, in best-arm identiﬁcation (BAI) problems[10, 6, 2], the evaluation index relates exclusively to the quality of the arm recommended at theend of the exploration phase.
Revised: Unlike the regret minimization setting [26], where the agent tries to maximize the cumulative return, in best-arm identiﬁcation (BAI) problems [10, 6, 2], the evaluation index relates exclusively to the quality of the arm recommended at the end of the exploration phase.

Original: More speciﬁcally, in the ﬁxed-conﬁdence setting [17], the agent aimsat identifying, with probability 1 − δ , the arm with the largest mean reward, while minimizing thesample complexity, i.e., the number of collected samples necessary to make the ﬁnal decision.
Revised: More speciﬁcally, in the ﬁxed-conﬁdence setting [17], the agent aims at identifying, with probability 1 − δ , the arm with the largest mean reward, while minimizing the sample complexity, i.e., the number of collected samples necessary to make the ﬁnal decision.

Original: Dueto its generality, the BAI framework encompasses a wide spectrum of applications, ranging frommedical trials [38], to hyper-parameter optimization [27], crowdsourcing [47], recommender systems[25], simulation-based studies in physics [43], and A/B testing [14].
Revised: Due to its generality, the BAI framework encompasses a wide spectrum of applications, ranging from medical trials [38], to hyper-parameter optimization [27], crowdsourcing [47], recommender systems [25], simulation-based studies in physics [43], and A/B testing [14].

Original: However, in practical cases, querying an arm to obtain a sample from its reward distribution mightbe expensive.
Revised: However, in practical cases, querying an arm to obtain a sample from its reward distribution might be expensive.

Original: For instance, in the context of physics simulation studies, where pulling an armcorresponds to the evaluation of a complex model under some given arm parameters, intensive useof computing power is required to obtain the reward we are interested in.
Revised: For instance, in the context of physics simulation studies, where pulling an arm corresponds to the evaluation of a complex model under some given arm parameters, intensive use of computing power is required to obtain the reward we are interested in.

Original: Nevertheless, simplerand, consequently, cheaper and biased, models might be available to the agent.
Revised: Nevertheless, simpler and, consequently, cheaper and biased, models might be available to the agent.

Original: Another example isstochastic planning, where the agent needs to compute the best action (i.e., arm) that maximizes thefuture sum of rewards over a maximum horizon H .
Revised: Another example is stochastic planning, where the agent needs to compute the best action (i.e., arm) that maximizes the future sum of rewards over a maximum horizon H .

Original: However, while in principle it is possible to planconsidering the entire horizon, cheaper and biased estimates can be obtained by cutting the depth ofthe search to h < H .
Revised: However, while in principle it is possible to plan considering the entire horizon, cheaper and biased estimates can be obtained by cutting the depth of the search to h < H .

Original: For these scenarios, the multi-ﬁdelity (MF) framework [9, 21, 22] that exploitsthis additional opportunity has recently gained attention in the literature.
Revised: For these scenarios, the multi-ﬁdelity (MF) framework [9, 21, 22] that exploits this additional opportunity has recently gained attention in the literature.

Original: The main rationale behind these methodologies is to exploit less expensive approximations in speciﬁc regions of the arm space tolimit the resources allocated to distinguish options that are far from optimal, and do not require highaccuracy.
Revised: The main rationale behind these methodologies is to exploit less expensive approximations in speciﬁc regions of the arm space to limit the resources allocated to distinguish options that are far from optimal, and do not require high accuracy.

Original: So far, most of the multi-ﬁdelity literature has focussed on black-box function optimization,with the goal of optimizing a target function f when multiple approximations of f are available[15, 33, 22, 34, 39, 40, 19, 28, 11].
Revised: So far, most of the multi-ﬁdelity literature has focussed on black-box function optimization, with the goal of optimizing a target function f when multiple approximations of f are available [15, 33, 22, 34, 39, 40, 19, 28, 11].

Original: Among these works, the ones more related to the bandit literatureare [22, 19, 39, 40, 11]; under different smoothness assumptions, these authors aimed at designingstrategies that explore the arm space to minimize the simple regret , i.e., the difference between thetrue optimal value of f and the best unbiased query made to f during the optimization process.
Revised: Among these works, the ones more related to the bandit literature are [22, 19, 39, 40, 11]; under different smoothness assumptions, these authors aimed at designing strategies that explore the arm space to minimize the simple regret , i.e., the difference between the true optimal value of f and the best unbiased query made to f during the optimization process.

Original: Otherworks studied cumulative regret analysis [21, 20], active learning [46, 32], reinforcement learning[9, 30, 41], and applied approaches [36, 44, 31].
Revised: Other works studied cumulative regret analysis [21, 20], active learning [46, 32], reinforcement learning [9, 30, 41], and applied approaches [36, 44, 31].

Original: In this paper, on the other hand, we focus on the understudied multi-ﬁdelity ﬁxed-conﬁdence BAIsetting.
Revised: In this paper, on the other hand, we focus on the understudied multi-ﬁdelity ﬁxed-conﬁdence BAI setting.

Original: More speciﬁcally, we study the minimization of the cost complexity , a novel metric that weintroduce by extending the notion of sample complexity to the multi-ﬁdelity setting, by weighting eachsample with the related cost.
Revised: More speciﬁcally, we study the minimization of the cost complexity , a novel metric that we introduce by extending the notion of sample complexity to the multi-ﬁdelity setting, by weighting each sample with the related cost.

Original: The goal of the agent is to output the optimal arm with probability 1 − δ ,while minimizing the total cost.
Revised: The goal of the agent is to output the optimal arm with probability 1 − δ , while minimizing the total cost.

Original: To this purpose, as commonly done in the literature [9, 21, 20, 39, 11],we assume that the agent has some knowledge of the quality of the approximator, which is usuallyexpressed in terms of upper bounds on the maximum bias that the ﬁdelities introduce on the arm set.
Revised: To this purpose, as commonly done in the literature [9, 21, 20, 39, 11], we assume that the agent has some knowledge of the quality of the approximator, which is usually expressed in terms of upper bounds on the maximum bias that the ﬁdelities introduce on the arm set.

Original: This term corresponds to the maximum distance between the mean of the desired (but expensive) armdistributions and their approximations.
Revised: This term corresponds to the maximum distance between the mean of the desired (but expensive) arm distributions and their approximations.

Original: Contributions.
Revised: Contributions.

Original: In this paper, we introduce the notion of cost complexity for the multi-ﬁdelity BAIsetting (Section 2).
Revised: In this paper, we introduce the notion of cost complexity for the multi-ﬁdelity BAI setting (Section 2).

Original: We derive a lower bound for this novel index, under the standard assumptionthat the agent is equipped with knowledge of the maximum bias that each ﬁdelity has w.r.t.
Revised: We derive a lower bound for this novel index, under the standard assumption that the agent is equipped with knowledge of the maximum bias that each ﬁdelity has w.r.t.

Original: theoriginal, expensive and unbiased bandit model (Section 3).
Revised: the original, expensive and unbiased bandit model (Section 3).

Original: As we shall show, our result highlightsseveral properties of the problem along with its challenges.
Revised: As we shall show, our result highlights several properties of the problem along with its challenges.

Original: Furthermore, we propose the Iterative
Revised: Furthermore, we propose the Iterative

Original: Imprecise Successive Elimination (IISE) algorithm (Section 4), for which, we derive cost complexityupper bounds that, under certain assumptions, mimic the structure of the lower bound, and that isprovably robust w.r.t.
Revised: Imprecise Successive Elimination (IISE) algorithm (Section 4), for which, we derive cost complexity upper bounds that, under certain assumptions, mimic the structure of the lower bound, and that is provably robust w.r.t.

Original: BAI algorithms that ignore the multi-ﬁdelity structure.
Revised: BAI algorithms that ignore the multi-ﬁdelity structure.

Original: We also show that theperformance of IISE can be further improved when a more ﬁne-grained knowledge of the behavior ofthe approximations is available to the agent (Section 5).
Revised: We also show that the performance of IISE can be further improved when a more ﬁne-grained knowledge of the behavior of the approximations is available to the agent (Section 5).

Original: After revising the literature (Section 6), wepropose numerical experiments on simulated domains that empirically verify our claims (Section 7).
Revised: After revising the literature (Section 6), we propose numerical experiments on simulated domains that empirically verify our claims (Section 7).

Original:  Setting
Revised:  Setting

Original: In the classical ﬁxed-conﬁdence BAI setting [17], the agent interacts with a bandit model ν :=( ν 1 , . ..
Revised: In the classical ﬁxed-conﬁdence BAI setting [17], the agent interacts with a bandit model ν := ( ν 1 , . .

Original: , ν K ) , where K ∈ N is the number of arms and ν
Revised: , ν K ) , where K ∈ N is the number of arms and ν

Original: i denotes the reward distribution associatedwith arm i ∈
Revised: i denotes the reward distribution associated with arm i ∈

Original: [ K ] := { 1 , . . .
Revised: [ K ] := { 1 , . . .

Original: , K } .
Revised: , K } .

Original: Denote with µ i the expectation of ν i and, assume, that the K73 arms are ordered such that µ [Equation]
Revised: Denote with µ i the expectation of ν i and, assume, that the K arms are ordered such that µ 1 > µ 2 ≥ · · · ≥ µ K .

Original: We assume that the reward distributions ν i are74 subgaussian with known scale parameter σ 2 .
Revised: We assume that the reward distributions ν i are subgaussian with known scale parameter σ 2 .

Original: At each interaction round t ∈ N , the agent selects anarm I t ∈
Revised: At each interaction round t ∈ N , the agent selects an arm I t ∈

Original: [ K ] and receives a reward R t sampled from ν I t .
Revised: [ K ] and receives a reward R t sampled from ν I t .

Original: Let T i ( t ) be the total number of pullsof arm
Revised: Let T i ( t ) be the total number of pulls of arm

Original: i ∈
Revised: i ∈

Original: [ K ] up to time t ∈ N .
Revised: [ K ] up to time t ∈ N .

Original: Given a conﬁdence level δ ∈ (0 , 1) , the goal of the agent is torecommend an arm (cid:98) I ( τ ) ∈
Revised: Given a conﬁdence level δ ∈ (0 , 1) , the goal of the agent is to recommend an arm (cid:98) I ( τ ) ∈

Original: [ K ] that corresponds to the optimal arm 1 with probability at least 1 − δ ,while minimizing the sample complexity ρ ( τ ) :
Revised: [ K ] that corresponds to the optimal arm 1 with probability at least 1 − δ , while minimizing the sample complexity ρ ( τ ) :

Original: = (cid:80) i ∈
Revised: = (cid:80) i ∈

Original: [ K ] E [ T i ( τ )] , where τ is the stopping time ofthe agent.
Revised: [ K ] E [ T i ( τ )] , where τ is the stopping time of the agent.

Original: Any algorithm that recommends the best arm with probability 1 − δ is said δ -correct.
Revised: Any algorithm that recommends the best arm with probability 1 − δ is said δ -correct.

Original: Forevery sub-optimal arm i > 1 , we deﬁne its gap w.r.t.
Revised: For every sub-optimal arm i > 1 , we deﬁne its gap w.r.t.

Original: the optimal arm as ∆ i := µ 1 − µ i .
Revised: the optimal arm as ∆ i := µ 1 − µ i .

Original: In this paper, we differ from this classical setting as, for each arm i ∈
Revised: In this paper, we differ from this classical setting as, for each arm i ∈

Original: [ K ] , we assume the agent hasaccess to M − 1 ∈ N ﬁdelity distributions 
Revised: [ K ] , we assume the agent has access to M − 1 ∈ N ﬁdelity distributions ν

Original: ν i, 1 , ν i, 2 , . .
Revised: i, 1 , ν i, 2 , . .

Original: ν i,M − 1 , each σ 2 -subgaussian, with meansµ i, 1 , µ i, 2 , . .
Revised: ν i,M − 1 , each σ 2 -subgaussian, with means µ i, 1 , µ i, 2 , . .

Original: . , µ i,M − 1 .
Revised: . , µ i,M − 1 .

Original: We unify the notation introducing a maximum ﬁdelity M to encode the true84 arm distributions the agent is interested in, i.e., ν i,M :
Revised: We unify the notation introducing a maximum ﬁdelity M to encode the true arm distributions the agent is interested in, i.e., ν i,M :

Original: = ν i with mean µ i,M := µ i .
Revised: = ν i with mean µ i,M := µ i .

Original: As in previous works[9, 21, 20, 39, 11], we assume that there is a relation encoding how much information distributions ofﬁdelity m < M contain w.r.t.
Revised: As in previous works [9, 21, 20, 39, 11], we assume that there is a relation encoding how much information distributions of ﬁdelity m < M contain w.r.t.

Original: to the true arm distribution.
Revised: to the true arm distribution.

Original: More speciﬁcally, for each ﬁdelity m ∈
Revised: More speciﬁcally, for each ﬁdelity m ∈

Original: [ M ] ,it is always possible to write µ i,m := µ i,M + ξ i,m for some appropriate ξ i,m ∈ R .
Revised: [ M ] , it is always possible to write µ i,m := µ i,M + ξ i,m for some appropriate ξ i,m ∈ R .

Original: Then, we assume88 that the agent has access to ξ m ≥ max i ∈
Revised: Then, we assume that the agent has access to ξ m ≥ max i ∈

Original: [ K ] | ξ i,m | , from which follows that | µ i,m − µ i,M | ≤ ξ m .
Revised: [ K ] | ξ i,m | , from which follows that | µ i,m − µ i,M | ≤ ξ m .

Original: We notice that ξ m represents an upper-bound on the maximum bias of ﬁdelity m ∈
Revised: We notice that ξ m represents an upper-bound on the maximum bias of ﬁdelity m ∈

Original: [ M ] .
Revised: [ M ] .

Original: At eachinteraction round t ∈ N , the agent selects an arm I t ∈
Revised: At each interaction round t ∈ N , the agent selects an arm I t ∈

Original: [ K ] , a ﬁdelity m t ∈
Revised: [ K ] , a ﬁdelity m t ∈

Original: [ M ] , and observes a reward
Revised: [ M ] , and observes a reward

Original: R t , drawn from ν I t ,m t .
Revised: R t , drawn from ν I t ,m t .

Original: The cost of gathering samples at ﬁdelity m is known and speciﬁed by λ m > 0 .
Revised: The cost of gathering samples at ﬁdelity m is known and speciﬁed by λ m > 0 .

Original: We make the standard assumption that λ 1 < λ 2 < · · · < λ M and ξ 1 > ξ 2 > · · · > ξ M := 0 .
Revised: We make the standard assumption that λ 1 < λ 2 < · · · < λ M and ξ 1 > ξ 2 > · · · > ξ M := 0 .

Original: At this93 point, let T i,m ( t ) be the number of pulls to arm
Revised: At this point, let T i,m ( t ) be the number of pulls to arm

Original: i ∈
Revised: i ∈

Original: [ K ] at ﬁdelity m ∈
Revised: [ K ] at ﬁdelity m ∈

Original: [ M ] up to time t ∈ N ; we94 deﬁne the cost complexity as:
Revised: [ M ] up to time t ∈ N ; we deﬁne the cost complexity as:

Original: [Equation]
Revised: [Equation]

Original: where τ is the stopping time at which the agent will output the optimal arm 1 with probability 1 − δ .
Revised: where τ is the stopping time at which the agent will output the optimal arm 1 with probability 1 − δ .

Original:  Cost Complexity Lower Bound
Revised:  Cost Complexity Lower Bound

Original: In this section, we discuss the intrinsic complexity of the multi-ﬁdelity BAI problem, by providingand analyzing a lower bound on the cost complexity (proof in Appendix B).
Revised: In this section, we discuss the intrinsic complexity of the multi-ﬁdelity BAI problem, by providing and analyzing a lower bound on the cost complexity (proof in Appendix B).

Original: Theorem 1.
Revised: Theorem 1.

Original: Consider a multi-ﬁdelity bandit model ν with Gaussian distributions
Revised: Consider a multi-ﬁdelity bandit model ν with Gaussian distributions

Original: ν i,m = N ( µ i,m , σ 2 )100 such that | µ i,m − µ i,M | ≤ ξ m for every i ∈
Revised: ν i,m = N ( µ i,m , σ 2 ) such that | µ i,m − µ i,M | ≤ ξ m for every i ∈

Original: [ K ] and m ∈ [ M ] .
Revised: [ K ] and m ∈ [ M ] .

Original: Then, for any δ -correct algorithm and101 δ ≤ 0 . 15 , it holds that:
Revised: Then, for any δ -correct algorithm and δ ≤ 0 . 15 , it holds that:

Original: [Equation]
Revised: [Equation]

Original: where KL( p, q ) is the Kullback-Leibler divergence between distributions p and q , ν 2 ,m = N ( µ 2 ,M +ξ m , σ 2 ) , ν 1 ,m = N ( µ 1 ,M − ξ m , σ 2 ) , M 1 :
Revised: where KL( p, q ) is the Kullback-Leibler divergence between distributions p and q , ν 2 ,m = N ( µ 2 ,M + ξ m , σ 2 ) , ν 1 ,m = N ( µ 1 ,M − ξ m , σ 2 ) , M 1 :

Original: = { m ∈
Revised: = { m ∈

Original: [ M ] : µ 1 ,m > E x ∼ ν 2 ,m [ x ] } and M i := { m ∈[ M ] : 104 E [Figure]
Revised: [ M ] : µ 1 ,m > E x ∼ ν 2 ,m [ x ] } and M i := { m ∈

Original: [Equation] [Figure] ,m [ x ] > µ i,m } for i > 1 .
Revised: [ M ] : E x ∼ ν 1 ,m [ x ] > µ i,m } for i > 1 .

Original: This bound reveals several properties of the multi-ﬁdelity ﬁxed-conﬁdence BAI problem.
Revised: This bound reveals several properties of the multi-ﬁdelity ﬁxed-conﬁdence BAI problem.

Original: A singleterm λ m KL − 1 ( ν i,m , ν 1 ,m ) with i (cid:54) = 1 can be interpreted as a lower bound on the minimum costthat any δ -correct algorithm needs to pay to conclude that arm i is sub-optimal.
Revised: A single term λ m KL − 1 ( ν i,m , ν 1 ,m ) with i (cid:54) = 1 can be interpreted as a lower bound on the minimum cost that any δ -correct algorithm needs to pay to conclude that arm i is sub-optimal.

Original: Suppose, for themoment, that M i contains any ﬁdelity m ∈
Revised: Suppose, for the moment, that M i contains any ﬁdelity m ∈

Original: [ M ] .
Revised: [ M ] .

Original: Then, once we ﬁx m ∈ M i , its related componentλ m KL − 1 ( ν i,m , ν 1 ,m ) is the cost-based multi-ﬁdelity counterpart of the usual KL − 1 ( ν i , ν 1 ) whichappears in the standard BAI lower bound [24].
Revised: Then, once we ﬁx m ∈ M i , its related component λ m KL − 1 ( ν i,m , ν 1 ,m ) is the cost-based multi-ﬁdelity counterpart of the usual KL − 1 ( ν i , ν 1 ) which appears in the standard BAI lower bound [24].

Original: This can be seen as a lower bound on the cost todiscard arm i using samples gathered at ﬁdelity m .
Revised: This can be seen as a lower bound on the cost to discard arm i using samples gathered at ﬁdelity m .

Original: Notice that, in this sense, the term ν 1 ,m replacesthe usual ν 1 to take the bias of ﬁdelity m into account.
Revised: Notice that, in this sense, the term ν 1 ,m replaces the usual ν 1 to take the bias of ﬁdelity m into account.

Original: The minimum, consequently, encodes a lowerbound on the cost of the most convenient ﬁdelity with which we can discard i .
Revised: The minimum, consequently, encodes a lower bound on the cost of the most convenient ﬁdelity with which we can discard i .

Original: The reason why wehave a minimum over M i instead of [ M ] is that, given a ﬁdelity m ∈
Revised: The reason why we have a minimum over M i instead of [ M ] is that, given a ﬁdelity m ∈

Original: [ M ] , m might not be useful todiscard i , since ∆ i might be too small for the bias that m introduces.
Revised: [ M ] , m might not be useful to discard i , since ∆ i might be too small for the bias that m introduces.

Original: This concept is further clariﬁedin the following remark:
Revised: This concept is further clariﬁed in the following remark:

Original: Remark 1.
Revised: Remark 1.

Original: Consider the following problem instance: for each m ∈
Revised: Consider the following problem instance: for each m ∈

Original: [ M ] , let µ 1 ,m = µ 1 ,M − ξ m andµ i,m = µ i,M + ξ m for i (cid:54)
Revised: [ M ] , let µ 1 ,m = µ 1 ,M − ξ m and µ i,m = µ i,M + ξ m for i (cid:54)

Original: = 1 .
Revised: = 1 .

Original: Then, Theroem 1 reduces to:
Revised: Then, Theroem 1 reduces to:

Original: [Equation]
Revised: [Equation]

Original: As we can see, if ∆ i < 2 ξ m for some m , its related component is not present in the lower bound.
Revised: As we can see, if ∆ i < 2 ξ m for some m , its related component is not present in the lower bound.

Original: Furthermore, it is interesting to note that the setting of Remark 1 represents a worst-case ﬁdelitydependent scenario.
Revised: Furthermore, it is interesting to note that the setting of Remark 1 represents a worst-case ﬁdelitydependent scenario.

Original: Indeed, each ﬁdelity m ∈
Revised: Indeed, each ﬁdelity m ∈

Original: [ M − 1] underestimates the reward of the optimal armand overestimates the ones of sub-optimal arms.
Revised: [ M − 1] underestimates the reward of the optimal arm and overestimates the ones of sub-optimal arms.

Original: Finally, we show how Theorem 1 highlights the main challenge of our setting.
Revised: Finally, we show how Theorem 1 highlights the main challenge of our setting.

Original: For each value of ξ m ,there exists a multi-ﬁdelity bandit model ν for which the lower bound is given by (cid:101) Ω((cid:80)
Revised: For each value of ξ m , there exists a multi-ﬁdelity bandit model ν for which the lower bound is given by (cid:101) Ω(

Original: Ki =2 λ M σ 2 ∆ 2 i )(given Remark 1, it is sufﬁcient to consider ∆ i such that the only ﬁdelity in the minimization set is
Revised: [Equation] ) (given Remark 1, it is sufﬁcient to consider ∆ i such that the only ﬁdelity in the minimization set is

Original: M ).
Revised: M ).

Original: It is clear that this cost complexity can be matched (up to log factors) by many BAI algorithmsavailable in the literature (e.g., [10]) that ignore the multi-ﬁdelity structure of the problem. 
Revised: It is clear that this cost complexity can be matched (up to log factors) by many BAI algorithms available in the literature (e.g., [10]) that ignore the multi-ﬁdelity structure of the problem. The main [Figure]

Original: The mainissue, in this sense, is the fact that the agent does not know, a priori, the values of ∆ i .
Revised: x x issue, in this sense, is the fact that the agent does not know, a priori, the values of ∆ i .

Original: To build analgorithm that achieves robustness (i.e., not being worse than these uninformed strategies), then, we need to control how much we spend to explore the arm distributions at ﬁdelity m < M .
Revised: To build an algorithm that achieves robustness (i.e., not being worse than these uninformed strategies), then, we need to control how much we spend to explore the arm distributions at ﬁdelity m < M .

Original: Moregenerally, this reasoning also extends to building algorithms that match the lower bound.
Revised: More generally, this reasoning also extends to building algorithms that match the lower bound.

Original: In thissense, we observe, even for the ﬁxed-conﬁdence BAI setting, the adversarial behavior that ﬁdelitymight have, as previously reported in [21] for the ﬁnite-armed bandit regret minimization setting.
Revised: In this sense, we observe, even for the ﬁxed-conﬁdence BAI setting, the adversarial behavior that ﬁdelity might have, as previously reported in [21] for the ﬁnite-armed bandit regret minimization setting.

Original:  Imprecise and Iterative Successive Elimination
Revised:  Imprecise and Iterative Successive Elimination

Original: In this section, we present our algorithm, Imprecise and Iterative Successive Elimination (IISE,
Revised: In this section, we present our algorithm, Imprecise and Iterative Successive Elimination (IISE,

Original: Algorithm 1), to solve multi-ﬁdelity ﬁxed-conﬁdence BAI problems.
Revised: Algorithm ?? ), to solve multi-ﬁdelity ﬁxed-conﬁdence BAI problems.

Original: In particular, while developingour method, we pursue the following two objectives.
Revised: In particular, while developing our method, we pursue the following two objectives.

Original: First, we want to guarantee robustness w.r.t.uninformed strategies that blindly use only the highest ﬁdelity M to identify the best arm.
Revised: First, we want to guarantee robustness w.r.t. uninformed strategies that blindly use only the highest ﬁdelity M to identify the best arm.

Original: Indeed,we expect a good agent to be able to reduce the total cost by exploiting the peculiar multi-ﬁdelitystructure of the problem.
Revised: Indeed, we expect a good agent to be able to reduce the total cost by exploiting the peculiar multi-ﬁdelity structure of the problem.

Original: Second, we aim at providing a strategy whose upper-bound structureresembles, as close as possible, the one of the lower bound presented in Section 3.
Revised: Second, we aim at providing a strategy whose upper-bound structure resembles, as close as possible, the one of the lower bound presented in Section 3.

Original: Our solution (pseudo-code in Algorithm  1) builds on the Successive Elimination algorithm [10].
Revised: Our solution (pseudo-code in Algorithm 1 and visual representation in Figure 1) builds on the Successive Elimination algorithm [10].

Original: More speciﬁcally, we proceed by epochs, one for each ﬁdelity m ∈
Revised: More speciﬁcally, we proceed by epochs, one for each ﬁdelity m ∈

Original: [ M ] , considered in increasingorder.
Revised: [ M ] , considered in increasing order.

Original: In each phase m ∈ [ M ] , we query all the active arms, i.e., those that have not been discardedyet, through their approximated distributions ν i,m at ﬁdelity m .
Revised: In each phase m ∈ [ M ] , we query all the active arms, i.e., those that have not been discarded yet, through their approximated distributions ν i,m at ﬁdelity m .

Original: We denote the set of active arms with S ⊆
Revised: We denote the set of active arms with S ⊆

Original: [ K ] .
Revised: [ K ] .

Original: IISE keeps usual empirical mean estimates ˆ µ i,m,t of the mean µ i,m of reward distributionsν i,m for each i ∈ S , and discards all the arms i for which the following applies:
Revised: IISE keeps usual empirical mean estimates ˆ µ i,m,t of the mean µ i,m of reward distributions ν i,m for each i ∈ S , and discards all the arms i for which the following applies:

Original: [Equation]
Revised: [Equation]

Original: , (3) where U ( t, δ, ξ ) is a symmetric bound on the empirical means (deﬁned below).
Revised: where U ( t, δ, ξ ) is a symmetric bound on the empirical means (deﬁned below).

Original: Condition (3)aims at eliminating all the active arms i whose estimated empirical mean upper bound falls belowthe empirical mean lower bound of another active arm j .
Revised: Condition (3) aims at eliminating all the active arms i whose estimated empirical mean upper bound falls below the empirical mean lower bound of another active arm j .

Original: More speciﬁcally, we set U ( t, δ, ξ ) :=
Revised: More speciﬁcally, we set U ( t, δ, ξ ) :=

Original: B ( t, δ ) + ξ = (cid:113) 2 σ 2 log 4 t 2 δ t + ξ .
Revised: B ( t, δ ) + ξ = (cid:113) 2 σ 2 log 4 t 2 δ t + ξ .

Original: As we can see, U ( t, δ, ξ ) is composed of the usual concentrationterm B ( t, δ ) plus the ﬁdelity bias ξ .
Revised: As we can see, U ( t, δ, ξ ) is composed of the usual concentration term B ( t, δ ) plus the ﬁdelity bias ξ .

Revised: As our proof highlights, this additional term is required to

Original: As our proof highlights, this additional term is required toensure that the optimal arm remains, with high probability, in the active set S until termination.
Revised: Require: Multi-ﬁdelity bandit model ν , conﬁdence δ , thresholds { α m } Mm =1 , bounds { ξ m } Mm = 1: S ← [ K ] 2: m ← 1 and t ←3: while | S | > 1 do 4: if α m ≥ 4 B (cid:0) t, δKM (cid:1) then 5: m ← m + 1 and t ←6: end if 7: Pull all arms in S at ﬁdelity m and t ← t +8: Update ˆ µ j,m,t for all j ∈ [ S ] 9: S ← S \ (cid:8) i ∈ S : ∃ j ∈ S : ˆ µ j,m,t − U (cid:0) t, δKM , ξ m (cid:1) ≥ ˆ µ i,m,t + U (cid:0) t, δKM , ξ m (cid:1)(cid:9) 10: end while 11: return S ensure that the optimal arm remains, with high probability, in the active set S until termination.

Original: 2 Theduration of each epoch (i.e., how much IISE tries to exploit ﬁdelity m ) is determined by a thresholdparameter α m (to be deﬁned later).
Revised: The duration of each epoch (i.e., how much IISE tries to exploit ﬁdelity m ) is determined by a threshold parameter α m (to be deﬁned later).

Original: At each round, the value of B (cid:0) t, δKM (cid:1) decreases and as soonas 4 B (cid:0) t, δKM (cid:1) drops below α m , IISE switches to the next ﬁdelity m + 1 .
Revised: At each round, the value of B (cid:0) t, δKM (cid:1) decreases and as soon as 4 B (cid:0) t, δKM (cid:1) drops below α m , IISE switches to the next ﬁdelity m + 1 .

Original: Notice that, to achieveδ -correctness, it is required to set α M to 0 , so that the last phase is guaranteed to identify the optimalarm, and α m > 0 for m < M , so that each phase is assured to terminate in a ﬁnite number of rounds.
Revised: Notice that, to achieve δ -correctness, it is required to set α M to 0 , so that the last phase is guaranteed to identify the optimal arm, and α m > 0 for m < M , so that each phase is assured to terminate in a ﬁnite number of rounds.

Original: Then, keeping generic thresholds α m that satisfy the previous conditions, we can show that IISE isδ -correct and its cost complexity is upper bounded by the following theorem (proof in AppendixC).
Revised: Then, keeping generic thresholds α m that satisfy the previous conditions, we can show that IISE is δ -correct and its cost complexity is upper bounded by the following theorem (proof in AppendixC).

Original: Theorem 2.
Revised: Theorem 2.

Original: If α m > 0 for every m ∈ [ M − 1] and α M = 0 , then, with probability at least 1 − δ , IISE162 returns the optimal arm 1 with cost complexity c ( τ ) upper bounded by:
Revised: If α m > 0 for every m ∈ [ M − 1] and α M = 0 , then, with probability at least 1 − δ , IISE returns the optimal arm 1 with cost complexity c ( τ ) upper bounded by:

Original: [Equation]
Revised: [Equation]

Original: where m i is the smallest m ∈
Revised: where m i is the smallest m ∈

Original: [ M ] for which ∆ i ≥ 4 ξ m + α m holds.
Revised: [ M ] for which ∆ i ≥ 4 ξ m + α m holds.

Original: We notice that the cost required by IISE to discard i (cid:54) = 1 is composed of two terms.
Revised: We notice that the cost required by IISE to discard i (cid:54) = 1 is composed of two terms.

Original: The ﬁrst one isthe cost of discarding arm i using samples at ﬁdelity m i , while the second one is the cost of trying todiscard arm at lower ﬁdelity m < m i .
Revised: The ﬁrst one is the cost of discarding arm i using samples at ﬁdelity m i , while the second one is the cost of trying to discard arm at lower ﬁdelity m < m i .

Original: As our proof reveals, m i is the smallest ﬁdelity m for which
Revised: As our proof reveals, m i is the smallest ﬁdelity m for which

Original: IISE ensures that arm i will be discarded.
Revised: IISE ensures that arm i will be discarded.

Original: 4.1 On the Choice of the Thresholds α m
Revised: 4.1 On the Choice of the Thresholds α m

Original: In this section, we highlight one of the main challenges of the MF setting, as mentioned in Section 3.
Revised: In this section, we highlight one of the main challenges of the MF setting, as mentioned in Section 3.

Original: Since we are unaware of the suboptimality gaps ∆ i , we may attempt to choose α m so as to minimize Require: Multi-ﬁdelity bandit model ν , conﬁdence δ , thresholds { α m } Mm =1 , bounds { ξ m } Mm = S ← [ K ] ←
Revised: Since we are unaware of the suboptimality gaps ∆ i , we may attempt to choose α m so as to minimize the cost complexity of Theorem 2 w.r.t.

Original: ←3:

Original: > do 4: if then 5: if 7:

Original: Pull

Original: Update ˆ t

Original: [ S ] 9:

Original: S ← S \ (cid:8) i ∈ S : ∃ j ∈ S : ˆ µ j,m,t − U (cid:0) t, δKM , ξ m (cid:1) ≥ ˆ µ i,m,t + U (cid:0) t, δKM , ξ m (cid:1)(cid:9) 10: end while 11: return S the cost complexity of Theorem 2 w.r.t.

Original: the worst-case choice of suboptimality gaps ∆ i , that is: 
Revised: the worst-case choice of suboptimality gaps ∆ i , that is: min

Original: [Equation]
Revised: [Equation]

Original: where α = ( α 1 , . . . , α M ) and ∆ = (∆ 2 , . .
Revised: where α = ( α 1 , . . . , α M ) and ∆ = (∆ 2 , . .

Original: . , ∆ K ) are the threshold and gap vectors.
Revised: . , ∆ K ) are the threshold and gap vectors.

Original: However, itis easy to verify that Equation (4) is minimized for α m = + ∞ for all m < M and α M = 0 , i.e., analgorithm that exploits the highest ﬁdelity M only.
Revised: However, it is easy to verify that Equation (4) is minimized for α m = + ∞ for all m < M and α M = 0 , i.e., an algorithm that exploits the highest ﬁdelity M only.

Original: As we can see, IISE reﬂects the peculiar issueof the MF-BAI lower bound.
Revised: As we can see, IISE reﬂects the peculiar issue of the MF-BAI lower bound.

Original: In the worst case, any ﬁdelity m <
Revised: In the worst case, any ﬁdelity m <

Original: M is not useful to reduce theidentiﬁcation cost since it introduces too much bias w.r.t.
Revised: M is not useful to reduce the identiﬁcation cost since it introduces too much bias w.r.t.

Original: ∆ i , regardless its cost.
Revised: ∆ i , regardless its cost.

Original: For this reason, asdone in previous works [21], we restrict the class of problems with some assumptions whose goal ismaking the ﬁdelity structure provably convenient.
Revised: For this reason, as done in previous works [21], we restrict the class of problems with some assumptions whose goal is making the ﬁdelity structure provably convenient.

Original: More speciﬁcally:
Revised: More speciﬁcally:

Original: Assumption 1 (Costs and Biases Relationship) .
Revised: Assumption 1 (Costs and Biases Relationship) .

Original: For every ﬁdelity m ∈ [ M − 1] , it holds that:
Revised: For every ﬁdelity m ∈ [ M − 1] , it holds that:

Original: [Equation]
Revised: [Equation]

Original: Assumption 1 deserves some comments.
Revised: Assumption 1 deserves some comments.

Original: First of all, it directly encodes a relationship between costsand biases.
Revised: First of all, it directly encodes a relationship between costs and biases.

Original: To provide an interpretation, we notice that the assumption is always satisﬁed for: (i) coststhat grow at least linearly 3 and (ii) ξ ’s that satisfy the following decay rate: (cid:80) m<m
Revised: To provide an interpretation, we notice that the assumption is always satisﬁed for: (i) costs that grow at least linearly 3 and (ii) ξ ’s that satisfy the following decay rate: (cid:80) m<m

Original: [Equation]
Revised: [Equation]

Original: The intuition behind the cost increase rate (i) is that, with sub-linear growth, attempting to discardarms with a cheaper ﬁdelity is too inconvenient w.r.t.
Revised: The intuition behind the cost increase rate (i) is that, with sub-linear growth, attempting to discard arms with a cheaper ﬁdelity is too inconvenient w.r.t.

Original: gathering more accurate samples for a slightlyhigher cost.
Revised: gathering more accurate samples for a slightly higher cost.

Original: For what concerns the ξ ’s decay rate, instead, condition (ii) limits the adversarial behaviorthat ﬁdelity might have.
Revised: For what concerns the ξ ’s decay rate, instead, condition (ii) limits the adversarial behavior that ﬁdelity might have.

Original: If two subsequent ξ ’s are very close together, little can be inferred fromswitching from the ﬁrst ﬁdelity to the second one.
Revised: If two subsequent ξ ’s are very close together, little can be inferred from switching from the ﬁrst ﬁdelity to the second one.

Original: This sort of problem was already highlighted in[21].
Revised: This sort of problem was already highlighted in [21].

Revised: We remark that Assumption 1, compared to Assumption 1 of [21] (cid:80) m<m

Original: We remark that Assumption 1, compared to Assumption 1 of [21] (cid:80) m<m 1 ( ξ m ) 2 ≤ 1 ξ 2 m , involvesboth costs and biases, which is something we expect since the main challenge of the setting is totrade-off between them.
Revised: [Equation] [Table] [Equation] m , involves both costs and biases, which is something we expect since the main challenge of the setting is to trade-off between them.

Original: Indeed, depending on the values of λ ’s it is possible to tolerate ξ ’s that decaylinearly; vice-versa, if ξ ’s decay fast enough, it is possible to tolerate costs that increase sub-linearly.
Revised: Indeed, depending on the values of λ ’s it is possible to tolerate ξ ’s that decay linearly; vice-versa, if ξ ’s decay fast enough, it is possible to tolerate costs that increase sub-linearly.

Original: Although Equation (5) might appear abstract, we show that it holds for a large class problems.
Revised: Although Equation (5) might appear abstract, we show that it holds for a large class problems.

Original: Consider a Markov Decision Process (MDP) [35] ( Y , A , p, r, η ) , where Y is the state space, A is the194 action space, p = { p h } h ≥ 1 a set of transitions kernels, r = { r h } h ≥ 1 a set of reward functions, and195 η ∈ [0 , 1] is a discount factor.
Revised: Consider a Markov Decision Process (MDP) [35] ( Y , A , p, r, η ) , where Y is the state space, A is the action space, p = { p h } h ≥ 1 a set of transitions kernels, r = { r h } h ≥ 1 a set of reward functions, and η ∈ [0 , 1] is a discount factor.

Original: When an agent takes action a in state y at step h it transitions to y (cid:48) withprobability p h ( y (cid:48)
Revised: When an agent takes action a in state y at step h it transitions to y (cid:48) with probability p h ( y (cid:48)

Original: | y, a ) , and receives a deterministic reward r h ( y, a ) ∈ [0 , 1] .
Revised: | y, a ) , and receives a deterministic reward r h ( y, a ) ∈ [0 , 1] .

Original: Given a state y ∈ Y , our197 goal is to ﬁnd the action that maximizes the cumulative discounted reward over the next H steps[-
-]
Revised: Given a state y ∈ Y , our goal is to ﬁnd the action that maximizes the cumulative discounted reward over the next H steps[-
-]

Original: Given an initial action, a way to compute a sample from its future cumulative discounted reward isto apply a Monte Carlo search algorithm (e.g., depth ﬁrst search) truncating the planning depth ath < H to obtain a cheaper but biased estimate of the cumulative discounted reward.
Revised: Given an initial action, a way to compute a sample from its future cumulative discounted reward is to apply a Monte Carlo search algorithm (e.g., depth ﬁrst search) truncating the planning depth at h < H to obtain a cheaper but biased estimate of the cumulative discounted reward.

Original: In this sense,this scenario ﬁts the ﬁxed-conﬁdence MF-BAI setting.
Revised: In this sense, this scenario ﬁts the ﬁxed-conﬁdence MF-BAI setting.

Original: Let M be the maximum depth, then, for eachdepth m ∈
Revised: Let M be the maximum depth, then, for each depth m ∈

Original: [ M ] , we set as λ m the number of generated nodes, that, assuming a constant branch factor Λ , is exponential in m (i.e., λ m = Λ m ).
Revised: [ M ] , we set as λ m the number of generated nodes, that, assuming a constant branch factor Λ , is exponential in m (i.e., λ m = Λ m ).

Original: Instead, for ξ m it is easy to obtain the following upper boundson the maximum bias ξ m = (
Revised: Instead, for ξ m it is easy to obtain the following upper bounds on the maximum bias ξ m = (

Original: η m +1 − η M +1 ) / (1 − η ) .
Revised: η m +1 − η M +1 ) / (1 − η ) .

Original: Now, it is possible to show that Assumption 1holds (proof in Appendix C).
Revised: Now, it is possible to show that Assumptionholds (proof in Appendix C).

Original: Proposition 1.
Revised: Proposition 1.

Original: Consider Λ ≥ 2 .
Revised: Consider Λ ≥ 2 .

Original: If λ m = Λ m and ξ m =
Revised: If λ m = Λ m and ξ m =

Original: ( η m +1 − η M +1 ) / (1 − η ) for all m ∈ [ M ] ,then Assumption 1 holds.
Revised: ( η m +1 − η M +1 ) / (1 − η ) for all m ∈ [ M ] , then Assumption 1 holds.

Original: 4.2 Most Convenient Fidelity Thresholds
Revised: 4.2 Most Convenient Fidelity Thresholds

Original: Given the structure of Theorem 1 and its interpretation, during a phase m ∈ [ M ] , we expect a goodchoice of thresholds α to let IISE discard all the arms i (cid:54) = 1 that should (and can) be discarded at m .
Revised: Given the structure of Theorem 1 and its interpretation, during a phase m ∈ [ M ] , we expect a good choice of thresholds α to let IISE discard all the arms i (cid:54) = 1 that should (and can) be discarded at m .

Original: Therefore, during m , we aim at concluding that a given arm is suboptimal only if there is no otherﬁdelity m > m in which gathering samples at m is more convenient, in term of identiﬁcation cost,w.r.t.
Revised: Therefore, during m , we aim at concluding that a given arm is suboptimal only if there is no other ﬁdelity m > m in which gathering samples at m is more convenient, in term of identiﬁcation cost, w.r.t.

Original: m .
Revised: m .

Original: We formalize the previous concept in the following deﬁnition.
Revised: We formalize the previous concept in the following deﬁnition.

Original: Deﬁnition 1 (Fidelity Convenience) .
Revised: Deﬁnition 1 (Fidelity Convenience) .

Original: Consider a sub-optimal arm i (cid:54)
Revised: Consider a sub-optimal arm i (cid:54)

Original: = 1 and a ﬁdelity m < M suchthat ∆ i > 4 ξ m .
Revised: = 1 and a ﬁdelity m < M such that ∆ i > 4 ξ m .

Original: We say that ﬁdelity m is convenient for arm i if:
Revised: We say that ﬁdelity m is convenient for arm i if:

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: In Equation (6), we are comparing the cost of discarding i gathering samples at phase m only withthe cost of discarding i using a more precise ﬁdelity m > m .
Revised: In Equation (6), we are comparing the cost of discarding i gathering samples at phase m only with the cost of discarding i using a more precise ﬁdelity m > m .

Original: Therefore, solving it for ∆ i , we identifythe gaps of the arms that can be conveniently discarded at ﬁdelity m rather than at m :
Revised: Therefore, solving it for ∆ i , we identify the gaps of the arms that can be conveniently discarded at ﬁdelity m rather than at m :

Original: [Equation]
Revised: [Equation]

Original: To discard all i that satisfy Equation (7), it is sufﬁcient to set thresholds α m for m < M as follows:
Revised: To discard all i that satisfy Equation (7), it is sufﬁcient to set thresholds α m for m < M as follows:

Original: [Equation]
Revised: [Equation]

Original: At this point, under Assumption 1, it is possible to prove that, with this peculiar threshold choice, thecost complexity of IISE can be rewritten in the following way (proof in Appendix C).
Revised: At this point, under Assumption 1, it is possible to prove that, with this peculiar threshold choice, the cost complexity of IISE can be rewritten in the following way (proof in Appendix C).

Original: Theorem 3.
Revised: Theorem 3.

Original: Under Assumption 1, selecting the thresholds α m as in Equation (8) , with probabilityat least 1 − δ , IISE returns the optimal arm with cost complexity c ( τ ) upper bounder by:
Revised: Under Assumption 1, selecting the thresholds α m as in Equation (8) , with probability at least 1 − δ , IISE returns the optimal arm with cost complexity c ( τ ) upper bounder by:

Original: [Equation]
Revised: [Equation]

Original: As we can appreciate, this result comes with two desired properties.
Revised: As we can appreciate, this result comes with two desired properties.

Original: First of all, IISE is provablyrobust w.r.t.
Revised: First of all, IISE is provably robust w.r.t.

Original: algorithms that ignore the multi-ﬁdelity structure of the problem.
Revised: algorithms that ignore the multi-ﬁdelity structure of the problem.

Original: We remark that tohave this nice property, Assumption 1 does not need to hold for each m ∈
Revised: We remark that to have this nice property, Assumption 1 does not need to hold for each m ∈

Original: [ M ] .
Revised: [ M ] .

Original: It is always possibleto pre-select a subset of ﬁdelity for which the assumption holds.
Revised: It is always possible to pre-select a subset of ﬁdelity for which the assumption holds.

Original: Secondly, the bound in Theorem3 closely mimics the behavior of the lower bound (Equation 2).
Revised: Secondly, the bound in Theorem 3 closely mimics the behavior of the lower bound (Equation 2).

Original: The only difference stands in thefact that 2 ξ m is now replaced with 4 ξ m .
Revised: The only difference stands in the fact that 2 ξ m is now replaced with 4 ξ m .

Original: We leave this gap to be ﬁlled in for future work.
Revised: We leave this gap to be ﬁlled in for future work.

Original: Notice,again, that the similarity also holds for the case in which we select a subset of ﬁdelity for which theassumption holds.
Revised: Notice, again, that the similarity also holds for the case in which we select a subset of ﬁdelity for which the assumption holds.

Original: In this case, however, our minimization will be on a restricted ﬁdelity set w.r.t.
Revised: In this case, however, our minimization will be on a restricted ﬁdelity set w.r.t.

Original: theoriginal one.
Revised: the original one.

Original: 4.3 Nearly-Optimal Identiﬁcation
Revised: 4.3 Nearly-Optimal Identiﬁcation

Original: As for the Successive Elimination algorithm, one can easily modify IISE so that it returns an (cid:15) -optimalarm.
Revised: As for the Successive Elimination algorithm, one can easily modify IISE so that it returns an (cid:15) -optimal arm.

Original: In this case, we can show that the identiﬁcation cost is upper bounded by (proof in Appendix C):
Revised: In this case, we can show that the identiﬁcation cost is upper bounded by (proof in Appendix C):

Original: [Equation]
Revised: [Equation]

Original: where K (cid:15) :
Revised: where K (cid:15) :

Original: = { i ∈
Revised: = { i ∈

Original: [ K ] : µ i > µ 1 − (cid:15) } .
Revised: [ K ] : µ i > µ 1 − (cid:15) } .

Original: As usual, for arms that are not (cid:15) -optimal we pay the original cost237 of Theorem 3.
Revised: As usual, for arms that are not (cid:15) -optimal we pay the original cost of Theorem 3.

Original: For the ones in
Revised: For the ones in

Original: K (cid:15) , instead, their cost is actually clipped by (cid:15) .
Revised: K (cid:15) , instead, their cost is actually clipped by (cid:15) .

Original: In multi-ﬁdelity, this has the further implication that, depending on the values of biases, costs and required accuracy, wemight never query the most expensive unbiased ﬁdelity M .
Revised: In multi-ﬁdelity, this has the further implication that, depending on the values of biases, costs and required accuracy, we might never query the most expensive unbiased ﬁdelity M .

Original: In particular, let us focus on a two-armedbandit problem with two ﬁdelity values, and consider ∆ 2 < (cid:15) .
Revised: In particular, let us focus on a two-armed bandit problem with two ﬁdelity values, and consider ∆ 2 < (cid:15) .

Original: Then, we will never query M if:
Revised: Then, we will never query M if:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term guarantees that IISE can discard i using m , the second one, instead encodes theconvenience of m w.r.t.
Revised: The ﬁrst term guarantees that IISE can discard i using m , the second one, instead encodes the convenience of m w.r.t.

Original: to M in terms of costs.
Revised: to M in terms of costs.

Original: As λ M increases, this term shrinks to zero.
Revised: As λ M increases, this term shrinks to zero.

Original: Whenλ M → + ∞ , the condition reduces to (cid:15) > 4 ξ m .
Revised: When λ M → + ∞ , the condition reduces to (cid:15) > 4 ξ m .

Original: This means that if ﬁdelity m is precise enough for thedesired accuracy (cid:15) , we completely neglect ﬁdelity M .
Revised: This means that if ﬁdelity m is precise enough for the desired accuracy (cid:15) , we completely neglect ﬁdelity M .

Original:  Breaking the Cost Barrier
Revised:  Breaking the Cost Barrier

Original: We now show that (a slightly modiﬁed version of) IISE can further reduce the cost complexity whenmore ﬁne-grained knowledge on the type of error that approximators introduce is available.
Revised: We now show that (a slightly modiﬁed version of) IISE can further reduce the cost complexity when more ﬁne-grained knowledge on the type of error that approximators introduce is available.

Original: The mainintuition behind the following reasoning is that what matters when exploiting ﬁdelity m <
Revised: The main intuition behind the following reasoning is that what matters when exploiting ﬁdelity m <

Original: M is notthe bias they introduce, but whether the arms do not preserve the order w.r.t.
Revised: M is not the bias they introduce, but whether the arms do not preserve the order w.r.t.

Original: the mean of the truereward distributions ν i,M .
Revised: the mean of the true reward distributions ν i,M .

Original: Indeed suppose to have access to a ﬁdelity with a very large bias but whichfully preserves the order on the means of the arms.
Revised: Indeed suppose to have access to a ﬁdelity with a very large bias but which fully preserves the order on the means of the arms.

Original: In this scenario, it is possible to run a standard
Revised: In this scenario, it is possible to run a standard

Original: BAI technique on this ﬁdelity only and still obtain the optimal arm.
Revised: BAI technique on this ﬁdelity only and still obtain the optimal arm.

Original: We now formalize how much acertain ﬁdelity m preserves the order with the novel concept of maximum bias variation γ m :
Revised: We now formalize how much a certain ﬁdelity m preserves the order with the novel concept of maximum bias variation γ m :

Original: [Equation]
Revised: [Equation]

Original: Under the assumption that γ 1 ≥ γ 2 ≥ · · · ≥ γ M := 0 are available to the learner, IISE can be conve-255 niently modiﬁed to obtain the following bound on the cost complexity (proofs, pseudocode, andformal statement in Appendix D):
Revised: Under the assumption that γ 1 ≥ γ 2 ≥ · · · ≥ γ M := 0 are available to the learner, IISE can be conve- niently modiﬁed to obtain the following bound on the cost complexity (proofs, pseudocode, and formal statement in Appendix D):

Original: [Equation]
Revised: [Equation]

Original: To obtain such a result, the only required modiﬁcations stand in the elimination condition, which nowbecomes ˆ µ j,m,t − B (cid:0) t, δKM (cid:1) ≥ ˆ µ i,m,t + B (cid:0) t, δKM (cid:1) + γ m , and in the deﬁnition of the thresholdsα m , in which 4 ξ ’s are swapped with 2 γ ’s. We will refer to this modiﬁed algorithm with IISE- γ .
Revised: To obtain such a result, the only required modiﬁcations stand in the elimination condition, which now becomes ˆ µ j,m,t − B (cid:0) t, δKM (cid:1) ≥ ˆ µ i,m,t + B (cid:0) t, δKM (cid:1) + γ m , and in the deﬁnition of the thresholds α m , in which 4 ξ ’s are swapped with 2 γ ’s. We will refer to this modiﬁed algorithm with IISE- γ .

Original: Asour proofs show, IISE- γ can also exploit upper bounds φ m on the true maximum bias variation (i.e.,φ m ≥ γ m ); the complexity is as in Equation (9), but γ m is replaced with φ m .
Revised: As our proofs show, IISE- γ can also exploit upper bounds φ m on the true maximum bias variation (i.e., φ m ≥ γ m ); the complexity is as in Equation (9), but γ m is replaced with φ m .

Original: We remark that, by deﬁnition, γ m ≤ 2 ξ m , which provably demonstrates the beneﬁts of exploitingorder-aware knowledge in the MF-BAI setting.
Revised: We remark that, by deﬁnition, γ m ≤ 2 ξ m , which provably demonstrates the beneﬁts of exploiting order-aware knowledge in the MF-BAI setting.

Original: Interestingly, we also notice that in the worst-casescenario (i.e., γ m = 2 ξ m ), Equation (9) implicitly recovers the cost complexity of Theorem 3.
Revised: Interestingly, we also notice that in the worst-case scenario (i.e., γ m = 2 ξ m ), Equation (9) implicitly recovers the cost complexity of Theorem 3.

Original: Practical Relevance of Maximum Bias Variation.
Revised: Practical Relevance of Maximum Bias Variation.

Original: Finally, we highlight that the knowledge aboutupper bounds φ m is available in a large class of problems.
Revised: Finally, we highlight that the knowledge about upper bounds φ m is available in a large class of problems.

Original: In particular, consider settings in whichmeans of ﬁdelity m <
Revised: In particular, consider settings in which means of ﬁdelity m <

Original: M are underestimations of the ones of ﬁdelity M , i.e., µ i,M − µ i,m ≥ 0 for alli ∈
Revised: M are underestimations of the ones of ﬁdelity M , i.e., µ i,M − µ i,m ≥ 0 for all i ∈

Original: [ K ] and m ∈ [ M ] .
Revised: [ K ] and m ∈ [ M ] .

Original: This is the case, for instance, of the stochastic planning application of Section4, but it also common in other settings such as training models with less iterations.
Revised: This is the case, for instance, of the stochastic planning application of Section 4, but it also common in other settings such as training models with less iterations.

Original: In these problems,it always holds that γ m ≤ ξ m (proof in Appendix D).
Revised: In these problems, it always holds that γ m ≤ ξ m (proof in Appendix D).

Original: Therefore, we can set φ m = ξ m in IISE- γ toobtain the improved cost complexity of Equation (9), where 2 γ m is replaced with 2 ξ m .
Revised: Therefore, we can set φ m = ξ m in IISE- γ to obtain the improved cost complexity of Equation (9), where 2 γ m is replaced with 2 ξ m .

Original:  Related Works
Revised:  Related Works

Original: In this section, we revise the literature with particular attention to best-arm identiﬁcation and themulti-ﬁdelity setting.
Revised: In this section, we revise the literature with particular attention to best-arm identiﬁcation and the multi-ﬁdelity setting.

Original: Best-Arm Identiﬁcation.
Revised: Best-Arm Identiﬁcation.

Original: The BAI setting has aroused the interest of the research community fora long time [10, 6, 2].
Revised: The BAI setting has aroused the interest of the research community for a long time [10, 6, 2].

Original: In particular, [10] proposed the Successive Elimination algorithm, togetherwith the ﬁrst gap-dependent upper bound on the sample complexity: O ( (cid:80) Ki =2 ∆ − 2 i (log ( δ − 1 ) +log ( K ) + log (∆ − 2 i )) .
Revised: In particular, [10] proposed the Successive Elimination algorithm, together with the ﬁrst gap-dependent upper bound on the sample complexity: O ( (cid:80) Ki =2 ∆ − 2 i (log ( δ − 1 ) + log ( K ) + log (∆ − 2 i )) .

Original: This algorithm is known to match the lower bound up to logarithmic factors.
Revised: This algorithm is known to match the lower bound up to logarithmic factors.

Original: Indeed, [29, 24] showed that, for all possible instances, every δ -correct algorithm requires at leastΩ( (cid:80) Ki =2 ∆ − 2 i log( δ − 1 ))
Revised: Indeed, [29, 24] showed that, for all possible instances, every δ -correct algorithm requires at least Ω( (cid:80) Ki =2 ∆ − 2 i log( δ − 1 ))

Original: samples to output the optimal arm with probability 1 − δ .
Revised: samples to output the optimal arm with probability 1 − δ .

Original: To overcome thelogarithmic mismatch, several works have proposed both tighter upper [12, 23, 16, 7, 13, 8] andlower bounds [16, 13, 8].
Revised: To overcome the logarithmic mismatch, several works have proposed both tighter upper [12, 23, 16, 7, 13, 8] and lower bounds [16, 13, 8].

Original: Concerning the BAI literature, we leverage on the change of distributionsarguments presented in [24] to build lower bounds for our novel cost complexity metric.
Revised: Concerning the BAI literature, we leverage on the change of distributions arguments presented in [24] to build lower bounds for our novel cost complexity metric.

Original: As for theupper bound, instead, to favor the simplicity of exposition and intuition, we build on the Successive
Revised: As for the upper bound, instead, to favor the simplicity of exposition and intuition, we build on the Successive

Original: Elimination algorithm.
Revised: Elimination algorithm.

Original: Clearly, our proofs can straightforwardly incorporate the techniques from themore reﬁned analysis [e.g., 23] to tighten the logarithmic dependencies.
Revised: Clearly, our proofs can straightforwardly incorporate the techniques from the more reﬁned analysis [e.g., 23] to tighten the logarithmic dependencies.

Original: Multiple Fidelity.
Revised: Multiple Fidelity.

Original: The use of multiple ﬁdelity has gathered particular attention from the Bayesian
Revised: The use of multiple ﬁdelity has gathered particular attention from the Bayesian

Original: Optimization (BO) [5] ﬁeld, with a large variety of algorithms that operate under different assumptionsand with different goals [15, 33, 34, 28].
Revised: Optimization (BO) [5] ﬁeld, with a large variety of algorithms that operate under different assumptions and with different goals [15, 33, 34, 28].

Original: In particular, a recent line of works [22, 19] has focusedon minimizing the simple regret suffered when a certain budget has been spent to optimize a targetfunction under smoothness conditions guaranteed by Gaussian processes [37].
Revised: In particular, a recent line of works [22, 19] has focused on minimizing the simple regret suffered when a certain budget has been spent to optimize a target function under smoothness conditions guaranteed by Gaussian processes [37].

Original: The study of the simpleregret has also been extended to black-box function optimization [39, 40, 11] under hierarchicalpartition assumptions on the target function.
Revised: The study of the simple regret has also been extended to black-box function optimization [39, 40, 11] under hierarchical partition assumptions on the target function.

Original: However, all previous work differs signiﬁcantly from thestudy we present in this paper.
Revised: However, all previous work differs signiﬁcantly from the study we present in this paper.

Original: Indeed, they optimize different goals w.r.t.
Revised: Indeed, they optimize different goals w.r.t.

Original: our novel cost complexitynotion and make structured assumptions about the target function (i.e., the arm space).
Revised: our novel cost complexity notion and make structured assumptions about the target function (i.e., the arm space).

Original: With mostof them [21, 20, 39, 11], we share the assumption on the knowledge of the upper bound on themaximum bias ξ m for each ﬁdelity.
Revised: With most of them [21, 20, 39, 11], we share the assumption on the knowledge of the upper bound on the maximum bias ξ m for each ﬁdelity.

Original: Of particular interest w.r.t.
Revised: Of particular interest w.r.t.

Original: our work, is the one of [11].
Revised: our work, is the one of [11].

Original: Inparticular, the authors highlighted the concept that what matters when exploiting ﬁdelity m < M ishow much they preserve the order rather than the bias they have.
Revised: In particular, the authors highlighted the concept that what matters when exploiting ﬁdelity m < M is how much they preserve the order rather than the bias they have.

Original: However, their assumption aboutorder preservation is expressed in a general form; with the maximum bias variation γ m , instead, wepropose a closed-form expression for this concept, and we quantitatively show its impact in terms ofbeneﬁts with respect to the upper bound on the maximum bias.
Revised: However, their assumption about order preservation is expressed in a general form; with the maximum bias variation γ m , instead, we [Table] propose a closed-form expression for this concept, and we quantitatively show its impact in terms of beneﬁts with respect to the upper bound on the maximum bias.

Original: We also showed that upper boundsφ m on this metric are known and available in a large class of problems, and that our algorithm cansuccessfully exploit this knowledge to reduce identiﬁcation cost.
Revised: We also showed that upper bounds φ m on this metric are known and available in a large class of problems, and that our algorithm can successfully exploit this knowledge to reduce identiﬁcation cost.

Original: The work most closely related to ours is [21].
Revised: The work most closely related to ours is [21].

Original: The authors consider the ﬁnite arm setting with knownupper bounds on the maximum bias for each ﬁdelity.
Revised: The authors consider the ﬁnite arm setting with known upper bounds on the maximum bias for each ﬁdelity.

Original: However, while we take into account theidentiﬁcation cost, in [21], the authors analyze a pseudo-regret notion that sets as an optimal strategythe one that invests the whole budget in querying the optimal arm.
Revised: However, while we take into account the identiﬁcation cost, in [21], the authors analyze a pseudo-regret notion that sets as an optimal strategy the one that invests the whole budget in querying the optimal arm.

Original: Interestingly, we got similarresults, in terms of structure, for what concerns the lower bound, although we remark that the prooftechniques are different since we deal with a different performance metric.
Revised: Interestingly, we got similar results, in terms of structure, for what concerns the lower bound, although we remark that the proof techniques are different since we deal with a different performance metric.

Original: As for the proposedalgorithm and the upper bounds, our proofs rely on novel concepts such as the ﬁdelity convenience(i.e. , Deﬁnition 1), which are tailored to the identiﬁcation cost we have analyzed.
Revised: As for the proposed algorithm and the upper bounds, our proofs rely on novel concepts such as the ﬁdelity convenience (i.e., Deﬁnition 1), which are tailored to the identiﬁcation cost we have analyzed.

Original: In addition, as wehave seen in Section 4, we make a different assumption w.r.t.
Revised: In addition, as we have seen in Section 4, we make a different assumption w.r.t.

Original: [21]: our assumption directly states arelationship between costs and biases, which is more reasonable since the goal of the multi-ﬁdelitysetting is to trade-off between the two.
Revised: [21]: our assumption directly states a relationship between costs and biases, which is more reasonable since the goal of the multi-ﬁdelity setting is to trade-off between the two.

Original: Finally, w.r.t.
Revised: Finally, w.r.t.

Original: [21], we analyze the effect of more ﬁne-grainedknowledge on different ﬁdelity, which provably leads to performance improvements.
Revised: [21], we analyze the effect of more ﬁne-grained knowledge on different ﬁdelity, which provably leads to performance improvements.

Original: To conclude, some works [9, 30, 41] have also investigated the use of multi-ﬁdelity simulators in
Revised: To conclude, some works [9, 30, 41] have also investigated the use of multi-ﬁdelity simulators in

Original: Reinforcement Learning (RL) [42] domains.
Revised: Reinforcement Learning (RL) [42] domains.

Original: Among these works, [9] develops ﬁxed conﬁdencealgorithms (up to a desired precision (cid:15) > 0 ) both for the RL setting and for the bandit one.
Revised: Among these works, [9] develops ﬁxed conﬁdence algorithms (up to a desired precision (cid:15) > 0 ) both for the RL setting and for the bandit one.

Original: However,their setting is signiﬁcantly different from ours as they do not consider each ﬁdelity to be related to agiven cost.
Revised: However, their setting is signiﬁcantly different from ours as they do not consider each ﬁdelity to be related to a given cost.

Original: Instead, they simply aim at minimizing the number of sub-optimal steps at ﬁdelity M ,while controlling the number of samples taken in ﬁdelity m <
Revised: Instead, they simply aim at minimizing the number of sub-optimal steps at ﬁdelity M , while controlling the number of samples taken in ﬁdelity m <

Original: M to ensure they are polynomial.
Revised: M to ensure they are polynomial.

Original:  Numerical Validation
Revised:  Numerical Validation

Original: In this section, we provide a numerical validation of our theoretical claims in both synthetic andsimulated domains.
Revised: In this section, we provide a numerical validation of our theoretical claims in both synthetic and simulated domains.

Original: To prove that IISE is successful at exploiting ﬁdelity, we compare it with: (i)
Revised: To prove that IISE is successful at exploiting ﬁdelity, we compare it with: (i)

Original: Successive Elimination (SE) [10], which exploits only maximum ﬁdelity, and with a version of IISEthat chooses the thresholds α m so as to exploit every ﬁdelity as much as possible (i.e., α m close to 0).
Revised: Successive Elimination (SE) [10], which exploits only maximum ﬁdelity, and with a version of IISE that chooses the thresholds α m so as to exploit every ﬁdelity as much as possible (i.e., α m close to 0).

Original: We call this modiﬁed version Maximum Fidelity Exploit (MFE) algorithm.
Revised: We call this modiﬁed version Maximum Fidelity Exploit (MFE) algorithm.

Original: Table 1 reports all the[Table] results in terms of the percentage of cost complexity of a given algorithm w.r.t.
Revised: Table 1 reports all the results in terms of the percentage of cost complexity of a given algorithm w.r.t.

Original: the one of Successive
Revised: the one of Successive

Original: Elimination.
Revised: Elimination.

Original: The value 100 corresponds to the relative cost complexity of SE.
Revised: The value 100 corresponds to the relative cost complexity of SE.

Original: Synthetic bandits.
Revised: Synthetic bandits.

Original: We start presenting some experiments on two synthetic bandits (Synthetic Aand Synthetic B) with randomly generated arms.
Revised: We start presenting some experiments on two synthetic bandits (Synthetic A and Synthetic B) with randomly generated arms.

Original: These experiments aim at validating our robustnessresults along with the beneﬁt of the order-aware knowledge.
Revised: These experiments aim at validating our robustness results along with the beneﬁt of the order-aware knowledge.

Original: Synthetic A setting parameters are K =2000 , M = 4 , λ = [1 , 10 , 100 , 1000] , ξ =
Revised: Synthetic A setting parameters are K = 2000 , M = 4 , λ = [1 , 10 , 100 , 1000] , ξ =

Original: [1 . 15 , 0 . 225 , 0 . 015 , 0] , γ = [0 . 3 , 0 .
Revised: [1 . 15 , 0 . 225 , 0 . 015 , 0] , γ = [0 . 3 , 0 .

Original: 001 , 0] ; for Synthetic B, instead, K = 1000 , M = 5 , λ
Revised: 001 , 0] ; for Synthetic B, instead, K = 1000 , M = 5 , λ

Original: =
Revised: =

Original: [16 , 64 , 256 , 1024 , 4096] , ξ = [1 . 15 , 0 .
Revised: [16 , 64 , 256 , 1024 , 4096] , ξ = [1 . 15 , 0 .

Original: 0105 , 0] ,γ = [0 . 3 , 0 . 1 , 0 .
Revised: 0105 , 0] , γ = [0 . 3 , 0 . 1 , 0 .

Original: To make the γ and ξ settings directly comparable, we generated the armssuch that the ﬁdelity index m is left unchanged (i.e., it simultaneously holds that λ 1 > λ 2 > · · · > λ M ,ξ 1 > ξ 2 > · · · > ξ M , and γ 1 > γ 2 > · · · > γ M ).
Revised: To make the γ and ξ settings directly comparable, we generated the arms such that the ﬁdelity index m is left unchanged (i.e., it simultaneously holds that λ 1 > λ 2 > · · · > λ M , ξ 1 > ξ 2 > · · · > ξ M , and γ 1 > γ 2 > · · · > γ M ).

Original: Further details on this generation process can be found339 in Appendix E. As we can appreciate from Table 1, IISE substantially reduces the cost complexityw.r.t.
Revised: Further details on this generation process can be found in Appendix E. As we can appreciate from Table 1, IISE substantially reduces the cost complexity w.r.t.

Original: SE, and, its order-aware version IISE- γ , obtains further improvements.
Revised: SE, and, its order-aware version IISE- γ , obtains further improvements.

Original: Table 1 also highlightsthe importance of choosing a good threshold α m .
Revised: Table 1 also highlights the importance of choosing a good threshold α m .

Original: Indeed, MFE insists in exploiting a given ﬁdelity,even when there is no convenience in doing so, thus obtaining signiﬁcantly worse results than SE.
Revised: Indeed, MFE insists in exploiting a given ﬁdelity, even when there is no convenience in doing so, thus obtaining signiﬁcantly worse results than SE.

Original: Yahtzee.
Revised: Yahtzee.

Original: We now apply our algorithm to the problem of choosing the ﬁrst action in the stochasticplanning domain introduced in Section 4.
Revised: We now apply our algorithm to the problem of choosing the ﬁrst action in the stochastic planning domain introduced in Section 4.

Original: More speciﬁcally, we consider the Yahtzee game [3].
Revised: More speciﬁcally, we consider the Yahtzee game [3].

Original: Yahtzee is a sequential dice game in which a player rolls 5 dice up to 3 times and then chooses aparticular move that assigns a score to the ﬁnal dice combination.
Revised: Yahtzee is a sequential dice game in which a player rolls 5 dice up to 3 times and then chooses a particular move that assigns a score to the ﬁnal dice combination.

Original: For instance, typical moves are:“Sixes” with a score given by the sum of dice with the number 6 , and “Yahtzee”, that assigns 50 pointsif all the dice show the same number, and 0 otherwise.
Revised: For instance, typical moves are: “Sixes” with a score given by the sum of dice with the number 6 , and “Yahtzee”, that assigns 50 points if all the dice show the same number, and 0 otherwise.

Original: The total number of these possible moves is14 (i.e., our branching factor Λ ).
Revised: The total number of these possible moves is 14 (i.e., our branching factor Λ ).

Original: The game proceeds in 13 rounds.
Revised: The game proceeds in 13 rounds.

Original: During each round t , the playerselects a move and obtains the corresponding points.
Revised: During each round t , the player selects a move and obtains the corresponding points.

Original: We consider the variant of Yathzee in which,if the player has already selected a certain move at t < t , then, at t , that move will lead to reward 0 .
Revised: We consider the variant of Yathzee in which, if the player has already selected a certain move at t < t , then, at t , that move will lead to reward 0 .

Original: Furthermore, during each round, the player will roll the 5 dice only once, after which a move has tobe selected.
Revised: Furthermore, during each round, the player will roll the 5 dice only once, after which a move has to be selected.

Original: Further details are provided in Appendix E. Table 1 shows that our method signiﬁcantlyreduces the total number of generated nodes, thus lowering successfully the total cost of identifyingthe best action, and, that exploiting the upper bounds on the maximum bias variation instead of ξ ’s isbeneﬁcial in terms of total cost.
Revised: Further details are provided in Appendix E. Table 1 shows that our method signiﬁcantly reduces the total number of generated nodes, thus lowering successfully the total cost of identifying the best action, and, that exploiting the upper bounds on the maximum bias variation instead of ξ ’s is beneﬁcial in terms of total cost.

Original: Finally, MFE overexploits ﬁdelity and underperforms SE.
Revised: Finally, MFE overexploits ﬁdelity and underperforms SE.

Original:  Conclusions 
Revised:  Conclusions and Discussion

Original: In this work, we have studied, for the ﬁrst time, the variant of the ﬁxed-conﬁdence BAI scenario inwhich the agent has access to multiple approximations with different costs and biases of the originalbandit model.
Revised: In this work, we have studied, for the ﬁrst time, the variant of the ﬁxed-conﬁdence BAI scenario in which the agent has access to multiple approximations with different costs and biases of the original bandit model.

Original: We proposed the novel notion of cost complexity to evaluate the performance that agiven algorithm obtains in these situations.
Revised: We proposed the novel notion of cost complexity to evaluate the performance that a given algorithm obtains in these situations.

Original: We derived a lower bound on this index, which showsseveral interesting properties of the problem, along with its main challenge.
Revised: We derived a lower bound on this index, which shows several interesting properties of the problem, along with its main challenges.

Original: We also presented anovel algorithm, IISE, which provably reduces the cost complexity compared to algorithms thatignore the availability of multiple ﬁdelity.
Revised: We also presented a novel algorithm, IISE, which provably reduces the cost complexity compared to algorithms that ignore the availability of multiple ﬁdelity.

Original: More speciﬁcally, IISE discards a sub-optimal arm i withthe most convenient, in terms of cost and ﬁdelity, thus closely mimicking the shape of the lowerbound.
Revised: More speciﬁcally, IISE discards a sub-optimal arm i with the most convenient, in terms of cost and ﬁdelity, thus closely mimicking the shape of the lower bound.

Original: Furthermore, we also shown that the maximum bias is not what actually matters in identifyingthe best arm; indeed, better guarantees can be obtained with the knowledge of the maximum biasvariation, an original concept that we introduce to encode how much a given ﬁdelity preserves order.
Revised: Furthermore, we also showed that the maximum bias is not what actually matters in identifying the best arm. Indeed, better guarantees can be obtained with the knowledge of the maximum bias variation, an original concept that we introduce to encode how much a given ﬁdelity preserves order.

Original: To conclude, we note that previous studies [45, 18, 1] highlighted the strong connections betweenﬁxed-conﬁdence BAI and learning near-optimal RL policies while minimizing the number of samplesgathered in the environment.
Revised: To close the problem from a theoretical perspective, it is reasonable to assume that both upper and lower bounds could be improved in settings that do not fall under Assumption 1. More speciﬁcally, the lower bound of Theorem 1, that considers the minimum over the entire ﬁdelity set, seems currently optimistic since it does not encode the cost that is needed to explore at other ﬁdelities. To tackle this issue, proof techniques that rely on the characteristic time function [13] might lead to tighter lower bounds. Consequently, adapting to the MF setting their well-known “Track and Stop" strategy [13] might provide upper bounds that asymptotically match the lower bound in its characteristic time form.

Revised: However, results obtainable from these approaches usually come at the cost of lower interpretability (and, depending on the nature of the resulting characteristic function, computational complexity).

Original: In this sense, our ideas pave the way for future work that aims atlearning these policies when multiple simulators are available.
Revised: On the other hand, Theorem 1 provides several intuitions behind the challenges of the Multi-Fidelity BAI setting, and IISE is computationally efﬁcient. Furthermore, whenever Assumption 1 does not hold, the cost of “exploring" lower ﬁdelities becomes more and more relevant. Indeed, Assumption 1 guarantees that there is a large enough “ﬁdelity gap" (i.e., ﬁdelity differs in terms of the joint costs-biases relationship speciﬁed by the assumption). The main issue in settings, where this gap is small, roots down to the chicken-egg problem of MF-BAI.

Original: Indeed, previous efforts in MF-RL[9, 30, 41] do not directly take the challenging cost-precision trade-off into account. Checklist
Revised: On one side, we would like to exploit these approximators to discard sub-optimal arms; on the other side, we do not know if these approximators will be useful (i.e., in the worst case, we pay an exploration cost). In the case, where the “ﬁdelity-gap" is small, this problem is repeated over and over for all these ﬁdelities close to one another (i.e., the cumulative sum of the exploration costs becomes non-negligible).

Original: The checklist follows the references. Please read the checklist guidelines carefully for information onhow to answer these questions. For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
Revised: In this sense, tighter lower bounds (such as the ones previously suggested) might shed light on whether such assumptions are necessary or not, or if tighter assumptions are possible/necessary.

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: This represents an exciting line for future research.

Original: For example: • Did you include the license to the code and datasets? [Yes] See Section ?? .
Revised: To conclude, we note that previous studies [45, 18, 1] highlighted the strong connections between ﬁxed-conﬁdence BAI and learning near-optimal RL policies while minimizing the number of samples gathered in the environment.

Original: • Did you include the license to the code and datasets?
Revised: In this sense, our ideas pave the way for future work that aims at learning these policies when multiple simulators are available.

Original: [No] The code and the data are proprietary. • Did you include the license to the code and datasets? [N/A]
Revised: Indeed, previous efforts in MF-RL [9, 30, 41] do not directly take the challenging cost-precision trade-off into account

Original: Please do not modify the questions and only use the provided macros for your answers. Note that the Checklist section does not count towards the page limit. In your paper, please delete this instructionsblock and only keep the Checklistsection heading above along with the questions/answers below.
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] See for instance the contributions subsection.
Revised: [Yes] See for instance the contributions subsection.

Original: It list, one by one, the content of each of the section of the paper.
Revised: It list, one by one, the content of each of the section of the paper.

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 4.
Revised: [Yes] See Section 4.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Did not ﬁnd any.
Revised: [No] Did not ﬁnd any.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] We stated formally all the theoretical results throughout the main text (i.e., Theorem 1, Theorem2, Theorem 3, Proposition 1).
Revised: [Yes] We stated formally all the theoretical results throughout the main text (i e., Theorem 1, Theorem 2, Theorem 3, Proposition 1).

Original: We highlighted the set of assumptions in each claim.
Revised: We highlighted the set of assumptions in each claim.

Original: For extensions (e.g. Equation (9) we provide formal statements in the Appendix, but set of assumptions has been speciﬁed also in the main text.
Revised: For extensions (e.g. Equation (9) we provide formal statements in the Appendix, but set of assumptions has been speciﬁed also in the main text.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] We provide a detailed appendix with complete proofs of all theoretical results.
Revised: [Yes] We provide a detailed appendix with complete proofs of all theoretical results.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] Experiments are detailed in Section 7 and further documented in 
Revised: [Yes] Experiments are detailed in Section 7 and further documented in Appendix

Original: Appendix E. Further details can be found within the codebase.
Revised: E. Further details can be found within the codebase.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running exper- iments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] As speciﬁed in Section 7, we report mean and  conﬁdence intervals using 100 runs.
Revised: [Yes] As speciﬁed in Section 7, we report mean and 95% conﬁdence intervals using 100 runs.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] Yes, see Appendix E. 
Revised: [Yes] Yes, see Appendix E. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [N/A] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]