Original: Multi-Fidelity Best-Arm Identiﬁcation
Revised: Multi-Fidelity Best-Arm Identiﬁcation

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AfﬁliationAddress email
Revised: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: In several real-world applications, a learner has access to multiple environmentsimulators, each with a different precision (e.g., simulation accuracy) and cost(e.g., computational time).
Revised: In several real-world applications, a learner has access to multiple environmentsimulators, each with a different precision (e.g., simulation accuracy) and cost(e.g., computational time).

Original: In such a scenario, the learner faces the trade-offbetween selecting expensive accurate simulators or preferring cheap impreciseones.
Revised: In such a scenario, the learner faces the trade-offbetween selecting expensive accurate simulators or preferring cheap impreciseones.

Original: We formalize this setting as a multi-ﬁdelity variant of the stochastic best-
Revised: We formalize this setting as a multi-ﬁdelity variant of the stochastic best-

Original: arm identiﬁcation problem, where querying the original arm is expensive, butmultiple and biased approximations (i.e., ﬁdelities) are available at lower costs.
Revised: arm identiﬁcation problem, where querying the original arm is expensive, butmultiple and biased approximations (i.e., ﬁdelities) are available at lower costs.

Original: The learner’s goal, in this setting, is to sequentially choose which simulator toquery in order to minimize the total cost, while guaranteeing to identify the optimalarm with high probability.
Revised: The learner’s goal, in this setting, is to sequentially choose which simulator toquery in order to minimize the total cost, while guaranteeing to identify the optimalarm with high probability.

Original: We ﬁrst derive a lower bound on the identiﬁcation cost, assuming that the maximum bias of each ﬁdelity is known to the learner.
Revised: We ﬁrst derive a lower bound on the identiﬁcation cost, assuming that the maximum bias of each ﬁdelity is known to the learner.

Original: Then,we propose a novel algorithm, Iterative Imprecise Successive Elimination (IISE),which provably reduces the total cost w.r.t.
Revised: Then,we propose a novel algorithm, Iterative Imprecise Successive Elimination (IISE),which provably reduces the total cost w.r.t.

Original: algorithms that ignore the multi-ﬁdelitystructure and whose cost complexity upper bound mimics the structure of the lowerbound.
Revised: algorithms that ignore the multi-ﬁdelitystructure and whose cost complexity upper bound mimics the structure of the lowerbound.

Original: Furthermore, we show that the cost complexity of IISE can be furtherreduced when the agent has access to a more ﬁne-grained knowledge of the errorintroduced by the approximators.
Revised: Furthermore, we show that the cost complexity of IISE can be furtherreduced when the agent has access to a more ﬁne-grained knowledge of the errorintroduced by the approximators.

Original: Finally, we numerically validate IISE, showingthe beneﬁts of our method in simulated domains.
Revised: Finally, we numerically validate IISE, showingthe beneﬁts of our method in simulated domains.

Original:  Introduction
Revised:  Introduction

Original: In the multi-armed bandit setting [26], the agent selects, at each interaction round, an arm andobserves a sample drawn from its reward distribution.
Revised: In the multi-armed bandit setting [25], the agent selects, at each interaction round, an arm andobserves a sample drawn from its reward distribution.

Original: Unlike the regret minimization setting [26],where the agent tries to maximize the cumulative return, in best-arm identiﬁcation (BAI) problems[10, 6, 2], the evaluation index relates exclusively to the quality of the arm recommended at theend of the exploration phase.
Revised: Unlike the regret minimization setting [25],where the agent tries to maximize the cumulative return, in best-arm identiﬁcation (BAI) problems[10, 6, 2], the evaluation index relates exclusively to the quality of the arm recommended at theend of the exploration phase.

Original: More speciﬁcally, in the ﬁxed-conﬁdence setting [17], the agent aimsat identifying, with probability 1 − δ , the arm with the largest mean reward, while minimizing thesample complexity, i.e., the number of collected samples necessary to make the ﬁnal decision.
Revised: More speciﬁcally, in the ﬁxed-conﬁdence setting [16], the agent aimsat identifying, with probability 1 − δ , the arm with the largest mean reward, while minimizing thesample complexity, i.e., the number of collected samples necessary to make the ﬁnal decision.

Original: Dueto its generality, the BAI framework encompasses a wide spectrum of applications, ranging frommedical trials [38], to hyper-parameter optimization [27], crowdsourcing [47], recommender systems[25], simulation-based studies in physics [43], and A/B testing [14].
Revised: Dueto its generality, the BAI framework encompasses a wide spectrum of applications, ranging frommedical trials [38], to hyper-parameter optimization [26], crowdsourcing [48], recommender systems[24], simulation-based studies in physics [44], and A/B testing [13].

Original: However, in practical cases, querying an arm to obtain a sample from its reward distribution mightbe expensive.
Revised: However, in practical cases, querying an arm to obtain a sample from its reward distribution mightbe expensive.

Original: For instance, in the context of physics simulation studies, where pulling an armcorresponds to the evaluation of a complex model under some given arm parameters, intensive useof computing power is required to obtain the reward we are interested in.
Revised: For instance, in the context of physics simulation studies, where pulling an armcorresponds to the evaluation of a complex model under some given arm parameters, intensive useof computing power is required to obtain the reward we are interested in.

Original: Nevertheless, simplerand, consequently, cheaper and biased, models might be available to the agent.
Revised: Nevertheless, simplerand, consequently, cheaper and biased, models might be available to the agent.

Original: Another example isstochastic planning, where the agent needs to compute the best action (i.e., arm) that maximizes thefuture sum of rewards over a maximum horizon H .
Revised: Another example isstochastic planning, where the agent needs to compute the best action (i.e., arm) that maximizes thefuture sum of rewards over a maximum horizon H .

Original: However, while in principle it is possible to planconsidering the entire horizon, cheaper and biased estimates can be obtained by cutting the depth ofthe search to h < H .
Revised: However, while in principle it is possible to planconsidering the entire horizon, cheaper and biased estimates can be obtained by cutting the depth ofthe search to h < H .

Original: For these scenarios, the multi-ﬁdelity (MF) framework [9, 21, 22] that exploitsthis additional opportunity has recently gained attention in the literature.
Revised: For these scenarios, the multi-ﬁdelity (MF) framework [9, 20, 21] that exploitsthis additional opportunity has recently gained attention in the literature.

Original: The main rationale behind these methodologies is to exploit less expensive approximations in speciﬁc regions of the arm space tolimit the resources allocated to distinguish options that are far from optimal, and do not require highaccuracy.
Revised: The main rationale behind these methodologies is to exploit less expensive approximations in speciﬁc regions of the arm space tolimit the resources allocated to distinguish options that are far from optimal, and do not require highaccuracy.

Original: So far, most of the multi-ﬁdelity literature has focussed on black-box function optimization,with the goal of optimizing a target function f when multiple approximations of f are available[15, 33, 22, 34, 39, 40, 19, 28, 11].
Revised: So far, most of the multi-ﬁdelity literature has focussed on black-box function optimization,with the goal of optimizing a target function f when multiple approximations of f are available[14, 33, 21, 34, 39, 40, 18, 27, 11].

Original: Among these works, the ones more related to the bandit literatureare [22, 19, 39, 40, 11]; under different smoothness assumptions, these authors aimed at designingstrategies that explore the arm space to minimize the simple regret , i.e., the difference between thetrue optimal value of f and the best unbiased query made to f during the optimization process.
Revised: Among these works, the ones more related to the bandit literatureare [21, 18, 39, 40, 11]; under different smoothness assumptions, these authors aimed at designingstrategies that explore the arm space to minimize the simple regret , i.e., the difference between thetrue optimal value of f and the best unbiased query made to f during the optimization process.

Original: Otherworks studied cumulative regret analysis [21, 20], active learning [46, 32], reinforcement learning[9, 30, 41], and applied approaches [36, 44, 31].
Revised: Otherworks studied cumulative regret analysis [20, 19], active learning [47, 32], reinforcement learning[9, 29, 42], and applied approaches [36, 45, 31].

Original: In this paper, on the other hand, we focus on the understudied multi-ﬁdelity ﬁxed-conﬁdence BAIsetting.
Revised: In this paper, on the other hand, we focus on the understudied multi-ﬁdelity ﬁxed-conﬁdence BAIsetting.

Original: More speciﬁcally, we study the minimization of the cost complexity , a novel metric that weintroduce by extending the notion of sample complexity to the multi-ﬁdelity setting, by weighting eachsample with the related cost.
Revised: More speciﬁcally, we study the minimization of the cost complexity , a novel metric that weintroduce by extending the notion of sample complexity to the multi-ﬁdelity setting, by weighting eachsample with the related cost.

Original: The goal of the agent is to output the optimal arm with probability 1 − δ ,while minimizing the total cost.
Revised: The goal of the agent is to output the optimal arm with probability 1 − δ ,while minimizing the total cost.

Original: To this purpose, as commonly done in the literature [9,21, 20,  39, 11],we assume that the agent has some knowledge of the quality of the approximator, which is usuallyexpressed in terms of upper bounds on the maximum bias that the ﬁdelities introduce on the arm set.
Revised: To this purpose, as commonly done in the literature [9, 20, 19, 39, 11],we assume that the agent has some knowledge of the quality of the approximator, which is usuallyexpressed in terms of upper bounds on the maximum bias that the ﬁdelities introduce on the arm set.

Original: This term corresponds to the maximum distance between the mean of the desired (but expensive) armdistributions and their approximations.
Revised: This term corresponds to the maximum distance between the mean of the desired (but expensive) armdistributions and their approximations.

Original: Contributions.
Revised: Contributions.

Original: In this paper, we introduce the notion of cost complexity for the multi-ﬁdelity BAIsetting (Section 2).
Revised: In this paper, we introduce the notion of cost complexity for the multi-ﬁdelity BAIsetting (Section 2).

Original: We derive a lower bound for this novel index, under the standard assumptionthat the agent is equipped with knowledge of the maximum bias that each ﬁdelity has w.r.t.
Revised: We derive a lower bound for this novel index, under the standard assumptionthat the agent is equipped with knowledge of the maximum bias that each ﬁdelity has w.r.t.

Original: theoriginal, expensive and unbiased bandit model (Section 3).
Revised: theoriginal, expensive and unbiased bandit model (Section 3).

Original: As we shall show, our result highlightsseveral properties of the problem along with its challenges.
Revised: As we shall show, our result highlightsseveral properties of the problem along with its challenges.

Original: Furthermore, we propose the Iterative
Revised: Furthermore, we propose the Iterative

Original: Imprecise Successive Elimination (IISE) algorithm (Section 4), for which, we derive cost complexityupper bounds that, under certain assumptions, mimic the structure of the lower bound, and that isprovably robust w.r.t.
Revised: Imprecise Successive Elimination (IISE) algorithm (Section 4), for which, we derive cost complexityupper bounds that, under certain assumptions, mimic the structure of the lower bound, and that isprovably robust w.r.t.

Original: BAI algorithms that ignore the multi-ﬁdelity structure.
Revised: BAI algorithms that ignore the multi-ﬁdelity structure.

Original: We also show that theperformance of IISE can be further improved when a more ﬁne-grained knowledge of the behavior ofthe approximations is available to the agent (Section 5).
Revised: We also show that theperformance of IISE can be further improved when a more ﬁne-grained knowledge of the behavior ofthe approximations is available to the agent (Section 5).

Original: After revising the literature (Section 6), wepropose numerical experiments on simulated domains that empirically verify our claims (Section 7).
Revised: After revising the literature (Section 6), wepropose numerical experiments on simulated domains that empirically verify our claims (Section 7).

Original:  Setting
Revised:  Setting

Original: In the classical ﬁxed-conﬁdence BAI setting [17], the agent interacts with a bandit model ν :=( ν 1 , . . .
Revised: In the classical ﬁxed-conﬁdence BAI setting [16], the agent interacts with a bandit model ν :=( ν 1 , . . .

Original: , ν K ) , where K ∈ N is the number of arms and ν
Revised: , ν K ) , where K ∈ N is the number of arms and ν

Original: i denotes the reward distribution associatedwith arm i ∈
Revised: i denotes the reward distribution associatedwith arm i ∈

Original: [ K ] := { 1 , . . .
Revised: [ K ] := { 1 , . . .

Original: , K } .
Revised: , K } .

Original: Denote with µ i the expectation of ν i and, assume, that the K 73 arms are ordered such that µ [Equation]
Revised: Denote with µ i the expectation of ν i and, assume, that the Karms are ordered such that µ 1 > µ 2 ≥ · · · ≥ µ K .

Original: We assume that the reward distributions ν i are 74 subgaussian with known scale parameter σ 2 .
Revised: We assume that the reward distributions ν i aresubgaussian with known scale parameter σ 2 .

Original: At each interaction round t ∈ N , the agent selects anarm I t ∈
Revised: At each interaction round t ∈ N , the agent selects anarm I t ∈

Original: [ K ] and receives a reward R t sampled from ν I t .
Revised: [ K ] and receives a reward R t sampled from ν I t .

Original: Let T i ( t ) be the total number of pullsof arm
Revised: Let T i ( t ) be the total number of pullsof arm

Original: i ∈
Revised: i ∈

Original: [ K ] up to time t ∈ N .
Revised: [ K ] up to time t ∈ N .

Original: Given a conﬁdence level δ ∈ (0 , 1) , the goal of the agent is torecommend an arm (cid:98) I ( τ ) ∈
Revised: Given a conﬁdence level δ ∈ (0 , 1) , the goal of the agent is torecommend an arm (cid:98) I ( τ ) ∈

Original: [ K ] that corresponds to the optimal arm 1 with probability at least 1 − δ ,while minimizing the sample complexity ρ ( τ ) :
Revised: [ K ] that corresponds to the optimal arm 1 with probability at least 1 − δ ,while minimizing the sample complexity ρ ( τ ) :

Original: = (cid:80) i ∈
Revised: = (cid:80) i ∈

Original: [ K ] E [ T i ( τ )] , where τ is the stopping time ofthe agent.
Revised: [ K ] E [ T i ( τ )] , where τ is the stopping time ofthe agent.

Original: Any algorithm that recommends the best arm with probability 1 − δ is said δ -correct.
Revised: Any algorithm that recommends the best arm with probability 1 − δ is said δ -correct.

Original: Forevery sub-optimal arm i > 1 , we deﬁne its gap w.r.t.
Revised: Forevery sub-optimal arm i > 1 , we deﬁne its gap w.r.t.

Original: the optimal arm as ∆ i := µ 1 − µ i .
Revised: the optimal arm as ∆ i := µ 1 − µ i .

Original: In this paper, we differ from this classical setting as, for each arm i ∈
Revised: In this paper, we differ from this classical setting as, for each arm i ∈

Original: [ K ] , we assume the agent hasaccess to M − 1 ∈ N ﬁdelity distributions
Revised: [ K ] , we assume the agent hasaccess to M − 1 ∈ N ﬁdelity distributions

Original: ν i, 1 , ν i, 2 , . .
Revised: ν i, 1 , ν i, 2 , . .

Original: ν i,M − 1 , each σ 2 -subgaussian, with meansµ i, 1 , µ i, 2 , . .
Revised: ν i,M − 1 , each σ 2 -subgaussian, with meansµ i, 1 , µ i, 2 , . .

Original: . , µ i,M − 1 .
Revised: . , µ i,M − 1 .

Original: We unify the notation introducing a maximum ﬁdelity M to encode the true 84 arm distributions the agent is interested in, i.e., ν i,M :
Revised: We unify the notation introducing a maximum ﬁdelity M to encode the truearm distributions the agent is interested in, i.e., ν i,M :

Original: = ν i with mean µ i,M := µ i .
Revised: = ν i with mean µ i,M := µ i .

Original: As in previous works[9,21, 20,  39, 11], we assume that there is a relation encoding how much information distributions ofﬁdelity m <
Revised: As in previous works[9, 20, 19, 39, 11], we assume that there is a relation encoding how much information distributions ofﬁdelity m <

Original: M contain w.r.t.
Revised: M contain w.r.t.

Original: to the true arm distribution.
Revised: to the true arm distribution.

Original: More speciﬁcally, for each ﬁdelity m ∈
Revised: More speciﬁcally, for each ﬁdelity m ∈

Original: [ M ] ,it is always possible to write µ i,m := µ i,M + ξ i,m for some appropriate ξ i,m ∈ R .
Revised: [ M ] ,it is always possible to write µ i,m := µ i,M + ξ i,m for some appropriate ξ i,m ∈ R .

Original: Then, we assume 88 that the agent has access to ξ m ≥ max i ∈
Revised: Then, we assumethat the agent has access to ξ m ≥ max i ∈

Original: [ K ] | ξ i,m | , from which follows that | µ i,m − µ i,M | ≤ ξ m .
Revised: [ K ] | ξ i,m | , from which follows that | µ i,m − µ i,M | ≤ ξ m .

Original: We notice that ξ m represents an upper-bound on the maximum bias of ﬁdelity m ∈
Revised: We notice that ξ m represents an upper-bound on the maximum bias of ﬁdelity m ∈

Original: [ M ] .
Revised: [ M ] .

Original: At eachinteraction round t ∈ N , the agent selects an arm I t ∈
Revised: At eachinteraction round t ∈ N , the agent selects an arm I t ∈

Original: [ K ] , a ﬁdelity m t ∈
Revised: [ K ] , a ﬁdelity m t ∈

Original: [ M ] , and observes a reward
Revised: [ M ] , and observes a reward

Original: R t , drawn from ν I t ,m t .
Revised: R t , drawn from ν I t ,m t .

Original: The cost of gathering samples at ﬁdelity m is known and speciﬁed by λ m > 0 .
Revised: The cost of gathering samples at ﬁdelity m is known and speciﬁed by λ m > 0 .

Original: We make the standard assumption that λ 1 < λ 2 < · · · < λ M and ξ 1 > ξ 2 > · · · > ξ M := 0 .
Revised: We make the standard assumption that λ 1 < λ 2 < · · · < λ M and ξ 1 > ξ 2 > · · · > ξ M := 0 .

Original: At this 93 point, let T i,m ( t ) be the number of pulls to arm
Revised: At thispoint, let T i,m ( t ) be the number of pulls to arm

Original: i ∈
Revised: i ∈

Original: [ K ] at ﬁdelity m ∈
Revised: [ K ] at ﬁdelity m ∈

Original: [ M ] up to time t ∈ N ; we 94 deﬁne the cost complexity as:
Revised: [ M ] up to time t ∈ N ; wedeﬁne the cost complexity as:

Original: [Equation]
Revised: [Equation]

Original: where τ is the stopping time at which the agent will output the optimal arm 1 with probability 1 − δ .
Revised: where τ is the stopping time at which the agent will output the optimal arm 1 with probability 1 − δ .

Original:  Cost Complexity Lower Bound
Revised:  Cost Complexity Lower Bound

Original: In this section, we discuss the intrinsic complexity of the multi-ﬁdelity BAI problem, by providingand analyzing a lower bound on the cost complexity (proof in Appendix B).
Revised: In this section, we discuss the intrinsic complexity of the multi-ﬁdelity BAI problem, by providingand analyzing a lower bound on the cost complexity (proof in Appendix A).

Original: Theorem 1.
Revised: Theorem 1.

Original: Consider a multi-ﬁdelity bandit model ν with Gaussian distributions
Revised: Consider a multi-ﬁdelity bandit model ν with Gaussian distributions

Original: ν i,m = N ( µ i,m , σ 2 ) 100 such that | µ i,m − µ i,M | ≤ ξ m for every i ∈
Revised: ν i,m = N ( µ i,m , σ 2 )such that | µ i,m − µ i,M | ≤ ξ m for every i ∈

Original: [ K ] and m ∈ [ M ] .
Revised: [ K ] and m ∈ [ M ] .

Original: Then, for any δ -correct algorithm and 101 δ ≤ 0 . 15 , it holds that:
Revised: Then, for any δ -correct algorithm andδ ≤ 0 . 15 , it holds that:

Original: [Equation]
Revised: [Equation]

Original: where KL( p, q ) is the Kullback-Leibler divergence between distributions p and q , ν 2 ,m = N ( µ 2 ,M +ξ m , σ 2 ) , ν 1 ,m = N ( µ 1 ,M − ξ m , σ 2 ) , M 1 :
Revised: where KL( p, q ) is the Kullback-Leibler divergence between distributions p and q , ν 2 ,m = N ( µ 2 ,M +ξ m , σ 2 ) , ν 1 ,m = N ( µ 1 ,M − ξ m , σ 2 ) , M 1 :

Original: = { m ∈
Revised: = { m ∈

Original: [ M ] : µ 1 ,m > E x ∼ ν 2 ,m [ x ] } and M i := { m ∈
Revised: [ M ] : µ 1 ,m > E x ∼ ν 2 ,m [ x ] } and M i := { m ∈

Original: [ M ] : 104 E
Revised: [ M ] : 104 E

Original: [Figure]

Original: [Equation]

Original: [Figure]
Revised: [Figure]

Original:  ,m [ x ] > µ i,m } for i > 1 .
Revised: 1 ,m [ x ] > µ i,m } for i > 1 .

Original: This bound reveals several properties of the multi-ﬁdelity ﬁxed-conﬁdence BAI problem.
Revised: This bound reveals several properties of the multi-ﬁdelity ﬁxed-conﬁdence BAI problem.

Original: A singleterm λ m KL − 1 ( ν i,m , ν 1 ,m ) with i (cid:54) = 1 can be interpreted as a lower bound on the minimum costthat any δ -correct algorithm needs to pay to conclude that arm i is sub-optimal.
Revised: A singleterm λ m KL − 1 ( ν i,m , ν 1 ,m ) with i (cid:54) = 1 can be interpreted as a lower bound on the minimum costthat any δ -correct algorithm needs to pay to conclude that arm i is sub-optimal.

Original: Suppose, for themoment, that M i contains any ﬁdelity m ∈
Revised: Suppose, for themoment, that M i contains any ﬁdelity m ∈

Original: [ M ] .
Revised: [ M ] .

Original: Then, once we ﬁx m ∈ M i , its related componentλ m KL − 1 ( ν i,m , ν 1 ,m ) is the cost-based multi-ﬁdelity counterpart of the usual KL − 1 ( ν i , ν 1 ) whichappears in the standard BAI lower bound [24].
Revised: Then, once we ﬁx m ∈ M i , its related componentλ m KL − 1 ( ν i,m , ν 1 ,m ) is the cost-based multi-ﬁdelity counterpart of the usual KL − 1 ( ν i , ν 1 ) whichappears in the standard BAI lower bound [23].

Original: This can be seen as a lower bound on the cost todiscard arm i using samples gathered at ﬁdelity m .
Revised: This can be seen as a lower bound on the cost todiscard arm i using samples gathered at ﬁdelity m .

Original: Notice that, in this sense, the term ν 1 ,m replacesthe usual ν 1 to take the bias of ﬁdelity m into account.
Revised: Notice that, in this sense, the term ν 1 ,m replacesthe usual ν 1 to take the bias of ﬁdelity m into account.

Original: The minimum, consequently, encodes a lowerbound on the cost of the most convenient ﬁdelity with which we can discard i .
Revised: The minimum, consequently, encodes a lowerbound on the cost of the most convenient ﬁdelity with which we can discard i .

Original: The reason why wehave a minimum over M i instead of [ M ] is that, given a ﬁdelity m ∈
Revised: The reason why wehave a minimum over M i instead of [ M ] is that, given a ﬁdelity m ∈

Original: [ M ] , m might not be useful todiscard i , since ∆ i might be too small for the bias that m introduces.
Revised: [ M ] , m might not be useful todiscard i , since ∆ i might be too small for the bias that m introduces.

Original: This concept is further clariﬁedin the following remark:
Revised: This concept is further clariﬁedin the following remark:

Original: Remark 1.
Revised: Remark 1.

Original: Consider the following problem instance: for each m ∈
Revised: Consider the following problem instance: for each m ∈

Original: [ M ] , let µ 1 ,m = µ 1 ,M − ξ m andµ i,m = µ i,M + ξ m for i (cid:54)
Revised: [ M ] , let µ 1 ,m = µ 1 ,M − ξ m andµ i,m = µ i,M + ξ m for i (cid:54)

Original: = 1 .
Revised: = 1 .

Original: Then, Theroem 1 reduces to:
Revised: Then, Theroem 1 reduces to:

Original: [Equation]
Revised: [Equation]

Original: As we can see, if ∆ i < 2 ξ m for some m , its related component is not present in the lower bound.
Revised: As we can see, if ∆ i < 2 ξ m for some m , its related component is not present in the lower bound.

Original: Furthermore, it is interesting to note that the setting of Remark 1 represents a worst-case ﬁdelitydependent scenario.
Revised: Furthermore, it is interesting to note that the setting of Remark 1 represents a worst-case ﬁdelitydependent scenario.

Original: Indeed, each ﬁdelity m ∈
Revised: Indeed, each ﬁdelity m ∈

Original: [ M − 1] underestimates the reward of the optimal armand overestimates the ones of sub-optimal arms.
Revised: [ M − 1] underestimates the reward of the optimal armand overestimates the ones of sub-optimal arms.

Original: Finally, we show how Theorem 1 highlights the main challenge of our setting.
Revised: Finally, we show how Theorem 1 highlights the main challenge of our setting.

Original: For each value of ξ m ,there exists a multi-ﬁdelity bandit model ν for which the lower bound is given by (cid:101) Ω( (cid:80)
Revised: For each value of ξ m ,there exists a multi-ﬁdelity bandit model ν for which the lower bound is given by (cid:101) Ω( (cid:80)

Original: Ki =2 λ M σ 2 ∆ 2 i )(given Remark 1, it is sufﬁcient to consider ∆ i such that the only ﬁdelity in the minimization set is
Revised: Ki =2 λ M σ 2 ∆ 2 i )(given Remark 1, it is sufﬁcient to consider ∆ i such that the only ﬁdelity in the minimization set is

Original: M ).
Revised: M ).

Original: It is clear that this cost complexity can be matched (up to log factors) by many BAI algorithmsavailable in the literature (e.g., [10]) that ignore the multi-ﬁdelity structure of the problem.
Revised: It is clear that this cost complexity can be matched (up to log factors) by many BAI algorithmsavailable in the literature (e.g., [10]) that ignore the multi-ﬁdelity structure of the problem.

Original: The mainissue, in this sense, is the fact that the agent does not know, a priori, the values of ∆ i .
Revised: The mainissue, in this sense, is the fact that the agent does not know, a priori, the values of ∆ i .

Original: To build analgorithm that achieves robustness (i.e., not being worse than these uninformed strategies), then, we need to control how much we spend to explore the arm distributions at ﬁdelity m < M .
Revised: To build analgorithm that achieves robustness (i.e., not being worse than these uninformed strategies), then, we need to control how much we spend to explore the arm distributions at ﬁdelity m < M .

Original: Moregenerally, this reasoning also extends to building algorithms that match the lower bound.
Revised: Moregenerally, this reasoning also extends to building algorithms that match the lower bound.

Original: In thissense, we observe, even for the ﬁxed-conﬁdence BAI setting, the adversarial behavior that ﬁdelitymight have, as previously reported in [21] for the ﬁnite-armed bandit regret minimization setting.
Revised: In thissense, we observe, even for the ﬁxed-conﬁdence BAI setting, the adversarial behavior that ﬁdelitymight have, as previously reported in [20] for the ﬁnite-armed bandit regret minimization setting.

Original:  Imprecise and Iterative Successive Elimination
Revised:  Imprecise and Iterative Successive Elimination

Original: In this section, we present our algorithm, Imprecise and Iterative Successive Elimination (IISE,
Revised: In this section, we present our algorithm, Imprecise and Iterative Successive Elimination (IISE,

Original: Algorithm 1), to solve multi-ﬁdelity ﬁxed-conﬁdence BAI problems.
Revised: Algorithm 1), to solve multi-ﬁdelity ﬁxed-conﬁdence BAI problems.

Original: In particular, while developingour method, we pursue the following two objectives.
Revised: In particular, while developingour method, we pursue the following two objectives.

Original: First, we want to guarantee robustness w.r.t.uninformed strategies that blindly use only the highest ﬁdelity M to identify the best arm.
Revised: First, we want to guarantee robustness w.r.t.uninformed strategies that blindly use only the highest ﬁdelity M to identify the best arm.

Original: Indeed,we expect a good agent to be able to reduce the total cost by exploiting the peculiar multi-ﬁdelitystructure of the problem.
Revised: Indeed,we expect a good agent to be able to reduce the total cost by exploiting the peculiar multi-ﬁdelitystructure of the problem.

Original: Second, we aim at providing a strategy whose upper-bound structureresembles, as close as possible, the one of the lower bound presented in Section 3.
Revised: Second, we aim at providing a strategy whose upper-bound structureresembles, as close as possible, the one of the lower bound presented in Section 3.

Original: Our solution (pseudo-code in Algorithm 1) builds on the Successive Elimination algorithm [10].
Revised: Our solution (pseudo-code in Algorithm 1) builds on the Successive Elimination algorithm [10].

Original: More speciﬁcally, we proceed by epochs, one for each ﬁdelity m ∈
Revised: More speciﬁcally, we proceed by epochs, one for each ﬁdelity m ∈

Original: [ M ] , considered in increasingorder.
Revised: [ M ] , considered in increasingorder.

Original: In each phase m ∈ [ M ] , we query all the active arms, i.e., those that have not been discardedyet, through their approximated distributions ν i,m at ﬁdelity m .
Revised: In each phase m ∈ [ M ] , we query all the active arms, i.e., those that have not been discardedyet, through their approximated distributions ν i,m at ﬁdelity m .

Original: We denote the set of active arms with
Revised: We denote the set of active arms with

Original: S ⊆
Revised: S ⊆

Original: [ K ] .
Revised: [ K ] .

Original: IISE keeps usual empirical mean estimates ˆ µ i,m,t of the mean µ i,m of reward distributionsν i,m for each i ∈ S , and discards all the arms i for which the following applies:
Revised: IISE keeps usual empirical mean estimates ˆ µ i,m,t of the mean µ i,m of reward distributionsν i,m for each i ∈ S , and discards all the arms i for which the following applies:

Original: [Equation]
Revised: [Equation]

Original: , (3) where U ( t, δ, ξ ) is a symmetric bound on the empirical means (deﬁned below).
Revised: where U ( t, δ, ξ ) is a symmetric bound on the empirical means (deﬁned below).

Original: Condition (3)aims at eliminating all the active arms i whose estimated empirical mean upper bound falls belowthe empirical mean lower bound of another active arm j .
Revised: Condition (3)aims at eliminating all the active arms i whose estimated empirical mean upper bound falls belowthe empirical mean lower bound of another active arm j .

Original: More speciﬁcally, we set U ( t, δ, ξ ) :=
Revised: More speciﬁcally, we set U ( t, δ, ξ ) :=

Original: B ( t, δ ) + ξ = (cid:113) 2 σ 2 log 4 t 2 δ t + ξ .
Revised: B ( t, δ ) + [Equation] = [Equation]

Original:  As we can see, U ( t, δ, ξ ) is composed of the usual concentrationterm B ( t, δ ) plus the ﬁdelity bias ξ .
Revised: + ξ . As we can see, U ( t, δ, ξ ) is composed of the usual concentrationterm B ( t, δ ) plus the ﬁdelity bias ξ .

Original: As our proof highlights, this additional term is required toensure that the optimal arm remains, with high probability, in the active set S until termination.
Revised: As our proof highlights, this additional term is required toensure that the optimal arm remains, with high probability, in the active set S until termination.

Original: 2 Theduration of each epoch (i.e., how much IISE tries to exploit ﬁdelity m ) is determined by a thresholdparameter α m (to be deﬁned later).
Revised: 2 Theduration of each epoch (i.e., how much IISE tries to exploit ﬁdelity m ) is determined by a thresholdparameter α m (to be deﬁned later).

Original: At each round, the value of B (cid:0) t, δKM (cid:1) decreases and as soonas 4 B (cid:0) t, δKM (cid:1) drops below α m , IISE switches to the next ﬁdelity m + 1 .
Revised: At each round, the value of B (cid:0) t, δKM (cid:1) decreases and as soonas 4 B (cid:0) t, δKM (cid:1) drops below α m , IISE switches to the next ﬁdelity m + 1 .

Original: Notice that, to achieveδ -correctness, it is required to set α M to 0 , so that the last phase is guaranteed to identify the optimalarm, and α m > 0 for m < M , so that each phase is assured to terminate in a ﬁnite number of rounds.
Revised: Notice that, to achieveδ -correctness, it is required to set α M to 0 , so that the last phase is guaranteed to identify the optimalarm, and α m > 0 for m < M , so that each phase is assured to terminate in a ﬁnite number of rounds.

Original: Then, keeping generic thresholds α m that satisfy the previous conditions, we can show that IISE isδ -correct and its cost complexity is upper bounded by the following theorem (proof in AppendixC).
Revised: Then, keeping generic thresholds α m that satisfy the previous conditions, we can show that IISE isδ -correct and its cost complexity is upper bounded by the following theorem (proof in AppendixB).

Original: Theorem 2.
Revised: Theorem 2.

Original: If α m > 0 for every m ∈ [ M − 1] and α M = 0 , then, with probability at least 1 − δ , IISE 162 returns the optimal arm 1 with cost complexity c ( τ ) upper bounded by: 
Revised: If α m > 0 for every m ∈ [ M − 1] and α M = 0 , then, with probability at least 1 − δ , IISEreturns the optimal arm 1 with cost complexity c ( τ ) upper bounder by: [Equation] δ

Original: [Equation]
Revised: [Equation]

Original:  where m i is the smallest m ∈
Revised: , where m i is the smallest m ∈

Original: [ M ] for which ∆ i ≥ 4 ξ m + α m holds.
Revised: [ M ] for which ∆ i ≥ 4 ξ m + α m holds.

Original: We notice that the cost required by IISE to discard i (cid:54) = 1 is composed of two terms.
Revised: We notice that the cost required by IISE to discard i (cid:54) = 1 is composed of two terms.

Original: The ﬁrst one isthe cost of discarding arm i using samples at ﬁdelity m i , while the second one is the cost of trying todiscard arm at lower ﬁdelity m < m i .
Revised: The ﬁrst one isthe cost of discarding arm i using samples at ﬁdelity m i , while the second one is the cost of trying todiscard arm at lower ﬁdelity m < m i .

Original: As our proof reveals, m i is the smallest ﬁdelity m for which
Revised: As our proof reveals, m i is the smallest ﬁdelity m for which

Original: IISE ensures that arm i will be discarded.
Revised: IISE ensures that arm i will be discarded.

Original: 4.1 On the Choice of the Thresholds α m
Revised: 4.1 On the Choice of the Thresholds α m

Original: In this section, we highlight one of the main challenges of the MF setting, as mentioned in Section 3.
Revised: In this section, we highlight one of the main challenges of the MF setting, as mentioned in Section 3.

Original: Since we are unaware of the suboptimality gaps ∆ i , we may attempt to choose α m so as to minimize
Revised: Since we are unaware of the suboptimality gaps ∆ i , we may attempt to choose α m so as to minimize

Original: Require: Multi-ﬁdelity bandit model ν , conﬁdence δ , thresholds { α m } Mm =1 , bounds { ξ m } Mm =
Revised: Require: Multi-ﬁdelity bandit model ν , conﬁdence δ , thresholds { α m } Mm =1 , bounds { ξ m } Mm =

Original: S ← [ K ]
Revised: ← [ K ]

Original: ← ←3:> do 4: ifthen 5: if 7:
Revised: ←3: do 4: if if 7:

Original: Pull
Revised: Pull

Original: Update ˆt
Revised: Update ˆ

Original: [ S ] 9:
Revised: S ] 9:

Original: S ← S \ (cid:8) i ∈ S : ∃ j ∈ S : ˆ µ j,m,t − U (cid:0) t, δKM , ξ m (cid:1) ≥ ˆ µ i,m,t + U (cid:0) t, δKM , ξ m (cid:1)(cid:9) 10: end while 11: return S the cost complexity of Theorem 2 w.r.t.
Revised: S ← S \ (cid:8) i ∈ S : ∃ j ∈ S : ˆ µ j,m,t − U (cid:0) t, δKM , ξ m (cid:1) ≥ ˆ µ i,m,t + U (cid:0) t, δKM , ξ m (cid:1)(cid:9) 10: end while 11: return S the cost complexity of Theorem 2 w.r.t.

Original: the worst-case choice of suboptimality gaps ∆ i , that is:
Revised: the worst-case choice of suboptimality gaps ∆ i , that is:

Original: [Equation]
Revised: [Equation]

Original: where α = ( α 1 , . . . , α M ) and ∆ = (∆ 2 , . .
Revised: where α = ( α 1 , . . . , α M ) and ∆ = (∆ 2 , . .

Original: . , ∆ K ) are the threshold and gap vectors.
Revised: . , ∆ K ) are the threshold and gap vectors.

Original: However, itis easy to verify that Equation (4) is minimized for α m = + ∞ for all m < M and α M = 0 , i.e., analgorithm that exploits the highest ﬁdelity M only.
Revised: However, itis easy to verify that Equation (4) is minimized for α m = + ∞ for all m < M and α M = 0 , i.e., analgorithm that exploits the highest ﬁdelity M only.

Original: As we can see, IISE reﬂects the peculiar issueof the MF-BAI lower bound.
Revised: As we can see, IISE reﬂects the peculiar issueof the MF-BAI lower bound.

Original: In the worst case, any ﬁdelity m <
Revised: In the worst case, any ﬁdelity m <

Original: M is not useful to reduce theidentiﬁcation cost since it introduces too much bias w.r.t.
Revised: M is not useful to reduce theidentiﬁcation cost since it introduces too much bias w.r.t.

Original: ∆ i , regardless its cost.
Revised: ∆ i , regardless its cost.

Original: For this reason, asdone in previous works [21], we restrict the class of problems with some assumptions whose goal ismaking the ﬁdelity structure  provably convenient.
Revised: For this reason, asdone in previous works [20], we restrict the class of problems with some assumptions whose goal ismaking the ﬁdelity structure convenient provably convenient.

Original: More speciﬁcally:
Revised: More speciﬁcally:

Original: Assumption 1 (Costs and Biases Relationship) .
Revised: Assumption 1 (Costs and Biases Relationship) .

Original: For every ﬁdelity m ∈ [ M − 1] , it holds that:
Revised: For every ﬁdelity m ∈ [ M − 1] , it holds that:

Original: [Equation]
Revised: [Equation]

Original: Assumption 1 deserves some comments.
Revised: Assumption 1 deserves some comments.

Original: First of all, it directly encodes a relationship between costsand biases.
Revised: First of all, it directly encodes a relationship between costsand biases.

Original: To provide an interpretation, we notice that the assumption is always satisﬁed for: (i) coststhat grow at least linearly 3 and (ii) ξ ’s that satisfy the following decay rate: (cid:80) m<m
Revised: To provide an interpretation, we notice that the assumption is always satisﬁed for: (i) coststhat grow at least linearly 3 and (ii) ξ ’s that satisfy the following decay rate: (cid:80) m<m

Original: [Equation]
Revised: [Equation]

Original: The intuition behind the cost increase rate (i) is that, with sub-linear growth, attempting to discardarms with a cheaper ﬁdelity is too inconvenient w.r.t.
Revised: The intuition behind the cost increase rate (i) is that, with sub-linear growth, attempting to discardarms with a cheaper ﬁdelity is too inconvenient w.r.t.

Original: gathering more accurate samples for a slightlyhigher cost.
Revised: gathering more accurate samples for a slightlyhigher cost.

Original: For what concerns the ξ ’s decay rate, instead, condition (ii) limits the adversarial behaviorthat ﬁdelity might have.
Revised: For what concerns the ξ ’s decay rate, instead, condition (ii) limits the adversarial behaviorthat ﬁdelity might have.

Original: If two subsequent ξ ’s are very close together, little can be inferred fromswitching from the ﬁrst ﬁdelity to the second one.
Revised: If two subsequent ξ ’s are very close together, little can be inferred fromswitching from the ﬁrst ﬁdelity to the second one.

Original: This sort of problem was already highlighted in[21].
Revised: This sort of problem was already highlighted in[20].

Original: We remark that Assumption 1, compared to Assumption 1 of [21] (cid:80) m<m 1 ( ξ m ) 2 ≤ 1 ξ 2 m , involvesboth costs and biases, which is something we expect since the main challenge of the setting is totrade-off between them.
Revised: We remark that Assumption 1, compared to Assumption 1 of [20] (cid:80) m<m [Equation] m ) 2 ≤ 1 ξ 2 m , involvesboth costs and biases, which is something we expect since the main challenge of the setting is totrade-off between them.

Original: Indeed, depending on the values of λ ’s it is possible to tolerate ξ ’s that decaylinearly; vice-versa, if ξ ’s decay fast enough, it is possible to tolerate costs that increase sub-linearly.
Revised: Indeed, depending on the values of λ ’s it is possible to tolerate ξ ’s that decaylinearly; vice-versa, if ξ ’s decay fast enough, it is possible to tolerate costs that increase sub-linearly.

Original: Although Equation (5) might appear abstract, we show that it holds for a large class problems.
Revised: Although Equation (5) might appear abstract, we show that it holds for a large class problems.

Original: Consider a Markov Decision Process (MDP) [35] ( Y , A , p, r, η ) , where Y is the state space, A is the 194 action space, p = { p h } h ≥ 1 a set of transitions kernels, r = { r h } h ≥ 1 a set of reward functions, and 195 η ∈ [0 , 1] is a discount factor.
Revised: Consider a Markov Decision Process (MDP) [35] ( Y , A , p, r, η ) , where Y is the state space, A is theaction space, p = { p h } h ≥ 1 a set of transitions kernels, r = { r h } h ≥ 1 a set of reward functions, andη ∈ [0 , 1] is a discount factor.

Original: When an agent takes action a in state y at step h it transitions to y (cid:48) withprobability p h ( y (cid:48)
Revised: When an agent takes action a in state y at step h it transitions to y (cid:48) withprobability p h ( y (cid:48)

Original: | y, a ) , and receives a deterministic reward r h ( y, a ) ∈ [0 , 1] .
Revised: | y, a ) , and receives a deterministic reward r h ( y, a ) ∈ [0 , 1] .

Original: Given a state y ∈ Y , our 197 goal is to ﬁnd the action that maximizes the cumulative discounted reward over the next H steps 

Revised: Given a state y ∈ Y , ourgoal is to ﬁnd the action that maximizes the cumulative discounted reward over the next H steps

Original: Given an initial action, a way to compute a sample from its future cumulative discounted reward isto apply a Monte Carlo search algorithm (e.g., depth ﬁrst search) truncating the planning depth ath < H to obtain a cheaper but biased estimate of the cumulative discounted reward.
Revised: Given an initial action, a way to compute a sample from its future cumulative discounted reward isto apply a Monte Carlo search algorithm (e.g., depth ﬁrst search) truncating the planning depth ath < H to obtain a cheaper but biased estimate of the cumulative discounted reward.

Original: In this sense,this scenario ﬁts the ﬁxed-conﬁdence MF-BAI setting.
Revised: In this sense,this scenario ﬁts the ﬁxed-conﬁdence MF-BAI setting.

Original: Let M be the maximum depth, then, for eachdepth m ∈
Revised: Let M be the maximum depth, then, for eachdepth m ∈

Original: [ M ] , we set as λ m the number of generated nodes, that, assuming a constant branch factor
Revised: [ M ] , we set as λ m the number of generated nodes, that, assuming a constant branch factor

Original: Λ , is exponential in m (i.e., λ m = Λ m ).
Revised: Λ , is exponential in m (i.e., λ m = Λ m ).

Original: Instead, for ξ m it is easy to obtain the following upper boundson the maximum bias ξ m = (
Revised: Instead, for ξ m it is easy to obtain the following upper boundson the maximum bias ξ m = (

Original: η m +1 − η M +1 ) / (1 − η ) .
Revised: η m +1 − η M +1 ) / (1 − η ) .

Original: Now, it is possible to show that Assumption 1holds (proof in Appendix C).
Revised: Now, it is possible to show that Assumption 1holds (proof in Appendix B).

Original: Proposition 1.
Revised: Proposition 1.

Original: Consider Λ ≥ 2 .
Revised: Consider Λ > 2 .

Original: If λ m = Λ m and ξ m =
Revised: If λ m = Λ m and ξ m =

Original: ( η m +1 − η M +1 ) / (1 − η ) for all m ∈ [ M ] ,then Assumption 1 holds.
Revised: ( η m +1 − η M +1 ) / (1 − η ) for all m ∈ [ M ] ,then Assumption 1 holds.

Original: 4.2 Most Convenient Fidelity Thresholds
Revised: 4.2 Most Convenient Fidelity Thresholds

Original: Given the structure of Theorem 1 and its interpretation, during a phase m ∈ [ M ] , we expect a goodchoice of thresholds α to let IISE discard all the arms i (cid:54) = 1 that should (and can) be discarded at m .
Revised: Given the structure of Theorem 1 and its interpretation, during a phase m ∈ [ M ] , we expect a goodchoice of thresholds α to let IISE discard all the arms i (cid:54) = 1 that should (and can) be discarded at m .

Original: Therefore, during m , we aim at concluding that a given arm is suboptimal only if there is no otherﬁdelity m > m in which gathering samples at m is more convenient, in term of identiﬁcation cost,w.r.t.
Revised: Therefore, during m , we aim at concluding that a given arm is suboptimal only if there is no otherﬁdelity m > m in which gathering samples at m is more convenient, in term of identiﬁcation cost,w.r.t.

Original: m .
Revised: m .

Original: We formalize the previous concept in the following deﬁnition.
Revised: We formalize the previous concept in the following deﬁnition.

Original: Deﬁnition 1 (Fidelity Convenience) .
Revised: Deﬁnition 1 (Fidelity Convenience) .

Original: Consider a sub-optimal arm i (cid:54)
Revised: Consider a sub-optimal arm i (cid:54)

Original: = 1 and a ﬁdelity m <
Revised: = 1 and a ﬁdelity m <

Original: M suchthat ∆ i > 4 ξ m .
Revised: M suchthat ∆ i > 4 ξ m .

Original: We say that ﬁdelity m is convenient for arm i if:
Revised: We say that ﬁdelity m is convenient for arm i if:

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: In Equation (6), we are comparing the cost of discarding i gathering samples at phase m only withthe cost of discarding i using a more precise ﬁdelity m > m .
Revised: In Equation (6), we are comparing the cost of discarding i gathering samples at phase m only withthe cost of discarding i using a more precise ﬁdelity m > m .

Original: Therefore, solving it for ∆ i , we identifythe gaps of the arms that can be conveniently discarded at ﬁdelity m rather than at m :
Revised: Therefore, solving it for ∆ i , we identifythe gaps of the arms that can be conveniently discarded at ﬁdelity m rather than at m :

Original: [Equation]
Revised: [Equation]

Original: To discard all i that satisfy Equation (7), it is sufﬁcient to set thresholds α m for m < M as follows:
Revised: To discard all i that satisfy Equation (7), it is sufﬁcient to set thresholds α m for m < M as follows:

Original: [Equation]
Revised: [Equation]

Original: At this point, under Assumption 1, it is possible to prove that, with this peculiar threshold choice, thecost complexity of IISE can be rewritten in the following way (proof in Appendix C).
Revised: At this point, under Assumption 1, it is possible to prove that, with this peculiar threshold choice, thecost complexity of IISE can be rewritten in the following way (proof in Appendix B).

Original: Theorem 3.
Revised: Theorem 3.

Original: Under Assumption 1, selecting the thresholds α m as in Equation (8) , with probabilityat least 1 − δ , IISE returns the optimal arm with cost complexity c ( τ ) upper bounder by:
Revised: Under Assumption 1, selecting the thresholds α m as in Equation (8) , with probabilityat least 1 − δ , IISE returns the optimal arm with cost complexity c ( τ ) upper bounder by:

Original: [Equation]
Revised: [Equation]

Original: As we can appreciate, this result comes with two desired properties.
Revised: As we can appreciate, this result comes with two desired properties.

Original: First of all, IISE is provablyrobust w.r.t.
Revised: First of all, IISE is provablyrobust w.r.t.

Original: algorithms that ignore the multi-ﬁdelity structure of the problem.
Revised: algorithms that ignore the multi-ﬁdelity structure of the problem.

Original: We remark that tohave this nice property, Assumption 1 does not need to hold for each m ∈
Revised: We remark that tohave this nice property, Assumption 1 does not need to hold for each m ∈

Original: [ M ] .
Revised: [ M ] .

Original: It is always possibleto pre-select a subset of ﬁdelity for which the assumption holds.
Revised: It is always possibleto pre-select a subset of ﬁdelity for which the assumption holds.

Original: Secondly, the bound in Theorem3 closely mimics the behavior of the lower bound (Equation 2).
Revised: Secondly, the bound in Theorem3 closely mimics the behavior of the lower bound (Equation 2).

Original: The only difference stands in thefact that 2 ξ m is now replaced with 4 ξ m .
Revised: The only difference stands in thefact that 2 ξ m is now replaced with 4 ξ m .

Original: We leave this gap to be ﬁlled in for future work.
Revised: We leave this gap to be ﬁlled in for future work.

Original: Notice,again, that the similarity also holds for the case in which we select a subset of ﬁdelity for which theassumption holds.
Revised: Notice,again, that the similarity also holds for the case in which we select a subset of ﬁdelity for which theassumption holds.

Original: In this case, however, our minimization will be on a restricted ﬁdelity set w.r.t. theoriginal one.
Revised: In this case, however, our minimization will be on a restricted ﬁdelity set w.r.t theoriginal one.

Original: 4.3 Nearly-Optimal Identiﬁcation
Revised: 4.3 Nearly-Optimal Identiﬁcation

Original: As for the Successive Elimination algorithm, one can easily modify IISE so that it returns an (cid:15) -optimalarm.
Revised: As for the Successive Elimination algorithm, one can easily modify IISE so that it returns an (cid:15) -optimalarm.

Original: In this case, we can show that the identiﬁcation cost is upper bounded by (proof in Appendix C):
Revised: In this case, we can show that the identiﬁcation cost is upper bounded by (proof in Appendix B):

Original: [Equation]
Revised: [Equation]

Original: where K (cid:15) :
Revised: where K (cid:15) :

Original: = { i ∈
Revised: = { i ∈

Original: [ K ] : µ i > µ 1 − (cid:15) } .
Revised: [ K ] : µ i > µ 1 − (cid:15) } .

Original: As usual, for arms that are not (cid:15) -optimal we pay the original cost 237 of Theorem 3.
Revised: As usual, for arms that are not (cid:15) -optimal we pay the original costof Theorem 3.

Original: For the ones in K (cid:15) , instead, their cost is actually clipped by (cid:15) .
Revised: For the ones in K (cid:15) , instead, their cost is actually clipped by (cid:15) .

Original: In multi-ﬁdelity, this has the further implication that, depending on the values of biases, costs and required accuracy, wemight never query the most expensive unbiased ﬁdelity M .
Revised: In multi-ﬁdelity, this has the further implication that, depending on the values of biases, costs and required accuracy, wemight never query the most expensive unbiased ﬁdelity M .

Original: In particular, let us focus on a two-armedbandit problem with two ﬁdelity values, and consider ∆ 2 < (cid:15) .
Revised: In particular, let us focus on a two-armedbandit problem with two ﬁdelity values, and consider ∆ 2 < (cid:15) .

Original: Then, we will never query M if:
Revised: Then, we will never query M if:

Original: [Equation]
Revised: [Equation]

Original: The ﬁrst term guarantees that IISE can discard i using m , the second one, instead encodes theconvenience of m w.r.t. to M in terms of costs.
Revised: The ﬁrst term guarantees that IISE can discard i using m , the second one, instead encodes theconvenience of m w.r.t to M in terms of costs.

Original: As λ M increases, this term shrinks to zero.
Revised: As λ M increases, this term shrinks to 0 .

Original: Whenλ M → + ∞ , the condition reduces to (cid:15) > 4 ξ m .
Revised: Whenλ M → + ∞ , the condition reduces to (cid:15) > 4 ξ m .

Original: This means that if ﬁdelity m is precise enough for thedesired accuracy (cid:15) , we completely neglect ﬁdelity M .
Revised: This means that if ﬁdelity m is precise enough for thedesired accuracy (cid:15) , we completely neglect ﬁdelity M .

Original:  Breaking the Cost Barrier
Revised:  Breaking the Cost Barrier

Original: We now show that (a slightly modiﬁed version of) IISE can further reduce the cost complexity whenmore ﬁne-grained knowledge on the type of error that approximators introduce is available.
Revised: We now show that (a slightly modiﬁed version of) IISE can further reduce the cost complexity whenmore ﬁne-grained knowledge on the type of error that approximators introduce is available.

Original: The mainintuition behind the following reasoning is that what matters when exploiting ﬁdelity m <
Revised: The mainintuition behind the following reasoning is that what matters when exploiting ﬁdelity m <

Original: M is notthe bias they introduce, but whether the arms do not preserve the order w.r.t.
Revised: M is notthe bias they introduce, but whether the arms do not preserve the order w.r.t.

Original: the mean of the truereward distributions ν
Revised: the mean of the truereward distributions ν

Original: i,M .
Revised: i,M .

Original: Indeed suppose to have access to a ﬁdelity with a very large bias but whichfully preserves the order on the means of the arms.
Revised: Indeed suppose to have access to a ﬁdelity with a very large bias but whichfully preserves the order on the means of the arms.

Original: In this scenario, it is possible to run a standard
Revised: In this scenario, it is possible to run a standard

Original: BAI technique on this ﬁdelity only and still obtain the optimal arm.
Revised: BAI technique on this ﬁdelity only and still obtain the optimal arm.

Original: We now formalize how much acertain ﬁdelity m preserves the order with the novel concept of maximum bias variation γ m :
Revised: We now formalize how much acertain ﬁdelity m preserves the order with the novel concept of maximum bias variation γ m :

Original: [Equation]
Revised: [Equation]

Original: Under the assumption that γ 1 ≥ γ 2 ≥ · · · ≥ γ M := 0 are available to the learner, IISE can be conve- 255 niently modiﬁed to obtain the following bound on the cost complexity (proofs, pseudocode, andformal statement in Appendix D):
Revised: Under the assumption that γ 1 ≥ γ 2 ≥ · · · ≥ γ M := 0 are available to the learner, IISE can be conveniently modiﬁed to obtain the following bound on the cost complexity (proofs, pseudocode, andformal statement in Appendix C):

Original: [Equation]
Revised: [Equation]

Original: To obtain such a result, the only required modiﬁcations stand in the elimination condition, which nowbecomes ˆ µ j,m,t − B (cid:0) t, δKM (cid:1) ≥ ˆ µ i,m,t + B (cid:0) t, δKM (cid:1) + γ m , and in the deﬁnition of the thresholdsα m , in which 4 ξ ’s are swapped with 2 γ ’s. We will refer to this modiﬁed algorithm with IISE- γ .
Revised: To obtain such a result, the only required modiﬁcations stand in the elimination condition, which nowbecomes ˆ µ j,m,t − B (cid:0) t, δKM (cid:1) ≥ ˆ µ i,m,t + B (cid:0) t, δKM (cid:1) + γ m , and in the deﬁnition of the thresholdsα m , in which 4 ξ ’s are swapped with 2 γ ’s. We will refer to this modiﬁed algorithm with IISE- γ .

Original: Asour proofs show, IISE- γ can also exploit upper bounds φ m on the true maximum bias variation (i.e.,φ m ≥ γ m ); the complexity is as in Equation (9), but γ m is replaced with φ m .
Revised: Asour proofs show, IISE- γ can also exploit upper bounds φ m on the true maximum bias variation (i.e.,φ m ≥ γ m ); the complexity is as in Equation (9), but γ m is replaced with φ m .

Original: We remark that, by deﬁnition, γ m ≤ 2 ξ m , which provably demonstrates the beneﬁts of exploitingorder-aware knowledge in the MF-BAI setting.
Revised: We remark that, by deﬁnition, γ m ≤ 2 ξ m , which provably demonstrates the beneﬁts of exploitingorder-aware knowledge in the MF-BAI setting.

Original: Interestingly, we also notice that in the worst-casescenario (i.e., γ m = 2 ξ m ), Equation (9) implicitly recovers the cost complexity of Theorem 3.
Revised: Interestingly, we also notice that in the worst-casescenario (i.e., γ m = 2 ξ m ), Equation (9) implicitly recovers the cost complexity of Theorem 3.

Original: Practical Relevance of Maximum Bias Variation.
Revised: Practical Relevance of Maximum Bias Variation.

Original: Finally, we highlight that the knowledge aboutupper bounds φ m is available in a large class of problems.
Revised: Finally, we highlight that the knowledge aboutupper bounds φ m is available in a large class of problems.

Original: In particular, consider settings in whichmeans of ﬁdelity m <
Revised: In particular, consider settings in whichmeans of ﬁdelity m <

Original: M are underestimations of the ones of ﬁdelity M , i.e., µ i,M − µ i,m ≥ 0 for alli ∈
Revised: M are underestimations of the ones of ﬁdelity M , i.e., µ i,M − µ i,m ≥ 0 for alli ∈

Original: [ K ] and m ∈ [ M ] .
Revised: [ K ] and m ∈ [ M ] .

Original: This is the case, for instance, of the stochastic planning application of Section4, but it also common in other settings such as training models with less iterations.
Revised: This is the case, for instance, of the stochastic planning application of Section4, but it also common in other settings such as training models with less iterations.

Original: In these problems,it always holds that γ m ≤ ξ m (proof in Appendix D).
Revised: In these problems,it always holds that γ m ≤ ξ m (proof in Appendix C).

Original: Therefore, we can set φ m = ξ m in IISE-
Revised: Therefore, we can set φ m = ξ m in IISE-

Original: γ toobtain the improved cost complexity of Equation (9), where 2 γ m is replaced with 2 ξ m .
Revised: γ toobtain the improved cost complexity of Equation (9), where 2 γ m is replaced with 2 ξ m .

Original:  Related Works
Revised:  Related Works

Original: In this section, we revise the literature with particular attention to best-arm identiﬁcation and themulti-ﬁdelity setting.
Revised: In this section, we revise the literature with particular attention to best-arm identiﬁcation and themulti-ﬁdelity setting.

Original: Best-Arm Identiﬁcation.
Revised: Best-Arm Identiﬁcation.

Original: The BAI setting has aroused the interest of the research community fora long time [10, 6, 2].
Revised: The BAI setting has aroused the interest of the research community fora long time [10, 6, 2].

Original: In particular, [10] proposed the Successive Elimination algorithm, togetherwith the ﬁrst gap-dependent upper bound on the sample complexity: O ( (cid:80) Ki =2 ∆ − 2 i (log ( δ − 1 )
Revised: In particular, [10] proposed the Successive Elimination algorithm, togetherwith the ﬁrst gap-dependent upper bound on the sample complexity: O ( (cid:80) Ki =2 ∆ − 2 i (log ( δ − 1 )

Original: +log ( K ) + log (∆ − 2 i )) .
Revised: +log ( K ) + log (∆ − 2 i )) .

Original: This algorithm is known to match the lower bound up to logarithmic factors.
Revised: This algorithm is known to match the lower bound up to logarithmic factors.

Original: Indeed, [29, 24] showed that, for all possible instances, every δ -correct algorithm requires at leastΩ( (cid:80) Ki =2 ∆ − 2 i log( δ − 1 ))
Revised: Indeed, [28, 23] showed that, for all possible instances, every δ -correct algorithm requires at leastΩ( (cid:80) Ki =2 ∆ − 2 i log( δ − 1 ))

Original: samples to output the optimal arm with probability 1 − δ .
Revised: samples to output the optimal arm with probability 1 − δ .

Original: To overcome thelogarithmic mismatch, several works have proposed both tighter upper [12, 23, 16, 7,13, 8] andlower bounds [16, 13, 8].
Revised: To overcome thelogarithmic mismatch, several works have proposed both tighter upper [12, 22, 15, 7, 8] and lowerbounds [15, 8].

Original: Concerning the BAI literature, we leverage on the change of distributionsarguments presented in [24] to build lower bounds for our novel cost complexity metric.
Revised: Concerning the BAI literature, we leverage on the change of distributions argumentspresented in [23] to build lower bounds for our novel cost complexity metric.

Original: As for theupper bound, instead, to favor the simplicity of exposition and intuition, we build on the Successive Elimination algorithm.
Revised: As for the upper bound,instead, to favor the simplicity of exposition and intuition, we build on the Successive Eliminationalgorithm.

Original: Clearly, our proofs can straightforwardly incorporate the techniques from themore reﬁned analysis [e.g., 23] to tighten the logarithmic dependencies.
Revised: Clearly, our proofs can straightforwardly incorporate the techniques from the more reﬁnedanalysis [e.g., 22] to tighten the logarithmic dependencies.

Original: Multiple Fidelity.
Revised: Multiple Fidelity.

Original: The use of multiple ﬁdelity has gathered particular attention from the Bayesian
Revised: The use of multiple ﬁdelity has gathered particular attention from the Bayesian

Original: Optimization (BO) [5] ﬁeld, with a large variety of algorithms that operate under different assumptionsand with different goals [15, 33, 34, 28].
Revised: Optimization (BO) [5] ﬁeld, with a large variety of algorithms that operate under different assumptionsand with different goals [14, 33, 34, 27].

Original: In particular, a recent line of works [22, 19] has focusedon minimizing the simple regret suffered when a certain budget has been spent to optimize a targetfunction under smoothness conditions guaranteed by Gaussian processes [37].
Revised: In particular, a recent line of works [21, 18] has focusedon minimizing the simple regret suffered when a certain budget has been spent to optimize a targetfunction under smoothness conditions guaranteed by Gaussian processes [37].

Original: The study of the simpleregret has also been extended to black-box function optimization [39, 40, 11] under hierarchicalpartition assumptions on the target function.
Revised: The study of the simpleregret has also been extended to black-box function optimization [39, 40, 11] under hierarchicalpartition assumptions on the target function.

Original: However, all previous work differs signiﬁcantly from thestudy we present in this paper.
Revised: However, all previous work differs signiﬁcantly from thestudy we present in this paper.

Original: Indeed, they optimize different goals w.r.t.
Revised: Indeed, they optimize different goals w.r.t.

Original: our novel cost complexitynotion and make structured assumptions about the target function (i.e., the arm space).
Revised: our novel cost complexitynotion and make structured assumptions about the target function (i.e., the arm space).

Original: With mostof them [21, 20, 39, 11], we share the assumption on the knowledge of the upper bound on themaximum bias ξ m for each ﬁdelity.
Revised: With mostof them [20, 19, 39, 11], we share the assumption on the knowledge of the upper bound on themaximum bias ξ m for each ﬁdelity.

Original: Of particular interest w.r.t.
Revised: Of particular interest w.r.t.

Original: our work, is the one of [11].
Revised: our work, is the one of [11].

Original: Inparticular, the authors highlighted the concept that what matters when exploiting ﬁdelity m < M ishow much they preserve the order rather than the bias they have.
Revised: Inparticular, the authors highlighted the concept that what matters when exploiting ﬁdelity m < M ishow much they preserve the order rather than the bias they have.

Original: However, their assumption aboutorder preservation is expressed in a general form; with the maximum bias variation γ m , instead, wepropose a closed-form expression for this concept, and we quantitatively show its impact in terms ofbeneﬁts with respect to the upper bound on the maximum bias.
Revised: However, their assumption aboutorder preservation is expressed in a general form; with the maximum bias variation γ m , instead, wepropose a closed-form expression for this concept, and we quantitatively show its impact in terms ofbeneﬁts with respect to the upper bound on the maximum bias.

Original: We also showed that upper boundsφ m on this metric are known and available in a large class of problems, and that our algorithm cansuccessfully exploit this knowledge to reduce identiﬁcation cost.
Revised: We also showed that upper boundsφ m on this metric are known and available in a large class of problems, and that our algorithm cansuccessfully exploit this knowledge to reduce identiﬁcation cost.

Original: The work most closely related to ours is [21].
Revised: The work most closely related to ours is [20].

Original: The authors consider the ﬁnite arm setting with knownupper bounds on the maximum bias for each ﬁdelity.
Revised: The authors consider the ﬁnite arm setting with knownupper bounds on the maximum bias for each ﬁdelity.

Original: However, while we take into account theidentiﬁcation cost, in [21], the authors analyze a pseudo-regret notion that sets as an optimal strategythe one that invests the whole budget in querying the optimal arm.
Revised: However, while we take into account theidentiﬁcation cost, in [20], the authors analyze a pseudo-regret notion that sets as an optimal strategythe one that invests the whole budget in querying the optimal arm.

Original: Interestingly, we got similarresults, in terms of structure, for what concerns the lower bound, although we remark that the prooftechniques are different since we deal with a different performance metric.
Revised: Interestingly, we got similarresults, in terms of structure, for what concerns the lower bound, although we remark that the prooftechniques are different since we deal with a different performance metric.

Original: As for the proposedalgorithm and the upper bounds, our proofs rely on novel concepts such as the ﬁdelity convenience(i.e.
Revised: As for the proposedalgorithm and the upper bounds, our proofs rely on novel concepts such as the ﬁdelity convenience(i.e.

Original: , Deﬁnition 1), which are tailored to the identiﬁcation cost we have analyzed.
Revised: , Deﬁnition 1), which are tailored to the identiﬁcation cost we have analyzed.

Original: In addition, as wehave seen in Section 4, we make a different assumption w.r.t.
Revised: In addition, as wehave seen in Section 4, we make a different assumption w.r.t.

Original: [21]: our assumption directly states arelationship between costs and biases, which is more reasonable since the goal of the multi-ﬁdelitysetting is to trade-off between the two.
Revised: [20]: our assumption directly states arelationship between costs and biases, which is more reasonable since the goal of the multi-ﬁdelitysetting is to trade-off between the two.

Original: Finally, w.r.t.
Revised: Finally, w.r.t.

Original: [21], we analyze the effect of more ﬁne-grainedknowledge on different ﬁdelity, which provably leads to performance improvements.
Revised: [20], we analyze the effect of more ﬁne-grainedknowledge on different ﬁdelity, which provably leads to performance improvements.

Original: To conclude, some works [9, 30, 41] have also investigated the use of multi-ﬁdelity simulators in
Revised: To conclude, some works [9, 29, 42] have also investigated the use of multi-ﬁdelity simulators in

Original: Reinforcement Learning (RL) [42] domains.
Revised: Reinforcement Learning (RL) [43] domains.

Original: Among these works, [9] develops ﬁxed conﬁdencealgorithms (up to a desired precision (cid:15) > 0 ) both for the RL setting and for the bandit one.
Revised: Among these works, [9] develops ﬁxed conﬁdencealgorithms (up to a desired precision (cid:15) > 0 ) both for the RL setting and for the bandit one.

Original: However,their setting is signiﬁcantly different from ours as they do not consider each ﬁdelity to be related to agiven cost.
Revised: However,their setting is signiﬁcantly different from ours as they do not consider each ﬁdelity to be related to agiven cost.

Original: Instead, they simply aim at minimizing the number of sub-optimal steps at ﬁdelity M ,while controlling the number of samples taken in ﬁdelity m <
Revised: Instead, they simply aim at minimizing the number of sub-optimal steps at ﬁdelity M ,while controlling the number of samples taken in ﬁdelity m <

Original: M to ensure they are polynomial.
Revised: M to ensure they are polynomial.

Original:  Numerical Validation
Revised:  Numerical Validation

Original: In this section, we provide a numerical validation of ourtheoretical claims in both synthetic andsimulated domains.
Revised: In this section, we provide a numerical validation of our claims in both synthetic and simulateddomains.

Original: To prove that IISE is successful at exploiting ﬁdelity, we compare it with: (i) 
Revised: To prove that IISE is successful at exploiting ﬁdelity, we compare it with: (i) Successive

Original: Successive Elimination (SE) [10], which exploits only maximum ﬁdelity, and with a version of IISEthat chooses the thresholds α m so as to exploit every ﬁdelity as much as possible (i.e., α m close to 0).
Revised: Elimination (SE) [10], which exploits only maximum ﬁdelity, and with a version of IISE that choosesthe thresholds α m so as to exploit every ﬁdelity as much as possible (i.e., α m close to 0).

Original: We call this modiﬁed version Maximum Fidelity Exploit (MFE) algorithm.
Revised: We call thismodiﬁed version Maximum Fidelity Exploit (MFE) algorithm.

Original: Table 1 reports all the 
Revised: Table 1 reports all the results in terms

Original: [Table]
Revised: [Table]

Original: results in terms of the percentage of cost complexity of a given algorithm w.r.t.
Revised: of the percentage of cost complexity of a given algorithm w.r.t.

Original: the one of Successive Elimination.
Revised: the one of Successive Elimination.

Original: The value 100 corresponds to the relative cost complexity of SE.
Revised: The value 100 corresponds to the relative cost complexity of SE.

Original: Synthetic bandits.
Revised: Synthetic bandits.

Original: We start presenting some experiments on two synthetic bandits (Synthetic Aand Synthetic B) with randomly generated arms.
Revised: We start presenting some experiments on two synthetic bandits (Synthetic Aand Synthetic B) with randomly generated arms.

Original: These experiments aim at validating our robustnessresults along with the beneﬁt of the order-aware knowledge.
Revised: These experiments aim at validating our robustnessresults along with the beneﬁt of the order-aware knowledge.

Original: Synthetic A setting parameters are K =2000 , M = 4 , λ = [1 , 10 , 100 , 1000] , ξ =
Revised: Synthetic A setting parameters are K =2000 , M = 4 , λ = [1 , 10 , 100 , 1000] , ξ =

Original: [1 . 15 , 0 . 225 , 0 . 015 , 0] , γ = [0 . 3 , 0 .
Revised: [1 . 15 , 0 . 225 , 0 . 015 , 0] , γ = [0 . 3 , 0 .

Original: 001 , 0] ; for Synthetic B, instead, K = 1000 , M = 5 , λ
Revised: 001 , 0] ; for Synthetic B, instead, K = 1000 , M = 5 , λ

Original: =
Revised: =

Original: [16 , 64 , 256 , 1024 , 4096] , ξ = [1 . 15 , 0 .
Revised: [16 , 64 , 256 , 1024 , 4096] , ξ = [1 . 15 , 0 .

Original: 0105 , 0] ,γ = [0 . 3 , 0 . 1 , 0 .
Revised: 0105 , 0] ,γ = [0 . 3 , 0 . 1 , 0 .

Original: To make the γ and ξ settings directly comparable, we generated the armssuch that the ﬁdelity index m is left unchanged (i.e., it simultaneously holds that λ 1 > λ 2 > · · · > λ M ,ξ 1 > ξ 2 > · · · > ξ M , and γ 1 > γ 2 > · · · > γ M ).
Revised: To make the γ and ξ settings directly comparable, we generated the armssuch that the ﬁdelity index m is left unchanged (i.e., it simultaneously holds that λ 1 > λ 2 > · · · > λ M ,ξ 1 > ξ 2 > · · · > ξ M , and γ 1 > γ 2 > · · · > γ M ).

Original: Further details on this generation process can be found 339 in Appendix E. As we can appreciate from Table 1, IISE substantially reduces the cost complexityw.r.t. SE, and, its order-aware version IISE- γ , obtains further improvements.
Revised: Further details on this generation process can be foundin Appendix D. As we can appreciate from Table 1, IISE substantially reduces the cost complexityw.r.t SE, and, its order-aware version IISE- γ , obtains further improvements.

Original: Table 1 also highlightsthe importance of choosing a good threshold α m .
Revised: Table 1 also highlightsthe importance of choosing a good threshold α m .

Original: Indeed, MFE insists in exploiting a given ﬁdelity,even when there is no convenience in doing so, thus obtaining signiﬁcantly worse results than SE.
Revised: Indeed, MFE insists in exploiting a given ﬁdelity,even when there is no convenience in doing so, thus obtaining signiﬁcantly worse results than SE.

Original: Yahtzee.
Revised: Yahtzee.

Original: We now apply our algorithm to the problem of choosing the ﬁrst action in the stochasticplanning domain introduced in Section 4.
Revised: We now apply our algorithm to the problem of choosing the ﬁrst action in the stochasticplanning domain introduced in Section 4.

Original: More speciﬁcally, we consider the Yahtzee game [3].
Revised: More speciﬁcally, we consider the Yahtzee game [3].

Original: Yahtzee is a sequential dice game in which a player rolls 5 dice up to 3 times and then chooses aparticular move that assigns a score to the ﬁnal dice combination.
Revised: Yahtzee is a sequential dice game in which a player rolls 5 dice up to 3 times and then chooses aparticular move that assigns a score to the ﬁnal dice combination.

Original: For instance, typical moves are:“Sixes” with a score given by the sum of dice with the number 6 , and “Yahtzee”, that assigns 50 pointsif all the dice show the same number, and 0 otherwise.
Revised: For instance, typical moves are:“Sixes” with a score given by the sum of dice with the number 6 , and “Yahtzee”, that assigns 50 pointsif all the dice show the same number, and 0 otherwise.

Original: The total number of these possible moves is14 (i.e., our branching factor Λ ).
Revised: The total number of these possible moves is14 (i.e., our branching factor Λ ).

Original: The game proceeds in 13 rounds.
Revised: The game proceeds in 13 rounds.

Original: During each round t , the playerselects a move and obtains the corresponding points.
Revised: During each round t , the playerselects a move and obtains the corresponding points.

Original: We consider the variant of Yathzee in which,if the player has already selected a certain move at t < t , then, at t , that move will lead to reward 0 .
Revised: We consider the variant of Yathzee in which,if the player has already selected a certain move at t < t , then, at t , that move will lead to reward 0 .

Original: Furthermore, during each round, the player will roll the 5 dice only once, after which a move has tobe selected.
Revised: Furthermore, during each round, the player will roll the 5 dice only once, after which a move has tobe selected.

Original: Further details are provided in Appendix E. Table 1 shows that our method signiﬁcantlyreduces the total number of generated nodes, thus lowering successfully the total cost of identifyingthe best action, and, that exploiting the upper bounds on the maximum bias variation instead of ξ ’s isbeneﬁcial in terms of total cost.
Revised: Further details are provided in Appendix D. Table 1 shows that our method signiﬁcantlyreduces the total number of generated nodes, thus lowering successfully the total cost of identifyingthe best action, and, that exploiting the upper bounds on the maximum bias variation instead of ξ ’s isbeneﬁcial in terms of total cost.

Original: Finally, MFE overexploits ﬁdelity and underperforms SE.
Revised: Finally, MFE overexploits ﬁdelity and underperforms SE.

Original:  Conclusions
Revised:  Conclusions

Original: In this work, we have studied, for the ﬁrst time, the variant of the ﬁxed-conﬁdence BAI scenario inwhich the agent has access to multiple approximations with different costs and biases of the originalbandit model.
Revised: In this work, we have studied, for the ﬁrst time, the variant of the ﬁxed-conﬁdence BAI scenario inwhich the agent has access to multiple approximations with different costs and biases of the originalbandit model.

Original: We proposed the novel notion of cost complexity to evaluate the performance that agiven algorithm obtains in these situations.
Revised: We proposed the novel notion of cost complexity to evaluate the performance that agiven algorithm obtains in these situations.

Original: We derived a lower bound on this index, which showsseveral interesting properties of the problem, along with its main challenge.
Revised: We derived a lower bound on this index, which showsseveral interesting properties of the problem, along with its main challenge.

Original: We also presented anovel algorithm, IISE, which provably reduces the cost complexity compared to algorithms thatignore the availability of multiple ﬁdelity.
Revised: We also presented anovel algorithm, IISE, which provably reduces the cost complexity compared to algorithms thatignore the availability of multiple ﬁdelity.

Original: More speciﬁcally, IISE discards a sub-optimal arm i withthe most convenient, in terms of cost and ﬁdelity, thus closely mimicking the shape of the lowerbound.
Revised: More speciﬁcally, IISE discards a sub-optimal arm i withthe most convenient, in terms of cost and ﬁdelity, thus closely mimicking the shape of the lowerbound.

Original: Furthermore, we also shown that the maximum bias is not what actually matters in identifyingthe best arm; indeed, better guarantees can be obtained with the knowledge of the maximum biasvariation, an original concept that we introduce to encode how much a given ﬁdelity preserves order.
Revised: Furthermore, we also shown that the maximum bias is not what actually matters in identifyingthe best arm; indeed, better guarantees can be obtained with the knowledge of the maximum biasvariation, an original concept that we introduce to encode how much a given ﬁdelity preserves order.

Original: To conclude, we note that previous studies [45, 18, 1] highlighted the strong connections betweenﬁxed-conﬁdence BAI and learning near-optimal RL policies while minimizing the number of samplesgathered in the environment.
Revised: To conclude, we note that previous studies [46, 17, 1] highlighted the strong connections betweenﬁxed-conﬁdence BAI and learning near-optimal RL policies while minimizing the number of samplesgathered in the environment.

Original: In this sense, our ideas pave the way for future work that aims atlearning these policies when multiple simulators are available.
Revised: In this sense, our ideas pave the way for future work that aims atlearning these policies when multiple simulators are available.

Original: Indeed, previous efforts in MF-RL[9, 30, 41] do not directly take the challenging cost-precision trade-off into account.
Revised: Indeed, previous efforts in MF-RL[9, 29, 42] do not directly take the challenging cost-precision trade-off into account.

Original: Checklist
Revised: Checklist

Original: The checklist follows the references.
Revised: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No], or[N/A] .
Revised: For each question, change the default [TODO] to [Yes] , [No] or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
Revised: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:
Revised: For example:

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [Yes] See Section ?? .
Revised: [Yes] See Section ?? .

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [No] The code and the data are
Revised: [No] The code and the data are

Original: proprietary.
Revised: proprietary.

Original: • Did you include the license to the code and datasets?
Revised: • Did you include the license to the code and datasets?

Original: [N/A]
Revised: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.
Revised: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the
Revised: Note that the

Original: Checklist section does not count towards the page limit.
Revised: Checklist section does not count towards the page limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...
Revised: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
Revised: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?
Revised: contributions and scope?

Original: [Yes] See for instance the contributions subsection.
Revised: [Yes] See for instance the contributions subsection.

Original: It list,
Revised: It list,

Original: one by one, the content of each of the section of the paper.
Revised: one by one, the content of each of the section of the paper.

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 4.
Revised: [Yes] See Section 4.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Did not
Revised: [No] Did not

Original: ﬁnd any.
Revised: ﬁnd any.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] We stated
Revised: [Yes] We stated

Original: formally all the theoretical results throughout the main text (i.e., Theorem 1, Theorem2, Theorem 3, Proposition 1).
Revised: formally all the theoretical results throughout the main text (i.e., Theorem 1, Theorem2, Theorem 3, Proposition 1).

Original: We highlighted the set of assumptions in each claim.
Revised: We highlighted the set of assumptions in each claim.

Original: For
Revised: For

Original: extensions (e.g. Equation (9) we provide formal statements in the Appendix, but set of
Revised: extensions (e.g. Equation (9) we provide formal statements in the Appendix, but set of

Original: assumptions has been speciﬁed also in the main text.
Revised: assumptions has been speciﬁed also in the main text.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] We provide a detailed
Revised: [Yes] We provide a detailed

Original: appendix with complete proofs of all theoretical results.
Revised: appendix with complete proofs of all theoretical results.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experi-
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Original: mental results (either in the supplemental material or as a URL)?
Revised: mental results (either in the supplemental material or as a URL)?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters,
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters,

Original: how they
Revised: how they

Original: were chosen)?
Revised: were chosen)?

Original: [Yes] Experiments are detailed in Section 7 and further documented in
Revised: [Yes] Experiments are detailed in Section 7 and further documented in

Original: Appendix E. Further details can be found within the codebase.
Revised: Appendix D. Further details can be found within the codebase.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running exper-
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running exper-

Original: iments multiple times)?
Revised: iments multiple times)?

Original: [Yes] As speciﬁed in Section 7, we report mean and
Revised: [Yes] As speciﬁed in Section 7, we report mean and

Original: conﬁdence intervals using 100 runs.
Revised: conﬁdence intervals using 100 runs.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes] Yes, see Appendix E.
Revised: [Yes] Yes, see Appendix D.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]
Revised: [N/A]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [N/A]
Revised: [N/A]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: information or offensive content?
Revised: information or offensive content?

Original: [N/A]
Revised: [N/A]

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]