Original: Learning on the Edge: Online Learning with Stochastic Feedback Graphs
Revised: Learning on the Edge: Online Learning with Stochastic Feedback Graphs

Original: Abstract
Revised: Abstract

Original: The framework of feedback graphs is a generalization of sequential decisionmaking with bandit or full information feedback.
Revised: The framework of feedback graphs is a generalization of sequential decisionmaking with bandit or full information feedback.

Original: In this work, we study an extension where the directed feedback graph is stochastic, following a distribution similar to the classical Erdős-Rényi model.
Revised: In this work, we study an extension where the directed feedback graph is stochastic, following a distribution similar to the classical Erdős-Rényi model.

Original: Specifically, in each round every edge in the graph is either realized or not with a distinct probability for each edge.
Revised: Specifically, in each round every edge in the graph is either realized or not with a distinct probability for each edge.

Original: We prove nearly optimal regret bounds of order min (cid:8) min ε p ( α ε /ε )
Revised: We prove nearly optimal regret bounds of order min (cid:8) min ε p ( α ε /ε )

Original: T, min ε ( δ ε /ε ) 1 / 3 T 2 / 3 (cid:9) (ignoring logarithmic factors), where α ε and δ ε are graph-theoretic quantities measured on the support of the stochastic feedback graph G with edge probabilities thresholded at ε .
Revised: T, min ε ( δ ε /ε ) 1 / 3 T 2 / 3 (cid:9) (ignoring logarithmic factors), where α ε and δ ε are graph-theoretic quantities measured on the support of the stochastic feedback graph G with edge probabilities thresholded at ε .

Original: Our result, which holds without any preliminary knowledge about G , requires the learner to observe only the realized out-neighborhood of the chosen action.
Revised: Our result, which holds without any preliminary knowledge about G , requires the learner to observe only the realized out-neighborhood of the chosen action.

Original: When the learner is allowed to observe the realization of the entire graph (but only the losses in the out-neighborhood of the chosen action), we derive a more efficient algorithm featuring a dependence on weighted versions of the independence and weak domination numbers that exhibits improved bounds for some special cases.
Revised: When the learner is allowed to observe the realization of the entire graph (but only the losses in the out-neighborhood of the chosen action), we derive a more efficient algorithm featuring a dependence on weighted versions of the independence and weak domination numbers that exhibits improved bounds for some special cases.

Original:  Introduction
Revised:  Introduction

Original: In this work we study an online learning framework for decision-making with partial feedback.
Revised: In this work we study an online learning framework for decision-making with partial feedback.

Original: In each decision round, the learner chooses an action in a fixed set and is charged a loss.
Revised: In each decision round, the learner chooses an action in a fixed set and is charged a loss.

Original: In our setting, the loss of any action in all decision rounds is preliminarily chosen by an adversary, but the feedback received by the learner at the end of each round t is stochastic.
Revised: In our setting, the loss of any action in all decision rounds is preliminarily chosen by an adversary, but the feedback received by the learner at the end of each round t is stochastic.

Original: More specifically, the loss of each action i (including I t , the one selected by the learner at round t ) is independently observed with a certain probability p
Revised: More specifically, the loss of each action i (including I t , the one selected by the learner at round t ) is independently observed with a certain probability p

Original: ( I t , i ), where the probabilities p ( i, j ) for all pairs i, j are fixed but unknown.
Revised: ( I t , i ), where the probabilities p ( i, j ) for all pairs i, j are fixed but unknown.

Original: This feedback model can be viewed as a stochastic version of the feedback graph model for online learning [Mannor and Shamir, 2011], where the feedback received by the learner at the end of each round is determined by a directed graph defined over the set of actions.
Revised: This feedback model can be viewed as a stochastic version of the feedback graph model for online learning [Mannor and Shamir, 2011], where the feedback received by the learner at the end of each round is determined by a directed graph defined over the set of actions.

Original: In this model, the learner deterministically observes the losses of all the actions in the out-neighborhood of the action selected in that round.
Revised: In this model, the learner deterministically observes the losses of all the actions in the out-neighborhood of the action selected in that round.

Original: In certain applications, however, deterministic feedback is not realistic.
Revised: In certain applications, however, deterministic feedback is not realistic.

Original: Consider for instance a sensor network for monitoring the environment, where the learner can decide which sensor to probe in order to maximize some performance measure.
Revised: Consider for instance a sensor network for monitoring the environment, where the learner can decide which sensor to probe in order to maximize some performance measure.

Original: Each probed sensor may also receive readings of other sensors, but whether a sensor successfully transmits to another sensor depends on a number of environmental factors, which include the position of the two sensors, but also their internal state (e.g., battery levels) and the weather conditions.
Revised: Each probed sensor may also receive readings of other sensors, but whether a sensor successfully transmits to another sensor depends on a number of environmental factors, which include the position of the two sensors, but also their internal state (e.g., battery levels) and the weather conditions.

Original: Due to the variability of some of these factors, the possibility of reading from another sensor can be naturally modeled as a stochastic event.
Revised: Due to the variability of some of these factors, the possibility of reading from another sensor can be naturally modeled as a stochastic event.

Original: Online learning with adversarial losses and stochastic feedback graphs has been studied before, but under fairly restrictive assumptions on the probabilities p ( i, j ).
Revised: Online learning with adversarial losses and stochastic feedback graphs has been studied before, but under fairly restrictive assumptions on the probabilities p ( i, j ).

Original: Let G be a stochastic feedback graph, represented by its probability matrix p ( i, j ) for i, j ∈ V where
Revised: Let G be a stochastic feedback graph, represented by its probability matrix p ( i, j ) for i, j ∈ V where

Original: V is the action set.
Revised: V is the action set.

Original: When p ( i, j ) =
Revised: When p ( i, j ) =

Original: ε for all distinct i, j ∈ V and for some ε > 0, then G follows the Erdős-Rényi random graph model.
Revised: ε for all distinct i, j ∈ V and for some ε > 0, then G follows the Erdős-Rényi random graph model.

Original: Under the assumption that ε is known and p ( i, i ) =
Revised: Under the assumption that ε is known and p ( i, i ) =

Original: 1 for all i ∈ V (all self-loops occur w.p. 1), Alon et al.
Revised: 1 for all i ∈ V (all self-loops occur w.p. 1), Alon et al.

Original: [2017] show that the optimal regret after T rounds is of order p T/ε , up to logarithmic factors.
Revised: [2017] show that the optimal regret after T rounds is of order p T/ε , up to logarithmic factors.

Original: This result has been extended by Kocák et al.
Revised: This result has been extended by Kocák et al.

Original: [2016a], who prove a regret bound of order pP t (1 /ε t ) when the parameter ε t of the random graph is unknown and allowed to change over time.
Revised: [2016a], who prove a regret bound of order pP t (1 /ε t ) when the parameter ε t of the random graph is unknown and allowed to change over time.

Original: However, their result holds only under rather strong assumptions on the sequence ε t for t ≥ 1.
Revised: However, their result holds only under rather strong assumptions on the sequence ε t for t ≥ 1.

Original: In a recent work, Ghari and Shen [2022] show a regret bound of order ( α/ε ) √ KT , ignoring logarithmic factors, when each (unknown) probability p ( i, j ) in G is either zero or at least ε for some known ε > 0, and all self-loops ( i, i ) have probability p ( i, i ) ≥ ε .
Revised: In a recent work, Ghari and Shen [2022] show a regret bound of order ( α/ε ) √ KT , ignoring logarithmic factors, when each (unknown) probability p ( i, j ) in G is either zero or at least ε for some known ε > 0, and all self-loops ( i, i ) have probability p ( i, i ) ≥ ε .

Original: Here α is the independence number (computed ignoring edge orientations) of the support graph supp( G ); i.e., the directed graph with adjacency matrix A ( i, j ) =
Revised: Here α is the independence number (computed ignoring edge orientations) of the support graph supp( G ); i.e., the directed graph with adjacency matrix A ( i, j ) =

Original: I { p ( i,j ) > 0 } .
Revised: I { p ( i,j ) > 0 } .

Original: Their bound holds under the assumption that supp( G ) is preliminarily known to the learner.
Revised: Their bound holds under the assumption that supp( G ) is preliminarily known to the learner.

Original: Our analysis does away with a crucial assumption that was key to prove all previous results.
Revised: Our analysis does away with a crucial assumption that was key to prove all previous results.

Original: Namely, we do not assume any special property of the matrix G , and we do not require the learner to have any preliminary knowledge of this matrix.
Revised: Namely, we do not assume any special property of the matrix G , and we do not require the learner to have any preliminary knowledge of this matrix.

Original: The fact that positive edge probabilities are not bounded away from zero implies that the learner must choose a threshold ε ∈ (0 , 1] below which the edges are deemed to be too rare to be exploitable for learning.
Revised: The fact that positive edge probabilities are not bounded away from zero implies that the learner must choose a threshold ε ∈ (0 , 1] below which the edges are deemed to be too rare to be exploitable for learning.

Original: If ε is too small, then waiting for rare edges slows down learning.
Revised: If ε is too small, then waiting for rare edges slows down learning.

Original: On the other hand, if ε is too large, then the feedback becomes sparse and the regret increases.
Revised: On the other hand, if ε is too large, then the feedback becomes sparse and the regret increases.

Original: To formalize the intuition of rare edges, we introduce the notion of thresholded graph supp( G ε ) for any ε > 0.
Revised: To formalize the intuition of rare edges, we introduce the notion of thresholded graph supp( G ε ) for any ε > 0.

Original: This is the directed graph with adjacency matrix A ( i, j ) =
Revised: This is the directed graph with adjacency matrix A ( i, j ) =

Original: I { p ( i,j ) ≥ ε } .
Revised: I { p ( i,j ) ≥ ε } .

Original: As the thresholded graph is a deterministic feedback graph G , we can refer to Alon et al.
Revised: As the thresholded graph is a deterministic feedback graph G , we can refer to Alon et al.

Original: [2015] for a characterization of minimax regret R T based on whether G is not observable ( R T of order
Revised: [2015] for a characterization of minimax regret R T based on whether G is not observable ( R T of order

Original: T ), weakly observable ( R T of order δ 1 / 3 T 2 / 3 ), or strongly observable ( R T of order √ αT ).
Revised: T ), weakly observable ( R T of order δ 1 / 3 T 2 / 3 ), or strongly observable ( R T of order √ αT ).

Original: Here α and δ are, respectively, the independence and the weak domination number of G ; see Section 2 for definitions.
Revised: Here α and δ are, respectively, the independence and the weak domination number of G ; see Section 2 for definitions.

Original: Let α ε and δ ε respectively denote the independence number and the weak domination number of supp( G ε ).
Revised: Let α ε and δ ε respectively denote the independence number and the weak domination number of supp( G ε ).

Original: As α ε and δ ε both grow when ε gets larger, the ratios α ε /ε and δ ε /ε capture the trade-off involved in choosing ε .
Revised: As α ε and δ ε both grow when ε gets larger, the ratios α ε /ε and δ ε /ε capture the trade-off involved in choosing ε .

Original: We define the optimal values for ε as follows:
Revised: We define the optimal values for ε as follows:

Original: [Equation]
Revised: [Equation]

Original: We adopt the convention that the minimum of an empty set is infinity and the relative arg min is set to 0.
Revised: We adopt the convention that the minimum of an empty set is infinity and the relative arg min is set to 0.

Original: The arg min are well defined: there are at most K 2 values of ε for which the support of G ε varies, and the minimum is attained in one of these values.
Revised: The arg min are well defined: there are at most K 2 values of ε for which the support of G ε varies, and the minimum is attained in one of these values.

Original: For simplicity, we let α ∗ =
Revised: For simplicity, we let α ∗ =

Original: α ε ∗ s and δ ∗ =
Revised: α ε ∗ s and δ ∗ =

Original: δ ε ∗ w .
Revised: δ ε ∗ w .

Original: Our first result can be informally stated as follows.
Revised: Our first result can be informally stated as follows.

Original: Theorem 1 (Informal) .
Revised: Theorem 1 (Informal) .

Original: Consider the problem of online learning with an unknown stochastic feedback graph G on T time steps.
Revised: Consider the problem of online learning with an unknown stochastic feedback graph G on T time steps.

Original: If supp( G ε ) is not observable for ε = ˜Θ( K 3 /T ) , then any learning algorithm suffers regret linear in T .
Revised: If supp( G ε ) is not observable for ε = ˜Θ( K 3 /T ) , then any learning algorithm suffers regret linear in T .

Original: Otherwise, there exists an algorithm whose regret satisfies (ignoring polylog factors in K and T )
Revised: Otherwise, there exists an algorithm whose regret satisfies (ignoring polylog factors in K and T )

Original: [Equation]
Revised: [Equation]

Original: This bound is tight (up to polylog factors).
Revised: This bound is tight (up to polylog factors).

Original: This result shows that, without any preliminary knowledge of G , we can obtain a bound that optimally trades off between the strongly observable rate p ( α ∗ /ε ∗ s ) T , for the best threshold ε for which supp( G ε ) is strongly observable, and the (weakly) observable rate ( δ ∗ /ε ∗ w ) 1 / 3 T 2 / 3 , for the best threshold ε for which supp( G ε ) is (weakly) observable.
Revised: This result shows that, without any preliminary knowledge of G , we can obtain a bound that optimally trades off between the strongly observable rate p ( α ∗ /ε ∗ s ) T , for the best threshold ε for which supp( G ε ) is strongly observable, and the (weakly) observable rate ( δ ∗ /ε ∗ w ) 1 / 3 T 2 / 3 , for the best threshold ε for which supp( G ε ) is (weakly) observable.

Original: Note that this result improves on Ghari and Shen [2022] bound ( α ε /ε ) √ KT , who additionally assume that supp( G ε ) and ε (a lower bound on the self-loop probabilities) are both preliminarily available to the learner.
Revised: Note that this result improves on Ghari and Shen [2022] bound ( α ε /ε ) √ KT , who additionally assume that supp( G ε ) and ε (a lower bound on the self-loop probabilities) are both preliminarily available to the learner.

Original: On the other hand, the algorithm achieving the bound of Theoremneed not receive any information (neither prior nor during the learning process) besides the stochastic feedback.
Revised: On the other hand, the algorithm achieving the bound of Theoremneed not receive any information (neither prior nor during the learning process) besides the stochastic feedback.

Original: We obtain positive results in Theorem 1 via an elaborate reduction to online learning with deterministic feedback graphs.
Revised: We obtain positive results in Theorem 1 via an elaborate reduction to online learning with deterministic feedback graphs.

Original: Our algorithm works in two phases: first, it learns the edge probabilities in a round-robin procedure, then it commits to a carefully chosen estimate of the feedback graph and feeds it to an algorithm for online learning with deterministic feedback graphs.
Revised: Our algorithm works in two phases: first, it learns the edge probabilities in a round-robin procedure, then it commits to a carefully chosen estimate of the feedback graph and feeds it to an algorithm for online learning with deterministic feedback graphs.

Original: There are two main technical challenges the algorithm faces: on the one hand, it needs to switch from the first to the second phase at the right time in order to achieve the optimal regret.
Revised: There are two main technical challenges the algorithm faces: on the one hand, it needs to switch from the first to the second phase at the right time in order to achieve the optimal regret.

Original: On the other hand, in order for the reduction to work, it needs to simulate the behaviour of a deterministic feedback graph using only feedback from a stochastic feedback graph (with unknown edge probabilities).
Revised: On the other hand, in order for the reduction to work, it needs to simulate the behaviour of a deterministic feedback graph using only feedback from a stochastic feedback graph (with unknown edge probabilities).

Original: We complement the positive results in Theorem 1 with matching lower bounds that are obtained by a suitable modification of the hard instances in Alon et al.
Revised: We complement the positive results in Theorem 1 with matching lower bounds that are obtained by a suitable modification of the hard instances in Alon et al.

Original: so as to consider stochastic feedback graphs.
Revised: so as to consider stochastic feedback graphs.

Original: Our last result is an algorithm that, at the cost of an additional assumption on the feedback (i.e., the learner additionally observes the realization of the entire feedback graph at the end of each round), has regret which is never worse and may be considerably better than the regret of the algorithm in Theorem 1.
Revised: Our last result is an algorithm that, at the cost of an additional assumption on the feedback (i.e., the learner additionally observes the realization of the entire feedback graph at the end of each round), has regret which is never worse and may be considerably better than the regret of the algorithm in Theorem 1.

Original: While the bounds in Theorem 1 are tight up to log factors, we show that the factors α ∗ /ε ∗ s and δ ∗ /ε
Revised: While the bounds in Theorem 1 are tight up to log factors, we show that the factors α ∗ /ε ∗ s and δ ∗ /ε

Original: ∗ w can be improved for specific feedback graphs.
Revised: ∗ w can be improved for specific feedback graphs.

Original: Specifically, we design weighted versions of the independence and weak domination numbers, where the weights of a given node depend on the probabilities of seeing the loss of that node.
Revised: Specifically, we design weighted versions of the independence and weak domination numbers, where the weights of a given node depend on the probabilities of seeing the loss of that node.

Original: On the technical side, we design a new importance-weighted estimator which uses a particular version of upper confidence bound estimates of the edge probabilities p ( i, j ), rather than the true edge probabilities, which are unknown.
Revised: On the technical side, we design a new importance-weighted estimator which uses a particular version of upper confidence bound estimates of the edge probabilities p ( i, j ), rather than the true edge probabilities, which are unknown.

Original: We show that the cost of using this estimator is of the same order as the regret bound achievable had we known p ( i, j ).
Revised: We show that the cost of using this estimator is of the same order as the regret bound achievable had we known p ( i, j ).

Original: Additionally, the algorithm that obtains these improved bounds is more efficient than the algorithm of Theorem 1.
Revised: Additionally, the algorithm that obtains these improved bounds is more efficient than the algorithm of Theorem 1.

Original: The improvement in efficiency comes from the following idea: we start with an optimistic algorithm that assumes that the support of G is strongly observable and only switches to the assumption that the support of G is (weakly) observable when it estimates that the regret under this second assumption is smaller.
Revised: The improvement in efficiency comes from the following idea: we start with an optimistic algorithm that assumes that the support of G is strongly observable and only switches to the assumption that the support of G is (weakly) observable when it estimates that the regret under this second assumption is smaller.

Original: The algorithm learns which regime is better by keeping track of a bound on the regret of the optimistic algorithm while simultaneously estimating the regret in the (weakly) observable case, which it can do efficiently.
Revised: The algorithm learns which regime is better by keeping track of a bound on the regret of the optimistic algorithm while simultaneously estimating the regret in the (weakly) observable case, which it can do efficiently.

Original: Additional related work.
Revised: Additional related work.

Original: The problem of adversarial online learning with feedback graphs was introduced by Mannor and Shamir [2011], in the special case where all nodes in the feedback graph have self-loops.
Revised: The problem of adversarial online learning with feedback graphs was introduced by Mannor and Shamir [2011], in the special case where all nodes in the feedback graph have self-loops.

Original: The results of Alon et al.
Revised: The results of Alon et al.

Original: [2015] (also based on prior work by Alon et al.
Revised: [2015] (also based on prior work by Alon et al.

Original: [2013], Kocák et al.
Revised: [2013], Kocák et al.

Original: [2014]) have been recently slightly improved by Chen et al.
Revised: [2014]) have been recently slightly improved by Chen et al.

Original: [2021], with tighter constants in the regret bound.
Revised: [2021], with tighter constants in the regret bound.

Original: Variants of the adversarial setting have been studied by Feng and Loh [2018], Arora et al.
Revised: Variants of the adversarial setting have been studied by Feng and Loh [2018], Arora et al.

Original: [2019], Rangi and Franceschetti [2019] and Van der Hoeven et al.
Revised: [2019], Rangi and Franceschetti [2019] and Van der Hoeven et al.

Original: [2021], who study online learning with feedback graphs and switching costs and online multiclass classification with feedback graphs, respectively.
Revised: [2021], who study online learning with feedback graphs and switching costs and online multiclass classification with feedback graphs, respectively.

Original: There is also a considerable amount of work in the stochastic setting [Liu et al., 2018, Cortes et al., 2019, Li et al., 2020].
Revised: There is also a considerable amount of work in the stochastic setting [Liu et al., 2018, Cortes et al., 2019, Li et al., 2020].

Original: Finally, Rouyer et al.
Revised: Finally, Rouyer et al.

Original: [2022] and Ito et al.
Revised: [2022] and Ito et al.

Original: [2022] independently designed different best-of-both-worlds learning algorithms achieving nearly optimal (up to polylogarithmic factors in T ) regret bounds in the adversarial and stochastic settings.
Revised: [2022] independently designed different best-of-both-worlds learning algorithms achieving nearly optimal (up to polylogarithmic factors in T ) regret bounds in the adversarial and stochastic settings.

Original: Following Mannor and Shamir [2011], we can consider a more general scenario where the feedback graph is not fixed but changes over time, resulting in a sequence G 1 , . .
Revised: Following Mannor and Shamir [2011], we can consider a more general scenario where the feedback graph is not fixed but changes over time, resulting in a sequence G 1 , . .

Original: , G T of feedback graphs.
Revised: , G T of feedback graphs.

Original: Cohen et al.
Revised: Cohen et al.

Original: [2016] study a setting where the graphs are adversarially chosen and only the local structure of the feedback graph is observed.
Revised: [2016] study a setting where the graphs are adversarially chosen and only the local structure of the feedback graph is observed.

Original: They show that, if the losses are generated by an adversary and all nodes always have a self-loop, one cannot do better than √ KT regret, and we might as well simply employ a standard bandit algorithm.
Revised: They show that, if the losses are generated by an adversary and all nodes always have a self-loop, one cannot do better than √ KT regret, and we might as well simply employ a standard bandit algorithm.

Original: Furthermore, removing the guarantee on the self-loops induces an Ω( T ) regret.
Revised: Furthermore, removing the guarantee on the self-loops induces an Ω( T ) regret.

Original: In Section 3, we are in a similar situation, as we also observe only local information about the feedback graph and the losses are generated by an adversary.
Revised: In Section 3, we are in a similar situation, as we also observe only local information about the feedback graph and the losses are generated by an adversary.

Original: However, we show that if the graphs are stochastically generated with a strongly observable support for some threshold ε , there is a p αT/ε regret bound.
Revised: However, we show that if the graphs are stochastically generated with a strongly observable support for some threshold ε , there is a p αT/ε regret bound.

Original: As a consequence, for ε not too small, observing only the local information about the feedback graphs is in fact sufficient to obtain better results than in the bandit setting.
Revised: As a consequence, for ε not too small, observing only the local information about the feedback graphs is in fact sufficient to obtain better results than in the bandit setting.

Original: Similarly, if there are no self-loops in the support but the support is weakly observable, then our regret bounds are sublinear rather than linear in T .
Revised: Similarly, if there are no self-loops in the support but the support is weakly observable, then our regret bounds are sublinear rather than linear in T .

Original: Alon et al.
Revised: Alon et al.

Original: [2013, 2017] and Kocák et al.
Revised: [2013, 2017] and Kocák et al.

Original: [2014] also consider adversarially generated sequences
Revised: [2014] also consider adversarially generated sequences

Original: G 1 , G 2 , . .
Revised: G 1 , G 2 , . .

Original: of deterministic feedback graphs.
Revised: of deterministic feedback graphs.

Original: In the case of directed feedback graphs, Alon et al.
Revised: In the case of directed feedback graphs, Alon et al.

Original: [2013] investigate a model in which G t is revealed to the learner at the beginning of each round t .
Revised: [2013] investigate a model in which G t is revealed to the learner at the beginning of each round t .

Original: Alon et al.
Revised: Alon et al.

Original: [2017] and Kocák et al.
Revised: [2017] and Kocák et al.

Original: [2014] extend this analysis to the case when
Revised: [2014] extend this analysis to the case when

Original: G t is strongly observable and made available only at the end of each round t .
Revised: G t is strongly observable and made available only at the end of each round t .

Original: In comparison, in our setting the graphs (or the local information about the graph) revealed to the learner (at the end of each round) may not even be observable, let alone strongly observable.
Revised: In comparison, in our setting the graphs (or the local information about the graph) revealed to the learner (at the end of each round) may not even be observable, let alone strongly observable.

Original: Despite this seemingly challenging setting for previous works, we nevertheless obtain sublinear regret bounds.
Revised: Despite this seemingly challenging setting for previous works, we nevertheless obtain sublinear regret bounds.

Original: Finally, Kocák et al.
Revised: Finally, Kocák et al.

Original: [2016b] study a feedback model where the losses of other actions in the out-neighborhood of the action played are observed with an edge-dependent noise.
Revised: [2016b] study a feedback model where the losses of other actions in the out-neighborhood of the action played are observed with an edge-dependent noise.

Original: In their setting, the feedback graphs G t are weighted and revealed at the beginning of each round.
Revised: In their setting, the feedback graphs G t are weighted and revealed at the beginning of each round.

Original: They introduce edge weights s t ( i, j ) ∈ [0 , 1] that determine the feedback according to the following additive noise model: s t ( I t , j ) ℓ t ( j ) + (1 − s t
Revised: They introduce edge weights s t ( i, j ) ∈ [0 , 1] that determine the feedback according to the following additive noise model: s t ( I t , j ) ℓ t ( j ) + (1 − s t

Original: ( I t , j ))
Revised: ( I t , j ))

Original: ξ t ( j ), where ξ t ( j ) is a zero-mean bounded random variable.
Revised: ξ t ( j ), where ξ t ( j ) is a zero-mean bounded random variable.

Original: Hence, if s t ( i, j ) = 1, then I t =
Revised: Hence, if s t ( i, j ) = 1, then I t =

Original: i allows to observe the loss of action j without any noise.
Revised: i allows to observe the loss of action j without any noise.

Original: If s t ( i, j ) = 0, then only noise is observed.
Revised: If s t ( i, j ) = 0, then only noise is observed.

Original: Note that they assume s t ( i, i )
Revised: Note that they assume s t ( i, i )

Original: = 1 for each i , implying strong observability.
Revised: = 1 for each i , implying strong observability.

Original: Although similar in spirit to our feedback model, our results do not directly compare with theirs.
Revised: Although similar in spirit to our feedback model, our results do not directly compare with theirs.

Original: Further work also takes into account a time-varying probability for the revelation of sideobservations
Revised: Further work also takes into account a time-varying probability for the revelation of sideobservations

Original: [Kocák et al., 2016a].
Revised: [Kocák et al., 2016a].

Original: While the idea of a general probabilistic feedback graph has been already considered in the stochastic setting [Li et al., 2020, Cortes et al., 2020], the recent work by Ghari and Shen [2022] seems to be the first one in the adversarial setting that generalizes from the Erdős-Rényi model to a more flexible distribution where they allow “edge-specific” probabilities.
Revised: While the idea of a general probabilistic feedback graph has been already considered in the stochastic setting [Li et al., 2020, Cortes et al., 2020], the recent work by Ghari and Shen [2022] seems to be the first one in the adversarial setting that generalizes from the Erdős-Rényi model to a more flexible distribution where they allow “edge-specific” probabilities.

Original: We remark, however, that the assumptions of Ghari and Shen [2022] exclude some important instances of feedback graphs.
Revised: We remark, however, that the assumptions of Ghari and Shen [2022] exclude some important instances of feedback graphs.

Original: For example, we cannot hope to employ their algorithm for efficiently solving the revealing action problem (see for example [Alon et al., 2015]).
Revised: For example, we cannot hope to employ their algorithm for efficiently solving the revealing action problem (see for example [Alon et al., 2015]).

Original: In a spirit similar to ours, Resler and Mansour [2019] studied a version of the problem where the topology of the graph is fixed and known a priori, but the feedback received by the learner is perturbed when traversing edges.
Revised: In a spirit similar to ours, Resler and Mansour [2019] studied a version of the problem where the topology of the graph is fixed and known a priori, but the feedback received by the learner is perturbed when traversing edges.

Original:  Problem Setting
Revised:  Problem Setting

Original: A feedback graph over a set V =
Revised: A feedback graph over a set V =

Original: [ K ] of actions is any directed graph G =
Revised: [ K ] of actions is any directed graph G =

Original: ( V, E ), possibly with self-loops.
Revised: ( V, E ), possibly with self-loops.

Original: For any vertex i ∈ V , we use N in G ( i ) =
Revised: For any vertex i ∈ V , we use N in G ( i ) =

Original: { j ∈ V : ( j, i ) ∈ E } to denote the in-neighborhood of i and N out G ( i ) =
Revised: { j ∈ V : ( j, i ) ∈ E } to denote the in-neighborhood of i and N out G ( i ) =

Original: { j ∈ V : ( i, j ) ∈ E } to denote its out-neighborhood (we may omit the subscript when the graph is clear from the context).
Revised: { j ∈ V : ( i, j ) ∈ E } to denote its out-neighborhood (we may omit the subscript when the graph is clear from the context).

Original: The independence number α ( G ) of a feedback graph G is the cardinality of the largest subset S of V such that, for all distinct i, j ∈ S , it holds that ( i, j ) and ( j, i ) are not in E .
Revised: The independence number α ( G ) of a feedback graph G is the cardinality of the largest subset S of V such that, for all distinct i, j ∈ S , it holds that ( i, j ) and ( j, i ) are not in E .

Original: We also use the following terminology for directed graphs G =
Revised: We also use the following terminology for directed graphs G =

Original: ( V, E )
Revised: ( V, E )

Original: [Alon et al., 2015].
Revised: [Alon et al., 2015].

Original: Any i ∈ V is: observable if N in G ( i ) ̸
Revised: Any i ∈ V is: observable if N in G ( i ) ̸

Original: = ∅ , strongly observable if i ∈ N in G ( i ) or V \ { i } ⊆
Revised: = ∅ , strongly observable if i ∈ N in G ( i ) or V \ { i } ⊆

Original: N in G ( i ), and weakly observable if it is observable but not strongly.
Revised: N in G ( i ), and weakly observable if it is observable but not strongly.

Original: The graph G is: observable if all i ∈ V are observable, strongly observable if all i ∈ V are strongly observable, and weakly observable if it is observable but not strongly.
Revised: The graph G is: observable if all i ∈ V are observable, strongly observable if all i ∈ V are strongly observable, and weakly observable if it is observable but not strongly.

Original: The weak domination number δ ( G ) of G is the cardinality of the smallest subset S of V such that for all weakly observable i ∈ V \ S there exists j ∈ S such that ( j, i ) ∈ E .
Revised: The weak domination number δ ( G ) of G is the cardinality of the smallest subset S of V such that for all weakly observable i ∈ V \ S there exists j ∈ S such that ( j, i ) ∈ E .

Original: In the online learning problem with a stochastic feedback graph, an oblivious adversary privately chooses a stochastic feedback graph G and a sequence ℓ 1 , ℓ 2 , . . .
Revised: In the online learning problem with a stochastic feedback graph, an oblivious adversary privately chooses a stochastic feedback graph G and a sequence ℓ 1 , ℓ 2 , . . .

Original: of loss functions ℓ t : V → [0 , 1].
Revised: of loss functions ℓ t : V → [0 , 1].

Original: At each round t = 1 , 2 , . . .
Revised: At each round t = 1 , 2 , . . .

Original: , the learner selects an action I t ∈ V to play and, independently, the adversary draws a feedback graph
Revised: , the learner selects an action I t ∈ V to play and, independently, the adversary draws a feedback graph

Original: G t from G (denoted by G t ∼ G ).
Revised: G t from G (denoted by G t ∼ G ).

Original: The learner then incurs loss ℓ t ( I t ) and observes the feedback (cid:8) ( i, ℓ t ( i )) : i ∈ N out G t
Revised: The learner then incurs loss ℓ t ( I t ) and observes the feedback (cid:8) ( i, ℓ t ( i )) : i ∈ N out G t

Original: ( I t ) (cid:9) .
Revised: ( I t ) (cid:9) .

Original: In some cases we consider a richer feedback, where at the end of each round t the learner also observes the realized graph G t .
Revised: In some cases we consider a richer feedback, where at the end of each round t the learner also observes the realized graph G t .

Original: The learner’s performance is measured using the standard notion of regret,
Revised: The learner’s performance is measured using the standard notion of regret,

Original: [Equation]
Revised: [Equation]

Original: # where I 1 , . .
Revised: # where I 1 , . .

Original: , I T are the actions played by the learner, and the expectation is computed over both the sequence
Revised: , I T are the actions played by the learner, and the expectation is computed over both the sequence

Original: G 1 , . .
Revised: G 1 , . .

Original: , G T of feedback graphs drawn i.i.d.
Revised: , G T of feedback graphs drawn i.i.d.

Original: from G and the learner’s internal randomization.
Revised: from G and the learner’s internal randomization.

Original: Fix any stochastic feedback graph G = { p ( i, j ) : i, j ∈ V } .
Revised: Fix any stochastic feedback graph G = { p ( i, j ) : i, j ∈ V } .

Original: We sometimes use e to denote a pair ( i, j ), in which case we write p e to denote the probability p ( i, j ).
Revised: We sometimes use e to denote a pair ( i, j ), in which case we write p e to denote the probability p ( i, j ).

Original: When G t = ( V, E t ) is drawn from
Revised: When G t = ( V, E t ) is drawn from

Original: G , each pair ( i,
Revised: G , each pair ( i,

Original: j ) ∈
Revised: j ) ∈

Original: V × V independently becomes an edge (i.e., ( i,
Revised: V × V independently becomes an edge (i.e., ( i,

Original: j ) ∈ E t ) with probability p ( i, j ).
Revised: j ) ∈ E t ) with probability p ( i, j ).

Original: For any ε > 0, we define the thresholded version G ε of G represented by { p ′ ( i, j ) : i, j ∈ V } , where p ′ ( i, j )
Revised: For any ε > 0, we define the thresholded version G ε of G represented by { p ′ ( i, j ) : i, j ∈ V } , where p ′ ( i, j )

Original: =
Revised: =

Original: p ( i, j ) I { p ( i,j ) ≥ ε } .
Revised: p ( i, j ) I { p ( i,j ) ≥ ε } .

Original: We also define the support feedback graph of G as the graph supp( G ) =
Revised: We also define the support feedback graph of G as the graph supp( G ) =

Original: ( V, E ) having E =
Revised: ( V, E ) having E =

Original: ( i, j ) ∈
Revised: ( i, j ) ∈

Original: V × V : p ( i, j ) > 0 } .
Revised: V × V : p ( i, j ) > 0 } .

Original: To keep the notation tidy, we write α ( G ) instead of α (supp( G )) and similarly for δ .
Revised: To keep the notation tidy, we write α ( G ) instead of α (supp( G )) and similarly for δ .

Original:  Block Decomposition Approach
Revised:  Block Decomposition Approach

Original: In this section, we present an algorithm for online learning with stochastic feedback graphs via a reduction to online learning with deterministic feedback graphs.
Revised: In this section, we present an algorithm for online learning with stochastic feedback graphs via a reduction to online learning with deterministic feedback graphs.

Original: Our algorithm
Revised: Our algorithm

Original: EdgeCatcher (Algorithm 3) has an initial exploration phase followed by a commit phase.
Revised: EdgeCatcher (Algorithm 3) has an initial exploration phase followed by a commit phase.

Original: In the exploration phase, the edge probabilities are learned online in a round-robin fashion.
Revised: In the exploration phase, the edge probabilities are learned online in a round-robin fashion.

Original: A carefully designed stopping criterion then triggers the commit phase, where we feed the support of the estimated stochastic feedback graph to an algorithm for online learning with (deterministic) feedback graphs.
Revised: A carefully designed stopping criterion then triggers the commit phase, where we feed the support of the estimated stochastic feedback graph to an algorithm for online learning with (deterministic) feedback graphs.

Original: 3. Estimating the Edge Probabilities
Revised: 3. Estimating the Edge Probabilities

Original: As a first step we design a routine, RoundRobin (Algorithm 1), that sequentially estimates the stochastic feedback graph until a certain stopping criterion is met.
Revised: As a first step we design a routine, RoundRobin (Algorithm 1), that sequentially estimates the stochastic feedback graph until a certain stopping criterion is met.

Original: The stopping criterion depends on a nonnegative function Φ that takes in input a stochastic feedback graph G together with a time horizon.
Revised: The stopping criterion depends on a nonnegative function Φ that takes in input a stochastic feedback graph G together with a time horizon.

Original: Let ˆ τ ≤ T/K be the index of the last iteration of the outer for loop in Algorithm 1.
Revised: Let ˆ τ ≤ T/K be the index of the last iteration of the outer for loop in Algorithm 1.

Original: We want to make sure that, for all τ ≤ ˆ τ , the stochastic feedback graphs ˆ G τ are valid estimates of the underlying G up to a Θ( ε τ ) precision.
Revised: We want to make sure that, for all τ ≤ ˆ τ , the stochastic feedback graphs ˆ G τ are valid estimates of the underlying G up to a Θ( ε τ ) precision.

Original: To formalize this notion of approximation, we introduce the following definition.
Revised: To formalize this notion of approximation, we introduce the following definition.

Original: Definition 1 ( ε -good approximation) .
Revised: Definition 1 ( ε -good approximation) .

Original: A stochastic feedback graph ˆ G =
Revised: A stochastic feedback graph ˆ G =

Original: { ˆ p e : e ∈ V 2 } is an ε -good approximation of G = { p e : e ∈ V 2 } for some ε ∈ (0 , 1] , if the following holds:
Revised: { ˆ p e : e ∈ V 2 } is an ε -good approximation of G = { p e : e ∈ V 2 } for some ε ∈ (0 , 1] , if the following holds:

Original: All the edges e ∈ supp( G ) with p e ≥ 2 ε belong to supp( ˆ G ) ; 2.
Revised: All the edges e ∈ supp( G ) with p e ≥ 2 ε belong to supp( ˆ G ) ; 2.

Original: For all edges e ∈ supp( ˆ G ) with p e ≥ ε/ 2 it holds that | ˆ p e − p e | ≤ p e / 2 ; 3.
Revised: For all edges e ∈ supp( ˆ G ) with p e ≥ ε/ 2 it holds that | ˆ p e − p e | ≤ p e / 2 ; 3.

Original: No edge e ∈ V 2 with p e < ε/ 2 belongs to supp( ˆ G ) .
Revised: No edge e ∈ V 2 with p e < ε/ 2 belongs to supp( ˆ G ) .

Original: We can now state the following theorem; we defer the proof in Appendix B.
Revised: We can now state the following theorem; we defer the proof in Appendix B.

Original: The proof follows from an application of the multiplicative Chernoff bound on edge probabilities.
Revised: The proof follows from an application of the multiplicative Chernoff bound on edge probabilities.

Original: Theorem 2.
Revised: Theorem 2.

Original: If RoundRobin (Algorithm 1) is run on the stochastic feedback graph G , then, with probability at least 1 − 1 /T , the estimate ˆ G τ is an ε τ -good approximation of G simultaneously for all τ ≤ ˆ τ , where ˆ τ ≤ T/K is the index of the last iteration of the outer for loop in Algorithm 1.
Revised: If RoundRobin (Algorithm 1) is run on the stochastic feedback graph G , then, with probability at least 1 − 1 /T , the estimate ˆ G τ is an ε τ -good approximation of G simultaneously for all τ ≤ ˆ τ , where ˆ τ ≤ T/K is the index of the last iteration of the outer for loop in Algorithm 1.

Original: // estimated feedback graph if Φ
Revised: // estimated feedback graph if Φ

Original: ≤ then
Revised: ≤ then

Original: 3. Block Decomposition: Reduction to Deterministic Feedback Graph
Revised: 3. Block Decomposition: Reduction to Deterministic Feedback Graph

Original: As a second step, we present BlockReduction (Algorithm 2) which reduces the problem of online learning with stochastic feedback graph to the corresponding problem with deterministic feedback graph.
Revised: As a second step, we present BlockReduction (Algorithm 2) which reduces the problem of online learning with stochastic feedback graph to the corresponding problem with deterministic feedback graph.

Original: Surprisingly enough, in order for this reduction to work, we do not need the exact edge probabilities: an ε -good approximation is sufficient for this purpose.
Revised: Surprisingly enough, in order for this reduction to work, we do not need the exact edge probabilities: an ε -good approximation is sufficient for this purpose.

Original: The intuition behind BlockReduction is simple: given that each edge e in supp( G ε ) appears in G t with probability p e ≥ ε at each time step t , if we wait for Θ (cid:0) (1 /ε )
Revised: The intuition behind BlockReduction is simple: given that each edge e in supp( G ε ) appears in G t with probability p e ≥ ε at each time step t , if we wait for Θ (cid:0) (1 /ε )

Original: ln( T ) (cid:1) time steps it will appear at least once with high probability.
Revised: ln( T ) (cid:1) time steps it will appear at least once with high probability.

Original: Applying a union bound over all edges, we can argue that considering ∆ =
Revised: Applying a union bound over all edges, we can argue that considering ∆ =

Original: Θ (cid:0) (1 /ε )
Revised: Θ (cid:0) (1 /ε )

Original: ln( KT ) (cid:1) realizations of the stochastic feedback graph, then all the edges in supp( G ε ) are realized at least once with high probability.
Revised: ln( KT ) (cid:1) realizations of the stochastic feedback graph, then all the edges in supp( G ε ) are realized at least once with high probability.

Original: Imagine now to play a certain action a consistently during a block B τ of ∆ time steps.
Revised: Imagine now to play a certain action a consistently during a block B τ of ∆ time steps.

Original: We want to reconstruct the average loss suffered by a ′ in B τ :
Revised: We want to reconstruct the average loss suffered by a ′ in B τ :

Original: [Equation]
Revised: [Equation]

Original: and we want to do it for all a ′ in the out-neighborhood of a .
Revised: and we want to do it for all a ′ in the out-neighborhood of a .

Original: Let ∆ τ ( a,a ′ ) be the number of times that the loss of a ′ is observed by the learner; i.e., the number of times that ( a, a ′ ) is realized in the ∆ time steps.
Revised: Let ∆ τ ( a,a ′ ) be the number of times that the loss of a ′ is observed by the learner; i.e., the number of times that ( a, a ′ ) is realized in the ∆ time steps.

Original: With this notation, we can define the natural estimator ˆ c τ ( a ′ ):
Revised: With this notation, we can define the natural estimator ˆ c τ ( a ′ ):

Original: [Equation]
Revised: [Equation]

Original: Conditioning on the event E τ ( a,a ′ ) that the edge ( a, a ′ ) in ˆ G is observed at least once in block
Revised: Conditioning on the event E τ ( a,a ′ ) that the edge ( a, a ′ ) in ˆ G is observed at least once in block

Original: B τ , we show in Lemma 2 in Appendix B that ˆ c
Revised: B τ , we show in Lemma 2 in Appendix B that ˆ c

Original: τ ( a ′ ) is an unbiased estimator of c τ ( a ′ ).
Revised: τ ( a ′ ) is an unbiased estimator of c τ ( a ′ ).

Original: Therefore, we can construct unbiased estimators of the average of the losses on the blocks as if the stochastic feedback graph were deterministic.
Revised: Therefore, we can construct unbiased estimators of the average of the losses on the blocks as if the stochastic feedback graph were deterministic.

Original: This allows us to reduce the original problem to that of online learning with deterministic feedback graph on the meta-instance given by the blocks.
Revised: This allows us to reduce the original problem to that of online learning with deterministic feedback graph on the meta-instance given by the blocks.

Original: The details of BlockReduction are reported in Algorithm 2, while the theoretical properties are summarized in the next result, whose proof can be found in
Revised: The details of BlockReduction are reported in Algorithm 2, while the theoretical properties are summarized in the next result, whose proof can be found in

Original: Appendix B.
Revised: Appendix B.

Original: Theorem 3.
Revised: Theorem 3.

Original: Consider the problem of online learning with stochastic feedback graph G , and let ˆ G be an ε -good approximation of G .
Revised: Consider the problem of online learning with stochastic feedback graph G , and let ˆ G be an ε -good approximation of G .

Original: Let A be an algorithm for online learning with arbitrary deterministic feedback graph G with regret bound R A N ( G ) over any sequence of
Revised: Let A be an algorithm for online learning with arbitrary deterministic feedback graph G with regret bound R A N ( G ) over any sequence of

Original: N losses in [0 , 1] .
Revised: N losses in [0 , 1] .

Original: Then, the regret of BlockReduction (Algorithm 2) run with input ( T, ε/ 2 , ˆ G , A ) is at most ∆ R A N (cid:0) supp( ˆ G ) (cid:1) + ∆ , where N =
Revised: Then, the regret of BlockReduction (Algorithm 2) run with input ( T, ε/ 2 , ˆ G , A ) is at most ∆ R A N (cid:0) supp( ˆ G ) (cid:1) + ∆ , where N =

Original: ⌊ T/ ∆ ⌋ and ∆ =
Revised: ⌊ T/ ∆ ⌋ and ∆ =

Original: ⌈ 4 ε ln( KT ) ⌉ .
Revised: ⌈ 4 ε ln( KT ) ⌉ .

Original: arbitrarily the remaining
Revised: arbitrarily the remaining

Original: ∆
Revised: ∆

Original: For online learning with deterministic feedback graphs we use the variants of Exp3.G contained in Alon et al.
Revised: For online learning with deterministic feedback graphs we use the variants of Exp3.G contained in Alon et al.

Original: Together with Theorem 3, this gives the following corollary; the details of the proof are in Appendix B.
Revised: Together with Theorem 3, this gives the following corollary; the details of the proof are in Appendix B.

Original: Corollary 1.
Revised: Corollary 1.

Original: Consider the problem of online learning with stochastic feedback graph G , and let ˆ G be an ε -good approximation of G for ε ≥ 1 /T and with support ˆ G .
Revised: Consider the problem of online learning with stochastic feedback graph G , and let ˆ G be an ε -good approximation of G for ε ≥ 1 /T and with support ˆ G .

Original: • If ˆ G is strongly observable with independence number α , then the regret of BlockRe- duction run with parameter ε/ 2 using Exp3.G for strongly observable graphs as base algorithm
Revised: • If ˆ G is strongly observable with independence number α , then the regret of BlockRe- duction run with parameter ε/ 2 using Exp3.G for strongly observable graphs as base algorithm

Original: A satisfies: R T ≤ 4 C s p ( α/ε ) T (cid:0) ln( KT ) (cid:1) 3 / 2 , where C s > 0 is a constant in the regret bound of A .
Revised: A satisfies: R T ≤ 4 C s p ( α/ε ) T (cid:0) ln( KT ) (cid:1) 3 / 2 , where C s > 0 is a constant in the regret bound of A .

Original: • If ˆ G is (weakly) observable with weak domination number δ , then the regret of BlockRe- duction run with parameter ε/ 2 using Exp3.G for weakly observable graphs as base algorithm
Revised: • If ˆ G is (weakly) observable with weak domination number δ , then the regret of BlockRe- duction run with parameter ε/ 2 using Exp3.G for weakly observable graphs as base algorithm

Original: A satisfies: R T ≤ 4 C w ( δ/ε ) 1 / 3 (cid:0) ln( KT ) (cid:1) 2 / 3 T 2 / 3 , where C w > 0 is a constant in the regret bound of A .
Revised: A satisfies: R T ≤ 4 C w ( δ/ε ) 1 / 3 (cid:0) ln( KT ) (cid:1) 2 / 3 T 2 / 3 , where C w > 0 is a constant in the regret bound of A .

Original: Note that we can explicitly compute valid constants C s = 12 + 2 √ 2 and C w = 8 directly from the bounds in Alon et al.
Revised: Note that we can explicitly compute valid constants C s = 12 + 2 √ 2 and C w = 8 directly from the bounds in Alon et al.

Original: 3. Explore then Commit to a Graph
Revised: 3. Explore then Commit to a Graph

Original: We are now ready to combine the two routines we presented, RoundRobin and BlockReduction , in our final learning algorithm, EdgeCatcher .
Revised: We are now ready to combine the two routines we presented, RoundRobin and BlockReduction , in our final learning algorithm, EdgeCatcher .

Original: EdgeCatcher has two phases: in the first phase, RoundRobin is used to quickly obtain an ε -good approximation ˆ G of the underlying feedback graph G , for a suitable ε .
Revised: EdgeCatcher has two phases: in the first phase, RoundRobin is used to quickly obtain an ε -good approximation ˆ G of the underlying feedback graph G , for a suitable ε .

Original: In the second phase, the algorithm commits to ˆ G and feeds it to BlockReduction .
Revised: In the second phase, the algorithm commits to ˆ G and feeds it to BlockReduction .

Original: The crucial point is when to commit to a certain (estimated) stochastic feedback graph.
Revised: The crucial point is when to commit to a certain (estimated) stochastic feedback graph.

Original: If we commit too early, we might not observe a denser support graph, which implies missing out on a richer feedback.
Revised: If we commit too early, we might not observe a denser support graph, which implies missing out on a richer feedback.

Original: If we wait for too long, then the exploration phase ends up dominating the regret.
Revised: If we wait for too long, then the exploration phase ends up dominating the regret.

Original: To balance this trade-off, we use the stopping function Φ. This function takes as input a probabilistic feedback graph together with a time horizon and outputs the regret bound that BlockReduction would guarantee on this pair.
Revised: To balance this trade-off, we use the stopping function Φ. This function takes as input a probabilistic feedback graph together with a time horizon and outputs the regret bound that BlockReduction would guarantee on this pair.

Original: It is defined as
Revised: It is defined as

Original: [Equation]
Revised: [Equation]

Original: (5) for the specific choice of Exp3.G as the learning algorithm
Revised: (5) for the specific choice of Exp3.G as the learning algorithm

Original: A adopted by BlockReduction .
Revised: A adopted by BlockReduction .

Original: Note that the dependence of Φ on the feedback graph G is contained in the topological parameters α ∗ and δ ∗ and the corresponding thresholds ε ∗ s and ε ∗ w , defined in Equations (1) and (2); see Appendix A for more details on their computation.
Revised: Note that the dependence of Φ on the feedback graph G is contained in the topological parameters α ∗ and δ ∗ and the corresponding thresholds ε ∗ s and ε ∗ w , defined in Equations (1) and (2); see Appendix A for more details on their computation.

Original: If we apply Φ to a stochastic feedback graph that is observable w.p. zero, its value is conventionally set to infinity.
Revised: If we apply Φ to a stochastic feedback graph that is observable w.p. zero, its value is conventionally set to infinity.

Original: Observe that, otherwise, the min is achieved for a specific ε ∗ and a specific G ∗ =
Revised: Observe that, otherwise, the min is achieved for a specific ε ∗ and a specific G ∗ =

Original: G ε ∗ .
Revised: G ε ∗ .

Original: In Appendix B, we provide a sequence of lemmas (Lemmas 3 and 4 in particular) showing
Revised: In Appendix B, we provide a sequence of lemmas (Lemmas 3 and 4 in particular) showing

Original: Environment : stochastic feedback graph G , sequence of losses ℓ 1 , ℓ 2 , . .
Revised: Environment : stochastic feedback graph G , sequence of losses ℓ 1 , ℓ 2 , . .

Original: . , ℓ T ;
Revised: . , ℓ T ;

Original: Input : time horizon T and actions V = { 1 , 2 , . .
Revised: Input : time horizon T and actions V = { 1 , 2 , . .

Original: , K } ; Let Φ defined as in Equation (5);
Revised: , K } ; Let Φ defined as in Equation (5);

Original: Run RoundRobin ( T, Φ , V ) and obtain ˆ G and ˆ ε ;
Revised: Run RoundRobin ( T, Φ , V ) and obtain ˆ G and ˆ ε ;

Original: Compute ˆ ε ∗ s and ˆ ε ∗ w for graph ˆ G as in Equations (1) and (2);
Revised: Compute ˆ ε ∗ s and ˆ ε ∗ w for graph ˆ G as in Equations (1) and (2);

Original: Let ˆ ε ∗ be the best threshold as in Equation (5); if ˆ ε ∗ =
Revised: Let ˆ ε ∗ be the best threshold as in Equation (5); if ˆ ε ∗ =

Original: ˆ ε ∗ s then Let A be Exp3.G for strongly observable feedback graph; else Let A be Exp3.G for weakly observable feedback graph;
Revised: ˆ ε ∗ s then Let A be Exp3.G for strongly observable feedback graph; else Let A be Exp3.G for weakly observable feedback graph;

Original: Let T ′ =
Revised: Let T ′ =

Original: T − ˆ τK be the remaining time steps; // ˆ τ as in RoundRobin
Revised: T − ˆ τK be the remaining time steps; // ˆ τ as in RoundRobin

Original: Run BlockReduction ( T ′ , ˆ ε ∗ / 2 , ˆ G ˆ ε ∗ , A ); that, if RoundRobin outputs an ε -good approximation of the graph, then the regret is bounded by a multiple of the stopping criterion evaluated at G .
Revised: Run BlockReduction ( T ′ , ˆ ε ∗ / 2 , ˆ G ˆ ε ∗ , A ); that, if RoundRobin outputs an ε -good approximation of the graph, then the regret is bounded by a multiple of the stopping criterion evaluated at G .

Original: Combined with Theorem 2, which tells us that RoundRobin does in fact output an ε -good approximation of the graph with high probability, this proves our main result for this section.
Revised: Combined with Theorem 2, which tells us that RoundRobin does in fact output an ε -good approximation of the graph with high probability, this proves our main result for this section.

Original: Theorem 4.
Revised: Theorem 4.

Original: Consider the problem of online learning with stochastic feedback graph G on
Revised: Consider the problem of online learning with stochastic feedback graph G on

Original: T time steps.
Revised: T time steps.

Original: If supp( G ε ( K,T ) ) is observable for ε ( K, T ) =
Revised: If supp( G ε ( K,T ) ) is observable for ε ( K, T ) =

Original: CK 3 (ln( KT )) 2 /T for a given constant C > 0 , then there exists an algorithm whose regret R T satisfies (ignoring polylog factors in K and T ) R T ≤ min np ( α ∗ /ε ∗ s ) T, (cid:0) δ ∗ /ε
Revised: CK 3 (ln( KT )) 2 /T for a given constant C > 0 , then there exists an algorithm whose regret R T satisfies (ignoring polylog factors in K and T ) R T ≤ min np ( α ∗ /ε ∗ s ) T, (cid:0) δ ∗ /ε

Original: ∗ w (cid:1) 1 / 3 T 2 / 3 o .
Revised: ∗ w (cid:1) 1 / 3 T 2 / 3 o .

Original:  Lower Bounds
Revised:  Lower Bounds

Original: In this section, we provide lower bounds that match the regret bound guaranteed by
Revised: In this section, we provide lower bounds that match the regret bound guaranteed by

Original: EdgeCatcher , shown in Theorem 4, up to polylogarithmic factors in K and T .
Revised: EdgeCatcher , shown in Theorem 4, up to polylogarithmic factors in K and T .

Original: These lower bounds are valid even if the learner is allowed to observe the realization of the entire feedback graph at every time step, and knows a priori the “correct” threshold ε to work with.
Revised: These lower bounds are valid even if the learner is allowed to observe the realization of the entire feedback graph at every time step, and knows a priori the “correct” threshold ε to work with.

Original: Theorem 5 summarizes the lower bounds whose proofs can be found in Appendix C.
Revised: Theorem 5 summarizes the lower bounds whose proofs can be found in Appendix C.

Original: Theorem 5 (Informal) .
Revised: Theorem 5 (Informal) .

Original: Let A be a possibly randomized algorithm for the online learning problem with stochastic feedback graphs.
Revised: Let A be a possibly randomized algorithm for the online learning problem with stochastic feedback graphs.

Original: Consider any directed graph G =
Revised: Consider any directed graph G =

Original: ( V, E ) with | V |
Revised: ( V, E ) with | V |

Original: ≥and any ε ∈ (0 , 1] .
Revised: ≥and any ε ∈ (0 , 1] .

Original: There exists a stochastic feedback graph G with supp( G ) =
Revised: There exists a stochastic feedback graph G with supp( G ) =

Original: G and, for a sufficiently large time horizon T , there is a sequence ℓ 1 , . . .
Revised: G and, for a sufficiently large time horizon T , there is a sequence ℓ 1 , . . .

Original: , ℓ T of loss functions on which the expected regret of A with respect to the stochastic generation of G 1 , . . .
Revised: , ℓ T of loss functions on which the expected regret of A with respect to the stochastic generation of G 1 , . . .

Original: , G T ∼ G is
Revised: , G T ∼ G is

Original: • Ω( p ( α ( G ε ) /ε ) T ) if G is strongly observable; • ˜Ω(( δ ( G ε ) /ε ) 1 / 3 T 2 / 3 ) if G is weakly observable; • Ω( T ) if G is not observable.
Revised: • Ω( p ( α ( G ε ) /ε ) T ) if G is strongly observable; • ˜Ω(( δ ( G ε ) /ε ) 1 / 3 T 2 / 3 ) if G is weakly observable; • Ω( T ) if G is not observable.

Original: The lower bound in the non-observable case is the same as Alon et al.
Revised: The lower bound in the non-observable case is the same as Alon et al.

Original: [2015, Theorem 6] with a deterministic feedback graph.
Revised: [2015, Theorem 6] with a deterministic feedback graph.

Original: The remaining lower bounds are nontrivial adaptations of the corresponding bounds for the deterministic case by Alon et al.
Revised: The remaining lower bounds are nontrivial adaptations of the corresponding bounds for the deterministic case by Alon et al.

Original: The main technical hurdle is due to the stochastic nature of the feedback graph, which needs to be taken into account in the proofs.
Revised: The main technical hurdle is due to the stochastic nature of the feedback graph, which needs to be taken into account in the proofs.

Original: The rationale behind the constructions used for proving the lower bounds is as follows: since each edge is realized only with probability ε , any algorithm requires 1 /ε rounds in expectation in order to observe the loss of an action in the out-neighborhood of the played action, whereas one round would suffice with a deterministic feedback graph.
Revised: The rationale behind the constructions used for proving the lower bounds is as follows: since each edge is realized only with probability ε , any algorithm requires 1 /ε rounds in expectation in order to observe the loss of an action in the out-neighborhood of the played action, whereas one round would suffice with a deterministic feedback graph.

Original: Note that Theorem 5 implies that, if supp( G ε ( K,T ) ) is not observable for ε ( K, T ) as in Theorem 4, then there is no hope to achieve sublinear regret, as the lower bounds for both strongly and weakly observable supports are linear in T for all ε ≤ ε ( K, T ).
Revised: Note that Theorem 5 implies that, if supp( G ε ( K,T ) ) is not observable for ε ( K, T ) as in Theorem 4, then there is no hope to achieve sublinear regret, as the lower bounds for both strongly and weakly observable supports are linear in T for all ε ≤ ε ( K, T ).

Original:  Refined Graph-Theoretic Parameters
Revised:  Refined Graph-Theoretic Parameters

Original: Although the results from Section 3 are worst-case optimal up to log factors, we may find that the factors p α ( G ε ) /ε
Revised: Although the results from Section 3 are worst-case optimal up to log factors, we may find that the factors p α ( G ε ) /ε

Original: and ( δ ( G ε ) /ε )
Revised: and ( δ ( G ε ) /ε )

Original: / 3 for strongly and weakly observable supp( G ε ) =
Revised: / 3 for strongly and weakly observable supp( G ε ) =

Original: G ε , respectively, may be improved upon in certain cases.
Revised: G ε , respectively, may be improved upon in certain cases.

Original: In particular, we show that, under additional assumptions on the feedback that we receive, we can obtain better regret bounds.
Revised: In particular, we show that, under additional assumptions on the feedback that we receive, we can obtain better regret bounds.

Original: To understand our results, we need some initial definitions.
Revised: To understand our results, we need some initial definitions.

Original: The weighted independence number for a graph H =
Revised: The weighted independence number for a graph H =

Original: ( V, E ) and positive vertex weights w ( i ) for i ∈ V is defined as
Revised: ( V, E ) and positive vertex weights w ( i ) for i ∈ V is defined as

Original: [Equation]
Revised: [Equation]

Original: where I ( H ) denotes the family of independent sets in H .
Revised: where I ( H ) denotes the family of independent sets in H .

Original: We consider two different weight assignments computed in terms of any stochastic feedback graph G with edge probabilities p ( i, j ) and supp( G )
Revised: We consider two different weight assignments computed in terms of any stochastic feedback graph G with edge probabilities p ( i, j ) and supp( G )

Original: =
Revised: =

Original: G .
Revised: G .

Original: They are defined as w −G ( i ) =
Revised: They are defined as w −G ( i ) =

Original: (cid:0) min j ∈ N in G ( i ) p ( j, i ) (cid:1) − 1 and w + G ( i ) = (cid:0) min j ∈ N out G ( i ) p ( i, j ) (cid:1) − 1 .
Revised: (cid:0) min j ∈ N in G ( i ) p ( j, i ) (cid:1) − 1 and w + G ( i ) = (cid:0) min j ∈ N out G ( i ) p ( i, j ) (cid:1) − 1 .

Original: Then, the two corresponding weighted independence numbers are α − w ( G ) =
Revised: Then, the two corresponding weighted independence numbers are α − w ( G ) =

Original: α w ( G, w −G ) and α + w ( G ) =
Revised: α w ( G, w −G ) and α + w ( G ) =

Original: α w ( G, w + G ).
Revised: α w ( G, w + G ).

Original: The parameter of interest for the results in this section is α w ( G ) =
Revised: The parameter of interest for the results in this section is α w ( G ) =

Original: α − w ( G ) + α + w ( G ).
Revised: α − w ( G ) + α + w ( G ).

Original: For more details on the weighted independence number see Appendix E. We also use the following definitions of weighted weak domination number δ w for a graph H and positive vertex weights w , and self-observability parameter σ :
Revised: For more details on the weighted independence number see Appendix E. We also use the following definitions of weighted weak domination number δ w for a graph H and positive vertex weights w , and self-observability parameter σ :

Original: [Equation]
Revised: [Equation]

Original: where D ( H ) denotes the family of weakly dominating sets in H .
Revised: where D ( H ) denotes the family of weakly dominating sets in H .

Original: In this section, we focus on the weighted weak domination number δ w ( G ) =
Revised: In this section, we focus on the weighted weak domination number δ w ( G ) =

Original: δ w ( G, w + G ).
Revised: δ w ( G, w + G ).

Original: To gain some intuition on the graph-theoretic parameters introduced above, consider the graph with only self-loops, also used in Example 1 below.
Revised: To gain some intuition on the graph-theoretic parameters introduced above, consider the graph with only self-loops, also used in Example 1 below.

Original: If all p ( i, i ) = ε , the learner needs to pull a single arm 1 /ε times for one observation in expectation, and K/ε times to see the losses of all arms once.
Revised: If all p ( i, i ) = ε , the learner needs to pull a single arm 1 /ε times for one observation in expectation, and K/ε times to see the losses of all arms once.

Original: However, when the edge probabilities are different we need to pull arms for P Ki =1 1 /p
Revised: However, when the edge probabilities are different we need to pull arms for P Ki =1 1 /p

Original: ( i, i ) times.
Revised: ( i, i ) times.

Original: The weighted independence number, weighted weak domination and self-observability generalize this intuition and tell us how many observations the learner needs to see all losses at least once in expectation.
Revised: The weighted independence number, weighted weak domination and self-observability generalize this intuition and tell us how many observations the learner needs to see all losses at least once in expectation.

Original: We now state the main result of this section.
Revised: We now state the main result of this section.

Original: Theorem 6 (Informal) .
Revised: Theorem 6 (Informal) .

Original: There exists an algorithm with per-round running time of O ( K 4 ) and whose regret is bounded (ignoring logarithmic factors) by
Revised: There exists an algorithm with per-round running time of O ( K 4 ) and whose regret is bounded (ignoring logarithmic factors) by

Original: [Equation]
Revised: [Equation]

Original: The regret bound in Theorem 6 follows from Theorem 11 in Appendix D.
Revised: The regret bound in Theorem 6 follows from Theorem 11 in Appendix D.

Original: The running time bound is determined by approximating δ w for all K 2 possible thresholds.
Revised: The running time bound is determined by approximating δ w for all K 2 possible thresholds.

Original: In each of the thresholded graphs, we can compute a (ln( K ) + 1)-approximation for the weighted weak domination number in O ( K 2 ) time by reduction to set cover [Vazirani, 2001].
Revised: In each of the thresholded graphs, we can compute a (ln( K ) + 1)-approximation for the weighted weak domination number in O ( K 2 ) time by reduction to set cover [Vazirani, 2001].

Original: Doing so only introduces an extra factor of order (ln( K )) 1 / 3 in the regret bound.
Revised: Doing so only introduces an extra factor of order (ln( K )) 1 / 3 in the regret bound.

Original: An important property of the bound in Theorem 6 is that it is never worse than the bounds obtained before.
Revised: An important property of the bound in Theorem 6 is that it is never worse than the bounds obtained before.

Original: The following example shows that the regret bound in Theorem 6 can also be better than previously obtained regret bounds.
Revised: The following example shows that the regret bound in Theorem 6 can also be better than previously obtained regret bounds.

Original: Example 1 (Faulty bandits) .
Revised: Example 1 (Faulty bandits) .

Original: Consider a stochastic feedback graph G for the K -armed bandit setting: p ( i, i ) =
Revised: Consider a stochastic feedback graph G for the K -armed bandit setting: p ( i, i ) =

Original: ε i ∈ (0 , 1] for all i ∈ V and p ( i, j ) = 

Revised: ε i ∈ (0 , 1] for all i ∈ V and p ( i, j ) =

Original: for all i ̸
Revised: for all i ̸

Original: =
Revised: =

Original: j .
Revised: j .

Original: In this case, the regret of EdgeCatcher is ˜ O
Revised: In this case, the regret of EdgeCatcher is ˜ O

Original: (cid:0)p KT/ (min i ε i )).
Revised: (cid:0)p KT/ (min i ε i )).

Original: On the other hand, Theorem 6 provides the bound ˜ O (cid:0)p T P i (1 /ε i ) (cid:1) , as α w ( G )
Revised: On the other hand, Theorem 6 provides the bound ˜ O (cid:0)p T P i (1 /ε i ) (cid:1) , as α w ( G )

Original: = 2 P i 1 /ε i .
Revised: = 2 P i 1 /ε i .

Original: In the special case when ε i = ε ∈ (0 , 1] for some i ∈ V while ε j = 1 for all j ̸ = i , the regret of EdgeCatcher is ˜ O ( p KT/ε ), while
Revised: In the special case when ε i = ε ∈ (0 , 1] for some i ∈ V while ε j = 1 for all j ̸ = i , the regret of EdgeCatcher is ˜ O ( p KT/ε ), while

Original: Theorem 6 guarantees a ˜ O ( p ( K + 1 /ε ) T ) regret bound.
Revised: Theorem 6 guarantees a ˜ O ( p ( K + 1 /ε ) T ) regret bound.

Original: □
Revised: □

Original: To derive these tighter bounds, we exploit the additional assumption that the realized feedback graph G t is observed at the end of each round.
Revised: To derive these tighter bounds, we exploit the additional assumption that the realized feedback graph G t is observed at the end of each round.

Original: This allows us to simultaneously estimate the feedback graph and control the regret, rather than performing these two tasks sequentially as in Section 3.
Revised: This allows us to simultaneously estimate the feedback graph and control the regret, rather than performing these two tasks sequentially as in Section 3.

Original: In particular, we use this extra information to construct a novel importance-weighted estimator for the loss, which for t > 1 is defined to be
Revised: In particular, we use this extra information to construct a novel importance-weighted estimator for the loss, which for t > 1 is defined to be

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original:  where ˆ P t ( i ) =
Revised: 6) where ˆ P t ( i ) =

Original: P j ∈ N inˆ Gt
Revised: P j ∈ N inˆ Gt

Original: ( i ) π t ( j )ˆ
Revised: ( i ) π t ( j )ˆ

Original: p t
Revised: p t

Original: ( j, i ) is the estimated probability of observing the loss of arm i at round t , π t ( i ) is the distribution we sample I t from, and ˆ G t is the support of the estimated graph ˆ G t .
Revised: ( j, i ) is the estimated probability of observing the loss of arm i at round t , π t ( i ) is the distribution we sample I t from, and ˆ G t is the support of the estimated graph ˆ G t .

Original: Note that we ignore losses that we receive due to missing edges in ˆ G t .
Revised: Note that we ignore losses that we receive due to missing edges in ˆ G t .

Original: We show that we pay an additive term in the regret for wrongly estimating an edge, which is why it is important to control which edges are in ˆ G t .
Revised: We show that we pay an additive term in the regret for wrongly estimating an edge, which is why it is important to control which edges are in ˆ G t .

Original: Ideally, we would use
Revised: Ideally, we would use

Original: P t ( i ) =
Revised: P t ( i ) =

Original: P j ∈ N inˆ Gt
Revised: P j ∈ N inˆ Gt

Original: ( i ) π t ( j ) p ( j, i ) rather than ˆ P t ( i ), as this is the true probability of observing the loss of arm i in round t .
Revised: ( i ) π t ( j ) p ( j, i ) rather than ˆ P t ( i ), as this is the true probability of observing the loss of arm i in round t .

Original: However, since we do not have access to p ( j, i ), we use instead an upper confidence estimate of p ( j, i ) for rounds t ≥ 2 given by
Revised: However, since we do not have access to p ( j, i ), we use instead an upper confidence estimate of p ( j, i ) for rounds t ≥ 2 given by

Original: [Equation]
Revised: [Equation]

Original: where ˜ p t ( j, i ) =
Revised: where ˜ p t ( j, i ) =

Original: 1 t − 1 P t − 1 s =

Revised: 1 t − 1 P t − 1 s =

Original: I { ( j,i ) ∈
Revised: I { ( j,i ) ∈

Original: E s } .
Revised: E s } .

Original: We denote by ˆ G UCB t
Revised: We denote by ˆ G UCB t

Original: the stochastic graph with edge
Revised: the stochastic graph with edge

Original: probabilities ˆ p t ( j, i ).
Revised: probabilities ˆ p t ( j, i ).

Original: Note that the support of ˆ G UCB t is a complete graph because ˆ p t ( j, i ) > 0 for all ( j, i ) ∈
Revised: Note that the support of ˆ G UCB t is a complete graph because ˆ p t ( j, i ) > 0 for all ( j, i ) ∈

Original: V × V .
Revised: V × V .

Original: These estimators for the edge probabilities are sufficiently good for our purposes whenever event K occurs, which we define as the event that, for all t ≥ 2, | ˜ p [Equation] [Equation]
Revised: These estimators for the edge probabilities are sufficiently good for our purposes whenever event K occurs, which we define as the event that, for all t ≥ 2, | ˜ p t ( j, i ) − p ( j, i )

Original: j, i )
Revised: |

Original: [Equation]
Revised: [Equation]

Original: An important property of ˜ ℓ t can be found in Lemma 1 below.
Revised: An important property of ˜ ℓ t can be found in Lemma 1 below.

Original: It tells us that we may treat ˜ ℓ t as if event K is always realized, i.e., ˆ p t ( j, i ) is always an upper bound estimator on p ( j, i ).
Revised: It tells us that we may treat ˜ ℓ t as if event K is always realized, i.e., ˆ p t ( j, i ) is always an upper bound estimator on p ( j, i ).

Original: The proof of Lemma 1 is implied by Lemma 6 in Appendix D.
Revised: The proof of Lemma 1 is implied by Lemma 6 in Appendix D.

Original: Lemma 1 (Informal) .
Revised: Lemma 1 (Informal) .

Original: Let e k denote the basis vector with e k ( i )
Revised: Let e k denote the basis vector with e k ( i )

Original: =
Revised: =

Original: I { i = k } as i -th entry for each i ∈
Revised: I { i = k } as i -th entry for each i ∈

Original: [ K ] .
Revised: [ K ] .

Original: The loss estimate ˜ ℓ t defined in (6) satisfies
Revised: The loss estimate ˜ ℓ t defined in (6) satisfies

Original: [Equation]
Revised: [Equation]

Original: Lemma 1 shows that we only suffer ˜ O
Revised: Lemma 1 shows that we only suffer ˜ O

Original: (cid:16)qP Tt =2 P
Revised: (cid:16)qP Tt =2 P

Original: Ki =1 π t ( i ) P t ( i ) (cid:17) additional regret compared to when we know p ( j, i ).
Revised: Ki =1 π t ( i ) P t ( i ) (cid:17) additional regret compared to when we know p ( j, i ).

Original: Lemma 1 also shows that ˜ ℓ t behaves nicely in the sense that, conditioned on K , we have ˜ ℓ t ( i ) ≤ ℓ t ( i ) P t ( i ) I { i ∈ N out
Revised: Lemma 1 also shows that ˜ ℓ t behaves nicely in the sense that, conditioned on K , we have ˜ ℓ t ( i ) ≤ ℓ t ( i ) P t ( i ) I { i ∈ N out

Original: Gt
Revised: Gt

Original: ( I t ) }∩{ i ∈
Revised: ( I t ) }∩{ i ∈

Original: N outˆ Gt
Revised: N outˆ Gt

Original: ( I t ) } .
Revised: ( I t ) } .

Original: This is a crucial property to bound the regret of our algorithm.
Revised: This is a crucial property to bound the regret of our algorithm.

Original: We show that, with a modified version of Exp3.G [Alon et al., 2015], the second sum on the right-hand side of (7) is bounded by a term of order qP
Revised: We show that, with a modified version of Exp3.G [Alon et al., 2015], the second sum on the right-hand side of (7) is bounded by a term of order qP

Original: Tt =
Revised: Tt =

Original: 2 P Ki =1 π t ( i ) P t ( i ) , meaning that the regret is also bounded similarly.
Revised: 2 P Ki =1 π t ( i ) P t ( i ) , meaning that the regret is also bounded similarly.

Original: Our final step is to prove that the above term is bounded in terms of the minimum of the weighted independence number and the weighted weak domination number plus self-observability.
Revised: Our final step is to prove that the above term is bounded in terms of the minimum of the weighted independence number and the weighted weak domination number plus self-observability.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: Nicolò Cesa-Bianchi, Federico Fusco and Dirk van der Hoeven gratefully acknowledge partial support by the MIUR PRIN grant Algorithms, Games, and Digital Markets (ALGADIMAR).
Revised: Nicolò Cesa-Bianchi, Federico Fusco and Dirk van der Hoeven gratefully acknowledge partial support by the MIUR PRIN grant Algorithms, Games, and Digital Markets (ALGADIMAR).

Original: Nicolò Cesa-Bianchi and Emmanuel Esposito were also supported by the EU Horizon
Revised: Nicolò Cesa-Bianchi and Emmanuel Esposito were also supported by the EU Horizon

Original: ICT-48 research and innovation action under grant agreement 951847, project ELISE.
Revised: ICT-48 research and innovation action under grant agreement 951847, project ELISE, and by the project “One Health Action Hub: University Task Force for the resilience of territorial ecosystems” funded by Università degli Studi di Milano.

Original: Federico Fusco was also supported by the ERC Advanced Grant 788893 AMDROMA “Algorithmic and Mechanism Design Research in Online Markets”.
Revised: Federico Fusco was also supported by the ERC Advanced Grant 788893 AMDROMA “Algorithmic and Mechanism Design Research in Online Markets”.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] All the results claimed in the abstract have proofs either in the main body or appendix (b) Did you describe the limitations of your work?
Revised: [Yes] All the results claimed in the abstract have proofs either in the main body or appendix (b) Did you describe the limitations of your work?

Original: [Yes] For each result we state the parameters under which the result holds.
Revised: [Yes] For each result we state the parameters under which the result holds.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] Our work is primarily theoretical and we do not foresee any potential negative societal impacts of our work.
Revised: [No] Our work is primarily theoretical and we do not foresee any potential negative societal impacts of our work.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes]
Revised: [Yes]

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] (b) Did you include complete proofs of all theoretical results?
Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Original: [Yes] Although some proofs may only appear in the appendix. 3.
Revised: [Yes] Although some proofs may only appear in the appendix. 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [N/A] (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [N/A] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [N/A] 4.
Revised: [N/A] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [N/A] (b) Did you mention the license of the assets?
Revised: [N/A] (b) Did you mention the license of the assets?

Original: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A] (d) Did you discuss whether
Revised: [N/A] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]