Thompson Sampling For Bandits With Cool-Down Periods
Authors: Jingxuan Zhu
Bin Liu
Venue: Accepted by TMLR ()

Abstract: This paper investigates a variation of dynamic bandits, characterized by arms that follow a periodic availability pattern. Upon a "successful" selection, each arm transitions to an inactive state and requires a possibly unknown cool-down period before becoming active again. We devise Thompson Sampling algorithms specifically designed for this problem, guaranteeing logarithmic regrets. Notably, this work is the first to address scenarios in which the agent lacks knowledge of each arm's active state. Furthermore, the theoretical findings extend to the sleeping bandit framework, offering a notably superior regret bound compared to existing literature.

Published in Transactions on Machine Learning Research (10/2025)
Thompson Sampling For Bandits With Cool-Down Periods
Jingxuan Zhu
Shajecs@@chinnteeeeeoo.ccr
B-Surfing Digital Life Technology Co., Ltd.
China Telecom
Bin Liu
bins Dieze.oryy.
E-Surfing Digital Life Technology Co., Ltd.
China Telecom
Reviewed on OpenReview: htips: // openreview. net/ form?? id- Ifv0ZS2mXm
Abstract
This paper investigates a variation of dynamic bandits, characterized by arms that follow a periodic availability pattern. Upon a "successful" selection, each arm transitions to an inactive state and requires a possibly unknown cool-down period before becoming active again. We devise Thompson Sampling algorithms specifically designed for this problem, guaranteeing logarithmic regrets. Notably, this work is the first to address scenarios in which the agent lacks knowledge of each arm's active state. Furthermore, the theoretical findings extend to the siceping bandit framework, offering a notably superior regret bound compared to existing literature.
1 Introduction
Within the realm of multi-armed bandits (MAB), agents navigate the delicate balance between exploration and exploitation to maximize cunulative rewards. The MAB framework has been applied to a diverse array of practical scenarios, inchuding but not limited to dynamic pricing (Misra et al. 2019 online learning of dialogue systems (Liu et al. 2018) and anomaly detection (Ding et al. 2019 Our research is focused on the stochastic multi-armed bandit problem, fundamental variant within this domain, where rewards for each action are drawn from stochastic distribution that is unknown the agent. The agent's objective is to discern and select the arm that promises the highest average reward over time.
The literature has introduced spectrum of algorithms to tackle this problem within the standard frame-
work, prominently featuring the Upper Confidence Bound (UCB) algorithm suite (Auer et al
2002 Garivier
& Cappé 2011
Kaufmann et al.) 2012), alongside "greedy" approaches (Langford & Zhang 2007 Vermorel
& Mohri 2005
and Thompson sampling method [Thompson] 1933). Additionally, foundations work by
1985 establishes a theoretical lower bound for the performance these contexts. In recent years, the research focus has shifted towards multi-agent variations of these problems, characterized by distinctions in agent bchavior (cooperative or competitive) (Zhang et al., 2021] and how information exchange is exe- cuted KKrihhamurthy & Wahlberg 2009). Notable advancements include distributed cooperative bandit
Lai et al.
settings (Bistritz & Leshem 2018 Landgrena et al. 2021 Zhu al.
2021
Martínez-Rai Rubio al. 2019 Zhu
& Liu 2023), where multiple agents collaboratively achieve objectives through distributed communication
networks; The "collision" setling (Nayyar et al. 2016 Liu & Zhao
2010 Bistritz & Leshem 2018 Lai el al.
2008), where agents "collide" when they select a same arm at the same time; and federated in information
structures (Shi Shen 2021 Shi et al.] 2021 Zhu et al. 2021 Dubey & Pentland, 2020 Li et al. 2020
Réda et al. 2022). which explore personalized and federated strategies for leveraging shared knowledge.
Despite the rich body of literature, there prevailing assumption that the set of arms and their reward distributions remain unchanged over time, simplification that often diverges from the dynamic nature of decision-making in real-world scenarios. Real-life applications frequently feature changing options and reward distributions, leading to the emergence of new topic: "dynamic bandit" or "non-stationary bandit"
Corresponding author

---

Published in Transactions on Machine Learning Research (10/2025)
problems. The literature encompasses variety of these scenarios, including the restless bandit framework where arm states evolve based on an irreducible Markow chain (Ortner et al., 2012). scenarios with a finite
number of distribution switches (Kocsis & Szepesvári 2006 Garivier & Moulines 2011 Auer et al. 2019) and the "sleeping bandit" problem where only a subset of arms is active at any given time (Kleinberg et al. 2010 Chalterjee et al. 2017
To address the dynamic bandit problem while maintaining logarithmic regret, our work builds upon the "sleeping bandit' framework, which accounts for the dynamic availability of options. To generalize this set- ting, we introduce novel extension by incorporating structured and predictable pattern in the distribu- tional changes. Specifically, we propose model where each option, or arm, is assigned unique 'cool-down' period, which may not be explicitly revealed to the agent. Upon a "successful" selection of an arm, it becomes temporarily inactive and only yield O-rewards. This model mirrors real-world scenarios where options are not continuously available, offering a more nuanced approach to the exploration-cxploitaaii dilemma. This setup finds relevance in situations where choices exhibit periodic accessibility, such as:
Cognitive Radio Networks In this scenario, cognitive radio (Wang & Liu 2010: Masonta et al. 2012) aims to assign the most suitable active channel (arm) for sequence of communication demands. Each channel possesses a distinct success rate for successful communication. Upon commincattion passing through a channel, it becomes occupied for a certain period (cool-down period), during which selecting it again would lead to a "collision' (Liu & Zhao 2010 resulting in the failure of subsequent communication attempts. To devise an optimal allocation strategy, the agent must effectively acquire knowledge about both the success rate and the cool-down duration of each channel.
Coupon Website In this context, we consider platform that collects promotional vouchers for shoppers. The availability of these vouchers to any given customer is subject to both cool-down constraints and targeling range limitations. These vouchers may only be redeemed once per day or week due to cool-down limitations, and they are distributed to invited users in specific ratios. The goal of the website is to maximize its aggregate positive feedback, measured by the successful redemption of coupons (positive feedback), whereas attempts to use a voucher during its cool-down period or by users not targeted by the promotion result. no reward (negative feedback).
Gaming In gaming scenarios, whether against computers other players, certain events abilities are designed with cool-down times to ensure fair competition. Additionally, different abilities yield varying outcomes. A player, seeking to secure victories and maximize the reward, must make strategic decisions about which abilitics to use and their timing, based on the cool-down durations and the specific benefits of each ability.
1.1 Contributions
This paper delves into special dynamic bandit problem where each arm undergoes cool-down period following a "successful' selection. The main contributions can be summarized as follows.
1. We present distinct Thompson-sampling based algorithms tailored for scenarios where the cool-down durations are either known or unknown to the agent. In both instances, we establish an upper bound of order Ollg T) for the regret.
2. To the best of our knowledge, this is the first paper that provides a solution to the bandit problem where the active state of each arm is unknown to the agent (the unknown cool-down duration scenario). Besides, when the agent is aware of the cool-down durations, our regret bounds boil down to the results in traditional bandit settings disrogarding cool-down periods.
3. Additionally, we extend our algorithm to the sleeping bandit framework, achieving significant improve- ments compared existing literature (Kleinberg et al. 2010 Chatterjee al. 2017).
1.2 Problem Formulation
Let us consider a situation where an agent is presented with M distinct arms or options. Each arm can be in one of two states at any given time: active or inactive. Even when inactive, an arm can still be pulled but would yield a O-reward. The set of arms that are active at time t is represented by A(t). For every discrete instance i, the agent selects arm all), guided by a predetermined policy, thereby receives
2

---

Published in Transactions on Machine Learning Research (10/2025)
reward, denoted as ((o(()()). For any given arm k, where l 1,..., M, and at each time t, should arm k be active, its reward X (8) (t) is governed by Bernoulli distribution characterized by unknown success rate p(t) within the interval (0, 1). This distribution can be represented as follows:
(1 = {
with probability p(A)
0 with probability 1 - -(()) given k E A(t).
Without loss of generality, it is assumed for simplicity that the success rates are sorted in descending order: p(1) > ((2) >.. ... > plM)) When l is inactive at time 4, x(() O.
An arm, upon being chosen and yielding 1-reward (succeeding in the Bernoulli trial) at time t, transitions into an inactive state, necessitating a cool-down period of length ((8) before it is reactivated. This inactivity period is mathematically formulated as:
1: Φ A(T), for t 1,..., 1(4) k A(T), for T = + ((k) +
Here, ((8) denotes the cool-down duration for arm k, which is constrained by a uniform upper bound D, such that (()) < D for any arm k = =, .M. We explore both scenarios in this paper where the cool-down duration (k)) is either known or unknown to the agent, with the agent being cognizant of the upper bound D in situations where ((k) remains undisclosed.
The agent's primary objective is to minimize its regret, which is defined the expected cumulative "loss" in reward resulting from not choosing the optimal arm at every instance and is mathematically described
as:
R(T) = Enplr()) - p(((()))
-
where (t) arg A(()) p() identifies the optimal arm at time t, which is with the highest success rate among all active arms.
1.3 Technical Challenges
In this section, we mainly present the technical challenges when the cool-down duration of each is unknown to the agent. Addressing the problem requires the agent to thoroughly explore both the success rale and the cool-down duralion for each arm. This exploration faces three significant technical hurdles:
Limited Exploration of Distributions: Insufficient exploration of the success rates of each arm may lead to inaccurate estimations by the agent. Consoquently, the agent may repeatedly select a sub-optimal arm, resulting in considerable regret.
Inadequate Exploration of Cool-Down Durations: Insufficient exploration on the cool-down duration can result in two types of errors: (1) the agent may erroncously choose an inactive arm that it believes to be active, or (2) it might overlook the optimal active arm due to a mistaken belief that it is inactive. Both errors lead to sub-optimal selections. Particularly in the first scenario, incorporating the O-reward obtained due to selecting an inactive arm into its success rate estimation could bias the estimate, potentially causing the regret to escalate linearly. We demonstrate in Section 6.2 the repercussions of not implementing a cool-down exploration strategy for this bandit problem.
Over-Exploration of Cool-Down Durations: In the process of validating the cool-down duration for arm, the agent may deliberately select arm presumed to be inactive to gauge the precision of its cool- down estimate. Consequently, this procedure mandates selecting an inactive arm at least once during each verification attempt, inherently resulting in loss in reward as a direct consequence of cool-down exploration. That is why excessive exploration of cool-down duration can exacerbate regret. Further elaboration on the ramifications of explooxpoooi of the cool-down durations is provided in Section
In conclusion, the agent must navigate both distribution (success rate) exploration and cool-down exploration simultaneously. This necessitates delicate equilibrium between the two explorations. Moreover, to mitigate
3

---

Published in Transactions on Machine Learning Research (10/2025)
the risk of sub-optimal selections stemming from excessive cool-down exploration, it is essential to devise strategy enabling the agent to strategically pause resume cool-down exploration based on its confidence level in the cool-down estimate.
2 Algorithms
To simplify the presentation, we generally exclude the factor t from most notations used in the pseudocode this section and Appendix
Algorithm 1: Known Cool-Down Durations
Input: total time T, upper bound D Initialization For k = 1,..., M, set (* = (8()) = 1. Set & = 0.
while & < T do
t= 1 for k A do
The agent samples ((k) from Beta(a B(K)
end
The agent selects arm a(t) arg m (k) if Xa(1) = then
// Thompaon Sampling
((e()) (a((t)) +1
else 10
g(a(e)) = g(a(t) +1
11
12
end
13
end
2.1 Known Cool- Down Duration
When the cool-down duration of cach arm is known to the agent, the problem is equivalent to finding the optimal arm within a given subset A()) at each time L. To achieve this, we made slight adjustments to the Thompson Sampling algorithm, sampling only from the active arm set rather than the entire arm set at each time step, as outlined in Algorithm
2.2 Unknown Cool-Down Duration
When the cool-down duration each arm is unknown to the agent, as discussed in Section crucial to strike balance between cool-down exploration and distribution exploration. The agent must first determine which arm select and decide the type of exploration to undertake before executing the corresponding actions. In the following, present the framework of the algorithm along with the underlying intuition. A table summarizing key concepts and variables used in the algorithm is provided in Table D The main structure of the algorithm is given in Algorithm 2 and the pseudocode of the modules used is included in Appendix B
1.3 it becomes
Decision Making
During each time step t, the agent chooses an arm a(t) from the belief arm set B(t) The belief arm set is collection of arms that is ready for either distribution exploration cool-down exploration. The selection for a(t) is governed by modified Thompson Sampling algorithm: in most cases, the agent adheres to the standard Thompson Sampling rule, except when an arm is deemed "urgent"" for cool-down exploration. This modification addresses scenarios where conventional Thompson Sampling might fail to adequately prioritize cool-down exploration for certain important arms (as defined in Section 3.1 Without sufficient cool-down exploration, the reward mean estimate for these arms may become unreliable, leading to suboptimal decisions. By incorporating this modification ion, the agent ensures that important arms receive the necessary exploration, balancing immediate reward optimization with long-term Decision-making efficacy. Additional details about the bifurcated decision-making approach provided in Section 6.3 and the full implementation of the decision-making process is described in Algorithmm
4f B()) 6, then the agent randomly selects arm according to A Algorithm

---

Published in Transactions on Machine Learning Research (10/2025)
Table 1: Summary of Critical Variables and Concepts
Concept
Definition or Role in the Algorithms
a (t),)
Parameters for agent's estimate of success rate (beta distribution)
B(t)
The belief arm set, which stands for the collection of arms that are ready for dis- tribution exploration cool-down exploration at time (check more information in CD CD_Explore (0) (t)).
(1)
A collection of arms that are at the end of the cool-down period in the agent's belief at time t.
CD "xplooere'' (8)
A variable used for estimating the active state of arm k. When CD Explore (t) = oo, arm k is ensured to be active and ready for distribu- tion exploration; when CD Eaplore (8) (1) s 0, arm k ready for cool-down exploration.
check (A)
The accumulated count of times the agent engages in cool-down exploration for arm i without attaining a 1-reward in (c= (k) (f))-run. This variable serves to gauge the agent's confidence in T.(²(()) (t).
The collection of time instances t satisfying (k) (t)
c-run
I
Important arm set, collection of arms that are the optimal active arms with high frequency, see Section 3.1
last (D(
The last time arm k is selected for cool-down exploration before time L. The actual cool-down duration of arm k. A variable characterizes the sufficiency of cool-down exploration: if ( S T,(k), then the cool-down exploration is considered as sufficient. The definition is in Section 3.2
1(k)
T,K))
I (t)
The estimated cool-down duration of arm at time t.
T,022 (1)
An intermediate variable used for testing if T. (1) reaches the actual cool-down duration (08), which is designed to satisly (())(()) < (x) (1), with the equality holds only when the agent has enough confidence to believe L (t) (8))
(()(())
((0))
The actual reward mean / success rale of arm k.
(t))(t)
The estimated success rate of arm i at time t, defined by (k)) (()))))))))))))
20
waillist (1)
A signal indicating whether the cool-down exploration of arm & is suspended. Specifically, wwillist (x) (1) = signifies cool-down exploration suspension at time for arm k, and otherwise.
Distribution Exploration
During distribution exploration, the agent takes single sample from arm a(t) and updates a (a(t))(t and g(a(t)) according to the following equations:
=(+1))=a()(())) + (1) 1),
g00 (() xx)
The distribution exploration process given in Algorithm in Appendix
B
Cool-Down Exploration
The cool-down exploration for each arm & comprises two stages: the quick jump stage and the confirmation stage, which is decided by comparing the estimated cool-down duration L (1) and an intermediate testing value I (k) (t). The quick jump stage occurs when (t) < (1) (t) 1. During this phase, the agent utilizes
bisection algorithm to promptly update the estimated cool-down duration I (8) (t) based on sampling
5

---

1/ if no arm is ready for either exploration, then randomly select an arm Published in Transactions on Machine Learning Research (10/2025)
Algorithm 2: Unknown Cool-Down Durations
Input: total time T, upper bound D
Initialization For k = 1,..., M, set (*)) = 3(h) = 1, I!(h = D, L(k] = min - 1, 21), (
CD "'plooeee" (k) oc, check (k) waitlist O, last CD 0. Set 0. test 0,
while & < T do
==++1
for k = 1,..., M do
The agent samples (00) from B g(k)
end
Let B be the set of arm k satisfying CD Explore (k) 5 or CD Explore are ready for cool-down exploration or distribution exploration
= =0 // those arns that
if + then
Let B' be collection of arm A in B satislying CD Explore (6) = L (k)
(k) +1
test
if 13' * and minkes last_ CD last CD CD(arg maxke 6(k)) // urgent cool-down exploration
requirement
11
then
a(t) arg minker last CD (8)
last_CD
// urgent cool-down exploration
13
else
14
at) = arg max (kk)
// Thompson Sampling rule
15
end
if CD Explore then
18
Distribution Exploration
else 19
last (D((()))
CD exploration
21
end
22
else
23
Random Sample
24
end
Post- Update
28
end
outcomes. The confirmation stage arises when ( = (t) - Here, the agent carefully refines its
confidence in L (k) (t). The cool-down exploration process is delineated in Algorithm in Appendix B
Suspending or Restarting of Cool-Down Exploration
To avoid insufficient and excessive cool-down exploration simultaneously, dynamic criterion is employed to restart, suspend, or terminate the cool-down exploration. This criterion measures agent's confidence on the estimated cool-down duration by comparing (1 (1-p(*) (t)) (he))) (+) with Flogr where ((A) (t) and check (k) (f) are defined in Table D and based on which, helps the agent to decide whether to suspend or restart the cool-down exploration for arm k. By doing so, we ensure very low probability insufficient cool-down exploration, see the analysis in Appendix A.4.2 while preventing excessive exploration, as detailed in Lemma Further
1
details and variable updates are provided in Algorithm D
3
Additional Concepts
3.1 Important Arms
To delincate the important arm sct, let us start by examining simplified scenario where the reward distri- bution for each arm constant p(*), rather than Bernoulli distribution, while maintaining all other settings. The agent, possesses knowledge of both (k) and ((k) for each 1,..., M. Tn this setup,

---

Published in Transactions on Machine Learning Research (10/2025)
the optimal policy becomes deterministic: the agent consistently selects arg max ((), Additionally, it is evident that the chosen arm follows periodic pattern given that all arms are initialized to be active at = O. Let P denote the repetitive sequence of labels corresponding to the arms chosen in each period, the important arm set then defined as the non-repetitive copy of P. Return to the bandit sctting studied in this paper, the following lemma shows the property of I defined as above.
Lemma If the bandit algorithm satisfies that for every a > O, the number of sub- sub-optimal selects is o(t"), then for any arm E I and E (0,1), P(TP(()) < (²) S O(e For any arm j $ I and w (0,1), P(TT((1) > (²) S ).
The above lemma shows that the frequency of an important arm being the optimal active arm is high while that of the other arms is extremely low.
3.2 "Sufficient" Cool-Down Exploration
We mentioned in Section 1.3 that. insufficient cool-down exporation is likely to result in a linear regret. To quantify the "sufficiency" of the cool-down exploration, we define ((k) as the largest value such that I remains unchanged if the cool-down duration of arm & changes from ((k) to L(k), this sense, for & I, L D.
We say the cool-down exploration for arm k is sufficient for arm & at time if and only if (8) (t) s (k).
Note that for each arm k 1,..., M, ( may still be updated after it reaches (k))
4 Theoretical Results and Comparisons
4.1 Known Cool-Down Durations
In the scenario where the cool-down duration ((*) is known to the agent for each arm k present the following lower bound result and an asymptotically optimal upper bound result using Algorithm | Theorem (Lower Bound). If the agent is aware of (00) for each arm k == I,...,M. M,, then the lower bound of the regret satisfies
( 1,12)) -p(5)
R(T) Σ
j=2 K(plt), p(minl)--1 log T c0.
where K(p. (pp,9) denotes the Kullbcck- Leibler divergence.
K(p, 9) = pllog P + (1 p) log 1-9
1-p p
q
Theorem 2 (Upper Bound). If the agent aware of the cool-down duration of each arm, then under Algorithm for any c, there exist a constant C(c) O such that the regret satisfies that
M j-1 (plt) ((t)) logT + C(e).
R(T) S (1+))
ΣΣ 1= K(pli,, p(mmints,1111))
Remark (Comparison with conventional bandit results). The classic bandit problem without consid- ering the dool-down durations can be regarded as a special instance within our framework, wherein each
arm k = =,... 1,...,M satisfies [(k) = O. [La et al. 1985) provides an asymptotic lower bound with order
MEM M (()) ((t))
K(plat T)g and Agrawal & Goyal, 2017) provides (near-oplimal) asymptolic upper bound ( -pt)-pis)
with order ((11 + E) E ( gpi logT") by using Thompson sampling. Note that I {1} when (k)) 0
for each ko = 1...., M, then the regret bound in Theorem satisfies that
(p()) 1-)((ttt) p((+))) log
R(T) < (1+c)
+ +Ce)
J=2 k(p p()) =11 + e)
(()) pli)
Σ
K ((pl)), p(1l) + C(e).
It is casy to see both our lower bound in Theorem and the upper bound in Theorem boil down to the classic results under the conventional bandil selting.

---

Published in Transactions on Machine Learning Research (10/2025)
The detailed analysis for Theorems 1 and 2 can be found in Appendix
A
4.2 Sleeping Bandit Scenario
In the general sleeping bandit framework addressed (Kleinberg et al. 2010 Chatterjee et al. 2017]; a
periodic pattern of arm availability is not presupposed. The set A(t), representing the available arms any given time t, may not adhere to any predictable pattern and is known before for all t 0. Nevertheless, it is assumed that A(t) is known to the agent at time t. Under these conditions, Algorithm 1 is still applicable to this scenario. Furthermore, by defining I = (I,..., M), the analysis conducted in the proof of Theorem 2 can be directly transferred to this context. Thus, we are able to establish the following regret upper bound for the general sleeping bandit setting.
Theorem 3. For the sleeping bandit scenario where the agent is aware of A(t) at each time instance t, then under Algorithom for any 0, there exists a constant C(c) = 0 such that the regret satisfies
j-1 (p(t) -p(it)) log +((c). ΣΣ 2=2
R(T) S (1+
((06)),10))
Remark 2 (Comparison with the sleeping bandit results). From Kleinberg el al. 2010 Challerjee el al. 2017), the regret upper bound satisfies that
R(T) log T
M
32(p!)) -((+1)) p()) -p()) + +o(1).
By using Pinsker's 's inequality, we cun further bound the regret in Theorem 3 as
M 1-1 (1 (1(0 - plitt))
R(T))
+ +0(1)
log T
K(pb),,
M (1 (p((0 - piitt!)
2(p() p(il) + o(1).
The regret bound we derive significantly surpasses those found in eristing literalure, primarily due to two key factors. Firstly, we employ the Thompson Sampling algorithm instead of UCB1, as utilized in Kleinberg et al., 2010), leading to a superior bound. Secondly, in comparison to [Chatterjee et al.) 2017), we inte- grate the near- optrnal bound technique outlined in (Agraaal & Goyal, 2017| into the time-vorying scenario, resulting an enhanced upper bound for the number of sub-optimal selections. We have also implemented numerical comparison, see Figure 7
4.3 Unknown Cool-Down Durations
Theorem (Upper Bound). For any > O, there exist constant Ci(c) = O(1) and Cz(e) = 00- loo such that under Algorithm the vegret satisfies
M
2(p(0) - p(++1)
+ + +1)
R(T)- S (1 + min
K (plt), p(mmin(s/2111))
M
(p(()) D(++12)
M
(1 + c)p(()
log
j=2 (ppti), ( p(mmi((.]I(1)
log - p(l))
j=2
+e(()) log C2(e),
Remark 3 (Comparison with known cool-down duration setting). The additional parts compared with the regret bound Theorem g is mostly due to the additional efforts the agent makes on learning the cool-down duration of each arm, as is analyzed in the proof of Theorem 4 The comparison of the practical performance of the agent under both settings we consider is given in Figure

---

Published in Transactions on Machine Learning Research (10/2025)
In below, provide the sketched proof of Theorem i the complete proof can be found in Appendix A
Sketched proof of Theorem In the proof of Theorem A identify four situations where loss in reward may occur: the first situation arises from insufficient distribution exploration, the second from insufficient cool-down exploration, and the third and fourth from excessive cool-down explorations. To quantily the error in the first situation, we utilize Corollaries g and 2 extending the conventional uncertainty analysis associated with Thompson Sampling to our problem sctting. For the second situation, we show in Lemmas X and f that the probability of excluding the optimal arm from B(I) due insufficient cool-down exploration diminishes over time under Algorithm The assessments for the third and fourth situations employ both the analysis in Lemma and Lemmna with the latter one establishing an upper bound on unnecessrry cool-down explorations once the agent's estimated cool-down duration for each arm reaches the true value. Note that the proof of Lemma necessitates an assessment. of the accuracy of (t) for each arm & and time t, which is provided in Lemma Σ
5 Numerical Evaluations
In this section, we evaluate the performance of our proposed algorithms. We consider a 10-armed bandit problem, each arm follows a Bernoulli distribution with distinct success rate. For all the experiments this section, we set the cool-down duration upper bound D as 10. The total time T' to be 100000 and we provide the averaged performance over 50 runs with the corresponding error bar.
This paper is the first address the bandit scenario in which the agent is unaware of whether each arm is active, referred to the unknown cool-down duration scenario. Consequently, direct baseline comparison is available for this setting. Instead, we benchmark our approach against existing methods from the sleeping bandit literature. The result is presented in Figure D Furthermore, we evaluate the agent's performance when the cool-down durations are known (Algorithm ! versus unknown (Algorithm 2 to quantify the performance degradation caused by the lack of this knowledge. We also investigate the special case where ((k) = for all & I,....,M, M, corresponding to the classic bandit framework. In this case, Algorithm | aligns with the conventional Thompson Sampling algorithm. The performance results for these scenarios are
provided in Figures 2a and 2b
400
sleeping bandit [31] Algorithm
350
300
250
reget
200
150
100
50
0
2000
4000
6000
8000
10000
time
Figure 1: Numerical comparison for Algorithm and that in Chatterjee el. al. 2017 in sleeping bandit scnario
Figure [ clearly illustrates the superiority of our algorithm compared to existing method in the sleeping bandit scenario, aligning with the theoretical comparison discussed in Remark / Examination of Figures 2a and 2b reveals that in both scenarios studied in this paper, the regret trends logarithmically. This observation

---

Published in Transactions on Machine Learning Research (10/2025)
140
120
cool-down durations unknown
cool-down durations unknown
cool-dow down durations known
cool-down durations known
120
100
100
80
reget
80
reget
60
60
40
40
20
20
time
x10
time
x10*
(a) ((k) is generated randomly for all arm k
(b) (() 0 for all arm k
Figure 2: Numerical comparison for Algorithm and Algorithm
aligns with the theoretical results presented in Theorem and Theorem Moreover, despite the integration of an additional cool-down exploration process and the complexity added by the decision-making bifurcation, the regret incurred when cool-down durations unknown does not significantly exceed that observed when the cool-down durations are known. This demonstrates the elfectiveness and advanlage of our specifically tailored cool-down exploration and decision- makiin processes.
6
Further Discussions and Simulations
6.1 More General Reward Setting
Although this paper focuses on the Bernoulli reward distribution for simplicity, both Algorithm and Algorithm 2 can be readily extended to more general stochastic reward settings. This follows from the fact that both algorithms employ the Thompson sampling principle, which naturally generalizes to arbitrary stochastic rewards (see Algorithm
(Agrawal & Goyal, 2012)). With corresponding modifications in the analysis, our results can thus be extended general stochastic reward distributions.
6.2 Naive Thompson Sampling
As mentioned in Section 1.3 employing conventional Thompson Sampling without any cool-down exploration process typically results in linear regret in scenarios where each arm has a nonzero cool-down duration. This is because doing so would result in the agent incorporating the O-reward obtained due selecting an inactive into its success rate estimation, which biases the estimate, and consequently may cause the regret to escalate linearly. To better illustrate this, conduct the following simulation by considering a 10-armed bandit problem, with each arm following Bernoulli distribution characterized by distinct success rates. The total time T is set to 10000. Figure depicts the averaged curve over 50 runs.
6.3 Bifurcation in Decision-
Decision-Making Making
For scenarios where the agent is unaware of the cool-down duration of each arm, introduce bifurcated decision-making in Algorithm i.e., lines 10-15 in Algorithm 2 As discussed in Decision Making in Section 2.2 the agent chooses between the optimal arm in the agent's 's belief decided by Thompson Sampling rule (line 14 in Algorithm 2 and the one that is the most urgent for cool-down exploration (line 12 in Algorithm 2 Whether an arm deemed as "urgent" for cool-down exploration is decided by two factors: (1) whether at the end of the current cool-down period (2) the last time selected for cool-down exploration (see line 10 in Algorithm 2 Such a decision-making criteria sometimes prompts the agent to choose arm other than the optimal arm in the agent's belief, as otherwise, the cool-down duration of some important arms may never be sufficiently explored. We will use example to further explain this. Suppose time there exists sub-optimal arm & satisfying (() (1a (t) ((()) 1. For such situation, whenever k becomes inactive, the agent has only one opportunity to initiate cool-down exploration
10

---

Published in Transactions on Machine Learning Research (10/2025)
1200
Algorithm
naive Thompson Sampling
1000
800
reget
600
400
200
0
0
2000
4000
6000
8000
10000
time
Figure 3: Simulation results for Algorithm and naive Thompson Sampling when cool-down durations are unknown
it. If, at that moment, there exists "superior' arm with a higher estimated success rate in the agent's 's belief arm set, without adding the decision- decision-making bifurcation, the agent is likely to choose the superior arm based on the Thompson Sampling rule instead of exploring the cool-down duration of arm k. this situation recurs frequently, the agent may never acquire sufficient exploration of the cool-down duration of arm k. We conduct the following simulation to better elucidate the necessity of the bifurcated decision-males process.
Consider the following example, there are three arms with success rates 0.9, 0.8, 0.5 and cool-down durations 2, O, respectively. Let D = 2. Since ((2) = O, arm always active. Let us assume the agent fails to obtain reward during the first 5 cool-down explorations of arm 2, which happens with probability at least 0. Given T' 10000, the performance results of the agent under Algorithm 2 and that without the decision-making bifurcation are illustrated in Figure
600
Algorithm 2
without decision-making bifurcation
400
reget
300
200
100
0
0
2000
4000
6000
8000
10000
time
Figure 4: Simulation results for Algorithm
and the algorithm without decision-making bifurcation
11

---

Published in Transactions on Machine Learning Research (10/2025)
In Figure 1 it is evident that the worst performance, along with the variance of regret, is significant. Since the probability of the situation in the simulation is constant-large value, it's plausible that the agent may experience linear regret without the designed decision-maiee making bifurcation.
6.4 Non-Stop Cool- Rowl-Down Exploration
As detailed in Section addressing the problem when the cool-down durations are unknown necessitates policy for the agent to strategically pause and resumne the cool-down exploration for each arm, thereby preventing excessive exploration of the cool-down duration. This process is delineated in Lines 3-20 of Algorithm g the subsequent simulation, we explore the ramifications of not implementing such a process. We examine a 10-armed bandit problem, where each arm follows a Bernoulli distribution characterized by distinct success rates. The total time T is to 10000, with total of 50 runs. The simulation results are presented Figure
It can be clearly observed from Figure that in the absence of criteria for pausing resuming cool-down exploration, the agent experiences linear regret, despite accurately estimating both the success rale and cool-down duration for each arm. This observation aligns with the explanation provided in Section 1.3
4000
Algorithm 2
3500
non-stop cool-down exploration
3000
2500
reget
2000
1500
1000
500
0
0
2000
4000
6000
8000
10000
time
Figure 5: Simulation results for Al Algorithm and the without stopping policy
7 Conclusion
We explore a variant of dynamic bandits in which each arm's availability follows periodic pattern. We have developed modified Thompson Sampling algorithms tailored for scenarios where the agent is either aware or unaware of the cool-down duration of each arm. Both adaptations are demonstrated to achieve logarithmic regret. Notably, the algorithm designed for the unknown cool-down duration scenario is the first to tackle the bandit problem where the active state of each arm unknown. Furthermore, in the case where the cool- down duration known-parwaiee particularriyy in the sleeping bandit setting our algorithm demonstrates superior performance compared to existing approaches (Kleinberg al. 2010 Chatterjee et al., 2017). Potential directions for future work include exploring dependencies among different arms and conducting detailed evaluation of empirical regret.
12

---

Published in Transactions on Machine Learning Research (10/2025)
References
Shipra, Agrawal and Navin Goyal. Analysis of thompson sampling for the multi-armes bandit problem. In Conference on learning theory, pp. 39 1. JMLR Workshop and Conference Proceedings, 2012.
Shipra Agrawal and Navin Goyal. Near-optimal regret bounds for thompson sampling. Journal of the ACM (JACM), 64(5):1 24, 2017.
P. Auer, N. DisacBianchi, and P. Fischer. Finito-time analysis of the multiarmed bandit problem. Machine Learning, 47:235 256, 2002.
Peter Auer, Pratik Gajane, and Ronald Ortner. Adaptively tracking the best bandit arm with an unknown number of distribution changes. In Conference on Learning Theory, pp.
PMLR, 2019.
I. Bistritz and A. Leshem. Distributed multi-player bandits game of thrones approach. In 32nd Interna- tional Conference on Neural Information Processing Systems, pp. 7222-72227777 -2322 2018.
Aritra Chatterjee, Ganesh Ghalme, Shweta Jain, Rohit Vaish, and Y Narahari. Analysis of thompson sampling for stochastic sleeping bandits. In UAI, 2017.
Kaize Ding, Jundong Li, and Huan Liu. Interactive anomaly detection on attributed networks. In Proceedings of the twelfth ACM international conference on web search and data mining, pp. 357 365, 2019.
A. Dubey and A. Pentland. Diffeventially--rivate federated linear bandits. In 34th Conference on Neural Information Processing Systems, 2020.
Aurélien Garivier and Olivier Cappé. The kl-ueb algorithm for bounded stochastic bandits and beyond. In Proceedings of the 24th annual conference on learning theory, pp. 359-376. JMLR Workshop and Confer- ence Proceedings, 2011.
Aurélien Garivier and Eric Moulines. On upper-confidence bound policies for switching bandit problems. In International conference on algorithmic learning theory, pp. 174-188. Springer, 2011.
Emilie Kaulmann, Olivier Cappé, and Aurélien Garivier. On bayesian upper confidence bounds for bandit problems. In Artificial intelligence and statistics, pp. 592 600. PMLR, 2012.
Robert Kleinberg, Alexandru Niculescu-Mizil, and Yogeshwer Sharma. Regret bounds for sleeping experts and bandits. Machine learning, 80(2-3):245 272, 2010.
Levente Koesis and Csaba Szepesvári. Discounted ucb. In 2nd PASCAL Challenges Workshop, volume 2, pp. 51-134, 2006.
V. Krishnamurthy and B. Wahlberg. Partially observed Markov decision process multiarmed bandits structural results. Mathematics of Operations Research, 34(2):287-302, 2009.
L. Lai, H. Jiang, and H.V. Poor. Medium access cognitive radio networks: competitive multi-armed bandit framework. In 42nd Asilomar Conference on Signals, Systems and Computers, pp. 98-102, 2008. doi: 10.1 10.1109/A (09/ACSSC.200 CCSSC.0008.5774333
Tze Leung Lai, Herbert Robbins, et al. Asymptotically efficient. adaptive allocation rules. Advances in applied mathematics, 6(1):4 22, 1985.
P. Landgrena, V. Srivastavab, and N.F. Leonarda. Distributed cooperative decision making in multi-agent multi-armed bandits. Automatica, 2021.
John Langford and Tong Zhang. The epoch-greedy algorithm for multi-armed bandits with side information. In Advances in Neural Information Processing Systems, volume 20, 2007.
T. Li, Song, and C. Fragouli. Federated recommendation system via differential privacy. In 2020 JEEE International Symposium on Information Theory (ISIT), pp. 2592-2597, 2020. doi: 10.1109/ISIT44484. 9020.9174297.
13

---

Published in Transactions on Machine Learning Research (10/2025)
Bing Liu, Tong Yu, Ian Lane, and Ole Mengshoel. Customized nonlinear bandits for online response selection in neural conversation models. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 32, 2018.
K. Liu and Q. Zhao. Distributed learning in multi-armed bandit with multiple players. IEEE Transactions Signal Processing, 58(11):667--6681 2010.
Mlabo,, V. Kanade, and P. Rebeschini. Decentralized cooperative stochastic bandits. In 33rd Conference on Neural Information Processing Systems, pp. 4531-4542, 2019.
Moshe Timothy Masonta, Mjumo Mzyece, and Ntsibane Ntlatlapa. Spectrum decision in cognitive radio networks: A survey. IEEE commumicalions surveys & lulorials, 15(3): 1088 1107, 2012.
Kanishka Misra, Eric M Schwartz, and Jacob Abernethy. Dynamic online pricing with incomplete information using multiarmed bandit experiments. Markeling Science, (5(2):226-255) 2019.
N. Nayyar, D. Kalathil, and R. Jain. On regret-optimal learning recent multi-player multi-armed bandits. IEEE Transactions on Control of Network Systems, 2016.
Ronald Ortner, Daniil Ryabko, Peter Auer, and Rémi Munos. Regret bounds for restless markov bandits. In International conference on algorithmic learning theory, pp. 214 228. Springer, 2012.
Clémence Réda, Sattar Vakili, and Emilie Kaufmann. Near-Optimal Collaborative Learning in Bandits. In NewrIPS 2022 36th Conference on Neural Information Processing System, Advances in Neural Processing Systems, New Orleans, United States, December 2022. URL https://hal hww.aaeccccccaaa000269999999 h
C. Shi and C. Shen. Federated multi-armed bandits. In 35th AAAI Conference Artificial Intelligence, 2021.
C. Shi, C. Shen, and J. Yang. Federated multi-armed bandits with personalization. CoRR, abs/ 211/2102.13101, 2021. URL
13101
William R Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285-25)) 294, 1933.
Joannes Vermorel and Mehryar Mohri. Multi-armed bandit algorithms and empirical evaluation. In European conference machine learning, pp. 437-448. Springer, 2005.
Beibei Wang and KJ Ray Liu. Advances in cognitive radio networks: survey. IEEE Journal of selected
topics in signal processing, 5(1): 23, 2010.
K. Zhang, Z. Yang, and T. Başar. Multi-agent reinforcement lcarning: A selective overview thoorics and algorithms. In Handbook of Reinforcement Leurning and Control, pp. 321 384. Springer, 2021.
J. Zhu and J. Liu. Distributed multi-armed bandits. IEEE Transactions Automatic Control, 68(5): 3025-300 3040-,040, 2023. Special Issue on Learning for Control.
Z. Zhu, J. Zhu, J. Liu, and Y. Liu. Federated bandit: a gossiping approach. Proceedings of the ACM Measurement and Analysis of Computing Systems, 5(1), 2021.
14


Reviews:
[{"review_id": "NWoNTW0fuV", "reviewer": "Reviewer_tro3", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The problem tackled (case of unknown active status of arms) is interesting and promising\n- Theoretical analysis is rigorous giving a strong mathematical grounding to the results presented\n- Paper is clearly written and reasoning and well explained, with interesting intuitions on the problem in section 3 and insights in section 6  \n\nWeaknesses:\nI believe the main limitation of the paper is the simplicity of the setting it chose: having a 1-0 reward seems rather limiting and I believe a lot of valuable insights would come from the case of slightly more complex rewards (e.g. trade-off between high reward and cool-down).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper tackles the case of bandits with an unknown cool-down period after selecting a given arm. More specifically, the rewards follow a Bernoulli distribution, with an additional cool-down. After introducing the problem and giving intuition on the main difficulties around it, it presents novel algorithms for both known and unknown cool-down cases, before deriving regret bounds for each and comparing them to the sleeping bandits case. The paper ends with some numerical experiments and additional insights.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "See weaknesses section. Other smaller items:\n- More context and the use cases of such framework\n- Limitations discussion", "broader_impact_concerns": "None.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6jeMNu3HJh", "reviewer": "Reviewer_ywM1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well motivated, and the scope of the research is clearly defined. Thompson Sampling is a particularly interesting algorithm to study due to the inherent complexity of its theoretical analysis, so any meaningful contribution in this area is valuable to the literature. I appreciated the effort made to list real-world scenarios where the proposed setting can model the environment effectively, such as radio networks, coupon websites, and gaming help ground the theory in practical relevance. The structure of the paper is clear and coherent. I liked how the authors summarized the main technical challenges within the main text, making clearer the contributions and novelty wrt to the existing body of knowldege and the possible impact to the literature also for what regards the theoretical part. The algorithms are well explained, and the notation is clear throughout. The results presented are strong and noteworthy, especially since they match the $O(log(T))$ regret bound. I also have particularly appreciated the discussion and analysis of the unknown cool-down duration setting, which in my view is essential for the real-world applicability of the model, and also conceptually as it explain how the algorithm behave when some parameters of particular importance are not known. However, I have serious concerns about the soundness of some of the results. In particular, I question the validity of Lemma 3 in the preliminaries. The authors claim that this lemma can be inferred from the works of Agrawal and Goyal, but from my understanding, this seems to be a potential misinterpretation. Specifically, it appears the authors conflate the concept of a round $t$, which is purely temporal, with the number of times an arm is selected. They seem to suggest that, on average, the first $L_i(T)$ selections of arm $i$ occur within the first $L_i(T)$ rounds. This seems implausible. Since $L_i(T)$ is not a random variable, the only scenario where this would be consistently true is one in which the algorithm selects in every possible realization the $i$-th arm $L_i(T)$ times in the first $L_i(T)$ rounds, that is impossible. This flawed reasoning  underlies also to the proof of Theorem 2, and therefore impacts the validity of the subsequent corollaries derived from it. Given its foundational role, I believe it is crucial for the authors to clarify precisely how Lemma 3 can be justified, or revise the arguments if a mistake is present. Additionally, I noticed some minor spelling mistakes in the paper’s figures (e.g., “reget” instead of “regret”), which should be corrected. Lastly, while not critical, I would suggest improving the presentation of the proofs section, especially given the paper’s heavy notation. For instance, in the section of Lemma 7, the derivations are quite dense and could benefit from clearer exposition and more step-by-step explanations.\n\n# After the discussion:\nIn the discussion the authors were able to solve the issues with the proofs, providing corrected explanations for their lemmas. I now think there is enough mathematical evidence to support the claims.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors study a dynamical bandit environment that is characterized by the periodical availability of the arms. In particular, they focus on crafting  Thompson Sampling-like algorithms that are capable to handle such an environment, ensuring low regret. They also prove the the performance surpasses that of state-of-the-art algorithms in the sleeping bandit scenario. Finally, they provide numerical simulations to validate the theoretical findings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The concerns about Lemma 3 must be addressed clearly and rigorously, the minor spelling mistakes should be revised as also the presentation of the proofs.", "broader_impact_concerns": "I do not think the paper needs a broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Dg2gOuSZSs", "reviewer": "Reviewer_Yg2p", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Novel setup which is well motivated and useful in various applications.\n2. Novel analysis which improves on existing literature.\n\nWeaknesses:\n1. The algorithm is extremely hard to parse. Even after multiple readings, I found it difficult to follow. I would urge authors to added a detailed description of the intended effects of steps of Algorithm 4/5/6 in the Appendix. For example, authors wrote in Section 3.2 \"The quick jump stage occurs when $L^{(k)}_{test}(t) < L^{(k)}_u(t) - 1\". What does this check do? How does it help the algorithm.\n\n2. Comparison with Sleeping bandits algorithm of Chatterjee et al. How is the Algorithm 1 different from that provided by Chatterjee et al.? The two algorithms both use Thompson sampling, with selecting arms only from the current active set, which is known to both algorithms because of known cool down periods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper works on an interesting problem of bandits with cool down periods where an arms become inactive for a possibly unknown bounded duration after playing and receiving a reward. The problem is useful with various applications as noted by the authors in cognitive radio networks, etc. to online advertising where showing a recommendation may not result till certain time in future as the user already interacted and now will not interact till a certain unknown cool down period. This setup is different from varying reward distribution literature as the later typically works with a fixed number of distribution changes or the distribution changes are continuous. The authors also compare with with sleeping bandits, a similar framework, where a known subset of arms is available for the agent.\n\nNow, the agent, along with learning the underlying reward distribution, also has to learn the cool down period. The same bandit explore-exploit dilemma applies for learning cool down period as well. For this setup, the authors provide a Thompson Sampling based algorithm with carefully constructed cool down exploration to learn the cool down period for the arms. The algorithm conducts urgent cool down exploration when the last cool down explore is not conducted from a sufficient time. Also the algorithm uses bisection search to divide the exploration of cool down period in available domain.\n\nFurther, the authors conduct regret analysis of the algorithm and prove that the algorithm has logarithmic regret growth.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Address the two weaknesses.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)