Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors study a prevalent medical problem where the treatment aims to keep a physiological variable in a safe range and preferably close to a target level. They propose ESCADA, a multi-armed bandit algorithm tailored for the above leveling task, to make safe, personalized, and context-aware dose recommendations.
strengths_and_weaknesses: Strength: 

1. They consider constraints on instantaneous outcomes and propose efficient algorithms to achieve their goal.

2. They provide safety guarantees and upper bounds on cumulative regret.


Weakness:

1. Is \alpha a tuning parameter?  There many too many tuning parameters, which makes the method hard to be used in practice.
How would the choice of Tmin Tmax affect the results of the algorithm? 

2. What is the complexity of the algorithm? How does it depend on the cardinality of dose sets?

questions: Questions:

1. In your problem setup, does T depend on z_n in (1)? 

2. How was the confidence interval estimated? This piece seems to be missing in the manuscript.
limitations: Please see the weakness and questions.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors investigate a problem that they refer to as leveling. Shortly, it is a prevalent medical problem in which the treatment aims to keep a physiological variable in a safe range and preferably close to a target level.

Their proposed algorithm is a multi-armed bandit-based for the leveling task, which aims to make safe, personalized, and context-aware dose recommendations. As a theoretical contribution, they derive probability upper bounds on its cumulative regret and safety guarantees.

Additionally, they conducted in silico experiments on the bolus-insulin dose allocation problem in type-1 diabetes mellitus disease by comparing their algorithm against the GP-UCB baseline.

strengths_and_weaknesses: The dose-finding problem is challenging, given the ethical concerns from online experimentation with real patients. This paper uses a simulator to bring experimental results. However, it compares its performance against a clinician for virtual adult patients.

It would be worth including the qualifications of the clinician that evaluate the unseen meal events. Possibly, in the appendix. 

As a minor improvement, the authors may increase the font size of figures, such as Figures 3-6.

questions: It is unclear if the GP-UCB was also initialized using the same scheme for TACO and TS, i.e., the dose calculators proposed in [55]. Also, it would be interesting to see the effect of using random initialization of all algorithms to understand the impact on the dose calculators through epochs.
limitations: The authors need to include the limitations of their method in the concluding remarks.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: Consider the safe dose allocation problem in precision medicine. This paper proposes a contextual multi-armed bandit algorithm with the objective to keep the outcomes close to a target level. The proposed algorithm has high probability upper bounds on cummulative regrets and also possess a two-sided safety guarantee.
strengths_and_weaknesses: # Strengths 
- The paper is well organized
- The objective is an interesting problem. Instead of maximizing the outcome, this paper aim to keep the expected outcome close to a target level.
- The 'TACO' algorithm is novel. Both exploration and exploitation are addressed. The action when all safe doses are sub-optimal is interesting.

#Weaknesses
- The figures in section 5 are too small to read.
questions: - Typo found in the pseudo code of TACO. In the line after 'else', I believe it should be $w_n(\mathbf{z}_n,d')$ instead of $d$.
- $\epsilon$ in definition 1 is unclear.
limitations: None.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper introduces and evaluates a multi-armed bandit (MAB) algorithm for insulin dose allocation related to Type-1 Diabetes (T1D). Its technical contributions in enhancing safety in T1D by preventing hyperglycemia and hypoglycemia by context-aware and personalised dosing of insulin before meals is fairly convincing, based on evaluations on an appropriately chosen simulator.
strengths_and_weaknesses: The paper introduces and evaluates a multi-armed bandit (MAB) algorithm for insulin dose allocation related to Type-1 Diabetes (T1D). Its technical contributions in enhancing safety in T1D by preventing hyperglycemia and hypoglycemia by context-aware and personalised dosing of insulin before meals are fairly convincing, based on evaluations on an appropriately chosen simulator. However, the evaluation should have differentiated the 30 simulated patients instead of presenting averaged results over all these patients, regardless of their age. For instance, glucose control of adolescents is substantially harder than for adults, and hence, I am used to separating adults, adolescents, and children in evaluating algorithms and medical interventions. Similarly, in order to contribute to precision medicine, as argued by the paper, for example, the target PPBG level should have been adjusted by this age group. Finally, I would have wanted to know more about the clinician that was one of the comparisons in the evaluation; I really liked this more human-centric aspect of the evaluation but would like to know more about this experiment so that future studies could follow the same experimental design if they wanted. Also it was unclear how the simulate patient cases to be analysed by this clinician were chosen and how the human judgement and decision-making was implemented in this part experiments. Last but not least, I could not find details about this experiment (e.g., the cases analysed, implementation of the assessment task, etc.) involving a human participant (i.e., the clinician) as a participant, or obtaining the related ethics approval and informed consent, from the paper or its supplement.
questions: Please separate the 10 adults, 10 adolescents, and 10 children in evaluations and consider if the target PPBG level should be adjusted by this age group as well.

Please explain more about the evaluation conducted by the clinician (e.g., the cases analysed, implementation of the assessment task, ethics approval, and informed consent)
limitations: Clinical aspects of the study could be clearer. For example, the guidelines by https://www.jmir.org/2016/12/e323/ may help. In particular, I encourage clarifying the rationale, clinical implications, and limitations of the model; see the JMIR paper for further details about these topics.
review_title: null

Meta-review: metareview: I have read all comments and responses carefully.

The reviewers recognized that the problem was a challenging one, and that the paper provides both practical and novel tools and theoretical analysis. However, the reviewers pointed to the lack of numerical studies in the paper (for example, more details about the human clinicians and the patients). That being said, the authors have addressed most constructive comments given by reviewers. 

Overall, reviewers agree that this is an important and yet underexplored problem and the authors have provided useful contributions. I, therefore, have decided to recommend the acceptance of the paper.
confidence: Certain
Decision: Accept