Venue: CLeaR 2022 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary: Develops an algorithm for discrete causal bandits that does not assume prior knowledge about the causal structure.
Main review: This paper looks good enough to be accepted. That said, the clarity should be improved. 

Please reword and be more specific when presenting definitions: “If we do not only observe the rewards but also some additional variables, and the rewards and these variables are generated according to a causal model, then the rewards are no longer independent, and this is called a Causal Bandit problem.”

Be more specific about which previous works you are referring to. “In this case, the **previously cited works** on this topic have shown that one can this additional structure to improve our performance.”

How is the context variable different from the regime indicator variable F_t used by Philip Dawid? https://arxiv.org/pdf/2004.12493 (See section 2). Note that while this reference is relatively new the use of such variables can be found in older papers by Dawid and in Pearl’s Causality book.

In the introduction, the authors state, “we develop a framework for Causal Bandits where everything is learned from scratch.” This is an exaggeration.  Later on in section 3.1 they state, “we introduce a novel information sharing estimator that relies on much less specific knowledge about the causal graph”. Please state explicitly the information being assumed about the causal graph.

Typos and spelling mistakes: “then much less restrictive **then** specifically **focussing** on” 

The algorithm is presented in the appendix. It should probably be moved to the main paper.
review_title: null

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: An O.K. paper
Main review: In this paper, the authors proposed a new bandit algorithm that can more efficiently estimate the causal effect via learning the causal graphs. The simulation results on the Sachs data show that there is improved cumulative regret compared to classical bandit approaches. I took a brief look at the proof and did not find any problem. Overall I think it is an O.K. paper and may be of interest to the CLeaR audience. Nevertheless, after carefully reviewed this manuscript, I still have the following questions:

1. Theorem 2 only shows that there is variance reduction by given a valid separating set. However, in practice all the separating sets are learned from the algorithm and the causal discovery algorithm may not be guaranteed to find the valid set for separation. Is there any theoretical analysis to leverage the estimation errors from causal discovery algorithms into the regret or variance bound?

2. In Section 5.2, the authors showed improved cumulative regret bound on Sachs data set. This means that when the causal graph is easy to learn, the proposed method can achieve a relatively nicer performance compared to orthodox bandit algorithms. I wonder how the performance would be when we instead apply it on a data set with more complicated causal graph. For example, when it is applied to a data with a dense causal graph or with a causal graph containing latent confounders, would it still perform well than the ordinary bandit methods? Or it would be the other way around?

Overall I think it is an O.K. paper, but considering the problems on theoretical analysis and real world applications mentioned above, the contribution seems rather marginal.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: Summary: This paper considers a causal bandit problem where a separating set between intervention and reward can be utilized through information sharing among different arms. With reduction in variance, the proposed causal bandit algorithm outperforms those without taking advantage of causal information obtainable through data and conditional independence tests.
Main review: I have reviewed several causal bandit papers and found that some of the papers contribution is simply using the parents of reward variable Y to improve the performance. In my review, I wrote that there may be other separating set which can yield better variance reduction. Hence, I was somehow expected this idea being realized.

The authors solution is simple and clean. The use of context variable (~ regime indicator) is also a nice addition.


Some questions:
Why does the algorithms in the appendix does not check a separating set on the fly?

Do we have to perform causal discovery every iteration or once in a while (depending on p-value one may skip a few iteration)? Do we have an online version of conditional independence test or causal discovery algorithm that can be utilized in this setting?

The result with direct independence testing is better than those with causal discovery algorithm. Would it be desirable to employ some kind of “optimism in the face of uncertainty” for a “separating set”?
review_title: null

Meta-review: This paper proposes a causal bandit algorithm that does not assume prior knowledge about the causal structure and efficiently estimates the causal effect using a separating set between intervention and reward among different arms. The simulation results show that the proposed algorithm outperforms causal bandit approaches that do not leverage causal information obtainable through data and conditional independence tests.

The reviewers agree that the paper proposes an attractive solution to the causal bandit problem. However, the clarity and exposition of the paper can be improved. Therefore, we recommend the authors take the reviewers' comments to heart and encourage them to incorporate their thoughts in preparing the final version of their manuscript.
Decision: Accept (Poster)