Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper addresses the Dynamic Causal Global Optimization (DCGO) problem that aims to find both the optimal set of manipulative variables and the optimal intervention levels to the selected variables. This is a time-series optimization that determines the best way to intervene in the present based on the results of interventions made in the past, and is considered to be a formulation of a particularly important problem e.g. in the medical field.
The authors make three assumptions about the DCGO problem:
(1) the causal structure is invariant with respect to time, 
(2) the function value of a variable Y_t is determined from the function value of its parent node, and the function value of the parent node can be expressed as the sum of the function value of the target node and that of the non-target node, 
(3) the DAG does not contain any unobserved confounding factors. 

Under these assumptions, it follows that the objective function of DCGO at time t can be expressed recursively using the results of interventions up to time t-1, and that the search space of DCGO is invariant with respect to time.
Under the above problem setup, the proposed method DCBO is an EI-based Bayesian optimization algorithm based on the Dynamic Causal Gaussian Process model whose mean and covariance function are defined by the recursive representation of the objective function.  

Experiments to validate the effectiveness of the proposed method have been conducted on both synthetic and real data. Experiments with synthetic data have been conducted in six situations: when DAGs and SCMs are stationary/non-stationary with respect to time, when the manipulative variables contain noise, when there are missing observations, when there is no causal relationship among manipulative variables, 
and when there are multiple intervening variables.
Two real data experiments were conducted: one on the problem of minimizing the unemployment rate in a closed economy (economics), and the other on the problem of finding an intervention that would reduce the phenomenon of a concentration of dead animals in a chemostat (biology).
main_review: Originality: This paper aims at the optimization of sequential interventions, i.e., identifying the variables to be intervened and determining the optimal way to intervene. 
This involves important problems, such as stage-by-stage optimization of multi-stage treatments in medicine. I think it is an important work to formulate this as an optimization problem under uncertainty, and to propose a framework for automatically searching for optimal interventions using a Bayesian optimization-type algorithm.

Quality: This paper technically sounds and the performance of the proposed method has been verified in various situations including real data. In particular, settings where there is no causal relationship between the manipurative variables that would disadvantage the proposed method are also considered (in fact, the existing methods outperforms the proposed method in terms of accuracy), and thus it seems that fair comparisons are conducted. Theoretically, two properties Theorem 1, which states that the objective function can be written recursively, and Proposition 3.1, which states that the search space is time-invariant, may be important in Bayesian optimization, a problem that involves optimizing a function that is expensive to begin with.

Clarity: This paper is well organized. It clearly explains problem formulation, why a general framework for dynamic causal global optimization is needed, the process of deriving the proposed method, and its theoretical considerations.

Significance: As mentioned above, the problem considered in this study may have important applications. For example, in terms of medical applications, a reinforcement learning-based method called "dynamic treatment regime" is being considered as another approach to sequential optimization of interventions. However, this method has several problems that need to be solved before it can be put to practical use, such as being complicated of the method itself and the difficulty of estimation. There are still very few examples of its application to real problems. 
It is expected that this and subsequent studies will provide a practical approach to these problems.

Questions 

- How realistic are the three assumptions (especially the first one that the causal structure is invariant with respect to time) introduced in section 2? Also, is it correct to interpret that the non-stationary case considered in the experiment deals with a situation that violates this assumption? In that case, I think the theoretical properties such as recursive representation of the objective function and constant search space are not guaranteed. How can we treat this?

- How do you determine the initial state \mathbb{M}_0　of the search space?

- In the equation that define the GP prior, the notation "m_{s, t}=... \wedge k_{s, t}=..." is confusing and better to be changed. 
e.g. m_{s, t}=..., k_{s, t}=...
limitations_and_societal_impact: The authors also consider experiments in problem settings where the proposed method does not work well.
I believe that there is no negative social impact from this research.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a model, Dynamic Causal Bayesian Optimization (DCBO), which builds upon existing models to find a sequence of interventions , optimizing the target variable at each time step in a causal system. It combines Causal Bayesian Optimization, which builds around casual information and finds the optimal intervention in a DAG without temporal evolution, with Dynamic Bayesian Network, which is used in time-series modelling and carries dependence assumptions that don’t imply causation. DCBO represents associations between features and the causality between input and outputs, both of which can change over time. The modeling therefore offers a novel approach for decision making in dynamic systems ie. find optimal intervention at time t. The paper compares the model’s results against algorithms which either account for temporal evolution or optimize the sequence of interventions of a system (as no model exists which does both). The algorithm is run on both interventional and observational data. Finally, synthetic experiments are performed, such as creating noisy variables, incorporating missing observational data, and creating non-stationary settings. 
main_review: The paper describes the modeling with mathematical rigor and precision. However, given the assumptions with which the framework is built, it is doubtful whether the approach could be used in any real-world scenario. Specifically, the modeling requires that the optimizing agent is allowed to perform a number of explorative interventions which do not change the underlying state of the system. It is difficult to identify a use case that satisfies this requirement. In section 4.2 (Line 342-351), the author provides “real experiments” that empoy real data to train a simulator/a system of ODE, which is then optimized. This is not an example of a “real experiment”.
The submission also does not elucidate how the proposed model identifies optimal interventions or treatment over a given time horizon.
The theoretical results obtained for both the synthetics and the real dataset do not show a substantial improvement from the already existing casual optimization models, with only slight improvements.
On a minor point, the mathematical notations are a bit cumbersome, and it is easy to get lost in a great variety of subscripts and quotes. Including a table summarizing the notation in one place would be particularly helpful.
limitations_and_societal_impact: One of the paper's main points is to show the advantage of introducing a dynamic extension to existing causal Bayesian optimization methods. However, in line 145, invariance of the causal structure is assumed. This is a really restrictive hypothesis that severely hinders the possible application of this framework.  
To be fair, the submission states that some of the issues we point out will be addressed in future work. However, this framework could not be applied to real-world problems without such work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper works with the Dynamic Causal Global Optimization problem. For a temporal structural causal model, which is a sequence of structural causal models with causality in time, meaning there are dependencies on the values of variables from previous step. The goal is to optimize the outcome variable by manipulating some observable variables (those are chosen by the model).

An algorithm called dynamic causal bayesian optimization is proposed to solve such a problem. The objective function $f_{s,t}$ is decomposed into the sum of a function depending on the optimized observed variables and another function depending on others (variables in the current time). Thus the optimization can be done step by step in time.

A dynamic causal GP model is introduced for practically implement the DCBO algorithm.

For experiments, both Synthetic and Real data are used, in several different structural causal models, to compare the behavior of DCBO, to that of CBO, ABO and BO.
main_review: (I) The problem and methods are noval as far as I know. 

(II) The results are well supported. The methods used are appropriate. It is a complete work.

(III) The paper is well organized, the structure is clear, important conclusions and concepts are provided in math environments (theorem / lemma / corollary / definition). All conclusions have formal proofs. However, there are some typos and font mismatches in the text and formulas.

(IV) This work solves the dynamic problem and builds a relation to previous CBO problem. It is difficult for me to determine how important this work is, because of my lack of experience in this area.

----------------------
Questions:
- The conclusion depends on the assumptions. So:
   - How likely the additivity of $f_{Y_t}$ is satisfied? What is the difficulty in solving the problem without additivity (I guess, the theorem 1 will no longer be of the same format, the sum of $f^Y$ and $f^{NY}$).
   - What will happen if the structure of causal model changes in time? (It is assumed $G(t)=G(0)$ in the assumption).
- The practical algorithm depends on Gaussian Process, how representative is GP in this problem?
- Is it possible to use other surrogate models, like deep GP?
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors extend the notion of casual Bayesian optimisation from a static to a dynamic scenario which, indeed, has many relevant applications. The paper theoretically analyses the proposed approach and run the algorithm in a set of real-world and synthetic data. This work is really interesting but I have some questions related to scalability and applicability that I hope the authors could help me understand. 
main_review: 
1) I found definition 1 a bit confusing in some places: 
1.1.) When the set of observed variables V are defined, the authors write that those variables in the model are determined by variables in U \cup V. So, the variables in V are defined by variables in the union of U and V. I am a bit confused about this as this somehow defies the set V through the set V.
1.2.) When the set of functions f_i is introduced, the authors wrote that each of those is mapping from the union U_i and Pa(U_i) to V_i, where Pa(U_i) is a subset of V. Hence, from this definition, it follows that mapping f_i for each element in the union U_i and Pa(U_i) should put in correspondence a value of variable V_i. However,  f_i(Pa(u_i), u_i) is then defined as a function for an ordered pair of elements Pa(u_i) and u_i. Wouldn't this mean that the function f_i is defined on the cartesian product Pa(U_i) x U_i rather than the union P(U_i) \cup U_i?
1.3.) When the authors define the entire set F of functions f_1,.... f_n it is said that F forms a mapping from U to V. But each f_i is mapping not just from U but also it takes as an input element from Pa(U)\in V. Hence, should not F be a mapping from U x V to V? 

2) Similar to standard assumptions in causal BO, the authors assume a known graph G. It is already rather hard to satisfy this assumption is standard causal BO without explicit prior knowledge. In this paper, does not this assumption entail even more prior knowledge? Shouldn't we also know how the graphs are varying over time as well?

3) The algorithm proposed is interesting. I am wondering if the authors might reframe this as an instance of reinforcement learning and solve the corresponding problem? In other words, can there exist any relations to reinforcement learning and/or dynamic programming? If so, how can I think of the policy? Would the current algorithm correspond to a greedy policy for example? 

4) In formulation (1) for DCGO, if we are allowed to execute interventions on all variables X_t, will it not be the best option for us to optimise Y_t? The reason I ask is that although the cost of intervention is introduced it is not a involved in the objective function (but involved in the acquisition function)?

4) Assumption 2, how restrictive is the additive assumption? In the experiments do we consider problems with such function f_{Y_t} and if yes, do we have a mechanism to check this property somehow?

5) How the authors restrict the search space was not clear to me. My confusion stems from assumption 1, wherein the dynamic case the search space of subsets is the same as in the static case. Still, it is not clear, what is the size of this collection in this static case? Is it the cardinality of the power set, i.e. 2^{|X_t|}?

6) Maybe removing the and operation from line 217 can aid readability. 

7) Can the authors elaborate/study the effect of the topology of the underlying DAG graph on the overall performance of the algorithm? 


limitations_and_societal_impact: I think the paper can benefit from a larger study of social impacts especially after motivating health-care in the abstract.
review_title: null

Meta-review: This paper addresses an extension of causal Bayesian optimization from static to dynamic scenario to find a sequence of optimal interventions in a dynamic causal system. The problem formulation and the method are novel. In addition,  the results are well supported by experiments.  In general, the paper is well written. All of reviewers agree that the paper contains interesting contributions that are deserved to be presented in the conference. 


Decision: Accept (Poster)