Paper: Twinned Interventional Flows (ICLR-2024)

Reviewer comment (Writing):
It’s poorly written and lacks coherence. The introduction doesn’t motivate the problem sufficiently - what is the precise setting? This should be stated in the first paragraph and not scattered around two pages of introduction (leaving a feeling of lack of focus). For example paragraph three is motivating a setting but doesn’t explicitly express it and then paragraph four starts talking about something different (stiff ODE) without ever defining it or commenting on the relevance of stiff ODEs with this setting. After reading the paper several times, still cannot understand why this specific setting suffers from stiff ODEs.

Author response:
1. **Coherence and motivation behind the combination of ideas.** 

    We appreciate your feedback and understand the importance of clarity in our writing. Our objective is to address a combination of real-world problems, each of which is carefully chosen based on its relevance to applications such as medical decision-making. The selection of ideas is motivated by the need to integrate knowledge from different fields to tackle complex systems effectively.

    The issue of stiffness, in particular, arose during the development of our method. Despite searching existing literature, we found no satisfactory solutions. Consequently, we devised our own approach, which demonstrated significant improvements in our experiments. Please see the visualization in Section 4.1, if it's sufficient to explain the problem.

    We acknowledge the importance of conveying these motivations more explicitly, and we will make adjustments to the text to provide a clearer explanation of why these specific problems are included in the paper. If you have additional suggestions or specific areas you would like us to address, please let us know.

Reviewer comment (Experiments):
The paper combines too many ideas and it doesn’t spend much time to explain and study each of the ideas properly. For example, the penalty for mitigating stiffness is not ablated sufficiently (stiff ODEs are not even defined in the paper which makes it very hard to read).

Author response:
1. **Coherence and motivation behind the combination of ideas.** 

    We appreciate your feedback and understand the importance of clarity in our writing. Our objective is to address a combination of real-world problems, each of which is carefully chosen based on its relevance to applications such as medical decision-making. The selection of ideas is motivated by the need to integrate knowledge from different fields to tackle complex systems effectively.

    The issue of stiffness, in particular, arose during the development of our method. Despite searching existing literature, we found no satisfactory solutions. Consequently, we devised our own approach, which demonstrated significant improvements in our experiments. Please see the visualization in Section 4.1, if it's sufficient to explain the problem.

    We acknowledge the importance of conveying these motivations more explicitly, and we will make adjustments to the text to provide a clearer explanation of why these specific problems are included in the paper. If you have additional suggestions or specific areas you would like us to address, please let us know.

2. **Regarding stiffness in ODEs.** 

    There is no clear measure of stiffness in the literature and our definition, "Stiff ODEs are numerically difficult to invert," is derived from Ernst Hairer’s observation that "stiff equations are problems for which explicit methods do not work" [1], combined with insights from [2] that errors can significantly magnify when ODEs are inverted.

    To address your concern, we will refine our wording and explicitly reference [1] to provide a clearer foundation for our definition. If you have any further suggestions or specific areas you would like us to enhance, please let us know.

    Regarding the visualization of stiffness, it is presented in Figure 4f compared to the performance when the penalty term is employed (4e).

    How would you like us to further visualize these problems?

Reviewer comment (Writing):
Related work and background section is out of focus and missing key parts as well. For example, why is causal discovery discussed? How is causal inference relevant? Is it a causality paper, if so where is the background section introducing key concepts?

Author response:
3. **Focus of related work and background section.** 

    In our paper, we approach causality from a specific angle, choosing not to delve deeply into causal discovery. We mention it mainly due to its relevance to the intersection of reinforcement learning (RL) and causal fields. Instead, our primary focus lies in recognizing the causal properties of the system, particularly addressing challenges like unobserved confounders. We achieve this by incorporating these considerations into the model design, ensuring flexibility to tackle such complexities.

    To accomplish our goals, we amalgamate tools from both the causal inference and RL domains. As a result, our background section delves into these topics, expecting a foundational understanding from the readers. We acknowledge the absence of exhaustive technical background knowledge in the paper, given its assumption of familiarity with these fields.

    Nevertheless, we are open to adjusting our focus if you deem any particular aspect irrelevant or have specific topics in mind that should receive more attention. Your insights are valuable in shaping the paper to meet the expectations of our audience.

Reviewer comment (Presentation):
The paper is appendix heavy. A lot of the content should have been surfaced in the main text. For example, assumptions C.{1,2,3} are essential and should be surfaced in the main. Significance of theoretical results should be discussed in main as well.

Author response:
4. **The balance between appendixes and main body.** 

   We share this concern with you, however the strict page limit forces us to make some compromises.

    If there are specific sections in the appendix that you believe could be integrated into the main body or should be discarded, please let us know. We are open to reorganizing the content to enhance the overall flow while acknowledging to the page limit.

Reviewer comment (Writing):
What are stiff ODEs and how are they related to this setting? Is it a common problem in the setting you are studying? Can you show visual examples in the main text? Motivate and demonstrate the issue and then solve it, that way the reader can understand why things are happening.

Author response:
2. **Regarding stiffness in ODEs.** 

    There is no clear measure of stiffness in the literature and our definition, "Stiff ODEs are numerically difficult to invert," is derived from Ernst Hairer’s observation that "stiff equations are problems for which explicit methods do not work" [1], combined with insights from [2] that errors can significantly magnify when ODEs are inverted.

    To address your concern, we will refine our wording and explicitly reference [1] to provide a clearer foundation for our definition. If you have any further suggestions or specific areas you would like us to enhance, please let us know.

    Regarding the visualization of stiffness, it is presented in Figure 4f compared to the performance when the penalty term is employed (4e).

    How would you like us to further visualize these problems?

Reviewer comment (Writing):
What is the setting precisely? Please write a sentence explaining it as simple as possible. This shouldn't be guessed by the reader.

Author response:
5. **The setting summarized in a sentence.**
    Our primary objective is to develop a predictive model that accurately captures the system
dynamics under various interventions and possible (unobserved) confounders. This model should be versatile enough to estimate the likelihood of events and make use of both observational and interventional data. We believe this framework is applicable to various real-world scenarios, particularly in fields like medicine.

Reviewer comment (Writing):
In paragraph 3, "which can be counterproductive" - it's not counterproductive only, it casts the problem non-identifiable. Can you please update the text?

Author response:
6. **Wording and additions.**
Thank you for your detailed suggestions on the wording, (e.g. wording in paragraph 3, and adding the term causal sufficiency). We appreciate your feedback and will incorporate the necessary changes in the upcoming version, which we aim to upload shortly. Your understanding of the delay is much appreciated. We look forward to continuing the discussion.

Reviewer comment (Writing):
What are stiff regions? This is a technical term and needs definition before used.

Author response:
7. **Definition of stiff regions.**

    We will add this definition to the text: By stiff regions we mean the regions of weight space where the corresponding NODE becomes stiff.

Reviewer comment (Presentation):
In related work, causal discovery discussion is unrelated, can you please discuss the relevance to your work or remove?

Author response:
3. **Focus of related work and background section.** 

    In our paper, we approach causality from a specific angle, choosing not to delve deeply into causal discovery. We mention it mainly due to its relevance to the intersection of reinforcement learning (RL) and causal fields. Instead, our primary focus lies in recognizing the causal properties of the system, particularly addressing challenges like unobserved confounders. We achieve this by incorporating these considerations into the model design, ensuring flexibility to tackle such complexities.

    To accomplish our goals, we amalgamate tools from both the causal inference and RL domains. As a result, our background section delves into these topics, expecting a foundational understanding from the readers. We acknowledge the absence of exhaustive technical background knowledge in the paper, given its assumption of familiarity with these fields.

    Nevertheless, we are open to adjusting our focus if you deem any particular aspect irrelevant or have specific topics in mind that should receive more attention. Your insights are valuable in shaping the paper to meet the expectations of our audience.

Reviewer comment (Novelty):
Are you focusing mostly on learning from offline data? If so, would discussion of offline RL be of relevance here?

Author response:
9. *Also, are you focusing mostly on learning from offline data? ..?*

    We appreciate your question. Our primary focus is on integrating additional offline data into the context of online learning. This essentially combines aspects of both offline and online learning from the RL field. Given the constraints of page limitations, we have chosen not to delve extensively into these topics, as our primary contribution lies in the theoretical results, specifically tailored for TIF and normalizing flows. If you have further suggestions on how to address these aspects more effectively within our limitations, we would be open to considering them.

Reviewer comment (Theory):
In POMDP formulation you say that rewards are not required, however if you remove rewards from the POMDP then it's just a graphical model (Decision Process requires rewards). What does the POMDP formulation buy you?

Author response:
10. *In POMDP formulation you say that rewards are not required...?*
    
    Thank you for your observation. We will modify the statement to clarify that we do not explicitly discuss rewards in the POMDP formulation. This adjustment aligns more accurately with our intent. In our RL setup, TIF can serve as a predictor of the state, and our focus is on demonstrating its performance without explicit consideration of rewards. The role of rewards, would be more relevant to policy optimization, which is not the topic of this paper.

Reviewer comment (Writing):
What is the "twinned space"?

Author response:
11. *What is the "twinned space”?*

    The term "twinned space" refers to the combined space of observed and latent variables. We acknowledge that this concept wasn't as clearly articulated in the text, even though it is a central element of our methodology. To address this, we will incorporate a formal definition of the "twinned space" in the revised version for better clarity.

Reviewer comment (Writing):
In 3.2 you talk about the stability of Monte Carlo estimate. Where are you using MC?

Author response:
12. *In 3.2 you talk about the stability of Monte Carlo estimate. Where are you using MC?*

    Thank you for bringing this to our attention. The Monte Carlo estimation is employed in the log-likelihood estimation algorithm detailed in Appendix D. In the revised version, we will explicitly reference Appendix D when discussing the use of Monte Carlo estimation and its stability.

Reviewer comment (Novelty):
You cite Manski 1989, can you please comment on the relevance of this paper to your work? It doesn't seem to comment on the use of "observational data helping learning the dynamics of discrete interventional settings".

Author response:
13. *You cite Manski 1989, can you please comment on the relevance of this paper to you work? It doesn't seem to comment on the use of "observational data helping learning the dynamics of discrete interventional settings”*

    The citation of Manski (1989) is included in our work due to his significant contribution to the proof of bounds with binary variables. Following the example set by [3], we aimed to acknowledge the foundational work on this problem. While Manski's paper may not directly address the use of observational data in learning the dynamics of discrete interventional settings, its inclusion is more contextualized in the broader history and foundations of the problem we are addressing.

Reviewer comment (Experiments):
Figure 6 - The plot doesn't match the performance of the baselines in the paper. Did you use the same code? Also, how many seeds did you run? Are you plotting the standard error? I don't see any variance on your method - is this because it's stable between seeds?

Author response:
14. *Figure 6 - The plot doesn't match the performance of the baselines in the paper. Did you use the same code? Also, how many seeds did you run? Are you plotting the standard error? I don't see any variance on your method - is this because it's stable between seeds?*

     Yes, we utilized the same code for the experiments; however, regrettably, we encountered challenges in reproducing the results of the baselines. ODE solver errors with their method posed significant obstacles. The plot represents outcomes from five distinct random seeds, and the standard error is illustrated over these seeds. While the variance of our method is depicted with red opacity fill, covering a broad range of results, it might not be clearly visible due to its extensive coverage.

Reviewer comment (Evaluation):
What is the take-away message from the half cheetah example? How is this experiment related to your setting?

Author response:
15. *What is the take-away message from the half cheetah example? How is this experiment related to your setting?*

    The reinforcement learning (RL) benchmark underscores the applicability of TIF in this domain. While it may not surpass other baselines in performance, it introduces distinct advantages, such as log likelihood estimation and anomaly detection. The crucial takeaway is that TIF integrates effectively into the RL framework, demonstrating its capability to handle real-world online data.

Reviewer comment (Theory):
The paper positions itself as both a reinforcement learning and a (causal) time-varying treatment effect estimation method but makes little effort to explain the connections between both. It inherits the assumptions typical for reinforcement learning, i.e., POMDP, but uses it for causal benchmarks, like interventional or counterfactual predictions. For example, benchmarks for interventional predictions (like the tumor generator) are based on a different set of assumptions, e.g., three assumptions from Seedat et al. (2022), and thus are not suitable for TIF.

Author response:
1. **Causal reinforcement learning: connection between RL and treatment effect estimation.**
Thank you for your comment. The connection between causal inference and RL fields is well studied by Bareinboim [1]. In our setting, the reinforcement learning set up (POMDP) is closely tied with causal treatment effect prediction; and our work can be seen as addressing two of the six main tasks in causal reinforcement learning, namely, (a) Generalized policy learning and (b) Counterfactual decision-making. Additionally, deciding when and where to intervene could be addressed with a good predictor that is continuous in nature.  

    Specifically, each action $a_t$ serves the role of a treatment, and time-varying confounders can be included in $l_t$. Effectively predicting treatment effects implies that the problem can be cast as an RL task, where the objective is to optimize the treatment based on the predictions. This perspective highlights the interplay between causal treatment effect prediction and RL, where a good predictor with continuous properties can inform decision-making in RL. Our POMDP formulation serves this purpose. 

    Hopefully this clarified the connections between these topics and we will make the according changes to the paper to highlight these topics.

Reviewer comment (Theory):
The paper claims to handle unobserved confounding, but in such cases, without further assumptions, the causal effects are non-identifiable.

Author response:
2. **Identifiability and our  assumptions.**
We understand your concerns about identifiability. Indeed, identifying the number of confounders or recovering a full causal graph in general is tedious and often infeasible. We also acknowledge that there are scenarios where data limitations may hinder learning of the true causal structure due to identifiability issues.

    Our primary goal here is to develop a predictive model that accurately captures the system dynamics under various interventions and possible (unobserved) confounders in our setting that treats the size of the latent space as a tunable parameter, and abstracts many practical situations with confounders. We therefore appeal to an approach that does not constrain the causal hierarchy within observed and unobserved variables and can be described in terms of a continuous POMDP.  

    Importantly, if desired, identifiability can be achieved with appropriate additional assumptions in our setting. Indeed, one way is identification via (weighted) time-independent sampling by adapting Tennenholtz et al (2020) [2]. This, however, is susceptible to the curse of dimensionality. Instead, one can invoke a (more amenable) reduction to proximal causal inference as illuminated by Bennett and Kallus [3].

    The set of assumptions provided in the paper are also consequence of this approach. The Overlap assumption by Seedat et al. [4] is listed in our set of assumption in Appendix C, as it necessary for deriving our theoretical results. However the two other assumptions by [4] are more related to the identifiability of the problem, which is not our focus here, and so were not listed. However we are happy to add these assumptions with a discussion on identifiability based on your feedback.

Reviewer comment (Experiments):
Some key statements in the paper were not supported by the experimental evidence, such as the claim that TIF are better than methods with low-dimensional embeddings, or that the method provides accurate density estimates (no comparison with other density estimators was provided).

Author response:
3. **Regarding the experimental evidence.**
 Our primary focus in the experimental section was to showcase the various  capabilities of the proposed methood TIF, but not to imply or assert its  superiority over low-dimensional embeddings like TE-CDE [4]. We emphasize the flexibility afforded by TIF due to its many strengths inherited from continuous normalizing flows, particularly in density estimation. In this context, it is crucial to note that while density estimation is primarily a theoretical framework, the stability of the Monte Carlo estimation algorithm, as outlined in Appendix D, requires further validation. Consequently, we refrained from making comparisons with, for example, Bayesian methods. 

    We are grateful to you for bringing the two new papers [5], [6] to our attention, which are important and highly relevant to our topic. Unfortunately these works were not published when we worked with the background study of TIF. We are happy the see that the field is evolving and we will cite and position these papers appropriately in the final version of the paper. However, due to time constraints, benchmarking against them is unfortunately not feasible during the response period.

     We believe that this work already makes significant contributions, and lays the foundation to investigate aspects such as those pointed by you as future work. Based on your feedback, we will also be sure to underscore the motivation underlying each of our experiments.

Reviewer comment (Reproducibility):
There are no implementation details provided for the benchmarks, the authors did not provide details on hyper-parameter tuning or the source code, making it unclear how to choose parameters such as latent dimensionality ($n - m$) or number of MC samples, and preventing verification of whether comparisons (e.g., between TIF and TE-CDEs in the interventional prediction benchmark) were fair.

Author response:
4. **Hyperparameter tuning.**
Thank you for pointing out the missing details on hyperparameter tuning (owing to space constraints). We appreciate your understanding, and we will certainly include this information in the upcoming revision shortly. In the meantime, we provide below a brief overview of our hyperparameter tuning process.

    We conducted all our experiments with extreme care and diligence to be fair to the baselines. We would be happy to share our code upon request and release it upon the acceptance of this work.

    For the pendulum data, hyperparameters such as the latent dimension were fine-tuned, and the same configuration was applied to the tumor growth experiment. We note that more extensive hyperparameter tuning could potentially lead to further improvements for TIF. The learning rate was determined through multiple runs with a small number of epochs.

    In the case of the RL benchmark, the latent dimensionality and model size were aligned with those of other benchmarks, with the latent dimension being an educated guess based on the dimensions of the Half Cheetah task. The primary parameter tuned in this scenario was the learning rate, with approximately five different options tested for each setup.

    We hope this brief overview provides clarity, and we will ensure to include a more detailed account in the revised version of the paper. If you have any additional suggestions or queries, we are happy to discuss further.

Reviewer comment (Presentation):
Table 1 mixes completely different methods (“compares apples with oranges”), e.g., causal inference methods with reinforcement learning methods; time-varying methods and cross-sectional methods. It is also unclear what property “making prediction in the presence of unobserved confounders” means, as this would require assuming a sensitivity model and performing partial identification of causal effects. There also seem to be wrong entries, such as the cross for De Brouwer et al. (2022) at “logp” (this paper assumes a likelihood model, so log-probability can be inferred), or Seedat et al. (2022) at “conf” (TE-CDEs are not suitable for hidden confounding).

Author response:
5. **Regarding Table 1.**
We position ourselves at the intersection of different fields, and specifically in the realm of causal reinforcement learning. Consequently, the topics covered in Table 1 are deliberated across a diverse range of models, as you rightly pointed out. Despite these discussions traditionally occurring within distinct fields, we view the problems in Table 1 as pertinent to real-world issues, arising e.g., in medical settings, that ought to be addressed in a more holistic way than the conventional paradigm. Therefore, in our perspective, there is merit in addressing them collectively, which the previous literature has not done as illustrated in Table 1.

    With the property “making prediction in the presence of unobserved confounders” we refer to whether or not the possible confounders are taken into account in the model design and discussed in the paper. 

    Regarding what you suspect as potentially incorrect entries, please allow us to elucidate our reasoning.   De Brower et al. use decoder encoder decomposition and are therefore limited to obtaining variational bounds of the log-likelihood, while TIF extends continuous normalizing flows and therefore can obtain the exact log-likelihood estimate.    

    Seedat et al. do make predictions in the presence of confounders as they say "TE-CDE uses domain adversarial training to learn a representation that adjusts for time-depending confounding and hence is suitable for causal estimation". However, the column under consideration does not stand for causal discovery, but rather whether the presence of possible confounders is discussed and taken into account in the design of the method. 

    Thanks for your comments on this. We will carefully revisit the wording of the table description to avoid any misunderstandings.

Reviewer comment (Writing):
Some of the notation and definitions in the paper were not properly introduced, such as a formal definition of POMDP, the meaning of $\hat{Q}_{t_j, t}$ in Eq. 1, the definition of “privileged data”, and the definition of “sub-flows”.

Author response:
6. **Notation and definitions.**
We apologise for any confusion due to definitions and choice of our notation. We would be happy to accommodate any suggestions you might have to explicate the POMDP framework beyond the current mathematical description of section 2.1. 

    We will also clarify the definition of $\hat{Q}_{t_j,t}$, to highlight that it refers to the model approximation of the change in the model state. It is a necessary term as it relates to these "sub flows". As the observations and actions are observed at discrete time points, we need to account for discontinuities in the derivative of the flow at these points. Thus the flow needs to be divided into continuous sub flows that must be summed over to obtain the approximation of the state at time t. We will be sure to include this at the beginning of section 3. 

    The term "privileged data" has been defined in Section 2.1 as "observations of the state that are not available for the intervening agent," and it is visually represented in Figure 1 with the red arrow. To provide further clarity, we can include a brief reminder or illustration in subsequent sections to ensure consistent understanding. How would you prefer us to elaborate on this matter?

Reviewer comment (Writing):
Some of the causal inference terminology is used inconsistently. For example, “counterfactual prediction” is sometimes used in the meaning of “interventional” (Sec. 4.2), and “counterfactual trajectories” are used instead of “interventional trajectories” (Sec. 4.1 “Counterfactual prediction”), although “interventional” and “counterfactual” denote fundamentally different concepts in causal inference.

Author response:
7. **Use of causal inference terminology.**
We appreciate your suggestion. In response, we will modify the wording in Section 4.2 to use "interventional trajectories" as it aligns more accurately with the setup. It's worth noting that while our terminology choice may differ from Seedat et al [4], we believe this adjustment will enhance clarity of our presentation.

    We argue that in section 4.1 (Counterfactual prediction) the term counterfactual trajectory is used correctly. Please refer to supplementary materials H for an in-depth discussion on this matter. The meaning of counterfactual trajectory is also illustrated in Figure 2 b. While we are fully aware of the distinction between the terms interventional and counterfactual, we welcome any specific suggestions or preferences you might have to enhance the presentation.

Reviewer comment (Writing):
The usage of the terms “observational” and “interventional” data is confusing in this paper. Usually, “observational data” is used for data where treatment assignment depends on observed and unobserved confounders, whereas “interventional data” refers to randomised control trials. In this paper, “observational” refers to data with unobserved confounders and “interventional” to data with all confounders observed.

Author response:
8. **Terms “observational” and “interventional” data**: 
We are sorry for any misunderstanding. We indeed use wording observational to refer to data where the treatment assignment depends on the observed and confounding variables and interventional on the case where the information about these confounders is missing (please see  2.1. where these terms are defined). We do follow the common standard as you described it, but instead of randomized policy in interventional data we state that the policy can be arbitrary and can also be learned via reinforcement learning. 

    While we recognize that "interventional" data often refers to randomized trials, we have found it challenging to identify a more suitable term for our setting. We are, however, open to any suggestions you might have in this regard.

Reviewer comment (Theory):
The authors seem to confuse the causal Markov condition and the Markov property of POMDP, which are two different things.

Author response:
9. **Causal Markov condition and Markov property of POMDP**:
We would like to respond to ”authors confused the causal Markov condition and Markov property of POMDP", emphasising that our usage is correct.

    By definition, POMDP satisfies the Markovian assumption with respect to $s_t$ [5]. Given our assumption that $o_{t_j} ,l_{t_j}$ are sufficient to fully describe $s_{t_j}$, the Markovian assumption also holds with respect to $o_{t_j} ,l_{t_j}$.    The Markovian causal model is illustrated in Figure 3.3 and it satisfies the parental Markov condition.

    Thus, both of these terms can be used in this context with parental Markovian condition arguably being more suitable for the causal discussion. Additionally, the most important conditional independences are outlined in the appendices for clarity. The matter is also discussed in Appendix C.

Reviewer comment (Writing):
What is exactly meant by the ability to perform “online prediction”?

Author response:
10. **Meaning of “online prediction”.** 
We appreciate your inquiry. In our context, "online prediction" signifies that information about the future state of the system is not available during the prediction process. Consequently, methods that rely on implicit (ODE) solvers or, for example, splines are not applicable. This notion of online prediction is crucial in real-world scenarios and reinforcement learning (RL) applications where foresight into future states is unavailable.

Reviewer comment (Experiments):
Why is the concatsquash block important for the architecture of TIF? Couldn’t it be done simpler?

Author response:
11. **Regarding the concatsquash block**: 
Thank you for bringing up the ConcatSquash block. Indeed, our framework does not necessitate its use.  Its selection as a component in our architecture was based on its demonstrated performance in the related literature on normalizing flows and neural ODEs. While we believe it intuitively fits our needs, we acknowledge that alternative layouts are also feasible.  Furthermore, tuning these architectures would likely enable further empirical and computational improvements.

Reviewer comment (Writing):
Despite the technical soundness, the paper seems difficult to follow, in lack of intuitive illustration of both newly introduced concepts or adopted approaches & notations. This makes it hard for readers to follow the logical reasoning and motivation for each specific modeling proposal.

Author response:
Thank you for your inputs. We appreciate your insights, and are currently working on enhancing readability, including, by incorporating more intuitive reasoning into the methods section and elucidating the experimental section as suggested. The revised version will be uploaded shortly.

Reviewer comment (Experiments):
The method section mainly focuses on "what" -- what was designed in the proposed approach, however without enough high-level intuitive illustration of "why". Similarly, in experiment section, the purpose, tasks and characteristics of the chosen dataset were not clearly presented, which makes it difficult to understand the motivation and analysis along without checking the external references.

Author response:
Thank you for your inputs. We appreciate your insights, and are currently working on enhancing readability, including, by incorporating more intuitive reasoning into the methods section and elucidating the experimental section as suggested. The revised version will be uploaded shortly.

Reviewer comment (Theory):
Due to the high time complexity of matrix inversion (in general O(n^3)), the computation cost would be significantly heavy for either large sample size or high data dimensionality.

Author response:
Thank you for the opportunity to shed light on the computational aspects of the proposed method.  We would like to emphasize that TIF does not necessitate full matrix inversion (which would indeed have been computationally expensive with a complexity of $O(D^3)$, where $D$ denotes the data dimension), instead inheriting the more tractable computation needs of continuous normalizing flows that can be solved more efficiently by integrating backward in time [1].

    Furthermore TIF inherits from the continuous flows normalizing a more efficient way of estimating the log likelihood.

    If the cost of evaluating TIF is $O(DH)$ where $D$ is the dimensionality of the data, and $H$ is the largest hidden dimension in the concatsquach block, then the time complexity of one transformation using a naive normalizing flow approach would be $O(DH + D^3)$

    The cubic format arises from using the change of variables to obtain complex densities $z \sim p_z(z)$ from a simple distribution $u \sim p_u(u)$ with transformation $T(u) =x$, as described in [1,2]

    $log(p_z(z))=log(p_u(u)) - log(|det J_T(u)|) ~, $

    where $J_T$ is the corresponding Jacobian matrix ($n \times n$), and the time complexity of computing the log determinant is $O(n^3)$.


    With continuous normalizing flows the complexity can be reduces to $O(DH + D^2)$, as the change in log probabilities can be written as [3, 2]

     $log (p(z_{t_1}))= log (p(z_{t_0})) - \int_{t_0}^{t_1} Tr \{J_{T}(z_t) \} dt~,$

    Where the time complexity of obtaining the exact Trace is $O(n^2)$ [1].

    Instead of exact Trace computation, we further reduce the time complexity to $O(DH + D)$ using \textit{Hutchinson’s trace estimator} to obtain an estimate of the trace [1,2,3].

    $Tr \{ J_{T}(z_t) \} = E_v [ v^T  J_{T}(z_t) v ]$

    where $v$ is a random vector with zero mean and unit covariance. We draw binary values for $v$ from a Rademacher Distribution. Reducing the time complexity to $O(n)$ [2].

    However the use of numerical ODE solvers introduces an additional cost as the transformation needs to be performed $L$ times, leading to an overall time complexity of $O(L(DH + D))$.  The impact of $L$ is significant in TIF, especially as the discontinuities of the derivative at the intervention points increase the number of steps taken by the numeric solver. Additionally, for the log likelihood sample, the model needs to be run forwards and backwards in time, i.e., $O(2L(DH + D))$. Finally, to obtain the Monte Carlo estimate (Appendix D), several samples are needed, which can, however, be computed in parallel.

Reviewer comment (Theory):
What's the overall time complexity of TIF?

Author response:
Thank you for the opportunity to shed light on the computational aspects of the proposed method.  We would like to emphasize that TIF does not necessitate full matrix inversion (which would indeed have been computationally expensive with a complexity of $O(D^3)$, where $D$ denotes the data dimension), instead inheriting the more tractable computation needs of continuous normalizing flows that can be solved more efficiently by integrating backward in time [1].

    Furthermore TIF inherits from the continuous flows normalizing a more efficient way of estimating the log likelihood.

    If the cost of evaluating TIF is $O(DH)$ where $D$ is the dimensionality of the data, and $H$ is the largest hidden dimension in the concatsquach block, then the time complexity of one transformation using a naive normalizing flow approach would be $O(DH + D^3)$

    The cubic format arises from using the change of variables to obtain complex densities $z \sim p_z(z)$ from a simple distribution $u \sim p_u(u)$ with transformation $T(u) =x$, as described in [1,2]

    $log(p_z(z))=log(p_u(u)) - log(|det J_T(u)|) ~, $

    where $J_T$ is the corresponding Jacobian matrix ($n \times n$), and the time complexity of computing the log determinant is $O(n^3)$.


    With continuous normalizing flows the complexity can be reduces to $O(DH + D^2)$, as the change in log probabilities can be written as [3, 2]

     $log (p(z_{t_1}))= log (p(z_{t_0})) - \int_{t_0}^{t_1} Tr \{J_{T}(z_t) \} dt~,$

    Where the time complexity of obtaining the exact Trace is $O(n^2)$ [1].

    Instead of exact Trace computation, we further reduce the time complexity to $O(DH + D)$ using \textit{Hutchinson’s trace estimator} to obtain an estimate of the trace [1,2,3].

    $Tr \{ J_{T}(z_t) \} = E_v [ v^T  J_{T}(z_t) v ]$

    where $v$ is a random vector with zero mean and unit covariance. We draw binary values for $v$ from a Rademacher Distribution. Reducing the time complexity to $O(n)$ [2].

    However the use of numerical ODE solvers introduces an additional cost as the transformation needs to be performed $L$ times, leading to an overall time complexity of $O(L(DH + D))$.  The impact of $L$ is significant in TIF, especially as the discontinuities of the derivative at the intervention points increase the number of steps taken by the numeric solver. Additionally, for the log likelihood sample, the model needs to be run forwards and backwards in time, i.e., $O(2L(DH + D))$. Finally, to obtain the Monte Carlo estimate (Appendix D), several samples are needed, which can, however, be computed in parallel.