Paper: Transport meets Variational Inference: Controlled Monte Carlo Diffusions (ICLR-2024)

Reviewer comment (Experiments):
The empirical results are not strong enough from both VI/transport side.

Author response:
Dear Reviewer ndxa,

We thank you for the time and effort taken to review our manuscript and for the many careful and insightful comments presented.

We believe that there might have been a small confusion and you have accidentally reviewed the wrong manuscript. It might be possible that you wrote your review based on a much earlier workshop version of this paper that was publicly available online before the start of the review period. Alternatively it may have been the case that you did not see the main contributions of our paper being a new sampling algorithm, which our extensive experimental section focuses on, instead providing comments on some additional toy 2D experiments that are only in the appendix.

We believe this may be the case since the focus of your feedback is oriented to experiments that are not present in the main manuscript that we have submitted on openreview. 

Furthermore, we sincerely request the reviewer to revisit the experimental section of our openreview submission, where we have high dimensional non-toy experiments on sampling from densities, which we believe is the main focus of this work, and quite different from the workshop submission. 

To explain why we believe this is the case we would like to point to the following comments and feedback the reviewer has provided:

1. “The ablation study of HJB-regulariser is weak.”
2. “that Figure 2 is more well behaved than non-regularized transport, we still found that Figure 3 is good enough in real practice.”

If you revisit the experiments in the main pdf, you will notice we have experiments on 8 different datasets (target densities) ranging in dimensions from 2 to 1600, and averaging a dimension of ~ 231 which is quite high-dimensional for our focus of sampling from unnormalised densities. We sincerely request that the reviewer draw conclusions based on these 8 experiments in the main text where we compare them to 7 different competing methods.

The reviewer's feedback refers only to the toy 2D experiments that are at the end of the appendix, which we are happy to address but we would reiterate that our main experimental section on sampling experiments is the focus of the paper, not the final appendix section, which we don't mention in the main text and is mainly included as an interesting aside. 

We would further like to remark that the HJB-regulariser is not our main proposed method / algorithm and as the title of the paper motivates, the controlled Monte Carlo diffusion sampler CMCD (which we have ablated thoroughly in our experiments) is the main contribution of this work. We apologise if this was a point of confusion. The HJB regulariser is mostly a conceptual/theoretical remark that is in line with our overarching framework whilst our experimental focus is on our proposed method for sampling (CMCD). Therefore, the minor empirical ablations on the HJB regularizer are left to the appendix and are by no means the central contribution of our paper. 

You will see in our main manuscript submitted here you can find quite a comprehensive array of experiments for sampling comparing our CMCD method to prior work (note these are not toy experiments):

1. Ionosphere Logistic Regression Target.       Dimension = 35
2. Log Sonar Logistic Regression Target.        Dimension = 61
3. Brownian Motion Posterior Target.               Dimension = 32
4. Lorenz Model Posterior Target.                    Dimension = 90
5. Seeds (Random effect regression) Target.   Dimension = 26
6. Log Gaussian Cox Process Target.              Dimension = 1600
7. Neals Funnel Target.                                     Dimension = 10
8. Toy Difficult GMM [1]                                     Dimension = 2

Note that across all these targets we significantly outperform an array of sampling methods that we compare to:

1. ULA [2]
2. UHA [3]
3. MCD-UHA [4,5]
4. MCD-UHA [4,5]
5. SMC          [6]
6. PIS            [7]
7. DDS          [8]


We would also like to highlight that these experiments are standard baselines for sampling used consistently across many prior works focused on sampling and partition function estimation [1,2,3,4,5,6,7,8]. Note that for sampling, unlike generative modelling, dimensions >= 50 are already considered to be very high dimensional, as many of the provided references suggest.  

Finally, to further the extensive nature of these experiments we would like to cite reviewer LiYc’s remark:

“Extensive experiments show that the CMCD method outperforms existing methods in relatively high dimensional examples.”

We hope the reviewer can agree that these experiments are significantly different and much more extensive than the toy 2D experiments from the appendix. We sincerely request the reviewer to reconsider the comprehensive sampling experiments we ran to support our new sampling algorithm as the core contribution, and we would be grateful if they could reevaluate their score accordingly.

Reviewer comment (Experiments):
The ablation study of HJB-regulariser is weak.

Author response:
Dear Reviewer ndxa,

We thank you for the time and effort taken to review our manuscript and for the many careful and insightful comments presented.

We believe that there might have been a small confusion and you have accidentally reviewed the wrong manuscript. It might be possible that you wrote your review based on a much earlier workshop version of this paper that was publicly available online before the start of the review period. Alternatively it may have been the case that you did not see the main contributions of our paper being a new sampling algorithm, which our extensive experimental section focuses on, instead providing comments on some additional toy 2D experiments that are only in the appendix.

We believe this may be the case since the focus of your feedback is oriented to experiments that are not present in the main manuscript that we have submitted on openreview. 

Furthermore, we sincerely request the reviewer to revisit the experimental section of our openreview submission, where we have high dimensional non-toy experiments on sampling from densities, which we believe is the main focus of this work, and quite different from the workshop submission. 

To explain why we believe this is the case we would like to point to the following comments and feedback the reviewer has provided:

1. “The ablation study of HJB-regulariser is weak.”
2. “that Figure 2 is more well behaved than non-regularized transport, we still found that Figure 3 is good enough in real practice.”

If you revisit the experiments in the main pdf, you will notice we have experiments on 8 different datasets (target densities) ranging in dimensions from 2 to 1600, and averaging a dimension of ~ 231 which is quite high-dimensional for our focus of sampling from unnormalised densities. We sincerely request that the reviewer draw conclusions based on these 8 experiments in the main text where we compare them to 7 different competing methods.

The reviewer's feedback refers only to the toy 2D experiments that are at the end of the appendix, which we are happy to address but we would reiterate that our main experimental section on sampling experiments is the focus of the paper, not the final appendix section, which we don't mention in the main text and is mainly included as an interesting aside. 

We would further like to remark that the HJB-regulariser is not our main proposed method / algorithm and as the title of the paper motivates, the controlled Monte Carlo diffusion sampler CMCD (which we have ablated thoroughly in our experiments) is the main contribution of this work. We apologise if this was a point of confusion. The HJB regulariser is mostly a conceptual/theoretical remark that is in line with our overarching framework whilst our experimental focus is on our proposed method for sampling (CMCD). Therefore, the minor empirical ablations on the HJB regularizer are left to the appendix and are by no means the central contribution of our paper. 

You will see in our main manuscript submitted here you can find quite a comprehensive array of experiments for sampling comparing our CMCD method to prior work (note these are not toy experiments):

1. Ionosphere Logistic Regression Target.       Dimension = 35
2. Log Sonar Logistic Regression Target.        Dimension = 61
3. Brownian Motion Posterior Target.               Dimension = 32
4. Lorenz Model Posterior Target.                    Dimension = 90
5. Seeds (Random effect regression) Target.   Dimension = 26
6. Log Gaussian Cox Process Target.              Dimension = 1600
7. Neals Funnel Target.                                     Dimension = 10
8. Toy Difficult GMM [1]                                     Dimension = 2

Note that across all these targets we significantly outperform an array of sampling methods that we compare to:

1. ULA [2]
2. UHA [3]
3. MCD-UHA [4,5]
4. MCD-UHA [4,5]
5. SMC          [6]
6. PIS            [7]
7. DDS          [8]


We would also like to highlight that these experiments are standard baselines for sampling used consistently across many prior works focused on sampling and partition function estimation [1,2,3,4,5,6,7,8]. Note that for sampling, unlike generative modelling, dimensions >= 50 are already considered to be very high dimensional, as many of the provided references suggest.  

Finally, to further the extensive nature of these experiments we would like to cite reviewer LiYc’s remark:

“Extensive experiments show that the CMCD method outperforms existing methods in relatively high dimensional examples.”

We hope the reviewer can agree that these experiments are significantly different and much more extensive than the toy 2D experiments from the appendix. We sincerely request the reviewer to reconsider the comprehensive sampling experiments we ran to support our new sampling algorithm as the core contribution, and we would be grateful if they could reevaluate their score accordingly.

Reviewer comment (Experiments):
From the transport side, although it is understandable that Figure 2 is more well behaved than non-regularized transport, Figure 3 is still good enough in real practice. Can you find an example where the non-regularized way fails?

Author response:
Dear Reviewer ndxa,

We thank you for the time and effort taken to review our manuscript and for the many careful and insightful comments presented.

We believe that there might have been a small confusion and you have accidentally reviewed the wrong manuscript. It might be possible that you wrote your review based on a much earlier workshop version of this paper that was publicly available online before the start of the review period. Alternatively it may have been the case that you did not see the main contributions of our paper being a new sampling algorithm, which our extensive experimental section focuses on, instead providing comments on some additional toy 2D experiments that are only in the appendix.

We believe this may be the case since the focus of your feedback is oriented to experiments that are not present in the main manuscript that we have submitted on openreview. 

Furthermore, we sincerely request the reviewer to revisit the experimental section of our openreview submission, where we have high dimensional non-toy experiments on sampling from densities, which we believe is the main focus of this work, and quite different from the workshop submission. 

To explain why we believe this is the case we would like to point to the following comments and feedback the reviewer has provided:

1. “The ablation study of HJB-regulariser is weak.”
2. “that Figure 2 is more well behaved than non-regularized transport, we still found that Figure 3 is good enough in real practice.”

If you revisit the experiments in the main pdf, you will notice we have experiments on 8 different datasets (target densities) ranging in dimensions from 2 to 1600, and averaging a dimension of ~ 231 which is quite high-dimensional for our focus of sampling from unnormalised densities. We sincerely request that the reviewer draw conclusions based on these 8 experiments in the main text where we compare them to 7 different competing methods.

The reviewer's feedback refers only to the toy 2D experiments that are at the end of the appendix, which we are happy to address but we would reiterate that our main experimental section on sampling experiments is the focus of the paper, not the final appendix section, which we don't mention in the main text and is mainly included as an interesting aside. 

We would further like to remark that the HJB-regulariser is not our main proposed method / algorithm and as the title of the paper motivates, the controlled Monte Carlo diffusion sampler CMCD (which we have ablated thoroughly in our experiments) is the main contribution of this work. We apologise if this was a point of confusion. The HJB regulariser is mostly a conceptual/theoretical remark that is in line with our overarching framework whilst our experimental focus is on our proposed method for sampling (CMCD). Therefore, the minor empirical ablations on the HJB regularizer are left to the appendix and are by no means the central contribution of our paper. 

You will see in our main manuscript submitted here you can find quite a comprehensive array of experiments for sampling comparing our CMCD method to prior work (note these are not toy experiments):

1. Ionosphere Logistic Regression Target.       Dimension = 35
2. Log Sonar Logistic Regression Target.        Dimension = 61
3. Brownian Motion Posterior Target.               Dimension = 32
4. Lorenz Model Posterior Target.                    Dimension = 90
5. Seeds (Random effect regression) Target.   Dimension = 26
6. Log Gaussian Cox Process Target.              Dimension = 1600
7. Neals Funnel Target.                                     Dimension = 10
8. Toy Difficult GMM [1]                                     Dimension = 2

Note that across all these targets we significantly outperform an array of sampling methods that we compare to:

1. ULA [2]
2. UHA [3]
3. MCD-UHA [4,5]
4. MCD-UHA [4,5]
5. SMC          [6]
6. PIS            [7]
7. DDS          [8]


We would also like to highlight that these experiments are standard baselines for sampling used consistently across many prior works focused on sampling and partition function estimation [1,2,3,4,5,6,7,8]. Note that for sampling, unlike generative modelling, dimensions >= 50 are already considered to be very high dimensional, as many of the provided references suggest.  

Finally, to further the extensive nature of these experiments we would like to cite reviewer LiYc’s remark:

“Extensive experiments show that the CMCD method outperforms existing methods in relatively high dimensional examples.”

We hope the reviewer can agree that these experiments are significantly different and much more extensive than the toy 2D experiments from the appendix. We sincerely request the reviewer to reconsider the comprehensive sampling experiments we ran to support our new sampling algorithm as the core contribution, and we would be grateful if they could reevaluate their score accordingly.

Reviewer comment (Novelty):
Although the HJB-regulariser may be useful in some cases, given the current justifications and evidence, its significance is not clear at this moment.

Author response:
Dear Reviewer ndxa,

We thank you for the time and effort taken to review our manuscript and for the many careful and insightful comments presented.

We believe that there might have been a small confusion and you have accidentally reviewed the wrong manuscript. It might be possible that you wrote your review based on a much earlier workshop version of this paper that was publicly available online before the start of the review period. Alternatively it may have been the case that you did not see the main contributions of our paper being a new sampling algorithm, which our extensive experimental section focuses on, instead providing comments on some additional toy 2D experiments that are only in the appendix.

We believe this may be the case since the focus of your feedback is oriented to experiments that are not present in the main manuscript that we have submitted on openreview. 

Furthermore, we sincerely request the reviewer to revisit the experimental section of our openreview submission, where we have high dimensional non-toy experiments on sampling from densities, which we believe is the main focus of this work, and quite different from the workshop submission. 

To explain why we believe this is the case we would like to point to the following comments and feedback the reviewer has provided:

1. “The ablation study of HJB-regulariser is weak.”
2. “that Figure 2 is more well behaved than non-regularized transport, we still found that Figure 3 is good enough in real practice.”

If you revisit the experiments in the main pdf, you will notice we have experiments on 8 different datasets (target densities) ranging in dimensions from 2 to 1600, and averaging a dimension of ~ 231 which is quite high-dimensional for our focus of sampling from unnormalised densities. We sincerely request that the reviewer draw conclusions based on these 8 experiments in the main text where we compare them to 7 different competing methods.

The reviewer's feedback refers only to the toy 2D experiments that are at the end of the appendix, which we are happy to address but we would reiterate that our main experimental section on sampling experiments is the focus of the paper, not the final appendix section, which we don't mention in the main text and is mainly included as an interesting aside. 

We would further like to remark that the HJB-regulariser is not our main proposed method / algorithm and as the title of the paper motivates, the controlled Monte Carlo diffusion sampler CMCD (which we have ablated thoroughly in our experiments) is the main contribution of this work. We apologise if this was a point of confusion. The HJB regulariser is mostly a conceptual/theoretical remark that is in line with our overarching framework whilst our experimental focus is on our proposed method for sampling (CMCD). Therefore, the minor empirical ablations on the HJB regularizer are left to the appendix and are by no means the central contribution of our paper. 

You will see in our main manuscript submitted here you can find quite a comprehensive array of experiments for sampling comparing our CMCD method to prior work (note these are not toy experiments):

1. Ionosphere Logistic Regression Target.       Dimension = 35
2. Log Sonar Logistic Regression Target.        Dimension = 61
3. Brownian Motion Posterior Target.               Dimension = 32
4. Lorenz Model Posterior Target.                    Dimension = 90
5. Seeds (Random effect regression) Target.   Dimension = 26
6. Log Gaussian Cox Process Target.              Dimension = 1600
7. Neals Funnel Target.                                     Dimension = 10
8. Toy Difficult GMM [1]                                     Dimension = 2

Note that across all these targets we significantly outperform an array of sampling methods that we compare to:

1. ULA [2]
2. UHA [3]
3. MCD-UHA [4,5]
4. MCD-UHA [4,5]
5. SMC          [6]
6. PIS            [7]
7. DDS          [8]


We would also like to highlight that these experiments are standard baselines for sampling used consistently across many prior works focused on sampling and partition function estimation [1,2,3,4,5,6,7,8]. Note that for sampling, unlike generative modelling, dimensions >= 50 are already considered to be very high dimensional, as many of the provided references suggest.  

Finally, to further the extensive nature of these experiments we would like to cite reviewer LiYc’s remark:

“Extensive experiments show that the CMCD method outperforms existing methods in relatively high dimensional examples.”

We hope the reviewer can agree that these experiments are significantly different and much more extensive than the toy 2D experiments from the appendix. We sincerely request the reviewer to reconsider the comprehensive sampling experiments we ran to support our new sampling algorithm as the core contribution, and we would be grateful if they could reevaluate their score accordingly.

Reviewer comment (Reproducibility):
I don't have major comments regarding weaknesses other than that the source code for the experimental validation needs to be provided in the supplement and not only via some possibly non-permanent web link.

Author response:
>  weaknesses other than that the source code for the experimental validation needs to be provided in the supplement

We have uploaded a zip file to the supplement with our source code as requested. We will also open-source the code on Github at the camera-ready stage (if the paper is accepted), where it will be available publicly.

Reviewer comment (Experiments):
In my opinion the paper could benefit from a shorter introduction and motivation and instead an extended experimental section with method limitations, experimental details, computational trade-offs, etc. for more applied audiences. Since this is likely not possible due to page limits, an extended treatment in the appendix would be very welcome.

Author response:
> In my opinion the paper could benefit from a shorter introduction and motivation and instead an extended experimental section with method limitations, experimental details, 

We have moved the HJB remarks which were not central to our story to the appendix. As a result we expanded the practical motivation of the work further and added Algorithm 1 detailing CMCD which should heavily benefit more applied audiences.

> computational trade-offs, etc. for more applied audiences. Since this is likely not possible due to page limits, an extended treatment in the appendix would be very welcome.

We have added Table 8 which details the sampling and loss complexity of the main methods that we compare to. In particular, we have also added wallclock times for inference (sampling + ln Z estimation) comparing several of the ULA + MCD-based methods and ours. We can observe that once trained, sampling time is comparable across approaches.

Reviewer comment (Experiments):
What are the training times of the method, for instance, in comparison to running SMC?

Author response:
>  What are the training times of the method, for instance, in comparison to running SMC?

It is not entirely straightforward to compare SMC to our approach taking training into account, as given our fixed training iterations we dramatically outperform SMC across challenging high dimensional targets. 

To carry out an insightful comparison, we chose the LGCP target, which is our most numerically intense target, and we phrase the following question:

“For how long do we have to train CMCD to outperform the best-run of SMC”

For this, we look at Figure 1 and see that at $K=8$, CMCD already outperforms SMC at $K=256$ with 2000 particles. So we choose these two approaches to compare to each other.  Below is a brief comparison of training-time calculation, where we have included the tuning time for SMC, which is akin to our training time (as without tuning SMC’s hyperparameters, ELBOs and ln Z estimations were much worse). 

| Method | Train + Sample Time (min) | ln Z                | ELBO                  |
|--------|---------------------------|---------------------|-----------------------|
| CMCD   | $33.12 \pm 0.12$          | $491.059 \pm 3.553$  | $469.475 \pm  0.2589$ |
| SMC    | $62.62 \pm 0.1$           | $477.162 \pm 4.998$ | $453.395 \pm 4.43$ |

Reviewer comment (Experiments):
It is quite unfortunate that there are no experiments for the loss proposed in Proposition 3.2 to back the fact that the proposed loss does not suffer from mode forgetting.

Author response:
Out of the alternate losses we discuss, it is the log-variance divergence (introduced after Equation 24)  that can handle mode collapse: 

$$ \mathcal{L}^{\mathrm{Var}}(\phi)=\mathrm{Var}\left(\ln \frac{\pi_T\left(\boldsymbol{Y}_T\right)}{\pi_0\left(\boldsymbol{Y}_0\right)}+\int_0^T \Delta \phi_t\left(\boldsymbol{Y}_t\right) \mathrm{d} t-\sigma \sqrt{2} \int_0^T \nabla \ln \pi_t\left(\boldsymbol{Y}_t\right) \circ \mathrm{d} \boldsymbol{W}_t-\sigma^2 \int_0^T\left|\nabla \ln \pi_t\left(\boldsymbol{Y}_t\right)\right|^2 \mathrm{~d} t\right)$$

The loss with the HJB/PINN regularizer in Proposition 3.2 does not address the issue of mode collapse, and its application is more relevant to different problems (relevant to molecular dynamics).

> Furthermore, it is not clear how this is implemented in practice since a Laplacian term is involved.

As you have pointed out, there are some Laplacian terms that are not immediately obvious how we could compute. However, as it turns out the term inside the variance is simply the RND between the forward and backwards SDEs, so we can use the exact same discretisation that we used for KL to discretise it and thus: 

$$ \mathcal{L}^{\mathrm{Var}}(\phi)  \approx \mathrm{Var}\left[\ln\frac{\pi_0(Y_0)}{\hat{\pi} (Y_T)}\prod_{k=0}^{K-1}\frac{\mathcal{N}(Y_{t_{k+1}} |Y_{t_k}  + (\nabla \ln \pi_{t_{k}}+\nabla \ln \phi_{t_k})(Y_{t_k}) \Delta t_k , 2 \sigma^2 \Delta t_k)}{\mathcal{N}( Y_{t_k} |Y_{t_{k+1}}   + (\nabla \ln \pi_{t_{k+1}}-\nabla \ln \phi_{t_{k+1}})(Y_{t_{k+1}}) \Delta t_k , 2 \sigma^2 \Delta t_k)}\right]$$


This makes this loss as scalable as the KL divergence CMCD objective that we ablate. In particular, it is a bit more scalable as the Variance can be taken w.r.t to any measure, thus allowing us to detach the samples and circumvent backpropagating through timesteps.

>  .. There are no experiments for the . ..

To address your main weakness, we explore the log-variance (logvar) loss in the very challenging 40 GMM target [2] for which reverse KL-based methods all mode collapse.
We compare CMCD KL to CMCD with logvar, and we demonstrate how it significantly outperforms the KL-based loss, both in ELBO but also qualitatively in its ability to overcome mode collapse. The results and plots illustrating mode collapse can be found in **Appendix F.3. (highlighted in olive)** in particular **Tabe 2, Figures 3 and 4**, and here is a table to summarise them

|                     |         ELBO        |        $\ln Z$       |   $\mathcal{W}_2$  |
|---------------------|:-------------------:|:--------------------:|:------------------:|
| *log-variance* loss |  -1.279 $\pm$ 0.096 | -0.065 $\pm$ $0.101$ | 0.0143 $\pm$ 0.001 |
| KL loss             | -2.286 $\pm$ 0.1109 |  -0.244 $\pm$ 0.3309 | 0.0441 $\pm$ 0.012 |


> …  for the loss proposed in Proposition 3.2 to back the fact that the proposed loss does not suffer from mode forgetting ..

The loss in proposition 3.2 (now moved to the appendix) solves a different problem to mode collapse. Rather than addressing mode forgetting, it deals with forgetting the reference SDE. This does not solve practical issues (e.g. such as mode collapse) in sampling or generative modelling, but it might be useful in transition path problems in MD. **In appendix G**, we do some small ablations and demonstrate across a series of toy targets (including an MD double well problem) that it does indeed do well in not forgetting the reference process.  To address the Laplacian term computation in this loss in this case, we used specialised architectures from OT [1], which allow for fast calculation of these terms.

Finally, we note we have added a series of further experiments and ablations to address the other reviewers as well as improved the presentation of our work.

We hope that this addresses your main weakness, and we sincerely request any further feedback that you may be able to provide us with regarding our comprehensive changes to the presentation of the paper, as well as additional ablations and baselines. Thank you for your continued feedback!

[1] Onken, D., Fung, S.W., Li, X. and Ruthotto, L., 2021, May. Ot-flow: Fast and accurate continuous normalizing flows via optimal transport. In Proceedings of AAAI

[2] Midgley, L.I., Stimper, V., Simm, G.N., Schölkopf, B. and Hernández-Lobato, J.M., 2022. Flow annealed importance sampling bootstrap. ICLR 2023

Reviewer comment (Reproducibility):
It is not clear how this is implemented in practice since a Laplacian term is involved.

Author response:
As you have pointed out, there are some Laplacian terms that are not immediately obvious how we could compute. However, as it turns out the term inside the variance is simply the RND between the forward and backwards SDEs, so we can use the exact same discretisation that we used for KL to discretise it and thus: 

$$ \mathcal{L}^{\mathrm{Var}}(\phi)  \approx \mathrm{Var}\left[\ln\frac{\pi_0(Y_0)}{\hat{\pi} (Y_T)}\prod_{k=0}^{K-1}\frac{\mathcal{N}(Y_{t_{k+1}} |Y_{t_k}  + (\nabla \ln \pi_{t_{k}}+\nabla \ln \phi_{t_k})(Y_{t_k}) \Delta t_k , 2 \sigma^2 \Delta t_k)}{\mathcal{N}( Y_{t_k} |Y_{t_{k+1}}   + (\nabla \ln \pi_{t_{k+1}}-\nabla \ln \phi_{t_{k+1}})(Y_{t_{k+1}}) \Delta t_k , 2 \sigma^2 \Delta t_k)}\right]$$


This makes this loss as scalable as the KL divergence CMCD objective that we ablate. In particular, it is a bit more scalable as the Variance can be taken w.r.t to any measure, thus allowing us to detach the samples and circumvent backpropagating through timesteps.

Reviewer comment (Presentation):
The presentation of the paper needs to be revised extensively as it is confusing and posits ideas without forewarning or motivation. The introduction is especially difficult to read, not due to technicality, but due to lack of clarity in writing and insufficient motivation. While the math is quite readable, the writing and presentation require improvements. For instance, the problem being tackled (CMCD) is not explicitly stated in the introduction, and only appears on page 7, eq. (22).

Author response:
Dear Reviewer yH6G,

We thank you for the time and the very detailed feedback, especially since the presentation difficulties you have encountered must have required more time in parsing through our manuscript, and we apologise for the same.

We would also like to thank you for the very positive comments regarding the strength of the technical contribution of our work. We really appreciate your feedback and suggestions for improvement despite the presentation of the paper, while still being able to provide excellent actionable feedback on our technical contribution.

We would like to work with you as much as possible through this discussion period to understand how we can further improve the presentation. We appreciate that you must have many other commitments but we would greatly appreciate it if you could engage with us in improving the presentation of our work.

We have uploaded a revised version of the manuscript where we have attempted to address some of the changes recommended by the reviewer, these changes have been coloured in **magenta/pink**. 

The revised manuscript (https://openreview.net/pdf?id=PP1rudnxiW) contains the following changes which are highlighted in the revised version.

1. We have mentioned that our focus is the sampling task from a very early stage in the introduction and we formally introduce the sampling task in a titled paragraph, connecting it to framework 1 and thus providing motivation to our setup. 
2. We have reworded the abstract to acknowledge the focus on the sampling task and we have removed the remark on the HJB regularisers from the abstract as this is not our central contribution. 
4. We have also highlighted that the EM ⇔ IPF correspondence mostly serves a conceptual purpose that connects two adjacent fields rather than an algorithmic one.
5. We have changed our contributions to shift the focus to CMCD and the sampling task.
6. On page 6, we removed non-central results such as the paragraph on HJB regularizers and the corollary that followed. As before, we highlight that the EM and IPF connection is conceptual rather than practical. 
7. Also on page 6 we explain more clearly the issues present with EM and IPF, emphasising that an accumulation of errors occurs due to their sequential nature. We then signpost to the next section motivating that our proposed method CMCD can be trained end to end with simultaneous updates and thus solve this sequential error accumulation issue.
8. We have also added pseudocode for sampling and $\ln Z$ estimation with CMCD and we hope that this also draws the focus/attention to the task and enhances the readability.

Additionally, we have also tried restructuring the introduction in a different way which can be found in this anonymised link https://anonymous.4open.science/r/CMCD-6BF6/rebuttal_version.pdf
(same link as the anonymised codebase), here we changed the way the framework is introduced to try and make it more amenable to sampling. We hope that one of these revisions addresses some of the presentation issues raised by the reviewer.

We hope that the revised presentation and rebuttal pave the way to addressing the reviewer's concerns and we are eager to hear the reviewer's feedback plus further points of improvement.

Reviewer comment (Writing):
The reviewer cannot suggest any single modification that could drastically improve the paper, because the overall contextual clarity is lacking, making it unhelpful to the wider community in its current state despite the technical contributions.

Author response:
> It pains me to say that I cannot suggest the authors any single thing that could drastically improve the paper. I appreciate the technical contribution. 

We are very sorry that the initial presentation of our work has made you feel this way, however, we still believe your comments have been very helpful and have led us to make significant restructuring changes across the paper.

Reviewer comment (Writing):
The first two pages of the introduction should be thoroughly revised to focus more on the problem addressed in the paper, rather than overloading the reader with related technical terms and connections.

Author response:
> At the very least, I think the first two pages of the introduction should be thoroughly revised to focus more on the problem addressed in the paper

We have added the sampling task (which is the task CMCD solves) to the start of the introduction and modified the signposting and motivational elements in the introduction to refer more to this task.

> Rather than overloading on related technical terms and connections. 

We have not fully removed the connections / related work from the introduction as you requested for the following reasons

1. Variational inference is effectively the flavour of the task we are solving. We are doing sampling through VI using ideas from optimal transport. This requires us to introduce and discuss VI (Framework 1) in the introduction.
2. ELBOs are used for solving the second part of the sampling problem, namely estimating the normalising constant, and thus it is essential for us to introduce these earlier on.
3. OT and coupling allow us to highlight some of the pitfalls that doing full-blown VI can have, and as part of the motivation we want to discuss these issues in the introduction since they motivate our proposed improvements.

Instead, we have tried to reinforce their relevance to the sampling problem.

Reviewer comment (Presentation):
Propositions that are not key contributions of the paper (for instance, Proposition 2.1) could be better placed in the supplemental materials.

Author response:
> I also think the propositions which are not key contributions of the pape … could be better placed in the supplements.

We have moved proposition 3.2 to the appendix as well as the section on HJB reguralisers which are not part of our central algorithmic contribution/experiments. We hope this allows the reader to have a more clear and contextual focus on the story.

Reviewer comment (Theory):
Question — Is eq. (10) the same as eq. (1)? The reviewer failed to find the difference.

Author response:
> Is eq. (10) the same as eq (1)? I failed to find the difference.

The following equations are taken from the unrevised version of the manuscript that you read.

Equation (1): 

$$\boldsymbol{z} \sim \nu(\boldsymbol{z}), \quad \boldsymbol{x} \mid \boldsymbol{z} \sim p^\theta(\boldsymbol{x} \mid \boldsymbol{z})$$

Equation (10): 

$$\mathcal{L}^{ext}(\phi, \theta) :=D(q^{\mu, \phi} ( y_{0: L} ) \vert\vert p^{\nu, \theta} ( y_{0: L}))$$

Equation (1) specifies a “static” generative model whilst equation (10) details the divergence between two pathwise (many latent) generative models. Is it possible the reviewer was referring to a different pair of equations? Maybe they meant equations 10 and 5: 

Equation (5)

$$ \mathcal{L}_D(\phi, \theta):=D\left(q^\phi(\boldsymbol{z} \mid \boldsymbol{x}) \mu(\boldsymbol{x}) \vert\vert p^\theta(\boldsymbol{x} \mid \boldsymbol{z}) \nu(\boldsymbol{z})\right)$$

Both Equations 10 and 5 define the loss as a divergence between backwards and forwards generative models. However, Equation 10 (as the text preceding it details) is a path KL divergence because the generative model has many intermediate latent variables (which becomes an SDE in the limit) whilst Equation 5 does not have any of these intermediate latents. It is a static/non-dynamic setup (you can think of the intermediaries as being marginalised out). 

The purpose of having both these equations is, as reviewer LiYc highlights (i.e. “I enjoyed reading this paper; it is highly pedagogical and sheds light … ”), pedagogical. We present our work and framework in a pedagogical manner, first starting from simple static generative models that readers may be familiar with from VAE literature, and then we slowly build up from there to the non-static / dynamic SDE setting.