Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors adapt the formalism of denoising diffusion models to learn physics trajectories. In particular, they consider a forward diffusion given by a differentiable physics operator. The noise of the forward diffusion is assumed to be inherent noise or measurement error. Once this is done, the authors learn the backward dynamics by backpropagating through the dynamics using the square distance between the observed forward point and generated point. They consider both Ordinary Differential Equation (ODE) and Stochastic Differential Equation (SDE) settings. They apply their method to 3 problems: a toy heat equation problem, a buoyancy-driven flow with obstacles and an isotropic turbulence problem. The method outperforms other neural network approaches to learn dynamics.
strength_and_weaknesses: STRENGTHS:
* I think the connection of differentiable physics and denoising diffusion models is interesting and is a nice application of this generative model framework to physics applications. I believe that this paper is a step towards the use of generative model for simulating trajectories in physics.
* The experimental section is quite strong. I especially enjoyed some of the findings regarding the differences between SDE and ODE which I think are both intriguing and interesting.

WEAKNESSES:
* For me the main weakness of the paper is how the training is conducted. By using the loss function the authors use in the paper they lose the flexibility of diffusion models which do not require unrolling the dynamics. If I'm right I think the authors did not apply classical diffusion models here because they did not have access to the transition kernel of the forward dynamics (since it depends on the differentiable physics operator). However there are ways to circumvent this issue by looking at the Implicit Score Matching loss, see for instance [1] (and [2] for an older reference in the score matching literature). If one does not want to use that loss then one can leverage the fact that the Euler-Maruyama discretization of the forward dynamics yield Gaussian steps (locally), see [3]. Both these approaches allow for efficient training of diffusion models even in the case where the forward dynamics is given by a differentiable physics operator. These losses are more computationally friendly and yield good results in practice

* Regarding the bidirectional optimization, recent papers on Schrodinger Bridges using diffusion models [3,4] rely on some bidirectional models. I think the authors should discuss the link between this approach and these papers. Finally, I think that the authors missed the opportunity to discuss [5] which advocates for dropping the noise in diffusion models. I think that this is not always beneficial, dropping the noise makes sense here, as the authors justify the presence of the noise by saying that it "is added to the data at each time step [and] can be regarded as either noise to the physical problem or as noise from a measurement process". Gaussian noise with such covariance seems rather arbitrary and seems to be present only for the convenience of having well-defined probability measures.

OTHER COMMENTS:
* In the reconstruction loss in the first experiment (Section 4.1). I don't really understand how this can work. If $T$ is large then the operator $P$ destroys all information and converges to a stationary state. Then how can we check that we recover $x_T$ (or are all $x_T$ the same in that case)?
* I would have appreciated more details on the bidirectional method.
* I also would have appreciated more explanations as of why SDE does not perform well in the buoyancy driven flow example.
* I am not competent enough to assess if the baselines chosen by the authors are relevant and will not comment on them.

[1] A Variational Perspective on Diffusion-Based Generative Models and Score Matching - Huang, Courville
[2] Estimation of Non-Normalized Statistical Models by Score Matching - Hyvarinen
[3] Diffusion Schrödinger Bridge with Applications to Score-Based Generative Modeling - De Bortoli, Thornton, Heng, Doucet
[4] Likelihood Training of Schrödinger Bridge using Forward-Backward SDEs Theory - Chen, Liu, Theodorou
[5] Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise - Bansal, Borgnia, Chu, Li, Kazemi, Huang, Goldblum, Goldstein
clarity,_quality,_novelty_and_reproducibility: The paper is quite clear except for a few points (see above comment).

The experimental section is quite thorough and overall the paper is of good quality.

The application of diffusion models to differentiable physics is new to my knowledge.

Sufficient experimental details are given.
summary_of_the_review: I think this paper represents a nice application of diffusion models to physics.
Apart from the training procedure which seems strange I think the methodology and experimental settings are valid.
I think this paper opens the door to more fruitful interactions between the diffusion models and physical systems.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper uses score matching SDEs for learning inverse problems with physical simulators. Different from common diffusion models, the forward SDE is changed: the drift is given by a physical model and the diffusion term does not grow to infinity as time increases. Training and inference algorithms based on SDEs and probabilistic ODEs are discussed. Empirical results are evaluated on several inverse problems and compared against baselines, demonstrating the empirical advantage of the proposed SMDP method.
strength_and_weaknesses: Strength:
- Empirical performance seems to be better compared to baselines.
- The idea is an interesting take at inverse problem solving using the idea of diffusion models.

Weakness:

The method section is not very clear. 
- For example, in $\{x_t^n\} \subset R^{d \times T}$ is confusing. What is $d$? With notation $d \times T$ does this imply that the trajectories are sampled at discrete times? The $t_{m}$ notation should come after you discuss Euler-Maruyama discretization steps?
- What is the loss function, and are there any principled justification to this? My understanding is that this is the loss function in (Zhang & Chen, 2021) but with the drift being non-trainable. If this is true, then it would help to make this explicit, as it helps understanding the paper by a great deal. While the technical novelty is lowered, this is still a non-trivial application of diffusion normalizing flows (since Zhang & Chen mostly considered generative modeling problems).

It is unclear why both ODE and SDE can be used. 
- The goal in inverse problems is to sample from the full posterior. In that sense, ODE will only give one solution, which as the paper suggests, does not cover many candidate solutions. This is also because ODE does not recover the measure of the entire path, just the marginals. Wouldn't this suggest that, assuming the forward model is correct, the SDE is the **only** correct way of sampling from the posterior, since it should match the path measures.

Evaluations are not precisely targeted towards posterior sampling.
- If my understanding is correct, then in Table 1, only one path is compared against. However, if we have a non-zero g, then there are naturally many paths with similarly high probability of occurring, and comparing against one path is going to place posterior sampling at a disadvantage. I wonder if it is possible to obtain a "ground truth" posterior and compare path distributions with that? For example, using metrics like maximum mean discrepancy or divergences based on KDEs.
- The paper address the above by having g=0 during evaluation. However, would this generate the same score function as in g=0.1 during training?


Not weakness, but curious questions:
- Why does the FNO perform so badly in Table 1?
clarity,_quality,_novelty_and_reproducibility: Clarity (Fair): the method section can use a bit of improvement, such as what is the loss function, and some notational changes from background to method section. Also, it would be nice to reiterate how the setting is different from standard diffusion models in generative settings (where we only care about the marginal at t=0).

Quality (High): the experiments are generally well-done, except for the fact that posterior sampling is not well evaluated.

Novelty (High): while this seems to be an application of diffusion normalizing flows, it is an interesting one beyond the scope of the original paper.

Reproducibility (Fair): while the authors promised code release, the method section does not clearly describe the precise loss function. Specifically, it would be nice to expand the line on "We update the weights θ of sθ by unrolling and backpropagating though all ODE solver steps"
summary_of_the_review: While this seems to be an interesting application of diffusion normalizing flows to physical inverse problems, I am a bit concerned by the clarity in the method section. There are some notation issues, and even the loss function is not precisely defined (either in the main paper or in the appendix). But I do think the method itself is valid and experiments generally support the validity of the method.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper describes an ODE-based model for inverting the time-evolution of a physical system, aiming to predict a distribution of initial states which could have given rise to this system. Their approach assumes access to a differentiable simulator of the physical system, and represents the reverse-time evolution of the system as either a PDE or an ODE that combines the simulator with a neural network force term. The paper proposes training this force term such that the reversed trajectories under the ODE are close (under L2 norm) to true (forward) trajectories generated by the physical system running in the normal direction. The authors present results for a few physical processes, including heat diffusion, buoyancy-driven flow, and isotropic turbulence, and find that their method yields good approximations of the initial state.

~~Although this approach is motivated by score matching and denoising diffusion generative models, and the authors spend considerable time explaining the background of these models, the actual proposed approach deviates in a number of ways from proper SDE-based diffusion approaches. It seems to me that this is not really a diffusion model in the sense used by the generative model community, and the probability-flow ODE appears to be used "incorrectly" from the perspective of score matching and distribution fitting. I think the proposed methodology would be better understood as a (empirically successful) parameterization and denoising-based training strategy for an inverse model, without reference to generative diffusion models or probability flow.~~

*Updated summary:* After discussion with the authors, it does seem like there is a connection to proper SDE-based diffusion, but this connection is not very well explained. In particular, while their training objective isn't exactly a maximum-likelihood objective for the SDE itself or for the probability-flow ODE, it can be seen as a maximum-likelihood objective for a different, perturbed ODE flow model. And if the "sliding window" used during training is sufficiently small, the optimum of their objective will approximate the score function of the original SDE/ODE. I still feel that these connections aren't explained very well in the current revision of the paper.
strength_and_weaknesses: Strengths:
- Using a learned model to guide a differentiable simulator is a good idea, and using noise to train it seems reasonable as well.
- Their proposed model does well on a variety of experiments, and in particular does better than either directly learning everything with a neural network, or simply running the simulator backwards.

Weaknesses:
- ~~The paper gives very few details about how the model is actually trained. Many details are missing regarding the actual loss used, the distribution of samples in the training set, what is fixed and what is learned, etc. The authors briefly mention "bidirectional training" but it is never described what this actually means.~~
- ~~Based on my understanding of the training procedure, it is not actually trained via score matching, the typical way generative diffusion models are trained. This is why it is necessary to differentiate through the physics; I do not believe such differentiation would be necessary using an actual score-matching objective.~~
  - *After discussion with the authors, it does seem like their objective can be viewed as an approximation of score matching. However, this approximation seems to depend on the "sliding window" used during training being small, and this requirement isn't discussed very clearly in the paper (although there's some discussion in the appendix).*
- ~~The authors appear to be mis-using the relationship between the diffusion SDE and probability flow ODE. In particular, running the probability flow ODE should *not* be interpreted as producing a trajectory sample. The probability flow ODE is a continuous flow such that the marginal distribution at each time is correct, but any two points along a given flow path are not necessarily paths from the same trajectory. However, the authors discuss using the probability flow ODE to generate a "solution" trajectory, and appear to train their model so that this trajectory is close (in L2) distance to entire sampled trajectories. I similarly do not think it is appropriate to treat samples from the SDE as being drawn from a posterior, given that the training process does not seem to be based on KL divergence (or any other mechanism for fitting a sensible posterior over trajectories).~~
  - *After discussion with the authors, it seems like the L2 distance objective can be seen as a maximum-likelihood for a different probabilistic model (not clearly described in the paper right now, but only mentioned in the appendix). And for a small enough sliding window, training this second model would require learning the correct score function, which could then be used to draw samples from the reverse SDE despite their objective not being a KL divergence with the reverse SDE..*
- ~~Overall there are multiple choices that seem very questionable from the perspective of probabilistic generative modeling.~~
- After discussion with the authors, the main remaining weakness I see is that the connections to score matching are not very well explained, and that some details which are presented as minor computational efficiency improvements are actually necessary modeling choices that enable their model to approximate the true score function.
  - The training process actually uses a small window around parts of the trajectory. This is briefly mentioned as an implementation detail in the new "Training and Inference" section, but after reading the appendix and discussing this with the authors, it seems that the small window is actually necessary for their model to recover the true score function. If this window is not used, the learned "score" function might differ substantially from the true score, and the connection to the real SDE/ODE might break down.
  - The authors refer to their objective as a maximum-likelihood bound for the probability flow ODE, but after discussion it appears that the objective is actually a maximum-likelihood bound for a *different* probabilistic model, involving additional Gaussian noise perturbations (which the authors motivate as modeling the numerical inaccuracy of the ODE solver). This perturbed model is not explicitly described in the current submission, except for a partial description in the appendix.
clarity,_quality,_novelty_and_reproducibility: ## Clarity

I found the proposed training method to be very unclear. Although the authors spend a long time discussing the objective of denoising score matching, as far as I can tell this is not actually the objective they use. However, their exact objective does not appear to be explicitly stated. They do show a loss $\mathcal{L}((\hat{x}_t), (x_t)) = ((\hat{x}_t) - (x_t))^2$ but $t$ is never defined, the actual definition of $\hat{x}$ and it's initial conditions are never stated, and in general the notation is quite confusing (e.g. are these supposed to be vectors or scalars?). The "bidirectional training" procedure is also never formally described, and neither is the "SMDP-ODE+noise" variant discussed in section 4.1.

*Update: The implementation of their approach and the actual training method used has been clarified substantially. However, I still think that the probabilistic interpretation of their model and the connections to score matching are very unclear.*

## Quality

It is difficult to fully assess the quality of the proposed approach without a clear statement of the training procedure. However, based on my understanding, the objective appears not to be based on score matching at all, and it seems incorrect to refer to their technique as a score matching approach. In particular, the proposed objective seems to be based on L2 distances of sampled trajectories. This is not how score-matching objectives are trained; score matching models are trained to approximate the gradients of the data distribution (as the authors discuss in the background equation (3)). Minimizing the L2 difference between sampled trajectories and learned trajectories that use a score-based parameterization is not the same as learning the score, and I do not think that the learned "score" function will necessarily have anything to do with the gradients of the perturbed data distribution. (The authors state that their loss is "in line with previous work". But although it does resemble the differentiable physics loss used by Um et al., it does not actually seem to be line with the loss in Zhang & Chen, or any other diffusion model I am aware of.)

Similarly, although it is true that the probability flow ODE and the diffusion SDE have the same marginal probability densities, it is not accurate to treat paths computed under the ODE flow as trajectories of the diffusion SDE, or even as an "average" of candidate trajectories. I don't think that minimizing the L2 difference between the probabilistic-flow ODE path and a concrete trajectory sample has any probabilistic interpretation, and so it seems incorrect to refer to this as representing a posterior distribution, or to the SMDP-SDE drawing samples from this posterior distribution.

Most of the experimental results seem high quality from a perspective of evaluating the ability of their model to solve an inverse problem. However, there were additional confusing and seemingly incorrect uses of the diffusion SDE and score matching throughout. For instance, the "Diffusion" baseline doesn't appear to be trained using a diffusion model loss, but instead using their L2 norm loss (is this true?). In section 4.2, the authors introduce a number of heuristic changes which don't make sense under their diffusion process assumption in equation (5) or for a standard score-matching model, such as adding i.i.d. noise to simulation states (instead of Brownian noise) and "decoupling" the guiding function and physics operator. Additionally, they discuss multiple variants using the "score" terminology despite this not having a relationship to the gradient of the data distribution.

*Update: It seems that there is actually a way to interpret this as a score-matching approach (see my discussion with the authors below), but this isn't explained very well, so this is more of a clarity issue than a quality issue.*

## Novelty
Inverting a physical system by using a backward PDE combined with a learned correction term seems novel to me, although it's possible I am unaware of some recent work here. Inferring the initial state is in some ways similar to the setting of [Frerix et al. (2021)](http://proceedings.mlr.press/v139/frerix21a/frerix21a.pdf) but there are also many differences. One very recent diffusion-based approach that incorporates physical system knowledge is [Wu et al. (2022)](https://arxiv.org/abs/2209.00865v1), but the motivation is to add inductive biases to the generative process, not to invert a particular physical system.

The authors claim this is the first work that leverages the reverse-diffusion theorem to solve inverse problems, ~~but as noted above I'm not convinced this method is actually using the reverse-diffusion theorem in a correct way.~~ (I'm not aware of any other works that have done this either, though.)

## Reproducibility
I do not think there are sufficient details about the training algorithm to reproduce this method. (The authors state that they will make the source code available after acceptance but it's not clear how reproducible their experiments will be.)

*Update: I think enough details have been provided in the new revision to make it possible to reproduce this method now.*
summary_of_the_review: This paper has some interesting ideas for using learned ODEs to find the initial states of a physical system conditioned on a final state, and the method seems to work well empirically. However, I think in its current form the paper makes many unjustified and incorrect claims about the relationship between this model and SDE-based score-matching diffusion generative models. The paper also omits many important details about the training procedure, which seems substantially different than prior work with diffusion models.

~~As such I do not think the paper should be accepted in its current state. I would be willing to raise my score if the authors clearly explain what their actual training algorithm is and remove the misleading claims about the relationship to score matching and probabilistic inference (or, alternatively, if they can justify the sense in which their objective is equivalent to score matching and explain why my concerns are unfounded).~~

*Updated review:* The authors have added the missing details about the training procedure, which is now much better explained. They have also added a derivation which claims their objective is a probabilistic lower bound that is (indirectly) connected to score matching. I have some issues with the way this is presented in the paper, but after discussion with the authors (see below) I've been convinced that there is indeed a sense in which the actual objective used (for a sufficiently small sliding window) could be viewed as approximately doing score matching. However, this connection and the dependence on the window size are not very well explained in the paper, and there are still some misleading claims about maximum-likelihood training. I have raised my score from 3 to 5, since I no longer think there are critical problems, but still I'm hesitant to recommend acceptance due to these remaining clarity issues.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary: The goal of this paper is to  model the distribution of initial macroscopic states for a given end state of a physical system, for which the dynamics is given by a differentiable simulator. This is done by setting up a Neural ODE L2 training objective to learn the so-called Probability Flow ODE, and then sampling from the associated reverse time SDE. The method is demonstrated on a toy heat equation problem, a buoyancy-driven flow with obstacles and an isotropic turbulence problem.

Strengths: Incorporating a strong prior in the form of a differentiable simulator in score function learning appears to be a novel way to formulate inverse problems.  In the experiments, the proposed model improves over directly learning an unstructured neural network highlighting the role of priors, or simply running the simulator backwards which does not capture stochasticity.

Weaknesses:  The discussion around weaknesses in the review process ended up converging around clarity  --  the paper makes several claims about the relationship between this model and SDE-based score-matching diffusion generative models, omitting necessary details about the training procedure and its connections to approximate score matching  
 / max-likelihood training (e.g. the dependence on the window size are not very well explained in the paper).  The lack of sufficient rigor makes it harder to appreciate the mechanisms that allow the method to do well and reduces the confidence in reproducibility. 


justification_for_why_not_higher_score: Please see critical weaknesses listed above.
Decision: Reject