Venue: ICLR 2022 Conference

Reviews:

final_score: rating: null
review_content: Dear other reviewers and author, 

Thank authors for detailed response, I think one concern some of our reviewers have is around the basic understanding (significance, novelty and usefulness?) of this method. 

After the authors response, I think I still failed to understand the motivation. Using neural networks, a lot of deep inverse models tries to learn directly the mapping from y to x space, see this below paper where it benchmarked a wide range of deep inverse solutions, this sounds like the "Tandem" model with a actual physical model replacing the forward proxy.

Ren et al. "Benchmarking deep inverse models over time, and the neural-adjoint method", NeurIPS 2020

Also the numerical evidence is confusing to me:
1. Why did the Newton method and Adam converge to drastically different performance? I think if the optimizer is well tuned, adam should converge to as good as a result as the Newton method although using more iterations, since the loss surface is the same? (This suggested to me that the optimizers are not well tuned for comparison)

2. Wall clock time of the experients: Second order information is definitely useful in optimization and they converge in fewer iterations than first order ones, however, it is not the mainstream optimizers for a single reason, computational speed. The wall clock convergence speed of them (I assume) are usually much slower for the majority of the problems, otherwise everyone should be using Newton method for optimizing their neural networks?

The authors included the wall clock calculation in the last page of the appendix, which is apperciated. In "time-critical applications" like "deciding whether to store or discard event recordings at the LHC", I highly doubt the physical gradient can be calculated using similar time as Adam, which is done using Auto-grad.

Would like to hear more about what other reviewers think as well. Maybe I am completely off and did not understand major part of it.
review_title: More discussion needed...

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper looks at the (inverse) problem of finding the some initial state given a final state of a physical system. It proposes training neural networks to predict the initial state for a given final state. These networks are trained iteratively by using inverse solvers (based on domain knowledge) that provide physical "targets" that are closer to the desired initial state than the current prediction. 
main_review: This work is clearly presented and proposes an interesting idea: using inverse solvers to provide targets that iteratively improve the neural network predictions of the initial state.

The case n=1 used is probably the most interesting, because the inverse step is indeed simply providing a slightly improved target to supervise the training of the network, without the need of labeled data and without having to run a full inverse solve.

The experiments performed, though on fairly simple domains, demonstrate that the method works and is able to generalize at least in the given domains (ie interpolate), given that data is sampled randomly.

As with many of these types of applications of machine learning to physical inference, it is questionable if the benefit of the fully trained model achieved at the end compensates the cost of training it for a long time on a limited domain, while also having to provide an already existing inverse solver (or something capable of approximating this inverse). It would be good for the authors could discuss further what use cases they foresee for this type of model in practice, where the costs of training and formulating a known inverse could outweight simply using a traditional methods. For the cases where the inverse is actually known, it would be important to provide a comparison to simply providing the inverse optimization.

-------

After reading the authors response, I maintain my previous evaluation.
summary_of_the_review: Despite some of the comments above, I believe the conceptual contribution and the experiments presented in this paper are of enough interest and robustness to warrant its acceptance. I am rating it as marginally above the acceptance threshold for now, expecting the authors to improve and respond to the comments above.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an approach for learning neural network-based approximators for inverse problems in physical domains. The crux of this approach involves rewriting the training loss function to capture the difference between (a) the NN's estimate of the problem input and (b) the solution obtained when using this estimate as an initial point for an approximate inverse problem solver. This loss function is then optimized via gradient descent. This approach is demonstrated on several physical systems, and shown to avoid issues due to multi-modality (which supervised training schemes fall prey to), instability in the gradients through the physical process (faced in some settings by first-order training methods applied to the "standard" inverse problem), and issues such as expense of computation or ill-conditioning (faced in some settings by training method using e.g. inverse Hessian information).
main_review: Strengths:
* The idea the paper proposes is simple but powerful: That writing down a particular (intuitive) proxy L2 loss for the original inverse problem yields stable gradient updates with direct physical intuition in the physical output space.
* The empirical demonstrations are extensive, and demonstrate the superior performance of the proposed method in physical settings with different salient properties (e.g., multi-modality, large sensitivity, singularities, etc.) 

Weaknesses:
* The paper is written so as to significantly oversell the generality of the proposed framework, which in turn significantly impacts clarity. There are several major examples of this:
	* The title "physical gradients" and several phrases throughout (e.g. in the abstract: "We replace the gradient of the physical process by a new construct") strongly imply that a new method of taking gradients will be introduced. This is not the case. Instead, the original inverse problem is rewitten such at the gradients that "fall out" have physical intuition.
    * The formulations in Equations 4 and 5 depend on $\mathcal{P}_n^{-1}$, which in various places is described as being derived using domain knowledge. However, $\mathcal{P}_n^{-1}$ is just a single step of gradient descent, and while this is mentioned, it is not emphasized. Notably, all experiments use this "single step of gradient descent" interpretation, and the physical interpretation of the resultant gradient depends on this as well. This point should be made much clearer.
    * Relatedly, the abstract indicates that the training approach "combines higher-order optimization methods with machine learning techniques." To the best of my knowledge, this does not seem to be true.
* Certain claims made in the paper are not fully substantiated. For instance:
    * The claims in Figure 1(b) about the strengths of the present method vs. GD and Newton's method are not fully substantiated/described.
    * On page 4, it is noted that the method can be applied under one of three conditions. However, only condition (iii) is ever explicitly used (as far as I understand), and the rationale as to why either of the other two conditions is sufficient is never described.
* The main body of the paper is not self-contained, causing issues with clarity that are then exacerbated by typos. In particular, Appendix A.1 should definitely be in the main body of the paper in order to make the overall method/intuition understandable. (Also, the last paragraph of page 4 mentions that x_n can be taken as a constant - this is, importantly, not the case, but requires digging into Appendix A.1 in order to understand.)

Minor comments (not affecting my score):
* The paragraph on "inverse physics via domain knowledge" is hard to understand until after the entire paper has been read; its clarity could be improved.
* Above equation 3: The authors might consider using a different subscript than "sup." Presumably this is meant to stand for "supervised," but could be confused for "supremum" given the optimization setting.
* In the experiments, the descriptions of what methods are being compared could be made much crisper and clearer. In addition, interpretation of the legend in e.g. Figure 2 should be made more self-contained.
* In all graphs, legends should be moved such that the lines are visible.
* In Figure 3, it is not clear what the x-axis is. The caption of 3(a) presumably has a typo, as convergence curves are not shown.
summary_of_the_review: The method provided by this paper is simple, powerful, and physically intuitive. However, there are significant issues with overselling, substantiation of claims, and clarity that mean I cannot recommend acceptance of this paper in its current form.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The present paper proposes a novel ansatz for solving inverse problems by means of deep learning. Specifically, the loss attached to a specific observation is defined as the difference between the neural network prediction and the output of an approximate iterative solver initialized with the current neural network prediction. The gradient of such loss (for a mini-batch of observations) w.r.t. to the neural network parameters, coined physical gradient, can then be used to optimize the neural network by standard variants of gradient descent. Potential advantages of this approach over the classical (un-)supervised approaches include the following: (1) One circumvents the need for (possibly ill-conditioned) automatic differentiation of the forward operator, (2) the varying initialization of the iterative solver enables adaptive convergence to different solutions (in case of a non-injective forward operator) throughout the neural network training, and (3) a few iterations of the solver in each step might suffice to provide a useful loss for the neural network. Moreover, the paper also explores the use of (Quasi-)Newton methods.  
The functionality of the proposed method is backed by theoretical results and four numerical experiments.
main_review: The omnipresence of inverse problems asks for stable and efficient deep learning-based solvers and new methods are highly desired.
While the proposed usage of so-called physical gradients is based on a rather minor modification of the supervised approach, it could overcome several drawbacks of existing methods and thus seems to be a research direction well worth pursuing. 
Unfortunately, the present paper does not sufficiently motivate and justify the usage of physical gradients and has several other issues.


- **Motivation:**

    The proposed method employs an iterative solver in each optimization step in order to create the current targets for the neural network. Thus, it seems that the neural network cannot outperform the iterative solver except for perhaps averaging out noise or adapting the initialization. This raises the question, why one should prefer the neural network over the iterative solver?  Similar to the supervised approach, it might be beneficial to replace the iterative solver by a neural network, as this could lead to faster inference times in case of a computationally expensive solver and allow for sensitivity analysis / uncertainty estimation via the trained neural network. The former, however, would drastically slow down the neural network training when using physical gradients as the iterative solver needs to be applied in every optimization step and the latter is not explored in the paper. 


- **Empirical evidence:**

    Unfortunately, the numerical experiments do not showcase the potential strengths of the proposed method. While a crucial part of this method is the initialization of the iterative solver using the current network prediction, this seems to only be used in the first toy example. In the other examples, the difference between the use of physical gradients and the supervised approach is not clear. Moreover, the proposed method might also work with only a few iterations of the iterative solver in each step. However, for all but the last example, an explicit solution is used instead of an iterative solver and there is no numerical evidence of how the method depends on the accuracy of the iterative solver.  Finally, all experiments use an infinite stream of synthetically created (mostly noiseless) data, which is usually not given in practice. 


- **Further comments:**

    Further issues, comments, and suggestions for improvements can be found in the following:
     
    (i) The statement that physical gradients behave more smoothly, both in magnitude and direction, appears in Subsection 3.4 and in the abstract. However, apart from the toy example in Figure 7, it seems that there is not enough theoretical or empirical evidence for that statement. 
    
    (ii) In Equations (1) and (2), $x^*$ and $\theta_*$ belong to the $\operatorname{argmin}$ (which is in general a set of minimizers, given that the minimum is attained), rather than being equal to the $\min$.
    
    (iii) I guess that $\mathcal{P}^{-1}_{n^*}(y^* | x_0)$ should, in general, not be equal to $x^*$ as stated in the introduction of Subsection 2.1. It should probably only be an approximation of the latter.
    
    (iv) In my opinion the sentences "Note that the dimensions of ..." and the subsequent one, are rather confusing. What is meant by "dimensions" in this context, why is this phenomenon counter-intuitive, and how does this relate to the vanishing/exploding gradient problem in *deep* networks?
    
    (v) It seems that the derivative of the forward operator is missing in the update rules in Figure 1 and also the expression for the Hessian given in the caption should be derived in more detail.
    
    (vi) I am not aware of a generic definition of a "sensitive" function (as mentioned several times in the paper) and think that this should be properly defined.
    
    (vii) The statement that jointly optimizing all inverse problems reduces the likelihood that an individual solution gets stuck in a local minimum, should, if it holds true in this generality, be referenced or sufficiently motivated.
    
    (viii) In Algorithm 1, it is unclear what it means to condition the inverse solver on two values $x_{k-1}$ and $y_k$.
    
    (ix) The first example does not fit into the framework described in Equations (1) and (2) as now $\mathcal{P}$ is parametrized by $a$ and $y^*$ is fixed.
    
    (x) In the last example, $y^*$ seems to correspond to the tuple $(m_0,m_t)$ rather than only $m_t$.
    
    (xi) It is not clear, how Assumption 3 on page 15 relates to the universal approximation theorem (as stated below), which deals with uniform approximation of a continuous function on a compactum rather than convergence of gradient descent.
    
    (xii) The factor $1/2$ seems to be missing in the loss $L$ in Subsection B.2.
    
    (xiii) In Subsection B.4 there appears $x_0$ which seems not to be defined.
    
    (xiv) Typos: *an* generic inverse solver; *we* found that second-order optimizers
summary_of_the_review: The paper proposes an interesting adaptation to the supervised approach of solving inverse problems by means of deep learning. Unfortunately, the included experiments are not representative enough to show whether the promised benefits of this approach hold true. In addition, there are several issues regarding the presentation of the material.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposed a new update method for neural network solving inverse problem by incorporating the physical information as prior into the gradient computation step. It leads to faster convergence and better convergence performance.
main_review: Frankly speaking, I did not understand the main goal of this paper. I am happy to adjust my scores when the concerns are addressed and there is a high chance that I did not understand vital parts of this paper.

The main method or the goal of the paper is illustrated page 2 Section 2 with equations (1) and (2). The inverse problems are usually framed into two different frameworks:
1. Optimization framework where x* = min_x L(x). The goal is to minimize the loss function which is defined as the difference between the target y and F(x) (F is the forward physical process). This is consistent with the paper's equation (1). However, this is not what this physics gradient is meant for.
2. Mapping learning framework where we learn (usually using NN) either a probabilistic mapping or deterministic mapping from target y space to the control parameter space x. That is we learn Inv: Y -> X. This seems to be what the authors are trying to illustrate from Equation (2). However I struggle to understand the notion of "This training scheme is unsupervised, i.e. no labels are required over the training data Y". How can the mapping paradigm be learned without any training data pairs (x, y) which comes from the actual physical process? Do the authors mean the P(NN(y*|theta)) can somehow skip being evaluated by the actual physical process? If not, which means the underlying physical process is called during training, then we have the ground truth pair of (x,y) during training, which means this counts as an supervised training scheme.

Unless the authors are referring to some more advanced techniques like Physics informed neural network where the PDE equation prior is hardcoded in the loss, I really can't see why this is a unsupervised task.

Secondly, I didn't quite understand the assumption of having an inverse problem solver P^{-1} in the first place. If we have a numerical inverse problem solver, why do we need to use a neural network to solve the inverse problem anyway?  The assumption of having access to the forward numerical process P is ok, but the access to an oracle P^{-1} really made me struggle to understand. 

This is the biggest concern I have, this method proposes to solve inverse problems using neural networks, but it needs access to a numerical inverse solver from domain knowledge, which seems to beats the whole purpose of using neural networks.

Finally, the empirical results. Given the above questions raised a mere misunderstanding on my part, all the "convergence speed comparison" is done on the unit of "iteration". However, I would assume that the numerical inverse solver P^{-1} would take a much longer time in wall clock sense than a neural network update. Therefore without this critical information about the actual running time of the inverse solver, the mere comparison between performance upon iterations is meaningless as the total time of Inv. Phys gradient can well exceed the Adam and even Adam + Newton.

My apology if I missed crucial parts of this paper, but I did spend some time on it and still struggle to understand things I mentioned above. If the above concerns are ultimately addressed by the rebuttal, the clarity of the delivery would need a lot of improvement.
summary_of_the_review: The basis for problem set up is unclear to me in general, therefore the contributions and results are completely compromised by that.
review_title: null

Meta-review: The paper introduces a method to solve inverse problems: given y, find x such that P(x)=y, for a given physical simulator P. A standard approach is to learn a neural net such that the inverse x=NN(y;\theta). The authors state that this is problematic because it is difficult to take "higher order" gradient information into consideration when using this standard approach. The method assumes that there is an approximate inverse solver inv(P) and discusses an alternative "Physical Gradient" objective that can incorporate knowledge of an approximate inv(P) and a neural network. The experiments are good though comparing performance on an iteration basis is not always fair since an iteration of the PG method can be much more expensive than standard approaches.

The biggest issue that reviewers had was the clarity of the presentation. The authors have made a reasonable attempt to correct this, but I'm inclined to agree with the general reviewer sentiment that the presentation is still not at the required level. I agree that there are many things that are not clear, including the confusing discussion in section 2.1 about how the method takes higher order information into consideration. It only becomes partially transparent later in the experiments what is meant by higher order information.

Overall, I feel this is the basis of a potentially valuable contribution but that the current presentation is quite confusing. As mentioned by others, I would also suggest to find a different name since Physical Gradient is also rather misleading. 

The following points were not part of the review process and I do not base the final decision on them, but the authors may want to consider the following: 

I believe there is also an error in the basic approach, or at least an approximation is made which is not explained. The error is that the approximate inv(P) depends also on the parameter \theta (since this is used to initialise inv(P)). This dependency is not taken into consideration in the paper. For example, in theorem 1 in appendix A, the calculation of the gradient dM/d\theta is incorrect since the authors assume that inv(P) is independent of \theta, which it is not (since the preconditioner value depends on \theta). If we do take this into account, we would need to know the derivative of inv(P|x) with respect to the preconditioner x. This dependency would alter the gradient, potentially considerably. The gradient in figure 2 for the PG is also incorrect. One may of course simply say that the paper discounts this correction term in order to retain tractability; however, this would need to be stated as an approximation.
Decision: Reject