Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents an adversarial approach that overcomes this limitation, called competitive PINNs (CPINNs). CPINNs train a discriminator that is rewarded for predicting mistakes the PINN makes. The discriminator and PINN participate in a zero-sum game with the exact PDE solution as an optimal strategy. This approach avoids squaring the large condition numbers of PDE discretizations, which is the likely reason for failures of previous attempts to decrease PINN errors even on benign problems. The numerical results show the accuracy. 


strength_and_weaknesses: Strength:  By introducing another neural network as the point weight function and following with a min-max optimization, the solution of the PDEs can be more accurate. 

Weakness: Introducing a new neural network besides the solution neural network means much more complex for training and becomes much tricker on different tasks. And the weakness is summarized as follows.
1. there is one similar paper in 2020 as McClenny, Levi, and Ulisses Braga-Neto. "Self-adaptive physics-informed neural networks using a soft attention mechanism." arXiv preprint arXiv:2009.04544 (2020). In that work, they use mask functions for the point weight, and there is no significant difference using a neural network in this work, and the min-max procedure is the same; thus, the novelty is not much. The story is under the structure of the games but still can not change the reality of using a neural network for the point weight adversarial update. No intrinsic difference.
2. The analysis in section 2.3 is not very firm and too simple, sometimes naive, and the analysis of the neural network is just for a linear mapping which should be at least a two-layer neural network if you really want to explain something. As far as I know, in traditional numerical PDEs, the linear combination of the basis is, of course, a very common and efficient way. And it works well even using the vanilla schemes, such as the finite element method. Further, what really makes the so-called neural PDE different is the approximation by nonlinear neural networks, and it is far not enough to use one linear combination to analyze the neural PDE. There is plenty of work for analyzing PINNs, a direct search may help your analysis.
3. The results of the experiments are good but also tricky as a new neural network besides the solution neural network is introduced. More neural networks with a min-max optimization will increase the difficulties in training, and it is already complex compared with the traditional numerical solvers. If you can prove in high dimensional problem, the method still achieves good performance, just like Bao et al. in the weak adversarial neural network (Zang, Yaohua, et al. "Weak adversarial networks for high-dimensional partial differential equations." Journal of Computational Physics 411 (2020): 109409.), it is much better. Further, in the experiments, such as nearly all of the l2 error figures, the L2 relative error curve with respect to the iteration is not stable, but the experiments are just stopped at best. See Figure 4; there is still a very large oscillation. And Figure 5 has a big shock and etc. Also, it is hard to determine which one enhanced the performance, the optimizer or the structure, see Table 1. 
4. The terms are a little confusing for those who are not familiar with neural PDEs and may lead misunderstandings. The work is very direct with adversarial updating point weight in the area of Neural PDE, but the terms, such as bets, game, etc., will confuse the readers. A direct statement would make things much easier.

clarity,_quality,_novelty_and_reproducibility: Clarity: Not clear. The work is very direct with adversarial updating point weight in the area of Neural PDE, but the terms, such as bets, game, etc., will confuse the readers. 

Quality: Good. But the stop iteration is not natural. See #3 above for the detail.

Novelty: Weak. See #1 above.

Reproducibility: Good. The authors provide the code, but the method seems hard to use in other cases for requiring careful training.


summary_of_the_review: Based on the weakness, I reject the paper.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduces a new model called CPINN, which is an agent based approach to the machine learning solution of PDEs. CPINN is an adversarial approach where the discriminator and PINN, the existing approach to solve PDEs using neural networks, plays a game in order to improve the solution precision of PDEs. The authors motivate their work by the fact that the existing PINNs approaches fail to generate precise solution. The authors show that the proposed model, CPINN, is well-behaved compared to the original PINN model, from the optimization/training perspective. The authors support their claim using multiple PDE problems.
strength_and_weaknesses: Strengths:
1-	Novelty 
2-	Well-written
3-	Experimental results

Weakness: 
Cannot think of any

clarity,_quality,_novelty_and_reproducibility: The paper is very well-written and clear. The idea is novel, to the best of my knowledge. I cannot comment on the reproducibility as I am not sure if enough details are provided for that end.
summary_of_the_review: I enjoyed reading this paper. Physics informed machine learning is a very important and hot problem and still in its infancy. The authors efforts in improving the performance of existing models for this problem needs to be highly acknowledged and rewarded. The authors were able to motivate their approach and support that using experimental study. Overall, I believe this paper has a high quality and ready for publication. I should only add that I am not fully familiar with the state of the arts in this topic and assumed that the authors reflected it well in the paper.  

One minor issue: I am curious to see similar plots to that of Figure 1 for other datasets.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper produces more accurate physics-informed neural networks (PINNs) by including adversarial training through a discriminator which is rewarded for predicting PINN errors.
strength_and_weaknesses: Strengths:
    • Claims are supported by reported results.
    • Concept is sound and sensible.
    • Making PINNs more accurate has good impact for the field.
    • Paper is well-written and easy to read and navigate.
    • Mathematics is well explained.
    • Citations are thorough.
    • Tradeoffs between this method and conventional PINNs are discussed.

Weaknesses:
    • Figures describing the method are not included, and they would make the method easier to understand in this case.
clarity,_quality,_novelty_and_reproducibility: The work is well-written and easy to follow. Bolded subheadings are used to create small sections, allowing for quick navigation of the paper. 

The experimental quality is quite high, testing on a variety of problems with good results.

The novelty of the work is middling but non-trivial, with the addition of a discriminator network to reduce error in the PINN. 

Reproducibility is possible with hyperparameters and implementational details available in the appendix.

summary_of_the_review: This paper seems to be a solid step forward in the field of physics-based neural networks – a field that has high impact on a number of domains. The paper is generally well-written. The novelty of the work is middling (maybe too straightforward because of the recent popularity of GANs) but non-trivial.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper introduce competitive physics-informed networks where two neural networks solve a partial differential equation by playing a zero-sum game. The formulation is built using a weak form. The authors claim that this weak form have smaller condition number. 
strength_and_weaknesses: Strength: The experiments look strong, and the writing is super clear to understand.

Weakness: Several claims are hand waving, and the reviewer is not convinced. I think if the paper is accepted at this stage, it will make confusion in the community. 

**Regards Experiment** The reviewer's first major concern is the relationship of this paper with [1] which also uses the weak formulation to train a PDE solution.  I think an experiment comparing these works is essential (using the same optimization algorithm).

The paper only includes results comparison with respect to the number of iterations. Comparison with respect to the computation time is also needed. For CPINN introduces a new NN and CGD have a matrix inverse. Further computation cost have been included.

**Regards Mni-max** min-max optimization is hard. The complexity lower bound proposed of minimax optimization has a term K_x K_y [2]. I guess using min-max has the same computational complexity as the original formulation.

To reduce the condition number of laplace squares, we can still use PINN to do this by considering
min_{u_1,u} ||u_1-\nabla u||^2+||f-\nabla dot u_1||^2
to solve Delta u = f. I don't think the minimax formulation is essential.

At the same time, the paper uses ADAM, which is an adaptive method. The adaptive methods can cancel the condition number. I'm confused by this mismatch of the experiment with the theory claimed. 

**Regards the Squared residuals.** Changing the loss into the Sobolev norm also introduces a further matrix A. But [3,4] showed this would only accelerate the training both in experiments and theory for the machine learning algorithms. This paper have considered the theory as [4] includes a machine learning model into the learning objective. Can the authors comment on this?

**Related work [5]** The reviewer thinks the paper is still valuable. [5] proposed a similar trick from the optimization community and figured out the improvement of this method.  I encourage the authors to think about combining the theory in [4] and [5] to figure out the true reason this method has an acceleration. 

[1] Yaohua Zang, Gang Bao, Xiaojing Ye, and Haomin Zhou. Weak adversarial networks for high-dimensional partial differential equations. Journal of Computational Physics, 411:109409, 2020

[2] Zhang J, Hong M, Zhang S. On lower iteration complexity bounds for the convex-concave saddle point problems. Mathematical Programming, 2022, 194(1): 901-935.

[3] Son H, Jang J W, Han W J, et al. Sobolev training for the neural network solutions of pdes[J]. arXiv preprint arXiv:2101.08932, 2021.

[4] Lu Y, Blanchet J, Ying L. Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent[J]. arXiv preprint arXiv:2205.07331, 2022.

[5] https://proceedings.neurips.cc/paper/2018/file/08048a9c5630ccb67789a198f35d30ec-Paper.pdf 


===

The author's response convinced me this is a worth pursuing direction, I've raised my score. 

clarity,_quality,_novelty_and_reproducibility: The writing of this paper is super great. I enjoyed reading this paper. 
summary_of_the_review: The experiments look strong, and the writing is super clear to understand. Several claims are hand waving, and the reviewer is not convinced. I think if the paper is accepted at this stage, it will make confusion in the community. 

I still consider this method have made a huge contribution to the PINN family. I suggest authors to investigate paper [5] in detail to see the benefit of this method.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper introduces competitive physics-informed networks where two neural networks solve a partial differential equation by playing a zero-sum game. The formulation is built using a weak form. The authors claim that this weak form have smaller condition number.

The paper originally received diverse reviews, and we had a dedicated discussion among all the reviewers. The author response also helped a lot. Eventually, as a consensus, we believe this paper has more strengths than weaknesses, and it would be good to accept it to the conference.

justification_for_why_not_higher_score: There are still concerns on the technical novelty - the augment to PINN is quite straightforward, although the execution of the idea looks good.
justification_for_why_not_lower_score: After the discussion, the reviewers believe the ICLR audience should be interested in seeing this paper.
Decision: Accept: poster