Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors provide each bound and its connetions on PINN, DeepONet, FNO, physics-informed DeepONet, and physics-informed FNO. These general connections are also applied to show that the neural network representation can overcome the curse of dimensionality.
strengths_and_weaknesses: ## Strengths
* While various studies measured the approximation error for a specific PDE for one method among PINN, FNO, and DeepONet, this paper measured the error by connecting all methods at once for the first time.

## Weaknesses
* Theorem 3.4, Theorem 3.8, and Theorem 3.9 talked about the existence of nn that reduces the PDE residual. I can't guarantee it's a good match. These theorems only talk about the possibility of sufficiently reducing the loss. Furthermore, I wonder if the boundary condition or initial condition is not considered. Without these conditions, I am not sure if reducing $\|\mathcal{L}(\mathcal{G}_\theta)\|$ enough will lead to the convergence to the exact solution.
* [9] as well as [*] is a study showing that the curse of dimension can be solved when the solution of PDE is approximated by nn. Regarding these papers, it is necessary to explain the differences between Theorem 4.2 and Theorem 4.4 described in Section 4.1.




[*] Marwah, Tanya, Zachary Lipton, and Andrej Risteski. "Parametric complexity bounds for approximating PDEs with neural networks." Advances in Neural Information Processing Systems 34 (2021): 15044-15055.
questions: ## Questions
* The discussion says that assumption 3.3 is a natural assumption for various PDEs, but I wonder if this is correct. For example, I wonder if it is an assumption that can be easily made for the examples used in Burgers' and other operator learning papers.
* There is confusion in the notation of $u_\theta$, $\mathcal{G}_\theta(v)$ and $\mathcal{u}^\varepsilon$. Assumption3.1 and Assumption3.3 are assumptions related to $\mathcal{u}^\varepsilon$ and $\mathcal{G}_\theta(v)$, but Theorem 3.4 uses these assumptions to create $u_\theta$ and $\mathcal{L}(u_\theta)$. In case of PINN, $u_\theta$ is learned while $v$ is fixed.
* For the term $\|u_\theta-u\|_{L^2(\partial([0,T]\times D)}$ on the left side from the trace inequality in (3.4),  I wonder if this term affects the fact that there exists a nn that can sufficiently reduce the PINN loss. 

## Minor
* Assumption 3.5. there is no $p^*$ is not in inequality (3.5)
* p.5 194 What is $L^2_N$?

limitations: The authors noted their limitations and future works well in the conclusion section.
review_title: null

final_score: rating: 8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a framework for analyzing the approximation  error of PINNs and operator learning. The framework was applied to obtain  error bounds for physics-informed operator learning methods for several class of PDEs. The error bounds show that dimension-independent convergence rate can be obtained if the solution is smooth enough. 
strengths_and_weaknesses:  - Strengths. This is a nice work on the error analysis of PINNs and operator learning, and to the best of my knowledge the first theoretical work on physics-informed operator networks. The main results are interesting, nontrivial and important for understanding the performance of operator learning in PDEs. The paper is very well written and easy to read. 
      
- Weaknesses. The paper is strong enough to be accepted. I do not have much to comment on the weakness, except minor comments on the few main results and typos. See the section of Questions for more details. 
questions: - The current results only prove the existence of some neural works achieving certain desired error accuracy. Can the authors comment on how the neural networks can be obtained? Especially, if one consider the empirical risk minimization approach, how does the generalization error scales?

- Theorem 4.6 concerns the approximation error of physical informed operator networks. The result is stated for the loss function $\|\mathcal{L}(\mathcal{G}_\theta)\|_{L^2}$. Can the author elaborate on whether this result implies an error bound for the operator itself, e.g. a bound for $\|\mathcal{G} -\mathcal{G}_{\theta}\|$? In addition, the Gaussian measure $\mu$ of the input functions requires subexponential decay in the KL expansion coefficients. Can this be relaxed?

- Is $p^\ast$ used in the statement of Assumption 3.5? If not, can the authors define it outside the Assumption?
    
- In Assumption 3.1, `` ...  there exist ... " there exist should be `` ... there exists ... "

- In Theorem B.5 of Appendix, are the indices $i$ and $j$ the same in equation (B.15)?

limitations: The authors adequately addressed the limitations in the discussion section. I am not ware of any potential negative societal impact of their work.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper proposes a framework to derive some approximation error bounds for neural networks to approximate the solution operators of initial value problems.
strengths_and_weaknesses: Strength:
Literature overview on recent operator learning such as DeepONets and Fourier Neural Operators is well written.

On the expressive power of neural networks for operators, I would suggest the following paper as an important previous work:

[GS19] Guss and Salakhutdinov, On Universal Approximation by Neural Networks with Uniform Guarantees on Approximation of Infinite Dimensional Maps, arxiv preprint:1910.01545

Weakness:
I found that the theoretical part is sloppy. First, the function classes are often lacking. For example, what is the differential operator L_a in Equation 2.1? Is G a bounded operator? In what norm? Secondly, many assumptions are incompletely stated without mentioning sufficient conditions when those are strictly satisfied. For example, in Assumption 3.1, both U^\epsilon and G are not defined. Dependence on a is unclear. Consequently, the main theorems and arguments are hard to read for mathematicians.

=== After Discussion with Authors
Theoretical details are clarified, and now I recognize the contributions clearer. So I raised my rating.
questions: Please refer to the previous section.
limitations: I suggest the authors to clarify the function classes of L_a, G, and U.
review_title: null

Meta-review: metareview: The paper studies approximation error bound for physics informed neural networks and operator learning. The result is technical sound and  useful for practical applications. The authors also adequately addressed concerns by the referees. The meta-reviewer recommends acceptance of the paper. 
confidence: Certain
Decision: Accept