Original: M ULTI - SCALE P HYSICAL R EPRESENTATIONS FOR A PPROXIMATING PDE S OLUTIONS WITH G RAPH N EURAL O PERATORS
Revised: M ULTI - SCALE P HYSICAL R EPRESENTATIONS FOR A PPROXIMATING PDE S OLUTIONS WITH G RAPH N EURAL O PERATORS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Representing physical signals at different scales is among the most challenging problems in engineering.
Revised: Representing physical signals at different scales is among the most challenging problems in engineering.

Original: Several multi-scale modeling tools have been developed to describe physical systems governed by Partial Differential Equations (PDEs).
Revised: Several multi-scale modeling tools have been developed to describe physical systems governed by Partial Differential Equations (PDEs).

Original: These tools are at the crossroad of principled physical models and numerical schema.
Revised: These tools are at the crossroad of principled physical models and numerical schema.

Original: Recently, data-driven models have been introduced to speed-up the approximation of PDE solutions compared to numerical solvers.
Revised: Recently data-driven models have been introduced to speed-up the approximation of PDE solutions compared to numerical solvers.

Original: Among these recent data-driven methods, neural integral operators are a class that learn a mapping between function spaces.
Revised: Among these recent data-driven methods neural integral operators are a class that learn a mapping between function spaces.

Original: These functions are discretized on graphs (meshes) which are appropriate for modeling interactions in physical phenomena.
Revised: These functions are discretized on graphs (meshes) which are appropriate for modeling interactions in physical phenomena.

Original: In this work, we study three multi-resolution schema with integral kernel operators that can be approximated with Message Passing Graph Neural Networks (MPGNNs).
Revised: In this work we study three multi-resolution schema with integral kernel operators that can be approximated with Message Passing Graph Neural Networks (MPGNNs)

Original: To validate our study, we make extensive MPGNNs experiments with well-chosen metrics considering steady and unsteady PDEs.
Revised: To validate our study we make extensive MPGNNs experiments with well-chosen metrics considering steady and unsteady PDEs

Original:  I NTRODUCTION AND MOTIVATION
Revised:  I NTRODUCTION AND MOTIVATION

Original: Principled modeling of physical phenomena gives rigorous and interpretable mathematical models, e.g. Differential Equations (DEs).
Revised: Principled modeling of physical phenomena gives rigorous and interpretable mathematical models, e.g. Differential Equations (DEs).

Original: However, solving these equations analytically is impossible in most practical cases.
Revised: However, solving these equations analytically is impossible in most practical cases.

Original: To circumvent that, one could seek for an approximated solution through numerical analysis.
Revised: To circumvent that, one could seek for an approximated solution through numerical analysis.

Original: When the phenomenon involves information and energy exchange in different range, methods with multi-scale modeling, e.g. multi-grid (multi-resolution) methods, are proposed for solving DEs.
Revised: When the phenomenon involves information and energy exchange in different ranges, methods with multi-scale modeling, e.g. multi-grid (multi-resolution) methods, are proposed for solving DEs.

Original: Interactions at different scales are modeled with a pyramidal discretization (Bergot & Duruﬂ´e, 2013; O’Malley et al., 2018).
Revised: Interactions at different scales are modeled with a pyramidal discretization (Bergot & Duruﬂ´e, 2013; O’Malley et al., 2018).

Original: With the multi-scale modeling, the solvers converge more quickly than single-scale methods (Lie et al., 2017; Passieux et al., 2010).
Revised: With the multi-scale modeling, the solvers converge more quickly than single-scale methods (Lie et al., 2017; Passieux et al., 2010).

Original: In Deep Learning (DL), many methods have been proposed for approximating PDE solutions on a regular grid at a single scale (Um et al., 2020; Thuerey et al., 2020).
Revised: In Deep Learning (DL), many methods have been proposed for approximating PDE solutions on a regular grid at a single scale (Um et al., 2020; Thuerey et al., 2020).

Original: However, in real world applications, the domain of PDEs is often discretized on meshes (represented as Euclidean graphs), where the nodes and the edges represent respectively the physical states and their interaction.
Revised: However, in real world applications, the domain of PDEs is often discretized on meshes (represented by Euclidean graphs where vertices are points in an Euclidean domain and the edges with the distance between those points), where the nodes and the edges represent respectively the physical states and their interaction.

Original: In this case, we use Graph Neural Networks (GNNs) instead, e.g. Pfaff et al.
Revised: In this case, we use Graph Neural Networks (GNNs) instead, e.g. Pfaff et al.

Original: ; Xu et al.
Revised: ; Xu et al.

Original: Some methods use U-net (Ronneberger et al., 2015), e.g. Wandel et al.
Revised: Some methods use U-net (Ronneberger et al., 2015), e.g. Wandel et al.

Original: (2021), to enable longrange interactions on regular grid data.
Revised: (2021), to enable long-range interactions on regular grid data.

Original: Recently, Multipole Graph Neural Operator (MGNO, Li et al., 2020a) introduces a new graph-based method in this category.
Revised: Recently, Multipole Graph Neural Operator (MGNO, Li et al., 2020a) introduces a new graph-based method in this category.

Original: It learns an operator mapping between two function spaces by a MPGNN with a multilevel graph.
Revised: It learns an operator mapping between two function spaces by a MPGNN with a multilevel graph.

Original: However, Li et al.
Revised: However, Li et al.

Original: (2020a) only focus on reducing computation cost of long-range correlation, inspired by fast multipole methods.
Revised: (2020a) only focus on reducing computation cost of long-range correlation, inspired by fast multipole methods.

Original: In this work, we explore new ways to extend the multi-scale modeling capacity with neural networks.
Revised: In this work, we explore new ways to extend the multi-scale modeling capacity with neural networks.

Original: We would like to shed light on different numerical schema and understand the reason for the choice of multi-scale schema from the lens of DL.
Revised: We would like to shed light on different numerical schema and understand the reason for the choice of multi-scale schema from the lens of DL.

Original: We observe in practice that the multi-scale DL models share a similar structure with some of the numerical schemas.
Revised: We observe in practice that the multi-scale DL models share a similar structure with some of the numerical schemes .

Original: For example, both composed of a straight-through downscaling and upscaling process, U-net shares a very similar structure with Vcycle schema in multi-scale numerical analysis (Jaysaval et al., 2016).
Revised: For example, both composed of a straight-through downscaling and upscaling process, U-net shares a very similar structure with V-cycle schema in multi-scale numerical analysis (Jaysaval et al., 2016).

Original: We then draw inspiration from discretized multi-resolution schema (Jaysaval et al., 2016) such as W-cycle and F-cycle to propose new architectures.[Figure] Iteration tt Iteration t +1 t
Revised: We then draw inspiration from discretized multi-resolution schema (Jaysaval et al., 2016) such as W-cycle and F-cycle to propose new architectures.

Original: [Figure]

Original: Iteration tt Iteration t +1 t + [Figure] In this paper, we propose new multi-scale DL architectures, based on the original MGNO, for learning representation of multi-scale physical signals by approximating the functional spaces of PDEs.
Revised: In this paper, we propose new multi-scale DL architectures, based on the original MGNO, for learning representation of multi-scale physical signals by approximating the functional spaces of PDEs.

Original: They are tested with steady and unsteady physical systems.
Revised: They are tested with steady and unsteady physical systems.

Original: This opens perspectives to rethink the architecture design for multi-scale problems and help practitioners using U-net to include these numerical schema variants in their study.
Revised: This opens perspectives to rethink the architecture design for multi-scale problems and help practitioners using U-net to include these numerical schema variants in their study.

Original: To the best of our knowledge, this is the ﬁrst work to explore new architecture designs for multi-scale problems within the Machine Learning (ML) community.
Revised: To the best of our knowledge, this is the ﬁrst work to explore new architecture designs for multi-scale problems within the Machine Learning (ML) community.

Original: We organize our paper as follows: in section 2, we describe formally Graph Neural Operators and the multi-resolution schema.
Revised: We organize our paper as follows: in section 2, we describe formally Graph Neural Operators and the multi-resolution schema.

Original: In section 3, we present our evaluation protocol and ablation study.
Revised: In section 3, we present our evaluation protocol and ablation study.

Original: Finally, in section 4, we conclude our work.
Revised: Finally, in section 4, we conclude our work.

Original:  N EURAL O PERATOR AND ITS GRAPH INSTANTIATIONS
Revised:  N EURAL O PERATOR AND ITS GRAPH INSTANTIATIONS

Original: Neural Operator (Kovachki et al., 2021). It aims at learning a map from a function deﬁned over a domain (typically Euclidean space) to another with parameterized models, especially neural networks (NNs).
Revised: Neural Operator (Kovachki et al., 2021) aims at learning a map from a function deﬁned over a domain (typically Euclidean space) to another with parameterized models, especially neural networks (NNs).

Original: The objective is to learn a map G θ : a ∈ F (cid:55)→ u ∈ F (cid:48) , where F , F (cid:48) are two function spaces deﬁned on a domain D .
Revised: The objective is to learn a map G θ : a ∈ F (cid:55)→ u ∈ F (cid:48) , where F , F (cid:48) are two function spaces deﬁned on a domain D .

Original: At each position in the domain x ∈ D ,with the input function a = v 0 , the solution u = v T is given by iteratively applying the following  transformation [Equation] ∈
Revised: At each position in the domain x ∈ D , the following iterative transformation is applied:

Original: [Equation]
Revised: [Equation]

Original: where θ = { W, b, φ } are parameters.
Revised: where θ = { W, b, φ } are parameters.

Original: W, b are local afﬁne transformation of v t ( x ) at each location x .
Revised: W, b are local afﬁne transformation of v t ( x ) at each location x .

Original: The parameterized kernel κ φ integrates v t ( x (cid:48) ) over all x (cid:48) ∈ D , with x (cid:48) ∼ ν
Revised: The parameterized kernel κ φ integrates v t ( x (cid:48) ) over all x (cid:48) ∈ D , with x (cid:48) ∼ ν

Original: and ν is a (probabilistic) measure over D .
Revised: and ν is a (probabilistic) measure over D .

Original: σ is a nonlinear Lipschitz activation function.
Revised: σ is a nonlinear Lipschitz activation function.

Revised: The iteration starts from t = 0 with the input function a = v 0 , the solution is the function at ﬁnal iteration T , i.e. u = v T .

Original: GNO and MGNO (Li et al., 2020b;a).
Revised: GNO and MGNO (Li et al., 2020b;a).

Original: Given that integrating over the whole domain D is intractable, one possible simpliﬁcation is to integrate only in a subdomain around x , i.e. s ( x ) ⊂ D .
Revised: Given that integrating over the whole domain D is intractable, one possible simpliﬁcation is to integrate only in a subdomain around x , i.e. s ( x ) ⊂ D .

Original: By limiting this subdomain to some i.i.d.
Revised: By limiting this subdomain to some i.i.d.

Original: sampled neighbors x [Equation]
Revised: sampled neighbors x (cid:48) ∈

Original: ∈ N ( x )  , we obtain:
Revised: N ( x ) ⊂ s ( x ) , we obtain:

Original: [Equation]
Revised: [Equation]

Original: which can be implemented with a message passing graph neural network, called GNO.
Revised: which can be implemented with a message passing graph neural network, called GNO.

Original: The graph isartiﬁcially constructed to connect the point at position x to its neighbors x (cid:48)such that ∀ x (cid:48) ∈
Revised: The graph is constructed by connecting the point at position x to its neighbors x (cid:48)

Original:  N ( x ) , (cid:107) x − x (cid:48) (cid:107) ≤ r .
Revised: such that ∀ x (cid:48) ∈ N ( x ) , (cid:107) x − x (cid:48) (cid:107) ≤ r .

Original: Here |N ( x ) | stands for the total number of the neighbors of x .
Revised: Here |N ( x ) | stands for the total number of the neighbors of x .

Original: However, when modeling long-range interactions is necessary, a large s ( x ) should be considered in GNO, which comes with a huge computational cost.
Revised: However, when modeling long-range interactions is necessary, a large s ( x ) should be considered in GNO, which comes with an expensive computational cost.

Original: To overcome this problem, MGNO considers the following multi-scale kernel matrix decomposition of the graph kernel:
Revised: To overcome this problem, MGNO considers the following multiscale kernel matrix decomposition of the graph kernel:

Original: [Equation]
Revised: [Equation]

Original: where K l,l (cid:48) is the kernel from scale l to l (cid:48) , the ﬁnest scale is 1 .
Revised: where K l,l (cid:48) is the kernel from scale l to l (cid:48) , the ﬁnest scale is 1 .

Original: To implement this multi-scale kernel, several architectural designs are proposed in the following section.
Revised: To implement this multi-scale kernel, several architectural designs are proposed in the following section.

Original: Multi-scale kernel implementations.
Revised: Multi-scale kernel implementations.

Original: We present in this section, three architectures for multiscale kernel implementation: V-MGNO, F-MGNO, and W-MGNO.
Revised: We present in this section, three architectures for multiscale kernel implementation: V-MGNO, F-MGNO, and W-MGNO.

Original: V-MGNO is proposed in the original paper (Li et al., 2020a).
Revised: V-MGNO is proposed in the original paper (Li et al., 2020a).

Original: The latter two architectures are inspired by multi-resolution methods (Jaysaval et al., 2016).
Revised: F-MGNO, and W-MGNO are inspired by multi-resolution methods (Jaysaval et al., 2016).

Original: The V-, F-, and W-MGNO are iterative processes and have in common three types of kernels as shown in Figure 1: downscale, intra-scale, and upscale kernels.
Revised: The V-, F-, and W-MGNO are iterative processes and have in common three types of kernels as shown in Figure 1: downscale, intra-scale, and upscale kernels.

Original: The input information is propagated in the multi-scale graph by a cascade of down-scale contraction, intrascale transformation, and up-scale expansion.
Revised: The input information is propagated in the multi-scale graph by a cascade of down-scale contraction, intrascale transformation, and up-scale expansion.

Revised: [Figure]

Revised: Iteration tt

Revised: Iteration t +1 t

Revised: [Figure]

Original:  Metrics.
Revised: Iteration tt Iteration t +1 t + [Figure] Metrics.

Original: The goal here is to empirically study the performances of these multi-resolution architectures and their learning stability.
Revised: The goal here is to empirically study the performances of these multi-resolution architectures and their learning stability.

Original: We deﬁne the following metrics to compare these multi-scale architectures:• Number of scales : indicates the number of scales from the ﬁnest to the coarsest one.
Revised: We deﬁne the following metrics to compare these multi-scale architectures: Number of scales : indicates the number of scales from the ﬁnest to the coarsest one

Original: The choice of the ﬁnest and the coarsest scales depends on the discretization schema, and also on the cutoff energy, as well as the computational budget.
Revised: The choice of the ﬁnest and the coarsest scales depends on the discretization schema, and also on the cutoff energy as well as the computational budget.

Original: • Intra-cycle Kernel Sharing : indicates whether the kernels are shared inside eachiteration.
Revised: • Intra-cycle Kernel Sharing : indicates whether the kernels are shared inside each

Revised: iteration

Original: It is because F- and W-MGNO may have several kernels for the same downsampling or upsampling action, e.g. between the coarsest two scales, W-MGNO (shown in Figure 1) need toseveral downscaling (red arrows) or upscaling (purple arrows).
Revised: This is because F- and W-MGNO may have several kernels for the same downsampling or upsampling action, e.g. between the coarsest two scales, W-MGNO (shown in Figure 1) need to

Original:  • Iteration Kernel Sharing : indicates if the kernels are shared across all iterations.
Revised: several downscaling (red arrows or upscaling (purple arrows). • Iteration Kernel Sharing : indicates if the kernels are shared across all iterations.

Original: This allows to study the stability and the complexity of optimizing iterative processes w.r.t.
Revised: This allows to study the stability and the complexity of optimizing iterative processes w.r.t.

Original: a given multi-scale architecture.
Revised: a given multi-scale architecture.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We evaluate the performance of our multi-resolution schema F-MGNO and W-MGNO on two families of PDEs, namely 2D steady-state of Darcy ﬂow and 1D viscous unsteady Burgers’ equation .
Revised: We evaluate the performance of our multi-resolution schema F-MGNO and W-MGNO on two families of PDEs, namely 2D steady-state of Darcy ﬂow and 1D viscous unsteady Burgers’ equation .

Original: Datasets.
Revised: Datasets.

Original: Darcy ﬂow .
Revised: Darcy ﬂow .

Original: We construct our ﬁrst dataset with the following 2-D steady-state PDE
Revised: We construct our ﬁrst dataset with the following 2-D steady-state PDE

Original: [Equation]
Revised: [Equation]

Original: x ) = 0 , x ∈ ∂ (0 , 1)a is a random piecewise constant function parameterizing the PDE, given some f a unique u is associated with a .
Revised: a is a random piecewise constant function parameterizing the PDE, given some f a unique u is associated with a .

Original: In experiments, we approximate the mapping ( a, ∇ a ) (cid:55)→ u .
Revised: In experiments, we approximate the mapping ( a, ∇ a ) (cid:55)→ u .

Original: Burgers’ equation .
Revised: Burgers’ equation .

Original: In the second dataset, we consider the following 1-D viscous unsteady PDE
Revised: In the second dataset, we consider the following 1-D viscous unsteady PDE

Original: [Equation]
Revised: [Equation]

Original: with periodic boundary conditions.
Revised: with periodic boundary conditions.

Original: In experiments, we approximate the mapping from the initial condition u 0 to the solution at time t = 1 , i.e. u 0 (cid:55)→ u ( · , 1) .
Revised: In experiments, we approximate the mapping from the initial condition u 0 to the solution at time t = 1 , i.e. u 0 (cid:55)→ u ( · , 1) .

Original: For both datasets, 100 examples were generated for train and 100 other ones for validation/test.
Revised: For both datasets, 100 examples were generated for train and 100 other ones for validation/test.

Original: Baseline methods.
Revised: Baseline methods.

Original: We compare our proposed F-MGNO and W-MGNO w.r.t.
Revised: We compare our proposed F-MGNO and W-MGNO w.r.t.

Original: GNO and the original V-MGNO.
Revised: GNO and the original V-MGNO.

Original: To study to what extent the kernel construction from operator learning standpoint is helpful, proposed architectures are also compared with MLP and GCN (Kipf & Welling, 2017).
Revised: To study to what extent the kernel construction from operator learning standpoint is helpful, proposed architectures are also compared with MLP and GCN (Kipf & Welling, 2017).

Original: Methods are categorized into single-scale (MLP, GCN, GNO) and multi-scale (V-, F-, W-MGNO).
Revised: Methods are categorized into single-scale (MLP, GCN, GNO) and multi-scale (V-, F-, W-MGNO).

Original: On learning stability.
Revised: On learning stability.

Original: During training, we found out that the multi-scale architectures with Kaiming initialization (He et al., 2015) used in the original paper may lead to divergence in training.
Revised: During training, we found out that the multi-scale architectures with Kaiming initialization (He et al., 2015) used in the original paper may lead to divergence in training.

Original: This is caused by extremely large loss and gradient at the beginning of the training.
Revised: This is caused by extremely large loss and gradient at the beginning of the training.

Original: One possible explication is that, as the input is transformed through many kernels, an improper initialization will amplify the norm of features along all the forward steps.
Revised: One possible explication is that, as the input is transformed through many kernels, an improper initialization will amplify the norm of features along all the forward steps.

Original: We therefore chose orthogonal initialization to better control the norm of  output.
Revised: We therefore chose orthogonal initialization to better control the norm of the output.

Revised: [Table]

Original: We did a broad hyperparameter search to tune the model.
Revised: We did a broad hyperparameter search to tune the model.

Original: This showed that the initialization was crucial in MGNO architectures, with a main importance for the learning rate and even more for the initialization gain of the orthogonal initialization.
Revised: This showed that the initialization was crucial in MGNO architectures, with a main importance for the learning rate and even more for the initialization gain of the orthogonal initialization.

Original: Results.
Revised: Results.

Original: We show our results in Table 1 for Darcy ﬂow and Burgers’ equation .
Revised: We show our results in Table 1 for Darcy ﬂow and Burgers’ equation .

Original: For all variants of
Revised: For all variants of

Original: MGNO, we report the results with best architectural metrics.
Revised: MGNO, we report the results with best architectural metrics.

Original: We found that multi-scales methods outperform single-scale ones in both training and test error.
Revised: We found that multi-scales methods outperform single-scale ones in both training and test error.

Original: For both datasets, multi-scale methods achieve a decrease of 80-90% training error compared to[Table] the single-scale baselines.
Revised: For both datasets, multi-scale methods achieve a decrease of 80-90% training error compared to the single-scale baselines.

Original: This suggests  a better modeling of long-range interaction improves the expressiveness of the neural network.
Revised: This suggests that a better modeling of long-range interaction improves the expressiveness of the neural network.

Original: The same improvement tendency was also observed at test time.
Revised: The same improvement tendency was also observed at test time.

Original: All variants of MGNO reduce the test error by 10-50%.
Revised: All variants of MGNO reduce the test error by 10-50%.

Original: This improvement is less signiﬁcant as it may be limited by number of training data samples.
Revised: This improvement is less signiﬁcant as it may be limited by number of training data samples.

Original: Among the multi-scale models, for both PDEs, more complex F-/W-MGNO performs slightly better in training, i.e. 2.69 with V-MGNO down to 2.23 (-17%) with W-MGNO for Darcy ﬂow, and 4.
Revised: Among the multi-scale models, for both PDEs, more complex F-/W-MGNO performs slightly better in training, i.e. 2.69 with V-MGNO down to 2.23 (-17%) with W-MGNO for Darcy ﬂow, and 4.

Original: with V-MGNO down to 3.19 (-25%) with F-MGNO for Burgers’.
Revised: with V-MGNO down to 3.19 (-25%) with F-MGNO for Burgers’.

Original: However, we didn’t perceive a signiﬁcant difference in test error.
Revised: However, we did not perceive a signiﬁcant difference in test error.

Original: We also conducted further ablation studies to compare the impact of architectural metrics.
Revised: We also conducted further ablation studies to compare the impact of architectural metrics.

Original: See Appendix C for details.
Revised: See Appendix B for details.

Original: Discussion.
Revised: Discussion.

Original: To conclude, we conﬁrm with our experiments that multi-scale methods is important for better modeling physical signals, by efﬁciently maintaining long-range interactions in  spatial domain.
Revised: To conclude, we conﬁrm with our experiments that multi-scale methods are important for better modeling physical signals, by efﬁciently maintaining long-range interactions in the spatial domain.

Original: Some improvements in training were observed in with F- and W-MGNO, which may indicate an increase in model expressiveness with F- and W-cycle schemas.
Revised: Some improvements in training were observed in with F- and W-MGNO, which may indicate an increase in model expressiveness with F- and W-cycle schemes .

Original: However, with MGNO implementation, the evidence is not strong enough to support this claim.
Revised: However, with MGNO implementation, the evidence is not strong enough to support this claim.

Original: We suggest further investigation with other graph-based approaches to better understand more complex cycles.
Revised: We suggest further investigation with other graph-based approaches to better understand more complex cycles.

Original: In order to better generalize to unseen data, more constraints could be added to control train/test trade-off.
Revised: In order to better generalize to unseen data, more constraints could be added to control train/test trade-off.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we empirically studied the challenging task of representing physical signals at different scales from DL perspective.
Revised: In this work, we empirically studied the challenging task of representing physical signals at different scales from DL perspective.

Original: To do so, we developed two novel multi-scale architectures inspired by discretized multi-resolution schema (Jaysaval et al., 2016) and a neural integral operator, which is motivated by multipole theory (Li et al., 2020a).
Revised: To do so, we developed two novel multi-scale architectures inspired by discretized multi-resolution schema (Jaysaval et al., 2016) and a neural integral operator, which is motivated by multipole theory (Li et al., 2020a).

Original: To the best of our knowledge, this is the ﬁrst work that proposes to study and design multi-scale DL architectures from numerical schema standpoint.
Revised: To the best of our knowledge, this is the ﬁrst work that proposes to study and design multi-scale DL architectures from numerical schema standpoint.

Original: We proposed two MPGNNs to approximate this neural integral operator and we implemented it via F-cycle and W-cycle schema.
Revised: We proposed two MPGNNs to approximate this neural integral operator and we implemented it via F-cycle and W-cycle schema.

Original: The latter are iterative processes and hence are challenging to optimize.
Revised: The latter are iterative processes and hence are challenging to optimize.

Original: We deﬁned a set of metrics to evaluate the learning stability of these iterative processes and their performances.
Revised: We deﬁned a set of metrics to evaluate the learning stability of these iterative processes and their performances.

Original: We validated our work on two types of PDEs discretized on graphs.
Revised: We validated our work on two types of PDEs discretized on graphs.

Original: We argue that this work could open perspectives to study novel multi-scale neural architectures, beyond U-net, and V-F-W-cycle schema, suitable for multi-scale and/or scarce data.
Revised: We argue that this work could open perspectives to study novel multi-scale neural architectures, beyond U-net, and V-F-W-cycle schema, suitable for multi-scale and/or scarce data.

Original: One may consider a further study of discretized multi-resolution schema including the properties of their architectures and optimization procedures.
Revised: One may consider a further study of discretized multi-resolution schema including the properties of their architectures and optimization procedures.

Original: This could help to design more interpretable and efﬁcient architectures.
Revised: This could help to design more interpretable and efﬁcient architectures.

Original: Moreover, we think that studying multi-scale neural architectures from discretized multi-resolution schema’s standpoint could help to get insights about the capabilities of multi-scale neural architectures to reproduce some properties of discretized multi-resolution schema.
Revised: Moreover, we think that studying multi-scale neural architectures from discretized multi-resolution schema’s standpoint could help to get insights about the capabilities of multi-scale neural architectures to reproduce some properties of discretized multi-resolution schema.

Original: A  R EPRODUCTIBILITY S TATEMENT
Revised:  R EPRODUCTIBILITY S TATEMENT

Original: Code for reproducing the experiments and results reported in this work will be made available before the date of the workshop. B P IPELINES Graph nodes are uniformly sampled over the domain with different sample number at each scale. Graph edges in each scale are calculated to all points in the same scale within a given distance.
Revised: We provide a GitHub repository to reproduce the experiments.

Original: For multi-scale models, edges between scales are also calculated likewise. Input node features are values of the input function v 0 ( x ) and their position x . Each model predicts u ( x ) . To avoid divergence in training, we use the orthogonal initialization across all models.
Revised: We used NVIDIA 12Go single GPUs to conduct the experiments.

Original: C A BLATION STUDIES
Revised: B A BLATION STUDIES

Original: We conducted large-scale ablation studies with results in Tables 2 and 3.
Revised: We conducted large-scale ablation studies with results in Tables 2 and 3.

Original: We analyze the impact of different metrics on impacts of architectural metrics on prediction errors: • The impact of number of scales : We observe that the more the scales, the lower the training error.
Revised: We analyze the impact of different metrics on impacts of architectural metrics on prediction errors: • The impact of number of scales : We observe that the more the scales, the lower the training error.

Original: This shows an increasing tendency in model expressiveness w.r.t.
Revised: This shows an increasing tendency in model expressiveness w.r

Revised: [Table]

Original:  scales.
Revised: t. scales.

Original: • The impact of kernel sharing : We observe that sharing parameters may help improving training.
Revised: • The impact of kernel sharing : We observe that sharing parameters may help improving training.

Original: However, no major differences in test are noticed when evaluating generalization to test samples.
Revised: However, no major differences in test are noticed when evaluating generalization to test samples.

Original: Note that different variants of MGNO are the same according to number of scales.
Revised: Note that different variants of MGNO are the same according to number of scales.

Original: For 2-scale models, V-, F-, and W-MGNO are equivalent.
Revised: For 2-scale models, V-, F-, and W-MGNO are equivalent.

Original: For 3-scale models, F- and W-MGNO are equivalent.
Revised: For 3-scale models, F- and W-MGNO are equivalent.

Original: [Table]
Revised: [Table]

Original: [Table] (cid:55)
Revised: (cid:51) 1.60±0.

Original: [Table]
Revised: 6.18±0.

Original: [Table] F-MGNO
Revised: 3.95 W-MGNO (cid:55) (cid:55) 2.23±0.

Original: [Table]
Revised: 5.91±0.

Original: -MGNO [Table] [Table] (cid:51) 
Revised: 28.2 W-MGNO (cid:51) (cid:55) 1.68±0.

Original: [Table]
Revised: 6.17±0. 15.8 W-MGNO (cid:55) (cid:51) 1.85±0. 6.07±0. 7.07 W-MGNO (cid:51) (cid:51) 1.57±0.

Revised: 6.29±0.

Revised: [Table]