PIG: Physics-Informed Gaussians as Adaptive Parametric Mesh Representations
Authors: Namgyu Kang
Jaemin Oh
Youngjoon Hong
Eunbyung Park
Venue: ICLR 2025 Poster (2025)
Keywords: Gaussians
Physics-informed Deep Learning
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose Physics-Informed Gaussians, an adaptive mesh representation in which Gaussian parameters are trained to dynamically adjust their positions and shapes.

Abstract: The numerical approximation of partial differential equations (PDEs) using neural networks has seen significant advancements through Physics-Informed Neural Networks (PINNs). Despite their straightforward optimization framework and flexibility in implementing various PDEs, PINNs often suffer from limited accuracy due to the spectral bias of Multi-Layer Perceptrons (MLPs), which struggle to effectively learn high-frequency and nonlinear components. Recently, parametric mesh representations in combination with neural networks have been investigated as a promising approach to eliminate the inductive bias of MLPs. However, they usually require high-resolution grids and a large number of collocation points to achieve high accuracy while avoiding overfitting. In addition, the fixed positions of the mesh parameters restrict their flexibility, making accurate approximation of complex PDEs challenging. To overcome these limitations, we propose Physics-Informed Gaussians (PIGs), which combine feature embeddings using Gaussian functions with a lightweight neural network. Our approach uses trainable parameters for the mean and variance of each Gaussian, allowing for dynamic adjustment of their positions and shapes during training. This adaptability enables our model to optimally approximate PDE solutions, unlike models with fixed parameter positions. Furthermore, the proposed approach maintains the same optimization framework used in PINNs, allowing us to benefit from their excellent properties. Experimental results show the competitive performance of our model across various PDEs, demonstrating its potential as a robust tool for solving complex PDEs. Our project page is available at https://namgyukang.github.io/Physics-Informed-Gaussians/

Published as a conference paper at ICLR 2025
PIG: PHYSICS-INFORMEN GAUSSIANS AS ADAPTIVE PARAMETRIC MESH REPRESENTATIONS
Namgyu Kang" Department of Artificial Intelligence
Jaemin Oh'
Department of Mathematical Sciences
Yonsei University
KAIST
Youngjoon H
Eunbyung Parkl Department of Artificial Intelligence Yonsei University
Department of Mathematical Sciences Scoul National University
ABSTRACT
The numerical approximation of partial differential equations (PDEs) using neu- ral networks has seen significant advancements through Physics-Infornen Neural Networks (PINNs). Despite their straightforward optimization framework and flexibility implementing various PDEs, PINNS often suffer from limited accu- racy due the spectral bias of Multi-Layer Perceptrons (MLPs), which struggle to effectively learn high-frequency and nonlinear components. Recently, paramet- ric mesh representations in combination with neural networks have been inves- tigated as a promising approach to eliminate the inductive bias of MLPs. How- ever, they usually require high-resolution grids and a large number of colloca- tion points to achieve high accuracy while avoiding overfitting, In addition, the fixed positions of the mesh parameters restrict their flexibility, making accurate approximation of complex PDEs challenging. To overcome these limitations, we propose Informed Gaussians (PIGs), which combine feature embeddings using Gaussian functions with a lightweight neural network. Our approach uses trainable parameters for the mean and variance of each Gaussian, allowing for dynamic adjustment of their positions and shapes during training. This adapt- ability enables our model to optimally approximate PDE solutions, unlike mod- els with fixed parameter positions. Furthermore, the proposed approach main- tains the same optimization framework used in PINNs, allowing us to benefit from their excellent properties. Experimental results show the competitive per- formance of our model across various PDEs, demonstrating its potential as a ro- bust tool for solving complex PDEs. Our project page is available at https: //namggyuang..iittae..
INTRODUCTION
Machine learning techniques have become promising tools for numerical solutions to partial dif- ferential equations (PDEs) (Raissi et al. 2017 Yu et al. 2018 Karniadakis et al. 2021 Finzi ct al. 2023 Gaby et al. 2024). A notable example is the Physics-Informed Neural Network (PINN) (Rasssi et al.] 2019), which leverages Multi-Layer Perceptrons (MLPs) and gradient- optimization algorithms. This approach circumvents the need for the inte-intessive mesh design prevalent in numerical methods and allows to solve both forward and inverse problems within the same optimization framework. With the increased computational power and the development of casy-to-use automatic differentiation software libraries (Abadi et al. 2015 Bradbury et al. 2018 Innes] 2018; Paszke et al.) 2019), PINNs have successfully tackled a broad range of challenging
PDES (Hu al. 2024c; Li al. 2024 Oh et al. 2024).
Although the neural network approach shows significant promise in solving PDEs, it has several lim itations. Training PINNs typically requires numerous iterations to converge (Saarinen et al. 1993
"quual contribution
Corresponding authors

---

Published as a conference paper at ICLR 2025
400 Iter.
800 Iter.
1200 Ite..
4000 Iter.
673
675
435
Training visualization of the Allen-Cahn equation (400. 800, 1200, 4000 training iter- ations): Each Gaussian displayed as the ellipsoids, exhibiting different positions and shapes ac- cording to the Gaussian parameters, mean and covariance. Since we adopt causal loss (Wang et al.) 2024c), the solution gradually approximated from 0 to Note that the Gaussians are densely aligned in the locations where the solution changes abruptly.
Figure 1:
Wang et al. 2021| De Ryck al. 2023). Despite recent techniques aimed at reducing computational costs, multiple forward and ward passes of neural networks are still necessary to update net- work parameters. Furthermore, obtaining more accurate approximations demands the use of wider and deeper neural networks, which enhances their expressiveness but significantly increases compu- tational costs (Cybenko 1989; Baydin al., 2018 Kidger & Lyons (020). In addition, inductive biases inherent in MLPs often hinder the accuracy of solution approximations.
well-known exam- ple is the spectral bias, which favors learning oow-frequency components of solutions and disturbs singular behaviors (Rahaman al. 2019). Although some solutions
capturing high- figh-frequency to the spectral bias have been proposed (Tancik et al.] [2020] SSizzmann et al. 2020), eliminating inductive biases from neural networks remains a challenge.
To address these issues, recent studies have explored combining classical grid-based representations with lightweight neural networks (Hui et al. 20118) Cao al., (223)). In this approach, the parametric grids map input coordinates to intermediate features, which are then processed by neural networks to produce the final solutions. By relying on high-resolution parametric grids for representational capacity, this approach presents the potential to reduce the impact of neural networks' inductive biases. Moreover, using lightweight neural networks significantly reduces computational demands, leading to faster training speeds compared to traditional approaches only using neural networks.
While promising, existing methods that combine parametric grids with neural networks face fun- damental challenge. The positions of the parameters (the locations of vertices) are predetermined by the grid resolutions and remain fixed during training. Since the optimal allocation of represen- tational capacity (determining where to place more vertices) is unknown, these methods typically use high-resolution grids that uniformly distribute many vertices across the entire input domain to achieve high accuracy. This approach results in using large set of learnable parameters, which often leads to overfitting issues, i.e., low PDE residual losses but inaccurate solutions. To mitigate this problem, large number of collocation points are sometimes used during training the expense of the increased computational costs.
In this work, introduce novel representation for approximating solutions to PDEs. Drawing inspiration from adaptive mesh-based numerical methods (Berger & Oliger 1984 Scol et al. 2016
and recent parametric grid representations (Li Lee 2021 [02]]Jang Jang et al.) 2023), 3) we propose Physics- Informed Gaussian (PIG) that learns feature embeddings of input coordinates, using a mixture of Gaussian functions. For given input coordinate, PIG extracts feature vector as the weighted sum of the feature embeddings held by Gaussians with their learnable parameters (positions and shapes). They are adjusted during the training process, and underlying PDEs govern this dynamic adjustment. To update the parameters of all Gaussians, leverage the well-established PINNs training framework, which employs numerous collocation points to compute PDE residuals and uses gradient-base based optimization algorithms.
The proposed approach offers several advantages over existing parametric grid methods. PIG dy- namically adjusts the computational mesh structure and the basis functions (Gaussians) to learn the feature embeddings. By following the gradient descent directions, the Gaussians move towards re- gions with high residual losses singularities, and this adaptive strategy allows for more efficient
2

---

Published as a conference paper at ICLR 2025
and precise solutions than the static uniform grid structures. In addition, Gaussian functions are in- finitely differentiable everywhere, allowing for the convenient computation of high-order derivatives for PDE residuals, and they can be seamlessly integrated into deep-learning computation pipelines. The final architecture of the proposed approach, presented in F (c), which combines the learn- able Gaussian feature embedding and the lightweight neural network is new learningbbase PDE solver that can provide more efficient and accurate numerical solutions.
We have tested the proposed method on an extensive of challenging PDEs RRassi et al. 2019
Wang et al. 202T] Kang et al. 2023 Wang et al. 2023 Cho et al., 2024 Wang et al. 2024b). The experimental results show that the proposed PIG achieved competitive accuracy compared to the existing methods that use large MLPs or high-resolution parametric grids. When the number of Gaussians PIG is comparable to the number of vertices in previous parametric grids, our method significantly outperformed existing approaches, demonstrating its superior efficiency. Furthermore, the proposed PIG shows significantly faster convergence speed than PINNS using large neural net- works, demonstrating its effectiveness promising learning-based PDE solver. Our contributions are summarized as follows.
We introduce Physics-Infornen Gaussians, an efficient and accurate PDE solver that uti- lizes learnable Gaussian feature embeddings and lightweight neural network.
We propose dynamically adaptive parametric mesh representation that effectively ad- dresses the challenges encountered in previous static parametric grid approaches.
We demonstrate that PIG achieves competitive accuracy and faster convergence with fewer parameters compared to state-of-thh-al methods, establishing its effectiveness and paving the way for new research avenues.
2 RELATED WORK
2.1 PHYSICS-INFORMER NEURAL NETWORKS
PINNs are class of machine learning algorithms designed to integrate physical laws into the learn- ing process, popularized by Raissi et al. (2019). This is achieved by incorporating the PDE residuals directly into the loss function, allowing the model to be trained using standard gradient- op- timization methods. PINNs have gained significant attention for their ability to handle complex problems Yang et al. et al. 2022b Hu et al.
2021, Pensoneault & Zhu 2024) including nigh-dimensional PDEs (Wang
2024b a) that are challenging for traditional numerical methods. They are particularly effective in scenarios where data is sparse or expensive to obtain, as they can incorporate prior knowledge about the physical system. Applications of PINNs span various domains, including fluid dynamics, solid mechanics, and electromagnetics, demonstrating their versatility and effective- ness in solving real-world problems (Cai al. 2021 Khan & Lowther 2022 Bastek & Kochmann 2023). Key advantages of PINNs include their mesh-Iree nature, the ability to easily incorporate boundary and initial conditions, and their flexibility in handling various types of PDEs. However, they also face challenges, such as the need for extensive computational resources and the difficulty in training deep networks to achieve accurate solutions. For example, Wang et al.] (2024b) typi- cally uses around 9 hidden layers with 256 hidden units (sometimes up to 18 layers) to achieve high accuracy. This requires massive computations to run the neural network, which involves multiple forward and backward passes to compute the gradients for PDE residual loss.
2.2 INFORMED PARAMETRIC GRID REPRESENTATIONS
Physics-informed parametric grid representations combine traditional grid- based methods with neu- ral networks to solve PDEs (Kang et al., 2023 Huang & Alkhalifah) 2024. Wang et al. 2024a Shishehbor et al.) 2024a). These representations have also been extensively explored in image, video, and 3D scene representations (Liu et al. 2020 Yu et al. 202 Fridovich-Keil et al. 2022 Müller et al.) 2022 Chen et al.) 2022 Sun et al. 2022 Fridovich- Keil et al. 2023) by training the models as supervised regression problems. By discretizing the domain into a grid and associating each grid point with trainable parameters, these methods leverage the structured nature of grids to capture spatial variations effectively. This hybrid approach maintains high accuracy and reduces computational costs compared purely neural network-based methods. Key benefits include the
3

---

Published as a conference paper at ICLR 2025
MLP
MLP
(a) PINN
(b) Parametric Grid
(c) PIG (ours)
Figure 2: (a) PINN directly takes input coordinates (four collocation points) inputs and produces outputs. (b) Parametric grids first map input coordinates to output feature vectors. Each vertex the grids holds learnable parameters, and output features are extracted through interpolation schemes. (c) The proposed PIG consists numerous Gaussians moving around within the input domain, and their shapes change dynamically during training. Each Gaussian has Iearnable parameters, and feature vector for an input coordinate the weighted sum the learnable parameters based on the distance to the Gaussians.
ability to handle high-resolution representations and integrate boundary conditions efficiently, which especially important for solving PDEs. However, the fixed grid structure lead to suboptimal allocation of representational capacity during training.
2.3 ADAPTIVE MESHI-BASED METHIODS
Adaptive mesh-based methods dynamically adjust the computational mesh minimize the error between approximated and true solutions. This process involves a posteriori error analysis, which estimates errors after solving, allowing for targeted mesh refinement. Such adaptivity is crucial in the numerical analysis as ensures efficient allocation computational resources, focusing on regions with high errors and thus improving overall accuracy and efficiency (Ainsworth & Oden 1993 1997).
There are also some studies non-uniform adaptive sampling methods in the context of PINNs. Lu et al. (202T) proposed a residual-based adaptive refinement method in their work with Deep- XDE, aiming enhance the training efficiency of PINNs (Wu al. 2023). More recently, Yang ct al. (2023b) introduced Dynamic Mesh-based Importance Sampling (DMIS), a novel approach that constructs dynamic triangular mesh efficiently estimate sample weights, significantly improv- ing both convergence speed and accuracy. Similarly. Yang et al. (2023a) developed an end-to-end adaptive sampling framework called MMPDE-Net, which adapts sampling points by solving the moving mesh PDE. When combined with PINNs to form MS-PINN, MMPDE-Net demonstrated notable performance improvements. While these adaptive methods offer significant benefits, they also introduce additional complexity into the PINN framework.
2.4 POINT-BASED REPRESENTATIONS
Irregular basnt-based representations have long been considered promising approaches for data rep- resentation, reconstruction, and processing (Qi et al.) 2017 Xu et al., 2022; Zhang et al. 2022). recent study in 3D scene representation utilized Gaussians remarkable performance in image rendering quality and training speed (Kerbl et al.) (023). The combination of Gaussian representation and neural networks has recently been explored in regress- ing images 3D signed distance functions, showing its great expressibility (Chen et al. 2023) While those studies share some architectural similarities with our method, they all primarily focus supervised regression problems reconstruct the visual signals. We developed the architecture suitable for effective PDE solvers and first showed that the Gaussian features and neural networks can be trained in unsupervised manner guided by the physical laws.
graphical primitive and showed
4

---

Published as a conference paper at ICLR 2025
3 METHODOLOGY
3.1 PRELIMINARY: PHYSICS-INFORMED NEURAL NETWORKS
Consider an abstract underlying equation,
D Diul(x)= f(x), JEΩCR.
[x=](x)= g(x);
rE 02,
where D is differential operator, and B is boundary operator which could contain the initial condition. The physics-informed neural network methods try to find an approximate solution by minimizing
+ / [B[u)(x)) g(x)
(())
(3)
where ue is a neural network with the set of network parameters 8, À is positive real number, and a surface measure. In practice, integrals are usually estimated via Monte Carlo integration. PINNs typically utilize automatic differentiation to compute the PDE residuals and Vo Vol( For more details, please refer to the original paper RRassi al. 2019).
3.2 PHYSICS -IFORMEDD GAUSSIANS
In this section, present the proposed Physics-Informen Gaussian representation (PIG) for nu- merical solutions to PDEs. It comprises two stages: Gaussian feature embedding (3.2.1 and feature refinement with lightweight MLP 3.2.2
3.2. LEARNABLE GAUSSIAN FEATURE EMBEDDING
Let {(i, Er, ; I,.... N be the set of Gaussian model parameters, where μ4; E Rd is position of a Gaussian and Σ; E $4 a covariance matrix. Each Gaussian has a learnable feature embedding Ji E R& for a feature dimension k. For simplicity, we consider A = 1. Given an input coordinate RRd, the learnable embedding FEg Rdd - IR. extracts Gaussian features follows.
FEg(2) Σ f.G(()), G((z) = e -dx-ma (1-()-)))
where N is the number of Gaussians and G, represents the i-th Gaussian function. FEg maps an input coordinate feature embedding by weighted sum of the individual features f. of each Gaussian. Extensions to k > 1 for enhanced expressiveness are provided in Appendix A.1
Gaussian features distant from the input coordinates do not contribute to the final feature embedding. while only neighboring Gaussian features remain significant. Similar to the previous parametric grid methods, which obtain feature embeddings by interpolating only neighboring vertices, this locality encourages the model to capture sigh-frequency details by effectively alleviating spectral bias.
All Gaussian parameters & are learnable and iteratively updated throughout the training process. This dynamic adjustment, akin to adaptive mesh-based numerical methods, optimizes the structure of the underlying Gaussian functions accurately approximate the solution functions. For example, Gaussians will migrate the regions with high-frequency 2L.
singular behaviors that require more computational parameters, following the gradients Tyc (see Figure 1.. Compared to the existing parametric grid approaches, which achieve this goal by uniformly mereasing grid resolution, the proposed method can build more errameter-efficient and optimal mesh structure.
3.2.2 LEARNABLE FEATURE REFINEMENT
Once the features are extracted, a neural network processes the feature to produce the solution out- puts.
(5) where NNg is a lightweight MLP with the parameter We employed a single hidden layer MLP with limited number of hidden units, adding negligible computational costs. Feature extraction
(z))(z) = (F)(FE((z)).
5

---

Published as a conference paper at ICLR 2025
plays a primary role in producing the final solution, while the MLP functions feature refinement mechanism. Even though Gaussian features are already universal approximators (see (.3). using a small MLP at the end improved the solution accuracy by large margin compared to the method without the MLP, i.e., (z) = (T)
3.2.3 PIG AS A NEURAL NETWORK
The proposed Gaussian feature embedding ad- mits a form of radial basis function (RBF) net- work. Figure 3 depicts the overall PIG archi- tecture as neural network. The first layer con- tains N (the number of Gaussians) RBF units, and input coordinate passes through all RBF units, (z(z), resulting in a N N-dimensional vec- tor. A single fully connected layer processes this vector to produce a k-dimensional feature vector. The weight matrix W E * in this layer corresponds to the feature vectors held by each Gaussian, i.e., W E B equals f; E R
FE
NNo
Figure 3: PIG as a neural network.
The extracted feature vector further pro- cessed by single hidden layer MLP (we used the tanh activation function) produce the final output, as depicted Figure Overall, PIG can be interpreted as an MLP with one input layer with N RBF units and two hidden layers (no activation for the first hidden layer, and tanh for the second hidden layer).
A related study by Bai et (2023) has explored solving various PDEs using RBF networks (Park
& Sandberg 1991 Buhmann 2000) within the framework of physics-informed machine learning. However, their approach differs from ours in that the positions of the basis functions are fixed. adjust dynamically, moving in
In contrast, our method allows the positions of the Gaussians directions that minimize the loss function. addition, we extract the feature vectors from Gaussians and further process them using shallow neural networks while they directly predict the solution output from the Gaussians.
3.3 UNIVERSAL APPROXIMATION THEOREM FOR PIGS
Here, we present the Universal Approximation Theorem (UAT) for PIGs. A PIG consists of two functions: FEg and NNp (see equation 5). We will prove the UAT only for FE&, as the UAT for PIGs follows directly from the standard UAT for MLPs. Given our earlier discussion the relationship between PIGs and radial basis function networks, we begin with the following UAT specific to radial basis function networks.
Theorem Park & Sandberg 1991)) Let K Rdd R. be an integrable bounded function such that K is continuous and
{ K(x) du #0.
(6)
Then the family Sk, defined as linear combinations of translations of K,
f.K(æ pa) AR
SK
is dense in C(Rd).
However, Theorem does not apply to PIGs, the feature embedding FE, in PIGs takes slightly different form:
FEO(#) Σ (z E,), where the key difference lies the presence of E. Notably, the
(8)
Sk
Σ fKKlz ui: Ex E. Rd Eg E sd sdd ....nE REN
(9)

---

Published as a conference paper at ICLR 2025
Methods
Allen-Cahn
Helmholtz
Nonlinear Diftusion
Flow Mixing
Klein-Gordon
PINN
4.02e-1
9.50e-3
3.43e-2
LRA
3.69e-3
PIXEL
8,86e-3
8.63e-4
SPINN
6.10e-3
2.90e-3
3.90e-3
JAX-PI
5.37e-5
PirateNet
2.24e-5
PIG (Ours)
1.04c-4
4.13e-5
2.69e-3
4.51e-4
2.76e-3
+ 1std
+ 4.122.,,
2.599-05,
= 6.55e-4,
1.74e--
+ 4.277e-4
best
5.93e-5
2.12e-5
1.44e-3
2.67e-4
2.36e-3
Table 1: Comparison of relative L errors across different methods. Three experiments were con- ducted using seeds 100, 200, and 300, with the mean and standard deviation presented in the table. The methods compared include PINN (Raissi et al.] 2019), Learning Rate Annealing (LRA) (Wang et al. 2021), PIXEL (Kang et al. 2023), SPINN (Cho et al. 2024), JAX-PI (Wang et al., 2023), and Pirate-Net (Wang et al., 2024b). For fair comparisons, we included the reported values from the respective references and omitted results that were not provided in the original papers.
with S denoting the set of positive definite matrices, contains Sk. Therefore, Sk is dense in ((RR). We summarize this the following corollary:
Corollary The sealar-valued, d-dimensional PIGs {NNo FEg((0,)) RP are dense in C(RR)..
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
To validate the effectiveness of PIGs, we conducted extensive numerical experiments on vari- ous challenging PDEs, including Allen-Cahn, Helmholtz, Nonlinear Diffusion, Flow Mixing, and Klein-Gordon equations (for more experiments, please refer to the Appendix). We used the Adam optimizer (Kingma & Bal 2014) for all equations except for the Helmholtz equation, in which the L-BFGS optimizer (Liu & Nocedal, 1989) was applied for a fair comparison to the base- line method PIXEL. For computational efficiency, considered. diagonal covariance matrix E= diag(o?, 03) and we will discuss non-diagonal cases in Section 4.3.3
4.2 EXPERIMENTAL RESULTS
4.2.1
(1+1)D ALLEN-CAHN EQUATION
We compared our method against one of the state-of-the-art PINN methods the Allen-Cahn equa- tion. JAX-PI (Wang et al., 2023). For the detailed description, please refer to Appendix A.2. As shown in Figure 4 method converges significantly faster and achieves competitive final accu- racy (see Table JAX-PI used modified MLP architecture and hidden layers with 256 hidden neurons. Thus, the number of parameters in JAX-PI is more than 250K, while ours used only around 20K parameters ((N, d, k) (4000, 2, 1)). Also, note that the relative L error curve in Figure 4 displayed per iteration, and computational costs per iteration of are significantly lower than JAX-PI (7.25 x 10- 10 vs. 1.67 10 1s-it/,t), which requires multiple forward and backward passes of the wide and deep neural network.
4.2.2 2D HELMHOLTZ EQUATION
Figure illustrates the numerical performance of our proposed PIG method for the 2D Helmholtz equation, comparing it to PIXEL (Kang et al. 2023), one of the state-of-the-art methods within the PINN family that uses parametric grid representations. mental setup is available in Appendix A.2.2 The experiments
more detailed description of the experi-
conducted using three different random seeds, with PIG achieving the best relative L2 error of 2.22 x 10 when employing the L-BFGS optimizer, and relative 7.2 error of 2.12 10-5 with the Adam optimizer (For fair com-

---

Published as a conference paper at ICLR 2025
Reference
Absolute Error (PIG)
Absolute Errur (JAX-PI)
L2 Error Curve
100
5003
00
5002
16"
18"0
10-07
12"
1990 1503 200
2508
Figure 4: Allen-Cahn Equation. Reference solution and absolute maps of PIG and of the state-of-the-an art methods (JAX-PI) to Allen-Cahn Equation (x-axis: t, y-axis: x). The rightmost depicts relative I error curve during the training process (X-axis: iterations, y-axis: L error). The experiment was conducted with three different seeds, and the best relative I error of PIG is 5.93 10-5
Referenoy
Absoluic Error (PIG)
Absolutc Eroo (PIXEL)
L2 Error Curve
340
1.0
500000
1.02
0.2200
10P
5.7
6775
E00015
PIXEL
0.0
520035
13
500
1.0
500
E00000
0.02
0.01000
-220
-0.3
100015
10-
-0.75
-0.
60
25
10
-100
-13
-05
00
05
3.0
SIECEO -1.02
1.0
-25
2.0
2.2
0.3900
13"45
1000
2000
Figure 5: 2D Helmholtz Equation. Reference solution and absolute error maps of PIG and one of the state-of-the-ee methods (PIXEL) to 2D Helmholtz Equation. The rightmost depicts a relative 2 error curve during the training process and the best relative L eтror of PIG is 2.22 x 10
parison, reported the result using L-BFGS since PIXEL used L-BFGS). Notably, the results show that PIG's error four times lower than that of PIXEL, highlighting the efficiency and accuracy of method. We did not compare against other state-of-the-arr methods, such JAX-PI Pirate- Net, as they did not conduct experiments this setting. While we could have used their codes, the sensitivity of PINN variants hyperparameters complicates fair comparisons.
4.2.3 (2+1)D KLEIN-GORDON EQUATION
Figure 6 presents the predicted solution profile for the Klein-Gordon equation, comparing our results with SPINN. The best relative 2 error achieved is 2.36 x 10-3, For further details, please refer to Appendix A.2.3
4.2.4 (2+1)D NONLINEAR DIFFUSION EQUATION
We evaluated the performance of PIGs on the (2+1) dimensional nonlinear diffusion equation, with visualizations presented Figure 18 The relative I,² error achieved 1.44 10-3 For details on the experimental setup, please refer to Appendix A.2.5
4.2.5 (2+1)D FLOW MIXING PROBLEM
Figure displays the numerical solutions and absolute errors for the (2+1) flow mixing problem. Our solutions closely match the reference. with PIG achieving the relative L error of 2.67 x 10 compared to 2.90 x 10-1 for SPINN, underlining the enhanced accuracy of PIG. Figure 19 presents solution profiles up to / 4. Additional details can be found in Appendix A.2.4
4.3 HYPERPARAMETER ANALYSIS AND ABLATION STUDY
In this section, present the experimental results to show the effects of each component of the pro- posed PIG (using MLP, learnable Gaussian positions, and dense covariance matrices), In addition, study the effect of the number of Gaussians, the size of MI.P and input dimensions.
8

---

Published as a conference paper at ICLR 2025
Reference
Absolute Error (PIG)
Absolute Error (SPINN)
Error Curve
3200
C210
0850
-35
1.0
-55
-16
30000 20005 30003 00300 $0300
Figure 6: Klein-Gordon Equation. Reference solution and absolute error maps of PIG and one of the state-of-the-ar methods (SPINN) to Klein-Gordon Equation. Both models used 16 collocation points. The rightmost panel depicts relative L error curve during the training process and the best relative L eπror of PIG is 2.36 x 10
Reference
Absolute Error (PIC)
Absolute Error (SPINN)
L2 Eiror Curve
0.005
10a
SPIN
0.00
2
0.03
0.033
0.002
10:2
-2
-05 -2
0.001
-2
0.011
10
0.000
-2
2
-4 -2
2
-2
2
Figure 7: Flow mixing problem. The best relative T,2 error of PIG 2.67 x 10-4, while its maximum absolute error is 4.47 10-3 In comparison, one of the state-of-the-ar methods, SPINN achieved 2.90 X 10-3 L error and showed a maximum absolute error of 3.23 x 10-2
4.3.1 THE NUMBER OF GAUSSIANS
In numerical analysis, there is general trend that the quality of the solution improves the mesh refined. Given our approach of using Gaussians as mesh points, we expect that the accuracy of PIGs will improve with an increased number of Gaussians. Tablteiilusttaats the accuracy improve- ments of PIGs to the number of Gaussians. Overall, we observe positive correlation between the number of Gaussians and improved accuracy. It important to note that achieving this trend can be challenging for other PINN-type methods.
# Gaussians
Flow Mixing
Nonlinear Diffusion
Allen-Cahn
200
6.07e-03
2.33e-03
1.83e-02
400
3.13e-03
2.22e-03
2.93e-03
600
1.50e-03
2.23e-03
2.75e-03
800
1 1.44e-03
1.95e-03
1.22e-03
1000
1.31e-03
7.33c-03
4.81e-04
1200
1.03e-03
3.96e-03
3,98e-04
Table 2: The number of Gaussians and approximation accuracy (Flow Mixing, Nonlinear Diffusion, and Allen-Cahn). The results indicate that increasing the number of Gaussians typically leads to decrease in relative L error.
4.3.2 MLP IMPACT AND ADAPTIVE GAUSSIAN POSITIONS
While FEş serves as a universal approximator, we found that adding a small MLP NN, significantly enhances performance. Additionally, our ablation study explores the effectiveness of allowing adap- tive Gaussian positions (learnable vs. fixed μ)). The results Table 3 illustrate that varying Gaussian positions/ μ improve accuracy, particularly when combined with the MLP. We also evalu- ate the sensitivity of PIGs to the width and input dimensions of the MLP, as summarized in Table Notably, no clear trend emerges, highlighting the robustness of PIGs MLP variations.

---

Published as a conference paper at ICLR 2025
(MLP, ja)
Allen-Cahn
Helmholtz
Nonlinear Diffusion
Flow Mixing
Klein-Gordon
(X, Fixed)
4.72e-03
3.97e-04
6.32e-03
4.33e-03
6.44e-02
(0. Fixed)
1.82e-03
2.12e-04
2.e-0-03
1.09c-03
2.69e-02
(X, Learn)
7.29e-05
1.86e-04
5.26e-03
7.93e-04
8.51e-03
(0, Learn)
7.27e-05
2.12e-05
1.44e-03
4.51e-04
2.76e-03
Table 3: Ablation study results MLP and j across various equations.
4.3.3 COVARIANCE MATRICES
Dense covariance matrices have the potential improved accuracy over diagonal covariance matri- ces, at the expense of increased computational and memory costs. We compared these two types of covariance matrices across several equations: the 2D Helmholtz equation, the Klein-Gordon equa- tion, the flow mixing problem, and the nonlinear diffusion equation. We found that training PIGs with dense covariance matrices from seratch did not result in accurate solutions. To address this issue, first trained PIGs with diagonal covariance matrices and then initialized dense covariance matrices with pre-trained diagonal elements. As presented Table 4 there were improvements in the Klein-Gordon and flow mixing equations. We believe that advanced training techniques and engineering would improve the performance of PIG with dense covariance matrices and leave it to future works.
Helmholtz
Klein-Gordon
Flow Mixing
Nonlinear Diftusion
Dense
5.17e-05
1.81e-03
3.48e-04
3.86e-03
Diagonal
2.12e-05
2.76e-03
4.51e-04
1,44e-03
Comparison of error levels between dense and diagonal covariance matrices in PIGs. For dense covariance matrix experiments, we first trained PIG using a diagonal covariance matrix and then fine-tuned full covariance matrix parameters initialized from the trained diagonal elements.
Table 4:
5 CONCLUSION AND LIMITATIONS
In this work, we introduced PIGs as novel method for approximating solutions to PDEs. By leveraging explicit Gaussian functions combined with deep learning optimization, P/Gs address the limitations of traditional PINNs that rely on MLPs. Our approach dynamically adjusts the positions and shapes of the Gaussians during training, overcoming the fixed parameter constraints of previous methods and enabling more accurate and efficient numerical solutions to complex PDEs. Exper- imental results demonstrated the superior performance of PIGs across various PDE benchmarks, showcasing their potential as a robust tool for solving nigh-dimensional and nonlinear PDEs.
Despite the promising results, PIGs have certain limitations that warrant further investigation. Firstly, the dynamic adjustment of Gaussian parameters introduces additional computational over- head. While this improves accuracy, it may also lead to increased training times, particularly for very large-scale problems. However, by leveraging the locality of Gaussians, we can limit the evaluations nearby Gaussians, which reduces the necessary computations and saves GPU memory. Secondly, the number of Gaussians is fixed at the beginning of training. Ideally, additional Gaussians should be allocated to regions requiring more computational resources capture more complex solution functions. We believe it is promising research direction and leave it to future work. Finally, complete convergence analysis the proposed method not yet available. While empirical results show improved accuracy and efficiency, theoretical understandings of the convergence properties would provide deeper insights and guide further enhancements.
ACKNOWLEDDGEENTS
This work was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education SRRF202/RIA2C1192579) the Korean govermment (MSIT) (RS-2023-0021) 10299900,, RSS2004-00333548) This work was also sup- ported by the Culture, Sports, and Tourism R&D Program through the Korea Creative Content
10

---

Published as a conference paper at ICLR 2025
Agency grant funded by the Ministry of Culture, Sports and Tourism in 2024 (Project Name: Re- search on neural watermark technology for copyright protection of generative Al 3D content, RS- 2024-00348469).
REPRODUCIBILITY
We are committed to ensuring the reproducibility our research. All experimental procedures, data sources, and algorithms used in this study clearly documented in the paper. We already submit- ted the codes and command lines to reproduce the part of the results in Table [ as supplementary materials. The code and datasets will be made publicly available upon publication, allowing others to validate our findings and build upon our work.
ETHICS STATEMENT
This research adheres to the ethical standards required for scientific inquiry. We have considered the potential societal impacts of our work and have found no clear negative implications. All experi- ments were conducted in compliance with relevant laws and ethical guidelines, ensuring the integrity of our findings. We are committed to transparency and reproducibility in our research processes.
REFERENCES
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthicu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wat- tenberg. Martin Wicke, Yuan Yu, and Xiaogiang Zheng. TensorFlow: Large-scale machine learn- ing heterogeneous systems, 2015. URL https:/ http//wwww.. www.:ttessrfoo.com org/ Software available from tensorflow.org.
Mark Ainsworth and J Tinsley Oden. A unified approach posteriori estimation using element residual methods. Numerische Mathematik, 65:23-50, 1993.
Mark Ainsworth and Tinsley Oden. posteriori error estimation in finite element analysis. Com- puter methods in applied mechanics and engineering, 142(1-2):1-88, 1997.
Jinshuai Bai, Gui- Qui-Rong Liu, Ashish Gupta, Laith Alzubaidi, Xi-Qiao Feng, and Yuan ToagTong Gu. Physics-informed radial basis network (pirbn): A local approximating neural network for solving nonlinear partial differential equations. Computer Methods in Applied Mechanics and Engineer- ing, 1629,, 2023.
Jan-Hendrik Bastek and Dennis M Kochmann. Physics-informed neural networks for shell struc- tures. European Journal of Mechonics-A/Solids 97:104849, 2023.
Atilim Gunes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation in machine learning: a survey. Journal of machine learning research, 18(153):1-43 2018.
Marsha J Berger and Joseph Oliger. Adaptive mesh refinement for hyperbolic partial differential equations. Journal of computational Physics, 53(3):484 512, 1984.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Miin and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. URL http:
// //github wtt compo/o//jaa//
Martin Dietrich Buhmann. Radial basis functions. Acta numerica, 9:1-38, 2000.

---

Published as a conference paper at ICLR 2025
Shengze Cai, Zhiping Mao, Zhicheng Wang, Minglang Yin, and George Em Karniadakis. Physics- informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica, 37(2)): 1727-1738, 2021. 2021.
Yadi Cao, Menglei Chai, Minchen Li, and Chenfanfu Jiang. Efficient learning of mesh-based phys- ical simulation with bi-stride multi-scale graph neural network. In International Conference Machine Learning, pp. 3541-3558. PMLR, 2023.
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: fields. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23 23-27, 2022, Proceedings, Part XXXII, pp. 333-350. Springer, 2022.
Tensorial radiance
Zhang Chen, Zhong Li, Liangchen Song, Lele Chen, Jingyi Yu, Junsong Yuan, and Yi Xu. Neurbf: A neural fields representation with adaptive radial basis functions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 4182-4194, October 2023.
Junwoo Cho, Seungtae Nam, Hyunmo Yang, Seok-Bae Yun, Youngjoon Hong, and Eunbyung Park. Separable physics-informed neural networks. Advances in Neural Information Processing Sys- tems, 36, 2024.
George Cybenko. Approximation by superpositions of sigmoidal function. Mathematics of control, signals and systems, 2(4):303-314, 1989.
Tim De Ryck, Florent Bonnet, Siddhartha Mishra, and Emmanuel de Bézenac. An operator preconditioning perspective on training in physics-informed machine learning. arXiv preprint arXiv:2310.05801, 2023.
Tobin A Driscoll, Nicholas Hale, and Lloyd N Trefethen. Chebfun guide, 2014.
Mare Finzi, Andres Potapezynski, Matthew Choptuik, and Andrew Gordon Wilson. A sta- ble and scalable method for solving initial value pdes with neural networks. arXiv preprint arXiv:2304. 14994, 2023.
Sara Pridovich-Keii. Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5501555011000 2022.
Sara Fridovich-Ked Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. CVPR, 2023.
Nathan Gaby, Xiaojing Ye, and Haomin Zhou. Neural control of parametric solutions for high- dimensional evolution pdes. SIAM Journal on Scientific Computing, 46(2):C155-C155, 2024.
Zheyuan Hu, Zekun Shi, George Em Karniadakis, and Kenji Kawaguchi. Hutchinson trace estima- tion for high-dimensional and high-order physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 424: 16883, 2024a.
Zheyuan Hu, Khemraj Shukla, George Em Karniadakis, and Kenji Kawaguchi. Tackling the curse of dimensionality with physics-informed neural networks. Neural Networks, pp. 106369, 2024b.
Zheyuan Hu, Zhongqiang Zhang, George Em Karniadakis, and Kenji Kawaguchi. Score-based physics-informed neural networks for high-dimensional fokker-planck equations. arXiv preprint arXiv:2402.07465, 2024c.
Xinquan Huang and Tariq Alkhalifah. Efficient physics-informed neural networks using hash en- coding. Journal of Computational Physics, 501:112760,, 2024. ISSN 0021-9991. doi: https://doi. 0tg/:0..0116/...pp2224.1111111
URL httpp::/wwwtt http://wwwwwwweweeeeeeteet/ http://eeett.cooceeeeeeeeeeeeeee axticle/p1I/S02229911100000
Tak-War Hui. Xiaoou Tang, and Chen Change Loy. Liteflownet: A lightweight convolutional neural network for optical flow estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8981 8981-8989, 2018
12

---

Published as a conference paper at ICLR 2025
Michael Innes. Don't unroll adjoint: Differentiating ssa-form programs. CoRR, abs/ 181000991, 2018. BRRLattp://axxiv wttpa//.wrrrbbbb/ll...000
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: Convergence and gen- eralization in neural networks. Advances in neural information processing systems, 31, 2018.
Hojun Jang, Minkwan Kim, Jinseok Bae, and Young Min Kim. Dynamic mesh recovery from partial point cloud sequence. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15074-15084, 2023.
Namgyu Kang, Byeonghyeon Lee, Youngjoon Hong, Seok-Bae Yun, and Eunbyung Park. Pixel: Physics-informed cell representations for fast and accurate pde solvers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 8186-8194, 2023.
George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422 440, 2021.
Aly-Khan Kassam and Lloyd N Trefethen, Fourth-order time-stepping for stiff pdes. SIAM Journal on Scientific Computing, 26(4): 1214-))2214223333 2005.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023.
Arbaaz Khan and David A Lowther. Physics informed neural networks for electromagnetic analysis. IEEE Transactions on Magnetics, 58(9):1-4, 2022.
Patrick Kidger and Terry Lyons Universal approximation with deep narrow networks. In Conference on learning theory, pp. 2306 2327. PMLR, 2020.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization, arXiv preprint 17X1v:1412.6980, 2014.
Chen Li and Gim Hee Lee. Coarse-to-fine animal pose and shape estimation. Advances in Neural Information Processing Systems, 34:1 1717777-11778. 2021.
Zhengyi Li, Yanli Wang, Hongsheng Liu, Zidong Wang, and Bin Dong. Solving the boltzmann equation with neural sparse representation. SIAM Journal Scientific Computing, 46(2):C186 C215, 2024.
Dong C Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1):503-522) 1989.
Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural sparse voxel fields. Advances in Neural Information Processing Systems, 33:15651-15663, 2020.
Lu Lu, Xuhui Meng. Zhiping Mao, and George Em Karniadakis. Deepxde: A deep learning library for solving differential equations. SIAM review, 63(1):208-228, 2021.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), (1):1111,, 2022.
Jaemin Oh, Seung Yeon Cho, Seok-Bae Yun, Eunbyung Park, and Youngjoon Hong. Separable physics-informed neural networks for solving the bgk model of the boltzmann equation. arXiv preprint arXiv:2403.06344 2024.
Jooyoung Park and Irwin W Sandberg. Universal approximation using sasall-bsi--fffttti net- works. Neural computation, 3(2):246-25 257,,,227511 1991.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Yang, Zach Vito. Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- high-performance deep learning library, 2019.
13

---

Published as a conference paper at ICLR 2025
Andrew Pensoneault and Xueyu Zhu. Efficient bayesian physics informed neural networks for in- verse problems via ensemble kalman inversion. Journal of Computational Physics, 508: 13006,, 2024.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. 2017.
Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks, In International conference on machine learning, pp. 5301-5310. PMLR, 2019.
Maziar Raissi, Paris Perdikaris, and George Em Karniadakis, Machine learning of linear differential equations using gaussian processes. Journal of Computational Physics, 348:683-663 2017.
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Max Rensen, Michael Weinmann, Benno Buschmann, and Elmar Eisemann. Physics-informed gaus-
sian splatting. Master's s thesis, 2024.
Sirpa Saarinen, Randall Bramley, and George Cybenko. Ill-conditioning in neural network training problems. SIAM Journal on Scientific Computing, ((33)663-714, 1993.
Yunchang Seol, Wei-Fan Hu, Yongsam Kim, and Ming-Chih Lai. An immersed boundary method for simulating vesicle dynamies in three dimensions. Journal of Computational Physics, 322: 125-141, 2016.
Mehdi Shishehbor, Shirin Hosseinmardi, and Ramin Bostanabad. Parametric encoding with atten- tion and convolution mitigate spectral bias of neural partial differential equation solvers. arXiv preprint arXiv:2403.. 15652, 2024a.
Mehdi Shishehbor, Shirin Hosseinmardi, and Ramin Bostanabad. Parametric encoding with atten- tion and convolution mitigate spectral bias of neural partial differential equation solvers. Struc- tural and Multidisciplinary Optimization, 67(7):128, 2024b.
Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances neural information processing systems, 33:7462-7473. 2020.
Cheng Sun, Min Sun, and Hwann-Tzong Chen. Direct voxel grid optimization: Super-fast con- vergence for radiance fields reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.
2022.
Panos Tamamidis and Dennis N Assanis. Evaluation of various high-order-accuracy schemes with and without flux limiters. International Journal for Numerical Methods in Fluids, 16(10):931- 948, 1993.
Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537-7547, 2020.
Haoxiang Wang, Tao Yu, Tianwei Yang, Hui Qiao, and Qionghai Dai. Neural physical sim- ulation with multi-resolution hash grid encoding. Proceedings of the AAAI Conference on Artificial Intelligence, 3866)5510-5415
Mar. 2024a. doi: 16.16/aa 3388.228444 URL https::/ http:///js.aaa..... org/ index.idxx.e//ii whtppppppp/aapep.. http/////aatiiceee/eee//////
Sifan Wang, Yujun Teng, and Paris Perdikaris, Understanding and mitigating gradient flow patholo- gies in abysicc-informed neural networks. SIAM Journal on Scientific Computing, A3055- A3081, 2021.
Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:1 10768, 2022a.
14

---

Published as a conference paper at ICLR 2025
Sifan Wang, Shyam Sankaran, Hanwen Wang, and Paris Perdikaris. An expert's guide to training physics-ifformed neural networks. arXiv preprint wrxiv:2308.08466 2023.
Sifan Wang, Bowen Li, Yuhan Chen, and Paris Perdikaris. Piratenets: Physics-informed deep learn- ing with residual adaptive networks. arXiv preprini arXiv:2402.003326, 2024b.
Sifan Wang, Shyam Sankaran, and Paris Perdikaris. Respecting causality for training physics- informed neural networks. Computer Methods in Applied Mechanics and Engineering, 421: 116813, 2024c.
Yifan Wang, Pengzhan Jin, and Hehu Xie. Tensor neural network and its numerical integration. arXiv preprint arXiv:2207.0275: 2022b.
Chenxi Wu, Min Zhu, Qinyang Tan, Yadhu Kartha, and Lu Lu. A comprehensive study of non- adaptive and residual-baseee adaptive sampling for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering. 403:115671, 2023.
Qiangeng Xu, Zexiang Xu, Julien Philip. Sai Bi, Zhixin Shu, Kalyan Sunkavalli, and Uirich Neu- mann. Point-nerf: Point-based neural radiance fields. In CVPR, pp. 5438-5448, 2022.
Liu Yang, Xuhui Meng, and George Em Karniadakis. B-pinns: Bayesian physics-informet neu- ral networks for forward and inverse pde problems with noisy data. Journal of Computational Physics, 425:109913, 2021.
Yu Yang, Qihong Yang, Yangtao Deng, and Qiaolin He. Mmpde-net and moving sampling physics- informed neural networks based on moving mesh method. arXiv preprint arXiv:2311.16167,
2023a.
Zijiang Yang, Zhongwei Qiu, and Dongmei Fu. Dmis: Dynamic mesh-based importance sampling for training physics-inforned neural networks. In Proceedings of the AAAI Conference on Artifi- cial Intelligence, volume 37, pp. 5375-5383, 2023b.
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering neural radiance fields. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5752-5761, 2021.
Bing Yu et al. The deep ritz method: a deep learning- baanid numerical algorithm for solving varia- tional problems. Communications in Mathematics and Statistics, 6(1):1-12, 2018.
Qiang Zhang, Seung-Hwan Baek, Szymon Rusinkiewicz, and Felix Heide. Differentiable point- based radiance fields for efficient view synthesis. arXiv preprint arXiv:2205. /4330, 2022.
A APPENDIX
A.1 ENHANCED GAUSSIAN FEATURE EMBEDDING
To enhance the expressive capability, different Gaussians be used for each feature dimension. The learaale Gaussian feature embedding FE(r; Φ) Rd - R² and the set of Gaussian model parameters ((1,, E,, Jt) : 1..... N) defined as previously described. Then, have different Gaussians for each feature dimension, where E R is the Gaussian position parame- ters, Pt.j E Edd denotes the position parameter for j-th feature dimension, and fr.j E IR represents j-th feature value. Similarly, Σi.j Sd is a covariance matrix for j-th feature dimension. Given an input coordinate x E Rd. j-th element of the learnable Gaussian feature embedding FE, o) is defined as follows,
0z) (h Gzy(a) =e ((=-w)))) "
(10)
where Gig is the Gaussian function using Gaussians parameters for j-th feature dimension.
15

---

Published as a conference paper at ICLR 2025
A.2 DETAILED DESCRIPTION OF EXPERIMENTS
A.2.1
(I+1)D ALLEN-CAHIN EQUATION
The Allen-Cahn equation is a one-dimensional time-dependent reaction-diffusion equation that de- scribes the evolutionary process of phase separation, which reads
0.0001 Mgz + 5u 5u 0, (x, E [-11,1] [0, 1].
(11)
with the periodic boundary condition
u( 1,, = u(1,t), Uy u (-11) = u1 (1,
(12)
The initial condition for the experiment was u 0) c00o)sTT:) We used the WTK-based loss balancing scheme (Wang et al. (022a) to mitigate the illcconit spectrum the neural tangent kernel (Jacot al. 2018). We used N 4000 Gaussians for training and diagonal covariance matrix for parameter efficiency, where the diagonal elements of the initial Σ were set to a constant value of 0.025. The 14, uniformly initialized following Uniform (0, 2] We used shallow MLP with one hidden layer with 16 hidden units, and the dimension of the Gaussian feature was k = 1.
Reference solution was generated by Chebfun (Driscoll et al. 2014), which utilizes the Fourier col- location method with N = 4096 Fourier modes with ETDRK4 time stepping (Kassam & Trefethen 2005) with a fixed time step At 1/200.
A.2.2 2D HELMHOLTZ EQUATION
The Helmholtz equation is the eigenvalue problem of the Laplace operator A V We consider the manufactured solution
u(x, y) sin(ar sin(agny), (a1, (4,
(13)
to the two-dimensional Helmholtz equation with the homogeneous Dirichlet boundary condition given by
Au + k2u = q, (x,g) E [-1, ==1,
(14)
where
q(x, y) = x22 sin (a, Tz) sin (azTy) (a) sin (a, πσ) sin (a2Ty) - (a2 2 sin (a, ..) sin (a2ny)
(15)
can be extracted from the solution u.
We used N = 3000 Gaussians in this experiment. The weights and scales of Gaussians were initial- ized following Uniform[-1, 1J and 0.1, respectively. The feature size of Gaussians was fixed at 4. The shallow MLP has 16 hidden nodes, and its network parameters were initialized by Glorot nor- mal. The inputs for the Gaussians were rescaled into [0, 1J2, therefore the positions were initialized following Uniform 00, 1/².
A.2.3 (2+1)D CLEIN-GORDE EQUATION
The Klein-Gordon equation is relativistic wave equation, which predicts the behavior of particle at high energies. We consider the manufactured solution
u(x, y, t) (x + y) cos(21) + zy s2n
(16)
to the (2+1) dimensional inhomogeneous Klein-Gordae equation
Au + u2 S. (I,y,4) () E 1)1 [0, 10).
(17)
where the forcing f, initial condition, and Dirichlet boundary condition are extracted from the man- ufactured solution In this experiment, we employed N 100 Gaussians and a shallow MLP whose input dimension 4 and hidden layer size is 16. The network parameters for the shal- low MLP were initialized by Glorot Normal. Every weight of Gaussian was initialized following Normal(O, 0.013). The scale parameter a,'s were initialized with a constant value of 0.5. Instead of direct usage of the computational domain [-1, 1]2 [0, 10). used linearly rescaled values [0, 2 for the inputs of Gaussians. Accordingly, position parameters of Gaussians were initialized following Uniform 0, 2/3.
16

---

Published as a conference paper at ICLR 2025
A.2.4 (2+1)D FLOW MIXING PROBLEM
A mixing procedure of two fluids in two-dimensional spatial domain could be described in the following equation
Ue + aur + buy = 0, (I. E [-4,4] x
[0, 4),
(18)
Ut y Mx, = V1 x
a(x.y) =
(19)
0t max r'
Ut,max r
4 = sech² (r ttan(()),
(20)
== v x² + y Ut.max 0.385.
(21)
The analytic solution is u(x, y,
tanh (ø cos(uct) 2 sin(wt)). where w / mmxx); The initial condition can be extracted from the analytic
see e.g.. Tamamidis & Assanis (993).. solution.
To predict the solution to the PDE, used N = 4000 Gaussians. The weights and scales were initialized to Normal(O, 0.01²) and 0.1, respectively. The size of Gaussian features was fixed at 4. MLP had 16 hidden nodes, and its parameters were initialized by Glorot normal. Inputs for the Gaussians were rescaled to [0, 2], hence the positions of Gaussians were initialized following Uniform 00, 2
A.2.5 (2+1)D NONLINEAR DIFFUSION EQUATION
The diffusion equation is a parabolic PDE describing the diffusion process of a physical quantity, such heat. We consider nonlinear diffusion equation for our benchmark, which reads
04 0.05 (l| + udn),
(., [-1, [0. 1].
(22)
un (x))) = 0.25g
0;0.2.0.3,
x; 001. + 0.4g
+ 0.3g
x: 0.550000 0,
v10
v15
/0
where
-
(x, and g(., y; a, b, There are three peaks at the initial time and the peaks spread out as time goes on.
We employed N 4000 Gaussians. The weights and scales of Gaussians were initialized to Normal 0.012) and 0.1, respectively. The size of Gaussian features was 4. The hyperbolic tan- gent MLP had only single hidden layer with 16 nodes, and its parameters were initialized by Glorot normal. The inputs for the Gaussians were rescaled into [0, 1 Correspondingly, the positions of Gaussians were initialized following Uniform (0, 1|3.
A.3 ADDITIONAL EXPERIMENTS
Here, we compare PIGs to PIRBNs Bai et al. 2023 Two equations in the PIRBN paper are chosen as benchmarks.
Equation (15) in Bai et al.
(203)::
0
2x.2 u(x 100) 4y²2 sin (2 100)) 0,
(23)
and ((100) u(101) 0. The exact solution is u(x) sin 100)). We considered 4.
Equation (30) in Bai al. 2023)
0
0.2 u(x) --27 2 (22 x) cos((222)) +.5 (in N² =2 x) sin (2Tx)
(24)
+ 167 - 20) cos(16Tr)) + 0.5 sin (16Tz) - 64 (xx 20)² sin sin(16
and = = = 0. The exact solution uxx) = ( (32 (2xx) + (~ sin(167z))
Referring to the numbers in 5 PIGs achieved error levels by two orders of magnitude smaller than PIRBNs. This improvement could attributed to the introduction of a tiny MLP and letting posi- tions move during training.
17

---

Published as a conference paper at ICLR 2025
Equation 23
Equation 24
PIRBNs PIGS
6.87e-03 3. 70--04
1.47e-02 9. 9.
1.79c-05 # 3.80e-06
1.14c-04 = I 1.19e-05
Table 5: Results of the comparison study between PIGs and PIRBNs for Equations 23 and 24 PIGs achieve lower errors than PIRBNs, highlighting their superior performance in both equations.
A.4 SEPARABLE PIGS
Separable PINNs have shown excellent performance across various PDEs CCho et al. 2024: Oh
al. 2024). When mesh points are tensor products of ID grids, the number of network forward passes of SPINNs scale linearly (Nd)), in contrast to the exponential scaling O(N") of traditional PINNs, which adopt a single MIP.
Here, we provide proof-of-concept for combining SPINNs and PIGs. Separable PIGs (SPIGs) might have the following form:
u(
Ta) R EII (t;;0)
(25)
where PIG, is the r-th component of the output vector.
Reference u(t.x.y)
Predicted u(t.x.y)
Absolute error
10
1.0
05
05
05
-1.0
-0.
-0.5
-1.0
-10
-1.
Figure
equation A.2.3 The relative 2 error of SPIG is 3.68 10-4.
2D L-shaped Poisson equation the two-dimensional Poisson equation defined on L-shaped do- main. Despite the non-tensor product nature of the computational domain, SPINNs can deal with such complex domains by masking outputs. Please refer (Cho et 2024) for the description of this benchmark problem. A SPIG achieved 1.89 x 10 relative L error for this problem, while SPINN solution 3.22 x 10-2
(2+1)D Klein-Gordon equation SPIG achieved 3.68 x 10-1 relative 2 error. PIG's best relative L error was 2.36 x 10 Please refer A.2.3 for description PDE. SPIG used modified MLP with 2 layer and 16 hidden features. The weights and scales were initialized to Normal(O, 0.012) and 0.1, respectively. position parameters of Gaussians were initialized following Uniform (0, 2 2500 Gaussians are used.
(3+1)D Klein-Gordon equation SPIG achieved 2.88 10 44 relative 2 error. SPINN's relative 1 emor was 1.20 x 10-3 Please refer to (Cho et al. problem. SPIG used modified MI.P with layer and 16 hidden features. The weights and scales initialized Normal(0, 0.012) and 0.25, respectively. position parameters of Gaussians were initialized following Uniform(0, (0, 2).. 2500 Gaussians are used.
2024) for the description of this benchmark
3D Helmholtz equation SPIG achieved 1.50 10-3 relative L error. SPINN's relative L error was 3.00 x 2 Please refer to (Cho et al. 2024 for the description of this benchmark problem. SPIG used modified MLP with 2 layers and 16 hidden features. The weights and scales were initial- ized to Normal(O, 0.012) and 0.05, respectively. position parameters of Gaussians were initialized
following Uniform 00, 2 2500 Gaussians are used.
18

---

Published as a conference paper at ICLR 2025
Reference u(x,y,
Predicted y.x.y.2)
Absolute error
10
10
18
05
05
-10
100
-10
00
-10
-1.
Figure 9: 3D Helmholtz equation
A.4
The relative L error of SPIG is 1.50 x 10-3
A.5 INVERSE PROBLEM
With observation data, the PINN framework can estimate unknown equation parameters by letting them be learnable. Here we consider (I+1)D Allen-Cahi equation
Up 10 Ugz + >u - 5u 0,
with unknown coefficient A. Other conditions the with Section A.2.1 We estimated
using reference solution as observation data. Figure I0 presents estimated À over iterations, clearly showing PIG's faster convergence.
Allen-Cahn
Truth
PIG
PINN
0
10
20
Training iterations
Figure 10: Allen-Cahn Inverse problem. The experiment was conducted on five different seeds (100, 200, 300, 400, 500). PIG showed better performance than PINN.
A.6 HIGH-DIMENSIONAL EQUATIONS
Hu et al. introduced stochasticity in the dimension during the gradient descent (SDGD) efficiently handle high-dimensional PDEs within the PINN framework Hu et al. (2024b). PIGs can utilize SDGD tackle extremely high dimensional PDEs, e.g., 100D Allen-Cahn, and Poisson equation. Specifically, let d = 100 and Bd {o E Rd
||ж||| I 1} be the domain. We consider a function
Vexact = (1 - ||x)
Σ
Ci sin (z; + cos(1:+1) + #:41 cos(2:))
as our exact solution, where G; ~ Normal 12). Our benchmark problems are the Poisson equation and the Allen-Cahn equation, which read
Au (Poisson) and Au =
(Allen-Cahn)
where g is induced from the exact solution.
Figure presents relative L error curves over iterations. Note that global polynomial-bate based meth- ods cannot handle such high dimensional equations due to the curse of dimensionality.
19

---

Published as a conference paper at ICLR 2025
100D Allen-Cahn
100D Poisson Best Error: 8.42c-03
Best Error: 8.88e-03
L.2 Rel. Error
L. Rel. Error
10:1
10
000 1000
1000
3000 2503 3000
3500
550 1000 Training iterations
1500
2008
2000
2020
3300
Training iterations
Figure 11: Relative L error curves for two high dimensional PDEs. Left: 100D Allen-Cahn equa- tion. Right: 100D Poisson equation. PIGs achieved 8.88 x 10-3, and 8.42 10 respectively.
A.7 LID-DRIVEN CAVITY
To further illustrate the effectiveness PIGs over traditional parametric mesh methods, we chose baseline and considered the lid-driven cavity problem
PGCAN Shishehbor al. (2024b) presented in the paper. The domain is [o, 1J2. The homogeneous Dirichlet boundary condition imposed except for the lid {(x; 1) E [0, 1]}. The governing equation 2D stationary incompressible Naiver-Stokes equation,
V u 0 p(u V)u - -pp 2V2
where the boundary conditions are given as follows:
u(0, g) u(1. y) (0,
u(x, 0) = (0. 0),
u(r, 1) (A (in())))0) 0), A 1
p(0,0) 0) = 0.
We used 2000 Gaussians. Covariance matrices were diagonal and initialized at 0.1 and positions were initialized following Uniform(0, 2).
Figure 12
depicts numerical results. PIG shows excellent agreement with the reference solution.
Figure illustrates faster convergence of PIGs compared to the baseline method PGCAN.
A.8 EXAMPLE FOR SPECTRAL BIAS
Figure 14 illustrates PIG's ability to approximate high-frequency functions. We considered
Helmholtz equation (see Section A.2.2) with high wavenumber (a1, a2) (10, 10) for bench- mark problem.
A.9 THE HISTOGRAM OF VARIANCES AND DISTANCES OF GAUSSIANS
Figure 15 shows the histograms the Gaussian parameters for two benchmark problems dis- cussed in Section 4.2.5 and Section 4.2.3 Readers may observe that the Gaussians in the right panels are more global and, therefore, more sparsely distributed compared to those in the left panels.
A.10 COMPARISON BETWEEN PIG AND SIREN
In this section, we compare the performance of PIG with SIREN Sitzmann et al. (2020) PIG is composed of feature embedding FE and lightweight neural network NNg. Here, investigate the effectiveness of SIREN when used either as a feature embedding or as lightweight neural network.
20

---

Published as a conference paper at ICLR 2025
0.00
015
6.8
0.18
Reference
0.0
0.50
0:6
0.08
0.-
0.00
0.25
4015
0.13
4.22
02-
0.2
0.30
0.2
4.18
4.22
4,27
0.
0.8
0.0
0.6
0.8
06
0.80
0.15
0.38
6.8
0.6
00
Prediction
0.6-
0.50
0:6
0.6
0.33
4.00
0.2
0.2
0.30
0.2
4.12
-0.35
0.23
00
0.2
08
0.8
0.0
0.8
0.6
0.61
0.00
0.8
0.00
0.8
08
Abs. Error (PIG)
0.00
0.00
0.6
9:6
0.00
0.6
0.00
0.01
0.2
8:00
0.2
4000
0.2
0.01
-000
0.0
.0.0
0.01
0.
0.2
0.8
3.0
0.6
0.6
0.8
0.
0.00
0.6
0.00
Abs. Error (PGCAN)
0.00
0.8
0.8
0.6
0.00
0.00
0.00
D2
6.00
0.2
0.00
3.000
0.01
0.0
0.00
0.0
4.00
0.0
0.00
0.6
0.6
Figure 12: Lid-driven cavity flow problem. PIG achieved 4.04 x 10 relative I error whereas the baseline parametric grid method PGCAN resulted in 1.22 10-3
When used as FE SIREN is implemented as MLP with layers, each containing 256 units, and (3x) as the activation function. As NNa, SIREN is shallow MLP with 16 units and sin(30x) activation function. It worth noting that using sin (30z) the activation function for the feature embedding FE& did not yield effective results.
FES NN,
Helmholtz
Flow Mixing
SIREN + ld
1.68e-03 + 2.02e-03
1.22e-()2 4.17e-03
1.18e-01 + 4.8
SIREN tanh
1.31e-03 + 8.26e-04
2.80e-02 2.50e-02
1.04e-01 + 8.61e-02
PIG SIREN
1.37e-05 + 1.64e-06
1.28e-03 = 1.09e-04
2.37e-02 + 4.62e-03
PIG + tanh
4.13e-05 + 2.59e-05
4.51e-04 1.74e-04
2.76e-03 1: 4.27e-04
Table 6: Comparison of PIG and SIREN performance. For all cases except the Helmholtz equation, the original PIG tanh formulation outperformed other methods. The improved performance of PIG SIREN on the Helmholtz equation may be attributed to the functional form of its exact solu- tion.
The results, summarized Table 6 indicate that SIREN as FEs did not perform notably well. However, when SIREN was employed NNg, it demonstrated excellent performance in solving the Helmholtz equation discussed in Section 4.2.2] This improvement likely due to the structural similarity between the SIREN activation and the functional form of the exact solution (equation 13
21

---

Published as a conference paper at ICLR 2025
Lid-Driven Cavity Eq.
10
PIG
PGCAN
L2 Rel. Error
10-3
10-9
40000
Training iterations
Figure 13: Relative I error curve of the lid-driven cavity problem. PIG achieved 4.04 x 10-4 and PGCAN which used the parametric grid method achieved 1.22 x 10-3,
Exact
Predict
Abs. Error
8.0
100
320
-050
Figure 14: 2D Helmholtz equation with high wavenumber (a1,a2) = (10, 10). PIG achieved relative 1 error of 7.09 10-3, while the parametric fixed grid method PIXEL reached relative L error of 7.47 x 10 PINN failed to converge.
A.11 COMPARISON WITH A CONCURRENT WORK
We conducted several experiments to compare PIG with the Physics-Informed Gaussian Splatting (PI-GS) method proposed by Rensen et al. 2024). Table 7 summarizes the relative L errors and computation times per iteration (shown in parentheses). Across the three benchmark problems, PIG consistently outperforms PI-GS in both accuracy and efficiency.
Burgers' equation (1)
Burgers' equation (2)
PIG
7.68 10 (0.28s/it)
1.08 x 10 (0.29s/it)
PI-GS
1.62 x 10 (1.5s/it)
2.61 x 10-1 (1.68s/it)
Table 7: Performance comparison of PIG and PI-GS across 3 benchmark problems. Results include relative L errors and computation times per iteration (s/it). Benchmarks are conducted on two variations of the (2+1)D Burgers equation.
Across all experiments, We considered [-2.5, 2.5] as spatial domain with zero Dirichlet boundary condition.
(2+1)D Burgers' Equation For 1/(10T), the (2+1)D Burgers' equation is given by:
Ou
Ou
Pu
82u
+
u
Øt
dr
dr
0y²
where u is scalar-valued function.
22

---

Published as a conference paper at ICLR 2025
Flow-Mixing Eq.
Klein-Gordon Eq.
20
5
Frequency
Frequency
4
3-
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Distance
Distance
1400
Min: 1.33e-06
Min: 5.65e-06
1200
Frequency
1000
Frequency
150
800
600
100
400
50
200
0
0.15
0.0
0.6
0.8
1.0
Variance
Variances
Figure 15: Histograms of the Gaussian parameters for the flow mixing problem and the Klein- Gordon equation. The upper panels display histograms of the minimum distances between the Gaussian centers, where distances >> 0 indicate the absence of mode collapse. The lower panels show histograms of the Gaussian variances, highlighting the non-degeneracy of the Gaussians.
In the first example, the initial condition is set as the probability density function (PDF) of a standard two-dimensional Normal distribution. In the second example, the initial condition the PDF of mixture of two Gaussians. Figures 16 and 17 illustrate the reference solutions and the corresponding PIG solutions.
A.12 ADDITIONAL FIGURES AND TABLES
MLP input dimension (k)
# Hidden units
2
3
7.77e-03
9.60c-03
7.68c-03
9.60e-03
8.55e-03
6.44e-03
1.06e-02
8.54e-03
16
8.24c-03
1.06e-02
1,21e-02
6.90e-03
32
7.14c-03
8.06e-03
1.22-02
6.87c-03
64
6.33e-03
7.50c-03
1.09c-02
9,48c-03
128
6.38e-03
6.88e-03
8.48e-03
7.47e-03
256
5.21e-03
6.60e-03
5.22e-03
5.40e-03
Table 8: The performance of different MLP configurations for the Helmholtz equation, displaying L relative errors at iteration 1,000 across various configurations of hidden units and MLP input dimensions. Overall, the results highlight the robustness the size of MLP, showing minimal variation in errors across different settings.
23

---

Published as a conference paper at ICLR 2025
Reference 0,0
Predicted
1.0
1.0
2-
0.8
0.8
0.6
0.6
>
0
0.4
0.4
-1
0.2
-2
2
-2
Reference at t 1.0
Predicted at t= 1.0
0.8
0.6
0.6
0
0.4
0.4
-1
0.2
0.2
-2
-2
2
-2
Figure 16: Prediction results of PIG for the first example of the (2+1)D Burgers' equation. PIG achieved relative L error of 7.68 x 10-4, with a computation time of 0.28 seconds per iteration. In contrast, PI-GS attained relative L error of 1.62 x 10-1 requiring 1.50 seconds per iteration.
Reference 0,0
Predicted 0.0
1.
1.0
2
0.8
0.8
0.6
0.6
>
>
0.4
-1
-2
0
2
-2
Reference 1.0
Predicted 1.0
1.0
2
0.8
0.8
1
0.6
0.6
0
0.4
0.4
0.2
0.2
-2
-2 0
2
-2
X
Figure 17: Prediction results of PIG for the second example of the (2+1)D Burgers' equation. PIG achieved a relative 122 error 1.08 10-3, with computation time of 0.29 seconds per iteration. In comparison, PI-GS attained relative I error of 2.61 x 10-1, requiring 1.68 seconds per iteration.
24

---

Published as a conference paper at ICLR 2025
Reference
Predicted
Reference
Predicted
1 0.5
t - 1.0
Figure 18: Non-linear diffusion equation 4.2.4 The experiment was conducted three different
seeds (100, 200, 300). The best relative L error is 1.44 x 10
t 0
t= 2
t 4
Fxact solution
-4
Predicted solution
Figure 19: Flow mixing equation 4.2.5
The experiment was conducted on three different seeds
(100, 200, 300). The best relative 1 emor is 2.67 x 10-4
25


Reviews:
[{"review_id": "ug3bB2Srpp", "reviewer": "Reviewer_77oA", "summary": "In this paper, the authors use deep Gaussian mixture models as parametric mesh representations to replace MLP in PINN, so that it performs better for high-frequency and non-linear components. Experiments for PDEs such as Allen-Cahn, Helmholtz, Nonlinear diffusion, etc., are conducted compared with several previous PINN derivatives, showing that the proposed method achieves sota solution accuracy.", "questions": "- With the work SIREN in mind, I wonder if the Gaussian basis is the best choice?\n- The idea is quite similar to 3D Gaussian splatting in rendering. However, there are differences. One easy way to explain is that, we can think of the volume rendering equation as a specific type of governing physics equation. Then the success of applying parametric mixed Gaussian in one specific type of equations doesn't mean it is the best idea for others. For example, there can be long-range interactions for certain PDEs, while Gaussian functions are local and lack such representation capability. I would like to have the authors' opinions on such comments.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- A theoretical proof of the universal approximation capability for the proposed method is provided.\n- The experiments are diverse, the baselines are sufficient, and the ablation study is detailed.\n- The writing is overall clear and detailed.", "weaknesses": "- Though diverse and rich, the experimental settings are all relatively simple, and lack comparisons to traditional methods.\n- Correct me if I am wrong, but I think only the solution accuracy is provided, without the computation time?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gnlH6tqhSL", "reviewer": "Reviewer_SQ8v", "summary": "The authors propose the Physics-Informed Gaussian (PIG) that learns feature embeddings of input coordinates, using a mixture of Gaussian functions. Compared to PINN, this paper applies Gaussians to input coordinates and makes the mean and covariance matrix learnable. The locality of Gaussians encourages the model to capture high-frequency details by effectively alleviating spectral bias.", "questions": "- I would be interested to see how the authors address the weaknesses above. Could authors provide additional evidence or context to support the novelty? \n- Have the authors conducted any experiments to demonstrate how the method alleviates spectral bias?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The paper is well-written and easy to follow. The code and experiments are of good quality. It provides a more concise and more parameter efficient feature embedding method compared to previous parametric grid representations method.", "weaknesses": "- The primary concern lies in the originality of this method. Though it’s considered as a feature embedding, mathematically, it is in the same spirit of a KAN layer with RBF. Also, it is not uncommon to consider Gaussian embedding in ML. \n\n- The paper states that the approach \"dynamically adjusts the positions and shapes of the Gaussians during training\" with parameters \"iteratively updated throughout the training process.\" The claims in the paper give an impression of adaptive sampling method based on the training. However, the approach does not actually take into account the residual or the loss function landscape. It would be better to provide a more rigid description on this. Also, in the experiments, the method is compared with different PINN approaches for each PDE, and the authors report the results without conducting experiments by themselves again. Though “the sensitivity of PINN variants to hyperparameters complicates fair comparisons”, it’s still necessary to conduct experiments with same setup and number of parameters to ensure a fair comparison. The average error of PIG is generally higher, but the authors only highlight the best of PIG. PIG is claimed to “enable more accurate and efficient approximations”, but the efficiency is not clear. Hence, it seems a lot of conclusions in this paper are overstated.\n\n- I would like to see how the spectral bias can be alleviated, either through experimental evidence or analytical justification.\n\n- For the 2D and 3D cases, what is the mathematical formulation for the Gaussian embedding? How is the parameter being initialized? Additionally, how are the boundary conditions (BC) being constrained?\n\n- The paper mentions that \"computational costs per iteration of our method are significantly lower than JAX-PI.\" A detailed report of these computational costs would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5EhOL4SkwX", "reviewer": "Reviewer_vuTU", "summary": "Physics-Informed Gaussians (PIG) is proposed, an efficient and accurate PDE solver that utilizes\nlearnable Gaussian feature embeddings and a lightweight neural network, counteracting the problems inherent in previous static parametric grid approaches. PIG model achieves competitive accuracy and faster convergence with fewer parameters compared to state-of-the-art methods.", "questions": "- (Line 245-) \"Similar to the previous parametric grid methods, which obtain feature embeddings by interpolating only neighboring vertices, this locality encourages the model to capture high-frequency details by effectively alleviating spectral bias\": Is it possible to mathematically prove that \"this locality encourages the model to capture high-frequency details by effectively alleviating spectral bias\" in some sense? This is just a question out of curiosity.\n\n- Is the title of Section 3.2.2 relevant?\n\n- Why is tanh used in the architecture (it is fine if there is no specific reason)? What about sin or swish activations?\n\n- Is there any issues about mode collapses ($\\mu_i \\rightarrow \\mu_j$ for all $i$ and $j$ and/or $\\sigma_i \\rightarrow 0$ or $\\infty$)?\n\n- I think the update scales of $W$ and $\\theta$ differ significantly from those of $\\mu_i$ and $\\sigma_i$. The optimal updates for $\\mu_i$ and $\\sigma_i$ likely depend on the scale of the simulation. Did the authors encounter any issues related to this?\n\n- It may be off-topic, but is the proposed method applicable to high-dimensional PDEs, such as $d\\gtrsim100$? How significant is the computational cost?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-written and easy to follow. I enjoyed reading it.\n\n- Motivation is clear.\n\n- Comparisons with previous works are well-discussed.\n\n- The proposed model is supported by both theoretical and empirical evidence.\n\n- Error bars are provided, enhancing the reliability of the experimental results.\n\n- Code is provided for reproducibility.", "weaknesses": "- (Minor) Figures should be vector images to avoid pixelization.\n\n- While I found no significant weaknesses, there may be room to further expand the discussion on the novelty and its potential impact on future research.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XDy62NujWb", "reviewer": "Reviewer_uWK5", "summary": "This paper proposes Physics-Informed Gaussians (PIGs), which incorporates Gaussian embeddings in the PINNs framework, to eliminate the inductive biases of neural networks.", "questions": "I may miss something but, is there any reason why choose Gaussian functions as feature embedding? Would other embedding functions fail in the proposed framework?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The strength of this paper is to overcomes the limitations of traditional PINNs.  The proposed method maintains the benefits of PINNs, improves performance in PDE approximation, and can be applied to various PDEs.", "weaknesses": "As mentioned in Discussion and Limitations, theoretical understanding of the convergence properties is lack, although the universality of PIG is discussed in Section 3.3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)