SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We evaluate GRAPHSPLINENETS on five dynamical systems of increasing challenge. Simulated datasets consist of three PDEs (partial differential equations), including Heat, Damped Wave, and Navier-Stokes Equations. The two empirical datasets include the Ocean and Black Sea datasets; the Black Sea introduces significant challenges in scalability in the number of nodes, complex dynamics, irregular boundary, and non-uniform meshing. All the models employ the same structure of encoder-processor-decoder for fair comparisons and the same amount of training data in each testing domain. While inputs of baseline models are directly all of the available data points, inputs of our OSC-based models are only an initialized  $12 \times 12$  collocation point unitary mesh at the initial state and fewer in time sample points. For the Black Sea dataset, we construct a mesh via Delaunay Triangulation with 5000 data points for baselines and 1000 for GRAPHSPLINENETS as done in Lienen and Gunnemann (2022). More details on datasets are available in Appendix B. We value open reproducibility and make our code publicly available<sup>5</sup>.
5.2 Evaluation metrics and baselines
We evaluated our model by calculating the mean square error (MSE) of rollout prediction steps with respectively ground truth for  $t \in [0, 5]$  seconds. We employ relevant baselines in the field of discrete-step graph models for dynamical system predictions. Graph convolution networks (GCNs) (Kipf and Welling, 2016) and GCN with a hybrid multilayer perceptron (MLP) model are employed as baselines in the ablation study. We also compare our approach with one widely used baseline that employs linear interpolation for physics simulations allowing for continuous predictions in space, i.e., GEN (Alet et al., 2019). A similar setup considering the inherent graph structure and employing MPNNs in mesh space is used by (Pfaff et al., 2021) (MeshGraphNet, MGN in our comparisons). We utilize MGN as the MPNN building block for our GRAPHSPLINENETS.
5.3 Quantitative Analysis
We consider a series of ablation models on four datasets to demonstrate the effectiveness of our model components in our approach in multiple aspects. Quantitative results of ablation study models are shown in Table 5.1 [right]. The ablated models are:
- MGN: MeshGraphNet model with 3 message passing layers from Pfaff et al. (2021).
- MGN+OSC(Post): model with 3 message passing layers and only post-processing with the OSC method, i.e., we firstly train a MGN model, then we use the OSC method to collocate the prediction as a final result without end-to-end training.
- MGN+OSC: MGN with OSC-in-the-loop that allows for end-to-end training.
- MGN+OSC+Adaptive: MGN+OSC model that additionally employs our adaptive collocation point sampling strategy.
Post processing vs end-to-end learning We show the effectiveness of end-to-end learning architecture by comparing the MGN+OSC(Post) and MGN+OSC models. Table 5.1 shows that MGN+OSC has a more accurate prediction than MGN+OSC(Post) by more than  $8\%$  percent across datasets. This can be explained by the fact that, since the OSC is applied end-to-end, the error between MGN prediction steps is backpropagated to the message passing layers. In contrast, the model has no way of considering such errors in the post-processing steps.
Adaptive collocation points We further show the effectiveness of the adaptive collocation strategy by comparing the  $\mathrm{MGN + OSC}$  and  $\mathrm{MGN + OSC + Adaptive}$ : Table 5.1 shows that  $\mathrm{MGN + OSC + Adaptive}$  has a more accurate prediction than  $\mathrm{MGN + OSC}$ , i.e., more than  $5\%$  improvement on long rollouts in the Black Sea dataset. Adaptive collocation points encourage those points to move to the most dynamic regions in the domain, which is not only able to place
$^{5}$ https://github.com/kaist-silab/graphsplinenets
Table 5.1: Mean Squared Error propagation and runtime for 5 second rollouts. GRAPHSPLINENETS consistently outperform baselines in both accuracy and runtime. Smaller is better (↓). Best in bold; second underlined.
Dataset	Metric	Baselines	GRAPHSPLINENETS
GCN	GCN+MLP	GEN	MGN	MGN+OSC(Post)	MGN+OSC	MGN+OSC+Adaptive
Heat Equation	MSE (×10-3)	6.87 ± 1.00	5.02 ± 0.89	2.92 ± 0.23	3.01 ± 0.38	1.68 ± 0.18	1.14 ± 0.11	1.07 ± 0.28
Runtime [s]	3.26 ± 0.12	3.02 ± 0.10	6.87 ± 0.10	6.99 ± 0.12	1.52 ± 0.09	1.38 ± 0.10	1.41 ± 0.12
Damped Wave	MSE (×10-1)	10.5 ± 1.65	9.90 ± 1.52	6.49 ± 0.62	7.82 ± 0.88	4.98 ± 0.29	4.60 ± 0.27	4.51 ± 0.31
Runtime [s]	0.95 ± 0.08	0.82 ± 0.07	1.13 ± 0.09	1.38 ± 0.10	0.45 ± 0.05	0.39 ± 0.04	0.42 ± 0.09
Navier Stokes	MSE (×10-1)	4.24 ± 0.95	3.91 ± 0.99	3.45 ± 0.24	3.66 ± 0.33	2.58 ± 0.28	2.21 ± 0.27	2.02 ± 0.30
Runtime [s]	0.91 ± 0.08	0.88 ± 0.07	1.01 ± 0.09	1.21 ± 0.10	0.51 ± 0.05	0.47 ± 0.04	0.49 ± 0.09
Ocean Currents	MSE (×10-1)	6.02 ± 1.12	5.23 ± 0.98	4.55 ± 0.52	4.75 ± 0.61	3.82 ± 0.49	3.61 ± 0.38	3.34 ± 0.44
Runtime [s]	3.56 ± 0.15	3.38 ± 0.13	7.11 ± 0.31	7.15 ± 0.25	1.61 ± 0.12	1.44 ± 0.09	1.57 ± 0.11
Black Sea	MSE (×10-1)	7.73 ± 0.28	7.12 ± 0.35	6.22 ± 0.31	6.41 ± 0.42	4.37 ± 0.33	4.23 ± 0.17	3.91 ± 0.27
Runtime [s]	8.27 ± 0.33	8.11 ± 0.28	12.81 ± 0.19	13.35 ± 0.27	5.87 ± 0.13	5.13 ± 0.21	5.79 ± 0.15
greater attention on hard-to-learn parts in space but also can let the OSC method implicitly develop a better representation of the domain. Empirical quantitative results on the five datasets are shown in Table 5.1. In the heat equation dataset, our approach reduces long-range prediction errors by  $64\%$  with only  $20\%$  of the running time compared with the best baseline model. In the damped wave dataset, our approach reduces errors by  $42\%$  with a  $48\%$  reduction in inference time.
Figure 5.1: Inference time benchmark: the OSC and adaptive collocation points result in a minor overhead; the overall scheme result in a fraction of the baseline running time.
In the Navier-Stokes dataset, our method reduces  $31\%$  long-range prediction errors while requiring  $37\%$  less time to infer solutions compared to the strongest baseline.
Inference times We show the effectiveness of the COLROW algorithm in accelerating the OSC speed by comparing the OSC method with one of the most commonly used algorithms for efficiently solving linear systems and the OSC with the COLROW solver in Fig. 3.2. Our differentiable OSC and adaptive collocation only result in a minor overhead over the graph neural network. The overall scheme is still a fraction of the total in terms of inference time as
shown in Fig. 5.4, while making GRAPHSPLINENETS more accurate.
5.4 Sensitivity Analysis
Interpolation and collocation methods We demonstrate the efficiency of the OSC method by comparing the combination of MGN with different interpolation and collocation methods, including linear interpolation, cubic interpolation, and B-spline collocation methods. These models are implemented in the end-to-end training loop, and we use these methods in both the time and space dimensions. Results are shown in Fig. 5.2 where we measured the mean square error and running time of 3 second rollouts predictions by varying the number of collocation points from  $(2 \times 2)$  to  $(16 \times 16)$ . The model MGN+OSC shows the best accuracy while having a shorter running time. Even though the linear interpolation can be slightly faster than the OSC, it shows a considerable error in the prediction and does not satisfy basic assumptions such as Lipschitz continuity in space.
Prediction Running Time [s]
Figure 5.2: GRAPHSPLINENETS (MGN+OSC+Adaptive) is Pareto-optimal compared to combinations with other interpolation methods.
<sup>6</sup>We employ for our experiments torch.linalg.solve, which uses LU decomposition with partial pivoting and row interchanges. While this method is numerically stable, it still has a  $O(n^3)$  complexity with a  $O(n^2 \log(n))$  lower bound when employing generic sparse solver algorithms.
Number of collocation points We study the effect of the number of collocation points on the 3 second rollout prediction error by testing the MGN,  $\mathrm{MGN + OSC}$ , and  $\mathrm{MGN + OSC + Adaptive}$  models. The MGN is directly trained with the whole domain data.
Figure 5.3: MSE vs the number of collocation points. MGN does not use collocation points, hence constant. The more the collocation points, the better the performance.
We use a different number of collocation points (from  $(2\times 2)$  to  $(28\times 28)$ ) in the MPNN process in the rest two models and then compare the output of the OSC with the whole domain to train. With the increase in the number of collocation points, Fig. 5.3 shows that the  $\mathrm{MGN + OSC}$  and  $\mathrm{MGN + OSC + Adaptive}$  achieve significant improvements in prediction accuracy over the MGN. The  $\mathrm{MGN + OSC + Adaptive}$  has a stable better performance than the  $\mathrm{MGN + OSC}$ , and the improvement is larger when there are fewer collocation points. The reason is that with fewer collocation points, the  $\mathrm{MGN + OSC}$  has insufficient ability to learn the whole domain. With the adaptive collocation point, the  $\mathrm{MGN + OSC + Adaptive}$  can focus on hard-to-learning regions during training to obtain overall better predictions.
Number of rollout steps We show the effectiveness of the OSC method in improving long-range prediction accuracy by comparing the MGN and  $\mathrm{MGN + OSC}$  model. Fig. 5.4 shows the  $\mathrm{MGN + OSC}$  can keep stable in long-range rollouts compare with the MGN.
The reason is that with the OSC, we can use fewer neural network rollout steps to obtain longer-range predictions, which avoids error accumulation during the multi-step rollouts and implicitly learns for compensating integration residual errors. In addition, end-to-end learning lets the neural networks in  $\mathrm{MGN + OSC}$  learn the states between rollout steps, making the prediction stable and accurate.
Number of Message Passing Layers We further showcase the sensitivity to the number of message-passing layers while adopting the same baseline processor for a fair comparison. As Fig. 5.5 shows, a higher number of message-passing layers generally helps in decreasing the MSE at the cost of higher runtime, reflecting find-
ings from prior work such as GNS (Alet et al., 2019) and MGN (Pfaff et al., 2021). Notably, GRAPHSPLINENETS exhibits a Pareto-optimal trade-off curve, outperforming MGN in terms of prediction accuracy and computational efficiency at different numbers of message-passing layers. Downsampling mesh nodes can help in robustness and generalization given the larger spatial extent of communication akin to Fortunato et al. (2022); the OSC plays a crucial part in handling the problem's continuity (e.g., in time) while adaptive collocation effectively helps the model to focus on regions with higher errors thus increasing the model performance.
Figure 5.4: MSE vs rollout steps. Our full model yields stable long rollouts.
Figure 5.5: Pareto plot for MSE and runtime at different # of layers.
MGN. MGN obtain an MSE of  $(4.52 \pm 0.23) \times 10^{-1}$  with a runtime of more than 80 seconds, while GRAPHSPLINENETS remarkably outperforms MGN with an MSE of  $(3.98 \pm 0.27) \times 10^{-1}$  and less than 9 seconds to generate a trajectory - an example of which is shown in Fig. 5.6.
Scalability to Higher Dimensions We showcase GRAPHSPLINENETS on a larger 3D dataset, namely the 3D Navier-Stokes equations dataset from PDEBench (Takamoto et al., 2022) (more details in Appendix B.7). The number of nodes in GraphSplineNets is 4096, which is more than  $4\times$  compared to the Black Sea Dataset. Moreover, the 3D NavierStokes includes a total of 5 variables: density, pressure, and velocities on 3 axes. For a fair comparison, we keep the model structure the same as in other experiments and compare GRAPHSPLINENETS against
Figure 5.6: Rollout visualization of density for 3D Navier-Stokes equations.
5.5 Qualitative analysis
We visualize the prediction of our model with baselines on the Black Sea dataset Fig. 5.7. Our model can accurately predict even with such complex dynamics from real-world data that result in turbulence phenomena while considerably cutting down the computational costs as shown in Table 5.1. We also provide series of qualitative visualizations with different datasets and baselines in Appendix C. Our model has a smoother error distribution and more stable long-range prediction. Thanks to the continuous predictions from GRAPHSPLINENETS, we can simulate high resolutions without needing additional expensive model inference routines, while the other two models can only achieve lower-resolution predictions. Baselines visibly accumulate errors for long-range forecasts; our model can lower the error with smoother and more accurate predictions in space and time.
Figure 5.7: Visualization of rollout forecasts for baselines and GRAPHSPLINENETS on the Black Sea dataset. First row: predictions of the wind speed norm. Second row: mean square error (MSE). Our full model, namely  $\mathrm{MGN + OSC(Adaptive)}$ , generates more accurate predictions than other baseline surrogate models.
In this work, we proposed GRAPHSPLINENETS, a novel approach that employs graph neural networks alongside the orthogonal spline collocation method to enable fast and accurate forecasting of continuous physical processes. We used GNNs to obtain predictions based on coarse spatial and temporal grids, and the OSC method to produce predictions at any location in space and time. Our approach demonstrated the ability to generate high-resolution predictions faster than previous methods, even without explicit prior knowledge of the underlying differential equations. Additionally, we proposed an adaptive collocation sampling strategy that improves the prediction accuracy as the system evolves. We demonstrate how GRAPHSPLINENETS are robust in predicting complex physics in both simulated and real-world data. We believe this work represents a step forward in a new direction in the research area at the intersection of deep learning and dynamical systems that aims at finding fast and accurate learned surrogate models.
A limitation of our approach is that, by utilizing the orthogonal spline collocation (OSC) method for interpolation, we assume some degree of smoothness in the underlying physical processes between collocation points; as such, our approach may struggle when modeling discontinuity points, which would be smoothed-out. Finally, we do note that despite benefits, deep learning models including GRAPHSPLINENETS often lack interpretability. Understanding how the model arrives at its predictions or explaining the underlying physics in a transparent manner can be challenging, hindering the model's adoption in domains where interpretability is crucial.
We want to express our gratitude towards the anonymous reviewers who greatly helped us improve our paper. This work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korean government(MSIT)(2022-0-01032, Development of Collective Collaboration Intelligence Framework for Internet of Autonomous Things).
Learning Efficient Surrogate Dynamic Models with Graph Spline Networks Supplementary Material
A Additional OSC Material 14
A.1 1-D OSC Example 14
A.2 2-D OSC Example 15
A.3 Simple Numerical Example 16
A.4 Numerical Analysis for interpolation and collocation methods 16
B Additional Experimental Details 18
B.1 Models and implementation details 18
B.2 Heat Equation 19
B.3 Wave Equation 19
B.4 2D Incompressible Navier-Stokes 20
B.5Ocean Currents 20
B.6 Black Sea 20
B.7 3D Compressible Navier-Stokes Equations 20
B.8 Hardware and Software 22
C Additional Visualizations 23
A Additional OSC Material
We further illustrate the OSC method by providing numerical examples in this section.
A.1 1-D OSC Example
For simplicity and without loss of generality, we consider the function domain as unit domain  $[0,1]$  and we set  $N = 3, r = 2$ , which means we will use a three-order three-piece function to simulate the 1-D ODE problem. We firstly choose the partition points as  $x_{i}, i = 0, \dots, 3, x_{0} = 0, x_{3} = 1$ . The number of partition points is  $N + 1 = 4$ . Distance between partition points can be fixed or not fixed. Then, based on the Gauss-Legendre quadrature rule, we choose the collocation points. The number of collocation point within one partition is  $r - 1 = 1$ , so we have in total  $N \times (r - 1) = 3$  collocation points  $\xi_{i}, i = 0, \dots, 3$ .
After getting partition points and collocation points, we will construct the simulator. Here we have three partitions; in each partition, we assign a 2 order polynomial
$$
a _ {0, 0} + a _ {0, 1} x + a _ {0, 2} x ^ {2}, x \in [ x _ {0}, x _ {1} ] \tag {4a}
$$
$$
a _ {1, 0} + a _ {1, 1} x + a _ {1, 2} x ^ {2}, x \in [ x _ {1}, x _ {2} ] \tag {4b}
$$
$$
a _ {2, 0} + a _ {2, 1} x + a _ {2, 2} x ^ {2}, x \in [ x _ {2}, x _ {3} ] \tag {4c}
$$
Note that these three polynomials should be  $C^1$  continuous at the connecting points, i.e., partition points within the domain. For example, Eq. (4a) and Eq. (4b) should be continuous at  $x_1$ , then we
can get two equations
$$
\left\{ \begin{array}{l l} a _ {0, 0} + a _ {0, 1} x _ {1} + a _ {0, 2} x _ {1} ^ {2} & = a _ {1, 0} + a _ {1, 1} x _ {1} + a _ {1, 2} x _ {1} ^ {2} \\ 0 + a _ {0, 1} + 2 a _ {0, 2} x _ {1} & = 0 + a _ {1, 1} + 2 a _ {1, 2} x _ {1} \end{array} \right. \tag {5}
$$
For boundary condition
$$
\hat {u} (x) = \left\{ \begin{array}{l} b _ {1}, x = x _ {0} \\ b _ {2}, x = x _ {3} \end{array} \right. \tag {6}
$$
we can also get two equations
$$
\left\{ \begin{array}{l l} a _ {0, 0} + 0 + 0 & = b _ {1} \\ a _ {1, 0} + a _ {1, 1} + a _ {1, 2} & = b _ {2} \end{array} \right. \tag {7}
$$
Let us summarize the equations we got so far. Firstly, our undefined polynomials have  $N \times (r + 1) = 9$  parameters. The  $C^1$  continuous condition will create  $(N - 1) \times 2 = 4$  equations, and the boundary condition will create 2 equations. Then we have  $N \times (r - 1)$  collocation points. For each collocation point, we substitute it to polynomials to get an equation. For example, if the ODE is
$$
\hat {u} (x) + \hat {u} ^ {\prime} (x) = f (x), x \in [ 0, 1 ] \tag {8}
$$
By substituting collocation point  $\xi_0$  into the equation, we can get
$$
\begin{array}{l} \hat {u} (\xi_ {0}) + \hat {u} ^ {\prime} (\xi_ {0}) = f (\xi_ {0}) \\ \Longrightarrow a _ {0, 0} + a _ {0, 1} \xi_ {0} + a _ {0, 2} \xi_ {0} ^ {2} + a _ {0, 1} + 2 a _ {0, 2} \xi_ {0} = f (\xi_ {0}) \tag {9} \\ \Longrightarrow a _ {0, 0} + a _ {0, 1} \left(\xi_ {0} + 1\right) + a _ {0, 2} \left(\xi_ {0} ^ {2} + 2 \xi_ {0}\right) = f \left(\xi_ {0}\right) \\ \end{array}
$$
Now we can know that the number of equations can meet with the degree of freedom of polynomials
$$
\overbrace {(r + 1) \times N} ^ {\text {P a r a m e t e r s}} = \overbrace {2} ^ {\text {B o u n d a r y}} + \overbrace {(N - 1) \times 2} ^ {C ^ {1} \text {C o n t i n u o u s}} + \overbrace {N \times (r - 1)} ^ {\text {C o l l o c a t i o n}} \tag {10}
$$
In this example, generated equations will be constructed to an algebra problem  $\mathbf{A}\mathbf{a} = \mathbf{f}$  where the weight matrix is an ABD matrix.
Figure A.1: Visualization of an ABD matrix.
$$
\mathbf {A} = \left[ \begin{array}{c c c c c c} 1 & 0 & 0 & 0 & 0 & 0 \\ 1 & \xi_ {0} + 1 & \xi_ {0} ^ {2} + 2 \xi_ {0} & 0 & 0 & 0 \\ 1 & x _ {1} & x _ {1} ^ {2} & - 1 & - x _ {1} & - x _ {1} ^ {2} \\ 0 & 1 & 2 x _ {1} & 0 & - 1 & - 2 x _ {1} \\ 0 & 0 & 0 & 1 & \xi_ {1} + 1 & \xi_ {1} ^ {2} + 2 \xi_ {1} \\ 0 & 0 & 0 & 1 & 1 & 1 \end{array} \right], \tag {11a}
$$
$$
\mathbf {a} = \left[ \begin{array}{l} a _ {0, 0} \\ a _ {0, 1} \\ a _ {0, 2} \\ a _ {1, 0} \\ a _ {1, 1} \\ a _ {1, 2} \end{array} \right], \mathbf {f} = \left[ \begin{array}{c} b _ {1} \\ f (\xi_ {0}) \\ 0 \\ 0 \\ f (\xi_ {1}) \\ b _ {2} \end{array} \right]. \tag {11b}
$$
By solving this problem, we can obtain the simulation results. Note that we can employ the COL-ROW sparse solver to solve the system efficiently (see § 3.4).
A.2 2-D OSC Example
For simplicity and without loss of generality, we consider the function domain as unit domain  $[0,1]\times [0,1]$ , and set  $N_{x} = N_{y} = 2$ ,  $r = 3$ . Partition points and collocation points selection is similar to the 1-D OSC method; we have  $N^2\times (r - 1)^2 = 16$  collocation points in total. For simplicity, we note the partition points at two dimensions to be the same, i.e.,  $x_{i},i = 0,1,2$ . Unlike the 1-D OSC
method, we choose Hermite bases to describe as the simulator, which keeps  $C^1$  continuous. As a case, the base function at point  $x_1$  would be
$$
H _ {1} (x) = f _ {1} (x) + g _ {1} (x)
$$
$$
f _ {1} (x) = \left\{ \begin{array}{l} \frac {\left(x - x _ {0}\right) \left(x _ {1} - x\right) ^ {2}}{\left(x _ {1} - x _ {0}\right) ^ {2}}, x \in \left(x _ {0}, x _ {1} \right] \\ \frac {\left(x - x _ {2}\right) \left(x - x _ {1}\right) ^ {2}}{\left(x _ {2} - x _ {1}\right) ^ {2}}, x \in \left(x _ {1}, x _ {2} \right] \end{array} \right. \tag {12}
$$
$$
g _ {1} (x) = \left\{ \begin{array}{l} + \frac {[ (x _ {1} - x _ {0}) + 2 (x _ {1} - x _ {0}) ] (x - x _ {0}) ^ {2}}{(x _ {1} - x _ {0}) ^ {3}}, x \in (x _ {0}, x _ {1} ] \\ + \frac {[ (x _ {2} - x _ {1}) + 2 (x - x _ {1}) ] (x _ {2} - x) ^ {2}}{(x _ {2} - x _ {1}) ^ {3}}, x \in (x _ {1}, x _ {2} ] \end{array} \right.
$$
We separately assign parameters to basis functions, i.e.  $H_{1}(x) = a_{1,i}f_{1}(x) + b_{1,i}g_{1}(x)$  for  $x$  variable in  $[x_0,x_1]\times [y_{i - 1},y_i]$  partition. Then the polynomial in a partition is the multiple combinations of base functions of two dimensions. For example, the polynomial in the partition  $[x_0,x_1]\times [y_0,y_1]$  is
$$
\left[ a _ {0, 1} ^ {x} f _ {0} (x) + b _ {0, 1} ^ {x} g _ {0} (x) + a _ {1, 1} ^ {x} f _ {1} (x) + b _ {1, 1} ^ {x} g _ {1} (x) \right]
$$
$$
\times \left[ a _ {0, 1} ^ {y} f _ {0} (y) + b _ {0, 1} ^ {y} g _ {0} (y) + a _ {1, 1} ^ {y} f _ {1} (y) + b _ {1, 1} ^ {y} g _ {1} (y) \right] \tag {13}
$$
Now we consider the freedom degree of these polynomials. From definition, we have  $2n(r - 1)(n + 1) = 24$  parameter. Considering boundary conditions, we have  $24 - 4 \times N = 16$  parameters. The number is equal with collocation points  $N^2 \times (r - 1)^2$ , which means we can get an algebra equation by substituting collocation points. Solving this equation, we can get the simulator parameters.
We can similarly have multiple basis functions and set parameters to the simulation result for the higher dimension OSC method. And then, select partition points and collocation points using the same strategy as the 2-D OSC method. The rest algebra equation generating and solving equations parts will not be different.
A.3 Simple Numerical Example
We set  $N = 3, r = 3$  to simulate the problem
$$
\left\{ \begin{array}{l} u + u ^ {\prime} = \sin (2 \pi x) + 2 \pi \cos (2 \pi x) \\ u (0) = 0 \\ u (1) = 0 \end{array} \right. \tag {14}
$$
we can get a simulation solution as follows, which is visualized in Fig. A.2.
$$
\hat {u} (x) = \left\{ \begin{array}{l} 6. 2 x - 0. 4 x ^ {2} - 3 1. 4 x ^ {3}, x \in [ 0, 1 / 3) \\ 1. 5 + 1. 6 x - 1 3. 8 x ^ {2} + 9 x ^ {3}, x \in [ 1 / 3, 2 / 3) \\ 2 8. 5 - 1 0 0 x + 1 0 8. 5 x ^ {2} - 3 7 x ^ {3}, x \in [ 2 / 3, 1 ] \end{array} \right. \tag {15}
$$
Figure A.2: Visualization of an OSC solution.
A.4 Numerical Analysis for interpolation and collocation methods
We compared the OSC with linear, bilinear, 0-D cubic, and 2-D cubic interpolation methods on four types of problems: 1-D linear, 1-D non-linear, 2-D linear, and 2-D non-linear problems. In these experiments, we tested different simulator orders of the OSC method. For example, we set the order of the simulator to 4 for 1-D linear problem and 2 for 2-D linear problem. When the order of the simulator matches the polynomial order of the real solution, OSC can directly find the real solution. For non-linear problems, increasing the order of the simulator would be an ideal way to get lower
loss. For example, we set the order of the simulator to 4 for 1-D non-linear problem and 5 for 2-D non-linear problem. Thanks to the efficient calculation of OSC, even though we use higher-order polynomials to simulate, we use less running time to get results.