SlowGuess/NeurIPS_2023_OCR huggingface dataset:
We perform 1000 experiments for each  $n$ . As losses for optimal solutions can be on widely different scales, we report the relative error between the loss obtained by CoLLie and the loss obtained by the algorithm based on CVXOPT, which seems to always find the optimal solution.
The time is measured on a single computer with an Intel Core i5-6500 CPU (4 cores) and 16GB of RAM.
E.1 Ablated D-CIPHER
The ablated version uses the standard MSE loss with estimated derivatives and thus solves the following optimization problem:
$$
\min  _ {g} \min  _ {| | \boldsymbol {\beta} | | _ {1}} \sum_ {d = 1} ^ {D} \sum_ {\boldsymbol {x} \in \mathcal {G}} \left(\sum_ {p = 1} ^ {P} \beta_ {p} \hat {\mathcal {E}} _ {p} [ \boldsymbol {v} ^ {(d)} ] (\boldsymbol {x}) - g (\boldsymbol {x}, \boldsymbol {v} ^ {(d)} (\boldsymbol {x}))\right) ^ {2} \tag {58}
$$
where  $\pmb{v}^{(d)}$  is the observed field,  $\mathcal{G}$  is the sampling grid, and  $\hat{\mathcal{E}}_p[\pmb{v}^{(d)}](\pmb{x})$  requires derivative estimation.
The ablated version uses the same symbolic regression algorithm to search over closed-form  $g$  and CoLLie for the inner optimization.
E.2 Implementation
Step 1. For the homogeneous heat equation and Burgers' equation we use dictionary  $\mathcal{Q} = \{u, \partial_t u, \partial_x u, \partial_x(u^2), \partial_x^2 u, \partial_x^2 (u^2)\}$ . For Kuramoto-Sivashinsky equation we use dictionary  $\mathcal{Q} = \{u, \partial_t u, \partial_x u, \partial_x(u^2), \partial_x^2 u, \partial_x^2 (u^2), \partial_x^3 u, \partial_x^3 (u^2), \partial_x^4 u, \partial_x^4 (u^2)\}$ . For the damped and forced harmonic oscillator we use the dictionary  $\{\partial_t, \partial_t^2\}$ , and for the wave and heat equations, we use  $\{\partial_t, \partial_x, \partial_t^2, \partial_t \partial_x, \partial_x^2\}$ .
Step 2. Field estimation is performed using the Gaussian Process Regression from the Python library scikit-learn [39]. The kernel is chosen to be the RBF kernel [60] with an added White kernel to account for noise. The observed field is initially standardized by subtracting the mean and dividing by the standard deviation. Then the GaussianProcessRegressor is fitted to the data. The estimated fields are generated by predicting the values of a trained Gaussian Process on a full integration grid
and then scaling back to their original range (by multiplying by the standard deviation and adding the mean).
Step 3. The search over the closed-form expression is performed using the symbolic regression library gplearn [52]. We use a custom fitness function that solves the inner optimization problem in Equation 14. This inner optimization is performed by CoLLie (Section 7). The integration is performed using Riemann sums.
Ablated version of D-CIPHER. The derivative estimation is performed by first fitting a Gaussian process (in the same way as in Step 2) and then using the finite difference to estimate the derivative in one of the coordinates for all points in the sampling grid. To obtain higher-order derivatives, a Gaussian process is fitted again and the derivative is once again calculated using the finite difference (possibly in a different direction than the first time).
E.3 Hyperparameters
Gaussian process regression. The kernel parameters of the Gaussian Process are automatically adjusted during training. The default bounds of the length scale of the RBF kernel and the noise level of the White kernel are used, i.e.,  $(1e - 5, 1e5)$ .
GPlearn. We do not perform parameter tuning for the gplearn library and use the same parameters as in D-CODE [42] except for the parsimony coefficient and the number of generations.
Table 6: Hyperparameters used in gplearn
Hyperparameter	Value
population size	15000
tournament size	20
p crossover	0.6903
p subtree mutation	0.1330
p hoist mutation	0.0361
p point mutation	0.0905
generations	20 and 30
The number of generations is chosen to be 30 for the damped and forced harmonic oscillator and 20 for the inhomogeneous heat and wave equations.
Please check [52] for the detailed description of these parameters.
We modify the implementation of the parsimony coefficient. The standard implementation adds to the loss the length of the equation multiplied by the parsimony coefficient. In our implementation, we increase the loss by the parsimony coefficient. This modification is performed because for different experiments we record the loss on widely different scales. To prevent tuning this parameter for every experimental setting we introduce a penalty that can work on different scales. The parsimony coefficient is chosen manually by performing experiments for a few values. The value used in the experiments is 0.05.
The set of allowed mathematical operations is:  $\{+, -, \times, \div, \sin, \exp, \log\}$
We want to emphasize that we use the same configuration of gplearn in D-CIPHER and its ablated version.
Integration and number of testing functions. For the damped and forced harmonic oscillator we use 10 testing functions and the integration step 0.01. For the inhomogeneous heat and wave equations we use 100 testing functions and integrate on a grid with steps  $\delta t = 0.01$  and  $\delta x = 0.01$ .
Derivative estimation in the ablated version of D-CIPHER. The Gaussian process is configured the same way as described above. The interval used in the finite difference method to estimate the derivative was chosen to be:  $10^{-3}$ .
E.4 Choice of equations
Equations used in Section 8.1 are canonical equations from physics that often appear in other works about PDE discovery [47]. The homogeneous heat equation is a second-order PDE that models how heat diffuses through a region. It contains the dissipative term  $\partial_x^2 u$ . Burgers' equation is second-order PDE used, for instance, in fluid mechanics or nonlinear acoustics [14]. It contains the advection term  $u\partial_x u$  and the diffusion term that prevents shock formation. Kuramoto-Sivashinsky equation is a fourth-order PDE used in modelling reaction-diffusion systems [27] and is known for its chaotic behavior [22].
In Section 8.2, we chose equations of physical significance that have an interesting  $\partial$ -free part that is not a linear combination (as discussed in Section 3). That makes them impossible to discover by the current methods. A forced and damped harmonic oscillator is a second-order ODE. Although D-CODE [42] can discover any closed-form first-order ODE, it cannot be used to discover second-order ODEs. Thus there is currently no algorithm capable of discovering this equation. Inhomogeneous heat and wave equations are second-order PDEs where the  $\partial$ -free part is a source (of heat and wave respectively). Moreover, the wave equation does not have the standard evolution form, as it does not involve the  $\partial_t$  term. Thus even without the source term, most of the current methods cannot be applied directly to discover this equation.
E.5 Data generation
Homogeneous heat equation. The fields were generated by solving the equation  $\partial_t u - \theta_1\partial_x^2 u = 0$  ( $\theta_{1} = 0.25$ ) with Neumann boundary conditions  $\partial_x u(t,0) = \partial_x u(t,X) = 0$  and an initial condition  $u(0,x) = u_0(x)$ , where  $u_{0}$  is randomly sampled from a Gaussian process. The equation is solved using the implicit BTCS scheme [25] with steps  $\delta t = 0.001$  and  $\delta x = 0.001$ . The observed field is generated by sampling  $(t,x)\in [0,T]\times [0,X]$ , evaluating the true field  $u(t,x)$  and adding Gaussian noise.  $T = 2$  and  $X = 2$  are used in the experiments.
Burger's equation. The fields are computed by solving  $\partial_t u + u\partial_x u - \theta_1\partial_x^2 U = 0$  ( $\theta_{1} = 0.2$ ) with an initial condition  $u(0,x) = u_0(x)$ , where  $u_{0}$  is randomly sampled from a Gaussian process, and with Dirichlet boundary conditions  $u(t,0) = u_0(0)$ ,  $u(t,X) = u_0(X)$ . The equation is solved using Crank-Nicolson scheme [57] with steps  $\delta t = 0.002$  and  $\delta x = 0.002$ . The observed field is generated by sampling  $(t,x)\in [0,T]\times [0,X]$ , evaluating the field  $u(t,x)$  and adding Gaussian noise.  $T = 2$  and  $X = 2$  are used in the experiments.
Kuramoto-Sivashinsky equation. The solution is the same as the one used in [47]. The observed field is generated by sampling  $(t,x)\in [0,T]\times [0,X]$ , evaluating the field  $u(t,x)$  and adding Gaussian noise.  $T = 100$  and  $X = 100$  are used in the experiments.
Damped and forced harmonic oscillator. The true fields are created by analytically solving the equation  $\partial_t^2 u(t) + 2\theta_1\theta_2\partial_t u(t) + \theta_2^2 u(t) = \theta_3\sin (\theta_4t)$ , where  $\theta_{1} = 0.5, \theta_{2} = 4.0, \theta_{3} = 5.0, \theta_{4} = 3.0$ , with random initial conditions for  $u(0)$  and  $\partial_t u(0)$ . The observed fields are then created by sampling  $t \in [0, T]$ , evaluating  $u(t)$ , and adding Gaussian noise.  $T = 2$  was used in the experiments.
Inhomogeneous heat equation. The true fields are computed by solving  $\partial_t u(t,x) - \theta_1\partial_x^2 u(t,x) = \theta_2e^{\theta_3t}$ , where  $\theta_{1} = 0.25, \theta_{2} = 1.25, \theta_{3} = 1.8$ , with Neumann boundary conditions  $\partial_x u(t,0) = \partial_x u(t,X) = 0$  and an initial condition  $u(0,x) = u_0(x)$ , where  $u_{0}$  is randomly sampled from a Gaussian process. The equation is solved using the implicit BTCS scheme [25] with steps  $\delta t = 0.001$  and  $\delta x = 0.001$ . The observed field is generated by sampling  $(t,x) \in [0,T] \times [0,X]$ , evaluating the true field  $u(t,x)$  and adding Gaussian noise.  $T = 2$  and  $X = 2$  are used in the experiments.
Inhomogeneous wave equation. The true fields are computed by solving  $\partial_t^2 u(t,x) - \theta_1\partial_x^2 u(t,x) = \theta_2e^t\sin (\theta_3t)$ , where  $\theta_{1} = 1.0,\theta_{2} = 2.0,\theta_{3} = 3.0$ , with Dirichlet boundary conditions  $u(t,0) = u_0(0)$ ,  $u(t,X) = u_0(X)$ , where  $u_{0}$  is randomly sampled from a Gaussian process and specifies the initial condition  $u(0,x) = u_0(x)$ . The equation is solved using the Implicit Difference Method [38] with steps  $\delta t = 0.001$  and  $\delta x = 0.001$ . The observed field was generated by sampling  $(t,x)\in [0,T]\times [0,X]$ , evaluating the true field  $u(t,x)$  and adding Gaussian noise.  $T = 2$  and  $X = 2$  were used in the experiments.
E.6 Experimental settings
Homogeneous heat equation
Noise ratio  $(\sigma_R)$  .. 0.001, 0.01, 0.1
Number of samples  $(D)$  : 10
Domain  $(\Omega)$  ..  $[0,2]\times [0.2]$
Grid  $(\mathcal{G})$  ..  $\{0,0.07,\dots ,2\} \times \{0,0.07,\dots ,2\}$
Burgers' equation
Grid  $(\mathcal{G})$  ..  $\{0,0.1,\dots ,2\} \times \{0,0.1,\dots ,2\}$
Kuramoto-Sivashinsky equation
Number of samples  $(D)$ : 1
Domain  $(\Omega)$  ..  $[0,100]\times [0.100]$
Grid  $(\mathcal{G})$  ..  $\{0,1.67,\dots ,100\} \times \{0,1.67,\dots ,100\}$
Damped and forced harmonic oscillator
Default values are in bold.
Noise ratio  $(\sigma_R)$  .. 0.001, 0.005, 0.01, 0.1, 0.2, 0.5
Number of samples  $(D)$  : 1, 2, 5, 10, 15
Domain  $(\Omega)$  : [0, 2]
Grid  $(\mathcal{G})$  ..  $\{0,0.08,\dots ,2\}$ $\{\mathbf{0},\mathbf{0.1},\ldots ,\mathbf{2}\}$ $\{0,0.13,\dots ,2\}$ $\{0,0.2,\dots ,2\}$ $\{0,0.4,\dots ,2\}$
Inhomogeneous heat equation
Noise ratio  $(\sigma_R)$  .. 0.05, 0.1, 0.2
Inhomogeneous wave equation
Noise ratio  $(\sigma_R)$  .. 0.001, 0.01, 0.015
E.7 Benchmarks
We use the implementation of PDE-FIND and WSINDy from the PySINDY library [24, 16].
An important hyperparameter in PDE-FIND and WSINDy is the library  $\Theta$  used. We impose that  $\mathcal{Q}$  and  $\Theta \cup \{\partial_t u\}$  have the same number of elements. Moreover, solutions given by PDE-FIND and WSINDy are scaled to have the L1 norm equal to 1. Both of these measures are undertaken to ensure that RMSE error is comparable between the algorithms.
For the homogeneous heat equation and Burgers' equation we use a library
$$
\Theta = \left\{u, \partial_ {x}, \partial_ {x} ^ {2} u, u \partial_ {x} u, u \partial_ {x} ^ {2} u \right\}.
$$
For the Kuramoto-Sivashinsky equation, we use a library
$$
\Theta = \left\{u, \partial_ {x}, \partial_ {x} ^ {2} u, \partial_ {x} ^ {3} u, \partial_ {x} ^ {4} u, u \partial_ {x} u, u \partial_ {x} ^ {2} u, u \partial_ {x} ^ {3}, u \partial_ {x} ^ {4} \right\}.
$$
In the experiments, we have not optimized for the derivative-free part as it is identically equal to 0 in all equations.
E.8 Correct functional form
To measure success probability we need to establish whether two closed-form functions match. The previous approach [42] considered their functional forms, i.e., expressions where all numeric constants are replaced by placeholders. By this measure, functions  $\sin(3x)$  and  $\sin(3.5x)$  match as they have the same functional form  $\sin(Cx)$ , where  $C$  is a placeholder.
However, this definition is quite restrictive because functions  $\sin(3x)$ ,  $\sin(3x) + 0.001$ ,  $1.001\sin(3x)$ , and  $\sin(3x + 0.001)$  all have different functional forms.
We consider it an open challenge to design a good metric that would meaningfully reflect whether the correct equation is discovered. We propose the following.
For a target function  $f$ , we consider its augmented form  $\tilde{f}$ , defined as  $\tilde{f}(x) = C_1 f(C_3 x + C_4) + C_2$ , where all  $C_i$  are placeholders. Then all numeric constants are turned into placeholders as well. In the end, we combine the constants. For instance,  $C_1 + C_2$  becomes just  $C_3$ .
As an example, let us consider a function  $f(x) = 1.3e^{2x}$ . The augmented functional form is created in the following way:
1. Augment:  $C_1 \times 1.3e^{2 \times (C_3 \times x + C_4)} + C_2$
2. Replace:  $C_1 \times C_5 e^{C_6 \times (C_3 \times x + C_4)} + C_2$
3. Combine:  $C_1e^{C_3x + C_4} + C_2$
We perform this procedure for the target function. We can now take the standard functional form of the candidate function and check whether it matches the augmented functional form of the target function, taking into account that some of the constants might not be present in the candidate expression.
To aid in this procedure, we use a Python library for symbolic mathematics, SymPy [37].
E.9 Computation time
The average computation time for a single experiment with the damped and forced harmonic oscillator is 281 seconds with a standard error of 4.5 seconds. The average computation time for a single experiment with an inhomogeneous heat equation is 68 minutes with a standard error of 38 seconds. This time is measured on a single computer with an Intel Core i5-6500 CPU (4 cores) and 16GB of RAM.
The experiments are run simultaneously on 5 computers like the one described above. The total time for all experiments (all seeds, all equations, all experimental settings, and both versions of D-CIPHER) is 65 hours.
E.10 Licenses
The licenses of the software used in this work are presented in Table 7
F Additional experiments and discussion
F.1 Sharpe-Lotka-McKendrick model
We test D-CIPHER on a population model called the Sharpe-Lotka-McKendrick model [59]. The model is described by the following equation
$$
\partial_ {t} u (t, a) + \partial_ {a} u (t, a) + m (a) u (t, a) = 0 \tag {59}
$$
Table 7: Software used and their licenses
Software	License
gplearn	BSD 3-Clause "New" or "Revised" License
cvxopt	GNU General Public License
cvxpy	Apache License
sympy	New BSD License
scikit-learn	BSD 3-Clause "New" or "Revised" License
numpy	liberal BSD license
pandas	BSD 3-Clause "New" or "Revised" License
scipy	liberal BSD license
python	Zero-Clause BSD license
mysindy	MIT License
where  $m(a)$  is age-specific mortality rate. We choose  $m(a) = 2e^{\theta a}$ . We set  $\theta = 1.5$  in the experiments.
We note that the derivative-free part of the target PDE cannot be expressed as a linear combination of functions from a finite dictionary if the parameters are not known a priori. D-CIPHER is uniquely positioned among other discovery algorithms as the only technique that can recover any mortality rate that can be represented as a closed-form expression. We show a comparison between D-CIPHER and the Ablated D-CIPHER in Table 8
Table 8: Simulation results for the Sharpe-Lotka-McKendrick model. We report the success probability of discovering the  $\partial$ -free part and the Average RMSE of the  $\partial$ -bound part. Standard deviations are shown in brackets
Method	Success Probability	Average RMSE)
σR=0.001	0.01	σR=0.001	0.01
D-CIPHER	0.6 (0.15)	0.5 (0.16)	0.007 (0.0008)	0.008 (0.0011)
Abl. D-CIPHER	0.2 (0.13)	0.2 (0.13)	0.017 (0.0009)	0.017 (0.0008)
F.2 Discovering systems
Discovering systems of PDEs is a much harder problem than discovering a single PDE. One of the issues is the fact that we would call indeterminism. It follows from the following fact. If a vector field  $\pmb{u}$  is a solution to two differential equations  $f_{1}$  and  $f_{2}$ , i.e.,
$$
f _ {1} (\boldsymbol {x}, \boldsymbol {u} ^ {(d)} (\boldsymbol {x}), \partial^ {[ K ]} \boldsymbol {u} ^ {(d)} (\boldsymbol {x})) = 0 \forall \boldsymbol {x} \in \Omega \tag {60}
$$
$$
f _ {2} (\boldsymbol {x}, \boldsymbol {u} ^ {(d)} (\boldsymbol {x}), \partial^ {[ K ]} \boldsymbol {u} ^ {(d)} (\boldsymbol {x})) = 0 \forall \boldsymbol {x} \in \Omega
$$
then it is also a solution to any linear combination of these equations, i.e.,
$$
\lambda_ {1} \times f _ {1} (\boldsymbol {x}, \boldsymbol {u} ^ {(d)} (\boldsymbol {x}), \partial^ {[ K ]} \boldsymbol {u} ^ {(d)} (\boldsymbol {x})) + \lambda_ {2} \times f _ {2} (\boldsymbol {x}, \boldsymbol {u} ^ {(d)} (\boldsymbol {x}), \partial^ {[ K ]} \boldsymbol {u} ^ {(d)} (\boldsymbol {x})) = 0 \forall \boldsymbol {x} \in \Omega \tag {61}
$$
for any  $\lambda_1, \lambda_2 \in \mathbb{R}$ . Moreover, equations can sometimes be differentiated to yield more equations.
Let us take as an example the Cauchy-Riemann equations defined as:
$$
\partial_ {x} u _ {1} - \partial_ {y} u _ {2} = 0 \partial_ {x} u _ {2} + \partial_ {y} u _ {1} = 0 \tag {62}
$$
Let us assume that we have a true vector field  $(u_{1}, u_{2})$  that satisfies both equations. Then the following equations are also satisfied
$$
\partial_ {x} u _ {1} - \partial_ {y} u _ {2} + \partial_ {x} u _ {2} + \partial_ {y} u _ {1} = 0 \tag {63}
$$
$$
\partial_ {x} u _ {1} - \partial_ {y} u _ {2} - \partial_ {x} u _ {2} - \partial_ {y} u _ {1} = 0
$$
We can also differentiate the Cauchy-Riemann equations to arrive at the Laplacian equations for  $u_{1}$  and  $u_{2}$ .
$$
\partial_ {x} ^ {2} u _ {1} + \partial_ {y} ^ {2} u _ {1} = 0 \tag {64}
$$
$$
\partial_ {x} ^ {2} u _ {2} + \partial_ {y} ^ {2} u _ {2} = 0
$$
We could also combine the first-order equations with second-order equations or consider even higher-order derivatives. Although all these equations are compatible with our vector field  $(u_{1}, u_{2})$ , not all of them are equally desirable to discover. That is why we believe that in any algorithm for discovering systems of differential equations substantial expert knowledge or inductive biases have to be encoded to guide the algorithm into the right equations.
Current methods do not consider systems of equations or consider a system of equations of a very particular form. In the latter case, each equation models a derivative with respect to time of a different scalar field. The system is assumed to look like this:
$$
\begin{array}{l} \partial_ {t} u _ {1} = f _ {1} (\boldsymbol {x}, \boldsymbol {u} (\boldsymbol {x}), \partial^ {[ K ]} \boldsymbol {u} (\boldsymbol {x})) \\ \partial_ {t} u _ {2} = f _ {2} (\boldsymbol {x}, \boldsymbol {u} (\boldsymbol {x}), \partial^ {[ K ]} \boldsymbol {u} (\boldsymbol {x})) \tag {65} \\ \end{array}
$$
$$
\partial_ {t} u _ {L} = f _ {L} (\boldsymbol {x}, \boldsymbol {u} (\boldsymbol {x}), \partial^ {[ K ]} \boldsymbol {u} (\boldsymbol {x}))
$$
In addition, the LHS of these equations is often assumed to only contain spatial derivatives.
D-CIPHER can be used to discover some systems of differential equations if enough prior knowledge is provided in the choice of the dictionary  $\mathcal{Q}$ . Moreover, the discovered equations are not required to have a particular evolution form as is the case in current approaches.
Firstly, we note that D-CODE [42] has been shown to discover a system of equations that looks like Equations 65 when all equations are first-order ODEs. As D-CIPHER reduces to D-CODE when applied to first-order ODEs, it is also capable of discovering such a system.
We demonstrate that D-CIPHER is able to discover both Cauchy-Riemann equations if we use two different dictionaries. Each of the dictionaries yields a different equation. Additionally, it is a well-known fact that if a vector field satisfies Cauchy-Riemann equations then the constituent scalar fields are harmonic, i.e., they satisfy Laplace's equation. Based on the same dataset we are also able to discover both Laplace's equations given another set of two different dictionaries. We note that Laplace's equation does not contain  $\partial_x$  term, so most of the current methods cannot be directly applied to discover this equation. The results are presented in Figure 7. The dictionaries used to discover each of the equations are the following.
For  $\partial_x u_1 - \partial_y u_2 = 0$  we use  $\mathcal{Q}_1 = \{\partial_x u_1, \partial_y u_2, \partial_x^2 u_1, \partial_y u_2\}$
For  $\partial_x u_2 + \partial_y u_1 = 0$  we use  $\mathcal{Q}_2 = \{\partial_x u_2, \partial_y u_1, \partial_x^2 u_2, \partial_y^2 u_1\}$
For  $\partial_x^2 u_1 + \partial_y^2 u_1 = 0$  we use  $\mathcal{Q}_3 = \{\partial_xu_1,\partial_yu_1,\partial_x^2 u_1,\partial_y^2 u_1\}$
For  $\partial_x^2 u_2 + \partial_y^2 u_2 = 0$  we use  $\mathcal{Q}_4 = \{\partial_xu_2,\partial_yu_2,\partial_x^2 u_2,\partial_y^2 u_2\}$
F.3 Increasing the size of the dictionary  $\mathcal{Q}$
We test D-CIPHER on the Sharpe-Lotka-McKendrick model (Equation 59) with different dictionaries. We start with a small dictionary  $\mathcal{Q}_1 = \{\partial_t u, \partial_a u\}$ , and we create every new dictionary from the previous one by adding one more extended derivative. The final dictionary contains 10 elements,  $\mathcal{Q}_{10} = \{\partial_t u, \partial_a u, \partial_a^2 u, \partial_t^2 u, \partial_t \partial_a u, \partial_a(u^2), \partial_a^2(u^2), \partial_t(u^2), \partial_t^2(u^2), \partial_t(u^3), \partial_a(u^3)\}$ . The Average RMSE of the  $\partial$ -bound part is shown in Figure 8. We do not observe any increase in average error. Note that in these experiments, we just focus on the  $\partial$ -bound part and do not optimize the  $\partial$ -free part. The relationship between the computation time and the size of the dictionary is represented in Figure 9.
F.4 Computational complexity
We want to emphasize that PDE discovery is not a time-critical application (usually this process is performed manually by scientists) and we believe D-CIPHER's computation time is acceptable for such a task. In this section, we describe which parts of the algorithms are most computationally intensive.
Figure 7: Simulation results for both Cauchy-Riemann equations and two Laplace's equations. We report the Average RMSE of the  $\partial$ -bound part in different noise settings
Figure 8: Simulation results for Sharpe-Lotka-McKendrick model. We report the Average RMSE of the  $\partial$ -bound part for different sizes of the dictionary  $\mathcal{Q}$
Computation in D-CIPHER is performed in Step 2 and Step 3.
Step 2. Computational complexity of Step 2 depends on the choice of the smoothing algorithm. We want to emphasize that the user can use any smoothing algorithm based on their domain knowledge and experience, including spline regression, LOWESS, and Kalman filters. Gaussian Process has time complexity  $\mathcal{O}(n^3)$  where  $n$  is the number of data points in a grid  $\mathcal{G}$ . D-CIPHER is specifically designed to work for sparse and noisy data, so we have not encountered major computational issues while performing Gaussian process regression. We also note that significant progress has been made in adapting Gaussian processes for datasets with many data points [28].