Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper proposes a Bayesian spline learning framework to discover Nonlinear dynamics (ODE or PDE) using sparse data, which is interpretable and is able to quantify the uncertainty. Technical proposals are made to implement this strategy.
strengths_and_weaknesses: The strengths of the paper are listed below:
1. The experiment results seem promising.
2. The proposed methodology is interpretable and is able to quantify the uncertainty.

The weakness of the paper are listed below:
1. The suggested method relies on the predefined library.
2. The authors did not include the training time to compare the result. 

Applying spline based learning with alternating direction optimization to train Nonlinear Dynamics are original, significant. However, I have further questions about the assumptions and results.

questions: I have several questions regarding clarifying the contribution of the proposed method.

1. In 3.3, the authors assume that Aleatoric Uncertainty is $\epsilon_1 \sim N(0,B)$ and Epistemic Uncertainty is $\epsilon_2 \sim N(0,P)$. Since two noises are independent, the sum of two noises is $N(0, P+B)$, which can be understood as Aleatoric Uncertainty only. Can you elaborate how the proposed methodology differentiate between Aleatoric Uncertainty $N(0,B)$ and Epistemic Uncertainty $N(0,P)$ and how it affects the result?
 
2. It looks like that the authors try to prove the spline fitting error bound with respect to the mean, which is not zero. Have you considered using this upper bound as a means of the Epistemic Uncertainty? Or have you considered other distributions(e.g. uniform distribution) for $\epsilon_2$ (Epistemic Uncertainty)?

3. Can you please provide additional details on how long it takes to train the model? The proposed method in the paper uses the posterior distribution to estimate uncertainty. However, the key challenge in the Bayesian Inference is estimating posterior distribution. I’m curious to see how much computation you need in order to make the result better.

limitations: 1. The authors recognize that the proposed method relies on a predefined library of candidate terms

2. There are some papers considering finding the best knot size $(\tau_{s+1} - \tau_{s})$ as a way to improve the expressibility of spline fitting function, for instance, "Neural Spline Flow", Conor Durkan, Artur Bekasov, Iain Murray and George Papamakarios. I think this method could potentially reduce the total computation required.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: In this paper, the authors:
- propose a system identification algorithm that leverages spline bases to fit sparsely fit noisy data, and provides parameter and prediction uncertainty (which can be used in an active learning / Bayesian optimization setting)
- show that the algorithm can be used to infer ODEs and PDEs, and with high level of noise
- show that the proposed algorithm performs well against two other standard / state of the art methods, but has tradeoffs at different noise levels
strengths_and_weaknesses: originality: I don't know the literature well enough to know if others have attempted similar approaches, but in comparison to standard SINDY, this work builds upon the idea of constructing the system from sparse basis functions but additionally provides a probabilistic output

quality: I did not check the math/code, but I believe the results are of high quality at a superficial level, and support the overall claim (i.e., uses splines to discovery system equations, is sparse, and provides uncertainty estimates). Unclear if it would work beyond the standard benchmark synthetic systems.

clarity: the description of the algorithm is a bit unclear (specific issues outlined below), but the overall goal, the components of the algorithm, and the results were clearly communicated

significance: system identification that tries to retrieve the underlying mechanistic model is especially important in the context of scientific applications with noisy observations; has potential to be broadly impactful in many areas of ML and science
questions: - in Figure 2, the discovered mean of the library terms are plotted against their actual values: what is the actual value in these cases for the VdP and Lorenz96 systems? If I understand correctly, the output of the algorithm are the weights of the splines at different orders, which do not relate to the actual dynamical equations for the systems at all, so how are the comparisons made? Or do the library terms CX2 and CX3 correspond to, e.g., first and second order terms such as x, and xy? It would be nice if this point could be clarified in the text

- in this vein, and it's very possible that I'm confused on this point, but the motivation was to find interpretable models that match the true system. For example, with SINDY, the library can be constructed with polynomial terms up to arbitrary orders, which can then be exactly matched with the original system. In this work, the basis functions are splines, which do not allow for such interpretations

- the schematic in Figure 1 demonstrates a nice effort in explaining the algorithm, but it's a bit crowded, and it's especially unclear how the loss functions in the red box relate to each other, and what the correspond to in the main? In particular, it's a bit unclear to me how the data fitting loss (L2) is combined with the logL loss?

- related to my confusion on the learning procedure, are the spline parameters separately learned (as indicated by the blue box being separated)? or is the whole thing trained end to end. I apologize for the confusion here and do not really have concrete suggestions for how to improve clarity.

- while I understand that the algorithm comparisons are conducted on the benchmark ODE/PDEs, it would be nice to see how the proposed method works with real noisy data that is routinely modeled with one of the systems (e.g., advection equation), or another i.e. predator-prey data with LotkaVolterra, or COVID numbers with SIR model

- figure fonts are extremely small, basically unreadable in many of them

- some terminology should be explained for a reader unfamiliar with SI, e.g., "control points"
limitations: a brief discussion on library choice is included, but a longer discussion is warranted regarding the splines, as well as how it performs in high noise regimes (especially in comparison with the other algorithms). Furthermore, potential negative societal impact should be discussed as such an algorithm can have broad consequences in accurately modeling natural systems.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper extended an existing spline-based model for fitting dynamic systems (ODEs) reported in [14]. There are two main changes: 1) Use tensor-product of splines to model both time and spatial dimensions. 2) Use Bayesian regression to fit the coefficients. With the first change, authors attempted to model PDEs with one spatial dimension (avection equation and Burgers' equation). 

The authors also claimed that the Bayesian estimates help with the data assimilation tasks, although the approach was not described in detail.


strengths_and_weaknesses: Strength:

The paper contains some reasonable ideas (tricks) that have not seemed to be explored before.

Weaknessses:

1. The paper contains quite a few typos in equations and counter-intuitive use of symbols (e.g., using $\hat{U}$ to represent data instead of an estimate). These problems can make the method difficult to understand. The author should have been more careful proof-reading the draft before submission.
2. The tensor-product of 1-d splines has severe limitations, particularly when the spatial dimension is higher. The authors should have made this limitation more clear.
3. The experiment results do not contain sufficient comparison with baseline methods (Only Table 1 contains the comparison of, I assume, the running time).  
questions: 1. Equation (3), for PDE, how do you model terms like $u_x$ and $u_{xx}$? Please give an example like Burger's equation. Is the representation in $R^{n\times s\times m}$?
2. Equation (10), what is $U$? Same for equation (13).
3. Table 1, what are the values?

limitations: The authors should clarify the limitations of splines in modeling higher-dimension data.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose a Bayesian Spline Learning method for equation discovery, which uses spline basis to handle the data scarcity and noise. The Bayesian uncertainty calibration techniques are used to approximate posterior distributions of the trainable parameters. The proposed method is built upon the ADO of PINN-SR, but outperforms PINN-SR and SINDy in all the metrics in the experiments of four equations.
strengths_and_weaknesses: Strengths:
- The motivation is clear and the used method makes sense in the current presentation.
- The experimental results support the claim that their method is more robust with 10% data with large noise (5% to 20%) on Van der Pol Oscillator, Lorenz 96, Advection Equation and Burgers' Equation. 
- The codes and theoretical properties of the algorithm are provided in the supplementary materials.

Weaknesses:
- It is not clear what are the analytical forms of the used four equations and what are the parsimonious equations discovered. There should be some descriptions of the discovered equations at least in the supplementary materials if not in the main manuscript.
- The presentation about why the propose method can benefit equation discovery can be further extended and described in detail.
questions: In the current presentation, it can still be hard to interpret why uncertainty calibration can benefit equation discovery against measurement noise and sparsity. We only know that the spline learning can perform differentiation accurately to enhance learning efficiency and the posterior distribution of spline, library and equation coefficients are estimated simultaneously. These explain the training time superiority but lacks in interpretation why the obtained uncertainty can benefit equation discovery/coefficient estimation, which is more at the core of the question. It makes it harder to understand why the proposed method has a smaller error than PINN-SR and SINDy, especially calibration method generally succeeds in efficiency of uncertainty estimation but hardly provides significant accuracy boost based on relevant studies of previous works on other topics such as traffic prediction, etc.
limitations: The authors mention the limitation of pre-defined library of candidate terms.
review_title: null

Meta-review: metareview: The reviewers are all leaning towards acceptance, with two having acknowledged the extensive author feedback. I encourage the authors to incorporate as clearly as possible the extensive work done during the rebuttal period into the text when updating their paper for publication.
confidence: Less certain
Decision: Accept