Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This manuscript proposes a novel symbolic physical learner (SPL) that extracts analytical formulas governing nonlinear dynamics from limited data. The SPL uses a Monte Carlo tree search (MCTS) agent to search for optimal expression trees based on data.
The specific differences of the SPL with respect to previous studies are the following three points. (1) replacing the expected reward in UCT scores with a maximum reward to better fit the equation discovery objective, (2) employing an adaptive scaling in policy evaluation which would eliminate the uncertainty of the reward value range owing to the unknown error of the system state derivatives, and (3) transplanting modules with high returns to the subsequent search as a single leaf node. 
The effectiveness and superiority of the SPL machine are demonstrated by numerical experiments comparing it to a baseline of state-of-the-art symbolic regression algorithms.
strength_and_weaknesses: strengths:
The strength of the research is its stable symbolic regression compared to prior methods.

weaknesses:
The following three points are drawbacks
1 There is no discussion or theoretical analysis of why the proposed method is effective. Therefore, I cannot eliminate the suspicion that the effectiveness of the proposed method over the prior methods depends on the superiority of the hyperparameter tuning.
I think it is good to show what kind of "prior physics knowledge in nonlinear dynamics" can be implemented by the proposed algorithm, and provide numerical experiments to demonstrate its effectiveness.

2 Where the stability of the estimation results due to hyperparameter settings is unclear.
Please explain the optimization method for hyperparameters and the stability of the estimation results with respect to variations in hyperparameters.

3 Where it is unclear whether the hyperparameter tuning of the prior method is appropriate.
It is same as above point. In particular, we would like to know why the results in Table 1 do not match the results of similar experiments presented in the paper proposing the prior method. (Table 1 in [Mundhenk et al., 2021])

clarity,_quality,_novelty_and_reproducibility: Quality:
The contents in the manuscript is not reliable due to the low quality of the manuscript.
The presence of even the following elementary typos in the ABSTRACTION would suggest that the paper is not complete. 

In abst:
PSL -> SPL

In page 3:
tre -> the

an an -> an

In page 4:
$n$ -> $\eta$

In page 5:
unpredicTable -> unpredictable

In page 7:
DSR -> (The abbreviation is used without declaration.)

Nguyen-12 in table 1:
$x^4-x^2+\frac{1}{2}y^2-y$ -> $x^4-x^3+\frac{1}{2}y^2-y$ (Please check the Table 1 of [Mundhenk et al., 2021])

Clarity:
It is unclear from the demonstrated numerical experiments what aspects of the proposed method are intended to demonstrate its effectiveness.

Originality:
The proposed method is a trivial improvement of the previous method, and the results obtained are not so different from those of the previous method, so the novelty of the proposed method is weak.
summary_of_the_review: Symbolic regression is an important technique for future physics research using machine learning. In addition, equation tree search using reinforcement learning, which was improved in this study, is an important technique that has shown its usefulness in other fields as well. As such, this research is important. On the other hand, the description and organization of the paper and the setting of numerical experiments are still insufficient, and I think that the paper should be accepted after these points are improved.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper comprehensively reviews the different families of methods to tackle the data-driven discovery of nonlinear dynamics by comparing their advantages and disadvantages. Based on these discussions, the papers propose a new model, the Symbolic Physics Learner (SPL) machine, empowered by MCTS, to discover the mathematical structure of nonlinear dynamics. The paper proposes three changes: achieving a new expected reward, using adaptive scaling in policy evaluation, and transplanting modules with high returns. SPL shows its effectiveness in a wide range of tasks for discovering the mathematical structure of nonlinear dynamics.

strength_and_weaknesses: **Strengths**

**1. The paper is well-organized and easy to follow.**

The organization and the writing of the whole paper are very attractive to me. The introduction and the background sections go over the different lines of existing work and the background one needs before diving into the proposed method in this paper. Figure 1 and Algorithm 1 clearly illustrate the high-level architecture and the underlying details of the method, respectively. Sections 4 to 7 discuss how the method could be applied to different benchmarks and tasks.

**2. The proposed method adopts three useful adjustments to the conventional MCTS.**

 The proposed Symbolic Physics Learner (SPL) machine carefully incorporates the three adjustments to the conventional MCTS, which largely improve the prediction performance and stability.



**3. The experimental sections are impressive to me.** 

The experiments are conducted on four separate cases. The results show a consistent improvement on a bunch of tasks for SPL.


**Weaknesses**

**1. The model novelty.** 

I am a little bit concerned about the novelty of the proposed method. It looks like an incremental model that makes several modifications on top of a popular model.

clarity,_quality,_novelty_and_reproducibility: The paper is high-quality and well-written. I can easily learn the comprehensive review of the existing work and the background of the problem the paper is working on. My only concern is that there might not be enough modeling novelty.

summary_of_the_review: The paper is attractive to me since there are various strengths in terms of problem formulation, modeling, experiments, and writing.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes Symbolic Physics Learner (SPL), a method for symbolic regression and discovery of dynamics.
The authors formalize mathematical operations using expression trees and context-free grammars,
and employ Monte Carlo Tree Search (MCTS) to explore the space and select
optimal solutions.
Furthermore, they propose adjustments to conventional MCTS, including adaptive
scaling and module transplantation.
The authors demonstrate the efficacy of the proposed method in a number of
settings and achieve superior performance compared to other
state-of-the-art methods.

strength_and_weaknesses: The paper is well written and easy to follow.
The authors perform a multitude of experiments on symbolic regression and
discovery of nonlinear dynamics.
Comparison with state-of-the-art related works clearly demonstrates the
shortcomings of related works and the efficacy of the proposed method.

I am missing a more detailed description and direct comparison with related works on MCTS for symbolic regression.
How is your work different from theirs?
Similarly, I would expect a quantitative comparison with existing MCTS methods for symbolic regression.

There is one ablation study on the importance of adjustments to MCTS (sec. 4.2),
yet the results are barely mentioned, and I miss some analysis on them.
You could have included the results (and the analysis) in the appendix.
It is surprising that removing each adjustment separately results in a performance lower than 60%.
Can you provide more detailed results, and perhaps explain why that might be the case?

clarity,_quality,_novelty_and_reproducibility: It is unclear to me how you can predict continuous constant values. Can you elaborate on that?
You mention in the limitations and in the appendix that the computational cost
for constant value estimation is high. Why is that the case?

In table B.1, the learned physics for the yellow whiffle ball and the bowling ball
involve big expressions with large constant values and multiple components.
They seem to be non-parsimonious according to your descriptions. Do these
equations show signs of overfitting, and if so, are there ways to address that in
your method?

In the Lorenz system in sec. 6 and in the double pendulum in sec. 7, what is the importance of Savitzky-Golay filter?
Would your method work without it, or is it integral to compute noise-free
derivatives? Is this filtering approach applicable in a
variety of settings? Could the introduced smoothness be harmful in a
setting without noise?

In the double pendulum in sec. 8, you employ some physics to guide the discovery. Why is that?
Is the overall system too hard to model otherwise?
What is the performance of other methods in this setting?

### Additional

Table 4 is out of the document margins. Please respect the document margins for
the camera ready version.

### Typos

- Abstract: optimistic policy -> optimal policy
- Abstract: PSL -> SPL
- p. 3, sec. 2: stikcs
- p. 3 sec. 3: at an an expandable node
- p. 5, Adaptive-scaled rewarding: unpredicTable

summary_of_the_review: I recommend that this paper is accepted. It is a well written paper with important technical contributions and strong quantitative results. A comparison with related works on MCTS for symbolic regression could help the readers appreciate its contributions more. More thorough ablation studies on the introduced components could further clarify their efficacy.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose a Monte Carlo tree search (MCTS) algorithm to generate optimal expression trees based on measurement data. The method is validated using Nguyen’s symbolic regression benchmark task (Uy et al., 2011), along with several other benchmarks.




strength_and_weaknesses: Strengths:

This is a strong paper, and has clear novelty in the area of symbolic regression.
The paper has extensive comparison to competing methods, with good results.


Weaknesses:

The theoretical aspects of the paper could be improved.

The theoretical basis of this approach should be highlighted in comparison to existing methods of symbolic regression. In addition, since it is based on stochastic methods for which guarantees have been given for other applications, this is a clear lack. Questions include: 
What type of sampling distributions are used, and can these be modified during search to improve the returned function?

clarity,_quality,_novelty_and_reproducibility: There are some missing issues in the paper.

References:

--Need a reference for eq. (1), or a note as to why this is a good measure.

--Sec. (3): the claim "It has been demonstrated that the MCTS agent continuously gains knowledge" needs a reference.

Need a more precise definition of  Recovery rate (Table 1). It is well understood that the recovered equations typically contain many more terms than the target equations, and this needs to be shown in the definition and results. Do you mean "regeneration of the precise equation with no additional terms"?

A key missing factor in the paper (Algorithm 1) is the specification of the regularization terms. Symbolic regression has major problems with limiting the size of the learned equations without good regularization terms; this is not addressed in the body of the paper and must be.




summary_of_the_review: This paper presents a novel approach to symbolic regression, and shows good improvement over existing methods. Overall, a strong contribution.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses:  The method proposes the use of Monte Carlo Tree Search to discover more accurately and robustly symbolic expressions for the observed data from nonlinear dynamical systems.

All reviewers agree on the novelty, as well as the relevance of the empirical findings.
justification_for_why_not_higher_score: The paper is unanimously vetted by the reviewers.
justification_for_why_not_lower_score: The paper is unanimously vetted by the reviewers.
Decision: Accept: notable-top-5%