Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a novel transformer-based model for symbolic regression (SR), which produces mathematical expression skeletons from data points. In the proposed method, a feature extractor using pointMLP is added and jointly trained using contrastive loss to realize efficient training. The experimental evaluation using benchmark functions demonstrates that the proposed method exhibits higher recovery rates and R2 scores compared to existing SR methods.
strength_and_weaknesses: **Strengths**
* Introducing the idea of feature extractor and contrastive learning into the transformer-based symbolic regression model is convincing. Although each component of the proposed method was previously proposed in the context of other tasks, incorporating such methods into the SR model seems to be novel.
* The numerical experiment supports the effectiveness of the proposed method.

**Weaknesses**
* In the experiment, the same dataset is used for training and testing. I am not confident whether such a setting is reasonable. From the machine learning perspective, the obtained model should be evaluated on test datasets.

**Comments**
* It would be nice if the proposed method was evaluated using recent benchmark datasets such as SRBench (https://github.com/cavalab/srbench).
* In Figure 2, the redder color might indicate the cosine similarity is closer to **0**.
* How is the computational cost of the proposed method compared to other SR methods?
clarity,_quality,_novelty_and_reproducibility: * This paper is well-written. The motivation and contribution are easily understood.
* Although the components of the proposed method are existing ideas, introducing such promising techniques into the transformer-based SR model is novel.
* The code is provided and the detailed setting of the experiment is reported.
summary_of_the_review: Basically, this paper is well-written, and the motivation and contribution are clear. The effectiveness of the proposed method is supported by the numerical evaluation.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a transformer-based method for symbolic regression using a new feature extractor and a joint supervised learning mechanism.  They evaluated the framework on synthetic datasets and compared it with several baseline methods. Experimental results show good performance of the proposed framework. 
strength_and_weaknesses: Strength: 

1. The problem is interesting. 
2. The proposed framework achieves good results. 

Weakness:

1. It would be better if the authors could further clarify the potential application of symbolic regression. 
2. It is unclear to me why the authors adopt a point cloud-like feature extractor (like PointMLP). It seems that the tackled problem is a special regression problem, so why not use 1D convolution or directly use a transformer to handle it? Also, there are some works tackling tabular data, like TabNet. The authors should further clarify the motivation of network design. 
3. Since the proposed framework was trained and evaluated on the same synthetic dataset. There may be overfitting problems. The authors should discuss this point. 
clarity,_quality,_novelty_and_reproducibility: The whole paper is well-written and proposes a new solution for an interesting problem.
summary_of_the_review: This is a good paper to tackle an interesting problem. While the potential application of the proposed technique and method design should be further clarified. The technical novelty of this paper seems limited.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper presents a symbolic regression algorithm that uses a transformer backbone to learn a new feature extractor via joint supervised learning mechanism. The ill-posed problem mentioned in the paper was addressed by using the joint supervised learning mechanism combining supervised contrastive learning, which strengthens the similarity of feature vectors from the same skeleton. Experiments were done to prove the efficacy of the method.

strength_and_weaknesses: This paper is motivated by a practical issue. I like the idea of using the supervision of the whole expression skeleton and the supervision of the preorder traversal of its expression tree in contrastive learning (maybe with data augmentation) to deal with the case that different instances of the same skeleton can have very different shapes, and instances of very different skeletons can be very close.
The authors did extensive experiments, the presented algorithm performs very well on the benchmark dataset.
The paper has covered a wide range of related work.
clarity,_quality,_novelty_and_reproducibility: The paper is easy to follow and straightforward. The novelty is not significant as it is a basic combination of contrastive learning and Transformers. The implementation is not hard but it works well.
summary_of_the_review: This paper proposes a good approach to tackle the symbolic regression problem. Although the presented algorithm seems to be a combination of contrastive learning and Transformers, its performance on benchmark data sets looks good.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: All reviewers agree that this work is well-written and easy to follow with clear motivation and contributions. The experimental experiments support the effectiveness of the proposed method.
justification_for_why_not_higher_score: The contribution of this work is a solid accept but has not reached the bar for spotlight.
justification_for_why_not_lower_score: All reviewers agree to accept the paper.
Decision: Accept: poster