Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a new sampling method for genetic programming based on the semantics. The idea is that for each step, the algorithm would sample a new variant with different semantics (based on some distance function) that is also different from previous historical samples. Based on this idea, the authors demonstrate that this method improves sample complexity on a bunch of tasks, including function regression, learning an optimizer, and learning RL losses.
strength_and_weaknesses: Strength:
- The paper provides several instantiations of the method, all seem quite interesting.

Weakness:
- The method seems pretty simple and not very surprising, so not sure how much conceptual novelty there is. I'm not an expert on genetic programming, so would leave it to other reviewers who have more domain expertise on this topic.
- For the learned optimizer experiment, there's no comparison with other methods. In fact there is a rich literature of previous works on this problem, i.e., how to select the hyperparameters in an algorithmic way. More comparison with them would be great.
- The experiments are definitely too small scale. E.g., for the learned optimizer, it's unclear whether it works on larger NN models, so the findings in this paper may be not relevant to modern deep learning. 
clarity,_quality,_novelty_and_reproducibility: The paper is written clearly with good quality. 
summary_of_the_review: This paper mainly provide experiments on small scale datasets and lack comparison with existing methods. I'm not an expert on genetic programing, so I'm not sure how important these tasks are for the community --- from my perspective the tasks (e.g., learning optimizer and learning RL loss) are not super relevant these days especially when the experiments are only carried out on small datasets. Thus, I'll vote for rejection based on my current understanding, but would love to follow other reviewer's suggestion if they are more confident on this topic.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a mutation operation to generate semantically close and diverse individuals in genetic programming. The proposed method generates a candidate program minimizing the mutation objective that takes into account the distance between parent, child, and previously evaluated programs. Then, the proposed method enables to the generation of a candidate program that is close but not too close to the parent and is not close to previously evaluated programs. To validate the effectiveness of the proposed concept, the authors apply the proposed method to simple regression, learning an optimizer, and learning reinforcement learning loss tasks. In the simple regression task, the proposed method can accelerate the search efficiency compared to existing mutation operations. In addition, the proposed method could find a better solution than existing algorithms in learning an optimizer and learning reinforcement learning loss tasks.
strength_and_weaknesses: [Strength]
* A novel mutation operation taking into account the semantic similarity is introduced.
* The effectiveness of the proposed method is demonstrated for not only simple problems but also meta-learning tasks.

[Weaknesses]
* The reviewer is not an expert on this topic and is not confident whether the proposed semantic distance between programs defined in (2) is novel or not. The technical novelty of the distance function should be clarified. 
* In the proposed method, it seems that minimizing the objective of (1) and distance calculation requires generated programs' execution, and such cost should not be ignored. The reviewer is not sure that the experimental comparison with existing methods was fair. It would be better to compare the performance under the same number of program executions or computational budgets.
* In the experiments on learning an optimizer and learning a reinforcement learning loss, although the use case of the proposed method is demonstrated, the advantage of the proposed method on such tasks compared to other genetic programming methods is unclear.
* The concept of the proposed method can be understandable. However, it seems difficult to re-implement it because the authors did not provide the code and detailed information on the implementation.

[Minor Comments]
What does `sin_big` mean in Figure 1-4?

clarity,_quality,_novelty_and_reproducibility: * The reviewer feels that the technical difference between the proposed method and existing semantic genetic programming methods is not clear. The technical contribution should be made clear.
* The advantage of the proposed method on meta-learning tasks against existing methods is not clear, although the authors mentioned such experimental comparison is difficult due to the cost of reproducing existing methods.
* The authors did not provide the code and did not report the detailed experimental settings. The reviewer thinks that it is hard to reproduce the experimental results.

summary_of_the_review: The topic treated in this paper is interesting and important. However, the reviewer feels that the technical novelty and empirical evidence of the advantage of the proposed method are unclear. Such weakness should be addressed to accept the paper.
review_title: null

final_score: rating: 1: strong reject
review_content: summary_of_the_paper: The paper proposes a semantic mutation operator for genetic programming (GP), which aims at creating new diverse candidate solutions while keeping a good connection with the parent solutions. Semantic GP is a common approach that aims at exploiting the behavior of programs or subprograms (e.g., the output generated for given test cases used for evaluation) instead of their structure. This signal has been shown as relevant to be included in the optimization methods of GP, optimization guided purely on the structure of GP solutions being non-smooth or continuous, with simple changes in the structure leading to huge differences in the behavior. The proposed semantic mutation operator is presented and evaluated for three specific problems, that is real-valued function approximation, gradient-descent-based optimization function, and a loss function for reinforcement learning.
strength_and_weaknesses: Strength
- Straightforward proposal for a novel semantic GP mutation operator.
- Apparently good empirical performances in three applicative contexts.

Weaknesses
- The proposal is relatively simple and incremental considering the work presented so far on semantic GP. There is no detailed overview of semantic GP, nor a good analysis of the similarity and differences of the current approach with the literature on the topic is missing.
- The experiments are done on specific problems and lack detailed comparison with other common semantic GP approaches.
- The paper writing quality is below average, and details on the method proposed are missing or unclear at times.
- The overall proposal appears relatively niched.

clarity,_quality,_novelty_and_reproducibility: - Sec. 3.1, presenting the mutation objective is relatively clear, although quite abstract, it presents the approach in terms of a generic distance measure.
- The generic function is detailed in Eq. 2 as a scaled L1 norm over the semantic variables, but still an abstract scaling function p_i(s(G)), where G is not clearly defined given that the distance is stated as d(J,K), so between J and K. What’s G, and what’s its use? According to Eq. 1, the G is in fact used as J=M(G) and K=G for Eq. 2. I’m confused by this notation, better explanations are required.
- The second term of Eq. 1, which is the diversity term, appears quite ad hoc. I get it does some kind of adjustment for getting diverse solutions, but is there any more formal or conceptual justifications for that part? I mean, why not just summing the L2 norm with all individuals in the current population as diversity?
- Is the scaling between the three components of Eq. 1 with \mu parameter enough and allow the best combination of these components. The scale may vary between these, would a distinct scaling factor between them allow better performances?
- It is stated at the beginning of Sec. 3.2 that “We minimize (1) by generating several candidate mutations, and then use CMA-ES to fine-tune the constants.”. Ok, but how many individuals are we generating before picking the best one according to Eq. 1? And what is the effect of the number of individuals generated on the quality of the mutations?
- In Sec. 3.2, details are missing on the raw mutation. For instance, how the choice of the replacement of z with the expressions done uniformly? How is the new node used and constructed? Is the new node replacing the current node directly, or a new subtree is made? And how about the arity, do we restrict the new node to have the same arity than the old one? If not, how do we proceed to enforce validity of the tree? Many details on that mutation operator are missing.
- Also, still in Sec. 3.2 on the raw mutation, it is stated that “c\in{0 1} is a “gating” constant that allows continuous linear interpolation between the old and new node and is initially set to a value such that the new node has the same value as the old node.” I see that this is the case for c=0, but how can this be enforced for c=1? (Conversely for zc expression, where the semantic is preserved for c=1, but not for c=0).
- In the “Optimizing” paragraph, it is stated that s_i = max{10^-5,min(|u_i|, |u_i-1|, |u_i+1|)}. What do the indices i stand for? Why set the u in +- 1 range? I don’t get this part, the tricks of that equation are unclear to me, I am not sure how this helps to handle large or small constants.
- In Sec. 3.3, paragraph “Sampling and/or mutation”, it is stated “With probability 0.2, this is re-evaluation, which allows reducing the fitness measurement error for potentially high fitness programs”. Wait, what is that? I mean, if you have the fitness of an individual that is not modified, you don’t need to re-evaluate it. If it is modified, then you should re-evaluate. I don’t get why and for what purpose we would re-evaluate some individuals picked at random, some explanations are required here.
- In the “Mutation rates and parsimony pressure”, it is stated that \mu and \beta are picked randomly according to some distribution for each mutation. That’s quite fancy, this is an extra layer of complexity over the proposed mutation operator. I would like to see an ablation study over these to demonstrate the added value of proceeding that way.
- Paragraph “Collecting sample inputs for the semantics”, it is stated that “the evaluation workers also collect program inputs when evaluating a program”. Why doing so, what’s the use? How does this affect the optimization process?
- Performance and ablation in Sec. 4.1 is interesting to show the effect of diversity and simplicity, but these results are for one case. I would have been good to conduct this ablation over several problems.

Using CMA-ES for constant optimization is a big deal in my opinion, as (200x128)=25600 fitness evaluations are spent on making this tuning for each mutated individual. We usually count the sample efficiency of GP in term fitness evaluation, and as such, this is very not sample efficient if we compare with other GP approaches that are not optimizing constants that way. I fear this may be the part of the optimization model that is making most of the job in terms of results.

As mentioned before, a big issue with the proposal is the lack of comparison with other semantic GP approaches, like the ones presented in Vanneschi et al. (2014), and more standard GP approach.

Vanneschi, L., Castelli, M., & Silva, S. (2014). A survey of semantic methods in genetic programming. Genetic Programming and Evolvable Machines, 15(2), 195-214. 

There is a lack of evaluating the approach on benchmark problems for GP, like symbolic regression functions (see SRBench, https://github.com/cavalab/srbench). That would allow us to properly evaluate and compare performance with the state of the art.

The application contexts proposed are interesting uses of the approach, although I am not convinced they are of practical interest. Principled approaches are much preferable to learned ad hoc criterion, unless a clear added value is presented (this is not the case here). It is even stated at the end of sec. 6.1 that the learned optimizer is not generalizing outside the task suite, exposing some kind of overfitting. That limits greatly the interest for such an approach.

summary_of_the_review: The proposal is simple, but the explanations are not very clear, many elements are not justified, there is a lack of comparison with other related methods and the problems tackled are non standard (not common benchmark in the field) nor of practical interest by themselves.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This work proposed a novel semantic genetic programming mutation operator, and demonstrated its effectiveness using a set of applications in machine learning. Overall this work provide an interesting idea that would help to improve GP in several contexts, but the reviewers have overall reservations about this work in that: the proposed method is relatively straightforward, the comparisons and experiments could be significantly enhanced. It will be helpful for the authors to take these comments and improve their work.
justification_for_why_not_higher_score: Overall the reviewers find this work to be of interest to audience in the area of GP, and it is recognized that the authors showed good performance in several specific contexts. Nonetheless, the overall novelty and scope of this work is limited, and the authors are encouraged to take the comments from the discussion to further enhance this work for future submission.
justification_for_why_not_lower_score: NA
Decision: Reject