Original: Re-Evaluating Chemical Synthesis Planning Algorithms
Revised: Re-Evaluating Chemical Synthesis Planning Algorithms

Original: Abstract
Revised: Abstract

Original: Computer-Aided Chemical Synthesis Planning (CASP) algorithms have the potential to help chemists predict how to make molecules, and decide which moleculesto prioritize for synthesis and testing.
Revised: Computer-Aided Chemical Synthesis Planning (CASP) algorithms have the potential to help chemists predict how to make molecules, and decide which moleculesto prioritize for synthesis and testing.

Original: Recently, several algorithms have beenproposed to tackle this problem, reporting large performance improvements.
Revised: Recently, several algorithms have beenproposed to tackle this problem, reporting large performance improvements.

Original: Inthis work, we re-examine current and prior State-of-the-Art synthesis planningalgorithms under controlled and identical conditions, providing a holistic viewusing several previously un-reported evaluation metrics which cover the commonuse-cases of these algorithms.
Revised: Inthis work, we re-examine current and prior State-of-the-Art synthesis planningalgorithms under controlled and identical conditions, providing a holistic viewusing several previously un-reported evaluation metrics which cover the commonuse-cases of these algorithms.

Original: In contrast to prior studies, we find that under strictcontrol, differences between algorithms are smaller than previously assumed.
Revised: In contrast to prior studies, we find that under strictcontrol, differences between algorithms are smaller than previously assumed.

Original: Ourfindings can guide users to choose the appropriate algorithms for specific tasks, aswell as stimulate new research in improved algorithms
Revised: Ourfindings can guide users to choose the appropriate algorithms for specific tasks, aswell as stimulate new research in improved algorithms

Original:  Introduction/Motivation
Revised:  Introduction/Motivation

Original: The synthesis of small organic molecules (SMOLs) is an essential part of drug and materials discovery.
Revised: The synthesis of small organic molecules (SMOLs) is an essential part of drug and materials discovery.

Original: To make a desired target molecule, one follows a synthesis route (also called plan), which consistsof usually multiple reactions one has to run starting from commercially available building blockmolecules to obtain the target.
Revised: To make a desired target molecule, one follows a synthesis route (also called plan), which consistsof usually multiple reactions one has to run starting from commercially available building blockmolecules to obtain the target.

Original: Thus, a route resembles a cooking recipe, where the target moleculeis the desired final dish, and the the building blocks are ingredients.
Revised: Thus, a route resembles a cooking recipe, where the target moleculeis the desired final dish, and the the building blocks are ingredients.

Original: The most common approachto come up with a route is by decomposing the target molecule recursively using formally reversedchemical reactions, until one reaches the building blocks.
Revised: The most common approachto come up with a route is by decomposing the target molecule recursively using formally reversedchemical reactions, until one reaches the building blocks.

Original: In chemistry, this technique is calledretrosynthesis .
Revised: In chemistry, this technique is calledretrosynthesis .

Original: In contrast to peptides, proteins, or DNA/RNA, which can be made by a small number of possiblechemical reaction types, the structural diversity of SMOLs requires thousands of different reactiontypes.
Revised: In contrast to peptides, proteins, or DNA/RNA, which can be made by a small number of possiblechemical reaction types, the structural diversity of SMOLs requires thousands of different reactiontypes.

Original: This makes route planning challenging, as reaction types correspond to different options thatcan be taken at each step in the recursive decomposition, which makes retrosynthesis hard even forthe best human chemists.
Revised: This makes route planning challenging, as reaction types correspond to different options thatcan be taken at each step in the recursive decomposition, which makes retrosynthesis hard even forthe best human chemists.

Original: Computational approaches for synthesis planning have been studied for at least six decades.[Todd,2005, Strieth-Kalthoff et al., 2020].
Revised: Computational approaches for synthesis planning have been studied for at least six decades [Todd,2005, Strieth-Kalthoff et al., 2020].

Original: Recently, the use of machine learning for synthesis planning hasled to considerable progress in particular in the quality of the proposed solutions, and to renewedinterest in the field [Segler et al., 2018, Coley et al., 2019, Kishimoto et al., 2019, Schwaller et al.
Revised: Recently, the use of machine learning for synthesis planning hasled to considerable progress in particular in the quality of the proposed solutions, and to renewedinterest in the field [Segler et al., 2018, Coley et al., 2019, Kishimoto et al., 2019, Schwaller et al.

Original: ,2020, Genheden et al., 2020, Chen et al., 2020, Kim et al., 2021, Xie et al., 2022].
Revised: ,2020, Genheden et al., 2020, Chen et al., 2020, Kim et al., 2021, Xie et al., 2022].

Original: Conceptually, synthesis planning algorithms have four components: 1) A reaction prediction module,to propose single-step retrosynthetic disconnections that correspond to feasible reactions in the forward direction, 2) a search or planning algorithm that can handle large branching factors, 3) abuilding block library, 4) ranking criteria for the returned routes, for example by cost.
Revised: Conceptually, synthesis planning algorithms have four components: 1) A reaction prediction module,to propose single-step retrosynthetic disconnections that correspond to feasible reactions in the forward direction, 2) a search or planning algorithm that can handle large branching factors, 3) abuilding block library, 4) ranking criteria for the returned routes, for example by cost.

Original: In this work, we re-evaluate component 2), the search algorithms.
Revised: In this work, we re-evaluate component 2), the search algorithms.

Original: We report preliminary findingswhich suggest that the performance of different search algorithms may be much more similar thanpreviously reported.
Revised: We report preliminary findingswhich suggest that the performance of different search algorithms may be much more similar thanpreviously reported.

Original: Specifically, after re-benchmarking two MCTS variants against the Retro*algorithm [Chen et al., 2020] on four distinct sets of test molecules, we find that both the time atwhich a solution is found and the diversity of solutions found is essentially indistinguishable for allalgorithms.
Revised: Specifically, after re-benchmarking two MCTS variants against the Retro*algorithm [Chen et al., 2020] on four distinct sets of test molecules, we find that both the time atwhich a solution is found and the diversity of solutions found is essentially indistinguishable for allalgorithms.

Original: We believe these results raise some important questions for the community, and help toset best practice for future researchers in this area.
Revised: We believe these results raise some important questions for the community, and help toset best practice for future researchers in this area.

Original:  Experimental Procedure
Revised:  Experimental Procedure

Original: Algorithm classes considered.
Revised: Algorithm classes considered.

Original: We focus our experiments on two algorithm classes:
Revised: We focus our experiments on two algorithm classes:

Original: MCTS[Browne et al., 2012] and Retro* [Chen et al., 2020], which is inspired by A*.
Revised: MCTS[Browne et al., 2012] and Retro* [Chen et al., 2020], which is inspired by A*.

Original: MCTS is chosenbecause of its initial successful results for synthesis planning [Segler et al., 2017], and the subsequentattention and adoption that it received from the chemistry community (e.g. [Segler et al., 2018, Coleyet al., 2019, Genheden et al., 2020]).
Revised: MCTS is chosenbecause of its initial successful results for synthesis planning [Segler et al., 2017], and the subsequentattention and adoption that it received from the chemistry community (e.g. [Segler et al., 2018, Coleyet al., 2019, Genheden et al., 2020]).

Original: Retro* [Chen et al., 2020] is selected as it reported a significantimprovement over MCTS , and because the algorithm operates on an AND/OR tree instead of an
Revised: Retro* [Chen et al., 2020] is selected as it reported a significantimprovement over MCTS , and because the algorithm operates on an AND/OR tree instead of an

Original: OR tree from Segler et al.
Revised: OR tree from Segler et al.

Original: [2018], Coley et al.
Revised: [2018], Coley et al.

Original: [2019], Genheden et al.
Revised: [2019], Genheden et al.

Original: [2020], which is a morenatural data structure for synthesis trees.
Revised: [2020], which is a morenatural data structure for synthesis trees.

Original: We did not run initial experiments using proof numbersearch
Revised: We did not run initial experiments using proof numbersearch

Original: [Kishimoto et al., 2019] at this time due to the lower performance of this method reported in
Revised: [Kishimoto et al., 2019] at this time due to the lower performance of this method reported in

Original: Genheden and Bjerrum [2022], although we intend to run these experiments at a later date.
Revised: Genheden and Bjerrum [2022], although we intend to run these experiments at a later date.

Original: Algorithms used.
Revised: Algorithms used.

Original: From these algorithm classes, we chose three specific algorithms: Retro* on
Revised: From these algorithm classes, we chose three specific algorithms: Retro* on

Original: AND/OR trees, MCTS as implemented by Segler et al.
Revised: AND/OR trees, MCTS as implemented by Segler et al.

Original: [2018], Coley et al.
Revised: [2018], Coley et al.

Original: [2019], Genheden et al.[2020] on OR trees, and a novel modified version of MCTS which uses an AND/OR tree (AO-MCTS).
Revised: [2019], Genheden et al.[2020] on OR trees, and a novel modified version of MCTS which uses an AND/OR tree (AO-MCTS).

Original: All algorithms are implemented in a similar way as part of a larger python package for retrosynthesis,ensuring that performance differences due to implementation style are minimal.
Revised: All algorithms are implemented in a similar way as part of a larger python package for retrosynthesis,ensuring that performance differences due to implementation style are minimal.

Original: All algorithms usethe same reaction model and policy, and we run each algorithm with two different value functions: aconstant value function, and a pre-trained neural network value function from Chen et al.
Revised: All algorithms usethe same reaction model and policy, and we run each algorithm with two different value functions: aconstant value function, and a pre-trained neural network value function from Chen et al.

Original: Thisis an important departure from previous papers, which used different value functions for differentalgorithms (often rollouts for MCTS), making it unclear how much of performance differences aredue to the algorithm and how much are due to the value function.
Revised: Thisis an important departure from previous papers, which used different value functions for differentalgorithms (often rollouts for MCTS), making it unclear how much of performance differences aredue to the algorithm and how much are due to the value function.

Original: Further details can be found in
Revised: Further details can be found in

Original: Appendix A.
Revised: Appendix A.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We chose to evaluate performance with two metrics: the time at which asolution is found, and the number of distinct solutions found over time.
Revised: We chose to evaluate performance with two metrics: the time at which asolution is found, and the number of distinct solutions found over time.

Original: The first metric was chosenas an improvement over previous papers, which simply report the fraction of molecules which haveat least one solution by a given time, which does not yield any insight about when within that timewindow solutions were found.
Revised: The first metric was chosenas an improvement over previous papers, which simply report the fraction of molecules which haveat least one solution by a given time, which does not yield any insight about when within that timewindow solutions were found.

Original: The second metric was chosen to represent route diversity.
Revised: The second metric was chosen to represent route diversity.

Original: Althoughchemists generally want CASP algorithms to propose multiple diverse synthesis routes, most previousworks have not evaluated diversity.
Revised: Althoughchemists generally want CASP algorithms to propose multiple diverse synthesis routes, most previousworks have not evaluated diversity.

Original: Our metric, which specifically reports the number of synthesisroutes found at time t such that no molecules in any two synthesis routes overlap, was chosen basedon the diversity-metric guidance from Xie et al.
Revised: Our metric, which specifically reports the number of synthesisroutes found at time t such that no molecules in any two synthesis routes overlap, was chosen basedon the diversity-metric guidance from Xie et al.

Original: Among other things, this measure of diversityguarantees that adding an additional solution to an existing set of solutions will never result inthe diversity decreasing: a property which does not hold for other proposed measures of diversity
Revised: Among other things, this measure of diversityguarantees that adding an additional solution to an existing set of solutions will never result inthe diversity decreasing: a property which does not hold for other proposed measures of diversity

Original: Use of caching.
Revised: Use of caching.

Original: When calculating the metrics above, we have followed previous works in usingthe number of calls to the reaction model as a measure of time, since this tends to be the mostcomputationally-intensive part of the search.
Revised: When calculating the metrics above, we have followed previous works in usingthe number of calls to the reaction model as a measure of time, since this tends to be the mostcomputationally-intensive part of the search.

Original: However, unlike previous works, we use a cache forthe reaction model, meaning that after the reactions for a molecule M are predicted once, if M isencountered again its reactions are retrieved from the cache instead of calling the model.
Revised: However, unlike previous works, we use a cache forthe reaction model, meaning that after the reactions for a molecule M are predicted once, if M isencountered again its reactions are retrieved from the cache instead of calling the model.

Original: This may sound like a minor implementation detail, but it actually has a significant impact on the evaluation.
Revised: This may sound like a minor implementation detail, but it actually has a significant impact on the evaluation.

Original: Often very large sub-trees can occur in multiple places during search: 1 without a cache, expandingeach occurrence of these subtrees will count against an algorithm’s time budget, whereas with a cachethese expansions are effectively free.
Revised: Often very large sub-trees can occur in multiple places during search: 1 without a cache, expandingeach occurrence of these subtrees will count against an algorithm’s time budget, whereas with a cachethese expansions are effectively free.

Original: Given that any realistic implementation of these algorithmswould use caching to minimize computational cost, we think that it is important to use caching whenmaking any sort of real-world comparison between algorithms.
Revised: Given that any realistic implementation of these algorithmswould use caching to minimize computational cost, we think that it is important to use caching whenmaking any sort of real-world comparison between algorithms.

Original: Experiments run.
Revised: Experiments run.

Original: For all experiments, each algorithm is allowed to run for a maximum of 300 son each molecule tested.
Revised: For all experiments, each algorithm is allowed to run for a maximum of 300 son each molecule tested.

Original: While previous papers tend to focus on a single testing setup (i.e. using asingle reaction model, set of purchasable molecules, and set of test molecules), we think that testingon multiple setups is helpful to obtain a comprehensive picture of the performance differences ofeach algorithm.
Revised: While previous papers tend to focus on a single testing setup (i.e. using asingle reaction model, set of purchasable molecules, and set of test molecules), we think that testingon multiple setups is helpful to obtain a comprehensive picture of the performance differences ofeach algorithm.

Original: Therefore we use the following setups.
Revised: Therefore we use the following setups.

Original: Retro* Hard: 190 hard molecules used in Chen et al.
Revised: Retro* Hard: 190 hard molecules used in Chen et al.

Original: [2020], using the reaction model andset of purchasable molecules from Chen et al.
Revised: [2020], using the reaction model andset of purchasable molecules from Chen et al.

Original: PaRoutes: 200 randomly chosen molecules from the n = 5 setting of PaRoutes [Genhedenand Bjerrum, 2022], using their pre-trained reaction model as the policy for each algorithm.3.
Revised: PaRoutes: 200 randomly chosen molecules from the n = 5 setting of PaRoutes [Genhedenand Bjerrum, 2022], using their pre-trained reaction model as the policy for each algorithm.3.

Original: ChemBL Hard: 200 randomly chosen “hard” molecules chosen from the ChemBL dataset,with the reaction model and purchasable molecules from Chen et al.
Revised: ChemBL Hard: 200 randomly chosen “hard” molecules chosen from the ChemBL dataset,with the reaction model and purchasable molecules from Chen et al.

Original: To create a set
Revised: To create a set

Original: of molecules equally or more challenging than the Retro*
Revised: of molecules equally or more challenging than the Retro*

Original: set,
Revised: set,

Original: molecules
Revised: molecules

Original: were chosen byapplying the pre-processing script from [Brown et al., 2019], keeping only molecules whosemolecular weight, Bertz coefficient, logP and TPSA were larger the mean of the respectivevalues in the Retro* hard set, and removing all purchasable molecules.
Revised: were chosen byapplying the pre-processing script from [Brown et al., 2019], keeping only molecules whosemolecular weight, Bertz coefficient, logP and TPSA were larger the mean of the respectivevalues in the Retro* hard set, and removing all purchasable molecules.

Original: GDB17: 200 randomly chosen molecules from gdb17 [Ruddigkeit et al., 2012], using thereaction model and set of purchasable molecules from Chen et al.
Revised: GDB17: 200 randomly chosen molecules from gdb17 [Ruddigkeit et al., 2012], using thereaction model and set of purchasable molecules from Chen et al.

Original: Summary.
Revised: Summary.

Original: Overall, the most important parts of our experimental procedure are:
Revised: Overall, the most important parts of our experimental procedure are:

Original: [Figure]
Revised: [Figure]

Original: • Algorithm Implementation:
Revised: • Algorithm Implementation:

Original: [Figure]
Revised: [Figure]

Original: – All algorithms implemented by us in a similar way as part of a common code base.
Revised: – All algorithms implemented by us in a similar way as part of a common code base.

Original: [Figure]
Revised: [Figure]

Original: – Use the same policies and value functions for all algorithms.
Revised: – Use the same policies and value functions for all algorithms.

Original: [Figure]
Revised: [Figure]

Original: • Analysis of performance
Revised: • Analysis of performance

Original: [Figure]
Revised: [Figure]

Original: – Show when each algorithm finds a solution, not just whether a solution is found at anarbitrary end time.
Revised: – Show when each algorithm finds a solution, not just whether a solution is found at anarbitrary end time.

Original: [Figure]
Revised: [Figure]

Original: – Measure and evaluate diversity of routes found in a principled way– Use caching of the reaction model and account for this when measuring algorithmruntime.
Revised: – Measure and evaluate diversity of routes found in a principled way– Use caching of the reaction model and account for this when measuring algorithmruntime.

Original: – Test in 4 different scenarios
Revised: – Test in 4 different scenarios

Original:  Results and Discussion
Revised:  Results and Discussion

Original: The results from our experiments are summarized in Figure 1.
Revised: The results from our experiments are summarized in Figure 1.

Original: The most important observation inthis figure is the high amount of overlap between the evaluation metrics for all different algorithms.
Revised: The most important observation inthis figure is the high amount of overlap between the evaluation metrics for all different algorithms.

Original: Looking at the left column, for all four test sets, the box plots for time of first solution for all differentalgorithms almost entirely overlap.
Revised: Looking at the left column, for all four test sets, the box plots for time of first solution for all differentalgorithms almost entirely overlap.

Original: Looking at the right column, for all test sets the algorithms allfind approximately the same number of non-overlapping routes, although they seem to find additionalroutes at slightly different rates (with no clear pattern across the different test sets).
Revised: Looking at the right column, for all test sets the algorithms allfind approximately the same number of non-overlapping routes, although they seem to find additionalroutes at slightly different rates (with no clear pattern across the different test sets).

Original: This result mayseem surprising, because it directly contradicts the findings of Chen et al.
Revised: This result mayseem surprising, because it directly contradicts the findings of Chen et al.

Original: [2020], who report that
Revised: [2020], who report that

Original: MCTS performs far worse than Retro*.
Revised: MCTS performs far worse than Retro*.

Original: However, our findings are very similar to those of Genhedenand Bjerrum [2022], who also report very little difference between MCTS and Retro*.
Revised: However, our findings are very similar to those of Genhedenand Bjerrum [2022], who also report very little difference between MCTS and Retro*.

Original: [Figure]
Revised: [Figure]

Original: Further examining Figure 1 yields several additional insights:
Revised: Further examining Figure 1 yields several additional insights:

Original: • Difficulty of different test sets: The easiest set seems to be that of PaRoutes [Genheden and
Revised: • Difficulty of different test sets: The easiest set seems to be that of PaRoutes [Genheden and

Original: Bjerrum, 2022], where all algorithms find a solution for 75% of the test molecules withinjust 10 calls to the reaction model.
Revised: Bjerrum, 2022], where all algorithms find a solution for 75% of the test molecules withinjust 10 calls to the reaction model.

Original: The hardest set seems to be GDB17, where over 75%of the molecules are not solved at all.
Revised: The hardest set seems to be GDB17, where over 75%of the molecules are not solved at all.

Original: The Retro* Hard and ChemBL Hard test sets are
Revised: The Retro* Hard and ChemBL Hard test sets are

Original: somewhere in between.
Revised: somewhere in between.

Original: • Effect of the value function:
Revised: • Effect of the value function:

Original: Comparing each algorithm run with a constant value function
Revised: Comparing each algorithm run with a constant value function

Original: (C) to that same algorithm run with a neural network value function (V) reveals that thevalue function seems to help, but only slightly.
Revised: (C) to that same algorithm run with a neural network value function (V) reveals that thevalue function seems to help, but only slightly.

Original: This is most visible on the Retro* Hard test
Revised: This is most visible on the Retro* Hard test

Original: set.
Revised: set.

Original: It is consistent with the results reported by Chen et al.
Revised: It is consistent with the results reported by Chen et al.

Original: [2020] that setting the value
Revised: [2020] that setting the value

Original: function to 0 for all molecules resulted in only a mild decrease in performance.
Revised: function to 0 for all molecules resulted in only a mild decrease in performance.

Original: We also analyzed the results measuring time instead by wall-clock time (Figure 2) and by number ofcalls to the value function (Figure 3).
Revised: We also analyzed the results measuring time instead by wall-clock time (Figure 2) and by number ofcalls to the value function (Figure 3).

Original: In general, the conclusions measuring time with wall-clock arevery similar to those measuring time with number of calls to the reaction model: perhaps unsurprisingsince calling the reaction model is the most time-consuming part of search.
Revised: In general, the conclusions measuring time with wall-clock arevery similar to those measuring time with number of calls to the reaction model: perhaps unsurprisingsince calling the reaction model is the most time-consuming part of search.

Original: However, when time ismeasured by number of calls to the value function, Retro* is decisively worse than MCTS: solutionsare found later, and fewer solutions are generally found by any given time.
Revised: However, when time ismeasured by number of calls to the value function, Retro* is decisively worse than MCTS: solutionsare found later, and fewer solutions are generally found by any given time.

Original: We believe this isbecause for MCTS, only a handful of molecules are evaluated after the reaction model is run (since asingle random action is taken), however for Retro* all molecules produced by the value function areevaluated with the reaction model (because the minimum cost path must be computed).
Revised: We believe this isbecause for MCTS, only a handful of molecules are evaluated after the reaction model is run (since asingle random action is taken), however for Retro* all molecules produced by the value function areevaluated with the reaction model (because the minimum cost path must be computed).

Original: This suggeststhat Retro* would not perform well if an expensive value function were used (e.g. rollouts).
Revised: This suggeststhat Retro* would not perform well if an expensive value function were used (e.g. rollouts).

Original:  Conclusions and Future Work
Revised:  Conclusions and Future Work

Original: After a careful re-implementation of existing methods, our main finding in Section 3 is that theperformance of different CASP algorithms is much more similar than previously reported.
Revised: After a careful re-implementation of existing methods, our main finding in Section 3 is that theperformance of different CASP algorithms is much more similar than previously reported.

Original: Thisseems to suggest either that newer algorithms are needed to improve upon the deficiencies of existingalgorithms, or that the choice of search algorithm is less important than previously thought – perhapsthe most important choice is a good policy and value function.
Revised: Thisseems to suggest either that newer algorithms are needed to improve upon the deficiencies of existingalgorithms, or that the choice of search algorithm is less important than previously thought – perhapsthe most important choice is a good policy and value function.

Original: Future work will include addingadditional algorithms such as proof number search [Kishimoto et al., 2019] to the benchmark, andexpanding our evaluation metrics to also include route quality
Revised: Future work will include addingadditional algorithms such as proof number search [Kishimoto et al., 2019] to the benchmark, andexpanding our evaluation metrics to also include route quality

Original: A Details on Algorithm Implementations
Revised: A Details on Algorithm Implementations

Original: All code was written in python.
Revised: All code was written in python.

Original: For retro*, the cost of all reactions was the negative log probabilityfrom the template classifier, as in [Chen et al., 2020].
Revised: For retro*, the cost of all reactions was the negative log probabilityfrom the template classifier, as in [Chen et al., 2020].

Original: The constant value function was given avalue of 0, and the outputs of the neural network value function was used without adjustment.
Revised: The constant value function was given avalue of 0, and the outputs of the neural network value function was used without adjustment.

Original: Forboth AND/OR and OR MCTS, the P-UCT bound function from Segler et al.
Revised: Forboth AND/OR and OR MCTS, the P-UCT bound function from Segler et al.

Original: [2018] was used with
Revised: [2018] was used with

Original: C = 10 , and P values obtained by applying softmax normalization to the log probabilities from thetemplate classifier with a softmax temperature T = 3 .
Revised: C = 10 , and P values obtained by applying softmax normalization to the log probabilities from thetemplate classifier with a softmax temperature T = 3 .

Original: The values of T and C were chosen from ahyperparameter sweep on a small set of sample molecules.
Revised: The values of T and C were chosen from ahyperparameter sweep on a small set of sample molecules.

Original: The constant value function had a valueof 0 .
Revised: The constant value function had a valueof 0 .

Original: 5 , and the values from the neural network value function were adjusted by f ( x ) =
Revised: 5 , and the values from the neural network value function were adjusted by f ( x ) =

Original: exp ( − x )(because this neural network predicts a cost , it needed to be adjusted so that higher cost implied lowervalue).
Revised: exp ( − x )(because this neural network predicts a cost , it needed to be adjusted so that higher cost implied lowervalue).

Original: To prevent infinite expansion, the search trees for all algorithms were limited to a maximumdepth of 20 for all tree types.
Revised: To prevent infinite expansion, the search trees for all algorithms were limited to a maximumdepth of 20 for all tree types.

Original: AND/OR MCTS is a simple variant of MCTS on AND/OR tree.
Revised: AND/OR MCTS is a simple variant of MCTS on AND/OR tree.

Original: Like OR MCTS, it is an iterativealgorithm where one starts at the root node, descends the tree to a leaf node, either expands orevaluates this node, then propagates a series of values upwards.
Revised: Like OR MCTS, it is an iterativealgorithm where one starts at the root node, descends the tree to a leaf node, either expands orevaluates this node, then propagates a series of values upwards.

Original: At an OR node, an AND node isselected using the same P-UCT method as in OR MCTS (whichever node has the highest value +exploration bound).
Revised: At an OR node, an AND node isselected using the same P-UCT method as in OR MCTS (whichever node has the highest value +exploration bound).

Original: At an AND node, all of its children are visited, and the value propagated upwardsis the minimum value of the children.
Revised: At an AND node, all of its children are visited, and the value propagated upwardsis the minimum value of the children.

Original: This is the key difference to normal MCTS, where only a singlechild of one node is visited.
Revised: This is the key difference to normal MCTS, where only a singlechild of one node is visited.

Original: B Additional Results
Revised: B Additional Results

Original: [Figure] [Figure]
Revised: [Figure] [Figure]