Automated Design of Metaheuristic Algorithms: A Survey
Authors: Qi Zhao
Qiqi Duan
Bai Yan
Shi Cheng
Yuhui Shi
Venue: Accepted by TMLR ()

Abstract: Metaheuristics have gained great success in academia and practice because their search logic can be applied to any problem with available solution representation, solution quality evaluation, and certain notions of locality. Manually designing metaheuristic algorithms for solving a target problem is criticized for being laborious, error-prone, and requiring intensive specialized knowledge. This gives rise to increasing interest in automated design of metaheuristic algorithms. With computing power to fully explore potential design choices, the automated design could reach and even surpass human-level design and could make high-performance algorithms accessible to a much wider range of researchers and practitioners. This paper presents a broad picture of automated design of metaheuristic algorithms, by conducting a survey on the common grounds and representative techniques in terms of design space, design strategies, performance evaluation strategies, and target problems in this field.

Published in Transactions on Machine Learning Research (02/2024)
Automated Design of Metaheuristic Algorithms: A Survey
Qi Zhao
zhaoo@@ssteeh..ede. edu..cn
Qiqi Duan
117498850mai..susttee edu.
Bai Yan'
yenbüsustech..dle
Shi Cheng
cheng@tsnm.edu.c
Yuhui Shit.f
shiyh edu.cn
Southern University of Science and Technology, China
Harbin Insfilute of Technology, China
Shoandi Normal Universily, China
Reviewed on OpenReview: https: // wptpr/veee.........mm net/ forum? id= ghtHauF52)
Abstract
Metaheuristics have gained great success in academia and practice because their search logic can be applied to any problem with an available solution representation, solution quality evaluation, and notion of locality. Mamually designing metaheuristic algorithms for solving a target problem is criticized for being laborious, error-prone, and requiring intensive specialized knowledge. This gives rise to increasing interest in automated design of metaheuristic algorithms. With computing power to fully explore potential design choices, the automated design could reach and even surpass human-level design and could make high- performance algorithms accessible to much wider range of rescarchers and practitioners. This paper presents a broad picture of automated design of metaheuristic algorithms,, by conducting a survey on the common grounds and representative techniques in terms of design space, design strategics, performance evaluation strategics, and target problems in this field.
1 Introduction
Metaheuristic algorithms are stochastic search methods that integrate local improvement with high-level strategies of escaping from local optima (Glover and Kochenberger, 2006). Representatives include genetic algorithm (GA) (Holland) 1973). simulated annealing (SA) (Kirkpatrick al. 1983), tabu search (Glover 1989), particle swarm optimization (PSO) (Kennedy and Eberhart 1995), ant colony optimization (ACO) (Dorigo et al. 1996), and memetic algorithms (Moscato 1999), etc. In contrast to analytical methods and problem--peciiie heuristics, metaheuristics representation, solution quality evaluation, and a certain notion of locality, in which locality denotes the ability to generate neighboring solutions via a heuristically-infommel function of one or more incumbent solutions (Swan et al. 2022). Such capability has enabled metaheuristics to gain broad success across various fields.
conduct search on any problem with available solution
Usually, human experts requested to manually tailor algorithms to target problem to obtain good enough solutions. Although many manually tailored algorithms have been reported to perform remarkably various problems, manual tailoring suffers from apparent limitations. First, the manual tailoring process could be laborious, which may cost the expert days or weeks conceiving, building up, and verifying the algorithms. Second, manual tailoring could be error-prone due to the high complexity the target problem
'Corresponding author.

---

Published in Transactions on Machine Learning Research (02/2024)
and the high degree of freedom in tailoring the algorithm. Third, the manual process is untraceable regarding what motivates certain design decisions, losing insights and principles for future reuse (Swan et al. 2022 Lastly, manual tailoring is unavailable in scenarios with an absence of human experts.
The automated design is a promising alternative to manual tailoring to address the limitations. It leverages today's 's increasing computing resources to create metaheuristic algorithms to fit for solving a target problem. Herein, the computer (partly) replaces human experts to conceive, build up, and verify design choices. Human experts can be involved but are not necessary during the process. Therefore, the automated design could make high- high-performance algorithms accessible to much broader range of researchers and practitioners. This is significant in response to the lack of time and labor resources. Furthermore, by leveraging computing power to fully explore potential design choices, the automated design could be expected to reach or even surpass human-level design. In the long run, it could be a critical tool in the pursuit of autonomous and general artificial intelligence.
By automatically tailoring algorithm components, structures, and hyperparameter values, the automated design could find either instantiations/ /variants of existing algorithms unseen algorithms
Related work:
with novel structures and component compositions (Stützle and López-Ibáñez 2019 Qu et al. 2020). In principle, the automated design can be conducted either offline by a target distribution of problem instances or online by the search trajectory (Swan et al.) 2022). Most automated design works follow the offline manner, which is significant for scenarios where one can afford a priori computational resources (for design) to subsequently solve many problem instances drawn from the target domain. We distinguish the relationship of automated design to related topics, i.e. automated algorithm selection (Kerschke et al., 2019), automated
algorithm configuration (Huang et al. 2020 Schede et al. 2022). adaptive operator selection and hyper-
heuristics (Burke et al.
2013
Pillay and Qu 2018 to avoid conceptual confusion:
Automated algorithm selection (Kerschke et al. 2019 chooses algorithms from a portfolio for solving a specific problem or problem instance. It alleviates the limitations of manual algorithm design by the selection mechanism that allocates suitable algorithms to different problems or problem instances. The output of the selection is one of the existing algorithms, instead of a customized algorithm for the target problem. From this perspective, automated algorithm selection is an independent topic with respect to automated design.
Automated algorithm configuration appears in two kinds of literature. The first kind is offline luning or online controlling hyperparameters of an existing algorithm (Fiben el. al., 1999 Huang et al. 2020). The output is an instantiation of the existing algorithm. The second kind involves both offline hyperparameter tuning and algorithm component composition via representing components as categorical parameters (Blot et al. 2017 Aydın et al. 2017 Blot et al. 2019 Sae-Dan et al., 2020: Tari et al.] 2020a). Despite subjecting to given algorithm templates, the second kind performs the same as a part of automated design, i.e., component composition and parameter configuration. Thus, the second kind of automated algorithm configuration falls into automated design.
Adaptive operator selection and hyperheuristics use high-level automated methods to select or gen- erate low-level heuristics/ heeurisics/meetaaeuriiii metaheuristiss (Burke et. al. 2019 The high-level methods are usually heuristics/mettan metheeeritataaeeesssssesss e.g., genetic programming (GP) (Koza) 1994), as well as machine learning and data mining (c.g., exploratory landscape analysis) methods; the primary goal of hyperheuristies
is raising the low-level heuristics' generality (Swan et al. 2017 Burke al. 2019 Pillay and Qu 2027). In comparison, methods for automated design include not only GP, but also those from other fields, e.g., from the hyperparameter optimization of automated machine learning (Hutter et al., 2019); goals of automated design include but are not limited to generality (Ye et al. 2022a). In this regard, the hyperheuristics that generate metaheuristic algorithms can be seen as a part of automated design of metahouristic algorithms.
Many efforts have been devoted to the automated design of metaheuristic algorithms in recent years. There have been several surveys (Kerschke al. 2019 Huang et al. 2020 Stützle and López-Tbáñcz 2019 Schode et al.] 2022) relating to some of the efforts, but a comprehensive compilation is lacking. The surveys in Kerschke et al. (2019); Huang et al. (2020): Schede al. (2022) are for automated algorithm selection
2

---

Published in Transactions on Machine Learning Research (02/2024)
configuration, the scopes of which differ from or only cover part of automated design according to the above concept discrimination. The work in Stützle and López--báñez (2019) is for automated design and focuses on the methods of design. However, apart from the methods, different types of design space, ways of algorithm representations, paradigms for searching design choices, and metrics for evaluating the designed algorithms are also essential for conducting automated design. A systematic survey of these essentials is necessary but still lacking.
This paper presents broad picture of automated design of metaheuristic algorithms, by conducting survey on the common grounds and representative techniques in this field. In the survey, we first provide a taxonomy of automated design of metaheuristic algorithms by formalizing the design process into four modules, i.e., design space, design strategies, performance evaluation strategies, and target problems. Unlike related surveys organized from the perspectives of methods for algorithm configuration/ design, our taxonomy involves important elements of automated design that have not been involved related surveys, such as types of design space, algorithm representations, and applications. Morcover, our taxonomy provides comprehensive understanding of the four main modules of the automated design process, which would allow interested readers to easily overview existing studies the modules of interest and make comparisons. Then, we overview the common grounds and representative techniques with regard to the four modules, respectively. We further discuss the strengths, weaknesses, challenges, and usability of these techniques, which would give researchers a comprehensive understanding of the techniques and provide practitioners with guidance on choosing techniques for different algorithm design scenarios. Finally, we list some research trends in the field, which would promote the development of automated design techniques for serving and facilitating metaheuristic algorithms for complicated problem-solving.
Contributions:
The rest of the paper is structured as: Section 2 inttroduces preliminaries of this survey; Sections 3 and review and discuss the research progress on the four main modules of automated design of metaheuristic algorithms, i.e, design space, design strategies, performance evaluation strategies, and target problems, respectively; Scction points out future research directions; finally Section concludes the paper.
2 Preliminaries
2.1 Formulation of Metaheuristic Algorithm Design
Given a target problem, through algorithm design, we would like to find an algorithm (or algorithms) fit for solving the problem:
arg max EE [Ep[(((()] iEI,PEP, AEs
(1)
where A is the algorithm to be designed; S is the design space, from where A can be instantiated; is instance from the target problem domain Z; P : S x I - R is metric that scores the performance of A by a run of A on i. Because metaheuristic algorithms conduct stochastic search, we need to estimate the expected performance over P, i.e. multiple runs of A result in multiple P
In reality, the distribution of problem instances I usually unknown; and one cannot exhaust all the instances during the design process. The common practice of settling this to consider a finite set of instances from I. Consequently, Eq. reformulated as
arg max E,, Ep [P|P()]]]]] & CI, .T), PEP,
(2)
AEs
where 1 the finite set of problem instances that are targeted at time (i.e., iteration of the design process. The instances either be fixed (i.c., I1 Ig design process. The output of solving Eq. 2 is an algorithm (or algorithms) with the best performance on the instances. To avoid the designed algorithms overfitting, the design process should be followed by a test on the designed algorithms' generalization to instances from IV 12, ... ,I].
Ir) dynamically changed during the
Since Equation is black-hox, it often solved an iterative manner.
3

---

Published in Transactions on Machine Learning Research (02/2024)
Designed algorithm
Performance Evaluation Strategy
Targeted Problem
Design Space
Design Strategy
Performance of the designed algorithm
Figure 1: Abstractive process of automated design of metaheuristic algorithms.
2.2 Automated Design of Metaheuristic Algorithms
We abstract the general process of automated design of metaheuristic algorithms into four modules, shown in Fig. [ First, the design space collects candidate design choices (e.g,, computational primitives, algorithmic operators, and hyperparameters), which regulates what algorithms can be designed in principle. Second, the design strategy provides a principle way to design algorithms by selecting and assembling design choices from the design space. Third, the performance evaluation strategy defines how measure the performance of the designed algorithms. The measured performance guides the design strategy to search for desired algorithms. Finally, because the design aims find algorithms to fit for solving target problem, the target problem acts as external data to support the performance evaluation. The four modules constitute this survey's taxonomy of related research efforts.
2.3 Survey Scope
As stated in Section [ the automated design, in principle, can be conducted either offline with a target distribution of problem instances or online with the search trajectory. Most automated design works follow the offline manner. In contrast, the online manner is often referred to in hyperheuristic adaptive operator selection literature (related surveys can be found in Burke et al. (2013) and Pillay and Qu (2018 ). The scope of this survey is the research efforts on oflline aulomated design.
3 Design Space
The design space defines what metahouristic algorithms can be found in principle. From the perspective of the elements within the design space, the current design space may be classified into two categories, namely, the design space with computational primitives and the design space with existing algorithmic operators. We first overview the research progress on the design space with computational primitives and discuss its strengths, weaknesses, and challenges in subsection 3.1 We then overview the design space with existing algorithmic operators, with discussion on its strengths, weaknesses, and challenges in subsection 3.2 Finally, we analyze the usability of the two categories and their corresponding algorithm representations at the end of the section.
3.1 Design Space with Computational Primitives
Computational primitives define the elementary actions for computation. They include: arithmetic prim- itives, c.g., +, *, /: trigonometric primitives, o.g., sin(), cos(); probabilistic and statistic primitives, e.g., max(), mean(); operating instructions, e.g., swap, duplicale; etc. The design space consisting of these primitives enables designing metaheuristie algorithms by choosing primitives from the space, combining the chosen primitives algorithmic operators, and composing the operators as an algorithm. The ways of representing algorithms over this design space include tree representation and linear array representation in the literature.
Tree represcntation with computational primitives: The binary GP-style tree (ozaa 1994 is the
dominant representation for algorithms designed over computational primitives (Poli et al. 2005a Burke
et al. 2010: Rodriex- and Ochoa 2011 Richter and Tauritz 2018 Richter et al. 2019 As shown
4

---

Published in Transactions on Machine Learning Research (02/2024)
X1
X2
Figure 2: Example of a metaheuristic operator represented by a binary GP-style tree. This example repre- sents the whole arithmetic crossover operator (Eiben et al. 2003), where x and x₂ are the input variables (i.e., two parent solutions), is weight parameter, and +, +,*,- are primitives. The operator performs w. X1 (1 w) X2 in mathematical form.
in Fig. 2 terminals of the tree are the inputs, including variables, parameters, and constants; non-terminals are the computational primitives. The operation specified by the computational primitive performs by using the values obtained from child nodes; this process repeats until the root node reached.
The tree was usually employed to represent a single algorithmic operator instead of an entire algorithm (Poli et al.
2005a b Burke et al.) 2010 Vázquez Rodriiguz and Ochoa 2011 Richter and Tauritz 2018 Richter
ct al.
2019). because representing an entire algorithm leads to a large tree and could be infeasible to find an
appropriate tree over the large search space (Woodward and Bail 2009). An algorithm template is required to insert the operalor represented by the tree to an algorithm. The template determines the algorithm structure, i.e. the execution order and logic of the involved operators. For example, in Poli et al. (2005a) and Poli et al.] (2005b), the particle update operator was represented by a tree and designed; then, the designed operator was inserted into the PSO template (i.e., a recursive process of particle velocity update, particle location update, and best particle update) (Kennedy and Eberhart 1995). Other examples include representing the mating selection operator of GA (Richter and Tauritz 2018 and the individual selection operator for mean-update in CMA-ES (Richter et al. 2019).
Due to GP tree's expressiveness in representing programs over the primitives, the tree representation has been widely incorporated into the design space with computational primitives. An open challenge of using the tree representation avoiding underfitting and overfitting. Current GP-based automated design literature presets the tree's size or structure (Richter al.) 2019 Nguyen et al. 2022), which can be seen as means of regularization that mitigates over-fitting More strategies, c.g., model (i.c., tree) selection that determines the optimal amount of model complexity (Hastie et al.) 2009), are worth investigaling in the automated design field to cope with the challenge. The GP generalization survey in Agapitos et al. (2019) provides a comprchensive analysis of these strategics. Another challenge is that the tree has boen limited to represent a single algorithmic operator. The indirect encoding (Liben and Smith 2015 Stanley et al.) 2019 may
alleviate this challenge. Its generative and developmental nature allows learning and reusing building blocks of the tree, which could help improve the search efficiency and scale up the complexity of the algorithm being represented (Eiben and Smith 2015
Linear array representation with computational primitives: It uses a linear array of computational primitives to represent a metaheuristic operator (Goldman and Tauritz 2011 Woodward and Swan 2012). An example is shown in Figure 3 The primitives execute sequentially along with the array; primitives are associated with indexes and parameters to determine which input variable(s) the primitive should execute on. Similar to the tree representation, the lincar array was utilized to represent a single algorithmic operator instead of an entire algorithm (Goldman and Tauritz 2011 Woodward and Swan 2012) because of the weak expressiveness of the linear array. Related works include representing the crossover Goldman and Tauritz, 2011) and mutation [Woodward and Swan 2012 operators of GAs.
A way of augmenting the expressiveness is using the Push language (Spector 2001 to map the linear array of primitives to an executable algorithm (Lones) 2019 Kamrath et al. 2020 Lones 2021). ). Push is a Turing- complete, stack-based typed language used in GP Spector 2001)). With Push, the primitives are typed, and each primitive executes upon its corresponding type stack [Lones] 2021), rather than executing sequentially along with the array. This based typed system enables the linear array to be more expressive and
5

---

Published in Transactions on Machine Learning Research (02/2024)
(swap(3; 5), merge (1, F, 0.7)]
Figure 3: Example of a metaheuristic operator represented by a linear array of computational primitives. The example is derived from Goldman and Tauritz (2011). It represents crossover operator consisting of swap and merge primitives. The indexes and parameters associated with the primitives are in brackets. This operator produces offspring by first swapping the 3rd clement of parent 1 and the 5th clement of parent 2, then merging the 1st element of parent 1 (by a weight of 0.7) with a random element of parent 2.
ensures the represented algorithm to be syntactically valid {Lones] 2021 with Push was used to represent individual- local search [Lones 2019 Kamrath et al. 2020) and population-based (Lones] 2021) algorithms.
The linear array representation
Overall, using the design space with computational primitives indicates that the design is from scratch with little human bias. Although unbiasedness makes the design space more difficult to search than biased ones, it provides the design space generalization to different target problems. Furthermore, this kind of design space has the potential to produce innovative algorithms that go beyond human experience and surpass the performance of existing algorithms. This potential is appealing and could attract more researchers and practitioners to study, analyze, interpret, and apply automated design techniques with the advancement of representations and methods for manipulating the representations.
3.2 Design Space with Algorithmic Operators
Operators, c.g., the single-point crossover (Eiben ct al. 2003), tournament selection (Eiben et al. 2003 greedy randomized construction (Feo and Resende 1995). and tabu list. mechanism (Glover 1986), are the functional components of an algorithm. They have been used building blocks to constitute the design space. This design space enables designing metaheuristic algorithms via choosing operators from the space, composing the chosen operators as an algorithm, and configuring endogenous hyperparameters. The ways of representing algorithms over this design space include linear array representation, graph representation, and tree representation in the literature.
Linear array representation with algorithmic operators: The linear array representation is the com- mon option in the literature with design space with algorithmic operators. Tt uses categorical identifiers to index operators and numerals to refer to (conditional) hyperparameter valuos; algorithm is subscquently represented by a linear array of the identifiers and numerals (Oltean] 2005 Diogan and Oltean 2009 Khud- aBukhsh et al.) 2009 Lopez-Ibanez and Stutzle, 2012 van Rijn et al. fixed length for casy manipulation. The common practice of exccuting the array of operators is predefining an algorithm template that maps the linear array to an execulable algorithm. For example, in Blot el al.) (2019), a local search template was constructed by four components, i.e., selection, exploration, perturba- tion, and archive; the template organizes the linear array of operators and parameters into local search algorithm. in Villalón et al. (2022). the PSO template (Shi and Eberhart 1998) was adopted to map the linear array to a variant of PSO.
| 2016). The array is normally with a
Oltean 2005 and Diogan and Oltean (2009) provided an alternative to the predefined template, in which each operator was associated with a sequence mumber; the operators executed according to the sequence. This results in unfolded algorithms with serial executions of operators. The Backus Naur Form grammar [Ryan et al., 1998) was introduced as another alternative to the template (Mascia al. 2013). It has been widely employed (Tavares and Pereira 2012; Mascia et al. 2014 Pagnozzi and Stützle 2019 Miranda and Prudéncio 2020). As shown in Fig. 4 the grammar is formalized as tuple (N, S.P. where N is the non-terminals (operators); T is the terminals (the algorithm's 's inputs, e.g., parameters); S E N is called axiom; P is production rules that manage the operators' execution. This grammar ensures the syntactic validity of the represented algorithm by appropriately setting the production rules (Ryan el al. 1998).
Graph representation with algorithmic operators: In contrast to the lincar array representation that limits the designed algorithms' structure within the predefined template or production rules, the graph is well-defined format describing arbitrary orderings of process, resulting in flexible algorithm structures.

---

Published in Transactions on Machine Learning Research (02/2024)
N Initialization, cressover, mutation, selection
number creas, preb reaet
<algerithm>
the following preduction rules:
cinstiallizatoon2::= uni form random | Latin hypercube sample.
<crossover>::= one-point и-point (number cross n). <mutation>::= reset(prob reset).
<selection>::- tournament | roulette wheel greedy.
number cross::= 1
prob reset::= 0.1 0.2 0.3.
<algorithm>::= <btthalaateanseeeeeooaaaaaaaaeed
(a) Grammar.
<algorithm>::- cuniform zandon imitialleation<ttornaaa mating selectionn<<oe-poii
crossovez><reset, (prob zeset=0.21 20 2xgrreed selection>
(b) An algorithm instantiated according the grammar.
Figure 4: Example of grammar for designing genetic algorithms,
(a), the production
rule <initialization> means that the initialization operator can be either uniform random Latin hypercube sample; the meaning of other rules is in the same fashion. According to the <algorithm> rule, two clitism GA structures can be reprosented: one scarches by crossover and mutation; the other searches by mutation only. An algorithm instantiated by the rules is shown in (b).
(0) tournament
(!) reset mutation
(2) greedy selection
(I) one-point crossover
mating selection
3
tounnmennt
Input
mating selection
reset mutation
greedy selection
Output
0
5
one-point crossover
Figure 5: Example of metaheuristic algorithm represented by a directed acyelic graph. The example is derived from Ryser- Welch et al. (0016). Nodes indexed with 2, 4 represent algorithmic operators. "(0)" for node 1 means that node connects from node 0; the meaning for other nodes is the same fashion. The excmplified algorithm consists of one input, one output, and three operators 1, 2, 3. Operator inactivated because it is not on the path from the input to the output.
Nodes of the graph refer to operators from the design space; directed edges manage the operators' execution flow. Specifically, the directed acyclic graph from Cartesian GP (Miller 2011) was employed in Kantschik et al. (1999); Shirakawa and Nagao (2009h; Kkyser-Welde et al. (2016). in which each node is connected from a previous node or the input in feed-forwand manner; the nodes on the path from the input to the output are activated and constitute the represented algorithm. An example is depicted in Fig.
A more flexible directed cyclic graph representation was proposed in Tisdale al. 2021 in which nodes were allowed to be connected from multiple previous nodes, and edges were associated with weights and orders to determine i) the computational resources allocated to nodes, ii) the inputs and outputs of nodes, and iii) the starting point of the represented algorithm. These features allow the graph to express various (μ/p+X) X) and (и/p, A) EA structures. The graph representation in Tisdale el al. (2021) is well-developed to express EA structures but lacks generalization to describe other types of metaheuristics For this, (Zhao et al.) (2022 2023a) proposed new graph representation, in which i) the directed edges enable sequence structures of operator compositions; ii) each node is open to connecting forward multiple nodes, allowing branch structures; and fii) cycles realize loop structures. These principles enable the graph to represent. different

---

Published in Transactions on Machine Learning Research (02/2024)
types of metaheuristic algorithms, e.g., algorithms with unfolded tandem operators, algorithms with inner loops of local search, and algorithms with multiple pathways.
Tree representation with algorithmic operators: The binary GP-style tree (Koza 1994) was also considered to represent an algorithm over operators (Smorodkina and Tauritz 2007 Rivers and Tauritz, 2013). Terminals of the tree are the algorithm's 's inputs. Non-terminals of the tree are the algorithm's 's operators and (conditional) hyperparameters from the design space. This representation is similar to the tree representation with computational primitives in Fig. 2 The difference is that the non-terminals change from computational primitives to operators.
Overall, the design space with existing algorithmic operators is more compact than that with computational primitives. Purthermore, using this design space allows the design inheriting from prior expert knowledge and experimentation. The compact and knowledge-induee and accessible with fewer computing resources than that over the space with computational primitives. The weakness is that the space may bias the designed algorithms in favor of human-made ones, thereby reducing the flexibility and innovation potential; besides, users may need to select and collect operators to form the space by themselves, which is new burden.
space makes the design over the space possibly easier
3.3 Usability
The usability of different kinds of design space and representations varies in different algorithm design scenar- ios. The design space with computational primitives generally requires a large amount of computing resources to fully explore. This design space would be desired for metaheuristic researchers because its innovation potential. The design space is not the first choice for practitioners and rescarchers from other communities because it is not easy for these users to interpret the algorithms derived from computational primitives. The tree represcntation is preferred when using this design space due to the tree's strong expressiveness and maturity along with the advancement of GP.
The design space with existing algorithmic operators can be dense, which makes the space accessible to scenarios with limited computing resources. Furthermore, the space can be formed with good design choices from prior algorithms and the user's domain expertise. Therefore, this space is recommended if such prior algorithms or expertise are available. If the target problem is not complicated and does not require a complex algorithm, the linear array reprosentation may be considered since it is easy to be implemented and manipulated; otherwise, the graph representation is advised because of its strong expressiveness. In addition, the graph representation is easy to understand, which is important for users outside the metaheuristic community.
Both the design space with computational primitives and that with algorithmic operators are mixed and conditional, no matter what representation used. That is, the space is a mix of discrete primitives/opera- tors and continuous hyperparameter value the chosen operator conditions what hyperparameters will be involved. The mixed and conditional nature requires specific design stralegies (detail in Section 4 to handle. Model-free heuristic methods may be required for directly manipulating the mixed and conditional space; otherwise, the space should be projected to another space with a single type of entities (o.g., discretizing continuous hyperparameter values) and align the representations with different conditional hyperparameters to a unifidd prototype, in order to leverage model-based and learning-baat based methods for manipulation.
4 Design Strategies
The design stratogy is a principled way to generate metaheuristic algorithms over the design space. Basod on whether there is statistical inference that guides the design process, current design strategies may be classified into two categories, i.c., model-free strategies and model-based strategios. We first overview the model-free design strategies and discuss their strengths, weaknesses, and challenges in subsection 4.2 We then overview the model-basee design strategies with a discussion on its strengths, weaknesses, and challenges in subsection
4.1 Finally, we analyze the usability of the two categories at the end of the section.
2ome hyperparameters may be discrete.
8

---

Published in Transactions on Machine Learning Research (02/2024)
4.1 Model-Free Strategies
Model-free stralegies search for desired algorithms in a trial-and-error fashion. They mainly include local search and evolutionary search in the literature.
Local scarch: It mainly refers to the iterative local search (Lourenço et al. 2003) utilized in the ParamILS pipeline (Blot et al. 2019 Tari et al.) 2020a KhusaBBukhhh el al. 2009]. It usually incorporated with the design space with algorithmic operators. The key steps are local improvement and global perturbation, in which the former changes one dimension of the algorithm representation; the latter reinitializes the current algorithm once reaching local optima (Hutter et al. 2009). The local search is often conducted over a dense design space with good choices from prior expert knowledge and experimentation. Therefore, the design strategy itself is often deemphasized.
2003 is a more popular design strategy and is Evolutionary search: Evolutionary search Eiben et al.
widely used in the GP-based algorithm design literature. The algorithm representation determines what specified evolutionary operations can be used. For the tree representation, the sub-tree crossover and muta-
tion are common options (Poli et al.) 2005a b Richter and Tauritz 2018 Richter et. al. 2019 Smorodkina
and Tauritz 2007 Rivers and Tauritz 2013 p. That is, for pair of tree representations, a node is randomly
selected from each tree, respectively, and then, the sub-trees starting at the selected nodes are exchanged
(crossover); each sub-tree starting at a randomlly selected node is reinitialized (mutation) (Eiben el al., 2003). For the linear array representation, various standard evolutionary crossover and mutation are avail-
able (Goldman and Tauritz 2011
Woodward and Swan 2012 Oltean 2005 Diogan and Oltean 2009 van
Rijn et al.] [016 Ryan al.)
1998 Ye et al.) 2022a). Evolutionary search is also employed as meta-learner
within the context of meta-learning, such learning the update rules of evolutionary strategies (FS) Lange
et al.
(2023) parameterized the update rule of ES by neural work with the self-attention mechanism to fulfill the invariance in the ordering of ES solutions; the parameters were optimized by evolutionary search, resulting in new ES capable of generalizing to unseen optimization problems.
2023 In particular, Lange et al.
Evolutionary search is an adaptive system (Holland 1962) that provides higher-level adaptation across different target problem instances than local search. The adaption is realized by evolving a population of algorithms fitting across the problem instances. The adaption can be further boosted by evolving from easy to difficult instances (if the difficulty is accountable), in which the knowledge found from easy instances can be anchors to enhance the adaption to difficult instances.
An open challenge behind evolutionary design strategics, as well as local scarch ones, is the lack of principles for handling the conditional design space, i.e., the chosen algorithmic operator conditions what hyperpa- rameters should be involved. The conditional design space essentially makes the algorithm design task to be bi-level. The upper-level searches for the optimal operator composition, and the nested lower-level tunes the hyperparamoters within the operators chosen at the upper level. Principle strategies for managing the bi-level search deserve more investigation.
4.2 Model-Based Strategies
Model-based strategies use stochastic statistical models to guide the algorithm design process. They include estimation of distribution, Bayesian optimization, reinforcement learning, and the emerging large language models in the literature.
Estimation of distribution: The "estimation of distribution" was the design strategy in irace (López Πbáñez et al., 2016)) It is usually incorporated with the design space with algorithmic operators and the linear array algorithm representation. Its main idea is to build and sample an explicit probabilistic model of the distribution of optimal algorithms over the design space. Specifically, each entity (an algorithmic operator or hyperparameter) of the linear array algorithm representation is assumed to follow particular distribution; the multinomial distribution and normal distribution are assumed for categorical operators/ pperators/hyperparaaa /bpperarppprrrerrsssssss/ and numerical hyperparameters, respectively. The probabilistic model of the distribution is formed in an
Note that irace did claim its design strategy as estimation of distribution [Lppe Iháiee et al. 2016 We abstract the
strategy an "estimation of distribution" for ease of description.
9

---

Published in Transactions on Machine Learning Research (02/2024)
Sample algorithms
Update surrogate with the new algorithu
Train surrogate to predict algorithms' performance
Sample new algorithm via maximizing acquisition function
Figure 6: Workflow of Bayesian optimization for algorithm design.
iterative fashion; each iteration consists of three steps: 1) sampling candidate algorithms according to the particular distribution: 2) selecting promising algorithms among the candidate algorithms by means of racing Maron and Moore, 1997) (detailed in Section 5.3); and 3) updating the distribution by the promising algorithms to approach the optimal distribution. The distribution at the initial iteration can be formed by randomly sampled algorithms or user-specified ones.
The "cstimation of distribution" is widely used in the literature (Lopez-bbanez and Stutzle 2012 Franzin
and Stützle, 2019; Bezerra et al., 2014 2016; Pagnozzi and Stützle] 2019 Diaz and López-Ibáñee 2021
Brum et al. 2022) because its fast convergence and the popularity of irace (López-Ibáñez el. al., 2016).
In its default setting, each entity of the linear array algorithm representation was assumed to follow an independent univariate distribution for case of manipulation (López-Ibáñcz et al. 2016). This indicates the orthogonality among the entities, which is usually not the case and may make the design premature. Learning the covariance of distribution and leveraging other advanced methods, e.g., smoothing the update of the distribution (Wierstra et al. 2014) may mitigate the premature issue.
Bayesian optimization: Bayesian optimization is popular in hyperparameter optimization in the auto- Lindauer et al. 2022p. which has been introduced is used to search the design space with algorithmic operators and manipulate the linear array algorithm representations. Its core idea is learning a surrogate that models the mapping from algorithms to their performance on the target problem. The sampled algorithm for training the surrogate is obtained by maximizing the acquisition function. The random forest can be adopted as the surrogale because algorithms should be represented as mix of numeral parameters and categorical operators/ parameters, and random forest supports the mixed training data. The expected improvement {Joncs et al. 1998) was utilized as the acquisition function (Hutter et al. 2011 duc to its closed-form solution assuming that the designed algorithm's 's performance follows a normal distribution. The workflow is depicted in Fig. 6
mated machine learning field (Hutter et al. 2011: 2019
in metaheuristic algorithu design (Tang et al.) 2021 Ye et al.) 2022a).
Bayesian optimization obcys similar iteration process (i.c., algorithm sampling and model update) with estimation of distribution but with two unique features. First, Bayesian optimization leverages low- complexity surrogate model to predict the algorithm's 's performance, which saves computational effort at the cost of potential prediction error. Second, the acquisition function balances the exploration and exploitation in sampling over the design space. The balance is realized by preferring a new sample (algorithm) with high predicted performance (exploitation) and uncertainty (exploration).
Bayesian optimization is acknowledged to perform well on low-dimensional problems. There would be room for improving its scalability to high-dimensional design scenarios. The choice of surrogate models is another open issue. Metaheuristic algorithm design is black-box, in which the landscape of algorithm performance prediction varies when facing different target problems. Apart from the random forest, it is worth having more surrogate choices, e.g., neural networks, to improve the surrogate's accuracy and cope with different design scenarios (regarding target problem types, data richness, computing resources, etc.). Besides, the models can only manipulate fixed-length lincar array representations of algorithms. The fixed-length array limits the designed algorithm's structures, subsequently weakening the innovation that can be discovered and leading the model-based strategees only to approach sub-region of the design space.
10

---

Published in Transactions on Machine Learning Research (02/2024)
Agent's action Sample algorithms basod on the policy
Environment
Evaluate algorithms' performance
Update the policy
Reward & State
Figure 7: Workflow of reinforcement learning for algorithm design.
Reinforcement learning: The Markov decision process was introduced to design metaheuristic algorithms in Adriaensen and Nowē (2016); : Meng and Qu (2021). It is executed over the design space with algorithmic operators. The state relers to the operators from the design space (Meng and Qui 2021) or the algorithms constituted by the operators from the design space AAriacnsen and Nowé] 2016). The transition probability between each pair of states is learned according to the constituted algorithms performance on the target problem instances. Operators and are more likely to be chosen. The Markov decision process reveals the intrinsic relationships between each pair of design choices and their contributions to the overall performance [Adriaensen and Nowé 2016
algorithmss with better performance have higher transaction probabilities
Tabular reinforcement learning was employed in Buzdalova et al. (2014); Meng and Qu (0021), in which a tabulation records the reward and penalty of each design choice. Modern reinforcement learning approaches, e.g.; deep Q-network (Mnih et al. 2015) and proximal policy optimization (Schulman et al.) 2017), were introduced recently (Schuchardt et al. 2019 Sharma et al., 2019 Yi et al.) 2023b The action space is the design space with algorithmic operators. The state refers to the information from evaluating the current algorithm on certain problem instances, including features of the instances (c.g., the number and capacity of vehicles when largeting vehicle routing problems (Yi et al. 2023b ), features of the algorithm's solutions to the instances (c.g., solutions' fitness improvement over initial solutions (Yi et al. 2023b)). ), etc. In particular, the action of [Sharma et al. (2019); Sun et al. (2021); Yi et al. (2023b a) is modifying a certain operator of the current algorithm, resulting an algorithm with adaptively changed operators during problem-solvee Instead of using a fully connected policy network that only considers the current problem-solving state, Sun et al. (2021) proposed to adopt recurrent neural network to capture the time-dependent relations from the information till the current state. While the policy is lypically learned from scratch, Shala et al. (2020) proposed to use existing ES as teacher and learn the policy of CMA-ES's step-size adaptation upon the teacher by the guided policy scarch (Levine and Abbeel 2014), which accelcrated the policy learning process. The general framework of reinforcement learning for metaheuristic algorithu design is given in Fig. 7
Although reinforcement learning is less used than local and evolutionary design strategies, is promising in automated algorithm design due to three appcaling features. The first is the fine-grained design. Re- inforcement learning can learn to design at each time step rather than at each episode (i.e., iteration in the context of iterative local or evolutionary search-based design strategies). This enables building up an algorithm operator-by-operatee or sennttive-by-primitiv which could be expected to be more efficient than building the whole algorithm at once. The second is the value function. The value function realizes long- term planning of the design with consideration of not only the current state but also future state during a problem-solving. The third is the neural network policy model that restores the design knowledge within the neurons, which enables future reuse to unseen problem instances or transfer to related problems by transfer learning Taylor and Stone 2009 Zhu et al.) 2023 or continual learning (Khetarpal et al. 2022 techniques.
One concern is that the algorithms derived from the policy are often in lincar array representations. Current studies limit the algorithms to the sequential execution of the array. It is worth studying encoding-decoding mapping between the representation manipulated by the agent's is action and the executable algorithm to bring innovations in algorithm structures. Another concern is that the reward cannot be calculated in scenarios with fine-grained design from scratch unless all actions have been collected form a complete algorithm. This arises the sparse reward challenge.
11

---

Published in Transactions on Machine Learning Research (02/2024)
Large language models: Language is complex, intricate system of human expressions Zhao et al, 2023b). By pre-training Transformer models (Vaswani et al. 2017) over large-scale corpora, large language models (LLMs) are equipped with human knowledge memorization and compasitionality capabilities. These capabilitics enable LLMs to approximate or even surpass human decisions at an unprecodented level of performance (Zhao el al. 2023b; Mialon el. al. 2023). Some works have proposed leveraging LLMs as design strategies to design metahcuristic algorithms. They leverage LLMs through in-context learning, i.c., providing LLMs with a textual prompt and several algorithm demonstrations, LLMs generate algorithms by completing the word sequence of the input text, without training gradient update (Zhao et al.) 2023b). In detail, PRonera-Paredes et al. (2024) proposed FunSearch. Given an algorithm template (e.g., a greedy algorithm)) FunSearch prompts a pre-trained LLM (Codey (cod, 2023) ) to generate a population of programs (code snippets) of the key component (e.g., the priority function of the greedy algorithm); the programs are then evaluated, and feasible ones are sent to a database; the best k programs in the database are selected as new demonstrations in the prompt for the next round of program gencration. The above process iterates to find better programs. Precisely, the prompt consists of description of the larget problem, the k demonstrations, and the algorithm template with the key component being empty. Such a prompt aims to let the LLM spot patterns across the best programs and complement the key component in the template based on the patterns. Besides, FunSearch employed an island model that evolves multiple subpopulations of programs, which encourages the discovery of diversified programs. FunSearch discovered new algorithms that improve upon current ones on the cap set and online bin packing problems. This effectiveness is believed to be attributed to the LLM that acts as a source of diverse programs with occasionally interesting ideas rather than using much context about the target problem (Romera- Paredes et al., 2024).
Concurrently, Liu et al. (0024) proposed using the algorithm evolution using large language model (AEL) to design the guided local search algorithm (Voudouris and Tsang (199). It adopted an LLM to perform the roles of initialization, crossover, and mutation of an evolutionary process for evolving algorithms. The prompt describes the algorithm design task, parent algorithms, prompt-specific hints, expected output, and other hints. The prompt-specific hints instruct the LLM to perform different roles. Experiments on the traveling salesman problem (TSP) showed that with GPT-3.5-turbo, AEL could evolve elite algorithms in two days with minimal human effort and no model training. Ye et al.] (2024) proposed language hyperheuristics (LHHs) that leverage LLMs as high-level automated methods to generate low-level hcuristics. In particular, the reflective evolution was presented, in which a generator LLM is for generating heuristics and reflector LLM is for reflecting hints from the heuristics to guide the next round of generation.
Overall, LLMs bootstrap from rich human knowledge, which could be promising engines for designing algo- rithms concerning designing from scratch. With LLMs' compositionality over the prior knowledge, an explicit design space no longer necessary, enabling to discover innovative algorithms. The above works (Romera- Paredes et al.) 2024 Liu et al.] 2024 Ye et a 2024) demonstrate such promise. Meanwhile, these works incorporate LLMs with the population- based evolutionary framework. The framework empowers LLMs with diversity and global search capacities (Wu et al. 2024 Chao et al.) 2024) and helps mitigate L.MSs halluci- nation incurred by the lack of fact-checking. A challenge is that a non-trivial prompt engincering is required to reliably extract the knowledge from LLMs. Besides being the core engine of designing algorithms,, LLMs human-machine machine (algorithm design systems) interaction. By building powerful interactive caxe- interfaces with LLMs, many complicated decisions affecting the algorithm design pro- cess would be simplified. This would also increase the interpretability of the algorithm design process by elaborating on the algorithm design report in textual and graphical forms (Tornede et al. 2023
could significantly improve human-
4.3 Usability
In principle, any search method might be employed to search over the design space to produce algorithms. As algorithm design is black-box, there is not yet consensus and dominant strategy for designing algorithms.
The usability of the design strategies depends on the (arget problem-solving scenarios. If the design space is formed with a few existing algorithmic operators and there is a predefined algorithm template (e.g., designing a variable neighborhood scarch algorithm from several neighborhood scarch operators), the local search, estimation of distribution, and Bayesian optimization could be recommended. Among them, using Bayesian optimization as the design strategy is particularly preferred when targeting expensive problems
12

---

Published in Transactions on Machine Learning Research (02/2024)
Table 1: Summary of design strategies.
Design Strategies
Key features
With little sesumption on algorithm being designed.
Local Search 4.2
Model-
2 Flexible be integrated with various algorithm represeutations.
Bvolutionary Search 1:2,3
Exploration over the design space and adaption across problem instances.
Statistical inference provides interpretation.
Designed algorithms are subject to predefined structure.
Estimation of Distribution 4,5,8
Simplified statistics may lead to fast but premature convergence,
Bayesian Optimization 4,5,6,7
Surrogate saves computational effort incurred by algorithm evalmation.
Model- Basee
Reinforcement Learning
Reveal intrinsic relationships and contributions of design choices.
Large Language Models 10.11
Fine-grained design, long-term planning, and online design. Bootstrap from buman knowledge, an explicit design space is unnecessary. User-friendly human-machine interaction.
in which the computational cost is dominated by function evaluations, since the low-complexity surrogate model can save function evaluations.
When manipulating the design space with computational primitives, evolutionary search, e.g., GP, would be the first choice. Due to the ability maintain population of diverse algorithms, evolutionary search is also suitable for scenarios with demands for algorithm portfolios. Reinforcement learning is appealing for scenarios with a large amount of problems from the same or similar domains, in which the policy model enables the reuse and transfer of the knowledge of prior algorithm design to new problems. It. is also appealing for online scenarios. That is, learning the design policy offline and then deploying the policy online to generate algorithms during open-ended problem-solving. summary of the design stratogics is given in Table
5 Performance Evaluation Strategies
The performance evaluation strategy defines how to measure the performance of the designed algorithms. To conduct performance evaluation, one should first choose performance metric, then evaluate the performance according to the metric and identify desired algorithms by performance comparison. Tn particular, reducing the time cost of performance evaluation is vital because performance evaluation normally occupics most of the computational cost. of the design process. Hence, we first summarize the performance metrics employed in the literature and discuss their usability in subsection 5.1 Then, we review the performance evaluation and comparison strategies in subsection 5.2 Finally, report the strategics for reducing the time cost of performance evaluation and discuss their strengths and weakness in subsection 5.3
5.1 Performance Metrics
The metrics include the algorithm's solution quality, running time, anytime performance, and jointly con- sidering multiple motrics in the literature.
Solution quality: A commonly used performance metric is the algorithm's solution quality within a fixed computational budget. The solution quality is usually summarizing value based on the objective function value and constraint satisfaction on the target problem. The black-box optimization benchmarking study in
Hansen et al. (2022) constructed solution quality metrics for different types of problems, e.g., single- higle-/multt. objective, constrained, and noisy problems.
Running time: It measures the algorithm's 's running time until the amg reaches a performance thresh- old. The threshold can be the solution quality or a particular solution. The metric is formulated as Ye et al. 2022ah:
P(Ali) = r(),
(3)
where i,), is the running time of algorithm on target problem instance till reaching the threshold The running time often refers to the number of function evaluations. The wall clock time and CPU
13

---

Published in Transactions on Machine Learning Research (02/2024)
time may also be considered but may bring about unreprodueible results due to the difference in hardware, programming language, and coding style.
Anytime performance: It measures the algorithm'' ability to return high-quality solutions within any computational budget. The common practice to measure the anytime performance is calculating the area under the curve (AUC) of the empirical cumulative distrribution function of running time. Given a set of running time points (Tulu U} and a set of solution quality thresholds {Eyler V}, the AUC is calculated Ye et al. 2022a
P(A()- = EvEy Lvev( 2 (.)
(4)
|0| * IV]
where Tu) is algorithm A's solution quality on target problem instance i within To running time; (-) returns 1 if i,), 2 6v, and return O otherwise; |U] and [V] return the numbers of elements in sets U and V, respectively. A larger AUC value indicates a better algorithm performance.
An alternative way to measure the anytime performance is by the hypervolume indicator (López-Ibánez and Stützle] 2014). That is, the solution quality and running time can be seen as two conflicting objectives, i.e, better solution quality often demands longer running time. Thus, the elements of algorithm's per- formance profile (solution quality versus running time) can be seen as a set of Parcto non-dominate points regarding the two objectives. Therefore, the hypervolume value (Zitzler and Thiele 1999) of the Pareto non- dominate points with respect to a reference point measures the anytime performance. Higher hypervolume values indicate better anytime performance. This method does not require setting a priori solution quality thresholds.
Multiple-metric: Simultancously considering multiple performance metrics is of interest in certain scenar- ios. in Blot et al. (2016), the solution quality and memory usage were jointly considered for scenarios with memory limitations. In Blot et al. (0017); Tari et al. (2020b); Bezerra et al. (2020). the solution convergence and solution diversity were considered for designing multi-objective algorithms, i.e., multi-objective design of multi-objective algorithms; they used the hypervolume (Zitzler and Thicle 1999) to mcasure the algo- rithin's is solution convergence and used the A spread to measure the solution diversity (Deb et al. 2002). By considering multiple performance metrics, the designed algorithm would be more generalized to the target problem-solving scenario than only pursuing a single metric.
Usability of the metrics: The solution quality has been employed in most literature. The reason is possibe because presetting the computational budget is easy when targeting the widely-used numerical benchmark problems (where much prior experimentation can be referenced for presetting the budget). The running time and AUC provide a performance guarantee that the designed algorithm could reach the predefined performance threshold within the time budget. The guarantee is significant to practical scenarios. Therefore, the running time and AUC are recommended if the threshold can be defined according to domain expertise or preliminary experimentation. Function evaluations should be the first. choice for quantifying the time budget if function evaluations dominate the computational cost. The reason is that it is fair and can provide reproducible results regardless of hardware, programming language, and coding style (Hooker) 1995). The anytime time performance is more informative than running time since it comprehensively reveals the performance across the algorithm execution (Hansen et al. 2022). It is preferred in scenarios with demands for anytime behavior, e.g., the designed algorithm deployed in scenarios with variable computational budgets.
5.2 Performance Evaluation and Comparison
Due to the stochastic nature, different runs (with different seeds) of metaheuristic algorithm on the same target problem instance result in inconsistent performance. During the algorithm design process, simply evaluating and comparing candidate algorithms by the performance of a single run is not meaningful. Current strategies of performance evaluation and comparison during the algorithm design process are as follows.
Conducting descriptive statistics on the performance: It is straightforward to use descriptive statis- tics, o.g., mcan or median, to evaluate and compare candidate algorithms (Ye et al. 2022a Junior et al., 2020 Vermetten et al. 2022), but they contain issues. When targeting multiple problem instances, the mean requires the performance values different instances within the same scale, which is hard to achieve. The
14

---

Published in Transactions on Machine Learning Research (02/2024)
median is robust to outliers but will ignore the majority of the data, e.g., an algorithm may get unqualified performance in 49% of the evaluations, but the median would not capture this.
Conducting statistical hypothesis test on the performance: Hypothesis test provides more statistical evidence of to what extent an algorithm performs significantly better than others. The Friedman test and 1-test have been employed in the literature (T.pee--bañezz et al.) 2016). The t-test assumes samples (i.e., performance metric values) following a normal distribution, while the Friedman test is nonparametric and may be more preferred. In particular, the p-value correction in multiple comparisons (i.e., simultancously comparing more than two algorithms) was suggested to be abandoned in López -iriz--bbizzz et al. (2016), consid- cring that valuu correction makes the test consorvative, subsequently lowering the discrimination among algorithms.
5.3 Reducing Time Cost of Performance Evaluation
The repetitive performance evaluation leads the algorithm design process to be computationally expensive. A number of strategies have been introduced to reduce the time cost of performance evaluation. They reduce the time cost by either reducing the number of performance evaluations or estimating the performance without a full evaluation.
5.3.1 Reducing the Number of Performance Evaluations
This strategy refers to evaluating on few target problem instances instead of on all instances. Intensification (Hutter et al. 2009 Blot ct al.) 2016) is a method following this strategy. It evaluates the candidate algorithms" performance instance by instance. The evaluation terminates, and the candidate algorithm is discarded if the algorithm's performance is worse than the incumbent on the current instance. Otherwise, the evaluation continues till it has been evaluated in all instances.
1997 is also a method that evaluates the performance instance by instance [óppez-bbãeez et al. 2016). The difference of racing from intensification (Hutter et al.) 2009 Blot et al, 2016) is that there is no incumbent, but all candidate algorithms are evaluated together. In detail, the candidate algorithms' performance is evaluated in the same instances; the candidates whose performance is statistically worse than least another candidate are discarded. The evaluation continues with the survival candidates on the remaining instances until the number of survivors reaches the minimum, the computational budget is exhausted, or all instances have been evaluated.
Racing (Maron and Moore
Another strategy for saving performance evaluations is only evaluating desired algorithms instead of all can- didate algorithms. The dual population of gender-based GA (Ansótegui et al. 2009 follows this strategy. It maintains multiple algorithms during the design process. The algorithins are separated into a competitive and non-competitive population. The competitive population is evaluated on the target problem instances and works on guiding the design towards finding promising algorithms. The non-compctitive population works on introducing diversity without evaluation. The two populations interact (hrough genetic recombination.
5.3.2 Estimating the Performance without Full Evaluation
One such strategy is using a low-complexity surrogate model to estimate the performance instead of exact ovaluation (run the algorithm on the target problem). The random forest acted as surrogate in Bartz- Beielstein al. (2005); |Httter et al. (2011); Wang al. (2017): Lindauer et al. (2022). Its input is vectorized representations of the candidate algorithms; the output is the estimated performance values. Thus, it is an end-to-end performance estimation method and is convenient for use.
Another strategy for estimating performance is early stopping the performance evaluation before the compu- tational budget is exhausted. Capping (Hutter et al. 2009 De Souza et al. 2022) is a mothod following this strategy. There are two capping versions the literature. The first version was proposed Hutter et al.) (2009h for scenarios with running time as the performance metric. It uses the best algorithm's running
*Incumbent the best algorithm found far in the design process. Its performance in all instances has been evaluated. The best algorithm can be either the best one found at the current iteration of the design process (trajectory-preservins capping) or the hest. one found so far the design process (aggressive capping) (Hutter et al. 2009).
15

---

Published in Transactions on Machine Learning Research (02/2024)
Table 2: Summary of strategies for reducing time cost of algorithm evaluation.
Methoda
Main ideas
Inteusification
Do not evaluste on the next problem instance performance worse than the incumbent.
Reducing the number of coaluations
Racing
Do not evaluate the next problem instance if performance statistically worse than at least another algorithm.
Dual population
Only evaluate desired algorithms.
Surrogate
Use low complexity surrogate approximate performance.
Estimating without a full evaluation
Hutter al
2009
Capping
Stop evaluating the current problem instance rumning time meets budget.
De Souza et al
2022 s Capping
Stop evaluating on the current problem instance once performance proflle exceeds bound.
time bound. An algorithm's performance evaluation stops once its rumning time reaches the bound. Reaching the bound indicates that the algorithm fails to perform better than the best algorithm until the time bound, which means that the algorithm can be discarded.
The second version of capping was proposed De Souza et al 2022 and for scenarios with anytime performance metrics. It also uses a bound to limit the performance evaluation. But the bound is a step function curve that describes previously evaluated algorithms' aggregated performance profile across multiple running time points. For new algorithm, its performance evaluation stops once its performance profile exceeds the bound at some running time points.
5.3.3 Usability
Intensification (Hutter et al. 2009 Blot al. 2016) and racing (López-Tháñez et al. 2016) evaluate per- formance instance by instance. They available if there
relatively large number of instances to discriminate algorithms. While intensification and racing can be compatible with any performance metrios, capping (Hutter et al.) 2009 De Souza et al. 2022) is specified to scenarios with runming time and anytime performance metrics.
Using a surrogate to partially replace performance evaluations is promising due to its end-to-end nature. However, common surrogates (e.g., random forest) require a fixed-length vectorized representation of the algorithm as input. Subsequently, predefined algorithm template is needed to avoid unnccessarily lengthy vectors. The template limits the designed algorithms to have the same structure. In this regard, it is worth developing algorithm embedding methods to transform algorithms with various structures into compact representations with the same form. Furthermore, currently employed surrogates are regression models estimating algorithms' performance values, which are not always reasonable. For example, suppose there are two algorithms (A1, A2] with ground truth performance values (0.9, 0.91), where 0.9 and 0.91 stand for A and Az's performance, respectively; and there are two estimations {0.89, 0.92) and (0.91,0.9), {0.91, 0.9] has smaller mean square error loss than (0.89, 0.92] but results in a wrong performance rank. Considering such cases, surrogates that can directly estimate algorithms' performance rank advised. A summary of the strategies is given in Table
6 Target Problems
This section reviews the problems that have been targeted in the literature. Related software is also reported.
6.1
Numerical Benchmark Problems
Many works, especially the carly ones, employ numerical benchmarks as target problems. Most of them con- sidered continuous optinization problems. The CEC 2005 real-valued parameter optimization benchmarks
(Suganthan et al.) 2005) were widely used (Poli et al.
2005a b Shirakawa and Nagao 2009 Miranda and
Prudéncio 2015; Lones
2019
Bogdanova al.] 2019
Kamrath et al.) 2020 Miranda and Prudéncio 2020;
Cruz-Duarte et 2020
Lones 2021 Tian al. 2023
The CEC 2014 sal-valued parameter optimization
benchmarks [iang et al.
2013
the black-box optimization benchmark suite (Hansen et al. 2009 and the
16

---

Published in Transactions on Machine Learning Research (02/2024)
comparing continuous optimizers platform used for the GECCO
Workshops Real-Parameter Black-Box [2017): Villalón et al. (2022); van Rijn et al. (0166)
Optimization Benchmarking were adopted in Aydm et al.
Tisdale et al. (2021), and (Ansótegui et al. 2015 Richter et al.
2019), respectively.
The DTLZ (Deb et al. 2005 and WFG (Huband et al., 2006 test suites acted as target problems in the
automated design of multi-objective metaheuristic algorithms
(Bezerra et al., 2016; 2020; de Lima and Pozo, Binary optimization benchmarks were also considered. The NKK-Ladssaape benchmarks (Kauffman
2017).
et al. 1993) were utilized in Goldman and Tauritz (2011); Richter and Tauritz (2018) because the fitness landscape properties can be easily controlled by the N and K parameters. The D-Trap suite (Deb and
Goldberg, 1993) was employed in Smorodkina and Tauritz 20077; Goldman and Tauritz (01 1: Richter and
Tauritz 20188) Rivers and Tauritz (2013).
A common experimental observation of these works is that automated design techniques are able to recur existing algorithms and produce new algorithms that perform better than the existing oncs. Due to the expressiveness of various problem landscape features, numerical benchmarks are also important for verifying new automated design techniques and empirically analyzing the techniques, e.g., analyzing the efficiency of different performance metrics YEe et al. 2022a). ). A common concern on the numerical benchmarks is that it needs to be clarified to what extent these artificial problems reffect the practice. In this regard, benchmark problems derived from the real world could be considered to stimulate the practicality of automated design techniques.
6.2 Practical Problems
Automated design of metaheuristic algorithms has been applied to some practical problem domains, although most of the problems come from experimental simulations. Job shop scheduling (JSS) is representative practical problem considered in the literature. In Mascia et al. (2013) 2014); Vázquez- Väzquez-Rodriguez and Ochoa
(2011; b; Pagnozzi and Stützle (2019); Bezerra et al. (2014); ; Franzin and Stützle (2019); Allaro-Fernández ct al. p020 Sac-Dan et al. (2020) : Brum et al. (0222). metaheuristic solvers were automatically designed for JSS with different objectives, e.g., makespan, flowtime, and total tardiness. The bi-objective JSS that si-
multancously minimizes makespan and flowtime, the dynamic bi-objective JSS, and the resource-constrained
JSS were further considered in Blot et al. (2017 2019); Nguyen al. 20014 2022 respectively.
In Sae-Dan et al. 20200;; Tavares and Pereira 2012 Ryser- Welch et al. (2016) Lopez-lbanez and Stutzle
(2012], the automated design was conducted within the ACO template for the TSP. The vehicle routing and nurse rostering problems were adopted as case studies in Qu et al. (2020); Meng and Qu (202I). The propo- sitional satisfiability (SAT) problem with instances from various distributions was targeted in Smorodkina and Taurity (2007); KhushaBukhhh et al.] (2009). The quadratic assignment, bin packing, and imbalanced data classification were considered in Fraazin and Stützle (2019); [Sae-Dan et al. (2020); Mascia al. (2014), and (Taii et al.) 2020b/a). respectively. The automated design techniques were applied to problems with real-world data in Zhao et al. (2022), in which solvers were automatically designed for the raw material stack- ing and rack placement problems from the warehouse management department of a. biomedical electronics company and were reported to be satisfactory.
The aforementioned works show great potential for automated metaheuristic algorithm design for practical problems. For example, for the JSS problem, new stochastic local search algorithms were designed by the EMILI framework and demonstrated to be superior to the state-of-the-art iterative greedy and local search algorithms when pursuing the makespan, flowtime, and total tardiness objectives (Pagnozzi and Stützle, 2019 Alfaro-Fernándes et al. 2020). Multipass heuristics were evolved by GP and outperformed the popu- lar houristics FIFD, WSPT, WEDD, etc. (Thiruvady et al. 2013 Pinedo 2012) in resource constraint job scheduling (Nguyen el al. 2022). For the TSP, new metaheuristic solvers were discovered by Cartesian GP from small-scale TSP instances and reported to generalize well to much larger instances (Ryser- Welch et al., 2016). For the SAT problem, stochastic local search algorithms were derived from SATenstein and signifi- cantly outperformed previous state-of the-rrartttttttto algorithms on various benchmark distributions (uhhhBukhhh
et al. 2009
17

---

Published in Transactions on Machine Learning Research (02/2024)
6.3 Software
Generally, choose design space, design strategy, and performance evaluation strategy from these reviewed in Sections and [ respectively, to build concrete pipeline for automated design, following the workflow in Fig. The established pipelines with open-source code that have been or can be introduced in designing metaheuristic algorithms are reported below.
irace: irace (López-lbáñez et al. 2016 is an iterative version the racing method (Maron and Moore 1997). irace for metaheuristic algorithm design uses (i) the design space with existing algorithmic operators and the linear array representation, (ii) "estimation of distributionn
the design strategy, and (iii) racing
as the performance evaluation strategy. irace has been applied to design metaheuristic algorithms with given templates (e.g., with templates of ACO [opeez-banee and Stutzle 2012), artificial bee colony (Aydm
et al. 2017), SA Franzin and Stützle 2019). PSO Villllónn et al. 2022 and design general metaheuristics
Bezerra et al.
2014
2016 Pagnozzi and Stützle 2019: Alfaro-Fernándee et al. 2020: Bezerra et al. 2020:
Sae- Dan et al. 2020
Diaz and López-lbáñez 2021; Brum et al. 2022). The R implementation irace is
available on its website
C++ interface of irace given in the Paradiseo software (Dreo et al. 2021
Aziz-Alaout al.
2021 and available on Githuty]
ParamILS: ParamILS (Hutter et al. 2009), as its name indicated, uses the iterative local search (Lourengo et al. 2003) for parameler configuration. It was employed in algorithm design by representing algorith-
mic operators as categorical parameters (Blot et al. 2019 Tarri et al., 2020a] KhudaBuuhhsh et al. 2009).
Apart from using existing algorithmic operators as design space and iterative local search as design strat- egy, ParamILS proposes capping as the performance evaluation stralegy. A multi-objective ParamILS was developed in Blot et al. (2016) jointly consider multiple performance metrics. The Ruby implementation of ParamILS is available on its website
2022) was originated to hyperparameter optimization in automated machine learning and was then employed to configure metaheuristic algorithms (Tang et al. 2021 Its difference from ParamILS is that it uses Bayesian optimization as the design strategy. The intensification and capping can be the performance evaluation strategy alternatively. The Python implementation of SMAC is available on Githul
SMAC: SMAC (Hutler et al. 2011 Lindauer et al.
Sparkle: Sparkle is recently released Python platform for selecting and configuring algorithms (van der Blom et al. 2022). Its current version uses SMAC as the configurator and could be extended to algorithm design. The platform provides easy use for non-experts and supports benchmarking and ablation analysis. The source code is available on its "
7 Research Trends
The development of automated metaheuristic algorithm design continues along a number research threads. Some of the rescarch trends are discussed in this section.
7.1 Design Space
Constructing design space with less user effort: The design space is critical since it determines what algorithms can be found in principle. Till now, the design space still requires users to manually construct. Guidelines or strategics for constructing the design space are important. One way to provide such guidelines or strategies may be benchmarking, i.e., collecting general design choices for different types of problems, i.e., continuous, discrete, and permutation, and offering maintainable and extendable platforms for users to use and analyze the design choices on particular problems. Another way automatically constructing the design space. This way is emerging in the neural architecture search field (Radosavovic et al. 2020 in which the
"https://iridiaaullba bie/rrr
/ paradisco
Phttps///wwwccs.u wbw.c...bcc..ooo.....oc wawwaasaaaaaaeeeeeeeeeeeeeeeeeee
9https://gtthuu.com/ lataaaam/mmaaaa...
10 1https: httpt:////bbbbuckt.rrrggg http:://ww.....aaaaaa///////////
18

---

Published in Transactions on Machine Learning Research (02/2024)
design space was evolved according to the performance of neural networks sampled from it (Ci et al. 2021 or co-evolved along with the architecture search process (Chen et al. 2021 These works pave way for design space construction.
Novel representations: The GP parse tree and linear array are dominant algorithm representations for design space with computational primitives and design space with algorithmic operators, respectively. For the GP tree, it has been limited to represent a single algorithmic operator. The idea of indirect encoding (Eiben and Smith 2015; Stanley et al., 2019) may break the limitation. Its generative and developmental nature allows learning and reusing building blocks of the tree, which could improve the search efficiency and scale up the complexity of the algorithm being represented (Eiben and Smith, 2015). For the linear array, it requires a given algorithm template, limiting the novelty in algorithm structures. The graph representation (Tisdale et al. 2021) would be promising because of its expressiveness in representing algorithm flows. An open challenge is that the graph is difficult to be directly manipulated. More studies are desired on the encoding and docoding of graph representation to make it casily manipulated and compatible with different design strategies and performance evaluation strategies.
7.2 Design Strategies
Designing diversified algorithms in parallel and distributed manners: Designing multiple algo- rithms in parallel facilitates exploration over the design space (Miikulainen and Forrest 2021 Ye et al,, 2022b). This ariscs an issue of how to mcasure the diversity (similarity) of the algorithms. Idcally, the similarity between algorithms is determined by not only the distance between their representations but also their similarity in terms of algorithmic structures and exhibited behaviors (Xu et al. 2016 Nikfarjam et al., 2022 Xiang et al. 2022). Quantifying algorithms' similarities and subsequently identifying unexplored design space deserves more attention. Moreover, the advances of distributed computing platforms (e.g., Hennessy and Patterson 2011 2019)) provide opportunities for scaling automated design in hosting more challenging and expensive problems, which also deserves attention.
Learning-based design strategies: Learning-based design strategies, e.g., reinforcement learning, offer appealing features, including fine-grained design, long-term planning of the design with the value function, and knowledge reuse with the policy model. These features promise reinforcement learning to generate algorithms to fit-for-purpose in online and open-ended problem-solving scemarios with changing problem characteristics, user requirements, and operating environments (Hart) 2017 Weyns et al., 2021). ) It is de- sirable to have more studies on developing advancod policics and value functions for learning basod design strategies.
7.3 Performance Evaluation Strategies
Novel performance comparison methods: While null hypothesis statistical Lests have been the common way of performance comparison during the algorithm design process, recent alternatives have shown great potential in complex scenarios. For example, the Bayesian analysis of (Rojjs-Delgadd et al. (2022) provided an inference of algorithm ranking and quantification of the uncertainty involved in the ranking. Quantifying the uncertainty is important, considering the stochastic nature of metaheuristic algorithms. The study in Yan et al. 2023) proposed cumulative distribution functions with an algorithm-indepenndntt binary filter condition to address the "cycle ranking" paradox, i.e., inconsistent ranking of algorithms in multiple comparisons. These techniques are worth integrating within the algorithm design process for more reliable performance comparison.
Surrogate-bssed performance estimators: The automated design would be time costly when targeting expensive problems. The cost be saved by evaluating on part of instances early stopping the evaluation. The former requires adequate instances to discriminate the performance of different algorithms, while the latter is available when using running time or anytime performance metrics. An alternative is to build a low-complexity surrogate model to estimate the performance. This surrogate estimator is end-ta endd which can be intograted with any performance metrics and is independent of the characteristics of the target problem. There are some challenges to building an accurate surrogate. For example, how to embed the algorithms with various structures to the input of the surrogate, how to estimate algorithms' performance
19

---

Published in Transactions on Machine Learning Research (02/2024)
rank rather than performance metric values, and how different surrogate models are workable for different design scenarios (regarding sample richness, computing resources, etc.).
7.4 Experimental and Theoretical Analysis, Practical Applications
Although many works have reported remarkable efficiency of automated design, it is unclear what design space, design strategy, and performance evaluation strategy contribute to that efficiency. Further experi- mental and theoretical analysis is required to reveal the impact of different kinds of design space and design strategies on different problems. Besides, a majority of existing works apply automated metaheuristic al- gorithm design to numerical benchmarks and classical simulated problems, c.g., JSS, TSP, and SAT. It is important to go beyond these numerical and simulated problems and contribute more to real-world problem- solving, e.g., incorporating problem-speciee
elements with the design process, enabling the designed algo- rithms to handle constraints and to decouple large-scale problem space. Benchmarking, platforms, and user interfaces are also appreciated to promote practical applications.
7.5 Intersection with Related Research Fields
Automated design of motaheuristic algorithms is closely related automated machine learning and meta- learning (Hutter et al. 2019 Advances from the related fields would fertilize the development, of the automated design of metaheuristic algorithms, such multi-fidelity performance evaluation to speed up the design process, model learning to generate surrogates from previous experience and learn policies for reinforce- ment learning-based design strategies. On the other hand, the automated design of metaheuristic algorithms presents key techniques for automated machine learning and meta-learning. For example, metahcuristic algorithms could be high-level methods for conducting automated machine learning and meta-learning; the GP tree-based representation with computational primitives paves a potential way for automatically pro- gramming machine learning algorithms. The interaction and integration of these fields may be promising thread for boosting innovations.
8 Conclusion
This paper surveyed the automated design of metaheuristic algorithms. In the survey, we have abstracted the concept and proposed taxonomy of automated design of metaheuristic algorithms by formalizing the design process into four parts, i.e., design space, design strategies, performance evaluation strategies, and target problems. We have then reviewed the advances regarding the four parts, respectively. The strengths, weakness, challenges, and usability of these works have been discussed. Finally, some of the research trends have been pointed out, Hopefully, this survey can provide a comprehensive and meticulous understanding of the automated design of metaheuristic algorithms, inspire interest leveraging automated design techniques to make perforrefreeece algorithms accessible to a broader range of researchers and practitioners, and boost automated design innovations to fuel the pursuit of autonomous and general artificial intelligence.
References
2023. Code models overview. code-models-overvie.
https://cloud. gowgleooooo...ooo.ooooo.. httpss///boodggggggeeeee//eerrrs htt.g//...a///ooaaaar///////aaaa
Steven Adriaensen and Ann Nowé. 2016. Towards a white box approach to automated algorithm design. In International Joint Conferences on Arlificial Intelligence. IJCAT, New York, NY, USA, 554-560
Alexandros Agapitos, Roisin Loughran, Miguel Nicolau, Simon Lucas, Michael O'Neill, and Anthony Brabazon. 2019. A survey of statistical machine learning elements in genetic programming, IEEE Trans- actions on Euolutionary Computation 23, 6 (Dec. 2019), 1029-1048.
Pedro Fernnoz-e Rubén Ruiz, Federico Pagnozzi, and Thomas Stützle. 2020. Automatic algorithm design for hybrid flowshop scheduling problems. Europeon Journal of Operational Research 282, (May 2020), 835 845.
20

---

Published in Transactions on Machine Learning Research (02/2024)
Carlos Ansótegui, Yuri Malitsky, Horst Samulowitz, Mcinolf Sellmann, and Kevin Tierney. 2015. Model-based genetic algorithms for algorithı configuration. In International Joinl Conference on Artificial Intelligence. IJCAI, Buenos Aires, Argentina, 733 739.
Carlos Ansótegui, Meinolf Sellmann, and Kevin Tierney. 2009. A gender-based genetic algorithm for the au- tomatic configuration algorithms. In International Conference on Principles and Practice of Constroint Programming. Springer, Lisbon, Portugal, 142-157.
Dogan Aydm,, Gürcan Yavuz, and Thomas Stützle. 2017. ABC-X: A generalized, automatically configurable artificial bee colony framework. Swarm Intelligence 11, (Feb. 2017), 1-38.
Amine Aziz-Alaoui, Carola Doerr, and Johann Dreo. 2021. Towards large scale automated algorithm de- sign by integrating modular benchmarking frameworks. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Lille, France, 1374-1374.
Thomas Bartz-Beiclstein, Christian WG Lasarczyk, and Mike Preuß. 2005. Sequential parameter optimiza- tion. In JEEE Congress on Evolulionary Compulation, Vol. 1. IEEE, Edinburgh, UK, 773-780
Leonardo CT Bezerra, Manuel Tópez-Thánez, and Thomas Stützle. 2014. Automatic design of evolutionary algorithms for multi-objective combinatorial optimization. In International Conference on Parallel Problem Solving from Nature. Springer, Ljubljana, Slovenia, 508 517.
Leonardo CT Bezerra, Manuel López-Tbánez, and Thomas Stützle. 2016. Automatic component-wise design multiobjective evolutionary algorithms. IEEE Transactions on Evolutionary Computation 20, (Jun. 2016), 403-417.
Leonardo CT Bezerra, Manuel López-Ibáñez, and Thomas Stützle. 2020. Automatically designing state-of- the-art multi-and many-objective evolutionary algorithms. Evolutionary Computation 28, 2 (Jun. 2020), 195-226.
Aymeric Blot, Holger H Hoos, Lactitia Jourdan, Maric- Éléén-r Kessaci-Marmion, and Heike Trautmann. 2016. MO-ParamILS: A multi-objective automatic algorithm configuration framework. In Internalional Conference on Learning and Intelligent Optimization. Springer, Ischia, Italy, 32-47.
Aymeric Blot, Elénoore Kessaci, Lactitia Jourdan, and Holger H Hoos. 2019. Automatic configuration of multi-objective local search algorithms for permutation problems. Evolutionary Compulation 27, 1 (Mar. 2019), 147-171.
Aymeric Blot, Alexis Pernet, Laetitia Jourdan, Marie-Éléonore Kessaci- Marrmac, and Holger H Hoos. 2017, Automatically configuring multi-objective local search using multi-objective optimisation. In International Conference on Evolutionary Multi- Criti--Citteoon Optimization. Springer, Münster, Germany, 61-76.
Anna Bogdanova, Jair Pereira Junior, and Claus Aranha.
2019. Franken-swarm: Grammatical evolution for the automatic generation of swarm-like meta-heuristies. In Proceedings of the Genatic and Evolutionary Computation Conference Comparion. Prague, Czech Republic, 411 412.
Artur Brum, Rubén Ruiz, and Marcus Ritt. 2022. Automatic generation of iterated greedy algorithuns for the permutattntoo flow shop scheduling problem with total completion time minimization. Computers & Industrial Engineering 163 (Jan. 2022), 107843.
Edmund K Burke, Michel Gendreau, Matthew Hyde, Graham Kendall, Gabriela Ochoa, Ender Özean, and Rong Qu. 2013. Sypper-herristics A survey of the state of the art Journal of the Operational Research Society 64, 12 (July 2013), 1695-1724.
Edmund K Burke, Matthew Hyde, Graham Kendall, and John Woodward. 2010. A genetic programming hyper-heuristcc approach for evolving 2-D strip packing heuristics. IEEE Transactions on Evolutionary Compulation 14, (Dec. 2010), 942 958.
21

---

Published in Transactions on Machine Learning Research (02/2024)
Edmund K Burke, Matthew R Hyde, Graham Kendall, Gabriela Ochoa, Ender Özean, and John R Wood- ward. 2019. A classification of hyper-heuristic approaches: Revisited. In Handbook of Melaheurislics. Springer, 453 477.
Arina Buzdalova, Vladislav Kononov, and Maxim Buzdalov. 2014. Selecting evolutionary operators using reinforcement learning: Initial explorations. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Vancouver, Canada, 1033 1036.
Wang Chao, Jiaxuan Zhao, Licheng Jiao, Lingling Li, Fang Liu, and Shuyuan Yang. 2024. A match made in consistency heaven: when large language models meet evolutionary algorithms. arxie preprinl arXiv:2401.10510 (2024).
Minghao Chen, Kan Wu, Bolin Ni, Houwen Peng, Bei Liu, Jianlong Fu, Hongyang Chao, and Haibin Ling. 2021. Searching the search space of vision transformer. In Advances in Neural Information Processing Systems, Vol. 34. Virtual, 8714 8726.
Yuanzheng Ci, Chen Lin, Ming Sun, Boyu Chen, Hongwen Zhang, and Wanli Ouyang. 2021. Evolving search space for neural architecture scarch. In Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal, Canada, 6659 6669.
Jorge M Cruz-Duarte, Ivan Amaya, José Carlos Ortiz-Bayliss, Santiago Enrique Conant-Pablos, and Hugo Terashima-Marín. 2020. A primary study hyper-heuristies to customise metaheuristics for continuous optimisation. In IEEE Congress on Evolutionary Computation. IEEE, Glasgow, UK.
Ricardo Henrique Remes de Lima and Aurora Trinidad Ramirez Pozo. 2017. A study auto-configurationn of multi-objective particle swarm optimization algorithm. In IEEE Congress on Evolutionary Computation. IEEE, Donostia, Spain, 718 725.
Marcelo De Souza, Marcus Ritt, and Manuel López-lbáñez. 2022. Capping methods for the automatic
configuration of optimization algorithms. Computers & Operations Research 139 (Mar. 2022), 105615.
Kalyanmoy Deb and David E Goldberg. 1993. Analyzing deception in trap functions. In Foundations of Genetic Algorithms. Vol. 2. Elsevier, 93-108.
Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiob- jective genetic algorithm: NSGA-IL. IEEE Transactions on Evolutionary Compulation 6, (Apr. 2002),
187.
Kalyanmoy Deb, Lothar Thiele, Marco Laumanns, and Eckart Zitzler. 2005. Scalable test problems for evolutionary multiobjective optimization. In Evolutionary Multiohjective Optimization. Springer, New York, NY, USA, 105 145.
Juan Esteban Diaz and Manuel López-lbáñez. 2021. Incorporating decision-naker's preferences into the automatic configuration of bi-objective optimisation algorithms. European, Journal of Operational Research 289, 3 (Mar. 2021), 1202-1222.
Laura Diogan and Mihai Oltean. 2009. Evolutionary design of evolutionary algorithms. Genetic Programming and Evolvable Machines 10, 3 (Mar. 2009), 263-30 303-306.
Marco Dorigo, Vittorio Maniezzo, and Alberto Colorni. 1996. Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 26, 1 (Feb. 1996), 29-41.
Johann Dreo, Arnaud Liefooghe, Sébastien Verel, Mare Schoenaner, Juan Merelo, Alexandre Quemy, Ben- jamin Bouvier, and Jan Gmyss 2021. Paradiseo: from a modular framework for evolutionary computation the automated design of metaheuristics: 22 years of Paradiseo. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Lille, France, 1522-1530.
Ágoston E Eiben, Robert Hinterding, and Zbigniew Michalewicz. 1999. Parameter control in evolutionary algorithms. IEFF Transactions on Evolutionary Computation 2 (July 1999), 124-141.
22

---

Published in Transactions on Machine Learning Research (02/2024)
Agoston E Eiben and Jim Smith. 2015. From evolutionary computation to the evolution of things. Nature 521, 7553 (May 2015), 476-482.
Agoston E Eiben, James E Smith, al. 2003. Introduction to evolulionary compuling. Vol. 53. Springer, New York, NY, USA.
Thomas Feo and Mauricio GC Resende. 1995. Greedy randomized adaptive search procedures. Journal of Global Optimization 6, (Mar. 1995), 109-133.
Alberto Franzin and Thomas Stützle. 2019. Revisiting simulated annealing: A component-based analysis. Computers & Operations Reseorch 104 (Apr. 2019), 191 206.
Fred Glover. 1986. Future paths for integer programming and links to artificial intelligence. Compulers 8 Operations Research 13, (1986), 533-549.
Fred Glover. 1989. Tabu search part I. ORSA Journal on Computing 1, 3 (Aug. 1989), 190-206.
Fred W Glover and Gary A Kochenberger, 2006. Handbook of metaheuristics. Vol. 57. Springer Science & Business Media.
Brian W Goldman and Daniel R Tauritz. 2011. Self-configuring crossover. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Dublin, Ireland, 575-582.
Nikolaus Hansen, Anne Auger, Dimo Brockhoff, and Tea Tušar. 2022. Anytime performance assessment in blackbox optimization benchmarking. IEFE Transactions on Evolutionary Compulation 26, 6 (Dec. 2022), 1293 1305.
Nikolaus Hansen, Steffen Finck, Raymond Ros, and Anne Auger. 2009. Real- ceal-parameter black-bor optimization benchmarking 2009: Noiseless function definitions. Technical Report. INRIA.
Emma Hart. 2017. Towards lifelong learning in optimisation algorithms. In Proceedings of the International Joint Conference on Computational Intelligence. Madeira, Portugal, 7 9.
Trevor Hastie, Robert Tibshirani, Jerome Friedman, and Jerome Н Friedman. 2009. The elements of statistical learning: data mining, inference, and prediction. Vol. 2. Springer.
John Hennessy and David A Patterson. 2011. Computer architecture: a quantitative approach. Elsevier.
John T. Hennessy and David A Patterson. 2019. A new golden age for computer architecture. Commun. ACM 62, 2 (Jan. 2019), 48 60.
John H Holland. 1962. Outline for a logical theory of adaptive systems. J. ACM 9, 3 (1962). 297-314.
John H Holland. 1973. Genetic algorithus and the optimal allocation of trials. SIAM J. Comput. 2, 2 (Jun. 1973), 88-105.
John N Hooker. 1995. Testing heuristics: We have it all wrong. Journal of Hearistics 1, (Sept. 1995).
33 42.
C. Huang, Y. Li, and X. Yao. 2020. A survey of automatic parameter tuning methods for metaheuristics. IEEE Transactions on Evolutionary Computation 24, 2 (Apr. 2020), 201-216.
Simon Huband, Philip Hingston, Luigi Barone, and Lyndon While. 2006. A review of multiobjective test problems and scalable test problem toolkit. IEEE Transactions on Evolutionary Computation 10, 5 (Oct. 2006), 477-506.
Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. 2011. Soquential modcl-bascd optimization for general algorithm configuration. In International Conference on Learning and Intelligent Oplimization. Springer, Rome, Italy, 507-523.
Frank Hutter, Holger H Hoos, Kevin Leyton-Brown and Thomas Stützle. 2009. ParamILS: An automatic algorithm configuration framework. Journal of Artificial Intelligence Research 36 (Oct. 2009), 267-306.
23

---

Published in Transactions on Machine Learning Research (02/2024)
Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. 2019. Automated machine learning: Methods, systems, challenges. Springer Nature, New York, NY, USA.
Donald R. Jones, Matthias Schonlau, and William J Welch. 1998. Efficient global optimization of expensive black-box functions. Journal of Global Optimization 13, 4 (Dec. 1998). 455-492.
Jair Pereira Junior, Claus Aranha, and Tetsuya Sakurai. 2020. A training difficulty schedule for effective search of neuriscuiistic design. In IEEE Congress on Evolutionary Computation. IEEE, Glasgow, UK, 1-8.
Nathaniel R Kamrath, Aaron Scott Pope, and Daniel R Tauritz. 2020. The automated design of local optimizers for memetic algorithms employing supportive cocvolution. In Proceedings of the Genetic and Evolutionary Compulation Conference Companion. Cancún, Mexico, 1889 1897.
Wolfgang Kantschik, Peter Dittrich, Markus Brameier, and Wolfgang Banzhaf. 1999. Meta-evolution in graph GP. In European Conference on Genetic Progromming. Springer, Göteborg, Sweden, 15-28.
Stuart A Kauffman et al. 1993. The origins of order: Self-organization and selection in evolution. Oxford University Press, Oxford, UK.
James Kennedy and Russell Eberhart. 1995. Particle swarm optimization. In Proceedings of ICNN'95- International Conference on Neural Networks. TERE, Perth, WA, Australia, 1942-1948.
P. Kerschke, H. H. Hoos, F. Neumann, and H. Trautmann. 2019. Automated algorithm selection: Survey and perspectives. Evolutionary Computation 21, (Mar. 2019), 1-47.
Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup. 2022. Towards continual reinforcement learning: A review and perspectives. Journal of Artificial Intelligence Research 75 (Dec. 2022), 1401-1476.
Ashiqur R KhudaBukhsh, Lin Xu, Holger H Hoos, and Kevin Leyton-Brown. 2009. SATenstein: Automati- cally building local search SAT solvers from components. In International Joint Conference on Artificial Intelligence. Pasadena, CA, USA, 517-524.
Scott Kirkpatrick, C Daniel Gelatt Jr, and Mario P Vecchi. 1983. Optimization by simulated annealing. Science 220, 4598 (May 1983), 671-680.
John R Koza. 1994. Genetic programming as a means for programming computers by natural selection. Statistics and Computing 4, 2 (Jun. 1994), 87 112.
Robert Tjarko Lange, Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dalibard, Chris Lu, Satinder Singh, and Sebastian Flemnerhag. 2023. Discovering evolution strategies via meta-black-box optimization. In International Conference on Learning Representations. Kigali, Rwanda.
Sergey Levine and Pieter Abbeel. 2014. Learning neural network policies with guided policy search under unknown dynamics. In Advances in Neural Information Processing Systems, Vol. 27. Montreal, Canada.
Jing Liang, Bo Y Qu, and Ponnuthurai N Saganthan. 2013. Problem definitions and evaluation criteria for the CEC 2014 special session and competition on single objective real-parameter numerical optimization. Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou China and Technical Report, Nanyang Technological University, Singapore 635 (2013), 490.
Marius Lindauer, Katharina Eggensperger, Matthias Feurer, André Biedenkapp. Difan Deng, Carolin Ben- jamins, Tim Ruhkopf, René Sass, and Frank Hutter. 2022. SMAC3: A versatile Bayesian optimization package for hyperparameter optimization. Journal of Machine Learning Research 23, 54 (Feb. 2022), 1-9.
Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. 2024. An example of evolutionary computation+ large language model beating human: Design of efficient guided local search. arXiv preprint ar Xix:22401.00051 (2024).
24

---

Published in Transactions on Machine Learning Research (02/2024)
Michael A Lones. 2019. Instruction-level design of local optimisers using push GP. In Proceedings of the Genelic and Fvolutionary Compulation Conference Companion. Prague, Czech Republic, 1487-1494.
Michael A Lones. 2021. Evolving continuous optimisers from scratch. Genetic Progremming and Evolvable Machines 22, (Oct. 2021), 395-428.
Manuel López-lbáñez Jérémie Sabois-Lacoste Leslie Pérez Cáceres, Mauro Birattari, and Thomas Stützle. 2016. The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives 3 (Sept. 2016), 43-58.
Manuel Lopez-lbanez and Thomas Stutzle. 2012. The automatic design of multiobjective ant colony opti- mization algorithms. IEEE Transactions on Evolutionary Computation 16, 6 (Dec. 2012), 861-875.
Manuel López-lbánez and Thomas Stützle. 2014. Automatically improving the anytime behaviour of opti- misation algorithms. European Journal of Operational Research 235, (June 2014), 562-
Helena R Lourengo, Olivier C Martin, and Thomas Stützle. 2003. Tterated local search. In Handbook of Metaheuristics. Springer, New York, NY, USA, 320 353.
Oden Maron and Andrew W Moorc. 1997. The racing algorithm: Model selection for lazy learners. Artificial Intelligence Review 11, (Feb. 1997), 193 225.
Franco Mascia, Manuel López-lbánez, Jérémie Dubois-Lacoste and Thomas Stützzle 2013. From grammars to parameters: Automatic iterated greedy design for the permutation flow-shop problem with weighted tardiness. In International Conference on Learning and Intelligent Optimization. Springer, Catania, Italy, 321 334.
Franco Mascia, Manuel López-Ibáñes Jérémie Dubois- Lacoste, and Thomas Stützle. 2014. Grammar- based generation of stochastic local search heuristics through automatic algorithm configuration tools. Computers & Operations Research 51 (Nov. 2014), 190 199.
Weiyao Meng and Rong Qu. 2021. Automated design of scarch algorithms: Learning on algorithmic compo- nents. Erpert Systems with Applications 185 (Dec. 2021), 115493.
Grégoire Mialon, Roberto Dessi, Maria Lomeli, Christoforos Nalmpantis, Ram Pasumuru, Roberta Raileamu, Baptiste Rozière, Timo Schick, Jane Dwivedi- Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. 2023. Augmented language models: survey. Transactions on Machine Learning Re- search (June 2023).
Risto Müikkulainen and Stephanie Forrest. 2021. A biological perspective evolutionary computation. Nature Machine Intelligence 3, (Jan. 2021), 9-15.
Julian F Miller. 2011. Cartesian genetic programming, In Cartesian Genetic Programming, Springer, New
York, NY, USA, 17-34.
Péricles Barbosa Miranda and Ricardo Bastos Prudéncio. 2015. GEFPSO: A framework for PSO optimization based on grammatical evolution. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Madrid, Spain, 1087-1094.
Péricles BC Miranda and Ricardo BC Prudéncio. 2020. A novel context-free grammar for the generation of pso algorithms. Natural Compuling 19, 3 (Mar. 2020),
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (Feb. 2015), 529 533.
Pablo Moscato. 1999. Memetic algorithms: A short introduction. In New Ideas in Optimization. McGraw- Hill, 219-234.
25

---

Published in Transactions on Machine Learning Research (02/2024)
Su Nguyen, Dhananjay Thiruaddy, Mengjie Zhang, and Damminda Alahakoon. 2022. Automated design of multipass heuristics for resource-constrained job scheduling with self-competitive genetic programming. IEEE Transactions on Cybermetics 52, 9 (Sept. 2022), 8603 8616.
Su Nguyen, Mengjie Zhang, Mark Johnston, and Kay Chen Tan. 2014. Automatic design of scheduling policies for dynamic multi-objective job shop scheduling via cooperative coevolution genetic programming. IEEE Transactions on Evolutionary Computation 18, 2 (Apr. 2014), 193-208.
Adel Nikfarjam, Aneta Neumann, and Frank Neumann. 2022. Evolutionary diversity optimisation for the traveling thief problem. In Proceedings of the Genetic and Evolutionory Computation Conference Com- panion. Boston, MA, USA, 749-756.
Mihai Oltean. 2005. Evolving evolutionary algorithms using linear genetic programming. Evolutionary Computation 13, (Sept. 2005), 387-410.
Federico Pagnozzi and Thomas Stützle. 2019. Automatic design of hybrid stochastic local search algorithms for permutation flowshop problems. European Journal of Operational Research 276, 2 (Jul. 2019), 409-421.
Nelishia Pillay and Rong Qu. 2018. Hyper-heuristics: USA.
Theory and applications. Springer, New York, NY,
Nelishia Pillay and Rong Qu. 2021. Assessing hyper-heuristic performance. Journal of the Operational Research Society 72, 11 (Aug. 2021), 2503-2516.
Michael L Pinedo. 2012. Scheduling. Vol. 29. Springer.
Riccardo Poli, Cecilia Di Chio, and William B Langdon. 2005a. Exploring extended particle swarms: A genetic programming approach. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Washington DC, USA, 169-176.
Riccardo Poli, William B Langdon, and Owen Holland. 2005b. Extending particle swarm optimisation via genetic programming. In European Conference on Genetic Programming. Springer, Lausanne, Switzerland, 291-300.
R. Qu, G. Kendall, and N. Pillay. 2020. The general combinatorial optimization problem: Towards automated algorithm design. IEEE Computational Intelligence Magazine 15, 2 (May 2020), 14-23.
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. 2020. Designing network design spaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognilion. Seattle, WA, USA, 10428-104366 10436.
Samuel N Richter, Michael G Schoen, and Daniel R Tauritz. 2019. Evolving mean-update selection methods for CMA-FS.. In Proceedings of the Genelic and Evolutionary Compulation Conference Companion. Prague, Czech Republic, 1513 1517.
Samuel N Richter and Daniel R Tauritz. 2018. The automated design of probabilistic selection methods for evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Kyoto, Japan, 1545-1552.
Rebecca Rivers and Daniel R Tauritz. 2013. Evolving black-box search algorithms employing genetic pro- gramming. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Amster- dam, The Netherlands,
Jairo Delgado, Josu Ceberio, Borja Calvo, and Jose A Lozano. 2022. Bayesian Performance Analysis for Algorithm Ranking Comparison. IEEE Transactions on Evolutionary Computation 26, 1281 1292 (Dec. 2022), 48-60.
Bernardino Rorera-Parodes, Mohammadamin Barckatain, Alexander Novikov, Matej Balog, M Pawan Ku- mar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. 2024. Mathematical discoveries from program search with large language models. Nature 625 (2024), 468 475.
26

---

Published in Transactions on Machine Learning Research (02/2024)
Conor Ryan, John James Collins, and Michael O Neill. 1998. Grammatical evolution: Evolving programs for arbitrary language. In European Conference on Genelic Programming. Springer, Paris, France, 83-96.
Patricia Syser-Welcke Julian F Miller, Jerry Swan, and Martin A Trefzer. 2016. Iterative Cartesian ge- netic programming: Creating general algorithms for solving travelling salesman problems. In European Conference on Genetic Programminng. Springer, Porto, Portugal, 294 310.
Wecrapan Sac-Dan, Maric-Eléonore Kessaci, Nadarajen Vecrapen, and Lactitia Jourdan. 2020. Time- dependent aulomatic parameter configuration of a local search algorithm. In Proceedings of the Genelic and Evolutionary Computation Conference Companion. Cancún, Mexico, 1898-1905.
Flias Schede, Jasmin Brandt, Alexander Tornede, Marcel Wever, Viktor Bengs, Byke Hüllermeier, and Kevin Tierney. 2022. A survey of methods for automated algorithm configuration. Journal of Artificial Intelligence Research 75 (Oct. 2022), 425-487.
Jan Schuchardt, Vladimir Golkov, and Daniel Cremers. 2019. Learning to evolve. arXiv:1905.0m 03389 (2019).
arXiv preprinl
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint (xXic:1007003//" (2017).
Gresa Shala, André Biedenkapp, Noor Awad, Steven Adriaensen, Marius Lindauer, and Frank Hutter. 2020. Learning step-size adaptation in CMA-ES. In International Conference on Porallel Problem Solving from Nalure. Springer, Leiden, The Netherlands, 691 191-706.
Mudita Sharma, Alexandros Komninos, Manuel López-Ibáñez, and Dimitar Kazakov. 2019. Deep rein- forcement learning based parameter control in differential evolution. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. Prague, Czech Republic, 709 717.
Yuhui Shi and Russell Eberhart. 1998. A modified particle swarm optimizer. In IEEE International Confer- ence on Evolutionary Compulation. IEEE, Anchorage, AK, USA, 69 73.
Shinichi Shirakawa and Tomoharu Nagao. 2009. Evolution of search algorithms using graph structured program evolution. In European Conference on Genetic Progrumming. Springer, T'ubingen, Germany, 109-120.
Ekaterina Smorodkina and Daniel Tauritz. 2007. Toward automating EA configuration: The parent selection stage. In IEEE Congress on Evolutionary Computation. IEEE, Singapore, 63 70.
Loe Spector 2001. Autoconstructive cvolution: Push, pushGP, and pushpop. In Proceedings of the Genetic and Evolutionary Compulation Conference Companion. San Francisco, CA, USA.
Kenneth O Stanley, Jeff Clune, Joel Lehman, and Risto Mikkuulannnn 2019. Designing neural networks through neuroevolution. Nature Machine Intelligence 1, (Jan. 2019), 24-35.
Thomas Stützle and Manuel López- Ibáñez.. 2019, Automated design of metaheuristic algorithms. In Handbook of Metaheuristics. Springer, New York, NY, USA, 541-579.
hurnuthurai N Suganthan, Nikolaus Hansen, Jing J Liang, Kalyanmoy Deb, Ying-Ping Chen, Anne Auger, and Santosh Tiwari. 2005. Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. KanGAL Report 2005005 (2005).
Jianyong Sun, Xin Liu, Thomas Bäck, and Zongben Xu. 2021. Learning adaptive differential evolution algo- rithm from optimization experiences by policy gradient. IEFE Transactions on Fvolutionary Computation 25, 4 (Aug. 2021), 666-680.
Jerry Swan, Steven Adriacnson, Alexander EI Brownlce, Kevin Hammond, Colin G Johnson, Ahmed Kheiri, Faustyna Krawiee, Juan Julián Merelo, Leandro Minku, Ender Özcan, et al. 2022. Metaheuristics "in the large". European Journal of Operational Research 297, 2 (Mar. 2022), 393 406.
27

---

Published in Transactions on Machine Learning Research (02/2024)
Jerry Swan, Patrick De Causmaecker, Simon Martin, and Ender Özcan. 2017. A re-characterization of hyper-heuristies. In Recent Developments of Melaheurislics. Springer, 75-89.
Ke Tang, Shengoai Liu, Peng Yang, and Xin Yao. 2021. Few-shots parallel algorithm portfolio construction via co-evolution. IEEE Transactions Exolutionary Computation 25, 3 (Jun. 2021), 595 607.
Sara Tari, Holger Hoos, Julie Jacques, Marie-Eléonos Kessaci, and Lactitia Jourdan. 2020a. Automatic configuration of multi-objective local search for imbalanced classification. In International Conference on Parallel Problem Solving from Nature. Springer, Leiden, The Netherlands, 65 77.
Sara Tari, Nicolas Szezepanski, Lucien Mousin, Julie Jacques, Marie-Eléonene Eléonore Kessaci, and Laetitia Jourdan. 2020b. Multi-objective automatic algorithm configuration for the classification problem of imbalanced data. In IEEE Congress Evolutionary Computation. IEEE, Glasgow, UK, 1-8.
Jorge Tavares and Francisco Percira. 2012. Automatic design of ant algorithms with grammatical evolution. In European Conference on Genelic Programming. Málaga, Spain, 206 -17.
Matthew E Taylor and Peter Stone. 2009. Transfer learning for reinforcement learning domains: A survey. Journal of Machine Learning Research 10, 7 (July 2009), 1685-
Dhananjay Thiruvady, Gaurav Singh, Andreas T Ernst, and Bernd Meyer. 2013. Constraint- ACO for a shared resource constrained scheduling problem. International Journal of Production Economics 141, (Jan. 2013), 230 242.
Ye Tian, Xingyi Zhang, Cheng He, Kay Chen Tan, and Yaochu Jin. 2023. Principled design of translation, scale, and rotation invariant variation operators for metaheuristics. Chinese Journal of Electronics 32, 1 (Jan. 2023), 111-129.
Braden Tisdalc, Deacon Seals, Aaron Scott Pope, and Daniel R Tauritz. 2021. Directing evolution: The aulomated design of evolutionary pathways using directed graphs. In Proceedings of the Genelic and Evolutionary Computation Conference Companion. Lille, France, 732-740.
Alexander Tornede, Difan Deng, Theresa Eimer, Joseph Giovanelli, Aditya Mohan, Tim Ruhkopf, Sarah Segel, Daphne Theodorakopoulos, Tanja Tornede, Henning Wachsmuth, et al. 2023. AutoMI. in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks. arXiv preprint arXiv:2306.0880 08107 (2023).
Koen van der Blom, Holger H Hoos, Chuan Luo, and Jeroen Rook. 2022. Sparkle: Towards Accessible Algorithhss for Improving the State of the Art in Solving Challenging Problems. IEEE Transactions on Evolutionary Compulation 26, 6 (Dec. 2022), 1351-1364.
Sander van Rijn, Hao Wang, Matthijs van Leeuwen, and Thomas Bäck. 2016. Evolving the structure of evolution strategies. In IEEE Symposium Series on Computational Intelligence. IEEE, Athens, Greece, 1-8.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. Long Beach, CA, USA.
José Antonio Rodriiz=o and Gabriela Ochoa. 2011. On the automatic discovery of variants of the NEH procedure for flow shop scheduling using genetic programming. Journal of the Operational Research Society 62, 2 (Aug. 2011), 381-396.
Diederick Vermetten, Hao Wang, Manuel Tópez-Thañez, Carola Doerr, and Thomas Bäck. 2022. Analyz- ing the impact of undersampling on the benchmarking and configuration of evolutionary algorithms. In Proceedings of the Genelic and Evolutionary Computation Conference Companion. 867 875.
CL Camacho Villalón, Marco Dorigo, and Thomas Stützle. 2022. PSO-X: A component-based framework for the automatic design of particle swarm optimization algorithms. IEEE Transactions on Evolutionary Computation 26, (Jun. 2022), 402-416.
28

---

Published in Transactions on Machine Learning Research (02/2024)
Christos Voudouris and Edward Tsang. 1999. Guided local search and its application to the traveling salesman problem. Europenn journal of operational research 113, 2 (Mar. 1999), 469-499.
Hao Wang, Bas van Stein, Michael Emmerich, and Thomas Back. 2017. A new acquisition function for Bayesian optimization based on the noneentgeneeating function. In IEEE International Conference on Systems, Man, and Cybernetics. IEEE, Banff, Canada, 507 512.
Danny Weyns, Thomas Bäck, Renè Vidal, Xin Yao, and Ahmed Nabil Belbachir. 2021. Lifelong computing, arXiv preprinl ar Xie:21108. 08808 (2021).
Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. 2014. Natural evolution strategies. The Journal of Machine Learning Research 15, (Mar. 2014), 949-980.
John R Woodward and Ruibin Bai. 2009. Why evolution is not a good paradigm for program induction: A critique of genetic programming. In Proceedings of the ACM/SIGEVO Summit on Genetic and Evolution- ary Compulation. Shanghai, China, 593 600.
John R Woodward and Jerry Swan. 2012. The automatic generation of mutation operators for genetic algo-
rithms. In Proceedings of the Genelic and Enolutionary Computalion Conference Compunion. Philadelphia, PA, USA, 67-74
Xingyu Wu, Sheng-hao Wu, Jibin Wu, Liang Feng, and Kay Chen Tan, 2024. Evolutionary computation the era of large language model: Survey and roadmap. ar preprint arXic:2401.1003, (2024).
Yi Xiang, Han Huang, Yuren Zhou, Sizhe Li, Chuan Luo, Qingwei Lin, Miqing Li, and Xiaowei Yang. 2022. Search-based diverse sampling from real-world software product lines. In IEEE/ACM International Conference on Software Engineering. Pittsburgh, PA, USA, 1945-1957.
Lin Xu, Ashiqur R KhudaBuukshh, Holger H Hoos, and Kevin Leyton-Brown. 2016. Quantifying the simi- larity of algorithm configurations. In International Conference on Learning and Intelligent Optimization. Springer, Ischia, Italy, 203 217.
Yuan Yan, Qunfeng Liu, and Yun Li. 2023. Paradox-free analysis for comparing the performance of opti- mization algorithms. IEEE Transactions on Evolutionary Compulation 27, (Oct. 2023), 1275-1287.
Furong Ye, Carola Doerr, Hao Wang, and Thomas Bäck. 2022a. Automated configuration of genetic algo- rithms by tuning for anytime performance. IEEF Transactions on Evolutionary Computation 26, 6 (Dec.
2022), 1526 1538.
Furong Ye, Diederick Vermetten, Carola Doerr, and Thomas Bäck. 2022b. Non-clitist selection among survivor configurations can improve the performance of irace. arXiv preprinl vxXiv:2203.09220 (2022).
Haoran Ye, Jiarui Wang, Zhiguang Cao, and Guojie Song. 2024. ReEvo: Large language models as hyper-
heuristics with reflective evolution. arXiv preprint arXiv:2402.011455 (2024).
Wenjie Yi, Rong Qu, and Licheng Jiao. 2023a. Automated algorithm design using proximal policy optimi- sation with identified features. Expert Systems with Applications 216 (Apr. 2023), 119461.
Wenjie Yi, Rong Qu, Licheng Jiao, and Ben Niu. 2023b. Automated design of metaheuristics using reinforce- ment learning within a novel general search framework. IEEE Transactions on Evolutionary Computation 27, 4 (Aug. 2023), 1072 1084.
Qi Zhao, Bai Yan, Xianglong Chen, Taiwei Hu, Shi Cheng, and Yuhui Shi. 2022. AutoOpt: A general frame- work for automatically designing metaheuristic optimization algorithms with diverse structures. arXiv preprint arXiv:2204.000 009998 (2022).
Qi Zhao, Bai Yan, Taiwci Hu, Xianglong Chen, Qiqi Duan, Jian Yang, and Yuhui Shi. 2023a. Au- toOptLib: Automatically tailoring metaheuristic optimizers via automated algorithm design. arXiv preprint ar Xrxx:22303.000 06536 (2023).
29

---

Published in Transactions on Machine Learning Research (02/2024)
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023b. A survey of large language models. arviv preprint arXiv:2303.1823 (2023).
Zhuangdi Zhu, Kaixiang Lin, Anil K Jain, and Jiayu Zhou. 2023. Transfer learning in deep reinforcement learning: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence 45, 11 (Nov. 2023). 13344 13362.
Eckart Zitzler and Lothar Thiele. 1999. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. JEEE Transactions Evolulionary Compulation 3, 4 (Nov. 1999),
257-271.
30


Reviews:
[{"review_id": "PmbH5e5AH5", "reviewer": "Reviewer_q9oD", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nThis is a survey of an important topic: automatic design of algorithms. Given the constant increase in computational power, one can imagine that systems will be able to invent novel and powerful algorithms. The survey focuses on metaheuristics, which is an important niche in artificial intelligence and operations research. \n\nThe survey covers a large number of papers and can be helpful to readers trying to understand \"what is out there\" in terms of automated design of metaheuristics. The survey is also thorough in the sense that it covers all the parts involved in the process of designing a system that will design metaheuristics. \n\nSummary of Weaknesses (see detailed comments for more information)\n\n1. The survey only superficially covers some of the topics, such as representation of the search space. \n2. The paper doesn't fully explain some of the branches of the literature. Often I missed examples that would help me understand what a group of papers achieved. Some of the topics require longer sections so they can be fully explained. The explanations are shallow to the point that I need to read the papers cited to understand the high-level idea of the paper. \n3. I normally expect a survey paper to be insightful, to teach me in a few pages something I would have to spend hours reading the literature to learn. This survey doesn't provide such insights. The closest it gets from providing insights is when the text points out that some of these areas require more research. \n4. The mathematical notation of the paper is confusing. \n\nDetailed Comments\n\nThe binary tree representation explained in the survey is specific to binary operators. That is, the tree used as an example in Figure 2 is binary because it deals with multiplication, addition, and subtraction. How about a more general representation such as Abstract Syntax Trees? That way, if the language defining the space included ternary operators, the tree would include nodes with three children, to follow the number of \"non-terminals\" in each rule. \n\nThe text mentions that the binary trees present challenges related to overfitting and underfitting. This is a misconception. It is not the representation that presents this challenge, but the space they represent. For example, one could use trees to represent all programs one can write in C++. This space will certainly allow for overfitting as one can write a C++ program that will match exactly whichever data they have. If the language only allows for arithmetic operations, then the system will likely suffer from underfitting, giving that one cannot represent many hypotheses in this space. \n\nAn example of the linear array representation would be helpful as I could not understand this representation from the description of the text. Similarly, an example of the grammar shown in Figure 3 would also be helpful. The example in Figure 4 would be more clear if the operations a, f, b, and g were concrete. The paper would probably benefit from having a running example for which we see many different representations of solutions to the example. \n\nWhy is it difficult to manipulate a graph? I believe this is another misconception. What the authors meant is that the space the graph represents is too large. This is because it is not difficult to manipulate the graph structure as one can easily define edit operations for graphs. \n\nSome terms are used in the text without explanation. For example, what are phenotypes? What is a bloated space? \n\nThe paper poses local search and evolutionary search as two different topics. However, to me, evolutionary search is a type of local search. It often considers many local points, but it is still performing local search, as opposed to enumerative search. \n\nThe paper mentions the advantages of Reinforcement Learning as the ability of transferring the value function and policy to other problems. However, it doesn't mention to possible issues with this approach. That is, often the value function and policy will not transfer from one task to the next, even if they are similar. \n\nI don't understand the approaches related to Estimation of Distribution. This is one of the branches of the literature that needs to be better explained. The best the paper does is to point at several papers that use this type of approach. \n\nIn Equation 1, the use of the expectation operator $E[⋅]$ is ambiguous, as it is not specified over which variable or set of variables the expectations are being computed. Function $P$ is defined over two parameters, but only one appears in the equation. Also, there is no need to add $A \\in S$ as a constraint; one just needs to write $A \\in S$ under the argmax operator. \n\nEquation 2 is equally confusing due to the expectations: over which variables are they being computed? \n\nI don't understand the notation $P(A)|i$ in Equation 3. Also, it isn't clear how the minimum between $r$ and $b$ can be meaningful. This is because they represent different quantities. \n\nEquation 5 doesn't make sense to me. How come $r(C) \\times N = r(C \\times N)$. The value of $C$ isn't defined.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a survey on different methodologies for automated design of metaheuristics. The paper also presents a taxonomy for the different components the user of such systems need to be concerned with: design space, design strategy, and performance evaluation. \n\nThe survey situates itself in the context of other related surveys, mentioning that previous surveys either focus on other problems (e.g., algorithm selection) or lack information.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would like to see a deep re-write of this paper, where the resulting paper will teach important insights, as opposed to only pointing at different papers. The re-write will have a running example where different representations and designed metaheuristics can be shown. The paper will be more general (e.g., in terms of representations of trees). What can someone who read of these papers conclude that someone who read each of the papers individually wouldn't be able to? \n\nFinally, the mathematical notation will be fixed. I believe there is at least one problem in every equation of the paper.", "broader_impact_concerns": "Not applicable.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3KhbiYY8D3", "reviewer": "Reviewer_FTmA", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "[Strengths]\n- The broad picture for the automated design of metaheuristic algorithms is provided, which is useful for people who want to start the research on this topic.\n- Surveying the automated design methods in terms of four modules, including design space, design strategies, performance evaluation strategies, and target problems, is somewhat novel.\n\n[Weaknesses]\n- Some state-of-the-art works are missing. Please see Requested Changes.\n- Learning-based design strategies might be the most interesting approach in the machine learning community. However, the contents regarding them are limited (only in 7.2).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper provides a survey of the automated design of metaheuristic algorithms (black-box optimizers). As automatically designed algorithms can reach and sometimes surpass manually designed ones, a lot of methods have been investigated so far. This paper presents the representative techniques from four aspects: design space, design strategies, performance evaluation strategies, and target problems. The contribution of this paper is to survey and provide the existing techniques for four aspects.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Some state-of-the-art works are missing. It would be better to include the recent approaches, especially a (meta)-learning-based approach, for the automated design of black-box optimization algorithms. For example, the following papers should be considered to be included in this survey.\n\nRobert Tjarko Lange, Tom Schaul, Yutian Chen, Tom Zahavy, Valentin Dalibard, Chris Lu, Satinder Singh, Sebastian Flennerhag:\nDiscovering Evolution Strategies via Meta-Black-Box Optimization. ICLR 2023\n\nMudita Sharma, Alexandros Komninos, Manuel López-Ibáñez, and Dimitar Kazakov. 2019. Deep reinforcement learning based parameter control in differential evolution. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO '19). Association for Computing Machinery, New York, NY, USA, 709–717. https://doi.org/10.1145/3321707.3321813\n\nJ. Sun, X. Liu, T. Bäck and Z. Xu, \"Learning Adaptive Differential Evolution Algorithm From Optimization Experiences by Policy Gradient,\" in IEEE Transactions on Evolutionary Computation, vol. 25, no. 4, pp. 666-680, Aug. 2021, doi: 10.1109/TEVC.2021.3060811.\n\nShala, G., Biedenkapp, A., Awad, N., Adriaensen, S., Lindauer, M., Hutter, F. (2020). Learning Step-Size Adaptation in CMA-ES. In: Bäck, T., et al. Parallel Problem Solving from Nature – PPSN XVI. PPSN 2020. Lecture Notes in Computer Science, vol 12269. Springer, Cham. https://doi.org/10.1007/978-3-030-58112-1_48", "broader_impact_concerns": "This paper does not have a concern on the ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "M3WAtU4nhU", "reviewer": "Reviewer_R38i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "As a survey paper, I find the main strengths are:\n* The paper is well structured and well written. While there are many in-depth discussions, I find the writing engaging and easy to follow. The paper should be very accessible to a wide range of audience from different communities (e.g., genetic programming, automated machine learning).\n* The role of the design space is highlighted, which is not emphasized in prior surveys. I think this is a great decision, since in practice, the design space has great impact on the search efficiency and the performance of the searched algorithm. The discussion is thoughtfully organized into two parts: space with low-level computational primitives and space with high-level algorithmic operators. For each space type, different representation options are given (tree, linear array, graph) and their pros and cons are clearly discussed.\n* The usability sections provide practical recommendation for researchers what options to consider for each module.\n* The authors provide valuable insights on why certain methods are appealing for certain cases. (e.g. the discussion of reinforcement learning algorithms in section 4.1)\n\nI don't see particular weaknesses in this survey.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Contributions\n* The authors formalize the auto-design of metaheuristic algorithms into four modules (design space, design strategy, performance evaluation strategy, and target problem). \n* Design options for each module are discussed. Each of the first three modules ends with a comment on usability of the options.\n* The design space module is highlighted, which is not covered in related surveys.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "[Optional] It could be useful to include a figure for the linear array representation (just like Figure 2 is for tree, and Figure 4 is for graph)", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In the initial reviews, reviewer R38i and reviewer FTmA appreciate the contributions and strengths of this survey, and also provide some suggestions (e.g., missing related works and contents, more illustrative figures) for further improvements. These issues have been well addressed during the rebuttal, and they recommend accepting this work. Reviewer q9oD gives a list of detailed and constructive criticism on the weaknesses in the initial review, and the authors provide a thorough response to tackle these concerns. In the official recommendation, reviewer q9oD finds that some of their concerns (especially those on writing) have been addressed, but some deeper feedback (the survey is superficial, lack of insight into existing literature) are overlooked. In addition, this reviewer wants to remind the authors that they can upload an edited version of the paper during the discussion phase, which could be very useful for the rebuttal. Finally, given the potential value of this survey for newcomers in the field, reviewer q9oD leans toward accepting this work.  

I read this paper in detail and also believe this work is valuable for anyone who want to work on automated metaheuristic algorithm design. One thing that can be improved is to include a brief discussion on large language model (LLM) for metaheuristic algorithm design. Very recently (and actually after the submission of this survey), some interesting works have been proposed to leverage the power of LLM to automatically design powerful metaheuristic algorithms to tackle different real-world application problems such as [1,2,3]. There are also two survey papers on LLM for evolutionary algorithms from the metaheuristic community [4,5]. Since LLM for algorithm design could be a promising research trend, adding a brief timely discussion on this topic could be very useful for the readers. This discussion can also further strengthen this survey on addressing reviewer FTmA's concern on limited content on learning-based algorithm design. Therefore, I recommend **accept with minor revision** for this work.    

In the revised paper, I expect the authors to: 

+ Double-check and ensure all the discussions/analyses provided in the rebuttal are well incorporated in the revised paper. 

+ Add a brief discussion on large language model for algorithm design.

Given that it is a survey paper, I also recommend a Survey Certification for this work.

[1] Mathematical discoveries from program search with large language models, Nature 2024.

[2] An example of evolutionary computation+ large language model beating human: Design of efficient guided local search, arXiv:2401.02051.

[3] ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution, arXiv:2402.01145.

[4] Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap,  arXiv:2401.10034.

[5] A match made in consistency heaven: when large language models meet evolutionary algorithms, arXiv:2401.10510.
Decision: Accept (poster)