Reviewerly/exHarmony huggingface dataset:
Evolutionary continuous dynamic optimization

tutorial Share on Evolutionary continuous dynamic optimization Authors: Danial Yazdani Southern University of Science and Technology, Shenzhen, China Southern University of Science and Technology, Shenzhen, ChinaView Profile , Xin Yao University of Science and Technology, Shenzhen, China University of Science and Technology, Shenzhen, ChinaView Profile Authors Info & Claims GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionJuly 2022 Pages 1230–1242https://doi.org/10.1145/3520304.3533643Published:19 July 2022Publication History 0citation48DownloadsMetricsTotal Citations0Total Downloads48Last 12 Months48Last 6 weeks4 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Exactly characterizable parameter settings in a crossoverless evolutionary algorithm

The number of offspring and the population size greatly influence the performance of the Plant Propagation Algorithm, a crossover-less metaheuristic, when applied to the Euclidean traveling salesman problem. However, with large volumes of experimental data, a clearly characterizable relation emerges, facilitating both an exact parameterization as well as opening up perspectives on a precisely predictable outcome. These results directly question its position between exact, approximation, and PTAS algorithms.
An explainable visualisation of the evolutionary search process

The comprehension of the Evolutionary Algorithm (EA) search process is often eluded by challenges of transparency inherent to black-box EAs, thus affecting algorithm enhancement and hyperparameter optimisation. In this work, we develop algorithm insight by introducing the Population Dynamics Plot (PopDP). PopDP is a novel and intuitive visualisation capable of visualising the population of solutions, the parent-offspring lineage, solution perturbation operators, and the search process journey. We apply PopDP to NSGA-II to demonstrate the insight attained and the effectiveness of PopDP for visualising algorithm search on a series of discrete dual- and many-objective knapsack problems of different complexities, and our results demonstrate that the method can be used to produce a visualisation in which the lineage of solutions can be clearly seen. We also consider the efficacy of the proposed explainable visualisation against emerging approaches to benchmarking explainable AI methods and consider the accessibility of the resulting visualisations.
Evolving parsimonious circuits through shapley value-based genetic programming

Evolutionary analog circuit design is a challenging task due to the large search space incurred by the circuit topology and device values. Applying genetic operators on randomly selected genes may make it difficult to identify which part of sub-circuit is beneficial to the evolution and even destroy useful sub-circuits, potentially incurring stagnation of the evolutionary process and bloat on the evolved circuits. In this paper, we propose a tree-based approach called Shapley Circuit Tree that incorporates Shapley values for quantifying the contribution of each function node of the circuit tree to the performance of the whole tree, to guide the evolutionary process. Our experiments on three benchmarks show that the proposed approach is able to evolve analog circuits with smaller area while converging faster than existing approaches.
CGP-NAS

Convolutional Neural Networks (CNNs) have had a remarkable performance in difficult computer vision tasks. In previous years, human experts have developed a number of specialized CNN architectures to deal with complex image datasets. However, the automatic design of CNN through Neural Architecture Search (NAS) has gained importance to reduce and possibly avoid human expert intervention. One of the main challenges in NAS is to design less complex and yet highly precise CNNs when both objectives conflict. This study extends Cartesian Genetic Programming (CGP) for CNNs representation in NAS through multi-objective evolutionary optimization for image classification tasks. The proposed CGP-NAS algorithm is built on CGP by combining real-based solutions representation and the well-established Non-dominated Sorting Genetic Algorithm II (NSGA-II). A detailed empirical assessment shows CGP-NAS achieved competitive performance when compared to other state-of-the-art proposals while significantly reduced the evolved CNNs architecture's complexity as well as GPU-days.
An effective metaheuristic-based pruning method for convolutional neural network

Selecting filters to be pruned and determining the pruning ratio are two important research issues of model compression because both of them will strongly impact the accuracy, size, and computation time of a neural network. In this study, an effective algorithm---based on a promising metaheuristic algorithm, search economics, to find the pruning ratio and the filter importance of each convolutional layer to remove unimportant filters---is proposed. Experimental results show that the proposed algorithm can find a better solution than other pruning methods compared in this study.
Selection hyper-heuristics

tutorial Share on Selection hyper-heuristics Authors: Ahmed Kheiri Lancaster University, Lancaster, UK Lancaster University, Lancaster, UKView Profile , Edward Keedwell University of Exeter, Exeter, UK University of Exeter, Exeter, UKView Profile Authors Info & Claims GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionJuly 2022 Pages 983–996https://doi.org/10.1145/3520304.3533655Published:19 July 2022Publication History 0citation48DownloadsMetricsTotal Citations0Total Downloads48Last 12 Months48Last 6 weeks8 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Multi-fidelity optimization method with asynchronous generalized island model for AutoML

AutoML frameworks seek to find the best configurations of machine learning techniques. A research field that has gained a great deal of popularity in recent years is the combined algorithm selection and hyperparameter optimisation (CASH) problem.
Coevolution of neural networks for agents and environments

Evolutionary strategies have recently shown great results in the field of policy search. Compared to some classical artificial neural networks used in reinforcement learning, evolutionary strategies generate populations of agents which are evaluated on a specific task. The algorithm detailed here demonstrates how artificial neural networks can be evolved in a process of neuroevolution and used as agents in the 2D-game Zelda, producing relevant behaviors. Moreover, to increase the diversity and quantity of available maps, this paper shows how it is possible to generate environments using evolutionary strategies and neuroevolution, as well as neural cellular automata. Finally, to evolve populations of environments and agents cohesively, a coevolution algorithm was developed. Results demonstrate the potential of coevolution in the field of videogames by creating a wide range of diverse environments, and by creating agent strategies to solve these levels. However, these results also highlight the complexity of continuously generating novelty as agents and maps tend to converge quickly on similar patterns.
Uncertainty in equation learning

Equation learning is a deep learning framework for explainable machine learning in regression settings, with applications in engineering and the natural sciences. Equation learners typically do not capture uncertainty about the model or its predictions, although uncertainty is often highly structured and particularly relevant for these kinds of applications. We show how simple, yet effective forms of Bayesian deep learning can be used to build structure and explainable uncertainty over a set of found equations. Specifically, we use a mixture of Laplace approximations, where each mixture component captures a different equation structure, and the local Laplace approximations capture parametric uncertainty within one family of equations. We present results on both synthetic and real world examples.
A verified application of genetic programming

In both academia and industry, Web services (WSs) technology (also called Web APIs in recent years) has been a fervid research target for years. In industry, software developers now often develop their application systems with diverse WSs on the internet (i.e., these developers often use service-oriented software engineering, SOSE). However, many companies and organizations, including most tech giants, such as Google, Meta, and Amazon, expose their services and business functionalities in the form of RESTful WSs (Web APIs) for external access (e.g., Amazon Web Services). As a focus of research in services computing, many topics and subjects regarding WSs have been identified, investigated, and addressed. A practical research issue that receives broad attention is the understanding (modeling) and prediction of the time-aware (time-varying) dynamic quality of service (QoS) of WSs.
Evolving spaceships with a hybrid L-system constrained optimisation evolutionary algorithm

Content generation for video games is a time-consuming process that impacts both game development and gameplay in the case of sandbox games. One approach to improve this process is to use procedural content generation, which can automatically generate diverse content for the game. Lindenmayer systems (L-systems) with handcrafted rules have successfully been used to generate content for games, but rely on undirected expansion of the rules. In contrast, by combining L-systems with an evolutionary algorithm for constrained optimisation, we are able to efficiently utilise simple L-system rules with directed search to find functionally-viable and appealing spaceship designs within the Space Engineers game.
Scalable evolutionary hierarchical reinforcement learning

This paper investigates a novel method combining Scalable Evolution Strategies (S-ES) and Hierarchical Reinforcement Learning (HRL). S-ES, named for its excellent scalability, was popularised with demonstrated performance comparable to state-of-the-art policy gradient methods. However, S-ES has not been tested in conjunction with HRL methods, which empower temporal abstraction thus allowing agents to tackle more challenging problems. We introduce a novel method merging S-ES and HRL, which creates a highly scalable and efficient (compute time) algorithm. We demonstrate that the proposed method benefits from S-ES's scalability and indifference to delayed rewards. This results in our main contribution: significantly higher learning speed and competitive performance compared to gradient-based HRL methods, across a range of tasks.
Cascades of evolutionary support vector machines

Support vector machines (SVMs) have been widely applied to binary classification, but their real-life applications are limited due to high time and memory complexities of training coupled with high sensitivity to the hyperparameters of the classifier. To train SVMs from large datasets, numerous techniques were proposed which select a subset out of all the data presented for training. However, it is challenging to determine the appropriate size of such a subset which may lead to sub-optimal performance. In this paper, we propose a new approach to building a cascade of SVMs, each of which is optimized using a memetic algorithm that selects a small subset of the training data and tunes the hyperparameters. The optimization at each level of the cascade is aimed at creating competence regions that altogether cover complementary parts of the input space. Our experiments performed over 12 synthesized datasets and 24 benchmarks revealed that our method outperforms other classifiers, including SVMs trained with the whole set as well as with a reduced set selected using other techniques. Furthermore, our cascade identifies the high-confidence regions in the input space, and the results confirm that they are characterized with increased classification accuracy obtained for the test data.
Entropy-based local fitnesses for evolutionary multiagent systems

Evolutionary multiagent systems have been successfully applied to many real world problems, including search and rescue and ocean exploration. However, as the number of agents increases in such problems, the evaluation function captures an individual agent's fitness less and less accurately. As a consequence, agents adopt a small set of acceptable behaviors that are neither optimal nor robust to environmental changes or teammate failures. Fitness shaping, intrinsic fitnesses, or multi-fitness learning alleviate some of these concerns but generally require domain knowledge or the functional form of the evaluation function. In this paper, we introduce Entropy-Based Local Fitnesses (EBLFs) that generate diverse behaviors for agents and produce robust team behaviors without requiring environmental knowledge. The key contribution of EBLFs is to inject a dense, entropy-based fitness into the agents' evolution without interfering with the sparse, high-level system evaluation function. Our results show that the agents using EBLFs learn new skills in difficult environments with sparse feedback without requiring domain knowledge. In addition, EBLFs generated new team-level behaviors that were not defined by a human operator, but beneficial to robust team performance.
Effects of encodings and quality-diversity on evolving 2D virtual creatures

How to jointly optimize the morphology and controller is a challenging problem in evolutionary robotics. Due to the large search space, both quality diversity algorithms and types of encodings have been employed to search the solution space more effectively. Here we compare Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) and a standard evolutionary algorithm as well as the effect of using a direct versus an indirect encoding. The results showed that the MAP-Elites algorithm found diverse solutions, yet the encodings accounted for a larger performance discrepancy. This indicates that the representation is at least as important as the optimization method for effectively creating robots.
Analyzing optimized constants in genetic programming on a real-world regression problem

Ephemeral random constants are commonly used for symbolic regression with genetic programming. However, due to their random nature, it is difficult for genetic programming to find proper constants. This often leads to large and complex non-interpretable solutions. Therefore, in this short paper, we analyze a method that replaces ephemeral random constants with constant tokens whose values are optimized during evolution with the Sequential Least Squares Programming method. The results achieved on the studied regression problem indicate that replacing ephemeral random constants by optimized constant tokens leads to solutions with a significantly lower error which are also smaller than the solutions generated by a standard GP approach using ephemeral random constants.
XCS on embedded systems

With the more widespread use of Learning Classifier Systems (LCS) in real-world applications, they are increasingly deployed on embedded computing systems with constrained computational power. We profile the most popular classifier system XCS on both an embedded system with an ARM CPU and an off-the-shelf desktop computer. A comparison of both systems yields distinct differences, as classifier deletion is the most runtime consuming function on the embedded system while matching dominates the execution on the desktop computer. To accelerate classifier deletion and make XCS more amenable for deployment on embedded systems, we propose and evaluate two different deletion mechanisms. While the first is an algorithmic modification of the original deletion approach preserving its behavior, the second is a hybrid scheme utilizing both global and local deletion. We find that the latter can achieve a speedup of up to 1.47 for the overall execution of XCS.
Benchmarking genetic programming in a multi-action reinforcement learning locomotion task

Reinforcement learning (RL) requires an agent to interact with an environment to maximize the cumulative rather than the immediate reward. Recently, there as been a significant growth in the availability of scalable RL tasks, e.g. OpenAI gym. However, most benchmarking studies concentrate on RL solutions based on some form of deep learning. In this work, we benchmark a family of linear genetic programming based approaches to the 2-d biped walker problem. The biped walker is an example of a RL environment described in terms of a multi-dimensional, real-valued 24-d input and 4-d action space. Specific recommendations are made regarding mechanisms to adopt that are able to consistently produce solutions, in this case using transfer from periodic restarts.
Learning classifier systems

tutorial Share on Learning classifier systems: cognitive inspired machine learning for eXplainable AI Authors: Abubakar Siddique Victoria University of Wellington, Wellington, New Zealand Victoria University of Wellington, Wellington, New ZealandView Profile , Will Browne Queensland University of Technology, Queensland, Australia Queensland University of Technology, Queensland, AustraliaView Profile Authors Info & Claims GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionJuly 2022 Pages 1081–1110https://doi.org/10.1145/3520304.3533664Published:19 July 2022Publication History 0citation54DownloadsMetricsTotal Citations0Total Downloads54Last 12 Months54Last 6 weeks10 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
A new grammatical evolution method for generating deep convolutional neural networks with novel topologies

Neuroevolution, a sub-field of AutoML, utilizes evolutionary algorithms to automate the process of creating Deep Neural Networks architectures. For problems with complex objects, such as neural networks, Grammar-based Evolutionary Algorithms (GEs) can be used to simplify the implementation and the experimentation by using grammar rules to describe what the components of the complex object are and how they can be connected, that is, they elegantly describe the search space of the problem. In this work, we propose a GE algorithm based on Structured Grammatical Evolution to generate deep convolutional neural networks. Our work has two major contributions: first, the neural networks may contain an arbitrary number and arrangement of skip connections; second, our skip connections may upscale lower-resolution inputs, allowing the generation of architectures such as U-Net. Our best model achieved 0.85 accuracy on CIFAR-10.
Reduction of genetic drift in population-based incremental learning via entropy regularization

The Population-Based Incremental Learning (PBIL) is a Bernoulli distribution-based evolutionary algorithm for binary black-box optimization. The PBIL updates the distribution parameter according to the samples generated from the current distribution and their rankings. In PBIL, when some distribution parameters are continuously updated randomly, undesirable convergence without sufficient exploration is observed. This behavior is called genetic drift and induces an increasing number of function evaluations and convergence to local optima. In particular, large update strength leads to genetic drift while faster search. The ways to deal with genetic drift are limited, such as decreasing the update strength, and there is a trade-off between search efficiency and stability. This paper proposes a method to reduce genetic drift in PBIL based on entropy regularization widely used in reinforcement learning. We introduce entropy regularization into PBIL as a penalty term or constraint. The experimental results on well-known benchmark problems show that the proposed entropy regularization can efficiently suppress genetic drift, decrease the number of function evaluations, and improve stability.
Improving the search of learning classifier systems through interpretable feature clustering

Learning Classifier Systems (LCS) are a well-known machine learning method, producing sets of interpretable rules in order to solve a variety of problems. Despite this, a common issue that these systems run into is the creation of unhelpful rules, caused by having multiple features in the data representing similar areas of knowledge. While we can logically know that these rules will not be useful in conjunction with each other, this is much more difficult for the algorithm to innately know.
A comparison of rule compaction algorithms for michigan style learning classifier systems

Learning Classifier Systems (LCSs) excel in data mining tasks, e.g. an LCS optimal model contains patterns that can precisely reveal how features identify classes for the explored problem. EC is stochastic, which leads to LCSs producing and keeping redundant rules that obscure the patterns. Thus, LCSs employ rule compaction methods to improve models by removing problematic rules. However, former compaction methods often fail to highlight the patterns and reduce accuracy after compaction. A survey of compaction methods is provided to investigate why compaction algorithms are incompetent in achieving expected performance. This work discovers a new LCS's optimal solution format, which refers to a model consisting of all correct unsubsumable rules in a given search space. This optimal solution is accurate in arbitrary noiseless datasets and contains interpretable patterns naturally. Thus, two compaction methods, specifying the production of such optimal solutions, are proposed. Successful compaction is demonstrated on complex and real problems with noiseless datasets, e.g. the 11-bits Majority-On problem that requires 924 different interacting rules in the optimum solution to be uniquely identified to enable correct visualization of the discovered knowledge. For the first time, the patterns contained in learned models for the large-scale 70-bits Multiplexer problem are visualized successfully.
Networks of evolution

Evolutionary computing has benefited several fields such as psychology, economics, and of course, computer science. These algorithms can tackle challenging real-world optimisation problems using selection, crossover, and mutation operators. Despite the rich literature examining evolutionary algorithms, there are unanswered questions concerning their behaviour and the interplay between operators. This work models genetic algorithms as dynamic networks where nodes represent the population and edges describe the inheritance link between individuals. Using the interaction networks as a proxy, we assessed the impact of different parameters, optimisation problems and operators (i.e. selection, crossover, mutation) on the algorithm's behaviour.