Reviewerly/exHarmony huggingface dataset:
On the effect of embedding hierarchy within multi-objective optimization for evolving symbolic regression models

Symbolic Regression is sometimes treated as a multi-objective optimization problem where two objectives (Accuracy and Complexity) are optimized simultaneously. In this paper, we propose a novel approach, Hierarchical Multi-objective Symbolic Regression (HMS), where we investigate the effect of imposing a hierarchy on multiple objectives in Symbolic Regression. HMS works in two levels. In the first level, an initial random population is evolved using a single objective (Accuracy), then, when a simple trigger occurs (the current best fitness is five times better than best fitness of the initial, random population) half of the population is promoted to the next level where another objective (complexity) is incorporated. This new, smaller, population subsequently evolves using a multi-objective fitness function. Various complexity measures are tested and as such are explicitly defined as one of the objectives in addition to performance (accuracy). The validation of HMS is performed on four benchmark Symbolic Regression problems with varying difficulty. The evolved Symbolic Regression models are either competitive with or better than models produced with standard approaches in terms of performance where performance is the accuracy measured as Root Mean Square Error. The solutions are better in terms of size, effectively scaling down the computational cost.
Adversarial example generation via genetic algorithm

In recent years, some studies showed that deep neural networks (DNNs) are vulnerable to being attacked by small perturbated examples. To satisfy the lexical, grammatical, and semantic constrain, some works proposed using black-box population-based optimization algorithms to attack neural networks in natural language processing. However, they are inefficient enough because they do not consider the characteristics of the text itself. Also, they are slow to close to the decision boundary. In this paper, we propose a more efficient attention based genetic algorithm adversarial attack method, called AGA. We use attention mechanism to pay more attention to the important tokens and utilize the multi-membered strategy to accelerate the search procedure. The result shows that our attack achieves a higher success rate with less than 136% of the number of queries than the existing methods.
Lexicase selection

tutorial Share on Lexicase selection Authors: Thomas Helmuth Hamilton College Hamilton CollegeView Profile , William La Cava Harvard Medical School Harvard Medical SchoolView Profile Authors Info & Claims GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionJuly 2022 Pages 1385–1397https://doi.org/10.1145/3520304.3533633Published:19 July 2022Publication History 0citation21DownloadsMetricsTotal Citations0Total Downloads21Last 12 Months21Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Accelerating neural architecture exploration across modalities using genetic algorithms

Neural architecture search (NAS), the study of automating the discovery of optimal deep neural network architectures for tasks in domains such as computer vision and natural language processing, has seen rapid growth in the machine learning research community. While there have been many recent advancements in NAS, there is still a significant focus on reducing the computational cost incurred when validating discovered architectures by making search more efficient. Evolutionary algorithms, specifically genetic algorithms, have a history of usage in NAS and continue to gain popularity versus other optimization approaches as a highly efficient way to explore the architecture objective space. Most NAS research efforts have centered around computer vision tasks and only recently have other modalities, such as natural language processing, been investigated in depth. In this work, we show how genetic algorithms can be paired with lightly trained objective predictors in an iterative cycle to accelerate multi-objective architectural exploration in the modalities of both machine translation and image classification.
Chaotic genetic bee colony

Evolutionary and Swarm algorithms show great effectiveness when performing feature selection, classification problems, and other optimization tasks. These scenarios highlight several algorithms such as Genetic Algorithm, Particle Swarm Optimization, Artificial Bee Colony, and Genetic Bee Colony (GBC). The last one is a combination of Genetic Algorithm and Artificial Bee Colony. Our study proposes an improvement over the GBC algorithm, including the Chaos Theory behavior in its foundation. In addition, we modified the Onlooker Bee and Scout Bee phase to improve the proposed model exploration and exploitation capabilities. Our novel proposal, Chaos-GBC, showed a very competitive performance compared to GBC and other metaheuristics. CGBC achieved the highest classification accuracy and the lowest average number of selected genes, showing the importance of our new proposal.
Biology inspired growth in meta-learning

Unlike biological systems most current neural networks models do not change their structure during learning. In most machine learning problems, it is important to choose an appropriate network structure because simple networks are likely to under-fit while complex networks are less plastic and more computationally expensive to train. To address this challenge, we introduce a dynamically growing network that starts from a small network and adds layers while training. We demonstrate an acceleration of learning using a simple dynamically growing network compared to other baseline models with fixed network structures on a meta-learning task.
QDax

Quality-Diversity (QD) algorithms are a well-known approach to generate large collections of diverse and high-quality policies. However, QD algorithms are also known to be data-inefficient, requiring large amounts of computational resources and are slow when used in practice for robotics tasks. Policy evaluations are already commonly performed in parallel to speed up QD algorithms but have limited capabilities on a single machine as most physics simulators run on CPUs. With recent advances in simulators that run on accelerators, thousands of evaluations can be performed in parallel on single GPU/TPU. In this paper, we present QDax, an implementation of MAP-Elites which leverages massive parallelism on accelerators to make QD algorithms more accessible. We show that QD algorithms are ideal candidates and can scale with massive parallelism to be run at interactive timescales. The increase in parallelism does not significantly affect the performance of QD algorithms, while reducing experiment runtimes by two factors of magnitudes, turning days of computation into minutes. These results show that QD can now benefit from hardware acceleration, which contributed significantly to the bloom of deep learning.
Highlights of semantics in multi-objective genetic programming

Semantics is a growing area of research in Genetic programming (GP) and refers to the behavioural output of a Genetic Programming individual when executed. This research expands upon the current understanding of semantics by proposing a new approach: Semantic-based Distance as an additional criteriOn (SDO), in the thus far, somewhat limited researched area of semantics in Multi-objective GP (MOGP). Our work included an expansive analysis of the GP in terms of performance and diversity metrics, using two additional semantic-based approaches, namely Semantic Similarity-based Crossover (SCC) and Semantic-based Crowding Distance (SCD). Each approach is integrated into two evolutionary multi-objective (EMO) frameworks: Non-dominated Sorting Genetic Algorithm II (NSGA-II) and the Strength Pareto Evolutionary Algorithm 2 (SPEA2), and along with the three semantic approaches, the canonical form of NSGA-II and SPEA2 are rigorously compared. Using highly-unbalanced binary classification datasets, we demonstrated that the newly proposed approach of SDO consistently generated more non-dominated solutions, with better diversity and improved hypervolume results.
XCSF under limited supervision

XCSF is a well-researched function approximator used for supervised, online regression learning tasks. Hereby, XCSF relies on receiving a ground truth after each prediction step in order to update its internal rulebase. As there exist various conceivable scenarios where the amount of external supervision might be limited, we investigate how XCSF can be adapted in order to be applicable in such environments. We change the update mechanism such that only inexperienced classifiers are updated, and introduce an external covering procedure as well as a cache for saving external supervisions. Our experiments show that, despite performing significantly worse than an non-limited XCSF, our adaptations still provide acceptable results and limit the necessity for external supervision noticeably.
Espresso to the rescue of genetic programming facing exponential complexity

Fitness computation is a crucial part of any population-based evolutionary strategy. In Genetic Programming (GP) each individual of a population is evaluated by comparing the output they return when fed some input with the expected output. The mapping input/output form the test cases, it describes the "behaviour" an individual should have when presented an instance of a problem. However, there exist situations in which the mapping is given as a function an individual should implement, thus consisting in combinations of several variables of the problem. This paper addresses the issue of efficiently computing the fitness of an individual evaluated on such test sets exponentially large in the number of variables. The inspiration came from digital electronics and more specifically the ESPRESSO-MV algorithm, accepting multi-valued variables. The proposed datastructure represents subsets of the solution space, hence allowing to compute the number of passed test cases as a single operation rather than enumerating all of them. The heuristics used rely on the Unate Recursive Paradigm (URP), some of the proposed algorithms are new while others come directly from ESPRESSO-MV.
Improving DSMGA-II performance on hierarchical problems by introducing preservative back mixing

Inspired from the optimal mixing in the linkage tree gene-pool optimal mixing evolutionary algorithm, the dependent structure matrix genetic algorithm II (DSMGA-II) is one of the state-of-the-art model-building genetic algorithms. It obtains the patterns from successful restricted mixing and uses them during the back mixing (BM) to remove spurious linkage information such that the linkage information in need stands out. However, such techniques can be misled in hierarchical problems where the linkages in latter layers are mistreated as spurious in BM. There has been diversity preservation scheme developed for DSMGA-II by monitoring the entropy, but such technique impairs the exploration ability of DSMGA-II on plateau. To solve this issue, this paper proposes the preservative back mixing (PBM), a modified version of BM. Specifically, PBM features (i) the adaptive back mixing that automatically adjusts the power of BM and (ii) the linkage preservation that preserves the linkages among complementary patterns. This paper conducted several experiments to show that, PBM improves DSMGA-II greatly on hierarchical problems including the hierarchical trap and hierarchical exclusive-or while the NFE performances on other problems are not compromised by much.
Towards landscape-aware parameter tuning for the (1 + ( <i>λ, λ</i> )) genetic algorithm for permutations

The choice of parameter values greatly affects the performance of evolutionary algorithms. Many current parameter tuning approaches require multiple runs of the tuned algorithm, which makes it hard to use them in budget-constrained environments. Recently, an approach to parameter tuning on binary string problems was proposed, which uses machine learning and fitness landscape analysis to transfer knowledge on good parameter choices between similar problem instances. This approach allows using the performance data obtained on simple benchmark problems with different fitness landscape features to quickly choose suitable parameters for a given problem instance.
Environments with local scopes for modules in genetic programming

Scoped environments are used in various programming languages to provide contexts within which a function's code can bind values and perform computations with them, without affecting bindings used by the rest of the program. The PushGP genetic programming system, which has produced state-of-the-art results in application areas, including software synthesis, allows programs to label sequences of instructions as modules. However, those modules can also modify bindings used by the rest of the program. This increases module coupling and might therefore make modular programs less evolvable than they otherwise could be. To rectify this, and to ensure that modules return single values, we implement scoped environments in PushGP, using a method that allows for the dynamic definition of arbitrary modules in a multi-type setting. We demonstrate that the use of scoped environments leads to an increase in the success rates for multiple software synthesis problems.
Implementing and evaluating parallel evolutionary algorithms in modern GPU computing libraries

In this paper, we describe FFEAT - a library for GPU-based implementation of evolutionary algorithms based on Torch. We discuss limitations of GPU computing and how they affect implementations of evolutionary algorithms and other population-based heuristics.
Analysis of neutral rewrite operator effects on arithmetic domain

We investigate the potential benefits of semantically neutral rewrites in Genetic Programming. These rewrites are expressed in the form of domain-specific axioms and building-block strategies of their application. We develop the strategies with the goal to improve best-of-run fitness convergence metric of evolutionary process. The approach is evaluated using benchmark problems defined on arithmetic domain.
TGPGAN

Before the advent of Generative Adversarial Networks (GANs), Evolutionary Computation approaches made up the majority of the state of the art for image generation. Adversarial models employing Deep Convolutional Neural Networks better fitted for GPU computing have been at this point more efficient. Nevertheless, motivated by recent successes in GPU-accelerated Genetic Programming (GP) and given the disposition of expression-based solutions towards image evolution, we believe in the prospect of using symbolic expressions as generators for GANs, instead of neural networks. In this paper, we propose a novel GAN model called TGPGAN, where the traditional convolutional generator network is replaced with a GP approach. The generator iteratively evolves a population of expressions that are then passed to the discriminator module along with real images for backpropagation. Our experimental results show that it is possible to achieve comparable results to a typical Deep Convolutional GAN while benefiting from the flexibility enabled by an expression-based genotype. Moreover, this work serves as a proof of concept for the evolution of symbolic expressions within adversarial models.
Quantum-enhanced selection operators for evolutionary algorithms

Genetic algorithms have unique properties which are useful when applied to black box optimization. Using selection, crossover, and mutation operators, candidate solutions may be obtained without the need to calculate a gradient. In this work, we study results obtained from using quantum-enhanced operators within the selection mechanism of a genetic algorithm. Our approach frames the selection process as a minimization of a binary quadratic model with which we encode fitness and distance between members of a population, and we leverage a quantum annealing system to sample low energy solutions for the selection mechanism. We benchmark these quantum-enhanced algorithms against classical algorithms over various black-box objective functions, including the OneMax function, and functions from the IOHProfiler library for black-box optimization. We observe a performance gain in average number of generations to convergence for the quantum-enhanced elitist selection operator in comparison to classical on the OneMax function. We also find that the quantum-enhanced selection operator with non-elitist selection outperform benchmarks on functions with fitness perturbation from the IOHProfiler library. Additionally, we find that in the case of elitist selection, the quantum-enhanced operators outperform classical benchmarks on functions with varying degrees of dummy variables and neutrality.
Facilitating the hybridization of parallel evolutionary algorithms in cluster computing environments

Evolutionary Algorithms (EAs) need domain-specific adaptations for achieving better results, tend to converge to suboptimal solutions and are computationally expensive when they are applied to complex and large-scale optimization problems. For addressing these challenges, hybridizing EAs with other algorithms and methods and parallelizing them in cluster computing environments represent essential solutions. In the present paper, a new software solution for supporting the hybridizations of parallel EAs is proposed. Unlike other software solutions for hybridizing EAs, the proposed software solution provides a flexible, generic and scalable mechanism for integrating any algorithmic approach like a Machine Learning (ML) algorithm to seed the initial population of parallel EAs. It is designed based on three modern software technologies, namely microservices, container virtualization and the publish/subscribe messaging paradigm. The applicability and generality of the presented software solution is tested by hybridizing the General Learning Evolutionary Algorithm and Method (GLEAM) with ML techniques for solving the problem of scheduling Distributed Energy Resources (DERs). The benchmarking tests are performed in a cluster computing environment. The obtained results show that the new software solution represents a successful approach to facilitate the hybridization of parallel EAs paving the road for future applications of EAs in several domains.
Towards optimizing neural networks' connectivity and architecture simultaneously with feature selection

We propose an extension of NeuroEvolution of Augmenting Topologies (NEAT), called Heterogeneous Activation Feature Deselection NEAT (HA-FD-NEAT) that evolves the weights and topography (architecture and activation functions) of Artificial Neural Networks (ANNs) while performing feature selection. The algorithm is evaluated against its ancestors: NEAT that evolves the weights and topology, FD-NEAT that evolves the weights and the topology while performing feature selection and HA-NEAT that evolves the topography and the weights. Performance is described by (i) median classification accuracy, (ii) computational efficiency (number of generations), (iii) network complexity (number of nodes and connections) and (iv) ability of selecting the relevant features. On the artificial 2-out-100 XOR problem, used as benchmark for feature selection, HA-FD-NEAT reaches a 100% accuracy in a few generations. It is significantly better than NEAT and HA-NEAT and exhibits the same performance as FD-NEAT. Even though HA-FD-NEAT needs to search in a bigger search space that includes weights, activation functions, topology and inputs, it is able to achieve the same performance as FD-NEAT. To conclude, the proposed method reduces the burden of human designers by determining the network's inputs, weights, topology and activation functions while achieving a very good performance.
A distributed particle swarm optimization algorithm for distributed clustering

With the rapid development of the Internet of Things, distributed clustering has emerged as an attractive research topic. Although evolutionary computation (EC) has been introduced as an effective approach to solve complex data clustering problems, few studies realize the use of EC for distributed clustering. In this paper, we intend to propose a distributed particle swarm optimization (PSO) for distributed clustering. The main idea of the work is to take advantage of the natural distributed computing feature of EC to solve distributed clustering. Since each site only knows a part of the data, the performance of distributed clustering algorithm usually deteriorates. To address this challenge, the proposed algorithm includes two main steps. Firstly, PSO is introduced in each local site to perform local data clustering. Secondly, to combine the local clustering results together, the K-means algorithm is used in the global site to generate the global clustering result. Experimental results demonstrate the overall performance superiority of the proposed algorithm compared with two other algorithms.
PreDive

The ever-present challenge in the domain of digital devices is how to test their behavior efficiently. We tackle the issue in two ways. We switch to an automated circuit design using Grammatical Evolution (GE). Additionally, we provide two diversity-based methodologies to improve testing efficiency. The first approach extracts a minimal number of test cases from subsets formed through clustering. Moreover, the way we perform clustering can easily be used for other domains as it is problem-agnostic. The other uses complete test set and introduces a novel fitness function hitPlex that incorporates a test case diversity measure to speed up the evolutionary process.
Introductory mathematical programming for EC

tutorial Share on Introductory mathematical programming for EC Author: Ofer M. Shir Tel-Hai College, ISRAEL Tel-Hai College, ISRAELView Profile Authors Info & Claims GECCO '22: Proceedings of the Genetic and Evolutionary Computation Conference CompanionJuly 2022 Pages 1067–1080https://doi.org/10.1145/3520304.3533630Published:19 July 2022Publication History 0citation22DownloadsMetricsTotal Citations0Total Downloads22Last 12 Months22Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Ant colony optimization for feature selection via a filter-randomized search heuristic

The combination of bio-inspired algorithms with the design of new search heuristics has allowed the development of innovative methods for combinatorial problems where the number of possible solutions is unaffordable. However, for years swarm intelligent methods have proven to be adequate to deal with these issues. Therefore, this paper proposes a new application of the Ant Colony Optimization (ACO) algorithm for Feature Selection (FS). Specifically, the proposed algorithm applies ACO for FS using a filter approach as a heuristic and makes use of a randomized search, resulting in a very innovative algorithm for variable reduction.