Reviewerly/exHarmony huggingface dataset:
Methods to Detect and Address Stall in Particle Swarm Optimization

Particle Swarm Optimization can experience stall in multi-modal search spaces. A stalled swarm is unable to converge and unable to find better solutions because all of its exploratory search solutions are rejected. Remedies to address the stall condition can benefit from knowing the particles that are still performing exploration, so we develop an efficient and accurate real-time, search space independent method to identify these particles. We confirm the benefit of identifying stalled particles through a modification designed for globally convex search spaces. We also discuss opportunities to differentiate search space landscapes and propose future research that can address non-globally convex search spaces.
A Feature and Parameter Selection Approach for Visual Domain Adaptation using Particle Swarm Optimization

To train a classifier on a specific domain, often called the target domain, we need labeled data. However, there might be non availability of the labeled data in this domain. In this scenario, we look for a related domain called the source domain, where availability of labeled data is abundant in number. The lack of availability of labeled data in the target domain poses a serious problem and several domain adaptation (DA) approaches have been put forward to cope up with this problem. Existing DA methods seek a subspace common between both the domains (source and target domains) where the distribution difference is minimal and perform manual parameter sensitivity tests to find apposite value of each parameter for their respective objective function. However, for distorted original data, obtaining a common subspace is a challenging task and condensing manual parameter sensitivity testing is also costly and a time-intensive process. To overcome these challenges, some DA methods consider particle swarm optimization (PSO) technique. However, none of the existing DA methods simultaneously tackle these challenges. Therefore, in this paper, we put forward a method called Feature and Parameter Selection approach for visual Domain Adaptation (FPSDA) to address these challenges. In FPSDA, a suitable subset of features across both the domains and an apposite value of each parameter are simultaneously chosen using a PSO approach. Moreover, to guide the PSO, the objective functions of Joint Geometrical and Statistical Alignment (JGSA) [1] method along with preserving original similarity of data is considered as an objective function for our proposed approach. Full Scale experiments on benchmark datasets for cross-domain adaptation verify that FPSDA performs better than many state-of-the-art classic machine learning and domain adaptation approaches.
XCSR with VAE using Gaussian Distribution Matching: From Point to Area Matching in Latent Space for Less-overlapped Rule Generation in Observation Space

This paper focuses on the matching mechanism of Learning Classifier System (LCS) in a continuous space and proposes a novel matching mechanism based on Gaussian distribution. This mechanism can match the “area” instead of the “point (one value)” in the continuous space unlike the conventional LCS such as XCSR (XCS with Continuous-Valued Inputs). Such an area matching contributes to generating the rules (called classifiers) with less-overlapped with other rules. Concretely, the proposed area matching mechanism employed in XCSR using VAE can generate appropriate classifiers for latent variables with high-dimensional inputs by VAE and create a human-interpretable observation space of human-interpretable classifiers. Since the latent variable in VAE is followed by Gaus-sian distribution, the following three matching mechanisms are compared: (i) the (single) point matching that selects the classifier which condition covers the mean of Gaussian distribution M; (ii) the multiple points matching that selects the classifier which condition covers the data sampled from Gaussian distribution (M, u); and (iii) the area matching that selects the classifier which condition roughly covers a certain area of Gaussian distribution (M, o). Through the intensive experiments on the high dimension maze problem, the following implications have been revealed: (1) the point matching in XCSR with VAE generates the ambiguous classifiers which conditions are overlapped with the other classifiers with the different action; (2) the sampling multiple points matching in XCSR with VAE has a potential of generating the less-overlapped classifiers by improving the data set through sampling. (3) the proposed area matching can generate the less-overlapped classifiers with the same learning steps, which corresponds to the time of the point matching.
Adversarial Robustness Assessment of NeuroEvolution Approaches

NeuroEvolution automates the generation of Artificial Neural Networks through the application of techniques from Evolutionary Computation. The main goal of these approaches is to build models that maximize predictive performance, some-times with an additional objective of minimizing computational complexity. Although the evolved models achieve competitive results performance-wise, their robustness to adversarial examples, which becomes a concern in security-critical scenarios, has received limited attention. In this paper, we evaluate the adversarial robustness of models found by two prominent Neu-roEvolution approaches on the CIFAR-10 image classification task: DENSER and NSGA-Net. Since the models are publicly available, we consider white-box untargeted attacks, where the perturbations are bounded by either the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$L_{2}$</tex> or the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$L_{\infty}$</tex> -norm. Similarly to manually-designed networks, our results show that when the evolved models are attacked with iterative methods, their accuracy usually drops to, or close to, zero under both distance metrics. The DENSER model is an exception to this trend, showing some resistance under the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$L_{2}$</tex> threat model, where its accuracy only drops from 93.70% to 18.10% even with iterative attacks. Additionally, we analyzed the impact of pre-processing applied to the data before the first layer of the network. Our observations suggest that some of these techniques can exacerbate the perturbations added to the original inputs, potentially harming robustness. Thus, this choice should not be neglected when automatically designing networks for applications where adversarial attacks are prone to occur.
Measuring the Effects of Increasing Dimensionality on Fitness-Based Selection and Failed Exploration

The rate of Successful Exploration is related to the proportion of search solutions from fitter attraction basins that are fitter than the current reference solution. A reference solution that moves closer to its local optimum (i.e. experiences exploitation) will reduce the proportion of these fitter solutions, and this can lead to decreased rates of Successful Exploration/increased rates of Failed Exploration. This effect of Fitness-Based Selection is studied in Particle Swarm Optimization and Differential Evolution with increasing dimensionality of the search space. It is shown that increasing rates of Failed Exploration represent another aspect of the Curse of Dimensionality that needs to be addressed by metaheuristic design.
An adaptive variant of jSO with multiple crossover strategies employing Eigen transformation

In this paper, new strategy options are developed for the adaptive jSO algorithm. The proposed variant of jSO is based on the competition of a binomial and exponential crossover. Moreover, an Eigen transformation approach is employed in the selected crossover with a given probability. The proposed variant of jSO is applied to the CEC 2022 benchmark set, which contains 12 functions with dimensionality <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$D=10$</tex> , 20. The proposed algorithm found the optima values in seven problems out of 24. When comparing the new variant of jSO with the original jSO algorithm, nine functions were improved, where two of them significantly.
Evolving Neural Networks for a Generalized Divide the Dollar Game

Divide the dollar is a simpler version of a game invented by John Nash to study the bargaining problem. The generalized divide the dollar game is an n-player version. Evolutionary algorithms can be used to evolve players for this game, but it has been previously shown representation has a profound effect on the success of the evolutionary search. Representation defines both the genome and the move (search) operator used by the evolutionary algorithm. This study investigates how well two representations for a 3-player generalized divide the dollar game, one using a differential evolution move operator and the other a CMA-ES move operator, can find good players implemented as neural networks. Our results indicate both representations can evolve very good player trios, but the CMA-ES representation tends to evolve fairer players.
Multi-Objective Roadmap Optimization for Multiagent Navigation

In this paper, we investigate multi-objective opti-mization of roadmaps for multi-robot path planning. We propose a new representation for roadmaps based on polygons and explore its potentials on various scenarios. In addition, we define three objective functions to estimate the suitability of each roadmap for navigation, and propose a modification of the well-known NSGA-II algorithm to optimize the roadmaps. In our experiments, we compare the quality of the proposed optimized roadmaps with those based on regular grids. The results show that in complex environments with obstacles, the optimized roadmaps perform much more efficient than those on regular grids. In addition, the performance of the optimization can be significantly improved by using the regular grids to initialize the optimization process.
Managing Diversity and Many Objectives in Evolutionary Design

A new approach to evolving a diversity of high-quality solutions for problems having many objectives is presented. Mouret and Clune's MAP-Elites algorithm has been proposed as a way to evolve an assortment of diverse solutions to a problem. We extend MAP-Elites in a number of ways. We introduce into MAP-elites the many-objective strategy called sum-of-ranks, which enables problems with many objectives (4 and more) to be considered in the MAP. We also enhance MAP-Elites by extending it with multiple solutions per cell (the original MAP-Elites saves only a single solution per cell). Different ways of selecting cell members for reproduction are also considered. We test the new MAP-Elites strategies on the evolutionary art application of image generation. Using procedural textures, genetic programming is used with upwards of 15 light-weight image features to guide fitness. The goal is to evolve images that share image features with a given target image. Our experiments show that the new MAP-Elites algorithms produce a large number of diverse solutions, which can be competitive in quality to those from standard GP runs.
A replacement scheme based on dynamic penalization for controlling the diversity of the population in Genetic Programming

Algorithms relating the amount of population's diversity to the elapsed period of execution have yielded important improvements. Particularly, schemes with a gradual shift from exploration to exploitation have excelled in several areas of Evolutionary Algorithms. A fairly recent method that applies this design principle is the Genetic Programming variant with Dynamic Management of Diversity (GP-DMD). GP-DMD applies a diversity-based replacement strategy that takes into account a user-defined function or policy that sets the amount of diversity desired in the population. Despite the improvements attained by GP-DMD, it is unable to precisely follow the user-defined policy in some cases. This calls into question its ability to perform a gradual shift from exploration to exploitation and hinders its extension to develop more complex dynamic and adaptive algorithms. This paper proposes the Genetic Programming variant with Controlled Dynamic Management of Diversity (GP-CDMD) which incorporates a novel replacement strategy that aims to improve its tracking capabilities. This is done through a probabilistic selection that takes into account the desired amount of diversity to restrict the diversity of the population. Results in the Symbolic Regression benchmark problem show a significant improvement in the tracking error, which results in features of the dynamics of the population that are more similar to the expected ones. This achievement facilitates the design of more complex diversity-based dynamic and adaptive optimizers and allows for better analyses on the implications of diversity in the GP area.
AGAVaPS - Adaptive Genetic Algorithm with Varying Population Size

Recently there is great interest in optimization, especially on meta-heuristic algorithms. Many works have proposed improvements for these algorithms for general and specific applications. In this paper the Adaptive Genetic Algorithm with Varying Population Size (AGAVaPS) is proposed, an improvement of Genetic Algorithm. On the AGAVaPS each solution has their own mutation rate and number of iterations that the solution will be in the population. The proposed optimizer is tested against six other well established optimizers on the CEC2017 single objective optimization benchmark functions considering coverage of the search space and quality of solution obtained. It is also tested for feature selection and Bayesian network structural learning. The evolution of the population size over the iterations is also analysed. The results obtained show that the AGAVaPS has a very competitive performance in both, coverage and quality of solution.
A Fast Randomized Local Search for Low Budget Optimization in Black-Box Permutation Problems

Low budget black-box optimization is a relevant topic in many practical applications with expensive objective functions or tight real-time constraints. Recently, there has been a growing interest in addressing combinatorial permutation problems in a low budget and black-box scenario. In this context, most of the previously proposed algorithms learn a probabilistic model which guides the search by trying to somehow indicate the most effective areas of the permutation search space. However, the large size and the inherent discontinuity of the permutation space may lessen the effectiveness of this approach when a low, or very low, budget of evaluations is considered. Moving from this consideration, in this work we present a simpler elitist trajectory-based algorithm for low budget black-box optimization of permu-tation problems. The proposed algorithm, namely FAT-RLS, is based on three core ideas: a randomized local search scheme, an adaptive perturbation strength and the use of a tabu structure. A series of experiments held on commonly adopted benchmark problems clearly shows that FAT-RLS obtains better or compara-ble effectiveness with respect to the previous proposals. Moreover, its negligible computational overhead is of particular interest in mission critical situations where tight real-time constraints have to be matched.
A Genetic Programming-Based Hyper-Heuristic Approach for Multi-Objective Dynamic Workflow Scheduling in Cloud Environment

With the popularity of cloud computing, many organizations process their workflow tasks in cloud resources based on the Pay-As-Per-Use model. Dynamic Workflow Scheduling (DWS) aims to allocate dynamically arriving workflow tasks to cloud resources with optimal makespan, cost, load-balancing, etc. To timely allocate arriving tasks, heuristics have been used to solve the DWS problem in cloud environment. However, most of them are manually designed, considering a single objective, and use simple features to allocate resources to workflow tasks. In practice, multiple objectives should be considered to provide trade-off heuristics for users to choose from. In this paper, we propose a genetic programming hyper-heuristic (GPHH) approach to automatically generate multiple heuristics for multi-objective DWS. Our experimental evaluation using benchmark datasets demonstrates the effectiveness of our proposed GPHH approach.
A conjugated evolutionary algorithm for hyperparameter optimization

With the recent upsurge in the use of deep learning and other computationally expensive machine learning models, hyperparameter optimization has become a quite important and widely researched area of study. Genetic algorithms, a subclass of evolutionary algorithms, have proven to be an effective approach and have been widely used in recent years. However, efficiently exploring the domain of possible solutions remains a challenging, and often computationally-expensive task. In this paper, we present a novel and efficient hyperparameter optimization strategy based on a genetic algorithms variant: Biased Random-key Genetic Algorithms (BRKGA). One of the main challenges of BRKGA is its limited capacity to explore the domain surrounding a particular individual. Although good genes will be preserved by its bias property, these genes are copied as they are, and even if a better solution exists in the close neighborhood of a particular gene it might never be explored. We tackle this problem by adding an exploitation component at the end of every evolutionary step, further exploring the hyperparameter domain. Several computational experiments on eight different publicly available datasets were performed to assess the effectiveness of the proposed approach and to prove it is a significant improvement over its predecessor. The results show that our proposed method outperforms, in terms of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$F_{1}$</tex> score of the resulting Artificial Neural Network, not only BRKGA but also other commonly used methods in most of the test cases.
Hybrid Tourism Recommendation System: A Multi-Objective Perspective

A smart recommendation method can greatly improve tourists' travel experience, and it is an important task for tourism recommendation systems to intelligently suggest scenic spots for tourists according to their historical visit records. Currently, the collaborative filtering and deep neural network-based methods occupy the mainstream of tourism recommendation systems. Although each type of recommendation methods is superior over the others in terms of different aspects, the performance of a single recommendation method is limited. In order to inherit the advantages of different types of recommendation methods, this work suggests a hybrid method for assembling multiple methods for tourism recommendation. Based on the scenic spots obtained by multiple recommendation methods, the proposed hybrid method uses two novel objectives to evaluate each scenic spot, and identifies the best <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$K$</tex> scenic spots via the techniques used in evolutionary multi-objective optimization. In comparison to existing recommendation methods and hybrid methods, the proposed hybrid method exhibits better performance on two public tourism datasets and a new dataset created based on the tourism information of Huangshan City.
Hyperparameters Adaptive Sharing Based on Transfer Learning for Scalable GPs

Gaussian processes (GPs) are a kind of non-parametric Bayesian approach. They are widely used as surrogate models in data-driven optimization to approximate the exact functions. However, the cubic computation complexity is involved in building GPs. This paper proposes hyperparameters adaptive sharing based on transfer learning for scalable GPs to address the limitation. In this method, the hyperparameters across source tasks are adaptively shared to the target task by the linear predictor. This method can reduce the computation cost of building GPs without losing capability based on experimental analyses. The method's effectiveness is demonstrated on a set of benchmark problems.
A comparative study of convolutional network models for the classification of abnormalities in mammograms

Breast cancer is the most common cancer and one of the main causes of death in women. Early diagnosis of breast cancer is essential to ensure a high chance of survival for the affected women. Computer-aided detection (CAD) systems based on convolutional neural networks (CNN) could assist in the classification of abnormalities such as masses and calcifications. In this paper, several convolutional network models for the automatic classification of pathology in mammograms are analyzed. As well as different preprocessing and tuning techniques, such as data augmentation, hyperparameter tuning, and fine-tuning are used to train the models. Finally, these models are validated on various publicly available benchmark datasets.
Multistage Diagnosis of Alzheimer’s Disease Based on Slice Attention Network

Alzheimer’s disease (AD) is a latent progressive neurodegenerative disease. Early detection can prevent further damage to patient’s health. We proposed a 3D abnormal perception depth residual network based on the squeeze and excitation module (RSE) and recurrent slice attention module (RSA). In our model, RSE captures the importance of different channels by integrating extrusion and excitation modules into residual blocks, while RSA aims to model 3D MRI images as slice sequences to capture the long-term dependence of different slices in different directions. Our model combine the context information of the abnormal area with local and spatial information. Experimental results show that the accuracy of our method is 87.5%, which is better than the most advanced model in terms of normal cognition (NC), early mild cognitive impairment (EMCI), late mild cognitive impairment (LMCI) and Alzheimer’s disease (AD) on the ADNI dataset. The CAM visualization results also show that our method can successfully highlight the most contributing regions of 3D MRI images.
Adaptive Online Domain Incremental Continual Learning

Continual Learning (CL) problems pose significant challenges for Neural Network (NN)s. Online Domain Incremental Continual Learning (ODI-CL) refers to situations where the data distribution may change from one task to another. These changes can severely affect the learned model, focusing too much on previous data and failing to properly learn and represent new concepts. Conversely, if a model constantly forgets previously learned knowledge, it may be deemed too unstable and unsuitable. This work proposes Online Domain Incremental Pool (ODIP), a novel method to cope with catastrophic forgetting. ODIP also employs automatic concept drift detection and does not require task ids during training. ODIP maintains a pool of learners, freezing and storing the best one after training on each task. An additional Task Predictor (TP) is trained to select the most appropriate NN from the frozen pool for prediction. We compare ODIP against regularization methods and observe that it yields competitive predictive performance.