Optimizing Neural Networks with Gradient Lexicase Selection

1 INTRODUCTION . Modern data-driven learning algorithms generally define an optimization objective , e.g. , a fitness function in genetic algorithms ( Holland , 1992 ) or a loss function in deep learning ( LeCun et al. , 2015 ) , that computes the aggregate performance on the training data for the purpose of acquiring the best parameters in order to solve the task . Taking the image classification problem as an example , most recent approaches use Cross-Entropy loss with gradient descent ( Bottou , 2010 ) and backpropagation ( Rumelhart et al. , 1985 ) to train deep neural networks ( DNNs ) . Despite the success that advanced DNNs can reach human-level performance on the image recognition task ( Russakovsky et al. , 2015 ) , one potential drawback for such aggregated performance measurement is that the model may learn to seek ” compromises ” during the learning procedure . To give an example , consider a situation that may happen during the training phase for a batch of ten cases : nine cases are correctly predicted with high probabilities , but one is wrong . The aggregated loss may cause the model learning to compromise the wrong case for higher probabilities on other cases , which may lead the optimization process to be stuck at local optima . This potential optimization problem can be referred to as an uncompromising problem ( Helmuth et al. , 2014 ) , that is , a problem for which it is not acceptable for a solution to perform sub-optimally on any one of the cases in exchange for better performance on others . It becomes relevant in the context of deep learning where the optimization objective may introduce the behaviors of seeking compromises . In order to improve the generalization of DNNs , it is important to maintain the diversity and generality of the representations contributed by every training case . From the literature , uncompromising problems have been recently explored in genetic programming ( GP ) and genetic algorithms ( GAs ) for tasks such as program synthesis . Among many methods that aim to mitigate this problem , lexicase selection ( Helmuth et al. , 2014 ; Spector , 2012 ) and its variants ( Helmuth et al. , 2016 ; Helmuth & Spector , 2015 ; Helmuth et al. , 2014 ; Liskowski et al. , 2015 ) have been shown to outperform others ( Fieldsend & Moraglio , 2015 ; Galvan-Lopez et al. , 2013 ; Krawiec & Liskowski , 2015 ) in a number of applications . Instead of using an aggregated fitness function for parent selection , lexicase selection gradually eliminates candidates as it proceeds to look at how the population fares at each data point in a shuffled dataset , in which way it can bolster the diversity and generality in populations . Recent work also shows that lexicase selection can be used in rule-based learning systems ( Aenugu & Spector , 2019 ) , symbolic regression ( La Cava et al. , 2016 ) , machine learning ( La Cava & Moore , 2020b ; a ) , and evolutionary robotics ( Huizinga & Clune , 2018 ; La Cava & Moore , 2018 ) to improve model generalization , especially in situations of partial or missing data . In this work , we aim to explore the application of lexicase selection in the task of optimizing deep neural networks . Taking advantage of the commonly-used gradient descent and backpropagation methods , we introduce Gradient Lexicase Selection , an optimization method for training deep neural networks that not only benefit from the efficiency of gradient-based learning but also improves the generalization of the networks using the outline of lexicase selection method in an evolutionary fashion . We test the proposed method on the basic image classification task on three benchmark datasets ( CIFAR-10 ( Krizhevsky et al. , 2009 ) , CIFAR-100 ( Krizhevsky et al. , 2009 ) , and SVHN ( Netzer et al. , 2011 ) ) . Experimental results show that gradient lexicase selection manages to improve the performance of the DNNs consistently across six different popular architectures ( VGG ( Simonyan & Zisserman , 2015 ) , ResNet ( He et al. , 2016 ) , DenseNet ( Huang et al. , 2017 ) , MobileNetV2 ( Sandler et al. , 2018 ) , SENet ( Hu et al. , 2018 ) , EfficientNet ( Tan & Le , 2019 ) ) . In addition , we perform further ablation studies to analyze the effectiveness and robustness of the proposed method from various perspectives . We investigate the trade-offs between exploration and exploitation by analyzing the effects of population size and momentum , and also show that our algorithm can produce better representation diversity , which is advantageous to the generalization of DNNs . 2 BACKGROUND AND RELATED WORK . Preliminaries of Lexicase Selection Lexicase selection is initially proposed as a parent selection method in population-based stochastic search algorithms such as genetic programming ( Helmuth et al. , 2014 ; Spector , 2012 ) . Follow-up work has shown that lexicase selection can effectively improve behavioral diversity and the overall performance and on a variety of genetic programming problems ( Helmuth et al. , 2016 ; Helmuth & Spector , 2015 ; Helmuth et al. , 2014 ; Liskowski et al. , 2015 ) . The key idea in lexicase selection is that each selection event consider a randomly shuffled sequence of test cases . As a result , lexicase selection sometimes selects specialist individuals that perform poorly on average but perform better than many individuals on one or more other cases . A more detailed description of lexicase selection is appended in Sec . 7.1 . Unlike methods such as tournament selection that use a single fitness value and thus tend to always select generalist individuals that have good average performance , lexicase selection does not base selection on an aggregated measure of performance . Such a difference allows lexicase selection to maintain higher population diversity by prioritizing different parts of the dataset during each selection event through the ordering of the cases . It has been shown empirically on a number of program synthesis benchmark problems that lexicase selection substantially outperforms standard tournament selection and typically maintains higher levels of diversity ( Helmuth et al. , 2016 ) . In a more general context , lexicase selection can be used in any case where a selection procedure occurs with regard to performance assessment of multiple candidates with a set of test cases . Recent work also explores the usage of lexicase selection in rule-based learning systems ( Aenugu & Spector , 2019 ) , symbolic regression ( La Cava et al. , 2016 ) , machine learning ( La Cava & Moore , 2020b ; a ) , and evolutionary robotics ( Huizinga & Clune , 2018 ; La Cava & Moore , 2018 ) . In this work , we aim to explore the effectiveness of lexicase selection in the context of deep learning optimization from the perspective of improving the diversity of gradient-based representation learning for better generalization . While there are also other parent selection methods ( Fieldsend & Moraglio , 2015 ; Galvan-Lopez et al. , 2013 ; Krawiec & Liskowski , 2015 ) that have been proposed to achieve similar goals , in this work we focus on investigating the usage of lexicase selection in deep learning . Further discussion of comparisons between lexicase selection to other selection methods are out of the scope of this work . Deep Neuroevolution and Population-based Methods While backpropagation ( Rumelhart et al. , 1985 ) with gradient descent has been the most successful method in training DNNs with fixed-topology in the past few decades ( LeCun et al. , 2015 ) , there are also attempts to train DNNs through evolutionary algorithms ( EAs ) . Such et al . ( 2017 ) proposed a gradient-free method to evolve the network weights by using a simple genetic algorithm , and was able to evolve a relatively deep network ( with 4 million parameters ) and demonstrated competitive results on several reinforcement learning benchmark problems . Jaderberg et al . ( 2017 ) proposed population-based training to optimize both model and the hyperparameters . Cui et al . ( 2018 ) proposed to alternate between the SGD step and evolution step to improve the average fitness of the population . Ding & Spector ( 2021 ) demonstrate the usage of selection-inspired methods as regularization of DNNs . The most relevant recent work is Pawełczyk et al . ( 2018 ) , which did a pilot study on combining simple GA schema with gradient-based learning , where gradient training is used as part of the mutation process . Although the method was tested only on one dataset , the results were encouraging and offered some insights on a deeper combination of GAs and gradient learning . Following this trend , this paper aims to explore a more efficient evolutionary framework that takes advantage of both SGD and lexicase selection , to improve the network generalization by treating image recognition as an uncompromising problem . From a broader perspective that is also closely related to this work , there has been a surge of interest in methods for Neural Architecture Search ( NAS ) ( Elsken et al. , 2019 ) , where evolutionary algorithms gain high popularity . A majority of methods ( Floreano et al. , 2008 ; Liu et al. , 2017 ; Miikkulainen et al. , 2019 ; Real et al. , 2019 ; 2017 ; Stanley & Miikkulainen , 2002 ; Xie & Yuille , 2017 ) use EA to search neural network topologies and use backpropagation to optimize network weights , and some others ( Stanley & Miikkulainen , 2002 ) use EA to co-evolve topologies along with weights . While our method can be easily extended to the NAS problem , this work focuses on training various fixed-topology networks in order to make fair comparisons to show the significance of using lexicase selection to improve model generalization . 3 GRADIENT LEXICASE SELECTION . Our goal is to integrate the idea of lexicase selection to improve the generalization of DNNs , and at the same time to the greatest extent keep the efficiency and effectiveness of the popular gradientbased learning . We propose Gradient Lexicase Selection as an optimization framework to combine the strength of these two methods . An overview of the algorithm is demonstrated in Alg . 1 . It has two major components , Subset Gradient Descent ( SubGD ) and Lexicase Selection , which we describe in details in this section as follows . 3.1 EVOLUTION WITH SUBSET GRADIENT DESCENT ( SUBGD ) . First , we introduce the general evolutionary framework that uses a combination of stochastic gradient descent and evolution . Given the network topology , we first initialize all the parameters W0 as the initial parent weight . For each generation , given a population size of p , we generate p instances of the model as p offspring with the same weight as the parent weight , namely , W ( 0 ) = W ( 1 ) = · · · = W ( p−1 ) = W0 . We then perform mutation on these offspring and use lexicase selection to select the parent for the next generation . For each generation , we have p instances of the model starting with the exact same weights . Instead of random mutations such as adding gaussian noise as commonly used in neuroevolution , we propose a gradient-based mutation method called subset gradient descent ( SubGD ) . Given the whole training dataset Strain , we divided it into p subsets with random sampling , as S ( 0 ) train , S ( 1 ) train , · · · , S ( p−1 ) train . We then train each model instance accordingly on one of the subsets using the normal mini-batch stochastic gradient descent . The mutation is done when all the training data is consumed , which is one epoch in traditional deep learning . There are several advantages of the proposed mutation method . First , since all the offspring are trained with different non-overlapping training samples , they are likely to evolve diversely , especially when data augmentation is also included . Secondly , each off-spring is trained using gradient descent , meaning they will be optimized efficiently towards the objective , comparing to random mutation methods such as gaussian noise . Thirdly , if implemented with distributed training , all the offspring can be trained simultaneously to further reduce computation time . In general , the subset gradient descent aims to find a balance between exploration and exploitation toward the whole evolution process . Algorithm 1 : Gradient Lexicase Selection Data : • data - the whole training dataset • candidates - set of p instances of the DNN model initialized with the same parameters Result : • an optimized DNN model // K training epochs for epoch = 1 : K do subsets← p equal-size subsets obtained through random sampling from the entire data without replacement Use gradient descent and backpropagation to optimize each of the p candidates on each of the p subsets respectively cases← randomly shuffled sequence of data to be used in lexicase selection candidate ids← number set of { 0 , 1 , 2 , · · · , p− 1 } parent← None for case in cases do Evaluate all the candidates that have their ids in candidate ids on case . if at least one candidate makes the correct prediction then candidate ids← the subset of the current candidate ids that the corresponding candidates make the correct prediction else candidate ids← the subset of one randomly sampled id from the current candidate ids end if candidate ids contains only one single id then parent← the corresponding candidate in candidates with the remaining id in candidate ids break end end if parent is None then parent← a randomly selected candidate in candidates with the remaining id in candidate ids end candidates← set of p instances of the DNN model copied with the same parameters as parent end return parent

Summary: This paper presents a neural network training strategy that leverages lexicase selection and evolutionary algorithms. Specifically, the training strategy involves two layers of loops to iteratively update model&select candidates, which searches the best optimization direction through greedy search. Experiments are conducted on two image datasets and show the proposed training strategy allows the learned model to reach better performance (accuracy). Further experiments investigate how hyperparameters would impact the final performance.