Optimizing Neural Networks with Gradient Lexicase Selection

1 INTRODUCTION . Modern data-driven learning algorithms generally define an optimization objective , e.g. , a fitness function in genetic algorithms ( Holland , 1992 ) or a loss function in deep learning ( LeCun et al. , 2015 ) , that computes the aggregate performance on the training data for the purpose of acquiring the best parameters in order to solve the task . Taking the image classification problem as an example , most recent approaches use Cross-Entropy loss with gradient descent ( Bottou , 2010 ) and backpropagation ( Rumelhart et al. , 1985 ) to train deep neural networks ( DNNs ) . Despite the success that advanced DNNs can reach human-level performance on the image recognition task ( Russakovsky et al. , 2015 ) , one potential drawback for such aggregated performance measurement is that the model may learn to seek ” compromises ” during the learning procedure . To give an example , consider a situation that may happen during the training phase for a batch of ten cases : nine cases are correctly predicted with high probabilities , but one is wrong . The aggregated loss may cause the model learning to compromise the wrong case for higher probabilities on other cases , which may lead the optimization process to be stuck at local optima . This potential optimization problem can be referred to as an uncompromising problem ( Helmuth et al. , 2014 ) , that is , a problem for which it is not acceptable for a solution to perform sub-optimally on any one of the cases in exchange for better performance on others . It becomes relevant in the context of deep learning where the optimization objective may introduce the behaviors of seeking compromises . In order to improve the generalization of DNNs , it is important to maintain the diversity and generality of the representations contributed by every training case . From the literature , uncompromising problems have been recently explored in genetic programming ( GP ) and genetic algorithms ( GAs ) for tasks such as program synthesis . Among many methods that aim to mitigate this problem , lexicase selection ( Helmuth et al. , 2014 ; Spector , 2012 ) and its variants ( Helmuth et al. , 2016 ; Helmuth & Spector , 2015 ; Helmuth et al. , 2014 ; Liskowski et al. , 2015 ) have been shown to outperform others ( Fieldsend & Moraglio , 2015 ; Galvan-Lopez et al. , 2013 ; Krawiec & Liskowski , 2015 ) in a number of applications . Instead of using an aggregated fitness function for parent selection , lexicase selection gradually eliminates candidates as it proceeds to look at how the population fares at each data point in a shuffled dataset , in which way it can bolster the diversity and generality in populations . Recent work also shows that lexicase selection can be used in rule-based learning systems ( Aenugu & Spector , 2019 ) , symbolic regression ( La Cava et al. , 2016 ) , machine learning ( La Cava & Moore , 2020b ; a ) , and evolutionary robotics ( Huizinga & Clune , 2018 ; La Cava & Moore , 2018 ) to improve model generalization , especially in situations of partial or missing data . In this work , we aim to explore the application of lexicase selection in the task of optimizing deep neural networks . Taking advantage of the commonly-used gradient descent and backpropagation methods , we introduce Gradient Lexicase Selection , an optimization method for training deep neural networks that not only benefit from the efficiency of gradient-based learning but also improves the generalization of the networks using the outline of lexicase selection method in an evolutionary fashion . We test the proposed method on the basic image classification task on three benchmark datasets ( CIFAR-10 ( Krizhevsky et al. , 2009 ) , CIFAR-100 ( Krizhevsky et al. , 2009 ) , and SVHN ( Netzer et al. , 2011 ) ) . Experimental results show that gradient lexicase selection manages to improve the performance of the DNNs consistently across six different popular architectures ( VGG ( Simonyan & Zisserman , 2015 ) , ResNet ( He et al. , 2016 ) , DenseNet ( Huang et al. , 2017 ) , MobileNetV2 ( Sandler et al. , 2018 ) , SENet ( Hu et al. , 2018 ) , EfficientNet ( Tan & Le , 2019 ) ) . In addition , we perform further ablation studies to analyze the effectiveness and robustness of the proposed method from various perspectives . We investigate the trade-offs between exploration and exploitation by analyzing the effects of population size and momentum , and also show that our algorithm can produce better representation diversity , which is advantageous to the generalization of DNNs . 2 BACKGROUND AND RELATED WORK . Preliminaries of Lexicase Selection Lexicase selection is initially proposed as a parent selection method in population-based stochastic search algorithms such as genetic programming ( Helmuth et al. , 2014 ; Spector , 2012 ) . Follow-up work has shown that lexicase selection can effectively improve behavioral diversity and the overall performance and on a variety of genetic programming problems ( Helmuth et al. , 2016 ; Helmuth & Spector , 2015 ; Helmuth et al. , 2014 ; Liskowski et al. , 2015 ) . The key idea in lexicase selection is that each selection event consider a randomly shuffled sequence of test cases . As a result , lexicase selection sometimes selects specialist individuals that perform poorly on average but perform better than many individuals on one or more other cases . A more detailed description of lexicase selection is appended in Sec . 7.1 . Unlike methods such as tournament selection that use a single fitness value and thus tend to always select generalist individuals that have good average performance , lexicase selection does not base selection on an aggregated measure of performance . Such a difference allows lexicase selection to maintain higher population diversity by prioritizing different parts of the dataset during each selection event through the ordering of the cases . It has been shown empirically on a number of program synthesis benchmark problems that lexicase selection substantially outperforms standard tournament selection and typically maintains higher levels of diversity ( Helmuth et al. , 2016 ) . In a more general context , lexicase selection can be used in any case where a selection procedure occurs with regard to performance assessment of multiple candidates with a set of test cases . Recent work also explores the usage of lexicase selection in rule-based learning systems ( Aenugu & Spector , 2019 ) , symbolic regression ( La Cava et al. , 2016 ) , machine learning ( La Cava & Moore , 2020b ; a ) , and evolutionary robotics ( Huizinga & Clune , 2018 ; La Cava & Moore , 2018 ) . In this work , we aim to explore the effectiveness of lexicase selection in the context of deep learning optimization from the perspective of improving the diversity of gradient-based representation learning for better generalization . While there are also other parent selection methods ( Fieldsend & Moraglio , 2015 ; Galvan-Lopez et al. , 2013 ; Krawiec & Liskowski , 2015 ) that have been proposed to achieve similar goals , in this work we focus on investigating the usage of lexicase selection in deep learning . Further discussion of comparisons between lexicase selection to other selection methods are out of the scope of this work . Deep Neuroevolution and Population-based Methods While backpropagation ( Rumelhart et al. , 1985 ) with gradient descent has been the most successful method in training DNNs with fixed-topology in the past few decades ( LeCun et al. , 2015 ) , there are also attempts to train DNNs through evolutionary algorithms ( EAs ) . Such et al . ( 2017 ) proposed a gradient-free method to evolve the network weights by using a simple genetic algorithm , and was able to evolve a relatively deep network ( with 4 million parameters ) and demonstrated competitive results on several reinforcement learning benchmark problems . Jaderberg et al . ( 2017 ) proposed population-based training to optimize both model and the hyperparameters . Cui et al . ( 2018 ) proposed to alternate between the SGD step and evolution step to improve the average fitness of the population . Ding & Spector ( 2021 ) demonstrate the usage of selection-inspired methods as regularization of DNNs . The most relevant recent work is Pawełczyk et al . ( 2018 ) , which did a pilot study on combining simple GA schema with gradient-based learning , where gradient training is used as part of the mutation process . Although the method was tested only on one dataset , the results were encouraging and offered some insights on a deeper combination of GAs and gradient learning . Following this trend , this paper aims to explore a more efficient evolutionary framework that takes advantage of both SGD and lexicase selection , to improve the network generalization by treating image recognition as an uncompromising problem . From a broader perspective that is also closely related to this work , there has been a surge of interest in methods for Neural Architecture Search ( NAS ) ( Elsken et al. , 2019 ) , where evolutionary algorithms gain high popularity . A majority of methods ( Floreano et al. , 2008 ; Liu et al. , 2017 ; Miikkulainen et al. , 2019 ; Real et al. , 2019 ; 2017 ; Stanley & Miikkulainen , 2002 ; Xie & Yuille , 2017 ) use EA to search neural network topologies and use backpropagation to optimize network weights , and some others ( Stanley & Miikkulainen , 2002 ) use EA to co-evolve topologies along with weights . While our method can be easily extended to the NAS problem , this work focuses on training various fixed-topology networks in order to make fair comparisons to show the significance of using lexicase selection to improve model generalization . 3 GRADIENT LEXICASE SELECTION . Our goal is to integrate the idea of lexicase selection to improve the generalization of DNNs , and at the same time to the greatest extent keep the efficiency and effectiveness of the popular gradientbased learning . We propose Gradient Lexicase Selection as an optimization framework to combine the strength of these two methods . An overview of the algorithm is demonstrated in Alg . 1 . It has two major components , Subset Gradient Descent ( SubGD ) and Lexicase Selection , which we describe in details in this section as follows . 3.1 EVOLUTION WITH SUBSET GRADIENT DESCENT ( SUBGD ) . First , we introduce the general evolutionary framework that uses a combination of stochastic gradient descent and evolution . Given the network topology , we first initialize all the parameters W0 as the initial parent weight . For each generation , given a population size of p , we generate p instances of the model as p offspring with the same weight as the parent weight , namely , W ( 0 ) = W ( 1 ) = · · · = W ( p−1 ) = W0 . We then perform mutation on these offspring and use lexicase selection to select the parent for the next generation . For each generation , we have p instances of the model starting with the exact same weights . Instead of random mutations such as adding gaussian noise as commonly used in neuroevolution , we propose a gradient-based mutation method called subset gradient descent ( SubGD ) . Given the whole training dataset Strain , we divided it into p subsets with random sampling , as S ( 0 ) train , S ( 1 ) train , · · · , S ( p−1 ) train . We then train each model instance accordingly on one of the subsets using the normal mini-batch stochastic gradient descent . The mutation is done when all the training data is consumed , which is one epoch in traditional deep learning . There are several advantages of the proposed mutation method . First , since all the offspring are trained with different non-overlapping training samples , they are likely to evolve diversely , especially when data augmentation is also included . Secondly , each off-spring is trained using gradient descent , meaning they will be optimized efficiently towards the objective , comparing to random mutation methods such as gaussian noise . Thirdly , if implemented with distributed training , all the offspring can be trained simultaneously to further reduce computation time . In general , the subset gradient descent aims to find a balance between exploration and exploitation toward the whole evolution process . Algorithm 1 : Gradient Lexicase Selection Data : • data - the whole training dataset • candidates - set of p instances of the DNN model initialized with the same parameters Result : • an optimized DNN model // K training epochs for epoch = 1 : K do subsets← p equal-size subsets obtained through random sampling from the entire data without replacement Use gradient descent and backpropagation to optimize each of the p candidates on each of the p subsets respectively cases← randomly shuffled sequence of data to be used in lexicase selection candidate ids← number set of { 0 , 1 , 2 , · · · , p− 1 } parent← None for case in cases do Evaluate all the candidates that have their ids in candidate ids on case . if at least one candidate makes the correct prediction then candidate ids← the subset of the current candidate ids that the corresponding candidates make the correct prediction else candidate ids← the subset of one randomly sampled id from the current candidate ids end if candidate ids contains only one single id then parent← the corresponding candidate in candidates with the remaining id in candidate ids break end end if parent is None then parent← a randomly selected candidate in candidates with the remaining id in candidate ids end candidates← set of p instances of the DNN model copied with the same parameters as parent end return parent

Summary: This paper extends the lexicase selection developed in the genetic programming community to apply the idea of lexicase selection to deep neural network training. In the proposed gradient lexicase selection, a parent network is selected based on the lexicase selection from several candidate networks trained using different subsets of the dataset. The authors apply the proposed method to several well-known convolutional neural networks (CNN) and show that the gradient lexicase selection can improve the performance of trained models.