Package: auto-cnn
AutoCNNThis project is an implementation of the paper"Automatically Designing CNN Architectures Using
Genetic Algorithm for Image Classification"How it worksThis is an algorithm that is able to full automatically find an optimal CNN (Convolutional Neural Network) architecture.There are two main building blocks to this algorithm:Skip LayerEach Skip Layer comprises of:The input first passes through:A convolution layerA batch normalizationA ReLU activation layerAnother convolution layerAnother batch normalization layer(all of the convolution layers have a kernel size of 3x3 and a stride of 1x1, the filter size is randomly chosen as a power of 2)The input also passes through a convolution of kernel and stride size 1x1 with a filter size being the same as the last convolution.
This "reshapes" the input in order to allow for element wise addingThe two outputs are combined in an add operation and then passed through a ReLU activation functionPooling LayerThis is either a Max Pooling or a Average Pooling layer, the kernel and the stride size are 2x2LayersLayer TypeLayer DocumentationConvolutiontf.keras.layers.Conv2DMaxPoolingtf.keras.layers.MaxPool2DAveragePoolingtf.keras.layers.AveragePooling2DActivationtf.keras.layers.ActivationAddtf.keras.layers.addBatchNormalizationtf.keras.layers.BatchNormalizationStepsTo do this the algorithm follows these steps:Create an random initial populationEvaluate the fitness of the population by training the CNNGenerating offsprings2 different CNN in the population are selected using aBinary Tournament SelectionmethodGiven a certain probability a crossover between the two parents might happenthe two CNNs are split into two and two new CNN are created by mixing the parent "genes"
-After all the new offsprings are created go through each of them and given a certain probability mutate the offspringa mutation are:add a skip layer: increases the complexity and depth of the networkadd a pooling layer: increases the depth but might decrease the complexity due to the nature of poolingremove a layer: reduce complexity and depthrandomize a layer: changes the parameters of a layer (i.e filter size, max or mean pooling)Evaluate the offspring fitnessFrom the offsprings and the parent population generate a new populationUntil N CNN have been selected:randomly select 2 CNN, add the one with the highest fitness to the listLook at if the CNN from the offspring and parent population was placed in the new populationif it is not replace the worst CNN by the bestRepeat step 2.Exampleimportosos.environ['TF_CPP_MIN_LOG_LEVEL']='3'# Removes Tensorflow debuggin ouputsimporttensorflowastftf.get_logger().setLevel('INFO')# Removes Tensorflow debugging ouputsfromauto_cnn.ganimportAutoCNNimportrandom# Sets the random seeds to make testing more consisentrandom.seed(42)tf.random.set_seed(42)defmnist_test():# Loads the data as test and train(x_train,y_train),(x_test,y_test)=tf.keras.datasets.mnist.load_data()# Puts the data in a dictionary for the algorithm to usedata={'x_train':x_train,'y_train':y_train,'x_test':x_test,'y_test':y_test}# Sets the wanted parametersa=AutoCNN(population_size=5,maximal_generation_number=4,dataset=data,epoch_number=5)# Runs the algorithm until the maximal_generation_number has been reachedbest_cnn=a.run()print(best_cnn)if__name__=='__main__':mnist_test()Given these parameters the structure that was chosen was this:CNN: 128-64Score (Test Accuracy): 0.9799000024795532ContributingIf you have any idea on improving the performance, adding more customizations or correcting mistakes, please make a pull request or create an issue. I'd be happy to accept any contributions!