# Automated Design of the Deep Neural Network Pipeline
###### Abstract  
Deep neural networks have proven to be effective in various domains, especially in natural language processing and image processing. However, one of the challenges associated with using deep neural networks includes the long design time and expertise needed to apply these neural networks to a particular domain. The research presented in this paper investigates the automation of the design of the deep neural network pipeline to overcome this challenge. The deep learning pipeline includes identifying the preprocessing needed, the feature engineering technique, the neural network to use and the parameters for the neural network. A selection pertubative hyper-heuristic (SPHH) is used to automate the design pipeline. The study also examines the reusability of the generated pipeline. The effectiveness of transfer learning on the generated designs is also investigated. The proposed approach is evaluated for text processing--namely, sentiment analysis and spam detection--and image processing--namely, maize disease detection and oral lesion detection. The study revealed that the automated design of the deep neural network pipeline produces just as good, and in some cases better, performance compared to the manual design, with the automated design requiring less design time than the manual design. In the majority of instances, the design was not reusable; however, transfer learning achieved positive transfer of designs, with the performance being just as good or better than when transfer learning was not used.  
automated design; deep neural network pipeline; transfer learning; sentiment analysis; spam detection; image segmentation; image classification 202212  
determines the preprocessing technique, augmentation technique, feature engineering technique, the neural network architecture and the neural network parameters. The approach was evaluated for sentiment analysis, spam detection, maize disease detection and oral lesion detection. The automated design was found to produce just as good, and in some instances better results than the manual design, with the manual design requiring more design time. The generated designs were not reusable. Transfer learning applied to the produced designs was found to be comparative, and in some instances improved performance. From the current state of the literature, no research has been conducted pertaining to the use of transfer learning in automated design, making this work the first study to examine the effect of transfer learning on the automated design of the deep neural network pipeline.  
Hence, the main contributions of this research are:  
* Automation of the deep neural network pipeline using a selection hyper-heuristic.
* An analysis of the reusability of the generated designs.
* An investigation into transfer learning applied to the generated designs.  
Section 2 provides an overview of the topics related to the research presented. Section 3.1 presents the selection hyper-heuristic employed for the automated design of the deep learning pipeline. The experimental setup employed to evaluate the SPHH for sentimental analysis, spam detection, maize disease detection and oral lesion detection is described in Section 3.3. The performance of SPHH for automated design, including reusability and transfer learning of the generated designs, is discussed in Section 4. A summary of the findings of the study and future extensions of this work is provided in Section 6.