# Comparison of Topologies Generated by Evolutionary Neural Architecture Search
## 1 Introduction  
Neural Architecture Search (NAS) has gained popularity in recent years as a powerful tool for the automated design of deep learning architectures. In this paper, we compare the performance of the resulting topologies generated by evolutionary NAS for two different tasks: grammaticality judgment for n-word ordering [1], and the MNIST dataset. Despite the differences in the tasks, we find that NAS leads to consistent loss reduction using the same tuning method, which cannot be easily designed by human experts. Specifically, we report the emergence of a distinctive fork-like structure that multiplies links between layers. To the best of our knowledge, this fork-like structure has not been previously reported in the literature and may have implications for various deep learning tasks [1; 2]. Previous research on the application of NAS to linguistic data has focused on improving its accuracy compared to existing language models [3; 4]. In this paper, we acknowledge that the fork-like structure does not necessarily outperform the traditional technique utilized in each task. Therefore, we have limited the scope of our analysis to the novelty of the fork-like structure and its potential application in various deep learning tasks. Neural Architecture Search (NAS) is a powerful technique for automating the design of Deep Neural Networks (DNNs) for a given task [5; 6]. It encompasses a broad set of techniques that automatically design the architecture of a DNN, including selecting the appropriate layers, their connections, and performing hyperparameter tuning [6]. However, NAS faces several challenges in terms of designing effective network architectures. Firstly, there is a challenge in selecting the appropriate network architecture to suit the specific task at hand. Secondly, there is a challenge in determining how to effectively integrate different components into the network topology, such as selecting appropriateactivation functions, designing attention mechanisms, and including skip connections. Finally, there is a challenge in finding the optimal hyperparameters to fine-tune the network's performance [7]. Each of these factors must be optimized and studied separately for each new task, which can be time consuming and computationally expensive. Nonetheless, NAS has shown tremendous potential in automating the design process and achieving state-of-the-art performance in various domains. This paper presents a modified version of Neural Architecture Search (NAS) using the Genetic Algorithm (GA), with chromosome non-disjunction as a novel deep neural network structuring methodology, as recently proposed in [8; 9]. Our approach leverages the power of Evolutionary Algorithms (EA) to automatically design the full network topology. The key advantage of our system is the utilization of the chromosome non-disjunction operation, which preserves information from both parents without losing any crucial data. This enables the fine-tuning of the previous neural network architecture, which is not possible with traditional cross-over operations, as they typically lose some information while obtaining new data. This paper presents a comparison of the resulting topologies generated by Neural Architecture Search (NAS) in various deep learning tasks, including grammaticality tasks for n-word ordering [1], and the MINIST dataset. We found that a particular fork-like structure used in the system provides an efficient solution to these tasks. Additionally, our comparison reveals that the language model for the grammaticality tasks created by NAS is somewhat similar to the model created by the MINIST dataset. The main contribution of this paper is that it provides an analysis of the fork-like structure found in these experiments. While our analysis is still in its preliminary stage, we observed that the fork-like structure reduces the loss during training, which suggests that it has potential as a useful tool in deep learning tasks. This paper consists of five sections. Section 2 introduces NAS with the genetic algorithm.