# Comparison of Topologies Generated by Evolutionary Neural Architecture Search
## 3 Experiments
### Experiment 1--Application of NAS to n-Words Ordering  
In this study, the researchers aimed to apply NAS to Korean grammaticality judgement tasks, in order to explore the possibility of automating the design of language models. Sincethe word order of a language is the product of complex syntactic operations [20,21], a successful neural architecture search in linguistic data would suggest that NAS is capable of designing language models effectively. Although some previous studies have proposed the application of NAS in language modeling, the researchers in this study took it further by adding a novel dataset that contains Korean-specific linguistic operations, which added great complexity to the language patterns. The findings of the experiment demonstrated that NAS can provide an architecture for the Korean language, which is a remarkable result. It is interesting to note that the architecture suggested by NAS was an unprecedented structure that could not have been designed manually, highlighting the potential of NAS to automate the design of language models [8].  
The dataset used in this experiment consists of four-word level sentences in Korean that involve seven different syntactic categories, including noun phrases, verb phrases, prepositional phrases, adjective phrases, adverbs, complementizer phrases, and auxiliary phrases. These combinations result in a total of 2401 possibilities. To ensure the grammaticality of each combination, we consulted the [PERSON] corpus and two linguists, and the distribution of grammaticality is provided below. These data are unique because we have added two syntactic operations that were not previously visible in the Sejong corpus. An example of a sentence containing scrambling and ellipsis is presented in Table 1, where the underlying sentence may contain more than four words as some words can be elided and the order can be changed. Overall, this approach presents a novel way of testing language processing using unique data that add complexity to the patterns tested.  
The dataset's grammaticality is presented in Figure 1, which shows 113 grammatical sentences represented by circles. Figure 1 is a five-dimensional plot that includes color coding. The X-axis represents the first word slot, while the Y and Z axes represent the second and third word slots, respectively. The color spectrum indicates the fourth word slot. The O/X values in the plot represent grammaticality, which is the output of the artificial neural network. Although our experiment is limited to four-word sentences, the Korean language adds complexity to the sentence patterns due to its syntactic operations. For instance, the sentence in Table 1 is grammatically correct despite having more than four underlying words. The experiment took approximately 10 h to complete.  
The resulting topology of the experiment is displayed in Figure 2, which reveals a similar fork-like structure to the one found in the previous experiment. This result is somewhat unexpected, as the syntactic phenomena underlying the Korean sentences in the dataset are much more complex than the ones used in the previous experiment. The success of the NAS in designing an architecture for the Korean language model is significant, as it implies that NAS can automate language model designing even for languages with intricate syntactic operations. The discovery of an unprecedented structure in the Korean language model topology highlights the potential of NAS to discover new and improved architectures that can be used to complete various natural language processing tasks.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{1}{c}{**[PERSON]**} & **[PERSON]** & **[PERSON]** & **[PERSON]** \\ \hline [PERSON] & pretty & [PERSON] & [PERSON] & home & said \\ \hline
1 st input & 2 nd input & & 3 rd input & 4 th input \\ \hline \multicolumn{4}{c}{’At home, [PERSON] said to [PERSON] that [PERSON] is pretty.’} \\ \end{tabular}
\end{table}
Table 1: Sample Sentence of Korean Grammatical Judgement Task.