###### Abstract  
In the realm of machine learning, the optimization of hyperparameters and the design of neural architectures entail laborious and time-intensive endeavors. To address these challenges, considerable research effort has been directed towards Automated Machine Learning (AutoML), with a focus on enhancing these inherent inefficiencies. A pivotal facet of this pursuit is Neural Architecture Search (NAS), a domain dedicated to the automated formulation of neural network architectures. Given the pronounced impact of network architecture on neural network performance, NAS techniques strive to identify architectures that can manifest optimal performance outcomes. A prominent algorithm in this area is Differentiable Architecture Search (DARTS), which transforms discrete search spaces into continuous counterparts using gradient-based methodologies, thereby surpassing prior NAS methodologies. Notwithstanding DARTS' achievements, a discrepancy between discrete and continuously encoded architectures persists. To ameliorate this disparity, we propose TA-DARTS in this study--a temperature annealing technique applied to the Softmax function, utilized for encoding the continuous search space. By leveraging temperature values, architectural weights are judiciously adjusted to alleviate biases in the search process or to align resulting architectures more closely with discrete values. Our findings exhibit advancements over the original DARTS methodology, evidenced by a 0.07%p enhancement in validation accuracy and a 0.16%p improvement in test accuracy on the CIFAR-100 dataset. Through systematic experimentation on benchmark datasets, we establish the superiority of TA-DARTS over the original mixed operator, thereby underscoring its efficacy in automating neural architecture design. t +
Footnote †: journal: Applied  
Many approaches have been explored for efficient search of optimal neural architecture [2; 7; 8; 9]. One main difficulty of NAS is that previous algorithms have performed searches over discrete spaces. This is due to the fundamental problem of NAS algorithms. In NAS, one of the main components is an operator. There can be many different operators in neural architecture, and NAS algorithms evaluate, select and combine those operators for constructing optimal architecture. Combining different operators involves selecting an appropriate operator from a discrete operator pool, and thus, it is a search over a discrete search space. After the NAS problem was raised, evolutionary algorithms [2; 7] and reinforcement learning [8; 9] algorithms were proposed and achieved considerable performance improvement. Recently, the differentiable architecture search (DARTS) algorithm has been in the spotlight because it has reduced the NAS problem into a differential architecture search problem [10]. DARTS performs its training for the optimal operator by calculating the linear weighted summation of diverse operators, which comprise discrete search space. The weights of the operators constitute continuous space using the Softmax function, enabling backpropagation. In the realm of Neural Architecture Search, the gradient-based approach encompasses the automatic generation of neural architectures, including sequences of layers as well as optimal architectural choices tailored to specific datasets. Consequently, even configurations akin to residual networks [11] can be accommodated. To determine the optimal configuration for a skip connection, our method involves searching through connections linking preceding and subsequent layers. While alternate approaches to building search models may exist, our work follows the One-Shot NAS algorithm framework [12], which entails creating a supernet where results from all potential operators are aggregated. DARTS facilitates the determination of optimal operators for individual inter-layer connections by parameterizing them through a Softmax function, thereby ensuring the validity of updates through backpropagation.