# Improved Q Network Auto-Scaling in Microservice Architecture
## 4 Experiment and Results  
To determine the performance of intelligent microservices, we first conducted a baseline experiment comparing different workloads in terms of relative performance. This section simulates the application's existing Horizontal Pod Autoscaler, the Autoscaler applying reinforcement learning to HPA, and the Autoscaler incorporating the proposed reinforcement learning algorithm. In the existing HPA work, the CPU allocation gradually increased for each new experiment, starting with the number of cores. To maximize each core, the response to the request was simulated by applying a load. The time using each core was measured while the maximum number of cores was limited to 10. The detailed experimental results are summarized in Table 2. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Name** & **Reference** & **Targets** & **Minopods** & **Maxpods** & **Replicas** & **Time** \\ \hline HPA & Deployment/HPA & 458\%/50\% & 1 & 10 & 1 & 1 min 14 s \\ \hline HPA & Deployment/HPA & 411\%/50\% & 1 & 10 & 2 & 1 min 32 s \\ \hline HPA & Deployment/HPA & 364\%/50\% & 1 & 10 & 3 & 1 min 47 s \\ \hline HPA & Deployment/HPA & 331\%/50\% & 1 & 10 & 4 & 2 min 2 s \\ \hline HPA & Deployment/HPA & 280\%/50\% & 1 & 10 & 5 & 2 min 35 s \\ \hline HPA & Deployment/HPA & 241\%/50\% & 1 & 10 & 6 & 2 min 44 s \\ \hline HPA & Deployment/HPA & 200\%/50\% & 1 & 10 & 7 & 2 min 59 s \\ \hline HPA & Deployment/HPA & 164\%/50\% & 1 & 10 & 8 & 3 min 17 s \\ \hline HPA & Deployment/HPA & 140\%/50\% & 1 & 10 & 9 & 3 min 31 s \\ \hline HPA & Deployment/HPA & 122\%/50\% & 1 & 10 & 10 & 4 min 24 s \\ \hline \hline \end{tabular}
\end{table} 
Table 2: HPA-based auto-scaling performance test results. Figure 4: Quad Q Network algorithm structure that derives Target \(Q\) values with the structures of the Quad Q Network with four Q Networks. Since there is no intelligent algorithm in the existing HPA work that is suitable for predicting overload with relation to the target share, the number of cores is opened one by one with the existing method. The work was confirmed to be gradually carried out. The number of cores started with one. All cores were opened in 4 min and 24 s. This method requires improvement because it is ineffective in responding to more traffic requests. To solve this problem, in Autoscaler work equipped with AI algorithms, the DQN algorithm has been embedded in the existing HPA method to make the most of each core. The learning procedure is as follows. The input data value was set as the percentage loaded on the CPU, which is the target value. And the compensation system aimed to reduce overload by expanding the Pod as quickly as possible. The target value was reduced to a minimum, and reaching the target quickly was set as the best compensation value. To prevent the use of the core without limitation, the CPU load was continuously checked. In addition, penalties were set up to prevent unnecessary use of pods. If CPU resources are needed when opening the core, +1 point is given. If CPU resources were optimized when opening the core, \(-1\) point is given. If it was difficult to judge, the learning was conducted by setting it to +0 point. The learning results are shown in Figure 5.  
The highest value was derived from the 41 st out of the 50 total number of learning, with a compensation value of 7, and overall, the compensation value is upward. The same load was given to the intelligent Autoscaler to perform simulations in a similar manner to the previous environment. The maximum number of cores was limited to 10. The time used for each core was measured. The detailed experimental results are listed in Table 3.  
The Autoscaler equipped with AI algorithms began with one core according to the target share. CPU allocation was rapidly increased for each new experiment. Since there was an intelligent algorithm suitable for predicting overload, the Autoscaler quickly opened the cores, rather than proceeding according to the one-by-one system. It was also confirmed that the work proceeded radically.