# Improved Q Network Auto-Scaling in Microservice Architecture
## 2 Literature Review: Related Work
### Auto-Scaling  
The auto-scaling of cloud resources has emerged as a subject of intensive research in recent years due to the fact that customized resource allocation is one of the key characteristics of increased adaptation to cloud computing [10]. Other taxonomies have been proposed to classify numerous techniques at the algorithm level, with Threshold-based Rules, Queuing Theory, and RL being the main categories. Various studies have also been conducted with a focus on public cloud providers such as Amazon ECS using Auto-Scaling [9]. Even with simplified implementations, developing appropriate threshold-based rules requires expert knowledge and precise application understanding [9; 10; 11; 12; 13]. In practice, we use the Kubernetes scaling engine, which makes appropriate automatic scaling decisions to handle the actual variations in incoming requests. It can also be set up for resource over-provisioning by introducing certain management parameters for cloud application providers. Providing adequate application expansion represents one of the most important elements for cloud providers, as flexible and skillful resource distribution is a key concept of cloud computing [11; 12]. Auto-scaling has been classified and studied using basic theoretical models or methods. Threshold-based: Studies focusing on threshold-based expansion rules have improved vertical and horizontal elasticity performance in cloud systems of lightweight virtualization technology [14; 15; 16]. Specifically, one study examined a resource utilization-based automatic expansion system that demonstrates Kubernetes' VPA through its ability to dynamically adjust container allocation in the Kubernetes cluster without interruption [14]. Further, \begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Reference** \\ \hline \multicolumn{3}{c}{Various Middleware Frameworks of Containers [21]} \\ \multicolumn{3}{c}{Programming Sensors and IoT Devices (Small VMs Supported by Python Runtime Environments such as Spring and NodeJS) [22; 23; 24; 25]} \\ \multicolumn{3}{c}{Application Efficiency (Resource Virtualization Technology for Hardware Flexibility) [26]} \\ \multicolumn{3}{c}{Lightweight Virtualization Solution [27]} \\ \multicolumn{3}{c}{Cloud} \\ \multicolumn{3}{c}{Computing} & Virtualization Application to Lower Overhead [28] \\ \multicolumn{3}{c}{Definition of Container [29]} \\ \multicolumn{3}{c}{Strengths of Containers, Lightening Up [30]} \\ \multicolumn{3}{c}{The Connection between Lightweight Virtualization and Microservice [31]} \\ \multicolumn{3}{c}{Strategies to Strengthen The Fuzzy Distribution of Microphone Services [32]} \\ \multicolumn{3}{c}{Cluster Intelligence-Based Strategies [33]} \\ \hline \multicolumn{3}{c}{Deep Q Network That Performs Learning by Updating the \(Q\) Value [34; 35; 36]} \\ \multicolumn{3}{c}{Reinforcement} & Deep Q Network Proceeds with Effective Learning by Resetting the \(Q\) Value [37; 38; 39; 40] \\ \multicolumn{3}{c}{Learning} & Double Deep Q Network’s Idea [41; 42] \\ \multicolumn{3}{c}{Double Deep Q Network’s Improvements [43; 44]} \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Cont. based on IBM's principle of autonomous computing MAPE-K, an Elastic Docker study was conducted that autonomously supports the vertical elasticity of docker containers [15]. The above two papers integrated container movement and reviewed the possibility of vertical auto-scaling. Another study aimed to improve horizontal auto-scaling [16]. Reinforcement learning-based: One study combined two reinforcement learning (RL) approaches--Q-learning and state-compensation-state-behavior (SARSA) algorithms--with self-adaptive fuzzy logic controllers that lead to dynamic resource allocation to the virtual machine (VM) [17]. Another work provided a threshold-based solution for the automatic scaling of horizontal containers that uses Q-learning to adjust scaling thresholds [18].