# Optimizing Container Placement in Data Centers by Deep Reinforcement Learning
## 1 Introduction  
Data centers are facilities that house servers providing computing power. These servers are installed in rack units alongside network and security equipment that facilitates connectivity. In addition, power supply systems and temperature- and humidity-control facilities are essential for maintaining optimal server operation. Data centers are increasingly becoming an important component of digital infrastructure because of rapid advancements in cloud computing [1]. With the continuous emergence of artificial intelligence (AI)-driven applications, this demand is expected to increase even further. The operating costs of data centers increase exponentially with the number of servers used. Purchasing additional servers increases direct server costs and also escalates expenses related to network and security equipment, power consumption, and cooling systems. Particularly, energy efficiency is an important issue in data centers [2, 3]. Therefore, reducing the number of active servers is important for reducing overall data center costs. To enhance efficiency, data centers are rapidly transitioning to cloud-based architectures using virtual machines and containers that leverage virtualization technology. Virtualization is considered one of the pioneering concepts initiating modern clouds [4]. Virtual machine technology is the execution environment used in cloud computing, whereas a container is a lightweight virtualization technology that provides an isolated virtual environment [5]. Given the increasing need for agility, lightweight containerized servers are gaining popularity [6]. However, containers are often not deployed in an optimized manner. Currently, data centers tend to scale resources up and down during periods of high and low traffic, respectively, leading to inefficient resource utilization, particularly of physical servers. Optimizing container placement on physical servers can mitigate resource wastage. By strategically allocating containers, unused servers can be switched to idle mode, serving as spares, thereby reducing unnecessary server purchases and improving overall operational efficiency [7]. Operational practices in data centers often allocate more resources than necessary. Operators tend to feel relieved when resource levels are significantly above their immediate requirements. Additionally, the business models of some orchestration software makers, which may involve licensing costs tied to the number of physical servers, can incentivize the deployment of software on more servers. This can lead to over-provisioning relative to actual demand. However, as long as operational stability is guaranteed, it is essential to minimize resource consumption for economic operations. Consequently, applying optimization algorithms to container placement is a critical strategy as they can determine the minimum number of required servers, allowing the surplus servers to be transitioned to an idle state, thereby enhancing overall efficiency. Placing multiple containers on each physical server presents a key challenge: the complexity of finding an optimal solution as the number of possible combinations increases exponentially. For instance, even in a relatively simple case of placing 13 containers on five servers, there are 2380 possible overlapping combinations. This problem can be expressed as a variant of the bin-packing problem, which is considered to be an NP-hard problem [8]. To address this, various optimization methods, such as heuristic, meta-heuristic, and deep reinforcement learning methods, have been explored in previously reported studies [1]. However, these studies have primarily focused on improving algorithmic efficiency rather than achieving effective placements. Furthermore, the inherent complexity of meta-heuristic and deep reinforcement learning methods can make them difficult to implement, and their application in real-world environments may be limited owing to an incomplete understanding of the containerized server systems.