# Optimizing Container Placement in Data Centers by Deep Reinforcement Learning
## 3 Problem Formulation  
Data centers house numerous physical servers on which virtualized servers are deployed. Virtualization allows a single physical resource to be separated into multiple logical or physical resources, which can then be combined into a single logical entity. The most common form of virtualization is server virtualization, which enables a single physical server to operate as multiple virtualized servers. Server virtualization includes virtual machines (VMs), where a separate operating system is installed on each instance, and containers, which share a host operating system without requiring separate OS installations. Compared to VMs, containers offer greater speed, flexibility, and efficiency by enabling rapid deployment, scalability, and lightweight operation while running only essential software. Due to these advantages, containers are becoming the dominant form of server virtualization in data centers [4].  
As the number of physical servers in a data center increases, the corresponding costs increase proportionally. Since the number of containers to be operated is fixed, it is important to minimize costs by reducing the number of active physical servers being operated through optimal placement of containers. Therefore, an optimal container deployment method is required to reduce the number of active physical servers and, consequently, the operating costs of data centers.  
Common tools for container deployment include Docker Swarm and Kubernetes. Docker Swarm offers container deployment capabilities integrated directly within Docker, allowing users already employing Docker to utilize Swarm without separate installation. Kubernetes, an open-source container orchestration tool developed by Google, is designed to effectively manage large-scale container deployments, leveraging its high scalability and flexibility. These orchestration tools typically support both manual deployments to specified physical servers and automatic server selection and deployment when no specific target is selected. Docker Swarm distributes containers across available resources using a round-robin approach [5]. Kubernetes places containers based on a combination of filtering and preference criteria [16]. These methods assess whether a container can be accommodated and determine the most suitable physical server for deployment.  
In contrast, optimal container placement seeks to minimize the number of active physical servers (PSs) by strategically distributing containers (CTs), as shown in Figure 1. This approach allows idle servers to be powered off, leading to significant reductions in server resource consumption, electricity usage, cooling requirements, and overall operational costs.  
When placing containers, CPU and memory capacity must be considered, while storage is managed separately by dedicated software and does not factor into placement decisions. Therefore, CPU and memory should be allocated as a single unit to PSs. Figure 2 shows the concepts of bin packing and container placement [7].  
Based on this approach, container CPU and memory resources must fit within the CPU and memory capacity of the physical server. While CPU overcommitment is possible, it is not used in this allocation strategy because overcommitment can cause containers to be terminated (\"killed\"), meaning only the physical CPU capacity is assigned. Similarly, memory is not overcommitted, as doing so can lead to significant performance degradation. Typically, 60% of the total memory capacity is considered the maximum allocation limit to ensure system stability. By setting predefined limits for CPU and memory usage, this method ensures stable and efficient resource allocation.  
Figure 1: Container optimization placement concept diagram. Three types of containers with different sizes (type1: blue, type2: magenta, type3: red) are given.