# Optimizing Container Placement in Data Centers by Deep Reinforcement Learning
## 6 Discussion and Conclusions  
The container placement problem in data centers is an important issue because it directly affects operating costs. This paper proposes an optimization algorithm for container placement that allows unused servers to transition to idle mode, reducing resource consumption. This algorithm demonstrated better performance than the benchmark heuristic methods, successfully minimizing active PSs and enhancing resource efficiency. Additionally, although many optimization approaches focus on container placement based on VMs, this study considered the container form directly along with stable operation methods typically employed in actual data center environments, offering advantages in operational stability and adaptability. This excellent performance was achieved using the actor-critic method, which, combined with the use of fixed-length state values and a structured reward system, also provided flexibility for application to various environments and suggests possibilities for future improvements. In the future, integrating container orchestration and DRL is expected to stimulate further research and discussion, contributing to broader goals such as IT operation automation and automatic optimization of cloud environments.  
Although the experimental results confirmed that DRL is an effective approach for optimizing container placement in data centers, it is impossible to determine whether the current optimization represents the absolute best solution, as evaluating all possible placements is infeasible. Therefore, better algorithms may emerge over time. The DRL agent proposed in this paper performed comparably to existing methods in some cases, and even when it performed better, the difference was not substantial, indicating room for further improvement.  
Several potential enhancements can be explored. First, this study used linear networks and Softmax for learning [19]. However, implementing long short-term memory networks could improve performance by remembering past states and actions to decide on future actions. In addition, instead of using a single network, employing multiple independent networks that exchange learning results could yield better performance.  
Further research is also needed to develop an optimal placement model for containers operating across multiple zones. Currently, the proposed method assumes that containers can be placed freely within a single zone. Future studies should explore container placement methods that distribute workloads across multiple zones, ensuring greater stability in the event of a failure in an entire zone or individual PSs. In addition, as traffic loads fluctuate, containers must be dynamically scaled. Thus, an optimal placement method that accounts for sudden traffic surges is essential. Finally, simultaneously optimizing all virtual servers and containers within a data center could further reduce costs, making ongoing research in this area crucial.  
Conceptualization, C.L.; Methodology, H.K.; Validation, H.K.; Formal analysis, H.K.; Investigation, H.K.; Supervision, C.L.; Project administration, C.L. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
Not applicable.  
The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author.  
**Acknowledgments:** We would like to thank the staff of the National Information Resources Service, including [PERSON], [PERSON], [PERSON], [PERSON] and [PERSON] for their expertise which greatly assisted our research.