# A Case Study on Virtual HPC Container Clusters and Machine Learning Applications
## 2 Related Work
### Docker in HPC  
Before containerization became firmly established in the field of computer system administration as a new technique for creating segregated environments with varied software stacks, traditional virtualization, also known as hardware-level virtualization, was the prevailing method. The roles of virtual machines and containers are similar in many areas, leading to the natural assumption that there might have been attempts to prepare an artificial cluster using the former method. Indeed, there are scientific works discussing how to do this ([PERSON] et al., 2018; [PERSON] et al., 2018; [PERSON] et al., 2018). [PERSON] et al. (2018) explored the stability and reliability of Docker containers running on Windows operating systems under resource-deficit conditions. Their work, focusing on stress and volume testing, was crucial for understanding the performance of Docker containers under extreme load conditions, a relevant aspect when considering the efficiency of virtual HPC clusters. The authors ([PERSON] et al., 2018) discussed the application of Zero Trust principles to Docker containers, proposing the Zero Trust Container Architecture. This framework is particularly relevant for ensuring security in containerized environments, an aspect that complements this study's focus on Docker in HPC. [PERSON] et al. (2018) presented a neural network approach for recommending suitable base images for Docker containers, a key aspect in optimizing container performance and efficiency. Their work aligns with this study's interest in enhancing Docker container performance in HPC environments. Traditional virtualization never gained sufficient popularity in real HPC clusters, although it was considered due to its potential use in providing supercomputers as multi-user systems ([PERSON] et al., 2018). Since containerization has often been compared with traditional virtualization, studies have emerged comparing the performance of both techniques ([PERSON] et al., 2018), including analyses of their application in HPC systems ([PERSON] et al., 2018; [PERSON] et al., 2018). Tests were conducted using the Kernel-based Virtual Machine hypervisor and the Docker containerization platform. They evaluated computational performance, the performance of data-intensive applications, network communication efficiency depending on network topology, and memory usage. Generally, the results favored the use of containerization over virtual machines to achieve the highest efficiency. However, Docker is not without flaws. The performance of default-configured containers is primarily degraded by Docker's layered file system and the Network Address Translation used in network communication, although the negative effects of both mechanisms can be mitigated ([PERSON] et al., 2018). A significant downside of both virtualization techniques is the increased overhead on input-output operations ([PERSON] et al., 2018), especially those performed on local resources shared with many systems, such as hard drives and bridged network interfaces. However, the overhead on CPU usage is negligible, regardless of the chosen technique ([PERSON] et al., 2018). Over recent years, hardware virtualization has only improved in performance ([PERSON] et al., 2018), thanks to the widespread adoption of supporting technologies such as the Intel VT and AMD-V series. Although traditional virtualization remains irreplaceable in many areas of computing (e.g., leased private virtual servers), it was containerization and its promised 'lightness' (in terms of computer resource load) that effectively encouraged those interested in HPC to conduct experiments and discover practical applications of containers in HPC systems, leading to the emergence of dedicated containerization platforms, such as Shifter [19]. Given the current state of technology, using traditional virtualization to prepare an artificial computational cluster is also questionable.