# A Case Study on Virtual HPC Container Clusters and Machine Learning Applications
## 1 Introduction  
Early computers were large, expensive to build, and had limited computational power. Initially, they were used for long and tedious calculations and were not part of the consumer market, accessible only to select government or academic organizations. The advent of minicomputers, despite their smaller size, did not immediately solve these issues, due to their still-high cost and limited capabilities. The technological breakthrough came with the miniaturization of devices and the development of high-speed local computer networks, which led to a significant reduction in the cost of electronic devices, making computers more accessible to the general public. This miniaturization also enabled the installation of multiple processors on a motherboard, leading to the development of supercomputers with hundreds or even thousands of processors. This advancement was significant because it coincided with growing interest in parallel algorithms, which divide larger tasks into smaller parts to be executed by multiple processors simultaneously. The physical limitations of processors and the inefficiency of overlocking due to heat generation led to a focus on multiprocessing and multi-core processors. Next, computational clusters, extending the concept of multiprocessing, created extensive parallel or distributed systems, offering vast computational power. Instead of investing in a single high-performance device, the idea is to network many units with less powerful but cheaper components, often virtualized to achieve a more cost-effective overall performance. Simultaneously, containerization has emerged as a solution to software portability issues. This technique involves combining an application with its dependencies into an image, which is then run on an isolated container, making the application independent of the specific computer platform and facilitating its distribution and deployment. Today, the goal is to build a virtual computational container cluster using containerization on the Docker platform. It was therefore essential to validate the cluster's functionality, evaluate its performance, and demonstrate a practical application. This article covers the following:  
1. Preparation of Dockerfile scripts for building images to launch a cluster on the Docker platform, including the installation of the Message Passing Interface (MPI) communication standard and the configuration of containers for distributed programs. 2. Development and a use of a distributed application using Machine Learning (ML), demonstrating the effects of parallel processing on computation time in a containerized structure. Our study focuses on the intersection of Docker and High-Performance Computing (HPC), exploring how Docker's containerization technology can be leveraged to enhance the performance and scalability of HPC systems. [PERSON] and [PERSON] [1] performed a detailed performance assessment of Docker-based MPI applications, demonstrating minimal overhead. [PERSON] et al. [2] and [PERSON] and [PERSON] [3] explored a Slurm-Kubernetes orchestration, running HPC workloads under Kubernetes environments. [PERSON] and [PERSON] [4] further investigated container performance metrics on \(\times 86\) and OpenPOWER systems. We delve into the practical implications of this integration, particularly in the realm of ML for anomaly detection in system logs, a domain that has seen limited exploration in the existing literature. This research aims to close the gap between the theoretical potential and practical application of Docker in HPC environments, providing valuable insights and contributions to this emerging field [5]. The article is divided into the following sections: Section 2 provides a comprehensive overview of the evolution and significance of computational clusters, detailing their transition from exclusive use in government and academic organizations to more widespread accessibility, due to technological advancements like miniaturization and the development of high-speed local computer networks.