# A Case Study on Virtual HPC Container Clusters and Machine Learning Applications
## 3 Container Architecture
### Container Clusters
Users are also left with a set of configurable options to be applied only at the start of the container. In case of their use, there is no need to rebuild the images. The node image should cooperate with any implementation where the process manager uses SSH for communication between nodes, but it is recommended to use one in which the mpirun program can also independently read the host's file in a location specified by an environmental variable. Several image variants are available, combining two base Linux distributions--Debian or Alpine--with two MPI implementations--MPICH or OpenMPI. Alpine's image minimizes network transfer, speeding up downloads and, in cloud environments, reducing data-transfer charges. Pursuing minimal size, however, has consequences. Alpine relies on the _musl_ C library, whereas most mainstream distributions (e.g., Debian) ship _glibc_. This difference may cause compatibility and portability issues, as software designed for _glibc_ may not compile cleanly against _musl_. Outside the Docker ecosystem, many developers prefer established distributions such as Debian, CentOS, or Fedora. To meet that preference, Debian-based images were also prepared, making the environment more familiar and secure for production use. Because the cluster is intended to lower the entry barrier to distributed programming, unfamiliar operating systems are avoided. Activating the cluster and opening the management console requires only two simple Docker commands, and automatic host discovery removes the need to list container addresses when running an MPI program. In addition, users can pass configuration options at container start without rebuilding the images. The node image works with any MPI implementation whose process manager communicates over SSH; ideally, mpirun should be able to read the hosts file from a path set by an environment variable. All images are available in the public Docker Hub repository and are named as follows:  
- pikrog/mpich--base image with MPICH,  
- pikrog/mpich-node--cluster-node image with MPICH,  
- pikrog/openmpi--base image with OpenMPI,  
- pikrog/openmpi-node--cluster-node image with OpenMPI. Image tags consist of two parts separated by a hyphen: the version and the distribution (alpine for Alpine, bullseye for Debian). Base-image versions reflect the MPI release; node images have independent numbering, because their functionality evolved faster than upstream MPI. A table mapping node-image tags to MPI versions is provided in the GitHub repository [38]. There are certain functional and user differences between the proposal in [22] and the one presented in this work. The original images allowed the use of only MPICH dependencies installed in the Alpine distribution. In the updated version, support for Debian (for the reasons described earlier) and the OpenMPI implementation, which appears to be more comprehensive and popular judging by the statistics in the official GitHub repository, was added. In the old solution, container configuration is achieved by passing arguments to the startup script and only allows the choice of the container's role (supervisory or worker) and the specification of two cluster host names for network discovery. In the new proposal, the range of settings was expanded, and they are fully modifiable using environmental variables. An arbitrary number of host names can also be specified for discovery. Many options were introduced with the deployment of the containerized cluster in a production environment in mind. A script for waiting for the readiness of a specified number of cluster nodes was also developed. This may prove useful for automating the launching of MPI applications when the cluster operates in distributed mode (i.e., on multiple physical machines, not just one). The prepared images allow for the use of both versions of MPI implementation, namely MPICH 4.1.1 (Appendices A.1 and A.2) and OpenMPI 4.1.5. Other Image Variants  
Dockerfile scripts have only been described for images using the Alpine distribution and MPICH implementation.