# HPC Cloud Architecture to Reduce HPC Workflow Complexity in Containerized Environments
## 2 Related Work
### Requirements of the HPC Workflow in a Containerized Environment  
From our review of related projects, we listed 10 requirements and compared them with those of our proposed platform in Table 1. These requirements were derived from our experiences of HPC users and system administrators. We define an HPC workflow as the flow of tasks that need to be executed to compute on HPC resources. Tasks in a containerized environment can be divided into image, template, and job (container and application) management.  
In Table 1, for a self-service interface requirement, the Sarus project designed image, template, and job management, which was implemented as command line interfaces. Our proposed platform was also designed to support image, template, and job management with integrated command line interfaces. Shifter and Sarus use Docker as their basic container solution. The EASEY project was designed based on a Charliecloud solution, which can be used in an HPC environment. The JEDI project supports various container solutions such as Docker, Charliecloud, and Singularity. We selected only Docker and Singularity for our base container solutions because we considered architectural ways to provide unprivileged containers with Docker instead of Charliecloud. For a rapid expansion of new configuration environments, we listed three functionalities: template-based image management, image-based job management, and image format conversion. Sarus supports all three features. We first attempted to implement two main features and included image format conversion between Docker and Singularity as our future work.  
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Requirements** & **Shifter** & **Sarus** & **EASEY** & **JEDI** & **Proposed Platform** \\ \hline A self-service interface & Image & Image, template, job & Image, template & Image & Image, template, job \\  & & & & Docker, Charliecloud, Singularity & Docker, Singularity \\  & & & & & \\ \hline A rapid expansion of new configuration environments: & & & & & \\ (1) & Template-based image management & (1), (3) & (1), (2), (3) & (1) & None & (1), (2) \\ (2) & Image-based job management & & & & & \\ (3) & Image format conversion & & & & & \\ A pay-as-used model (on-demand billing) & None & None & None & None & Yes \\ Performance similar to that of bare-metal servers & None & Yes & Yes & None & Yes \\ Auto-provisioning that includes virtual instances and applications & Yes & Yes & None & None & Yes \\ Workload management & Image & Image, job (batch scheduler) & None & None & Image, job (batch scheduler) \\ Resource auto-scaling & None & None & None & None & Yes (image management) \\ Multitenancy capability & Yes & Yes & None & None & Yes \\ Portability of application & Yes & Yes & Yes & None & Yes \\ \hline \hline \end{tabular}
\end{table}
Table 1: Comparison of requirement of projects vs. the proposed platform.  
For a pay-as-used model, we implemented an on-demand billing system by collecting metering data with an existing authentication and authorization policy not supported in other projects. Sarus and EASEY use the host network with containers through multinode computing. We also evaluated container networking performance with tuning in our previous research. Shifter and Sarus support auto-provisioning that includes containers and inside running applications. For workload management, Sarus supports both image and job management. We referred to the architecture of this project. For resource auto-scaling, no projects support it. We implemented it for image management to distribute a request load. For multinetancy capability, Shifter and Sarus were designed to use a shared resource pool but provide various separated services to users. All projects support portability of application with container solutions.