# Performance Analysis of Container Effect in Deep Learning Workloads and Implications
## 1 Introduction  
As deep learning is widely used in various IoT (Internet-of-Things) services [1; 2], the demand for AI (artificial intelligence) technologies in the mobile edge environment is continuously increasing [3; 4]. Specifically, there is an increasing trend to run deep learning workloads using virtualized environments such as virtual machines and containers. This is because virtualization has the effect of improving computational elasticity and efficiency, which are important considerations in deep learning workloads [3]. The execution cycle of a deep learning workload can be divided into three phases: training, servicing, and monitoring, as shown in Figure 1. During the training phase, users need to install the configuration for the framework, provision the required computing resources, and then proceed with training. When training is complete, the model is exported and the API server is installed. The model is then deployed for service. In the service phase, users should monitor the model's performance to determine whether the model is overfitting or underfitting the training data. When the model's performance degrades, the user must train the model again. Therefore, configuring and managing deep learning infrastructure is a complicated and time-consuming task. All of these steps can be performed in the data center or cloud, but some tasks such as data collection and post-training monitoring are increasingly shifted to the edge side. Reducing energy consumption and privacy risks is important in mobile edge as continuous training is performed by using data collected from mobile devices and sensors [5]. Container-based deep learning has been introduced to provide such features in a managed service [6; 7]. Containers provide several merits compared to traditional virtual machines for running deep learning workloads. That is, containers are lightweight and portable, so it is easy to deploy and manage containerized deep learning workloads. Containers also share the host OS, thereby minimizing the virtualization overhead. Thus, containers are efficient for compute-intensive deep learning workloads that require heavy resources. Another important merit of a container is that it provides easy management for configuration settings. In a virtual machine platform, each guest machine should have its own configuration files, thus keeping track of changes and ensuring the configurations are difficult. Moreover, in container environments, it is easy to recreate the same environments and dependencies used to train and deploy models. In contrast, in virtual machine environments, each guest machine has a set of dependencies, so recreating the same environment on another machine is not simple [8]. Despite these advantages, using containers to run deep learning workloads also presents some challenges. One of the biggest challenges is that containers are less isolated than virtual machines. This means that performance degradation is more likely to occur due to resource conflicts [9]. For example, if two containers try to access the same resource, a performance conflict may occur. Since containers share the host OS, they compete for system resources and need careful resource management techniques to prevent performance degradation. Fortunately, the overhead of containerization is not significant in traditional workloads. For example, running multiplayer gaming and video streaming workloads in a containerized edge environment incurs only a small amount of Docker overhead without affecting data processing in each container [10]. Also, as containers are frequently used for application deployment, network resources become a major bottleneck due to the large amount of data transfer between nodes [11]. However, this is not the case for deep learning workloads, which are resource-intensive and require more computation than communication [12]. Thus, containerization may introduce other resource bottlenecks such as CPU, memory, and storage.