# Performance Analysis of Container Effect in Deep Learning Workloads and Implications
## 1 Introduction
As deep learning is increasingly used in image processing in various service fields such as manufacturing and medicine, the importance of efficient resource management in containerized deep learning continues to grow [13; 14]. Moreover, deep learning workloads have different resource usage patterns from traditional workloads, making efficient resource management more difficult [15; 16]. Traditional workloads often have consistent resource usage patterns. For example, web servers or database servers repeatedly process the same tasks, such as http requests or query processing, so the number of requests varies, but the characteristics of the workload are uniform, making it relatively easy to cope with resource requirements [17]. In contrast, deep learning workloads are known to have very different resource usage patterns during the training and inference phases [18]. In the training phase, the model is learned on the data set, and  
Figure 1: Execution cycle of a deep learning workload. the parameters are updated. During this process, resource usage may show sudden spiky patterns. On the other hand, the inference phase uses models to predict new data and resource usage is not high and is relatively uniform. It has been reported that deep learning workloads temporarily cause excessive memory usage during the training phase and that the bias in data accessed is weaker than traditional workloads because they rely less on hot data [19]. These characteristics make resource management more difficult compared to traditional workloads. Specifically, due to the spiky and bursty nature of memory usage in deep learning workloads, performance can rapidly deteriorate if insufficient memory space is allocated. However, equipping a large memory capacity to handle such situations will waste resources in the remaining time. So resource allocation that allows each container to run smoothly without over-allocation is a challenging issue. In addition, weak bias in data access makes it more difficult to design efficient caching mechanisms and limits performance gains through caching. Comprehensively monitoring data access patterns can be helpful in identifying and remediating resource usage issues. In particular, extracting and analyzing system traces from the host layer provides precise resource usage for each container, allowing for better resource allocation by accurately characterizing container workloads. In this article, we investigate the resource requirements in the context of containerized deep learning workloads and aim to find performance implications for such environments. To do this, we extract and analyze the event traces of deep learning workloads in container environments and compare them to traces collected by running the same workloads on host systems. By doing so, we can see the impact of containerization in deep learning with respect to resource contention and performance overhead. We also investigate the impact of multi-tenancy by running deep learning workloads simultaneously in different containers to highlight challenges in resource management issues. By analyzing system calls and event traces generated by container-based deep learning workloads, we provide implications for resource management in a containerized environment. Specifically, we observe that write-back operations to storage can be a major cause of performance bottlenecks. We also show that resource contention is significant in multi-tenant environments. To cope with this situation, we introduce a preliminary solution that adopts an intermediate non-volatile flushing layer to alleviate the performance bottleneck of storage write-backs. Instead of flushing to traditional storage, our solution effectively hides the write-back overhead by absorbing hot data in fast NVM media, improving I/O latency by 82% on average. The remainder of this article is organized as follows. Section 2 briefly summarizes previous studies related to this article.