# Performance Analysis of Container Effect in Deep Learning Workloads and Implications
## 4 Analysis of Deep Learning Event Traces
### Basic Event Analysis  
We use the Linux perf tool to profile system calls and events generated by container processes, Docker daemons, and host processes to quantify the performance effects of each system configuration. The container process executes deep learning workloads on a container based on a virtualized form. The Docker daemon is a process that handles Docker API requests and manages containers. Specifically, it builds, runs, and manages images, containers, and other Docker objects. The host process runs deep learning workloads directly on the host OS without virtualization. Table 5 shows basic event and system call traces extracted from the container process, Docker daemon, and host process. For system calls, the most frequently invoked events and their number of invocations are listed. As we see from this table, most system calls are related to memory allocation and I/Os. In terms of CPU performance, running workloads in Docker containers increases CPU cycles by approximately 50% compared to running workloads directly on the host machine. As we can see, the pure container overhead caused by the daemon process is not significant, but various inefficiencies resulting from the addition of a layer of containers increase the overhead.  
When analyzing block-related system calls in the container environment, we observe a significant number of block_rq_insert events. In particular, by analyzing the call graphs associated with these events in Table 6, we see that there is a significant number of write-back operations due to dirty pages within the container. This sync process flushes all dirty pages to storage, causing write-back operations to wait for the flush.  
We also measure page-faults and swaps occurring on the container-based and host systems. As can be seen in Table 5, the number of page-faults in the container-based system is more than twice that of the host system. There are two types of page-faults: major faults, which involve swap I/O, and minor faults, which can be handled by kernel mapping without storage access [30]. The increased number of page-faults in container-based systems suggests that containers may be more susceptible to I/O performance problems than traditional OSs. This is because containers share the file system and host  
\begin{table}
\begin{tabular}{l l l} \hline
**Dataset** & **Scale** & **Model** \\ \hline \multirow{2}{*}{Text} & Small & Mobile-BERT [26] \\  & Large & LSTM [27] \\ \hline \multirow{2}{*}{Image} & Small & SqueezeNet [28] \\  & Large & AlexNet [29] \\ \hline \end{tabular}
\end{table}
Table 4: Model configuration of our experiments.  
OS, which can lead to contention and fragmentation. This results in poor performance and increased I/O load in container-based systems. To alleviate the performance degradation of page swapping, it is important to carefully manage the memory management algorithms allocated to containers. For example, it should be ensured that containers have sufficient memory and do not execute workloads that are known to generate a lot of page-faults. Also, it is necessary to generate different container runtimes to provide better isolation from the host system.