# Efficient Use of GPU Memory for Large-Scale Deep Learning Model Training
## 5 Experiment and Performance Analysis
### Results of GPU Page Fault Profiling Using Nvprof  
In this section, we profile the experiment (mini-batch size: 32, max sequence: 256) performed in the previous section when the data size is larger than the GPU memory size using nvprof, a profiling tool of NVIDIA, and analyze the profiling results. When the data size required for deep learning is larger than the GPU memory, the memory advise setting that shows the best performance is when the intermediate result data are prefetched, the model parameters are set as _Preferred location_, and input data are set to _Read mostly_. Further, the memory advise setting that showed the lowest performance is the case where the intermediate results are set as the _Preferred location_ without giving memory advise to the model parameters and input data. Table 3 summarizes the profiling results of GPU page fault profiling. Table 3 is the result of profiling learning for 10 iterations by applying each memory advise. Table 3 shows that the number of page faults due to unified memory. When the memory advise is not applied, the page faults occur the most. The difference in the elapsed time for data migration of unified memory according to memory advise is so small that it is meaningless. Further, data migration takes more time for Host to Device than Device to Host. Therefore, looking at Table 3, as can be expected from the results shown through the graphs of the Figures 8 and 9 in the previous section, the memory advise with the best performance generated the least number of page faults. We used NVIDIA Visual Profiler, a graphic profiling tool, to check the pattern of page faults according to each memory advise. Figure 10 are visualizing the profiling results using NVIDIA Visual Profiler. Figure 10 is the case where no memory advise is applied, and Figure 10,\(c\) are the results of applying the memory advise showing the highest performance and the memory advise showing the lowest performance, respectively. In the upper red area of Figure 10a-c, you can check the pattern of page faults due to unified memory when deep learning is performed. NVIDIA Visual Profiler divides the execution of the process into equal-width sections, and expresses the percentage of time it takes to resolve GPU page faults in each section as a color. As can be seen in the upper red area of each figure, colors appear in a similar pattern regardless of memory advise when deep learning is performed. This shows that page faults occur in a similar pattern regardless of memory advise when deep learning is performed. In the blue area at the bottom of Figure 10a-c, you can check the time it takes to resolve the page fault according to the color of each section. The width of each section of unified memory in NVIDIA Visual Profiler is the total execution time divided by the same number; therefore, even with the same color in the three pictures, the time taken to resolve the page fault is the shortest when the memory advise that shows the best performance is applied. From the profiling results, we can confirm that GPU page faults can be reduced by applying the optimal memory advise. In addition, by reducing GPU page faults, the overhead caused by page faults is reduced, and thus, operator resources of the GPU can be used more efficiently. Figure 11 shows the pattern of GPU page faults when training a single mini-batch. The area marked in blue in Figure 11 is the area performing feed forwarding and the area marked in red is the area performing backpropagation. Figure 11 shows that most sections are displayed in green when feed forwarding is performed. Further, when performing backpropagation, it can be seen that the ratio of the section marked in yellow is high. The green section means that the percentage of the time it takes to resolve the page fault is 40% to 50%, and the yellow section means 60% to 70%. Through this, we can see that the ratio of the time of the operation is delayed due to a page fault is greater when performing