# Transformer: An OS-Supported Reconfigurable Hybrid Memory Architecture
## 4 Evaluation
### Experimental Results  
Figure 7 shows the page access counts of programs tracked our page access monitor. The total pages represent the amount of pages scanned by the page hotness monitoring module during the runtime of the program, i.e., the amount of pages the program has \begin{table}
\begin{tabular}{c c} \hline \hline
**CPU** & **Intel Xeon Gold 6230 2.10 GHz** \\ \hline DRAM & 128 GigaBytes \\ NVM & 512 GigaBytes \\ Disk & 320 GigaBytes \\ Operating System & CentOS 7.7 \\ Kernel & Linux 5.1.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Configurations of our hybrid memory system. \begin{table}
\begin{tabular}{c c} \hline \hline SPEC2006 & mcf, wrf, sjeng, libquantum, aster, Ibm \\ PARSEC & streamcluster, blackscholes \\ Graph500 & BFS \\ MIX1 & libquantum+aster+Ibm \\ MIX2 & libquantum+mcf+sjeng \\ MIX2 & streamcluster+blackscholes \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Workloads. touched in memory. Hot pages represent the amount of memory pages that exceeds the hot page threshold. For a relative small hotness threshold, the proportion of hot pages is rather high for applications with a good data locality or memory-intensive applications, such as sjeng, lbm, and streamcluster. Figure 8 shows that the ratio of hot pages varies with the time for different workloads in SPEC CPU2006. We can find that the hot page ratio of sjeng varies significantly, while mcf, libquantum, aster, and lbm are relatively stable. The memory access intensity of mcf is rather low, and the proportion of hot pages always remains low. Since libquantum accesses memory intensively, the proportion of hot pages is always high. Aster has a good temporal locality of memory accesses, and thus generates large amounts of hot pages in a very brief period of time and a high percentage of hot pages. Similarly, lbm has a good spatial locality, and thus a large portion of hot pages are identified. Figure 9 shows that Transformer can effectively improve the performance of applications in SPEC CPU2006, PARSEC, and Graph500. libquantum shows the best performance in Transformer because libquantum is a memory-intensive application with good data locality. As most memory accesses are spread over a small portion of memory space, these memory pages reach the threshold of hot pages faster than other programs. On the other hand, libquantum only requires a few page migrations due to a small memory footprint, and thus causes less performance overhead. The aster has similar performance in the H.S. and dynamic modes, but much better than that in the P.S. mode. The reason is that  
Figure 8: The proportion of hot pages identified in SPEC CPU2006. Figure 7: Numbers of total accessed pages and hot pages. aster has good temporal locality, and pages are accessed in a very short time. In the H.S. mode, DRAM is able to cache a large number of hot pages and these hot pages have a high reuse rate. In our dynamic mode, aster is actually running in the hierarchical mode which has a similar performance to the H.S. mode. In contrast, lbm runs more efficiently in the P.S. mode because of good spatial locality. In the P.S. mode, as one migration contains 256 B data, lbm is able to get quick access to the adjacent data of the current hot data in DRAM. Furthermore, we find that the execution time of programs has a slight effect on the performance improvement. For example, the performance improvement of blackscholes is not significant. Since the execution time of blackscholes is only 2 min in our experiment, it cannot identify most hot pages and migrate them to DRAM in a very short time, and thus can only achieve slight performance improvement from page migrations. Therefore, we can reduce the threshold of hot pages slightly for these applications to migrate more pages. Figure 10 shows the breakdown of execution time in different hybrid memory architectures. We can find that libquantum spends more than 90% of execution time in the hierarchical hybrid memory.